Independent third-party performance data — land-grant programs that test every
entered brand side-by-side with replication + LSD stats. This is the legitimate
way to get Pioneer / DEKALB / Brevant / Channel performance the corpus can't
scrape directly (data_type=trial, results[] shape; falls through the trial
chunker).
- illinois_vt_trials (30 docs, 1,392 rows) — U of Illinois VT. Per-region XLSX
(openpyxl), corn + soy + WHEAT, 2024+2025. Rich per-site agronomic metadata;
corn-following-corn vs -soybean kept distinct.
- iowa_icpt_trials (24 docs, 674 rows) — Iowa State ICPT. ASP.NET GridView
(viewstate postback for year/district), corn + soy by district x season.
- ohio_ocpt_trials (69 docs, 4,647 rows) — OSU/CFAES OCPT. Report PDF
(pdfplumber; per-site column groups split by header Yield-token count +
x-coord footnote bucketing), corn + soy per site, 2024+2025.
91 distinct seed brands across the three; majors confirmed present in the
independent rankings: DEKALB 395, Golden Harvest 249, Channel 241, NK 212,
Xitavo 135, LG 103, Pioneer 88, Asgrow 59. (A brand only appears where it
ENTERED a given program — e.g. Brevant not in Iowa, DEKALB/Channel not in
Illinois — true negatives, not parse gaps.)
- rag/chunk.py: gated `include_region` on _render_gh_plot_chunk; the 3 university
sources route through it so the region/district is in the embedded chunk +
labeled "variety trial (cross-vendor, independent third-party)". Existing plot
sources (gh/lg/agrigold/proharvest) unchanged.
- requirements.txt: openpyxl (Illinois XLSX; scrape-time only).
- sources.json + README/CLAUDE/lessons: registered + attributed; lessons
trial-data + Pioneer entries updated (Pioneer/DEKALB performance now available
indirectly via these trials).
Validation: all 123 chunk via rag.chunk.chunks_from_trial (0 errors), 0
out-of-range yields, 0 dup keys. Public land-grant data; attribution recorded in
each tos_note. CI rebuilds the index from the committed corpus.
The scaffold-era README was out of sync with the shipped product:
- Vendor counts stale (recon estimates, not actual deployed counts)
- Trial data sources (gh_plot_reports + agripro_trials) entirely
unmentioned
- Tool list listed `corpus_status` (doesn't exist) and missed both
`lookup_variety` and `search_trials`
- Build-phase table showed everything as "pending" / "next" but
Phases 1-8 + 11 all shipped
Rewrite to reflect the deployed state:
- Corpus inventory: 760 variety records + 4,313 trial documents =
5,073 chunks across 6 sources
- All 6 MCP tools documented with their purpose
- Eval baseline table (hybrid+rerank wins 100%, P@1 90%, MRR 0.905)
with the surprising findings (dense alone is noise; hybrid w/o
rerank is WORSE than BM25 alone)
- Deploy mechanics: Watchtower chain, 4-GPU embedder pool, shared
llama-rerank sidecar with the network-attach gotcha
- Status table: ✅ on the phases that shipped, deferred work list
(becks_pfr, 2023 plot backfill, NK trials, Channel Seed brand)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
claude
justin