Add university-extension variety trials: Illinois VT + Iowa ICPT + Ohio OCPT (+123 trial docs)

Independent third-party performance data — land-grant programs that test every entered brand side-by-side with replication + LSD stats. This is the legitimate way to get Pioneer / DEKALB / Brevant / Channel performance the corpus can't scrape directly (data_type=trial, results[] shape; falls through the trial chunker). - illinois_vt_trials (30 docs, 1,392 rows) — U of Illinois VT. Per-region XLSX (openpyxl), corn + soy + WHEAT, 2024+2025. Rich per-site agronomic metadata; corn-following-corn vs -soybean kept distinct. - iowa_icpt_trials (24 docs, 674 rows) — Iowa State ICPT. ASP.NET GridView (viewstate postback for year/district), corn + soy by district x season. - ohio_ocpt_trials (69 docs, 4,647 rows) — OSU/CFAES OCPT. Report PDF (pdfplumber; per-site column groups split by header Yield-token count + x-coord footnote bucketing), corn + soy per site, 2024+2025. 91 distinct seed brands across the three; majors confirmed present in the independent rankings: DEKALB 395, Golden Harvest 249, Channel 241, NK 212, Xitavo 135, LG 103, Pioneer 88, Asgrow 59. (A brand only appears where it ENTERED a given program — e.g. Brevant not in Iowa, DEKALB/Channel not in Illinois — true negatives, not parse gaps.) - rag/chunk.py: gated `include_region` on _render_gh_plot_chunk; the 3 university sources route through it so the region/district is in the embedded chunk + labeled "variety trial (cross-vendor, independent third-party)". Existing plot sources (gh/lg/agrigold/proharvest) unchanged. - requirements.txt: openpyxl (Illinois XLSX; scrape-time only). - sources.json + README/CLAUDE/lessons: registered + attributed; lessons trial-data + Pioneer entries updated (Pioneer/DEKALB performance now available indirectly via these trials). Validation: all 123 chunk via rag.chunk.chunks_from_trial (0 errors), 0 out-of-range yields, 0 dup keys. Public land-grant data; attribution recorded in each tos_note. CI rebuilds the index from the committed corpus.
2026-06-10 08:35:50 -04:00
parent 0bac06b7b6
commit 54094a0d43
255 changed files with 105410 additions and 13 deletions
@@ -0,0 +1,423 @@
+{
+  "source": "iowa_icpt_trials",
+  "source_key": "icpt-soybeans-2025-south-early",
+  "data_type": "trial",
+  "vendor": "Iowa State University",
+  "brand_aggregator": "Iowa Crop Performance Tests publishes",
+  "brand": "Iowa Crop Performance Tests",
+  "crop": "soybeans",
+  "state": "IA",
+  "state_abbrev": "ia",
+  "year": 2025,
+  "region": "District South",
+  "district": "South",
+  "season": "Early Season",
+  "cooperator_sites": [
+    "Atlantic",
+    "Corning",
+    "Chariton",
+    "Batavia",
+    "Crawfordsville"
+  ],
+  "experiment_mean_yield": 64,
+  "results": [
+    {
+      "rank": 1,
+      "brand": "Pioneer",
+      "product": "P31T64E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 3.1,
+        "Yield": 66.9,
+        "WestYield": 67.2,
+        "EastYield": 64.8,
+        "Atlantic": 73.9,
+        "Corning": 66.3,
+        "Chariton": 61.4,
+        "Batavia": 63.5,
+        "Crawfordsville": 69.7
+      }
+    },
+    {
+      "rank": 2,
+      "brand": "ISU",
+      "product": "IAS29C4",
+      "traits": "Conv",
+      "metrics": {
+        "MG": 2.9,
+        "Yield": 66.2,
+        "WestYield": 66.9,
+        "EastYield": 63.5,
+        "Atlantic": 75.1,
+        "Corning": 65.3,
+        "Chariton": 60.5,
+        "Batavia": 60.8,
+        "Crawfordsville": 69.2
+      }
+    },
+    {
+      "rank": 3,
+      "brand": "Xitavo",
+      "product": "XO 3105E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 3.1,
+        "Yield": 65.9,
+        "WestYield": 65.3,
+        "EastYield": 63.5,
+        "Atlantic": 70.1,
+        "Corning": 69.8,
+        "Chariton": 56,
+        "Batavia": 66,
+        "Crawfordsville": 68.5
+      }
+    },
+    {
+      "rank": 4,
+      "brand": "ISU",
+      "product": "IAS29C3",
+      "traits": "Conv",
+      "metrics": {
+        "MG": 2.9,
+        "Yield": 65.8,
+        "WestYield": 67.9,
+        "EastYield": 61.7,
+        "Atlantic": 72.4,
+        "Corning": 70.8,
+        "Chariton": 60.5,
+        "Batavia": 55.8,
+        "Crawfordsville": 68.8
+      }
+    },
+    {
+      "rank": 5,
+      "brand": "Nutech/G2 Genetics",
+      "product": "30N06E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 3,
+        "Yield": 65.7,
+        "WestYield": 67.5,
+        "EastYield": 60.3,
+        "Atlantic": 76.3,
+        "Corning": 70.7,
+        "Chariton": 55.5,
+        "Batavia": 55,
+        "Crawfordsville": 70.5
+      }
+    },
+    {
+      "rank": 6,
+      "brand": "Dyna-Gro",
+      "product": "S31EN96",
+      "traits": "E3",
+      "metrics": {
+        "MG": 3.1,
+        "Yield": 65.6,
+        "WestYield": 68.7,
+        "EastYield": 60,
+        "Atlantic": 76.7,
+        "Corning": 71.8,
+        "Chariton": 57.5,
+        "Batavia": 54.8,
+        "Crawfordsville": 67.8
+      }
+    },
+    {
+      "rank": 7,
+      "brand": "ISU",
+      "product": "IAS32C1",
+      "traits": "Conv",
+      "metrics": {
+        "MG": 3.2,
+        "Yield": 65.3,
+        "WestYield": 64.5,
+        "EastYield": 63.2,
+        "Atlantic": 68.2,
+        "Corning": 68.3,
+        "Chariton": 56.9,
+        "Batavia": 61.9,
+        "Crawfordsville": 70.7
+      }
+    },
+    {
+      "rank": 8,
+      "brand": "Nutech/G2 Genetics",
+      "product": "28N05E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 2.8,
+        "Yield": 65.1,
+        "WestYield": 65.2,
+        "EastYield": 61.9,
+        "Atlantic": 71.7,
+        "Corning": 67.9,
+        "Chariton": 56,
+        "Batavia": 60,
+        "Crawfordsville": 69.6
+      }
+    },
+    {
+      "rank": 9,
+      "brand": "Viking/Blue River",
+      "product": "32B6",
+      "traits": "Conv",
+      "metrics": {
+        "MG": 3.2,
+        "Yield": 64.9,
+        "WestYield": 65.2,
+        "EastYield": 61.7,
+        "Atlantic": 75.3,
+        "Corning": 63.7,
+        "Chariton": 56.7,
+        "Batavia": 57.2,
+        "Crawfordsville": 71.2
+      }
+    },
+    {
+      "rank": 10,
+      "brand": "Renk",
+      "product": "G2900E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 2.9,
+        "Yield": 64.7,
+        "WestYield": 65.7,
+        "EastYield": 60.3,
+        "Atlantic": 70.2,
+        "Corning": 72.4,
+        "Chariton": 54.4,
+        "Batavia": 55.9,
+        "Crawfordsville": 70.5
+      }
+    },
+    {
+      "rank": 11,
+      "brand": "Nutech/G2 Genetics",
+      "product": "32N05E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 3.2,
+        "Yield": 64.3,
+        "WestYield": 66.7,
+        "EastYield": 60.4,
+        "Atlantic": 72.3,
+        "Corning": 68.3,
+        "Chariton": 59.3,
+        "Batavia": 56.4,
+        "Crawfordsville": 65.4
+      }
+    },
+    {
+      "rank": 12,
+      "brand": "Golden Harvest",
+      "product": "GH3035E3",
+      "traits": "E3",
+      "metrics": {
+        "MG": 3,
+        "Yield": 64,
+        "WestYield": 64.9,
+        "EastYield": 59.7,
+        "Atlantic": 71.9,
+        "Corning": 69.4,
+        "Chariton": 53.5,
+        "Batavia": 59.3,
+        "Crawfordsville": 66.5
+      }
+    },
+    {
+      "rank": 13,
+      "brand": "Xitavo",
+      "product": "XO 2926E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 2.9,
+        "Yield": 63.8,
+        "WestYield": 64.5,
+        "EastYield": 59.3,
+        "Atlantic": 71,
+        "Corning": 69.5,
+        "Chariton": 52.9,
+        "Batavia": 57.1,
+        "Crawfordsville": 67.8
+      }
+    },
+    {
+      "rank": 14,
+      "brand": "ISU",
+      "product": "IAS31C2",
+      "traits": "Conv",
+      "metrics": {
+        "MG": 3.1,
+        "Yield": 63.7,
+        "WestYield": 65.8,
+        "EastYield": 60.2,
+        "Atlantic": 72.1,
+        "Corning": 66.7,
+        "Chariton": 58.6,
+        "Batavia": 56.2,
+        "Crawfordsville": 65.7
+      }
+    },
+    {
+      "rank": 15,
+      "brand": "NK Seeds",
+      "product": "NK28-G7E2S",
+      "traits": "E3S",
+      "metrics": {
+        "MG": 2.8,
+        "Yield": 63.6,
+        "WestYield": 63.7,
+        "EastYield": 60.7,
+        "Atlantic": 68.2,
+        "Corning": 68.1,
+        "Chariton": 54.8,
+        "Batavia": 60.3,
+        "Crawfordsville": 66.9
+      }
+    },
+    {
+      "rank": 16,
+      "brand": "Pioneer",
+      "product": "P30A75E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 3,
+        "Yield": 63.3,
+        "WestYield": 63.1,
+        "EastYield": 61.6,
+        "Atlantic": 69.6,
+        "Corning": 63.6,
+        "Chariton": 56,
+        "Batavia": 61.2,
+        "Crawfordsville": 67.5
+      }
+    },
+    {
+      "rank": 17,
+      "brand": "NK Seeds",
+      "product": "NK30-A9E3",
+      "traits": "E3",
+      "metrics": {
+        "MG": 3,
+        "Yield": 62.5,
+        "WestYield": 64.6,
+        "EastYield": 58.9,
+        "Atlantic": 70.3,
+        "Corning": 66.6,
+        "Chariton": 56.9,
+        "Batavia": 53.7,
+        "Crawfordsville": 66
+      }
+    },
+    {
+      "rank": 18,
+      "brand": "Xitavo",
+      "product": "XO 2865E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 2.8,
+        "Yield": 62.5,
+        "WestYield": 62.2,
+        "EastYield": 59.7,
+        "Atlantic": 70.7,
+        "Corning": 63.4,
+        "Chariton": 52.4,
+        "Batavia": 59.5,
+        "Crawfordsville": 67.2
+      }
+    },
+    {
+      "rank": 19,
+      "brand": "Asgrow",
+      "product": "AG29XF5",
+      "traits": "RR2XF",
+      "metrics": {
+        "MG": 2.8,
+        "Yield": 62.4,
+        "WestYield": 65.3,
+        "EastYield": 57.7,
+        "Atlantic": 70.4,
+        "Corning": 67.8,
+        "Chariton": 57.7,
+        "Batavia": 50.5,
+        "Crawfordsville": 65
+      }
+    },
+    {
+      "rank": 20,
+      "brand": "Confluence Genetics",
+      "product": "BH31Q146",
+      "traits": "Conv",
+      "metrics": {
+        "MG": 3.1,
+        "Yield": 62,
+        "WestYield": 62.1,
+        "EastYield": 59.5,
+        "Atlantic": 67.8,
+        "Corning": 62.4,
+        "Chariton": 56.2,
+        "Batavia": 57.6,
+        "Crawfordsville": 64.7
+      }
+    },
+    {
+      "rank": 21,
+      "brand": "ISU",
+      "product": "IAS29C1",
+      "traits": "Conv",
+      "metrics": {
+        "MG": 2.9,
+        "Yield": 61.9,
+        "WestYield": 62,
+        "EastYield": 59.2,
+        "Atlantic": 65.3,
+        "Corning": 66.3,
+        "Chariton": 54.2,
+        "Batavia": 57.3,
+        "Crawfordsville": 66.2
+      }
+    },
+    {
+      "rank": 22,
+      "brand": "Pioneer",
+      "product": "P28A65E",
+      "traits": "E3",
+      "metrics": {
+        "MG": 2.8,
+        "Yield": 60.8,
+        "WestYield": 62.5,
+        "EastYield": 56.1,
+        "Atlantic": 70.6,
+        "Corning": 65.1,
+        "Chariton": 52,
+        "Batavia": 51.7,
+        "Crawfordsville": 64.6
+      }
+    },
+    {
+      "rank": 23,
+      "brand": "Confluence Genetics",
+      "product": "BX32Q405",
+      "traits": "Conv",
+      "metrics": {
+        "MG": 3.2,
+        "Yield": 59.5,
+        "WestYield": 61,
+        "EastYield": 57,
+        "Atlantic": 64.6,
+        "Corning": 62.1,
+        "Chariton": 56.5,
+        "Batavia": 51.6,
+        "Crawfordsville": 62.9
+      }
+    }
+  ],
+  "n_results": 23,
+  "source_urls": [
+    "https://www.croptesting.iastate.edu/Soybean/SoybeanDistrict2.aspx"
+  ],
+  "tos_note": "Footer 'Copyright (c) ...ISU...All rights reserved' (no automation clause, no robots.txt); public ICPT data; low request rate; attribute Iowa Crop Performance Tests / ISU.",
+  "fetched_at": "2026-06-10T12:13:24.976682+00:00",
+  "scraper_version": "0.1.0"
+}