Add 4 independent seed brands: Latham, Stine, 1st Choice, Burrus (+623 varieties)
Four independent regional brands across IA/IN/IL (variety-identity sources,
each parsed into structured characteristics_groups so ratings embed):
- latham (264: 155 corn + 109 soy) — Latham Hi-Tech Seeds, Alexander IA.
WordPress REST enum (/wp-json/wp/v2/varieties) + /products/<slug>/ detail
HTML. Scale 1-9 LOWER=better (reversed, like NK/AgriPro).
- stine (217: 58 corn + 159 soy) — Stine Seed, Adel IA (largest US
independent). sitemap enum + /{crop}/traits/<slug>/<code>/ detail HTML.
Corn 1-9 (9=best); soy qualitative.
- first_choice (78: 52 corn + 22 soy + 4 wheat) — 1st Choice Seeds,
Rushville IN (employee-owned). Per-crop sitemap -> detail HTML. Scale
0-10 higher=better. ~40 older corn pages thin at source; wheat
identity-only.
- burrus (64: 38 corn + 26 soy) — Burrus Seed, Arenzville IL. Seedware
JSON API. Scale 1-10 (10=best). Brands Burrus/Power Plus/DONMARIO.
robots ai-train=no + named-bot blocks; operator opted in, scraper uses a
non-blacklisted UA + honors Crawl-delay 10.
All 623 validated through rag.chunk.chunks_from_variety (0 errors; 6
identity-only pages from source gaps). No chunk.py change needed (identity
sources auto-route to chunks_from_variety).
Docs:
- sources.json: 4 entries + Hoegemeyer added to _excluded_sources. The
Corteva ToU (shared across pioneer.com / hoegemeyer.com / therightseed.com
/ corteva.com + the Vylor spinoff) bans scrapers + competitive use, so the
whole Corteva family is one excluded ToU domain.
- docs_mcp/lessons.md: rating-scales updated with all 4 directions +
an explicit cross-vendor warning (Latham 1=best vs Stine/Burrus higher=best
— never compare raw numbers without _scale_direction).
- README + CLAUDE corpus inventory: now 2,268 variety + 6,787 trial records.
CI rebuilds the index from the committed corpus.
This commit is contained in:
@@ -29,16 +29,25 @@ and the `crop_seed_api_lessons` tool).
|
||||
| LG Seeds (AgReliant) | 🟢 | 170 | `lgseeds.com` JSON XHR (+ `lg_plot_reports` trials) |
|
||||
| Golden Harvest (Syngenta) | 🟢 | 139 | sitemap.xml + server-rendered HTML + Syngenta CDN PDFs (+ `gh_plot_reports` trials) |
|
||||
| NK (Syngenta) | 🟢 | 122 | static HTML + Syngenta CDN PDFs (shares fetcher with Golden Harvest) |
|
||||
| **Latham Hi-Tech Seeds** (independent, IA) | 🟢 | **264** | WordPress REST enum (`/wp-json/wp/v2/varieties`) + `/products/<slug>/` detail HTML. Scale 1-9 **LOWER=better** (reversed) |
|
||||
| **Stine Seed** (independent, IA — largest US) | 🟢 | **217** | custom PHP; `sitemap.xml` enum + `/{crop}/traits/<slug>/<code>/` detail HTML. Corn 1-9 (9=best); soy qualitative |
|
||||
| LG Seeds (AgReliant) | 🟢 | 170 | `lgseeds.com` JSON XHR (+ `lg_plot_reports` trials) |
|
||||
| Golden Harvest (Syngenta) | 🟢 | 139 | sitemap.xml + server-rendered HTML + Syngenta CDN PDFs (+ `gh_plot_reports` trials) |
|
||||
| NK (Syngenta) | 🟢 | 122 | static HTML + Syngenta CDN PDFs (shares fetcher with Golden Harvest) |
|
||||
| **ProHarvest Seeds** (independent, IL) | 🟢 | **119** | WordPress REST API (`/wp/v2/seed` + `/seed/<slug>/` detail pages) (+ `proharvest_plots` trials) |
|
||||
| AgriGold (AgReliant) | 🟢 | 111 | `agrigold.com` server-rendered HTML (+ `agrigold_plot_reports` trials) |
|
||||
| **1st Choice Seeds** (independent, IN) | 🟢 | **78** | WordPress (CPTs not in REST); per-crop sitemap → detail HTML. Scale 0-10 higher=better. corn/soy/wheat |
|
||||
| **Burrus Seed** (independent, IL) | 🟢 | **64** | Seedware JSON API (`burrus25.seedware.net`, callback+Referer). Scale 1-10 (10=best). robots `ai-train=no` — operator opted in |
|
||||
| Ebbert's Seeds (independent, OH/IN) | 🟢 | 29 | WordPress per-crop catalog pages, verbatim body |
|
||||
| AgriPro (Syngenta wheat) | 🟢 | 24 | Drupal Views form, server-rendered HTML |
|
||||
| Beck's PFR | 🟡 | 2,089 | Public Sanity GROQ API at `mc8v24rf.api.sanity.io` (no auth) |
|
||||
| Beck's products | 🟡 | 860 | Same Sanity API — identity-only until SeedIQ XHR is sniffed |
|
||||
| Pioneer (Corteva) | 🔴 | — | DROP. ToS bans automation; dealer locator login-gated too |
|
||||
| Pioneer + Hoegemeyer + Brevant (Corteva) | 🔴 | — | DROP. Shared corteva.com ToU bans automation (scrapers + "competitive service"). Treat ALL `*.corteva.com / pioneer.com / hoegemeyer.com / therightseed.com` + Vylor brands as one excluded ToU domain |
|
||||
|
||||
Trial-only sources (cross-vendor yield plots, `data_type=trial`): `gh_plot_reports`, `lg_plot_reports`, `agrigold_plot_reports`, `proharvest_plots`, `agripro_trials`. See the README corpus table for counts.
|
||||
|
||||
> **Scale-direction warning (read before any cross-vendor numeric comparison):** the independents do NOT agree on direction. Bayer + Stine(corn) + ProHarvest(disease) + Burrus = HIGHER is better (Burrus 1-10, others 1-9). **Latham + NK + AgriPro = LOWER is better (1 = best).** 1st Choice = 0-10 higher=better. Stine soy is qualitative. Always consult each record's `_scale_direction` (the chunker attaches it) before comparing numbers across brands.
|
||||
|
||||
**Build priority order** (shared-infra first → biggest yield):
|
||||
1. `bayer_seeds` — lift-and-shift from crop-chem-docs' Bayer scraper
|
||||
2. `golden_harvest` — biggest unique Syngenta brand
|
||||
|
||||
Reference in New Issue
Block a user