Files

T

justin 0e625553e5 gh_plot_reports corpus (4,299 plots) + concurrency + 4-GPU pool

CORPUS — 4,299 GH plot reports added (3,797 written + 502 from the
earlier slow run + 319 sitemap-listed URLs that 404'd as
discontinued). Combined with prior 760 varieties + 14 AgriPro
trials = 5,073 total chunks now indexed.

scrape/sources/gh_plot_reports.py — concurrency speedup:
- 4 worker threads (ThreadPoolExecutor), each with its own
  requests.Session for connection-pool efficiency.
- Shared class-level rate limiter (0.25 sec between ANY two
  requests across all threads). Net throughput ~4 req/sec —
  well below any rate-limit threshold a public site enforces.
- Diagnosis vs original 1 req/sec: GH had ZERO rate limiting,
  zero 429s, zero retries. The 1 sec self-throttle was just too
  conservative. Bench:
    1 worker  / 1.0 sec throttle:  ~0.4 plots/sec (190 min ETA)
    4 workers / 0.25 sec throttle: ~3 plots/sec  (~25 min actual)

rag/chunk.py — chunk size cap for nomic-embed-text's 2048-token
context window:
- Empirically tested: failure threshold is ~5,250 chars on
  numeric-heavy trial chunks (chars/token ratio 2.4 vs 3.5 for
  prose). Cap at 4,500 chars to be safely under at worst-case
  2.2 chars/token.
- Applied to BOTH variety and trial chunks. Marked truncated
  chunks with metadata.embed_truncated = True; FULL text stays
  in the on-disk .md for get_page to return verbatim.

.gitea/workflows/{refresh,image-only}.yml — OLLAMA_URL pool
restructured for the 4 GPU-pinned endpoints. Bench (50-chunk
batches on nomic-embed-text):

    .0.125:11434  (RTX 40-series)  242 embeds/sec  ← weight ×4
    .0.2:11436    (GPU-pinned)     108 embeds/sec  ← weight ×2
    .0.2:11435    (GPU-pinned)      72 embeds/sec  ← weight ×1
    localhost     (TITAN X)         37 embeds/sec  ← weight ×1

Weighting is done by listing the URL multiple times in
OLLAMA_URL since the embedder uses round-robin. .0.2:11434 is
explicitly EXCLUDED — it isn't pinned to a specific GPU.

Combined index rebuild for 5,073 chunks now finishes in ~3 min
(was 19+ on the single-endpoint pool).

Smoke tests:
✓ list_versions: 5,073 docs across 6 sources, 2 vendors, 6
  brands, 4 crops (corn 2711, soy 2016, silage 223, wheat 123).
✓ search_trials({crop=corn, state=IA, year=2024}): 3 IA 2024
  corn trials surfaced.
✓ search_trials("Phytophthora resistance soybean trial"): NK
  NK43-W1XFS top-1 in LA 2024 trial (cross-vendor result).
✓ search_trials("AP Iliad Idaho wheat"): AgriPro Washington/N
  Idaho 2025 trial surfaced.
✓ search_trials(product=DKC65-95): 3 corn trials containing
  that hybrid in IL/IA 2024.
✓ search_trials(product=NK1701): 3 corn trials in AR/MS 2024.
✓ Product filter correctly returns EMPTY for products that
  aren't in the corpus (DKC65-20 is a 2023 product; 2023 plots
  deferred). Anti-hallucination contract preserved.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-25 16:46:35 -04:00

3.4 KiB

Raw Blame History

Silage yield trial — Wisconsin, 2024

Source: Golden Harvest plot report (cross-vendor head-to-head)
Crop: Silage
State: Wisconsin (WI)
Year: 2024
Plot ID: 2465531
Cooperator: Tom Kopp Farms
Planted: 2024-05-10
Harvested: 2024-09-16
Population: 37,000 seeds/acre
Row width: 30"
URL: https://www.goldenharvestseeds.com/silage/plot-report/wi/2024/2465531

Results (top-down by rank)

Rank	Brand	Product	Traits	Ton/Acre	Milk Per Acre	Milk Per Ton	Beef Per Acre	Beef Per Ton
1	Legacy	Seeds	Experimental	33.0	3520.0	34858.0	252.0	2618.0
2	Midwest	Seed	Genetics MW 09-90 PCE	32.3	3396.0	32949.0	233.0	2481.0
3	Pioneer	P14830Q	-	31.8	3604.0	34412.0	270.0	2579.0
4	Brevant	B09C43V	-	30.9	3082.0	28561.0	204.0	2229.0
5	Brevant	B08V35Q	-	30.6	3525.0	32351.0	257.0	2432.0
6	Legacy	Seeds	LC554-23 SS	30.5	3475.0	31774.0	259.0	2441.0
7	Legacy	Seeds	LC644-22 DV	30.4	3292.0	30058.0	220.0	2277.0
8	Legacy	Seeds	Experimental	30.0	3434.0	30924.0	245.0	2343.0
9	Legacy	Seeds	LC572-22 SS	29.9	3383.0	30316.0	234.0	2289.0
9	Legacy	Seeds	Experimental	29.9	3750.0	33623.0	284.0	2481.0
11	Dairyland	Seed	HiDF-5033V	29.6	3594.0	31902.0	267.0	2390.0
12	Legacy	Seeds	Experimental	29.4	3647.0	32135.0	262.0	2391.0
13	Pioneer	P13777V	-	29.3	3390.0	29826.0	235.0	2255.0
14	Renk	Seed	RK773TRE	29.2	3487.0	30589.0	240.0	2289.0
15	Pioneer	P0924Q	-	29.1	3612.0	31502.0	267.0	2356.0
15	Enogen	E107C1-D	-	29.1	3608.0	31490.0	260.0	2350.0
17	Legacy	Seeds	LC623-21 DV	28.9	3552.0	30778.0	252.0	2302.0
18	Legacy	Seeds	Experimental	28.8	3725.0	32237.0	290.0	2451.0
19	Enogen	E105Z5-D	-	28.5	3633.0	31041.0	285.0	2374.0
20	Brevant	B06F91Q	-	28.4	3395.0	28931.0	235.0	2203.0
21	Pioneer	P1089AMXT	-	28.1	3437.0	29018.0	243.0	2187.0
22	Brevant	B12C01Q	-	27.6	3449.0	28544.0	248.0	2155.0
23	Dairyland	Seed	HiDF-4808V	27.5	3677.0	30333.0	269.0	2255.0
24	Pioneer	P0720Q	-	27.4	3753.0	30823.0	283.0	2291.0
25	Brevant	B09T70SX	-	27.2	3241.0	26402.0	218.0	2015.0
26	Legacy	Seeds	LC605-23 PCE	27.1	3427.0	27870.0	234.0	2100.0
27	Pioneer	P08527V	-	26.9	3796.0	30603.0	286.0	2256.0
27	NK	NK0440-AT	-	26.9	3626.0	29274.0	262.0	2181.0
29	Dairyland	Seed	HiDF-5000Q	25.6	3410.0	26237.0	233.0	1981.0
30	Brevant	B05D47V	-	25.5	3738.0	28650.0	281.0	2120.0
31	Legacy	Seeds	Experimental	25.2	3642.0	27540.0	259.0	2035.0
32	Renk	Seed	RK811PCE	25.1	3367.0	25356.0	240.0	1952.0
33	Pioneer	P09312V	-	24.8	3299.0	24510.0	215.0	1858.0
34	NK	NK0440-AT	-	24.3	3369.0	24541.0	234.0	1862.0
35	Legacy	Seeds	Experimental	22.2	3540.0	23592.0	257.0	1768.0
36	Renk	Seed	RK700SSTX	21.7	3438.0	22406.0	245.0	1696.0

Top 5 by Ton/Acre: Seeds (Legacy) 33.0, Seed (Midwest) 32.3, P14830Q (Pioneer) 31.8, B09C43V (Brevant) 30.9, B08V35Q (Brevant) 30.6.

3.4 KiB Raw Blame History

Silage yield trial — Wisconsin, 2024

Results (top-down by rank)

3.4 KiB

Raw Blame History