feat: citations, eval sidecar/pvalue/trace, nomic prefixes

mcp 2.x already on main. Does not replace the variety chunker
(one chunk per variety is the anti-hallucination contract).

- Numbered [1] citations on search_docs / search_trials
- Eval JSONL sidecar, k-curve, eval.pvalue, eval.trace
- Nomic prefixes at embed time only; stored text unprefixed

Closes #25
This commit is contained in:
2026-09-30 08:22:50 -04:00
parent 4de5aaa02f
commit d131e93252
12 changed files with 509 additions and 53 deletions
+41
View File
@@ -0,0 +1,41 @@
# seed-mcp retrieval eval — k=5
_21 golden queries × 4 retrievers_
## Summary
| Retriever | Passed | Recall | P@1 | MRR | Avg ms |
|---|---|---|---|---|---|
| **hybrid+rerank** | 21/21 | 100.00% | 90.48% | 0.905 | 2064 |
| **bm25** | 20/21 | 95.24% | 80.95% | 0.833 | 5 |
| **hybrid** | 15/21 | 71.43% | 61.90% | 0.619 | 73 |
| **dense** | 14/21 | 66.67% | 38.10% | 0.440 | 79 |
**Recall** = % of queries where ≥1 top-k chunk satisfied the spec. **P@1** = % where the very first result satisfied it. **MRR** = mean of `1 / rank-of-first-satisfying-result` (0 if missed).
## Per-query results
| Query | bm25 | dense | hybrid | hybrid+rerank |
|---|---|---|---|---|
| `DKC62-08RIB ratings` | ✅ #1 | ❌ | ❌ | ✅ #1 |
| `AG29XF4 disease ratings` | ✅ #1 | ❌ | ❌ | ✅ #1 |
| `WB6430 westbred wheat` | ✅ #1 | ❌ | ❌ | ✅ #1 |
| `E085Z5 corn` | ✅ #1 | ❌ | ❌ | ✅ #1 |
| `AP Iliad wheat performance` | ✅ #1 | ❌ | ❌ | ✅ #1 |
| `drought tolerant corn for sandy soil short season Iowa` | ✅ #2 | ✅ #1 | ✅ #1 | ✅ #1 |
| `soybean cyst nematode SCN resistant variety` | ✅ #1 | ✅ #1 | ✅ #1 | ✅ #1 |
| `Phytophthora resistance Rps3a soybean` | ✅ #1 | ✅ #2 | ✅ #1 | ✅ #1 |
| `XtendFlex soybean Northern Plains` | ❌ | ✅ #1 | ✅ #1 | ✅ #1 |
| `Hard Red Spring wheat stripe rust resistance` | ✅ #1 | ✅ #3 | ✅ #1 | ✅ #1 |
| `Soft White Winter wheat Pacific Northwest` | ✅ #1 | ✅ #5 | ✅ #1 | ✅ #1 |
| `Goss's Wilt resistance corn` | ✅ #1 | ✅ #1 | ✅ #1 | ✅ #1 |
| `best corn 2024 Iowa` | ✅ #1 | ✅ #1 | ✅ #1 | ✅ #1 |
| `Indiana corn yield comparison 2024` | ✅ #1 | ✅ #1 | ✅ #1 | ✅ #1 |
| `AP Iliad Idaho wheat trial` | ✅ #1 | ✅ #5 | ✅ #1 | ✅ #1 |
| `DKC65-95 corn yield in trials` | ✅ #1 | ❌ | ✅ #1 | ✅ #1 |
| `NK1701 corn trials head to head` | ✅ #1 | ❌ | ❌ | ✅ #1 |
| `silage corn high milk per acre dairy` | ✅ #1 | ✅ #1 | ✅ #1 | ✅ #1 |
| `soybean 2025 Minnesota top performers` | ✅ #1 | ✅ #1 | ✅ #1 | ✅ #1 |
| `Pioneer P1142 hybrid recommendation` | ✅ | ✅ | ✅ | ✅ |
| `DKC65-20 yield Alabama trial` | ✅ | ✅ | ✅ | ✅ |