2 Commits

Author	SHA1	Message	Date
justin	af44d7a102	Phase 11 + Phase 6 GPU move ... ## Phase 11 — Curated agronomy / label-handling knowledge layer docs_mcp/lessons.md: 13 topic-anchored markdown sections covering the LLM-side context a farmer-advisor needs alongside the raw label corpus — - how-to-use-this-corpus - epa-signal-words - rei-phi-fundamentals - rup-handling - supplemental-labels-24c-2ee - tank-mix-fundamentals - resistance-management-hrac-frac-irac - glufosinate-application-rules - dicamba-application-rules - lake-erie-watershed-ohio - scn-and-other-seed-treatment-context - drift-management-essentials - how-to-format-recommendations Each Topic block is independently retrievable via the new MCP tool: ppls_api_lessons(topic="rup-handling") Or with no topic to get the full TOC, or with a substring to match-and-return matching sections ("dicamba" → dicamba-application-rules). Tool docstring instructs the LLM to call this proactively before any pesticide recommendation so the recommendation lands with regulatory framing, resistance-group callouts, RUP applicator language, and the canonical recommendation format — not just a rate from a label. ## Phase 6 — Reranker moved to GPU on trashpanda Stopped the local CPU container and started on trashpanda's Tesla P4 (8 GB VRAM) via: docker run -d --name llama-rerank --restart unless-stopped --gpus all \ -p 8082:8080 \ ghcr.io/ggml-org/llama.cpp:server-cuda \ -hf gpustack/jina-reranker-v2-base-multilingual-GGUF:Q8_0 \ --reranking --host 0.0.0.0 --port 8080 -ngl 99 The :server-cuda image variant (not :server) is required for CUDA backend; -ngl 99 offloads all layers to GPU. Latency: 50-doc rerank dropped from ~23 s on CPU to ~0.7-1.5 s on the Tesla P4 — production-grade interactive speeds. deploy/rerank-docker.md updated with the trashpanda deploy recipe, troubleshooting (mostly "did you use server-cuda?"), and a perf reference table. The MCP server's RERANK_URL just points at http://10.10.1.65:8082 now. GPU eval still completing in background; results land in eval/results/with_rerank_gpu.md as a follow-up commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-24 12:10:09 -04:00
justin	278fe5f456	Phase 6: reranker sidecar (jina-reranker-v2-base via llama.cpp) ... Wires the docs_mcp/server.py reranker hook into a real backend: ghcr.io/ggml-org/llama.cpp:server \\ -hf gpustack/jina-reranker-v2-base-multilingual-GGUF:Q8_0 \\ --reranking --host 0.0.0.0 --port 8080 Setup recipe at deploy/rerank-docker.md. The MCP server already honors RERANK_URL (added in Phase 7+8 commit); setting it to http://<host>:8082 turns on rerank automatically. ## Eval results (35 queries, k=5, pool=50) | Retriever | MRR | Recall@5 | nDCG@5 | |----------------|-------|----------|--------| | dense | 0.027 | 0.086 | 0.041 | | bm25 | 0.544 | 0.586 | 0.524 | | hybrid-rrf | 0.114 | 0.114 | 0.108 | | dense+rerank | 0.171 | 0.143 | 0.149 | | hybrid+rerank | 0.672 | 0.638 | 0.621 | ← winner The reranker fixes hybrid's failure mode (dense noise polluting the fused pool) by scoring each (query, chunk) pair independently. Net: hybrid+rerank gives +24% MRR over BM25-only. Smoke test for the reranker itself (query: "soybean herbicide for waterhemp", 4 candidates): index=1 SENCOR metribuzin waterhemp soybean → score=0.84 ← right index=3 Headline wheat fungicide → score=-2.80 index=2 Lorsban corn rootworm → score=-2.91 index=0 Roundup fallow burndown → score=-3.44 Strong separation between the right doc and the rest. ## Production gotchas - CPU-only reranker is slow (~23s for a 50-doc pool). For interactive use put it on GPU (`--gpus all`); ~10-20× faster. - jina-reranker rejects the ENTIRE batch if any pair exceeds n_ctx_train=1024 — server truncates each doc to 2000 chars before sending. Already handled in _rerank_pool. Per-query rerank report at eval/results/with_rerank.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-24 10:50:03 -04:00