justin/hvm-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Default search_docs to dense; make rerank opt-in (#17)

Browse Source 
Co-authored-by: claude <[email protected]>
This commit was merged in pull request #17.
This commit is contained in:
claude
2026-09-29 22:45:25 -04:00
committed by claude
parent 92219878d9
commit dc4a96e8d5
5 changed files with 60 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+11 -11
View File
@@ -17,7 +17,7 @@ once deployed.
| Tool | Use |
|---|---|
| `search_docs` | BM25-default search with optional version / platform / bundle filters; cross-encoder reranked when `RERANK_URL` is set |
| `search_docs` | Dense-default search with optional version / platform / bundle filters; rerank only when `RERANK_ENABLED=true` |
| `get_page` | Full markdown of one page with metadata header + source URL |
| `list_versions` | Discover available versions, doc types, and bundle slugs |
| `list_cluster` | Cross-version peers of a page (synthesized from same-GUID overlap) |
@@ -51,19 +51,19 @@ peer mapping is free (no fuzzy matching needed).
## Retrieval
Eval against 22 hand-curated golden queries — see
[`eval/results/baseline.md`](eval/results/baseline.md):
Eval against 22 hand-curated golden queries. May 2026 baseline is
[`eval/results/baseline.md`](eval/results/baseline.md); after nomic
prefixes + heading-recursive chunking (2026-09-30, live index):
| Retriever | MRR | Recall@5 | nDCG@5 | latency |
| Retriever | P@1 | MRR | Recall@5 | nDCG@5 |
|---|---:|---:|---:|---:|
| dense (Ollama nomic-embed-text) | 0.539 | 0.621 | 0.558 | 88 ms |
| BM25 (SQLite FTS5) | 0.880 | 0.909 | 0.883 | 3 ms |
| hybrid (dense + BM25 + RRF) | 0.692 | 0.818 | 0.713 | 69 ms |
| **bm25 + jina-rerank** | **0.920** | **0.939** | **0.927** | 490 ms (CPU) / ~50 ms (GPU) |
| **dense** (nomic prefixes) | **0.955** | **0.966** | **0.985** | **0.972** |
| hybrid (dense + BM25 + RRF) | 0.955 | 0.961 | 0.955 | 0.955 |
| BM25 (SQLite FTS5) | 0.864 | 0.882 | 0.909 | 0.886 |
| bm25 + jina-rerank | 0.773 | 0.827 | 0.848 | 0.816 |
HPE docs use controlled vocabulary, so lexical match dominates; the
cross-encoder cleans up the long tail. See PLAN.md Phase 7/8 for the
reasoning.
`search_docs` defaults to dense. Rerank is opt-in (`RERANK_ENABLED=true`)
— it now hurts. Full table: [`eval/results/post-prefix.md`](eval/results/post-prefix.md).
## Architecture