justin/hvm-docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

eval: new baseline on the 4-endpoint embed pool index #9

Merged
justin merged 1 commits from eval-baseline-20260610 into main 2026-06-10 20:38:25 -04:00
Conversation 0 Commits 1 Files Changed 1
+141
justin commented 2026-06-10 20:38:23 -04:00
Owner
Copy Link
Copy Source

New eval/results/baseline-20260610.md: 22 queries against the prod image index rebuilt today on the 4-endpoint GPU pool with the resilient embedder (PR #8). vs the 2026-05-22 baseline: dense MRR 0.539→0.924, bm25+rerank 0.920→0.959, hybrid_rrf+rerank 0.875→0.960. Reranker legs ran against the prod jina-rerank. Provenance header in the file.

🤖 Generated with Claude Code

New `eval/results/baseline-20260610.md`: 22 queries against the prod image index rebuilt today on the 4-endpoint GPU pool with the resilient embedder (PR #8). vs the 2026-05-22 baseline: dense MRR 0.539→0.924, bm25+rerank 0.920→0.959, hybrid_rrf+rerank 0.875→0.960. Reranker legs ran against the prod jina-rerank. Provenance header in the file. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
justin added 1 commit 2026-06-10 20:38:24 -04:00
eval: new baseline on the 4-endpoint embed pool index 2e10279beb 
22 queries against the prod image index rebuilt today on the expanded
GPU pool with the resilient embedder (PR #8): dense MRR 0.539→0.924,
bm25+rerank 0.920→0.959, hybrid_rrf+rerank 0.875→0.960 vs the
2026-05-22 baseline. No regression from mixed-provenance embeddings.

Co-Authored-By: Claude Fable 5 <[email protected]>
justin merged commit 992d66e3d1 into main 2026-06-10 20:38:25 -04:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
ai-ready
Scoped and ready to implement
 autopilot
Drive end-to-end autonomously (claude-deck)
 autopilot-careful
Autopilot may scope/implement but never auto-merge
 bug
Defect
 feature
New capability
 needs-human
Autopilot stalled or vetoed — a human must act
 P0
Critical
 P1
High
 P2
Normal
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
justin (Justin Paul)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: justin/hvm-docs#9