feat: port template upgrades (citations, eval, chunking, nomic prefixes)

mcp 2.x already on origin/main (#10). Does not change BM25-first
search_docs — n=6 is too small to flip the default.

- Numbered [1] citations via docs_mcp/format.py
- Eval P@1 + JSONL sidecar + eval.pvalue + eval.trace
- Heading-recursive chunker, keep chunk-0 and MAX_CHARS=4000
- Nomic prefixes at embed time only; stored text unprefixed

Closes #12
This commit is contained in:
2026-09-30 08:33:12 -04:00
parent 6897fbb19a
commit fda572f59e
13 changed files with 812 additions and 77 deletions
+5 -2
View File
@@ -18,7 +18,7 @@ import chromadb
from chromadb.config import Settings
from .chunk import chunks_from_page
from .embeddings import embedding_function
from .embeddings import EMBED_DOC_PREFIX, embed_texts, embedding_function
log = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
@@ -78,9 +78,12 @@ def upsert_to_chroma(records: list[dict]) -> int:
total = 0
for i in range(0, len(records), BATCH):
chunk = records[i:i + BATCH]
texts = [r["text"] for r in chunk]
vectors = embed_texts(texts, prefix=EMBED_DOC_PREFIX, ef=embedding_function())
col.upsert(
ids=[r["id"] for r in chunk],
documents=[r["text"] for r in chunk],
documents=texts,
embeddings=vectors,
metadatas=[r["metadata"] for r in chunk],
)
total += len(chunk)