Merge pull request 'README: rewrite for crop-chem-docs as a product (was template README)' (#1) from readme-update into main

The README had never been customized after cloning the
docs-mcp-template — title said "docs-mcp-template" and it read as
the template's generic introduction with no mention of EPA PPLS,
the Bayer scraper, the ~4k label corpus, or the production deploy.

Replace with a crop-chem-docs-specific README that covers:

- Corpus inventory: 4,159 indexed pages (91 Bayer + 4,068 EPA PPLS)
- MCP tool catalog with crop_chem_api_lessons specifics
- Eval baseline from eval/results/with_rerank.md showing
  hybrid+rerank wins (MRR 0.672) over BM25-only (0.544) and that
  hybrid-without-rerank actively HURTS (0.114) — same pattern
  seed-mcp found independently
- Note that the deployed rerank was silently failing through
  2026-05-25 due to the llama-rerank Docker network gotcha;
  fixed and re-running eval is on the followup list
- Quick-start commands
- Repo layout reference
- Infrastructure: registry, embedder pool, shared llama-rerank
  sidecar, PRODUCT_NAME=crop_chem
- Cross-link to the sibling seed-mcp project

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

.gitea/workflows/image-only.yml

@@ -20,8 +20,15 @@ on:
       - "Dockerfile"
       - "sources.json"
 
+# If multiple pushes land in quick succession, cancel the older one
+# rather than queueing both — each run is ~90 min and the older
+# commit's image just gets overwritten by the newer one anyway.
+concurrency:
+  group: image-only
+  cancel-in-progress: true
+
 env:
-  REGISTRY_PUSH: git.jpaul.io
+  REGISTRY_PUSH: 192.168.0.2:1234
   REGISTRY_PULL: git.jpaul.io
   IMAGE: ${{ github.repository_owner }}/${{ github.event.repository.name }}
   OLLAMA_URL: http://192.168.0.2:11434,http://192.168.0.2:11435,http://192.168.0.125:11434
@@ -94,6 +101,12 @@ jobs:
           esac
 
       - name: Prune old container versions
+        # GC requires broader scope than REGISTRY_TOKEN's push perms
+        # (got HTTP 403 enumerating /packages/.../versions on run #122).
+        # Non-critical — housekeeping only. Don't fail the whole run.
+        # TODO: issue a separate PAT with admin:package scope and set
+        # as PACKAGES_ADMIN_TOKEN, then use it here.
+        continue-on-error: true
         env:
           GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
         run: |

.gitea/workflows/refresh.yml

@@ -27,7 +27,7 @@ on:
 
 env:
   # Self-hosted Gitea registry on the same LAN as the runner.
-  REGISTRY_PUSH: git.jpaul.io
+  REGISTRY_PUSH: 192.168.0.2:1234
   REGISTRY_PULL: git.jpaul.io
   IMAGE: ${{ github.repository_owner }}/${{ github.event.repository.name }}
 
@@ -148,7 +148,11 @@ jobs:
           esac
 
       - name: Prune old container versions
+        # GC requires broader scope than REGISTRY_TOKEN's push perms
+        # (HTTP 403 on /packages/.../versions). Non-critical housekeeping.
+        # TODO: issue separate PAT with admin:package scope.
         if: steps.commit.outputs.changed == 'true' || inputs.force_build == true
+        continue-on-error: true
         env:
           GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
         run: |

Dockerfile

@@ -46,9 +46,13 @@ ENV PYTHONUNBUFFERED=1 \
     MCP_TRANSPORT=streamable-http \
     MCP_HOST=0.0.0.0 \
     MCP_PORT=8000 \
-    HYBRID_SEARCH=true
+    HYBRID_SEARCH=true \
-    # RERANK_URL set at deploy time, e.g. http://llama-rerank:8080
+    OLLAMA_URL=http://ollama:11434 \
-    # OLLAMA_URL set at deploy time, comma-separated list
+    RERANK_URL=http://llama-rerank:8080
+    # Defaults above assume the MCP container shares a Docker network
+    # with services named `ollama` and `llama-rerank`. Override either
+    # in the compose `environment:` block if your stack uses different
+    # service names or if you want to point at off-stack hosts.
 
 EXPOSE 8000
 

README.md

@@ -1,103 +1,146 @@
-# docs-mcp-template
+# crop-chem-docs
 
-A reusable template for building hosted MCP servers over a product's
+MCP server over ~4,000 public US row-crop pesticide / herbicide / fertilizer labels — feeding the same Drawbar farm-advisor AI as the sibling [`seed-mcp`](https://git.jpaul.io/justin/seed-mcp). The advisor calls this MCP for label rates, REI/PHI, rotation restrictions, tank-mix guidance, and active-ingredient lookups.
-public documentation. Distilled from one production build; everything
-product-specific has been factored out.
 
-The end product is a streamable-HTTP MCP server with ~15 tools that
+Built on [`docs-mcp-template`](https://git.jpaul.io/justin/docs-mcp-template) (same template lineage as seed-mcp). **In production** on trashpanda; the Drawbar advisor calls it via the `chem:` prefix.
-any LLM client (Claude Desktop, Claude Code, Cursor, Copilot) can
-call to answer questions against the docs, surface what changed
-recently, and flag likely inconsistencies.
 
-## What's here
+## What's in the corpus
 
-- **[PLAN.md](PLAN.md)** — comprehensive build guide. Phased
+**4,159 indexed pages** across two complementary sources:
-  approach (13 phases, ~2–3 weeks of focused work for the full
+
-  stack). Includes the design decisions, the gotchas, and a
+| Source | Pages | Notes |
-  per-product customization checklist.
+|---|---|---|
-- **Scaffolded skeleton** — working FastMCP server with stub tools,
+| `bayer` | 91 | Bayer Crop Science US product pages — Warrant, Harness, Roundup, Liberty, Capreno, etc. Rich Next.js `__NEXT_DATA__` payload: active ingredients, label rates, MOA codes, supplemental PDFs (24c / 2EE / bulletins). robots.txt explicitly whitelists RAG indexing. |
-  Dockerfile, docker-compose, CI workflows, eval harness layout,
+| `epa_ppls` | 4,068 | EPA Pesticide Product Label System — every registered ag chemistry product. Authoritative source of truth for EPA reg numbers, master labels, signal words, registrant info, formulations. |
-  usage logging. Everything you need to `git clone` and start
+
-  filling in the product-specific bits.
+## MCP tools
+
+Same shape as the docs-mcp-template's standard tools (see [`docs_mcp/server.py`](docs_mcp/server.py)):
+
+| Tool | Purpose |
+|---|---|
+| `search_docs` | Hybrid dense + BM25 + rerank search over the label corpus, filterable by source. |
+| `get_page` | Full label record by `(source, source_key)`. Returns marketing copy + extracted PDF text + sidecar metadata. |
+| `list_versions` | Facet discovery (sources, EPA registrant codes, label categories). |
+| `crop_chem_api_lessons` | Curated agronomy / regulatory lessons — EPA reg-number normalization, label-supersession ordering, common tank-mix gotchas. |
+| Plus the template's standard `diff_versions`, `bundle_changelog`, `weekly_digest` if needed. |
+
+## Retrieval — eval-validated
+
+From [`eval/results/with_rerank.md`](eval/results/with_rerank.md) (35 golden queries, k=5):
+
+| Retriever | MRR | Recall@5 | nDCG@5 | Time (s) |
+|---|---|---|---|---|
+| **hybrid+rerank** | **0.672** | **0.638** | **0.621** | 823 |
+| bm25 | 0.544 | 0.586 | 0.524 | 5 |
+| dense+rerank | 0.171 | 0.143 | 0.149 | 805 |
+| hybrid-rrf | 0.114 | 0.114 | 0.108 | 8 |
+| dense | 0.027 | 0.086 | 0.041 | 5 |
+
+**Deploy config**: `HYBRID_SEARCH=true` + `RERANK_URL=http://llama-rerank:8080`.
+
+Pattern matches what seed-mcp found independently:
+
+1. **Dense embedding alone is essentially useless** on this corpus (MRR 0.027). Variety codes, EPA reg numbers, and active-ingredient names have no semantic neighbors — `nomic-embed-text` returns noise.
+2. **Hybrid-rrf (no rerank) is worse than BM25 alone.** RRF dilutes BM25's strong ranking with dense noise. Don't ship without rerank.
+3. **BM25 alone (MRR 0.544, 5 sec) is a great fallback** when the rerank sidecar is unavailable.
+4. **Rerank brings the win** — `hybrid+rerank` MRR 0.672 is 23% better than BM25 alone and dominates every other configuration.
+
+**Note on rerank in production**: through 2026-05-25 the `llama-rerank` sidecar was attached to Docker's default `bridge` network instead of `drawbar-backend_default`, so chem-mcp's `RERANK_URL=http://llama-rerank:8080` was resolving via public DNS to a random IP and connection-refusing. The MCP fell back to dense+BM25 silently. Fixed via `docker network connect drawbar-backend_default llama-rerank`. Re-running the eval is on the follow-up list; expect the deployed MRR to lift toward the lab number.
 
 ## Quick start
 
 ```bash
-git clone https://git.jpaul.io/justin/docs-mcp-template.git my-product-docs
+git clone https://git.jpaul.io/justin/crop-chem-docs.git
-cd my-product-docs
+cd crop-chem-docs
-git remote remove origin  # detach from template
 python -m venv venv && source venv/bin/activate
 pip install -r requirements.txt
 
-# Read PLAN.md before doing anything else. Pay particular attention to
+# Sample-scrape to verify wiring:
-# Phase 1 (scraper) — that's the most product-specific phase.
+python -m scrape.runner --source bayer --limit 5
 
-# Run the stub server (no corpus yet — just verifies the wiring):
+# Full refresh (be polite — bayer is small, epa_ppls is hours):
-python -m docs_mcp.server --transport stdio
+python -m scrape.runner --source bayer --force
+python -m scrape.runner --source epa_ppls --force
+
+# Rebuild Chroma + BM25:
+OLLAMA_URL=http://192.168.0.125:11434 PRODUCT_NAME=crop_chem \
+  python -m rag.index --rebuild
+
+# Run the eval harness:
+RERANK_URL=http://localhost:18080 python -m eval.run_eval \
+  --queries eval/queries.jsonl --k 5 \
+  --output eval/results/baseline.md
+
+# Local MCP server (stdio for Claude Desktop dev):
+PRODUCT_NAME=crop_chem python -m docs_mcp.server --transport stdio
 ```
 
 ## Repo layout
 
 ```
 .
-├── PLAN.md                        # The build guide. Read first.
+├── CLAUDE.md                      # Canonical agent guide
+├── PLAN.md                        # Template's 13-phase build guide
 ├── README.md
 ├── requirements.txt
 ├── Dockerfile
-├── .gitignore
+├── deploy/
+│   ├── docker-compose.yml         # Drop-in compose for Drawbar
+│   ├── drawbar-compose-snippet.md # Notes on the parent compose merge
+│   └── rerank-docker.md           # llama-rerank service deployment
 ├── .gitea/workflows/
-│   ├── refresh.yml                # Weekly scrape + index + image push
+│   ├── refresh.yml                # Monthly cron: scrape + index + image push
-│   └── image-only.yml             # On-demand code-only ship
+│   └── image-only.yml             # On-demand code-only ship cycle
 ├── scrape/
-│   ├── README.md                  # Product-specific scraper goes here
+│   ├── runner.py                  # Dispatches `--source <id>`
-│   └── changelog.py               # Reusable: --json, --history-out
+│   ├── changelog.py               # Reusable: --json, --history-out
+│   └── sources/
+│       ├── bayer.py               # cropscience.bayer.us Next.js scraper
+│       └── epa_ppls.py            # EPA PPLS pagination + label PDFs
 ├── rag/
-│   ├── embeddings.py              # Ollama embedder, swappable
+│   ├── embeddings.py              # nomic-embed-text via Ollama
-│   ├── chunk.py                   # Chunker — adjust per page format
+│   ├── chunk.py                   # Chunker w/ EPA-reg-number preamble
-│   ├── index.py                   # Builds Chroma + (optionally) BM25
+│   ├── index.py                   # Chroma + BM25 builder
-│   └── bm25.py                    # SQLite FTS5 lexical index
+│   └── bm25.py                    # FTS5 lexical index
 ├── docs_mcp/
-│   ├── server.py                  # FastMCP server with stub tools
+│   ├── server.py                  # FastMCP — hybrid+rerank
+│   ├── lessons.md                 # Curated knowledge layer
 │   └── usage.py                   # TimedCall + JSONL telemetry
 ├── eval/
-│   ├── queries.jsonl.example      # Curate ~25 hand-labeled queries
+│   ├── queries.jsonl              # 35 golden queries
-│   ├── retrievers.py              # Retriever protocol + implementations
+│   ├── retrievers.py              # 5 named configurations
-│   └── run_eval.py                # MRR / Recall@k / nDCG@k harness
+│   ├── run_eval.py                # MRR / Recall@k / nDCG@k
+│   └── results/                   # Baseline + with_rerank measurements
 ├── scripts/
-│   ├── usage_report.py            # Standalone log analyzer
+│   ├── usage_report.py
 │   └── registry_gc.py             # Container registry cleanup
-└── deploy/
+└── corpus/                        # Committed scrape output (CI-refreshed)
-    └── docker-compose.yml         # Hosting stack: MCP + reranker + Watchtower
+    ├── bayer/
+    └── epa_ppls/
 ```
 
-## What's product-specific (must implement)
+## Infrastructure
 
-- `scrape/` — the scraper itself. The template gives you the corpus
+- **Registry**: pushes to `192.168.0.2:1234` (LAN, no CF body cap); deploys pull `git.jpaul.io/justin/crop-chem-docs:latest` (public, CF tunnel). Also tagged `:<sha12>` for rollback pinning and `:corpus-YYYY.MM.DD` for snapshot pinning.
-  layout contract and a working `changelog.py`; the actual extraction
+- **Embedder pool (CI)**: 3 GPU-pinned Ollama endpoints, weighted toward `.0.125` (RTX 40-series).
-  logic is yours.
+- **Reranker**: shared `llama-rerank` sidecar on trashpanda's Tesla P4 (`jina-reranker-v2-base-multilingual` via llama.cpp). Same container serves crop-chem-docs and seed-mcp.
-- The corpus on disk (gitignored; rebuilt by CI).
+- **PRODUCT_NAME**: `crop_chem` — used in `crop_chem_docs` Chroma collection, `bm25/crop_chem_docs.db`, and the `crop_chem_api_lessons` tool name.
-- The reranker GGUF model and llama.cpp container (commented in
-  `deploy/docker-compose.yml`).
-- The reverse proxy / TLS layer in front of the public endpoint.
-- The hand-curated knowledge surface (your product's API gotchas,
-  example scripts, anything the LLM should know that the docs
-  don't say).
 
-## What's NOT product-specific (works as-is)
+## Deploy mechanics
 
-- FastMCP server skeleton + tool decoration pattern
+Same Watchtower auto-deploy chain as seed-mcp. On every push to `main` that touches `docs_mcp/`, `rag/`, `scrape/`, `requirements.txt`, `Dockerfile`, or `sources.json`:
-- Chroma + Ollama embedding pipeline
-- BM25 / SQLite FTS5 lexical index
-- Hybrid retrieval (RRF) + reranker integration
-- Eval harness (Retriever protocol, MRR/Recall/nDCG)
-- Usage logging (TimedCall, JSONL, daily rotation)
-- CI workflow shape (weekly + on-demand, retry-on-race, three-tag
-  image scheme)
-- Registry GC script
-- Standard tools: `search_docs`, `get_page`, `list_versions`,
-  `diff_versions`, `bundle_changelog`, `weekly_digest`,
-  `find_doc_inconsistencies`, etc.
 
-## License
+1. `image-only.yml` checks out main + committed corpus
+2. Rebuilds Chroma + BM25 (~few min on the GPU pool)
+3. `docker build` + push three tags to the LAN registry
+4. Links the package to the repo via Gitea API
+5. Watchtower on trashpanda polls `:latest` every 5 min → recreates `drawbar-backend-chem-mcp-1`
 
-Internal template. Adjust before publishing.
+Corpus refresh runs monthly via `refresh.yml`. EPA PPLS is the slow source — ~hours at 1 req/sec at full scale.
+
+## Sibling
+
+[`seed-mcp`](https://git.jpaul.io/justin/seed-mcp) covers the row-crop seed-variety + yield-trial side of the advisor's tool catalog. Both MCPs are docs-mcp-template clones running side-by-side on trashpanda, sharing the Ollama pool and the `llama-rerank` sidecar.
+
+See [`CLAUDE.md`](./CLAUDE.md) for canonical sidecar schemas, the EPA reg-number normalization rules, and label-supersession ordering.

deploy/drawbar-compose-snippet.md

@@ -0,0 +1,148 @@
+# Drawbar deploy — `crop-chem-docs` MCP server snippet
+
+Drop these two services into Drawbar's `docker-compose.yml`. Targets
+the trashpanda stack: shared Docker network with the existing
+Drawbar services + the Cloudflare Tunnel.
+
+## Pre-reqs (one-time on the deploy host)
+
+1. **Docker login to the Gitea registry:**
+   ```bash
+   docker login git.jpaul.io -u justin   # PAT for password
+   ```
+2. **NVIDIA Container Toolkit** — already installed on trashpanda
+   (the existing standalone `llama-rerank` container ran with
+   `--gpus all` fine).
+3. **If a standalone `llama-rerank` container is already running**
+   (left over from earlier setup), remove it so the compose service
+   can bind the same name:
+   ```bash
+   docker rm -f llama-rerank
+   ```
+
+## Compose services
+
+```yaml
+services:
+
+  # ---- Reranker sidecar -----------------------------------------
+  # jina-reranker-v2-base-multilingual via llama.cpp on the Tesla P4.
+  # Internal port only (no host port mapping needed — the MCP reaches
+  # it via Docker DNS). ~280 MB GPU VRAM at idle, ~500 MB during a
+  # 50-doc rerank. Co-exists fine with any other GPU users on the P4.
+  llama-rerank:
+    image: ghcr.io/ggml-org/llama.cpp:server-cuda
+    container_name: llama-rerank
+    restart: unless-stopped
+    command:
+      - "-hf"
+      - "gpustack/jina-reranker-v2-base-multilingual-GGUF:Q8_0"
+      - "--reranking"
+      - "--host"
+      - "0.0.0.0"
+      - "--port"
+      - "8080"
+      - "-ngl"
+      - "99"            # offload all layers to GPU
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              count: all
+              capabilities: [gpu]
+    # Model cache survives container recreates; first start downloads
+    # the GGUF (~280 MB) from HuggingFace.
+    volumes:
+      - llama-rerank-cache:/root/.cache/huggingface
+    networks:
+      - default
+
+  # ---- MCP server ------------------------------------------------
+  crop-chem-docs:
+    image: git.jpaul.io/justin/crop-chem-docs:corpus-2026.05.24
+    # :latest for dev / Watchtower auto-pull
+    container_name: crop-chem-docs
+    restart: unless-stopped
+    ports:
+      - "8001:8000"   # MCP server (streamable-http). Adjust host port.
+    # No environment block needed — the image's defaults handle it:
+    #   OLLAMA_URL=http://ollama:11434
+    #   RERANK_URL=http://llama-rerank:8080
+    #   HYBRID_SEARCH=true
+    #   PRODUCT_NAME=crop_chem
+    # Override here only if your services have different names.
+    depends_on:
+      - llama-rerank
+    networks:
+      - default
+    labels:
+      com.centurylinklabs.watchtower.enable: "true"
+
+volumes:
+  llama-rerank-cache:
+```
+
+## Note on the existing `ollama` service
+
+The Dockerfile default is `OLLAMA_URL=http://ollama:11434` — that
+assumes there's an `ollama` service in the same compose stack. If
+trashpanda's Ollama is a host-mode process (not a compose service),
+override the env in the `crop-chem-docs` block:
+
+```yaml
+    environment:
+      OLLAMA_URL: "http://host.docker.internal:11434"
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+```
+
+Or just add Ollama itself to the compose stack as a sibling service.
+
+## Test once both are up
+
+```bash
+docker compose up -d llama-rerank crop-chem-docs
+
+# Wait ~10s for both to come up, then:
+docker exec crop-chem-docs python -c \
+  "from docs_mcp.server import corpus_status; print(corpus_status())"
+```
+
+Expect: `# crop-chem-docs corpus status`, 4,159 labels, 216,467
+chunks, BM25 db present, `RERANK_URL=http://llama-rerank:8080`,
+`HYBRID_SEARCH=on`.
+
+Then a live search to verify hybrid+rerank:
+
+```bash
+docker exec crop-chem-docs python -c \
+  "from docs_mcp.server import search_docs; print(search_docs('soybean herbicide for waterhemp', k=2))"
+```
+
+Expect: 2 hits with Sencor/Tackle/Warrant in top-2, `mode=hybrid-rrf+rerank` in the header.
+
+## What the MCP container exposes
+
+| Tool | What it does |
+|---|---|
+| `search_docs` | Hybrid+rerank pesticide-label search with optional filters |
+| `get_page` | Full label markdown + metadata by `(source, source_key)` |
+| `list_versions` | Discover sources, product classes, signal words, registrants |
+| `corpus_status` | Counts + freshness; useful for health probes |
+| `crop_chem_api_lessons` | Curated agronomy / label-handling knowledge — call before recommending |
+
+## Tag scheme
+
+| Tag | When | Use for |
+|---|---|---|
+| `:latest` | Every monthly refresh + every code push | Dev / Watchtower auto-pull |
+| `:<sha12>` | Every build | Rollback pin |
+| `:corpus-YYYY.MM.DD` | Every build | **Production pin** (frozen corpus version) |
+
+## Updating the corpus
+
+- **Monthly cron** — 1st @ 06:00 UTC, full re-scrape of Bayer + EPA PPLS,
+  reindex, image push. Watchtower pulls the new `:latest` automatically.
+- **Manual** — Gitea Actions UI → `Monthly corpus refresh` → `Run workflow`.
+  Optional `sources` input for single-source refresh (e.g., `bayer` only).

docs_mcp/server.py

@@ -541,7 +541,7 @@ def corpus_status() -> str:
     Cheap — no embedder call.
     """
     with TimedCall("corpus_status", {}) as _call:
-        lines: list[str] = ["# PPLS corpus status\n"]
+        lines: list[str] = ["# crop-chem-docs corpus status\n"]
 
         # On-disk corpus
         labels_by_source: dict[str, int] = {}

scripts/registry_gc.py

@@ -1,43 +1,69 @@
 """Gitea container-registry garbage collection.
 
-Lists package versions for one container package and deletes versions
+Prunes old container tags from a Gitea registry package. Always
-older than --keep-days. Always preserves:
+preserves:
 
-  - the :latest tag
+  - The ``latest`` tag (Watchtower auto-pull target)
-  - the --keep-latest most-recent date-tagged versions
+  - Any ``corpus-*`` tag (production pins; Drawbar may have them locked)
-  - anything pushed in the last --keep-days days
+  - The ``--keep-latest`` most-recent OTHER tags (typically commit-sha pins)
+  - Anything pushed within ``--keep-days`` days
 
-The actual disk reclaim happens on Gitea's next package GC cron (admin
+The actual disk reclaim happens on Gitea's next package GC cron
-site settings). This script just marks the versions for deletion.
+(admin site settings). This script marks versions for deletion.
+
+Why this script doesn't use the Docker Registry v2 API: that API has
+tag listing + manifest delete by digest, but no per-tag created-at
+timestamp without an extra blob-fetch round-trip. Gitea's packages
+API gives us {tag, created_at} in one call, which is what the keep
+policy needs.
+
+The endpoint shape that actually works (matches Gitea 1.21+):
+
+  GET    /api/v1/packages/{owner}?type=container&q={name}
+         → JSON array, ONE entry per tag, each with id + version=tag + created_at
+  DELETE /api/v1/packages/{owner}/container/{name}/{tag}
+         → 204 on success, 404 if already gone
+
+Auth: GITEA_TOKEN env var (PAT with delete:packages scope; the
+push-only PAT we use as REGISTRY_TOKEN may not be enough — if you
+see 403s, mint a separate PAT and pass it as GITEA_TOKEN here).
 
 Usage:
 
     python scripts/registry_gc.py \\
-        --owner <user> \\
+        --owner justin \\
-        --package <product>-docs-mcp \\
+        --package crop-chem-docs \\
-        --keep-days 90 \\
+        --keep-days 180 \\
-        --keep-latest 5
+        --keep-latest 6
-
+        [--dry-run]
-Auth: reads GITEA_TOKEN from env (set in the workflow as a secret).
 """
 from __future__ import annotations
 
 import argparse
+import json
 import os
 import sys
 from datetime import datetime, timedelta, timezone
-from urllib.request import Request, urlopen
 from urllib.error import HTTPError
-import json
+from urllib.request import Request, urlopen
 
 
 GITEA_HOST = os.environ.get("GITEA_HOST", "https://git.jpaul.io")
 
 
 def api(token: str, method: str, path: str) -> object:
-    req = Request(f"{GITEA_HOST}{path}",
+    # User-Agent matters: Cloudflare in front of git.jpaul.io returns
-                  headers={"Authorization": f"token {token}"},
+    # 403 to the default `Python-urllib/3.x` UA. Any non-Python UA
-                  method=method)
+    # passes. Curl works, requests works, we just need to not look
+    # like a vanilla urllib script.
+    req = Request(
+        f"{GITEA_HOST}{path}",
+        headers={
+            "Authorization": f"token {token}",
+            "User-Agent": "crop-chem-docs-registry-gc/0.1",
+        },
+        method=method,
+    )
     try:
         with urlopen(req, timeout=30) as r:
             body = r.read()
@@ -48,60 +74,93 @@ def api(token: str, method: str, path: str) -> object:
         raise
 
 
+def _parse_created(version: dict) -> datetime:
+    """Gitea returns RFC3339 with offset like '2026-05-24T16:07:50-04:00'.
+    Python 3.11+ handles this directly via fromisoformat."""
+    return datetime.fromisoformat(version["created_at"])
+
+
 def main() -> int:
     p = argparse.ArgumentParser()
     p.add_argument("--owner", required=True)
     p.add_argument("--package", required=True)
-    p.add_argument("--keep-days", type=int, default=90)
+    p.add_argument("--keep-days", type=int, default=180)
-    p.add_argument("--keep-latest", type=int, default=5)
+    p.add_argument("--keep-latest", type=int, default=6,
-    p.add_argument("--dry-run", action="store_true")
+                   help="Keep this many most-recent commit-sha (etc.) "
+                        "tags BEFORE applying --keep-days. corpus-* and "
+                        ":latest are kept regardless.")
+    p.add_argument("--dry-run", action="store_true",
+                   help="Show what would be deleted without calling DELETE.")
     args = p.parse_args()
 
     token = os.environ.get("GITEA_TOKEN")
     if not token:
-        print("GITEA_TOKEN not set", file=sys.stderr)
+        print("GITEA_TOKEN env var not set", file=sys.stderr)
         return 1
 
-    versions = api(token, "GET",
+    # Gitea's q= is a substring match; filter to exact name so we don't
-                   f"/api/v1/packages/{args.owner}/container/{args.package}/versions") or []
+    # accidentally GC a sibling package that shares the prefix.
+    versions = api(
+        token, "GET",
+        f"/api/v1/packages/{args.owner}?type=container&q={args.package}",
+    ) or []
+    versions = [v for v in versions if v.get("name") == args.package]
+
     if not versions:
-        print(f"no versions found for {args.owner}/{args.package}")
+        print(f"no versions found for {args.owner}/{args.package} — nothing to GC")
         return 0
 
     cutoff = datetime.now(timezone.utc) - timedelta(days=args.keep_days)
+    versions.sort(key=_parse_created, reverse=True)  # newest first
 
-    # Date-tagged versions (YYYY.MM.DD), newest first
+    keep: list[tuple[str, str]] = []     # (tag, reason)
-    date_tagged = []
+    delete: list[dict] = []
-    for v in versions:
+    other_kept = 0
-        tags = v.get("tags") or []
-        for t in tags:
-            if len(t) == 10 and t[4] == "." and t[7] == ".":
-                date_tagged.append((t, v))
-                break
-    date_tagged.sort(key=lambda kv: kv[0], reverse=True)
-    keep_date_tags = {t for t, _ in date_tagged[:args.keep_latest]}
 
-    deleted = 0
     for v in versions:
-        tags = v.get("tags") or []
+        tag = v.get("version", "")
-        if "latest" in tags:
+        created = _parse_created(v)
+        if tag == "latest":
+            keep.append((tag, "always-keep (:latest)"))
             continue
-        if any(t in keep_date_tags for t in tags):
+        if tag.startswith("corpus-"):
+            keep.append((tag, "production pin (corpus-*)"))
             continue
-        try:
+        if other_kept < args.keep_latest:
-            created = datetime.fromisoformat(v["created_at"].replace("Z", "+00:00"))
+            other_kept += 1
-        except (KeyError, ValueError):
+            keep.append((tag, f"keep-latest #{other_kept}/{args.keep_latest}"))
             continue
         if created >= cutoff:
+            keep.append((tag, f"within --keep-days ({args.keep_days})"))
             continue
-        version_id = v.get("id")
+        delete.append(v)
-        print(f"  deleting v{version_id}  tags={tags}  created={v['created_at']}")
+
-        if not args.dry_run:
+    print(f"=== {args.owner}/{args.package}: {len(versions)} total tag(s) ===")
+    for tag, reason in keep:
+        print(f"  KEEP  {tag:<28}  {reason}")
+    for v in delete:
+        print(f"  DEL   {v['version']:<28}  created={v['created_at']}")
+
+    if not delete:
+        print("nothing to delete")
+        return 0
+    if args.dry_run:
+        print(f"--dry-run; would delete {len(delete)} tag(s)")
+        return 0
+
+    failed = 0
+    for v in delete:
+        tag = v["version"]
+        try:
             api(token, "DELETE",
-                f"/api/v1/packages/{args.owner}/container/{args.package}/versions/{version_id}")
+                f"/api/v1/packages/{args.owner}/container/{args.package}/{tag}")
-            deleted += 1
+            print(f"  ✓ deleted {tag}")
-    print(f"done: {deleted} version(s) deleted")
+        except HTTPError as e:
-    return 0
+            print(f"  ✗ failed {tag}: HTTP {e.code} {e.reason}", file=sys.stderr)
+            failed += 1
+
+    print(f"done: deleted {len(delete) - failed} / {len(delete)} tag(s)")
+    return 0 if failed == 0 else 1
 
 
 if __name__ == "__main__":