Commit Graph
3 Commits
Author SHA1 Message Date
claudeandClaude Opus 4.8 cd30d0dc7a feat(scrape): auto-discover HVM versions + add 9.0 to corpus
HVM (HPE Morpheus VM Essentials) 9.0.0 shipped June 2026; the MCP had
nothing about it. Add it and stop hand-adding a docId per release.

scrape/bundles.py now AUTO-DISCOVERS new versions (discover_specs, on by
default): it scans the DocPortal docId range above the highest known
bundle and classifies each abstract. On-by-default in CI's refresh.yml,
so future releases flow into the corpus with no code change. Gotchas
encoded:
  - classify on the abstract BODY, not the <h1> (a Release Notes h1 is
    just "v9.0.0 Release Notes" with no product name)
  - EXCLUDE the interleaved siblings that share the same version string:
    Morpheus Enterprise, Morpheus Central, Storage Integration Pack
  - failsoft: a scan error logs a warning and falls back to known bundles
  - self-healing + idempotent: prior bundles.json persists discoveries;
    the scan anchor rolls forward; re-runs add no duplicates

Discovered + scraped for 9.0.0:
  - User Manual   sd00008058en_us (414 pages; Deployment Guide now folded in)
  - Release Notes sd00008079en_us

corpus: +412 pages (9.0), sidecar topic_cluster peers re-linked to 9.0
across the 8.1.x bundles (shared GUIDs). Local reindex: 3645 chunks
(956 are 9.0.0); 9.0 content verified retrievable end-to-end.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MNQRS6F7N9Uwh1SMn3xz2g
2026-07-15 11:40:14 -04:00
justin 6e938c05c4 feat: Qualification Matrix + QuickSpecs bundles (#5) 2026-05-22 15:05:13 -04:00
justinandClaude Opus 4.7 7a491ba9e4 scrape: HVM bundles + runner for HPE Support DocPortal
Phase 1: scrape User Manual (8.1.0/.1/.2), Release Notes (8.1.0/.1/.2),
and the unversioned Deployment Guide. Total ~1,160 pages, 9.7 MB markdown.

Discovers via the anonymous JSON API at /hpesc/public/api/document/{docId}:
/toc walks the page tree (for TOC-paginated docs), /render?page=GUID
fetches per-page HTML, /document/{docId} returns the whole body for
single-doc shapes like Release Notes.

Runner converts DITA-source HTML to clean markdown (strips Notices/
Acknowledgments/Abstract boilerplate), writes corpus/<bundle>/<page>.{md,json},
then a finalize pass synthesizes topic_cluster.clustered_topics by GUID
overlap across versions (HPE GUIDs are stable cross-version — confirmed
374/376/376 with 100% overlap on shared pages).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-05-22 13:06:26 -04:00