fix(scrape): auto-discover new Morpheus versions; pin 9.0.0 #4

2026-07-15T11:14:07-04:00

claude commented

2026-07-15 11:14:07 -04:00

Problem

A report came in that Morpheus 9.0.0 docs are missing from the MCP. Root cause: scrape/bundles.py had a hardcoded BUNDLES list of 8.1.x docIds with no version discovery. The weekly refresh re-scraped the same fixed docIds forever; --force only re-fetches known pages, never new bundles. 9.0.0 shipped under brand-new HPE docIds that nothing looked for.

Fix (scrape/bundles.py only — downstream is data-driven)

Pin 9.0.0: UM sd00008014en_us (toc, 623 pages), RN sd00008017en_us (single). Confirmed by live portal probe 2026-07-15.
_discover_enterprise(): sweeps the docId range forward each run and appends newer Enterprise UM/RN bundles automatically, so 9.0.1+ land without a code edit. Filters sibling products (VM Essentials, Morpheus Central), matches title+abstract (RN <h1> is just "vX.Y.Z Release Notes"), prefers the lowest docId on HPE mirror-dupes.
Best-effort: per-id errors swallowed; any top-level failure falls back to the pins. Disable via DISCOVER_VERSIONS=0 / --no-discover.

Verification

Against only the 8.1.x pins, discovery surfaces 9.0.0 on its own. Full run: 9 bundles / 2334 pages in ~18s. Next refresh will scrape 9.0.0 into the corpus + reindex + push image.

## Problem A report came in that Morpheus **9.0.0** docs are missing from the MCP. Root cause: `scrape/bundles.py` had a hardcoded `BUNDLES` list of 8.1.x docIds with **no version discovery**. The weekly refresh re-scraped the same fixed docIds forever; `--force` only re-fetches *known* pages, never *new* bundles. 9.0.0 shipped under brand-new HPE docIds that nothing looked for. ## Fix (scrape/bundles.py only — downstream is data-driven) - **Pin 9.0.0**: UM `sd00008014en_us` (toc, 623 pages), RN `sd00008017en_us` (single). Confirmed by live portal probe 2026-07-15. - **`_discover_enterprise()`**: sweeps the docId range forward each run and appends newer Enterprise UM/RN bundles automatically, so 9.0.1+ land without a code edit. Filters sibling products (VM Essentials, Morpheus Central), matches title+abstract (RN `<h1>` is just "vX.Y.Z Release Notes"), prefers the lowest docId on HPE mirror-dupes. - **Best-effort**: per-id errors swallowed; any top-level failure falls back to the pins. Disable via `DISCOVER_VERSIONS=0` / `--no-discover`. ## Verification Against only the 8.1.x pins, discovery surfaces 9.0.0 on its own. Full run: 9 bundles / 2334 pages in ~18s. Next refresh will scrape 9.0.0 into the corpus + reindex + push image.

claude added 1 commit 2026-07-15 11:14:07 -04:00

fix(scrape): auto-discover new Morpheus versions; pin 9.0.0 7849c00507

The BUNDLES catalog was a hardcoded list of 8.1.x docIds with no
version discovery, so the weekly refresh never picked up Morpheus
Enterprise 9.0.0 (published May 2026 under new HPE docIds). --force
only re-fetches known pages; it never finds new bundles.

- Pin 9.0.0 explicitly: User Manual sd00008014en_us (toc, 623 pages)
  and Release Notes sd00008017en_us (single). docIds confirmed by
  probing the HPE DocPortal on 2026-07-15.
- Add _discover_enterprise(): each run sweeps the docId range forward
  from the pinned floor and appends any newer Enterprise User Manuals
  / Release Notes, so future versions (9.0.1, ...) land without a code
  edit. Filters sibling products (VM Essentials, Morpheus Central),
  matches title+abstract (the RN <h1> is only "vX.Y.Z Release Notes"),
  and deterministically prefers the lowest docId on HPE's mirror dupes.
- Best-effort: per-id errors are swallowed and any top-level failure
  falls back to the pinned list. Gate off with DISCOVER_VERSIONS=0 or
  --no-discover.

Verified live: against only the 8.1.x pins, discovery surfaces 9.0.0
on its own; full run yields 9 bundles / 2334 pages in ~18s.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01LFowQzJu7k97QLCRDSAeh1

claude merged commit 2922161cc7 into main

2026-07-15 11:17:27 -04:00

claude deleted branch fix/discover-new-versions

2026-07-15 11:17:27 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: justin/morpheus-docs#4