fix(scrape): make vision OCR a per-bundle allowlist, not a corpus sweep
The first vision pass OCR'd every large image and, because the "transcribe the table" prompt makes qwen render ANY image as a markdown table, it turned 60 product UI screenshots into junk "tables" in the corpus. In practice the only image-only DATA in this corpus is the release-schedule matrix — the upgrade-compatibility matrices etc. are already native HTML tables, no OCR needed. - Add BundleSpec.ocr (default False); the runner passes the OCR-enabling session only for bundles that opt in. Set ocr=True only on morpheus_release_schedule. - Purge the 60 screenshot cache entries (keep the release-schedule one). The next --force refresh regenerates every other page's .md WITHOUT the bogus "## Image transcriptions" section, cleaning the deployed corpus + index. - Update vision.py / refresh.yml docs to the allowlist model. Also add scripts/vision_report.py — a self-contained HTML report pairing each OCR'd source image with its transcription (uncertain-first) for accuracy inspection. Regenerate anytime from corpus/.vision-cache/. Verified: release schedule still transcribes (cache hit); cache is back to 1 entry; other bundles now pass no session so their images are left alone. Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_01LFowQzJu7k97QLCRDSAeh1
This commit is contained in:
+20
-10
@@ -14,7 +14,8 @@
|
||||
"Published": "February 2026"
|
||||
},
|
||||
"landing_page": "GUID-709AAADB-A9C1-40B6-AD22-958EE7E6F312",
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007510en_us"
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007510en_us",
|
||||
"ocr": false
|
||||
},
|
||||
{
|
||||
"slug": "morpheus_user_manual_8_1_1",
|
||||
@@ -31,7 +32,8 @@
|
||||
"Published": "March 2026"
|
||||
},
|
||||
"landing_page": "GUID-709AAADB-A9C1-40B6-AD22-958EE7E6F312",
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007621en_us"
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007621en_us",
|
||||
"ocr": false
|
||||
},
|
||||
{
|
||||
"slug": "morpheus_user_manual_8_1_2",
|
||||
@@ -48,7 +50,8 @@
|
||||
"Published": "April 2026"
|
||||
},
|
||||
"landing_page": "GUID-709AAADB-A9C1-40B6-AD22-958EE7E6F312",
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007732en_us"
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007732en_us",
|
||||
"ocr": false
|
||||
},
|
||||
{
|
||||
"slug": "morpheus_release_notes_8_1_0",
|
||||
@@ -65,7 +68,8 @@
|
||||
"Published": "February 2026"
|
||||
},
|
||||
"landing_page": "sd00007496en_us",
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007496en_us"
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007496en_us",
|
||||
"ocr": false
|
||||
},
|
||||
{
|
||||
"slug": "morpheus_release_notes_8_1_1",
|
||||
@@ -82,7 +86,8 @@
|
||||
"Published": "March 2026"
|
||||
},
|
||||
"landing_page": "sd00007610en_us",
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007610en_us"
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007610en_us",
|
||||
"ocr": false
|
||||
},
|
||||
{
|
||||
"slug": "morpheus_release_notes_8_1_2",
|
||||
@@ -99,7 +104,8 @@
|
||||
"Published": "April 2026"
|
||||
},
|
||||
"landing_page": "sd00007733en_us",
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007733en_us"
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00007733en_us",
|
||||
"ocr": false
|
||||
},
|
||||
{
|
||||
"slug": "morpheus_user_manual_9_0_0",
|
||||
@@ -116,7 +122,8 @@
|
||||
"Published": "May 2026"
|
||||
},
|
||||
"landing_page": "GUID-709AAADB-A9C1-40B6-AD22-958EE7E6F312",
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00008014en_us"
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00008014en_us",
|
||||
"ocr": false
|
||||
},
|
||||
{
|
||||
"slug": "morpheus_release_notes_9_0_0",
|
||||
@@ -133,7 +140,8 @@
|
||||
"Published": "May 2026"
|
||||
},
|
||||
"landing_page": "sd00008017en_us",
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00008017en_us"
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sd00008017en_us",
|
||||
"ocr": false
|
||||
},
|
||||
{
|
||||
"slug": "morpheus_quickspecs",
|
||||
@@ -148,7 +156,8 @@
|
||||
"abstract": "",
|
||||
"dates": {},
|
||||
"landing_page": "a50009231enw",
|
||||
"source_url": "https://www.hpe.com/psnow/doc/a50009231enw"
|
||||
"source_url": "https://www.hpe.com/psnow/doc/a50009231enw",
|
||||
"ocr": false
|
||||
},
|
||||
{
|
||||
"slug": "morpheus_release_schedule",
|
||||
@@ -165,6 +174,7 @@
|
||||
"Published": ""
|
||||
},
|
||||
"landing_page": "sf000111242en_us",
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sf000111242en_us"
|
||||
"source_url": "https://support.hpe.com/hpesc/public/docDisplay?docId=sf000111242en_us",
|
||||
"ocr": true
|
||||
}
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user