The "Run evals" step ran 24 API calls sequentially with no request timeout, so
it was slow and could stall indefinitely if one call hung.
- bin/lib/anthropic.mjs: per-request timeout (120s) via AbortController + retry
(2x, backoff) on 429/5xx/timeout. Fails fast on 4xx (bad key/model).
- evals/run-evals.mjs: run (case × model) tasks through a concurrency pool
(default 4, --concurrency to tune); preserves result order.
- eval-leaderboard.yml: job timeout-minutes: 20 as a safety net.
Applies to the next run. The hardening also benefits the Action runner and
`generate`, which share the client.
Claude-Session: https://claude.ai/code/session_016JWn5jRD5tcEFKrubjQ6Px
Co-authored-by: Claude <noreply@anthropic.com>
Three features riding 2026 trends (agentic CI, codegen, evals), sharing one
dependency-free Anthropic client (bin/lib/anthropic.mjs).
1. GitHub Action (action/) — run any skill in a consumer repo's CI:
uses: mohitagw15856/pm-claude-skills/action@main. Composite action +
run.mjs (loads the bundled SKILL.md, calls the API, exposes result as a
step output / file). Docs with auto-PR-description example.
2. generate command — `npx pm-claude-skills generate --from <url|file>` turns
a team's docs into a SKILL.md following the authoring standard
(bin/generate.mjs, wired into the CLI; needs ANTHROPIC_API_KEY).
3. Skill evals + Leaderboard — evals/run-evals.mjs runs each case across models
and scores output with an LLM judge (structure/completeness/usefulness/
grounding); scripts/build-leaderboard.mjs renders web/leaderboard.html
(built in the Pages deploy, falls back to clearly-labelled example data).
Linked from README, catalog, and playground.
Offline-testable parts verified (prompt building, skill loading, graceful
errors, leaderboard render). SkillCheck/audit/exports all green.
Claude-Session: https://claude.ai/code/session_016JWn5jRD5tcEFKrubjQ6Px
Co-authored-by: Claude <noreply@anthropic.com>
mohitagw15856