Leaderboard workflow: open a PR instead of pushing to protected main (#45)
The eval run worked (12 scored runs) but the final step failed: it pushed
evals/results.json directly to main, which the branch ruleset blocks
("Changes must be made through a pull request").
- eval-leaderboard.yml: replace the direct commit/push with
peter-evans/create-pull-request@v7 (branch eval-results), add
pull-requests: write. Merging that PR triggers the Pages deploy (which
watches evals/results.json) to publish real numbers.
- evals/README documents the PR flow + the required "Allow GitHub Actions to
create and approve pull requests" setting.
Claude-Session: https://claude.ai/code/session_016JWn5jRD5tcEFKrubjQ6Px
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
+7
-3
@@ -30,9 +30,13 @@ back to `results.example.json` (clearly labelled) so the page renders before you
|
||||
|
||||
### No local key? Run it in CI
|
||||
|
||||
Add an `ANTHROPIC_API_KEY` repo secret, then go to **Actions → "Update Skill Leaderboard"
|
||||
→ Run workflow**. It runs the evals, commits `evals/results.json`, and the Pages deploy
|
||||
re-renders the public leaderboard with real numbers — no laptop required.
|
||||
1. Add an `ANTHROPIC_API_KEY` repo secret.
|
||||
2. Enable **Settings → Actions → General → Workflow permissions → "Allow GitHub Actions to
|
||||
create and approve pull requests"** (so the workflow can open its results PR — `main`
|
||||
requires PRs).
|
||||
3. **Actions → "Update Skill Leaderboard" → Run workflow.** It runs the evals and opens a
|
||||
PR with `evals/results.json`. **Merge that PR** and the Pages deploy re-renders the
|
||||
public leaderboard with real numbers — no laptop required.
|
||||
|
||||
## Add a case
|
||||
|
||||
|
||||
Reference in New Issue
Block a user