justin/ai-workflow-course
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Use python3 as the canonical command name course-wide (#104) (#105)
CI / check (push) Successful in 7s
Details
Sync course wiki / sync-wiki (push) Successful in 4s
Details

Browse Source
This commit was merged in pull request #105.
This commit is contained in:
Claude (agent)
2026-06-23 20:25:05 -04:00
parent 7f439212ac
commit 95e5911957
102 changed files with 380 additions and 378 deletions
Download Patch File Download Diff File Expand all files Collapse all files
blog/16-unit5-ai-in-the-loop.md
+5 -5
View File
@@ -40,9 +40,9 @@ The lab makes this concrete and local: no hosted bot account required. You run a
```bash
cd modules/24-assistive-agents/lab
python reviewer.py prompt # builds: your committed rubric + the diff
python3 reviewer.py prompt # builds: your committed rubric + the diff
# (paste into your AI, save its JSON to my-review.json)
python reviewer.py apply my-review.json
python3 reviewer.py apply my-review.json
```
The diff it's reviewing has a real trap planted in it: a new `clear` command that prints "cleared all tasks" but never actually calls `save()`, so `tasks.json` is untouched. Did your AI catch it? Either way, *you* make the merge call, and you learn exactly how much this reviewer is worth before the stakes go up.
@@ -70,7 +70,7 @@ The lab runs the whole thing locally against the `tasks-app`, and the best part
```bash
git checkout -b agent/delete-command
python agent_runner.py issue-to-pr issue-delete-command.md --simulate bad
python3 agent_runner.py issue-to-pr issue-delete-command.md --simulate bad
# → ruff + pytest run, a test fails, the script refuses to call the work ready.
# Exit code non-zero. No PR. Nothing reached main.
```
@@ -124,8 +124,8 @@ The lab is the punchline of the whole series. You run the same eval set against
```bash
cd modules/27-evals/lab
python run_eval.py candidates/current_model # 100%, exit 0, your baseline
python run_eval.py candidates/swapped_model # 60%, exit 1, blocked
python3 run_eval.py candidates/current_model # 100%, exit 0, your baseline
python3 run_eval.py candidates/swapped_model # 60%, exit 1, blocked
```
The "swapped model" is a stand-in for the day a cheaper model ships, or your provider deprecates the one you're on, or someone edits the agent's prompt. The easy cases still pass (this output would sail through a casual skim), but the eval caught a regression a skim would have missed, *and the non-zero exit code means a pipeline would have blocked the merge.* That's a **regression eval**, and it's the moment this course's thesis stops being a slogan and becomes a procedure you run from the keyboard.