De-slop: remove every em-dash + banned words across all modules + capstone (#94)
Sync course wiki / sync-wiki (push) Successful in 4s
Sync course wiki / sync-wiki (push) Successful in 4s
Co-authored-by: claude <claude@jpaul.io> Co-committed-by: claude <claude@jpaul.io>
This commit was merged in pull request #94.
This commit is contained in:
@@ -34,7 +34,7 @@ def judge(candidate_text: str) -> dict:
|
||||
key = os.environ.get("EVAL_JUDGE_KEY")
|
||||
model = os.environ.get("EVAL_JUDGE_MODEL")
|
||||
if not (url and key and model):
|
||||
return {"score": None, "reason": "judge not configured — abstaining (set EVAL_JUDGE_* to enable)"}
|
||||
return {"score": None, "reason": "judge not configured; abstaining (set EVAL_JUDGE_* to enable)"}
|
||||
|
||||
payload = json.dumps({
|
||||
"model": model,
|
||||
@@ -72,7 +72,7 @@ if __name__ == "__main__":
|
||||
# about the candidate changed. The ruler is itself made of rubber.
|
||||
#
|
||||
# So: use a programmatic grader (run_eval.py) wherever a deterministic check is
|
||||
# possible — that is most of the time. Reach for an LLM judge only for genuinely
|
||||
# possible; that is most of the time. Reach for an LLM judge only for genuinely
|
||||
# open-ended output, and CALIBRATE it first: hand-label ~20 examples yourself,
|
||||
# run the judge on them, and confirm it agrees with you before you let it gate
|
||||
# anything. An uncalibrated judge is a vibe with a number attached.
|
||||
|
||||
Reference in New Issue
Block a user