Generate the course textbook into the wiki from modules/ (CI sync) #72

Merged
claude merged 1 commits from feat/wiki-sync into main 2026-06-22 18:53:12 -04:00
5 changed files with 368 additions and 0 deletions
Showing only changes of commit f0cb2382c8 - Show all commits
+51
View File
@@ -0,0 +1,51 @@
# Render the course (single source of truth = modules/) into the Gitea wiki on
# every push to main. The wiki is generated BUILD OUTPUT — never hand-edit it.
#
# Prerequisites (one-time):
# 1. An Actions runner is attached to this repo/org (Settings -> Actions -> Runners).
# 2. A repo secret WIKI_TOKEN holds a token with write access to the wiki
# (a PAT/deploy token scoped to repository write is enough — NOT a site-admin token).
# 3. The wiki is initialized (it is — created with a Home page).
name: Sync course wiki
on:
push:
branches: [main]
paths:
- 'modules/**'
- 'capstone/**'
- 'README.md'
- 'tools/build_wiki.py'
- '.gitea/workflows/sync-wiki.yml'
workflow_dispatch: {}
jobs:
sync-wiki:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v6
with:
python-version: '3.x'
- name: Render and push the wiki
shell: bash
env:
WIKI_TOKEN: ${{ secrets.WIKI_TOKEN }}
run: |
set -euo pipefail
if [ -z "${WIKI_TOKEN:-}" ]; then
echo "::error::WIKI_TOKEN secret is not set — see this workflow's header."
exit 1
fi
base="git.jpaul.io/justin/ai-workflow-course"
git clone "https://claude:${WIKI_TOKEN}@${base}.wiki.git" wiki
python3 tools/build_wiki.py --repo-root . --out wiki \
--web-base "https://${base}" --branch main --host gitea
cd wiki
git config user.name "claude"
git config user.email "claude@jpaul.io"
git add -A
if git diff --cached --quiet; then
echo "wiki already up to date"; exit 0
fi
git commit -m "docs(wiki): sync from modules/ @ $(echo "$GITHUB_SHA" | cut -c1-8)"
git push origin HEAD
+52
View File
@@ -0,0 +1,52 @@
# Render the course (single source of truth = modules/) into the GitHub wiki on
# every push to main. The wiki is generated BUILD OUTPUT — never hand-edit it.
# This activates on the GitHub mirror; the Gitea copy uses .gitea/workflows/.
#
# Prerequisites (one-time on the mirror):
# 1. The wiki must be INITIALIZED first — create any page once in the GitHub UI,
# otherwise the <repo>.wiki.git remote does not exist and the clone fails.
# 2. A repo secret WIKI_TOKEN holds a PAT with wiki/repo write. The default
# GITHUB_TOKEN CANNOT push to the wiki repo, so a PAT is required.
name: Sync course wiki
on:
push:
branches: [main]
paths:
- 'modules/**'
- 'capstone/**'
- 'README.md'
- 'tools/build_wiki.py'
- '.github/workflows/sync-wiki.yml'
workflow_dispatch: {}
jobs:
sync-wiki:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v6
with:
python-version: '3.x'
- name: Render and push the wiki
shell: bash
env:
WIKI_TOKEN: ${{ secrets.WIKI_TOKEN }}
run: |
set -euo pipefail
if [ -z "${WIKI_TOKEN:-}" ]; then
echo "::error::WIKI_TOKEN secret is not set — see this workflow's header."
exit 1
fi
repo="${GITHUB_REPOSITORY}" # owner/repo
git clone "https://x-access-token:${WIKI_TOKEN}@github.com/${repo}.wiki.git" wiki
python3 tools/build_wiki.py --repo-root . --out wiki \
--web-base "https://github.com/${repo}" --branch main --host github
cd wiki
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add -A
if git diff --cached --quiet; then
echo "wiki already up to date"; exit 0
fi
git commit -m "docs(wiki): sync from modules/ @ $(echo "$GITHUB_SHA" | cut -c1-8)"
git push origin HEAD
+9
View File
@@ -16,6 +16,15 @@ built on a branch and merged through review — exactly the motion the modules t
---
## Read it as a book
The lessons are rendered into the **[Wiki](../../wiki)** as a navigable textbook (unit-by-unit
sidebar, one page per module). The wiki is generated from `modules/` and kept in sync
automatically — it's build output, so read it there but **edit the lessons here in `modules/`**.
See [`tools/`](tools/) for the generator and the sync workflows.
---
## Who this is for
IT professionals who are fluent in an AI chat window and comfortable with ops concepts — **not
+41
View File
@@ -0,0 +1,41 @@
# tools/
## `build_wiki.py` — render the course into a wiki
The course wiki (the "textbook" view under the **Wiki** tab) is **generated build
output**. The single source of truth is `modules/**/README.md` + `capstone/README.md`.
**Never hand-edit the wiki** — edits there are overwritten on the next sync.
`build_wiki.py` is host-agnostic: it writes Markdown pages into a wiki working
directory (`Home.md`, `_Sidebar.md`, `_Footer.md`, one page per module + capstone),
rewriting `lab/…` and repo-root links to absolute URLs back in the main repo (so the
runnable labs stay in the repo and are linked, never copied into the wiki).
### Run it manually
```bash
# clone the wiki repo (Gitea)
git clone https://git.jpaul.io/justin/ai-workflow-course.wiki.git wiki
# render into it
python3 tools/build_wiki.py --repo-root . --out wiki \
--web-base https://git.jpaul.io/justin/ai-workflow-course --branch main --host gitea
# publish
cd wiki && git add -A && git commit -m "docs(wiki): sync from modules/" && git push
```
For GitHub, use `--host github` and `--web-base https://github.com/<owner>/<repo>`.
### Automated sync (CI)
- **Gitea:** `.gitea/workflows/sync-wiki.yml` runs on every push to `main` that
touches `modules/**`, `capstone/**`, `README.md`, or this generator.
- **GitHub:** `.github/workflows/sync-wiki.yml` does the same on the mirror.
Both need, one time:
1. an **Actions runner** attached (Gitea: Settings → Actions → Runners);
2. a repo secret **`WIKI_TOKEN`** with wiki write (a scoped PAT/deploy token — *not*
a site-admin token; on GitHub the default `GITHUB_TOKEN` can't push the wiki);
3. the wiki **initialized** once (Gitea is already initialized; on GitHub create one
page in the UI first so `<repo>.wiki.git` exists).
+215
View File
@@ -0,0 +1,215 @@
#!/usr/bin/env python3
"""Render the course into wiki pages from the single source of truth (modules/).
Host-agnostic: this writes Markdown into a target wiki working directory. A thin
per-host CI wrapper clones the host's `<repo>.wiki.git`, runs this script, then
commits and pushes. The same script feeds both the Gitea and GitHub wikis — only
the `--host` URL flavor and the CI wrapper differ.
The wiki is GENERATED BUILD OUTPUT. Never hand-edit it; edit modules/ and let CI
re-render. Labs are NOT copied into the wiki — lesson pages link to the runnable
files back in the main repo.
Usage:
build_wiki.py --repo-root . --out <wiki-dir> \
--web-base https://git.jpaul.io/justin/ai-workflow-course \
--branch main --host gitea
"""
from __future__ import annotations
import argparse
import re
from pathlib import Path
# Unit structure (module numbers), from the syllabus. Drives the textbook nav.
UNITS = [
("Unit 1 — Get out of the chat window", range(1, 8)),
("Unit 2 — Make it shareable, reviewable, recoverable", range(8, 13)),
("Unit 3 — Automate the checking and shipping", range(13, 20)),
("Unit 4 — Extend the AI into your systems", range(20, 24)),
("Unit 5 — AI in the Loop", range(24, 28)),
]
def src_url(web_base: str, branch: str, host: str, path: str) -> str:
"""Browser URL to a file in the main repo (host-specific path scheme)."""
path = path.rstrip("/")
if host == "github":
return f"{web_base}/blob/{branch}/{path}"
return f"{web_base}/src/branch/{branch}/{path}" # gitea / forgejo
def first_h1(text: str, fallback: str) -> str:
for line in text.splitlines():
if line.startswith("# "):
return line[2:].strip()
return fallback
def short_title(h1: str) -> str:
"""'Module 1 — The Copy-Paste Problem' -> 'The Copy-Paste Problem'."""
m = re.match(r"^Module\s+\d+\s*[—:-]\s*(.+)$", h1)
return m.group(1).strip() if m else h1
def rewrite_lab_links(body: str, web_base: str, branch: str, host: str, slug: str) -> str:
"""Make ](lab/...) point at the runnable file in the main repo, not the wiki."""
def repl(m: re.Match) -> str:
rel = m.group(1) # e.g. 'lab/Dockerfile' or 'lab/'
return f"]({src_url(web_base, branch, host, f'modules/{slug}/{rel}')})"
return re.sub(r"\]\((lab/[^)]*)\)", repl, body)
# Repo root files that may be linked from prose; in the wiki these must point back
# at the main repo, not at a (nonexistent) wiki page of the same name.
ROOT_FILES = ("AGENTS.md", "README.md", "the-workflow-syllabus.md", "handoff.md", "_TEMPLATE.md")
def rewrite_repo_file_links(body: str, web_base: str, branch: str, host: str) -> str:
pattern = r"\]\((" + "|".join(re.escape(f) for f in ROOT_FILES) + r")\)"
return re.sub(
pattern,
lambda m: f"]({src_url(web_base, branch, host, m.group(1))})",
body,
)
def banner(source_path: str, source_url: str) -> str:
return (
f"> 📖 _This page is generated from [`{source_path}`]({source_url}). "
f"**Edit the source, not the wiki** — edits here are overwritten on the next sync. "
f"Run the hands-on labs from the repo, linked inline._\n"
)
def discover_modules(repo_root: Path):
"""Return [(number:int, slug:str, path:Path)] sorted by number."""
mods = []
for d in sorted((repo_root / "modules").iterdir()):
if not d.is_dir():
continue
m = re.match(r"^(\d+)-", d.name)
if m and (d / "README.md").exists():
mods.append((int(m.group(1)), d.name, d))
return sorted(mods, key=lambda t: t[0])
def home_intro(repo_root: Path) -> str:
"""Pull title/tagline/thesis from the repo README (up to its first '## ')."""
readme = (repo_root / "README.md").read_text(encoding="utf-8")
out = []
for line in readme.splitlines():
if line.startswith("## "):
break
out.append(line)
return "\n".join(out).strip()
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--repo-root", default=".")
ap.add_argument("--out", required=True, help="wiki working directory to write into")
ap.add_argument("--web-base", required=True, help="main repo web base URL")
ap.add_argument("--branch", default="main")
ap.add_argument("--host", choices=["gitea", "github"], default="gitea")
args = ap.parse_args()
repo_root = Path(args.repo_root).resolve()
out = Path(args.out).resolve()
out.mkdir(parents=True, exist_ok=True)
mods = discover_modules(repo_root)
if not mods:
print("no modules found", flush=True)
return 1
titles: dict[int, str] = {}
slugs: dict[int, str] = {}
# --- per-module pages -------------------------------------------------
for num, slug, path in mods:
body = (path / "README.md").read_text(encoding="utf-8")
h1 = first_h1(body, f"Module {num}")
titles[num] = h1
slugs[num] = slug
source_path = f"modules/{slug}/README.md"
body = rewrite_lab_links(body, args.web_base, args.branch, args.host, slug)
body = rewrite_repo_file_links(body, args.web_base, args.branch, args.host)
page = (
banner(source_path, src_url(args.web_base, args.branch, args.host, source_path))
+ "\n"
+ body
+ "\n"
)
(out / f"{slug}.md").write_text(page, encoding="utf-8")
# --- capstone ---------------------------------------------------------
cap = repo_root / "capstone" / "README.md"
cap_title = None
if cap.exists():
body = cap.read_text(encoding="utf-8")
cap_title = first_h1(body, "Capstone — The Full Loop")
source_path = "capstone/README.md"
# capstone labs may reference modules/.. ; only lab/ links are rewritten elsewhere,
# capstone has none, so copy through with a banner.
body = rewrite_repo_file_links(body, args.web_base, args.branch, args.host)
page = (
banner(source_path, src_url(args.web_base, args.branch, args.host, source_path))
+ "\n"
+ body
+ "\n"
)
(out / "capstone.md").write_text(page, encoding="utf-8")
# --- _Sidebar.md (textbook nav) --------------------------------------
sb = ["### [📖 Home](Home)\n"]
for unit_title, rng in UNITS:
present = [n for n in rng if n in slugs]
if not present:
continue
sb.append(f"**{unit_title}**\n")
for n in present:
sb.append(f"- [{n} · {short_title(titles[n])}]({slugs[n]})")
sb.append("")
if cap_title:
sb.append("**Finale**\n")
sb.append(f"- [{short_title(cap_title)}](capstone)")
sb.append("")
(out / "_Sidebar.md").write_text("\n".join(sb) + "\n", encoding="utf-8")
# --- Home.md (landing + TOC) -----------------------------------------
intro = rewrite_repo_file_links(home_intro(repo_root), args.web_base, args.branch, args.host)
home = [intro, "\n## Contents\n"]
for unit_title, rng in UNITS:
present = [n for n in rng if n in slugs]
if not present:
continue
home.append(f"### {unit_title}\n")
for n in present:
home.append(f"- **[{titles[n]}]({slugs[n]})**")
home.append("")
if cap_title:
home.append("### Finale\n")
home.append(f"- **[{cap_title}](capstone)**")
home.append("")
home.append(
"\n---\n> 📖 _This wiki is generated from the "
f"[course repo]({args.web_base}) — edit `modules/` there, not these pages._"
)
(out / "Home.md").write_text("\n".join(home) + "\n", encoding="utf-8")
# --- _Footer.md -------------------------------------------------------
(out / "_Footer.md").write_text(
f"_Generated from the [ai-workflow-course repo]({args.web_base}) • "
"the model is the cheap, swappable part; the workflow is the durable skill._\n",
encoding="utf-8",
)
pages = len(mods) + (1 if cap_title else 0) + 3
print(f"wrote {pages} wiki files to {out} ({len(mods)} modules"
f"{' + capstone' if cap_title else ''} + Home/_Sidebar/_Footer)", flush=True)
return 0
if __name__ == "__main__":
raise SystemExit(main())