Provenance had no system-level owner: ownership was only per-tree
(TreeMembership), so a self-hosted instance had no operator account and no
instance-admin surface. This adds one, declared by environment per the project's
twelve-factor rule.
- OWNER_EMAIL (comma-separated): the account(s) named here are instance owners.
Derived at request time — no DB column, no migration, can't drift from the env,
survives DB resets. is_instance_owner()/InstanceOwner dependency in api/deps.py.
- Ownership requires a VERIFIED email (independent of REQUIRE_EMAIL_VERIFICATION).
Registration is open, so without this an attacker could seize the role by
registering the owner address first; verification ties it to inbox control.
- GET /api/v1/admin/instance (owner-only): operational status — version, env,
user/tree counts, configured AI providers. Deliberately exposes no tree data
or PII: instance ownership is an operator role, NOT a privacy-engine bypass.
- /users/me reports is_instance_owner; frontend gains an owner-only /admin page
and a conditional sidebar link (server-enforced, not just client-hidden).
Found-and-fixed by an adversarial security review before merge: the
verified-email land-grab (above) and a frontend null-deref where the admin page
crashed on 401/5xx instead of failing closed.
Docs: .env.example + ARCHITECTURE (notes the not-a-privacy-bypass boundary and
the verified-email requirement). Tests: owner matching, the land-grab guard,
/users/me, and owner-only /admin. Suite 96 passing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Defense-in-depth for the deploy pipeline. Today a backend image shipped ahead
of an un-applied migration; the Tree model selected columns the DB didn't have
yet, so every trees query 500'd with an opaque UndefinedColumnError and the UI
showed no trees. The root cause (deploys not running migrations) is fixed
separately; this makes the *symptom* impossible to miss.
- app/core/schema_version.py: compare the DB's stamped alembic head to the
head(s) baked into the image's migration scripts. A DB with no alembic_version
table (e.g. a create_all test DB) is treated as current, so this stays quiet
outside real deployments. Uses to_regclass so a missing table never poisons
the caller's transaction.
- /health/ready: returns 503 with an explicit "drift: db=… expected=…" message
when the schema is behind, instead of reporting ready and serving 500s.
- Startup lifespan: logs CRITICAL on drift (advisory — never blocks startup).
Liveness (/health) is untouched, so a drifted container isn't killed into a
crash-loop — it's loudly degraded and self-heals once migrations apply.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
The operator decides which model providers exist (env / registry — Anthropic,
OpenAI, x.AI, Ollama, several at once). The *tree owner* decides who uses which:
- Members' assistant -> one configured provider (or none)
- Recommender (association/connection finder) -> one configured provider (or none)
- Owner -> may use any configured provider
Backend: two nullable columns on `trees` (ai_member_provider,
ai_recommender_provider) + migration; `configured_llm_providers()` exposes the
registry as {name, model} with no secrets; owner-gated GET/PATCH
/trees/{id}/ai validate names against the configured set. Frontend: owner-only
"AI models" page with a dropdown per role, graceful 403 for non-owners, and a
sidebar link.
Per-model-within-a-provider selection is a follow-up; today each provider maps
to its single configured model.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Extends the #215 abstraction:
- OpenAICompatibleLLMProvider / OpenAICompatibleEmbeddingProvider — one impl (via
the official openai SDK) covers OpenAI, xAI (api.x.ai/v1), Ollama
(…:11434/v1), OpenRouter, etc.; they differ only by base_url, key, and model.
- Registry factory: build_llm_providers() / build_embedding_providers() return
every provider whose credentials are configured, so you can run several
concurrently. get_llm_provider(name)/get_embedding_provider(name) select by
name, falling back to default_*_provider, then Null.
- Per-provider env config (ANTHROPIC_*, OPENAI_*, XAI_*, OLLAMA_*) +
DEFAULT_LLM_PROVIDER / DEFAULT_EMBEDDING_PROVIDER; documented in .env.example.
Defaults keep AI off (empty registry).
Embeddings now have real backends (OpenAI/Ollama), still separate from the LLM
since Anthropic offers no embeddings endpoint. Tests cover multi-provider
selection, default resolution, disabled-without-credentials, and null fail-loud.
Full suite 87 passed.
Relates to #215.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Implements non-negotiable #1: the AI assistant never writes autonomously. Every
assistant/contributor "write" emits a ChangeProposal — a structured diff a human
approves, edits, or rejects. Design: docs/design/change-proposal.md.
Structural guarantee: a proposal's operations reach the DB ONLY via
change_proposal_service.apply(), which requires the actor be an editor and
dispatches each op through the normal editing services (person/name/event/
relationship/source/citation create/update/delete) — so every change passes the
privacy engine and is audited as the approving human. propose() only inserts a
pending row; it performs no domain mutation. Model providers stay read-only, so
no model response can mutate tree data.
- ChangeProposal model + migration (status pending|applied|rejected, origin
assistant|contributor, JSONB operations, reviewer + apply_error).
- Service: propose / list / get / apply (with optional edited ops) / reject /
delete; a dispatcher mapping ops → editing services. v1 applies ops in order,
not cross-op transactional (single-op is atomic; documented).
- API /trees/{id}/proposals + a frontend review page (approve/reject; editor-
gated) and sidebar entry.
Tests: proposal doesn't apply until approved; reject doesn't apply; non-editor
member can see but not apply; multi-op; approve-with-edits; apply-error keeps it
pending. Full suite 87 passed; single alembic head.
Closes#214
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Adds the vendor-agnostic seam the AI assistant + match-ranking plug into:
- LLMProvider / EmbeddingProvider ABCs (base.py). LLM and embeddings are
SEPARATE abstractions — Anthropic has no embeddings endpoint, so each is
configured independently and either can be off.
- NullLLMProvider / NullEmbeddingProvider — the default; fail loud with a clear
"not configured" error so AI-off deployments don't silently no-op.
- AnthropicLLMProvider — first concrete LLM impl, via the official anthropic SDK
(default model claude-opus-4-8). A local provider (e.g. Ollama) would be
another subclass of the same interface.
- Factory in deps.py (get_llm_provider / get_embedding_provider) selects by
env (MODEL_PROVIDER / EMBEDDING_PROVIDER); documented in .env.example.
Providers are read-only text/vector producers — they never touch the DB, so the
"AI never writes autonomously" invariant (CLAUDE.md #1) holds; writes will go
through ChangeProposal (#214).
Tests: provider selection (null default, anthropic when keyed, fallback without
key) + null providers raise. 81 passed.
Closes#215
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Move backup from a documented procedure to `deploy/backup.sh`: dumps Postgres
(pg_dump --clean --if-exists, gzipped) and archives the MinIO /data directory
into a single timestamped bundle under backups/. Reads config from the compose
.env with the same defaults the stack uses; optional BACKUP_RETAIN_DAYS prunes
old bundles (cron-friendly). BACKUP.md documents usage + the restore procedure
(kept manual/documented rather than an untested destructive script).
Closes#196
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
TreeMembership was enforced on every read/write but had no API or UI to manage
members — trees were effectively single-user, breaking full-CRUD (NN#8).
Backend (/trees/{id}/members): list (members only — the list exposes emails, so
non-members never see it, even on public trees); add an existing user by email
(owner only, 404 if no such account, 409 if already a member); PATCH role;
DELETE. A tree must always keep ≥1 owner (demote/remove of the sole owner → 409).
All changes audited.
Frontend: a Members page (owner gets add-by-email + per-member role select +
remove; others see a read-only list) and a sidebar entry.
Test covers the full lifecycle + every guard. Suite 77 passed.
Closes#145
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Export emitted SOUR records but never the per-fact SOUR links, so a
Provenance→Provenance round-trip destroyed the sources graph (citations were
dropped). Emit citation links on the facts they sit on:
- person-level → 1 SOUR @Sx@ (2 PAGE)
- name-level → 2 SOUR under 1 NAME
- event-level → 2 SOUR under the event (incl. partnership events in FAM)
- relationship → 1 SOUR under FAM
Citations whose source didn't export are skipped.
Test: a person + event citation round-trips through export→import into a fresh
tree with their pages intact. GEDCOM suite 6 passed.
Closes#169
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Backlog §2.10: registration issued a live session and email_verified_at was
written but never read, so an unverified user had full access and there was no
switch to require verification.
Add REQUIRE_EMAIL_VERIFICATION (default false). When true:
- resolve_session_user returns None for a user whose email_verified_at is null —
the single read-side gate covering every authenticated request, incl. the
session minted at registration.
- login raises 403 ("email not verified") instead of issuing a useless token.
Default false on purpose: self-hosts without SMTP, and accounts created before
this gate existed (email_verified_at null), must not be locked out. Operators
enable it once mail works and accounts are verified. Documented in .env.example.
Tests: default-off keeps unverified accounts working; on → register's session
won't resolve (401), login is 403, and after verify-email both work. 75 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Unset sex renders blue (male-colored), which is misleading next to a confirmed
male partner. Add a Cleanup action that proposes the opposite sex for an unset
partner of someone whose sex is set (couples are opposite-sex in practice — a
confirmed-male husband ⇒ a female wife). People whose known partners disagree
are skipped as ambiguous.
It's a preview the user reviews and approves in the Cleanup tool (reusing the
existing gender apply path + audit) — not an autonomous write. Backend:
guess_gender_by_spouse + GET /cleanup/gender/from-spouse. Frontend: an "Infer
from spouse" button feeding the existing proposal list. Test covers
propose-opposite, skip-no-partner, skip-already-set, apply, and re-preview.
Full suite 73 passed; frontend build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
The public tree chart was fixed at 3 ancestors / 2 descendants. Add the same
Generations controls the member view has (slider + number stepper + "All" per
direction), applied live around the focused person.
Extracts the member page's inline DepthControl into a shared
components/depth-control.tsx and uses it in both, so they stay in sync. The
public chart gains anc/prog depth state + an apply effect (setAncestryDepth/
setProgenyDepth + updateTree) mirroring the member behavior.
tsc clean; next build passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
The public layout forced max-w-5xl on every /p page, so the tree chart was
cramped. Mirror the member shell: the public layout now drops the max-width for
the tree page (/p/<id>) only, giving the chart the full canvas (74vh to match
the member view), while the page keeps its heading and people list in a
centered max-w-5xl column. Person detail (/p/<id>/persons/<pid>) and /explore
stay narrow.
tsc clean; next build passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Two gaps from review of the public surface:
- The public tree page showed only a list of names. Add the family-chart
hourglass (PublicTreeChart) above the directory — the same renderer the
member tree view uses, including the cycle-sanitisation that guards against a
bad graph, fed by redacted public data. Click a card to recenter; "Open"
links to the person's public page. Centers on the tree's home person.
- The homepage had no path to /explore. Add an "Explore" nav link and an
"Explore public trees" hero button.
tsc clean; next build passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
A no-login directory of shared trees, backed by GET /api/v1/public/trees:
- /explore: searchable grid of public trees; debounced name search. Because the
backend adds `site_members` trees when a valid session is present, signed-in
users see more with no client-side branching.
- PublicHeader extracted and shared by /p and /explore (logo, Explore, Sign in).
- "Explore" entry added to the authed sidebar.
tsc clean; next build passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Adds the public viewing surface in the UI — shareable, no-login pages backed by
the redaction-safe /api/v1/public API:
- /p/[treeId]: tree name + searchable people directory (living people show as
"Living person"; counts; links to person pages).
- /p/[treeId]/persons/[personId]: person detail — events, alternate names, and
parents/partners/children as links to other public person pages.
- app/p/layout.tsx: slim public header (logo + Sign in), no app sidebar.
- robots.ts: allow /p/, disallow the authenticated app sections.
- Trees list: a "Public page ↗" link on every non-private tree so the owner can
grab the shareable URL.
Client-rendered (same-origin fetch via Caddy). Follow-up (needs a frontend
SSR→backend base URL + a compose/env deploy step, so not auto-applied by
Watchtower): true server-rendering for SEO, a dynamic sitemap of public trees,
and per-page noindex for unlisted/site_members.
tsc clean; next build passes (both routes dynamic).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
A logged-in NON-member of a public/unlisted tree could read living people's
dates, real alternate names, and media (incl. downloading photos) through the
family-view endpoints — only the person LIST was redacted; list_events,
list_relationships, list_names, list_media gated on can_view_tree alone.
For non-members, these now delegate to the same visibility-filtered reads the
public surface uses (person_visibility-driven): living-person events/names
dropped, relationships touching a hidden person dropped, media limited to
full-visibility persons, and media download (get_media → media_content) 404s
for a redacted/unlinked person's media. Members are unchanged.
Adds list_public_relationships_for_person / list_public_media / can_view_media
to public_view_service. Test: an authed non-member sees no living-person PII
across events/names/relationships/media and can't download a living person's
file, while the owner still sees everything. Full suite: 72 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Output of a multi-agent gap analysis comparing Provenance against commercial
(Ancestry/MyHeritage/FamilySearch) and open-source (GRAMPS/Gramps Web/webtrees)
genealogy software: 15 research lenses, 580 raw features deduped into a 17-
category taxonomy, 302 features assessed against the codebase (have/partial/
planned/missing) with statuses verified against the actual code.
Includes an executive summary, per-category backlog with status/importance/
effort/phase, a quick-wins shortlist, and strategic differentiators. Statuses
reflect the repo at analysis time (before tree-visibility phases 1-3); a couple
of flagged items (e.g. the site_members tier) are already closed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Adds the anonymous read surface (/api/v1/public) — the privacy-critical core.
- CurrentUserOrNone dependency: optional auth that never 401s (anonymous OK).
- public_view_service: every projection passes through privacy.person_visibility.
persons redacted (living → "Living person", hidden dropped); relationships
only when both endpoints non-hidden; events only for FULL-visibility persons
(partnership events only when both partners full); names only for FULL
persons. Not-viewable trees raise 404 (not 403) so the surface can't probe
for private trees. Media deferred (higher-sensitivity; own pass later).
- public router: read-only directory + tree + persons/relationships/events +
person detail/names/events. Directory lists `public` to all and adds
`site_members` for authenticated callers; never lists unlisted/private.
- PublicTreeRead omits owner_id.
Tests (ran locally — CI does not run pytest): an anonymous end-to-end leak test
asserting a living person's real name, alias, and birth year appear in NO public
response while the deceased person's data does; plus private=404, unlisted
viewable-by-link-but-unlisted, site_members requires login, and directory
visibility. Full suite: 70 passed. Regenerated openapi.json + TS client.
Note: the AUTHED list endpoints still leak per-person for non-members
(pre-existing) — fixed next, separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
can_view_tree() now distinguishes anonymous vs authenticated non-members so the
four-level model is enforceable:
- public / unlisted → anyone, including anonymous (unlisted gated only by the
link, so the API must never *list* it)
- site_members → any authenticated account (denies anonymous)
- private → members only
Members (any role) always view; soft-deleted trees stay hidden from everyone.
person_visibility (living-person redaction) is unchanged.
Tests: a full can_view_tree matrix across {anonymous, logged-in non-member,
member} × {public, unlisted, site_members, private}, plus deleted-tree-hidden
and the site_members anon-vs-logged-in case. Adds `engine`/`db_session` fixtures
(refactored out of `client`) so the engine can be unit-tested directly,
including the anonymous path that has no HTTP endpoint yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
First step of the public-viewing feature (design: docs/design/tree-visibility.md).
No non-member behavior change yet — this only widens the vocabulary and UI.
- TreeVisibility gains `site_members` (any authenticated user of the instance),
giving the four-level model: public / site_members / unlisted / private.
- Alembic migration adds the enum value via an autocommit block (ALTER TYPE
ADD VALUE can't run in a transaction on older Postgres); downgrade is a no-op
since PG can't drop an enum value.
- Regenerated openapi.json + frontend TS client.
- Trees-list dropdown now offers Private / Public – Members / Unlisted / Public
with an explanatory tooltip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Tree visibility was set to private with no UI to change it — the trees list
only displayed the value as text. Add a private/unlisted/public dropdown on
each tree card that PATCHes visibility immediately (optimistic), pulled out of
the card's navigation Link so it doesn't trigger a page change. Honors the
"everything configurable / full CRUD in the UI" invariants. Living people stay
protected by the privacy engine regardless of tree visibility.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Depth was hardcoded (3 ancestors, 2 descendants). Add a controls row to set
each direction independently — a slider plus a number stepper, with an "All"
toggle per direction — applied around whoever is currently focused.
- ancestor/descendant depth held in state; effective value is a large cap
when "All" is on (the chart only renders people that exist, so the cap is
free).
- changes apply to the live chart via setAncestryDepth/setProgenyDepth +
updateTree without a full rebuild.
- fan mode (ancestors only) takes the ancestor depth via its `generations`
prop, capped at 8 to avoid the radial layout's 2^n blow-up; its descendants
control is disabled with a note.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Setting a person's sex meant clicking Edit, opening a dropdown, and saving.
Replace the read-only ♂/♀ symbol next to the name with an always-visible
two-button segmented control that PATCHes immediately on click (gender-only;
backend PATCH is exclude_unset so the name/other fields are untouched).
Clicking the active sex clears it. The full edit form still offers gender for
completeness.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
Adding a marriage/partnership event used a plain <select> for the spouse,
which is unusable on a large tree — you can't search, only scroll. Swap it
for the existing PersonCombobox (already used by the relationship form), which
filters by name as you type. No onCreate, so it still resolves to an existing
person id, which is what the partnership-event handler requires.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
justin