174 Commits

Author SHA1 Message Date
justin 7043532c3b Merge pull request 'Cleanup tool: mark deceased by a child's birth year' (#254) from cleanup-deceased-by-child into main
build-backend / build (push) Successful in 29s
build-frontend / build (push) Successful in 1m29s
2026-06-11 11:08:52 -04:00
justin 1340d1957f Cleanup tool: "mark deceased by a child's birth year" rule
Adds a preview/apply rule to the Cleanup tool for parents who have NO birth date
of their own (so the existing born-on-or-before rule can't reach them) but who
have a child born long ago — they're necessarily deceased. This is the gap that
left ~56 parents in the Paul tree as "unknown".

- cleanup_service.preview_deceased_by_child(year): parents of any child born
  on/before the cutoff, excluding already-deceased; returns child_birth_year.
- GET /trees/{id}/cleanup/deceased-by-child?born_on_or_before=1900. Apply reuses
  the existing POST .../cleanup/deceased (same audited mark-deceased path).
- Frontend: a new card in the Cleanup tool (year input → preview → select →
  apply), preview-first like the rest of the tool.

Test covers preview (finds the no-birthdate parent of a pre-cutoff child,
excludes modern-child parents), child_birth_year, apply, and re-preview drop.
Suite 106 passing.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 11:08:50 -04:00
justin e24a7cfcc9 Merge pull request 'Tree cards: living/unset-sex people render gray, not blue' (#253) from living-and-unset-cards-gray into main
build-frontend / build (push) Successful in 1m28s
2026-06-11 10:37:27 -04:00
justin 07944e329e Tree cards: render unset-sex / redacted "Living person" in gray, not blue
The chart mapped gender as `=== "female" ? "F" : "M"`, so anything non-female —
including null — became "M" (blue). On the public site, redacted living people
(whose gender the privacy engine nulls) all showed blue regardless of real sex,
and anywhere a person's sex was simply unset they also showed blue (misleading).

Map male→"M", female→"F", and everything else→null, which family-chart renders
as `card-genderless`. So living/redacted people render gray (and never imply a
sex), and unset-sex people render gray instead of defaulting to male/blue.
Applied to both the member tree (tree/page.tsx) and the public chart
(public-tree-chart.tsx), which share chart.css. Also bumped the genderless color
from the library's washed-out `lightgray` to a warm mid-gray that matches the
muted male/female tones and the brand palette.

Privacy note: `_redact` already nulls gender, so this is purely the client color
mapping — no sex leak, just a correct neutral rendering.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 10:37:25 -04:00
justin a33a88e558 Merge pull request 'docs: note the spouse-layout fix is upstreamed' (#252) from docs-upstream-spouse-fix into main 2026-06-11 09:33:21 -04:00
justin fe8349819f docs: note the spouse-layout fix is upstreamed (donatso/family-chart#105)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 09:33:19 -04:00
justin e745fb5d4d Merge pull request 'Move cardToMiddle fix into the family-chart patch (+ document patches)' (#251) from family-chart-patch-cardtomiddle into main
build-frontend / build (push) Successful in 1m27s
2026-06-11 09:21:32 -04:00
justin e0573e6be2 Move cardToMiddle vertical-centering fix into the family-chart patch
Fold the fly-to vertical-centering fix into our patch-package patch (alongside
the existing spouse-layout fix) instead of compensating in app code, and revert
the in-app workaround so the two don't double-correct.

- patches/family-chart+0.9.0.patch: cardToMiddle now scales datum.y by the zoom
  k in both dist builds (.js + .esm.js), matching datum.x. Verified the patch
  applies cleanly (patch-package --error-on-fail).
- tree/page.tsx: the cardToMiddle caller passes raw y again (the patched library
  does the scaling now); pre-scaling here too would double-correct. Behavior is
  identical to the previous in-app fix — both center the node exactly.
- CLAUDE.md: documents the two family-chart patches, how to regenerate them, and
  that both should be upstreamed. The cardToMiddle fix is submitted upstream
  (donatso/family-chart#103, issue #102); the spouse-layout fix is a TODO.

The frontend Dockerfile already COPYs patches/ before npm ci, so the fix is in
the production build.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 09:21:30 -04:00
justin 3731d77d4b Merge pull request 'Fix fly-to vertical centering at non-1 zoom levels' (#250) from fix-fly-to-vertical-centering into main
build-frontend / build (push) Successful in 1m28s
2026-06-11 08:58:38 -04:00
justin bf1576252b Fix fly-to vertical centering at non-1 zoom levels
Clicking ×N sometimes flew to a blank area far below the tree. Cause:
family-chart's cardToMiddle scales datum.x by the zoom factor k but not datum.y
(`y = height/2 - datum.y`, missing the ·k), so vertical centering is only
correct at k=1 and drifts by datum.y·(k−1) at any other zoom — worse the deeper
the person sits. That's why it worked only when the view happened to be near 1:1.

Compensate by pre-multiplying the y we pass to cardToMiddle by the current
scale, cancelling the library's missing ·k. x was already correct.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:58:36 -04:00
justin 0ed6ba4505 Merge pull request 'Tree: clicking ×N flies to the person's other copy' (#249) from tree-fly-to-duplicate into main
build-frontend / build (push) Successful in 1m31s
2026-06-11 08:47:59 -04:00
justin ed263cf9a7 Tree: clicking ×N flies to the person's other copy (not just flashes)
On a large tree the duplicate's other copy is usually off-screen, so flashing
in place wasn't enough. Clicking the ×N badge now pans/zooms the view to center
the other copy and flashes it on arrival; clicking again cycles through the
remaining copies (for a person drawn 3+ times).

Uses family-chart's exported handlers: cardToMiddle centers a datum (read from
the target card_cont's bound x/y, falling back to its transform attr), keeping
the current zoom level via getCurrentZoom. Verified against the lib: the svg's
parent (f3Canvas) holds the zoom object, and cards are positioned by datum x/y —
same coordinate space cardToMiddle expects. Falls back to an in-place flash if
the zoom object isn't ready. Frontend only; supersedes the flash-only behavior.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:47:44 -04:00
justin f7666ad30b Merge pull request 'Tree: Legend by the pan/zoom hint + clickable ×N duplicate badges' (#248) from tree-legend-and-duplicate-flash into main
build-frontend / build (push) Successful in 1m28s
2026-06-11 08:32:53 -04:00
justin 690a6da659 Tree: a Legend by the pan/zoom hint, and clickable ×N duplicate badges
Two small tree-view aids prompted by "why do some people show ×2".

- Legend: a hover/focus "Legend" link next to the "drag to pan…" hint, explaining
  the ×N badge (a person drawn N times in the view because they connect through
  more than one line — a shared ancestor or an intermarriage), the gender card
  colors, and the pan/zoom/recenter controls.
- The ×N badge is now clearly clickable (cursor + hover state); clicking it
  flashes every copy of that person in the current view (a bronze outline pulse),
  so you can spot where else they appear. Implemented by delegating on the chart
  container and matching the d3-bound person id across cards; capture-phase +
  stopPropagation so a badge click flashes instead of recentering.

Frontend only. Honest follow-up: flashing finds copies that are on-screen; a true
"fly to an off-screen copy" needs d3-zoom transform work (the chart pans by
transform, not scroll) — a later enhancement.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:32:35 -04:00
justin e7115023e1 Merge pull request 'Person page: server-side search; stop loading the whole tree' (#247) from person-page-server-search into main
build-backend / build (push) Successful in 38s
build-frontend / build (push) Successful in 1m30s
2026-06-11 08:29:32 -04:00
justin 58400ffdf7 Person page: server-side search; stop loading the whole tree
The person page fetched the entire tree on every open — all persons (to build a
name map + power the relative pickers) and all events (to find partnership
events). On a 2k-person tree that's a ~230KB person list + ~600KB event list per
view. Now it loads only what the page shows:

Frontend:
- The relationship & spouse pickers use the backend's fuzzy pg_trgm search
  (debounced, typo-tolerant) instead of substring-filtering a preloaded array —
  better search, and no need to preload every person. PersonCombobox gained an
  `onSearch` server mode (client `people` mode still works).
- The page drops the all-persons and all-events fetches; it resolves just this
  person's relatives' names via GET /persons?ids=..., and reads partnership
  events from the per-person events endpoint.

Backend:
- GET /trees/{id}/persons?ids=a,b,c — batch by id (privacy-filtered, names
  batched), for relative-name display.
- list_events_for_person (member path) now also returns the person's partnership
  events, so the page needn't scan every event in the tree.

Adversarial review (frontend logic + backend/privacy) found no issues. Suite 105
passing.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:29:13 -04:00
justin 629bfa1367 Merge pull request 'Fix list_persons N+1 (the ~4s person-page load)' (#246) from fix-person-list-n-plus-one into main
build-backend / build (push) Successful in 37s
2026-06-11 08:00:47 -04:00
justin 1562febdcf Fix list_persons N+1 (the ~4s person-page load)
Opening any person page on a large tree took 4-5s on an idle server. Root cause:
list_persons looped over every person calling privacy.person_visibility (which
issues TWO get_membership_role queries per call) AND _attach_primary_name (one
name query per person). On the reporter's 2,324-person tree that's ~7,000
serialized DB round-trips per page load — the person page fetches the full
person list to build its name-lookup map.

Fix:
- Resolve the viewer's membership role ONCE. Members see the whole tree (full),
  so skip the per-person privacy engine entirely.
- Add _attach_primary_names: one batched names query (person_id IN (...),
  ordered the same as the single-person query so it picks the same name) instead
  of one per person.
- Apply the same batching to the non-member path, search_persons, the deleted-
  persons list, and public_view_service.list_public_persons.

Member-path list_persons goes from ~3·N queries to ~3 total. Other tree-wide
list endpoints (events/relationships/media/citations) were already flat selects.

Adds a regression test that asserts list_persons issues a constant number of
queries (not proportional to person count). Suite: 103 passing.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:00:30 -04:00
justin 265f5f4e7a Merge pull request 'Close citation/source living-person leak; add on-demand tree purge' (#245) from citation-redaction-and-tree-purge into main
build-backend / build (push) Successful in 32s
build-frontend / build (push) Successful in 1m26s
2026-06-10 22:39:15 -04:00
justin a6179037c2 Close citation/source living-person leak; add on-demand tree purge
Two changes.

1. Privacy fix (NN#2/NN#3) — the citation and source list endpoints gated only
   on can_view_tree, so a non-member on a public/unlisted/site_members tree could
   enumerate citations and sources tied to a redacted living person, leaking that
   the person exists and has sourced facts (and possibly their name via a source
   title). #46 closed this for events/media/names/relationships but not
   citations/sources. Now citation_service.list_citations and
   source_service.{list_sources,get_source} delegate non-member reads to
   public_view_service, mirroring the #46 pattern:
   - citations: shown only when the cited fact resolves to FULL-visibility
     person(s) — covers the person_id, name_id, event_id (person or both-partner),
     and relationship_id (both-partner) target paths.
   - sources: shown only when they back at least one visible citation; a withheld
     source 404s (don't reveal it exists).
   Tests cover all four citation target types + source withholding + member-sees-all.

2. On-demand tree purge — owners can permanently delete a soft-deleted tree now
   instead of waiting out the 30-day auto-purge window. POST /trees/{id}/purge
   (owner-only): the tree must already be in the trash, and the caller retypes its
   name to confirm. Media objects are deleted from storage, then a single
   DELETE on trees cascades all tree-owned rows via the tree_id ON DELETE CASCADE;
   the audit entry survives (tree_id SET NULL). Frontend adds a "Delete forever"
   button to the Recently-deleted list. No migration.

Suite: 102 passing.
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-10 22:38:59 -04:00
justin 7ed3ddd448 Merge pull request 'docs: bring all documentation current with shipped work' (#244) from docs-catch-up into main 2026-06-10 21:05:46 -04:00
justin 447daf7fa8 docs: bring all documentation current with shipped work
A multi-agent audit of every doc against the code surfaced ~50 stale/missing
items (the roadmap/status docs and the backlog had fallen behind the code).
This catches them up:

- CLAUDE.md: phase status was ~3 phases stale ("Phase 1 is next" while Phase 1 +
  chunks of 2 & 4 shipped). Rewrote the status list; added a model-provider
  tech-stack entry; updated repo-layout (integrations objectstore/models,
  deploy backup.sh/dev compose).
- ARCHITECTURE.md: §6 privacy engine described 3 visibility levels — corrected to
  the shipped 4 (adds site_members); documented per-tree AI policy on Tree,
  LLMProvider/EmbeddingProvider split + registry, ChangeProposal origin/status/
  operations, verified-email session gate, instance-owner role, schema-drift
  guard, and the env_file config model.
- PRD.md: 4-level visibility in US-040/§5.5, instance-owner role (§5.1/§5.11),
  per-tree AI policy (§5.8), §8 sequencing annotated with shipped status, header
  date/status bumped.
- README.md: 4-level privacy; softened "Full GEDCOM 7" to the 5.5.1/7 common
  subset; noted backups + instance-owner admin; moved property/land to an
  explicit "where it's headed" (no property models exist yet).
- BACKLOG.md: flipped ~15 shipped-but-open rows to Have (ChangeProposal, provider
  abstraction, GEDCOM citation export, membership management, operator backup,
  email-verification gate, per-tree AI policy, instance owner, the whole
  visibility/public-viewing/child-resource-redaction cluster #41-#51/#46), and
  reconciled the executive summary, "current defects" list, quick wins, and
  differentiators. Left genuinely-open items (citation/source redaction, sitemap,
  per-tree noindex, scoped-token API) accurately open.
- .env.example: dropped "SMTP wired in a later phase"; documented the worker
  purge knobs, S3_PRESIGN_TTL, COOKIE_NAME; removed a stray duplicate line.
- design/: tree-visibility.md and change-proposal.md marked Shipped; corrected
  the redaction approach (reuses member schemas, not a separate PublicPersonRead)
  and the apply() rollback claim (v1 is not cross-op transactional), and marked
  rate-limiting/sitemap/noindex as deferred.

No code changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-10 21:05:29 -04:00
justin 0388b9b99f Merge pull request 'compose: drive app config from .env (env_file, blanket passthrough)' (#243) from compose-env-file into main 2026-06-10 08:46:16 -04:00
justin 00f403defa compose: drive backend/worker/migrate config from .env (env_file)
Replace the per-setting environment allow-list with `env_file: .env` on the
three app-image services, so every setting in app/core/config.py is configurable
from .env with no compose edit. This kills the recurring trap where a documented
env var (OWNER_EMAIL, the AI keys, SMTP, APP_BASE_URL) silently didn't reach the
app because it wasn't on the hand-maintained list.

`env_file` is `required: false` so local/CI without a .env still works (falls
back to ${VAR:-default} interpolation + code defaults). The small `environment:`
block that remains is only for values that must NOT come from .env:
  - RUN_MIGRATIONS=1 (backend) — a deploy flag, not an app setting.
  - DATABASE_URL — pinned to the compose-internal host, because the code default
    points at localhost (wrong inside the network). environment wins over
    env_file, so this is a safety net if .env ever omits it.

Trade-off (accepted, see comment): env_file also injects infra secrets
(POSTGRES_*, MINIO_*, CLOUDFLARE_TUNNEL_TOKEN) into the app process env; the app
ignores unknown vars (pydantic extra="ignore").

Verified on prod: DATABASE_URL resolves to postgres:5432, RUN_MIGRATIONS=1 and
OWNER_EMAIL intact, COOKIE_SECURE=true (no posture change), health 200, trees
200. The earlier explicit AI/SMTP/OWNER passthrough is now subsumed by this.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-10 08:46:00 -04:00
justin 519f1c31b5 Merge pull request 'compose: forward AI provider + mailer/SMTP env to the backend' (#242) from compose-ai-smtp-passthrough into main 2026-06-10 08:39:04 -04:00
justin 3a1395b6af compose: forward AI provider + mailer/SMTP env to the backend
Follow-up to the OWNER_EMAIL passthrough. The backend service env block is an
explicit allow-list, so the documented model-provider keys (ANTHROPIC_*,
OPENAI_*, XAI_*, OLLAMA_*, DEFAULT_*_PROVIDER, LLM_MAX_TOKENS,
EMBEDDING_DIMENSIONS) and mailer settings (MAILER, SMTP_*, APP_BASE_URL,
REQUIRE_EMAIL_VERIFICATION) never reached the container — setting them in .env
was a no-op. The AI assistant/policy and the SMTP mailer run in the backend, so
forward them here.

Side fix: APP_BASE_URL was likewise dropped, so outbound email links used the
code default http://localhost instead of the configured domain. Now forwarded
(verified live: backend reports APP_BASE_URL=https://provenance.paul.farm).

Worker is left as-is (it consumes neither today); it'll need the model vars when
embedding/matching jobs land. Alternative to this growing allow-list is
`env_file: .env` on the service — deferred to avoid forwarding unrelated secrets.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-10 08:38:49 -04:00
justin 2712ae469b Merge pull request 'compose: forward OWNER_EMAIL to the backend container' (#241) from compose-forward-owner-email into main 2026-06-09 23:22:59 -04:00
justin 88beb9650f compose: forward OWNER_EMAIL to the backend container
The instance-owner feature reads OWNER_EMAIL, but the backend service's
environment block is an explicit allow-list that didn't include it — so setting
it in .env never reached the app (is_instance_owner always saw "" → no owner).
Add the passthrough.

NOTE: the same allow-list omits the AI provider keys (ANTHROPIC_API_KEY,
OPENAI_*, XAI_*, OLLAMA_*) and SMTP settings, so those documented env vars also
don't currently reach the backend on this deployment. Worth a follow-up
(forward them explicitly, or switch the service to env_file) so .env actually
drives all configuration per the twelve-factor rule.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 23:22:48 -04:00
justin 15504ba6e1 Merge pull request 'Instance owner/operator role (env-declared via OWNER_EMAIL)' (#240) from instance-owner into main
build-backend / build (push) Successful in 29s
build-frontend / build (push) Successful in 1m29s
2026-06-09 23:17:08 -04:00
justin c5631d3eab Add an instance owner/operator role (env-declared via OWNER_EMAIL)
Provenance had no system-level owner: ownership was only per-tree
(TreeMembership), so a self-hosted instance had no operator account and no
instance-admin surface. This adds one, declared by environment per the project's
twelve-factor rule.

- OWNER_EMAIL (comma-separated): the account(s) named here are instance owners.
  Derived at request time — no DB column, no migration, can't drift from the env,
  survives DB resets. is_instance_owner()/InstanceOwner dependency in api/deps.py.
- Ownership requires a VERIFIED email (independent of REQUIRE_EMAIL_VERIFICATION).
  Registration is open, so without this an attacker could seize the role by
  registering the owner address first; verification ties it to inbox control.
- GET /api/v1/admin/instance (owner-only): operational status — version, env,
  user/tree counts, configured AI providers. Deliberately exposes no tree data
  or PII: instance ownership is an operator role, NOT a privacy-engine bypass.
- /users/me reports is_instance_owner; frontend gains an owner-only /admin page
  and a conditional sidebar link (server-enforced, not just client-hidden).

Found-and-fixed by an adversarial security review before merge: the
verified-email land-grab (above) and a frontend null-deref where the admin page
crashed on 401/5xx instead of failing closed.

Docs: .env.example + ARCHITECTURE (notes the not-a-privacy-bypass boundary and
the verified-email requirement). Tests: owner matching, the land-grab guard,
/users/me, and owner-only /admin. Suite 96 passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 23:16:45 -04:00
justin 6fbad3106d Merge pull request 'Guard against schema drift (readiness 503 + loud startup log)' (#239) from schema-drift-guard into main
build-backend / build (push) Successful in 32s
2026-06-09 21:56:08 -04:00
justin 94b5caa7e5 Guard against schema drift: fail readiness + log loudly when DB is behind code
Defense-in-depth for the deploy pipeline. Today a backend image shipped ahead
of an un-applied migration; the Tree model selected columns the DB didn't have
yet, so every trees query 500'd with an opaque UndefinedColumnError and the UI
showed no trees. The root cause (deploys not running migrations) is fixed
separately; this makes the *symptom* impossible to miss.

- app/core/schema_version.py: compare the DB's stamped alembic head to the
  head(s) baked into the image's migration scripts. A DB with no alembic_version
  table (e.g. a create_all test DB) is treated as current, so this stays quiet
  outside real deployments. Uses to_regclass so a missing table never poisons
  the caller's transaction.
- /health/ready: returns 503 with an explicit "drift: db=… expected=…" message
  when the schema is behind, instead of reporting ready and serving 500s.
- Startup lifespan: logs CRITICAL on drift (advisory — never blocks startup).

Liveness (/health) is untouched, so a drifted container isn't killed into a
crash-loop — it's loudly degraded and self-heals once migrations apply.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 21:55:21 -04:00
justin f8fa23c1f6 Merge pull request 'Per-tree AI model policy (owner-only admin view)' (#238) from ai-model-policy into main
build-backend / build (push) Successful in 32s
build-frontend / build (push) Successful in 1m27s
2026-06-09 20:53:07 -04:00
justin c6b1e72130 Per-tree AI model policy (owner-only admin view)
The operator decides which model providers exist (env / registry — Anthropic,
OpenAI, x.AI, Ollama, several at once). The *tree owner* decides who uses which:

- Members' assistant -> one configured provider (or none)
- Recommender (association/connection finder) -> one configured provider (or none)
- Owner -> may use any configured provider

Backend: two nullable columns on `trees` (ai_member_provider,
ai_recommender_provider) + migration; `configured_llm_providers()` exposes the
registry as {name, model} with no secrets; owner-gated GET/PATCH
/trees/{id}/ai validate names against the configured set. Frontend: owner-only
"AI models" page with a dropdown per role, graceful 403 for non-owners, and a
sidebar link.

Per-model-within-a-provider selection is a follow-up; today each provider maps
to its single configured model.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 20:52:30 -04:00
justin ceafb299d6 Merge pull request 'Model providers: OpenAI/xAI/Ollama + run several at once' (#237) from multi-provider-openai-xai-ollama into main
build-backend / build (push) Successful in 32s
2026-06-09 18:39:20 -04:00
justin de50f2c803 Model providers: OpenAI/xAI/Ollama + run several at once (registry)
Extends the #215 abstraction:
- OpenAICompatibleLLMProvider / OpenAICompatibleEmbeddingProvider — one impl (via
  the official openai SDK) covers OpenAI, xAI (api.x.ai/v1), Ollama
  (…:11434/v1), OpenRouter, etc.; they differ only by base_url, key, and model.
- Registry factory: build_llm_providers() / build_embedding_providers() return
  every provider whose credentials are configured, so you can run several
  concurrently. get_llm_provider(name)/get_embedding_provider(name) select by
  name, falling back to default_*_provider, then Null.
- Per-provider env config (ANTHROPIC_*, OPENAI_*, XAI_*, OLLAMA_*) +
  DEFAULT_LLM_PROVIDER / DEFAULT_EMBEDDING_PROVIDER; documented in .env.example.
  Defaults keep AI off (empty registry).

Embeddings now have real backends (OpenAI/Ollama), still separate from the LLM
since Anthropic offers no embeddings endpoint. Tests cover multi-provider
selection, default resolution, disabled-without-credentials, and null fail-loud.
Full suite 87 passed.

Relates to #215.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 18:39:19 -04:00
justin 9187c0a791 Merge pull request 'Fix #214: ChangeProposal (propose-then-confirm)' (#236) from change-proposal into main
build-backend / build (push) Successful in 31s
build-frontend / build (push) Successful in 1m25s
2026-06-09 15:44:41 -04:00
justin abaa8efdd5 Fix #214: ChangeProposal (propose-then-confirm)
Implements non-negotiable #1: the AI assistant never writes autonomously. Every
assistant/contributor "write" emits a ChangeProposal — a structured diff a human
approves, edits, or rejects. Design: docs/design/change-proposal.md.

Structural guarantee: a proposal's operations reach the DB ONLY via
change_proposal_service.apply(), which requires the actor be an editor and
dispatches each op through the normal editing services (person/name/event/
relationship/source/citation create/update/delete) — so every change passes the
privacy engine and is audited as the approving human. propose() only inserts a
pending row; it performs no domain mutation. Model providers stay read-only, so
no model response can mutate tree data.

- ChangeProposal model + migration (status pending|applied|rejected, origin
  assistant|contributor, JSONB operations, reviewer + apply_error).
- Service: propose / list / get / apply (with optional edited ops) / reject /
  delete; a dispatcher mapping ops → editing services. v1 applies ops in order,
  not cross-op transactional (single-op is atomic; documented).
- API /trees/{id}/proposals + a frontend review page (approve/reject; editor-
  gated) and sidebar entry.

Tests: proposal doesn't apply until approved; reject doesn't apply; non-editor
member can see but not apply; multi-op; approve-with-edits; apply-error keeps it
pending. Full suite 87 passed; single alembic head.

Closes #214

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 15:44:40 -04:00
justin 251a10a087 Merge pull request 'Fix #215: pluggable LLM + embedding provider abstraction' (#235) from model-provider-abstraction into main
build-backend / build (push) Successful in 28s
2026-06-09 12:51:03 -04:00
justin 330543f9ce Fix #215: pluggable LLM + embedding provider abstraction
Adds the vendor-agnostic seam the AI assistant + match-ranking plug into:
- LLMProvider / EmbeddingProvider ABCs (base.py). LLM and embeddings are
  SEPARATE abstractions — Anthropic has no embeddings endpoint, so each is
  configured independently and either can be off.
- NullLLMProvider / NullEmbeddingProvider — the default; fail loud with a clear
  "not configured" error so AI-off deployments don't silently no-op.
- AnthropicLLMProvider — first concrete LLM impl, via the official anthropic SDK
  (default model claude-opus-4-8). A local provider (e.g. Ollama) would be
  another subclass of the same interface.
- Factory in deps.py (get_llm_provider / get_embedding_provider) selects by
  env (MODEL_PROVIDER / EMBEDDING_PROVIDER); documented in .env.example.

Providers are read-only text/vector producers — they never touch the DB, so the
"AI never writes autonomously" invariant (CLAUDE.md #1) holds; writes will go
through ChangeProposal (#214).

Tests: provider selection (null default, anthropic when keyed, fallback without
key) + null providers raise. 81 passed.

Closes #215

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 12:51:01 -04:00
justin d540dc3f32 Merge pull request 'Fix #196: one-command operator backup (pg_dump + MinIO)' (#234) from operator-backup-script into main 2026-06-09 12:45:35 -04:00
justin 8652425413 Fix #196: one-command operator backup (pg_dump + MinIO)
Move backup from a documented procedure to `deploy/backup.sh`: dumps Postgres
(pg_dump --clean --if-exists, gzipped) and archives the MinIO /data directory
into a single timestamped bundle under backups/. Reads config from the compose
.env with the same defaults the stack uses; optional BACKUP_RETAIN_DAYS prunes
old bundles (cron-friendly). BACKUP.md documents usage + the restore procedure
(kept manual/documented rather than an untested destructive script).

Closes #196

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 12:45:33 -04:00
justin 3a7728f1dc Merge pull request 'Fix #145: tree membership management (list / add / role / remove)' (#233) from membership-management into main
build-backend / build (push) Successful in 27s
build-frontend / build (push) Successful in 1m29s
2026-06-09 12:43:31 -04:00
justin eb0350733b Fix #145: tree membership management (list / add / role / remove)
TreeMembership was enforced on every read/write but had no API or UI to manage
members — trees were effectively single-user, breaking full-CRUD (NN#8).

Backend (/trees/{id}/members): list (members only — the list exposes emails, so
non-members never see it, even on public trees); add an existing user by email
(owner only, 404 if no such account, 409 if already a member); PATCH role;
DELETE. A tree must always keep ≥1 owner (demote/remove of the sole owner → 409).
All changes audited.

Frontend: a Members page (owner gets add-by-email + per-member role select +
remove; others see a read-only list) and a sidebar entry.

Test covers the full lifecycle + every guard. Suite 77 passed.

Closes #145

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 12:43:30 -04:00
justin 6d3147e86d Merge pull request 'Fix #169: keep citation links on GEDCOM export' (#232) from fix-gedcom-citation-export into main
build-backend / build (push) Successful in 29s
2026-06-09 12:37:04 -04:00
justin b4434cb5dd Fix #169: keep citation links on GEDCOM export
Export emitted SOUR records but never the per-fact SOUR links, so a
Provenance→Provenance round-trip destroyed the sources graph (citations were
dropped). Emit citation links on the facts they sit on:
- person-level → 1 SOUR @Sx@ (2 PAGE)
- name-level   → 2 SOUR under 1 NAME
- event-level  → 2 SOUR under the event (incl. partnership events in FAM)
- relationship → 1 SOUR under FAM
Citations whose source didn't export are skipped.

Test: a person + event citation round-trips through export→import into a fresh
tree with their pages intact. GEDCOM suite 6 passed.

Closes #169

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 12:37:03 -04:00
justin 39e3eac3df Merge pull request 'Security: gate sessions on verified email (opt-in)' (#53) from security-require-email-verification into main
build-backend / build (push) Successful in 35s
2026-06-09 11:22:55 -04:00
justin 660fe7b37f Security: gate sessions on verified email (opt-in)
Backlog §2.10: registration issued a live session and email_verified_at was
written but never read, so an unverified user had full access and there was no
switch to require verification.

Add REQUIRE_EMAIL_VERIFICATION (default false). When true:
- resolve_session_user returns None for a user whose email_verified_at is null —
  the single read-side gate covering every authenticated request, incl. the
  session minted at registration.
- login raises 403 ("email not verified") instead of issuing a useless token.

Default false on purpose: self-hosts without SMTP, and accounts created before
this gate existed (email_verified_at null), must not be locked out. Operators
enable it once mail works and accounts are verified. Documented in .env.example.

Tests: default-off keeps unverified accounts working; on → register's session
won't resolve (401), login is 403, and after verify-email both work. 75 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 11:22:54 -04:00
justin 5485dd2077 Merge pull request 'Cleanup: infer a missing sex from a known-sex spouse (preview → approve)' (#52) from cleanup-sex-from-spouse into main
build-backend / build (push) Successful in 26s
build-frontend / build (push) Successful in 1m28s
2026-06-09 10:59:10 -04:00
justin 05d2773e25 Cleanup: infer a missing sex from a known-sex spouse (preview → approve)
Unset sex renders blue (male-colored), which is misleading next to a confirmed
male partner. Add a Cleanup action that proposes the opposite sex for an unset
partner of someone whose sex is set (couples are opposite-sex in practice — a
confirmed-male husband ⇒ a female wife). People whose known partners disagree
are skipped as ambiguous.

It's a preview the user reviews and approves in the Cleanup tool (reusing the
existing gender apply path + audit) — not an autonomous write. Backend:
guess_gender_by_spouse + GET /cleanup/gender/from-spouse. Frontend: an "Infer
from spouse" button feeding the existing proposal list. Test covers
propose-opposite, skip-no-partner, skip-already-set, apply, and re-preview.

Full suite 73 passed; frontend build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 10:59:08 -04:00
justin 768c68cbe0 Merge pull request 'Public tree view: add generation depth controls (shared with member view)' (#51) from public-tree-depth-controls into main
build-frontend / build (push) Successful in 1m29s
2026-06-09 10:35:44 -04:00
justin 7d6fbce87e Public tree view: add generation depth controls (shared with member view)
The public tree chart was fixed at 3 ancestors / 2 descendants. Add the same
Generations controls the member view has (slider + number stepper + "All" per
direction), applied live around the focused person.

Extracts the member page's inline DepthControl into a shared
components/depth-control.tsx and uses it in both, so they stay in sync. The
public chart gains anc/prog depth state + an apply effect (setAncestryDepth/
setProgenyDepth + updateTree) mirroring the member behavior.

tsc clean; next build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 10:35:43 -04:00
justin 12ba0a0fb6 Merge pull request 'Public tree view: full-width canvas like the member view' (#50) from public-tree-fullwidth into main
build-frontend / build (push) Successful in 1m29s
2026-06-09 10:29:20 -04:00
justin 150d69e5ac Public tree view: full-width canvas like the member view
The public layout forced max-w-5xl on every /p page, so the tree chart was
cramped. Mirror the member shell: the public layout now drops the max-width for
the tree page (/p/<id>) only, giving the chart the full canvas (74vh to match
the member view), while the page keeps its heading and people list in a
centered max-w-5xl column. Person detail (/p/<id>/persons/<pid>) and /explore
stay narrow.

tsc clean; next build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 10:29:18 -04:00
justin 053ce357ac Merge pull request 'Public view: add tree chart + homepage Explore links' (#49) from public-tree-chart-and-explore-link into main
build-frontend / build (push) Successful in 1m25s
2026-06-09 09:44:24 -04:00
justin 269cae556f Public view: add tree chart + homepage Explore links
Two gaps from review of the public surface:
- The public tree page showed only a list of names. Add the family-chart
  hourglass (PublicTreeChart) above the directory — the same renderer the
  member tree view uses, including the cycle-sanitisation that guards against a
  bad graph, fed by redacted public data. Click a card to recenter; "Open"
  links to the person's public page. Centers on the tree's home person.
- The homepage had no path to /explore. Add an "Explore" nav link and an
  "Explore public trees" hero button.

tsc clean; next build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:44:23 -04:00
justin 0df44e7e59 Merge pull request 'Visibility phase 5: public /explore directory + search' (#48) from visibility-phase5-explore into main
build-frontend / build (push) Successful in 1m28s
2026-06-09 09:34:21 -04:00
justin 7a5c5f2882 Visibility phase 5: public /explore directory + search
A no-login directory of shared trees, backed by GET /api/v1/public/trees:
- /explore: searchable grid of public trees; debounced name search. Because the
  backend adds `site_members` trees when a valid session is present, signed-in
  users see more with no client-side branching.
- PublicHeader extracted and shared by /p and /explore (logo, Explore, Sign in).
- "Explore" entry added to the authed sidebar.

tsc clean; next build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:34:20 -04:00
justin 20c7fbd8d6 Merge pull request 'Visibility phase 4: no-login public viewer pages + robots' (#47) from visibility-phase4-public-pages into main
build-frontend / build (push) Successful in 1m28s
2026-06-09 09:31:58 -04:00
justin b8405ced07 Visibility phase 4: no-login public viewer pages + robots
Adds the public viewing surface in the UI — shareable, no-login pages backed by
the redaction-safe /api/v1/public API:

- /p/[treeId]: tree name + searchable people directory (living people show as
  "Living person"; counts; links to person pages).
- /p/[treeId]/persons/[personId]: person detail — events, alternate names, and
  parents/partners/children as links to other public person pages.
- app/p/layout.tsx: slim public header (logo + Sign in), no app sidebar.
- robots.ts: allow /p/, disallow the authenticated app sections.
- Trees list: a "Public page ↗" link on every non-private tree so the owner can
  grab the shareable URL.

Client-rendered (same-origin fetch via Caddy). Follow-up (needs a frontend
SSR→backend base URL + a compose/env deploy step, so not auto-applied by
Watchtower): true server-rendering for SEO, a dynamic sitemap of public trees,
and per-page noindex for unlisted/site_members.

tsc clean; next build passes (both routes dynamic).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:31:56 -04:00
justin 91a7ce1dc2 Merge pull request 'Fix leak: redact per-person on authed non-member reads' (#46) from fix-authed-nonmember-redaction into main
build-backend / build (push) Successful in 31s
2026-06-09 09:26:54 -04:00
justin 8b91326481 Fix leak: redact per-person on authed non-member reads
A logged-in NON-member of a public/unlisted tree could read living people's
dates, real alternate names, and media (incl. downloading photos) through the
family-view endpoints — only the person LIST was redacted; list_events,
list_relationships, list_names, list_media gated on can_view_tree alone.

For non-members, these now delegate to the same visibility-filtered reads the
public surface uses (person_visibility-driven): living-person events/names
dropped, relationships touching a hidden person dropped, media limited to
full-visibility persons, and media download (get_media → media_content) 404s
for a redacted/unlinked person's media. Members are unchanged.

Adds list_public_relationships_for_person / list_public_media / can_view_media
to public_view_service. Test: an authed non-member sees no living-person PII
across events/names/relationships/media and can't download a living person's
file, while the owner still sees everything. Full suite: 72 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:26:53 -04:00
justin 671b560768 Merge pull request 'docs: add product backlog (genealogy feature gap analysis)' (#45) from add-product-backlog into main 2026-06-09 09:19:22 -04:00
justin 6a5ef4d392 docs: add product backlog (genealogy feature gap analysis)
Output of a multi-agent gap analysis comparing Provenance against commercial
(Ancestry/MyHeritage/FamilySearch) and open-source (GRAMPS/Gramps Web/webtrees)
genealogy software: 15 research lenses, 580 raw features deduped into a 17-
category taxonomy, 302 features assessed against the codebase (have/partial/
planned/missing) with statuses verified against the actual code.

Includes an executive summary, per-category backlog with status/importance/
effort/phase, a quick-wins shortlist, and strategic differentiators. Statuses
reflect the repo at analysis time (before tree-visibility phases 1-3); a couple
of flagged items (e.g. the site_members tier) are already closed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:19:21 -04:00
justin 3810b65de0 Merge pull request 'Visibility phase 3: redaction-safe public read API + leak test' (#44) from visibility-phase3-public-api into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m25s
2026-06-09 09:17:55 -04:00
justin 9820a77d25 Visibility phase 3: redaction-safe public read API + leak test
Adds the anonymous read surface (/api/v1/public) — the privacy-critical core.

- CurrentUserOrNone dependency: optional auth that never 401s (anonymous OK).
- public_view_service: every projection passes through privacy.person_visibility.
  persons redacted (living → "Living person", hidden dropped); relationships
  only when both endpoints non-hidden; events only for FULL-visibility persons
  (partnership events only when both partners full); names only for FULL
  persons. Not-viewable trees raise 404 (not 403) so the surface can't probe
  for private trees. Media deferred (higher-sensitivity; own pass later).
- public router: read-only directory + tree + persons/relationships/events +
  person detail/names/events. Directory lists `public` to all and adds
  `site_members` for authenticated callers; never lists unlisted/private.
- PublicTreeRead omits owner_id.

Tests (ran locally — CI does not run pytest): an anonymous end-to-end leak test
asserting a living person's real name, alias, and birth year appear in NO public
response while the deceased person's data does; plus private=404, unlisted
viewable-by-link-but-unlisted, site_members requires login, and directory
visibility. Full suite: 70 passed. Regenerated openapi.json + TS client.

Note: the AUTHED list endpoints still leak per-person for non-members
(pre-existing) — fixed next, separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:17:41 -04:00
justin 3ff03b037b Merge pull request 'Visibility phase 2: privacy-engine branches on viewer auth state' (#43) from visibility-phase2-privacy into main
build-backend / build (push) Successful in 29s
2026-06-09 09:08:17 -04:00
justin 84a743f5b9 Visibility phase 2: privacy-engine branches on viewer auth state
can_view_tree() now distinguishes anonymous vs authenticated non-members so the
four-level model is enforceable:
- public / unlisted → anyone, including anonymous (unlisted gated only by the
  link, so the API must never *list* it)
- site_members → any authenticated account (denies anonymous)
- private → members only
Members (any role) always view; soft-deleted trees stay hidden from everyone.
person_visibility (living-person redaction) is unchanged.

Tests: a full can_view_tree matrix across {anonymous, logged-in non-member,
member} × {public, unlisted, site_members, private}, plus deleted-tree-hidden
and the site_members anon-vs-logged-in case. Adds `engine`/`db_session` fixtures
(refactored out of `client`) so the engine can be unit-tested directly,
including the anonymous path that has no HTTP endpoint yet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:08:04 -04:00
justin e6dfe39e84 Merge pull request 'Visibility phase 1: add site_members value + 4-option dropdown' (#42) from visibility-phase1-enum into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m26s
2026-06-09 09:02:01 -04:00
justin 4a3fe983fa Visibility phase 1: add site_members value + 4-option dropdown
First step of the public-viewing feature (design: docs/design/tree-visibility.md).
No non-member behavior change yet — this only widens the vocabulary and UI.

- TreeVisibility gains `site_members` (any authenticated user of the instance),
  giving the four-level model: public / site_members / unlisted / private.
- Alembic migration adds the enum value via an autocommit block (ALTER TYPE
  ADD VALUE can't run in a transaction on older Postgres); downgrade is a no-op
  since PG can't drop an enum value.
- Regenerated openapi.json + frontend TS client.
- Trees-list dropdown now offers Private / Public – Members / Unlisted / Public
  with an explanatory tooltip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 08:54:45 -04:00
justin 251652a935 Merge pull request 'Trees list: inline visibility selector (private/unlisted/public)' (#41) from tree-visibility-control into main
build-frontend / build (push) Successful in 1m30s
2026-06-09 08:39:42 -04:00
justin dc1b6aac01 Trees list: inline visibility selector (private/unlisted/public)
Tree visibility was set to private with no UI to change it — the trees list
only displayed the value as text. Add a private/unlisted/public dropdown on
each tree card that PATCHes visibility immediately (optimistic), pulled out of
the card's navigation Link so it doesn't trigger a page change. Honors the
"everything configurable / full CRUD in the UI" invariants. Living people stay
protected by the privacy engine regardless of tree visibility.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 08:36:20 -04:00
justin f93327f5d3 Merge pull request 'Tree view: configurable generation depth (ancestors/descendants + All)' (#40) from configurable-tree-depth into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 22:21:04 -04:00
justin c86771034c Tree view: configurable generation depth (ancestors/descendants + All)
Depth was hardcoded (3 ancestors, 2 descendants). Add a controls row to set
each direction independently — a slider plus a number stepper, with an "All"
toggle per direction — applied around whoever is currently focused.

- ancestor/descendant depth held in state; effective value is a large cap
  when "All" is on (the chart only renders people that exist, so the cap is
  free).
- changes apply to the live chart via setAncestryDepth/setProgenyDepth +
  updateTree without a full rebuild.
- fan mode (ancestors only) takes the ancestor depth via its `generations`
  prop, capped at 8 to avoid the radial layout's 2^n blow-up; its descendants
  control is disabled with a note.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 22:20:07 -04:00
justin b51b65de80 Merge pull request 'Person page: one-click sex setter (no edit mode)' (#39) from quick-set-sex into main
build-frontend / build (push) Successful in 1m25s
2026-06-08 22:03:32 -04:00
justin 93c22b4bcf Person page: one-click sex setter (no edit mode)
Setting a person's sex meant clicking Edit, opening a dropdown, and saving.
Replace the read-only ♂/♀ symbol next to the name with an always-visible
two-button segmented control that PATCHes immediately on click (gender-only;
backend PATCH is exclude_unset so the name/other fields are untouched).
Clicking the active sex clears it. The full edit form still offers gender for
completeness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 21:42:59 -04:00
justin 7255920135 Merge pull request 'Person page: make marriage-event spouse picker searchable' (#38) from searchable-marriage-spouse into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 21:30:48 -04:00
justin 62513ee22e Person page: make marriage-event spouse picker searchable
Adding a marriage/partnership event used a plain <select> for the spouse,
which is unusable on a large tree — you can't search, only scroll. Swap it
for the existing PersonCombobox (already used by the relationship form), which
filters by name as you type. No onCreate, so it still resolves to an existing
person id, which is what the partnership-event handler requires.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 21:29:27 -04:00
justin ac0b9818dd Merge pull request 'Tree view: center a person between multiple spouses' (#37) from center-spouse-layout into main
build-frontend / build (push) Successful in 1m26s
2026-06-08 19:56:59 -04:00
justin 182a5dab16 Tree view: center a person between multiple spouses
family-chart 0.9.0 stacks all of a person's spouses on one gender-determined
side, so someone with two spouses (e.g. a woman with two husbands) renders
with both spouses piled above/below her and ambiguous child lines.

Patch the library (via patch-package) so the person stays centered and their
spouses split to alternating sides — spouse 1 above, spouse 2 below, further
spouses farther out — and order each couple's children to match, so children
descend from between the correct pair without crossed lines:

- setupSpouses: keep the person centered; place spouses at alternating
  offsets and recenter the cluster on the person's slot.
- sortChildrenWithSpouses: order children by spouse order (gender-independent)
  to match the new spouse positions.

Adds patch-package + a postinstall hook, and COPY patches into the Dockerfile
deps stage so the patch applies during `npm ci` in CI. Verified the patch
re-applies on a clean install and the production build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 19:56:17 -04:00
justin 77b78410ff Merge pull request 'Tree view: add "Back to default person" recenter link' (#36) from add-default-person-link into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 15:25:04 -04:00
justin fe1e0171ff Tree view: add "Back to default person" recenter link
Once you recenter the tree on someone, there was no quick way back to the
tree's home/default person. Add a header link (shown only when a home person
is set and you're not already on them) that recenters the chart on
home_person_id via the existing goTo() — works in landscape, portrait, and
fan modes. Labels with the home person's name for clarity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 15:15:39 -04:00
justin 9dbdae975a Merge pull request 'Preserve focused person across tree/people/detail navigation' (#35) from improve-tree-people-navigation into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 15:07:10 -04:00
justin c5a2a7f0d4 Preserve focused person across tree/people/detail navigation
The Tree view, People (Family) view, and person detail page each tracked
the "current person" independently, so moving between them reset you to the
home person. The detail page's "← Back to tree" link also pointed at the
People view (not the Tree) and carried no person, so it always landed on the
default person.

Make the focused person a URL-encoded concept that travels across views:

- Tree and People views read ?focus=<id> on load and mirror the focused
  person back into the URL via router.replace (no history spam), so leaving
  and returning keeps you centered where you were. Bookmarks/shared links
  also resolve to the right person.
- "Open person" links carry ?from=tree | ?from=people.
- The detail page's back link is now origin-aware: "← Back to Tree" →
  /tree?focus=<id> or "← Back to People" → /?focus=<id>, returning you in
  place instead of to the home person.
- Add a "View in tree →" link on the detail page — the previously missing
  direct jump from a person to the tree re-rooted on them.
- person→person relationship links (and create-relative redirect) pass
  `from` through so click-chains keep their anchor.

Also gitignore *.tsbuildinfo (Next build artifact).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 14:48:32 -04:00
justin 8c36785197 Merge pull request 'Prevent duplicate relationships; harden tree render against cycles' (#34) from prevent-duplicate-links into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m23s
2026-06-08 11:35:12 -04:00
justin fae1162ff8 Prevent duplicate relationships; harden tree render against bad graphs
Root cause of the blank Jung tree: a child double-linked to the same parent
(and, generally, any cycle) made family-chart recurse forever.

Backend (the real fix):
- create_relationship now rejects an equivalent existing edge → 409.
  parent_child is directional (parent→child); partnership/sibling match the
  pair in either order. So you can't link the same two people the same way
  twice. (GEDCOM import already deduped; manual creates didn't.)

Frontend (defense in depth so data can never blank the view):
- Tree view sanitizes the graph before rendering: dedupes parents/spouses,
  drops self-links, and greedily breaks ancestor cycles (a person can't be
  their own ancestor); children are derived from the kept edges. The render is
  wrapped in try/catch and shows a note instead of a blank canvas, telling you
  which conflicting links were skipped.
- Person page surfaces the 409 ("They're already linked that way.").

59 backend tests pass (incl. dup-rejection + reverse-parent-child allowed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 11:35:11 -04:00
justin 1025f86657 Merge pull request 'Cleanup: list people with no sex set + inline set' (#33) from cleanup-unset-sex into main
build-frontend / build (push) Successful in 1m25s
2026-06-08 10:43:10 -04:00
justin a53858f920 Cleanup: list people with no sex set + inline set
Adds a "People with no sex set" section to the Cleanup page — lists everyone
whose gender is still null with inline ♂ Male / ♀ Female buttons (and a link to
their page). Refreshes after the source-match and first-name guess passes, so
it's the manual mop-up for whatever those leave behind.

Frontend only (reuses person list + PATCH) — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 10:43:08 -04:00
justin 941f9827c1 Merge pull request 'Cleanup: best-guess sex from first name (offline dictionary)' (#32) from gender-name-guess into main
build-backend / build (push) Successful in 33s
build-frontend / build (push) Successful in 1m26s
2026-06-08 10:30:36 -04:00
justin 6ec852a23a Cleanup: best-guess sex from first name (offline dictionary)
A "Guess from first name" option in the Cleanup gender section: a bundled,
curated given-name -> sex dictionary (weighted English + German for the first
real tree) proposes sex for people who don't have it set. Deterministic, offline,
no model. Genuinely ambiguous names (Marion, Frances, Jordan, …) are excluded
from both sets so they're left for a human. Reuses the existing preview/apply
gender flow, so every guess is reviewed before saving.

No migration. 56 backend tests pass; frontend builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 10:30:35 -04:00
justin 7405ec762f Merge pull request 'Tree Cleanup tool: bulk deceased / gender-from-source / name fixes (preview-first)' (#31) from tree-cleanup into main
build-backend / build (push) Successful in 27s
build-frontend / build (push) Successful in 1m28s
2026-06-08 10:17:02 -04:00
justin aa62ca490e Tree Cleanup tool: bulk fixes with preview → approve
A new per-tree Cleanup page (and cleanup_service + endpoints), each fix
preview-first per the propose-then-approve rule:

- Mark deceased by birth year: lists people born ≤ a cutoff (default 1930) not
  already deceased; apply sets is_living=false for the ones you keep checked.
- Set sex from a source GEDCOM: upload the source .ged (it carries SEX); matches
  by name and proposes sex only where it's missing — far more accurate than
  guessing from first names. Review, then apply.
- Names that look broken: flags date-in-surname / date-in-given / no-surname /
  packed given names, with inline editable given+surname; fix the checked ones.

No migration (uses existing columns). 55 backend tests pass (preview+apply for
all three); frontend builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 10:17:01 -04:00
justin 97f7a9e0ff Merge pull request 'Show a sex symbol after the name on the person page' (#30) from person-sex-symbol into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 09:16:40 -04:00
justin cd4ccb4ac8 Show a sex symbol after the name on the person page
A blue ♂ (male) or pink ♀ (female) symbol now follows the person's name in the
detail header, using the same gender tints as the tree cards. Nothing shows when
sex is unknown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 09:16:38 -04:00
justin 6696015970 Merge pull request 'Full light/dark theme toggle; brand-aware connector lines' (#29) from light-dark-theme into main
build-frontend / build (push) Successful in 1m26s
2026-06-07 11:49:01 -04:00
justin e8839b15a0 Full light/dark theme toggle; brand-aware connector lines
- Theme is now class-based (.dark on <html>) with a System/Light/Dark toggle in
  the sidebar, persisted to localStorage and applied pre-paint by an inline
  script (no flash). Replaces the prefers-color-scheme-only behavior, so a phone
  on a light OS theme can still choose dark and vice versa.
- New brand-derived --line token (Ink at 55%): a dark line on the light paper,
  light on dark. The family-chart tree connectors had the library's default
  white stroke and were invisible in light mode — now they use --line, as do
  the pedigree brackets and the fan-chart sectors.
- Light/dark tokens use the exact brand palette (Ink/Muted flip; Bronze/Paper
  constant).

Frontend only — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:48:59 -04:00
justin 548e883d82 Merge pull request 'Discoverable Add Person + inline create-new when linking relatives' (#28) from create-person-ux into main
build-frontend / build (push) Successful in 1m27s
2026-06-07 11:30:16 -04:00
justin 37ac49767e Make creating a person obvious; inline "create new" when linking relatives
- Family view gets a prominent "+ Add person" button that creates a person and
  opens their page to fill in details (previously you could only add a person
  via the empty-state form or by linking from another person).
- The person page's relationship picker (PersonCombobox) now offers
  "+ Create '<typed name>'" when the person doesn't exist yet: it creates them,
  links them in the chosen role (parent/child/partner/sibling), and jumps to
  their new page to edit — no more create-then-go-back-and-link.

Frontend only — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:30:14 -04:00
justin 9b04bcefba Merge pull request 'Account export / restore-into-new-tree / delete' (#27) from account-export-restore-delete into main
build-backend / build (push) Successful in 26s
build-frontend / build (push) Successful in 1m27s
2026-06-07 11:26:06 -04:00
justin e9b2436ce0 Account export / restore-into-new-tree / delete
New account_service + endpoints under /users/me:
- GET /me/export — zip of every owned tree (account.json + media blobs).
- POST /me/import — restore a backup into NEW trees (ids remapped, media
  re-uploaded); non-destructive, never touches existing data.
- DELETE /me — soft-delete the user, their owned trees, and revoke sessions;
  guarded by retyping the account email.

Settings page wires all three (export download, restore upload, delete with
typed-email confirmation). No migration — uses existing tables + soft-delete.

52 backend tests pass (export→restore round-trip + delete guards); frontend builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:26:04 -04:00
justin 8903e480cf Merge pull request 'Link media to people (person page + media page)' (#26) from media-person-linking into main
build-frontend / build (push) Successful in 1m24s
2026-06-07 11:19:27 -04:00
justin d27cc5dddc Link media to people (person page + media page)
The Media model already carried person_id/event_id/source_id and the upload
route already accepted person_id — this surfaces it in the UI:

- Person page: a Media card lists media linked to that person, uploads new
  files already linked ("Upload & link"), links existing unlinked media, and
  unlinks.
- Media page: each item gets a person picker to link/unlink.

Frontend only — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:19:25 -04:00
justin 943f459b91 Merge pull request 'Shared marriage events; deterministic parent ordering' (#25) from marriage-event-parent-order into main
build-frontend / build (push) Successful in 1m23s
2026-06-07 11:15:55 -04:00
justin 5106538934 Shared marriage events; deterministic parent ordering
- Partnership life events (marriage/divorce/engagement) now attach to the
  couple's relationship, not each person. The add-event form asks for the
  spouse, finds-or-creates the partnership, and writes ONE event on it — shown
  on both partners' pages ("· with <spouse>"), entered once. Event values
  (RELI/OCCU detail) now render too.
- Family-view pedigree orders parents deterministically (father on top, mother
  below, stable fallback when gender is unknown) instead of by which link was
  created first.

Frontend only — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:15:54 -04:00
justin 2669543e56 Merge pull request 'Account menu + Settings (change password), per-tree home person, full-width tree' (#24) from account-settings-home-person into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m24s
2026-06-07 11:05:41 -04:00
justin 0262ed3d97 Account menu + Settings (change password); per-tree home person; full-width tree
- Sidebar bottom-left now shows the signed-in user; clicking opens a menu with
  Settings and Sign out. New /settings page: account info + change password
  (POST /auth/change-password, re-verifies current password). Export/restore/
  delete are stubbed there for the next pass.
- Per-tree default/home person: tree.home_person_id (migration) + TreeUpdate/
  Read; the tree and family views open focused on it; the person page gets a
  "Set as default" control and "Default person" badge. Cleared if that person
  is deleted. Complements the account-level "this is me" link.
- Tree visualization now fills the content area (AppShell drops the max-width
  column on the /tree route); other pages stay centered.
- Audit records are coerced JSON-safe (UUIDs/enums), so PATCHing UUID fields
  like home_person_id audits cleanly.

50 backend tests pass; migration up/down verified; frontend builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:05:04 -04:00
justin 9ee960c4ef Merge pull request 'Auto-apply migrations on deploy (entrypoint + one-shot service)' (#23) from deploy-auto-migrate into main
build-backend / build (push) Successful in 26s
2026-06-07 10:54:31 -04:00
justin 7f640649b9 Auto-apply migrations on deploy (entrypoint + one-shot service)
So a deploy never needs a manual `alembic upgrade head`:

- Backend image gains an entrypoint that runs `alembic upgrade head` before
  uvicorn when RUN_MIGRATIONS=1 (set on the backend service). This self-migrates
  even on a Watchtower in-place image swap, which doesn't re-run one-shot jobs.
- A one-shot `migrate` service covers the `docker compose up` path; backend and
  worker depend on it completing, which also serializes it with the backend
  entrypoint so alembic never runs concurrently. `upgrade head` is idempotent.

Activating this needs the updated compose on the host once (Watchtower only
swaps images, not the compose file / env). After that, migrations are automatic.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:50:28 -04:00
justin a8929c2862 Merge pull request 'Global Import menu entry + mobile drawer nav' (#22) from nav-global-import-mobile into main
build-frontend / build (push) Successful in 1m29s
2026-06-07 10:41:12 -04:00
justin b90ba53a3f Merge pull request 'GEDCOM: duplicate-aware import + maiden/married + RELI/NOTE mapping' (#21) from gedcom-import-dedupe into main
build-frontend / build (push) Has been cancelled
build-backend / build (push) Successful in 30s
2026-06-07 10:41:08 -04:00
justin c4e9d69e00 Merge pull request 'Alternate names, self-person link, deletion integrity + dangling people' (#20) from names-deletion-self into main
build-backend / build (push) Has been cancelled
build-frontend / build (push) Has been cancelled
2026-06-07 10:41:02 -04:00
justin 0673896133 Merge pull request 'Tree search + click-rebuild; searchable relationship picker; gender dropdown' (#19) from tree-search-combobox into main
build-frontend / build (push) Has been cancelled
2026-06-07 10:40:59 -04:00
justin 1164841950 Global Import in the menu; mobile drawer nav
- Add a top-level "Import" entry to the sidebar and a global /import page, so
  you can start a tree from a GEDCOM without first creating an empty one. The
  import flow now picks its destination (new tree, or an existing one) — the
  tree-scoped page reuses the same <GedcomImport> with a fixed destination and
  keeps Export.
- Extract the sidebar chrome into <AppShell> and give small screens a working
  menu: a hamburger opens the full sidebar as a slide-in drawer (it was just a
  logo + "Trees" link before). Used by both /trees and /import.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:40:01 -04:00
justin 5824e70895 GEDCOM: duplicate-aware import + typed name/attribute mapping
Duplicate detection (the "merge / skip / overwrite" the user asked for):
- New POST /gedcom/preview dry-runs the file and flags incoming people that
  resemble existing ones (name similarity via difflib + birth-year guard;
  high/medium score). No writes.
- /gedcom/import takes default_action (new|skip|merge|overwrite) + per-xref
  resolutions {xref: {action, target_id}}:
    new       create as a new person (current behavior)
    skip      link families to the existing person, copy nothing
    merge     attach the incoming names (as alternates), events, citations,
              and notes onto the existing person
    overwrite soft-delete the existing person, import the incoming one fresh
  Relationship creation is deduped so a merge can't double an edge.

Richer record mapping (covers the user's repo's GEDCOM):
- Multiple NAME records honor their TYPE; _MARNM (and NICK) import as typed
  alternate names — maiden stays primary, married becomes a "married" Name.
- RELI -> a "religion" event with the value in detail; OCCU/EDUC values too.
- NOTE -> person notes (and event notes); NOTE/RELI are no longer "unmapped".
- Export round-trips name TYPE.

Verified against the user's 2185-person export: 0 unmapped tags. 48 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:35:55 -04:00
justin 04ccdbf96a Alternate names (maiden/married), self-person link, deletion integrity
Names (the genealogy standard: maiden name primary, married/alias as typed
alternates):
- Name model already supported multiple typed names; expose full CRUD —
  NameCreate/Read/Update schemas, name_service (one-primary invariant,
  promote-on-delete), nested /persons/{id}/names routes.
- Person page gains a Names card: add/edit/delete + "make primary", with a
  curated name_type dropdown (birth/maiden, married, alias, nickname, …).

Self-person ("who am I"):
- users.self_person_id FK (use_alter for the users<->persons<->trees cycle)
  + migration; PATCH /users/me/self-person; "This is me" / "This is you"
  on the person page. Soft-deleting the linked person clears it.

Deletion integrity (fixes the broken tree view):
- delete_person now soft-deletes the relationships touching the person, so no
  dangling edges remain; family-chart also filters links to missing people.
- Optional cascade=true recursively deletes descendants (GEDCOM cleanup);
  the person page asks "only this person" vs "with all descendants".
- DELETE returns {deleted: n}.

Family view surfaces "Not connected to anyone" so dangling people aren't lost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:21:12 -04:00
justin f165ccb941 Tree search + click-rebuild; searchable relationship picker; gender dropdown
- Tree page: add a "Find a person" search box that jumps the chart to a
  match and rebuilds the hourglass (parents/grandparents/partner/children)
  around them. Clicking any card recenters via family-chart's default
  behavior (setAncestryDepth 3 / setProgenyDepth 2), syncing focus through
  setAfterUpdate for the "Open profile" link.
- Person detail: replace the relationship "add" <select> with a
  type-to-filter PersonCombobox so long people lists are searchable.
- Person detail: gender is now a Male/Female dropdown, not free text.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 09:58:45 -04:00
justin e0fb924a1d Merge pull request 'Full-CRUD sweep (API): update for tree/source/citation/relationship/media' (#18) from crud-sweep-backend into main
build-backend / build (push) Successful in 30s
2026-06-07 09:53:18 -04:00
justin cf5518c7ec Full-CRUD sweep: update endpoints for tree, source, citation, relationship, media
Closes the rule #8 gap at the API layer: PATCH endpoints + service updates for Tree (name/description/visibility), Source, Citation (page/detail/confidence), Relationship (qualifier/notes), and Media (title/attachment) — editor-gated and audited. Every core entity now has create/read/update/delete. Edit UIs for these land in the frontend batch. 37 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 09:53:17 -04:00
justin 26df03cfd7 Merge pull request 'Edit people + events; existing-person picker; full-CRUD rule' (#17) from crud-edits into main
build-backend / build (push) Successful in 27s
build-frontend / build (push) Successful in 1m25s
2026-06-07 09:35:56 -04:00
justin ab064bce6e Edit UI for people and life events; existing-person picker in family view
Person detail: an Edit form for name + gender + living status + privacy, and inline edit of each life event (type + structured date). Family view: the add-relative buttons now search existing people (link the real person) or create new — preventing duplicate spouses/parents — and adding a child to someone with one spouse links both parents.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 09:35:55 -04:00
justin 76b7f453c1 Add update (CRUD) for events and people; record the full-CRUD invariant
Events and people are now editable, not write-once: PATCH /events/{id} (type, structured date, place, notes) and PATCH /persons/{id} (vitals, privacy, and the primary name's given/surname). CLAUDE.md gains rule #8: every stored object must support full CRUD in API and UI — historical research is constant correction. Tests cover both updates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 09:35:55 -04:00
justin 438d2db2e7 Merge pull request 'Tree layout toggles + fan + card->profile + server search' (#16) from phase2-tree-toggles into main
build-frontend / build (push) Successful in 1m26s
2026-06-07 08:01:32 -04:00
justin 99913ada94 Tree layout toggles (landscape/portrait/fan), card->profile, server search
Tree page gets Landscape/Portrait/Fan toggles: landscape & portrait via family-chart's orientation; a hand-rolled radial Fan chart of ancestors (rings per generation, click to recenter). Clicking a card recenters and updates an 'Open <name> →' link to that person's profile. The People directory search now hits the server-side pg_trgm fuzzy endpoint (debounced) so it spans the whole tree, not just the loaded page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 08:01:31 -04:00
justin 584b323121 Merge pull request 'Fuzzy search (pg_trgm) + living-person protection' (#15) from phase2-search-privacy into main
build-backend / build (push) Successful in 30s
2026-06-07 07:55:14 -04:00
justin 4788ae7723 Add fuzzy name search (pg_trgm) and living-person protection
Fuzzy search: pg_trgm extension + trigram GIN indexes on name parts and a GET /trees/{id}/persons?q= search ranked by trigram similarity (finds Mueller for 'muller'), privacy-filtered. Living-person protection: the privacy engine now derives possibly-living status (explicit flag, else no death fact + birth within ~100y or unknown) and returns 'redacted' for non-members of public/unlisted trees; the service minimises those records ('Living person', no vitals). Members are unaffected. 31 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 07:55:13 -04:00
justin 51f0066e61 Merge pull request 'Interactive Tree view (pan/zoom genealogy chart)' (#14) from interactive-tree into main
build-frontend / build (push) Successful in 1m21s
2026-06-06 23:07:04 -04:00
justin bfa6c0782a Add an interactive Tree view (pan/zoom genealogy chart)
Researched how FamilySearch/Geni/MyHeritage lay out trees (switchable pedigree/portrait/fan, an interactive canvas with pan/zoom + click-to-recenter, gender colors, birth-death years) and built a real Tree page on the MIT d3 library family-chart instead of a flat list. Ancestors + descendants around a focus person, click any card to recenter, drag to pan, scroll to zoom — scales to large imported trees. Tree is now the first per-tree sidebar item and the default when opening a tree; People keeps the searchable directory + add/edit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 23:07:02 -04:00
justin 2f21e767f3 Merge pull request 'Scalable people directory' (#13) from people-directory into main
build-frontend / build (push) Successful in 1m20s
2026-06-06 22:54:10 -04:00
justin f6bcf198ee Make the people index a scalable scrollable directory
A flat wrap of every person didn't scale to imported trees. Replace it with a bounded (max-height, scrollable) searchable directory: clean name + birth–death-year rows, focus highlight, a result count, and a 200-row cap with a 'refine your search' notice so a thousand-person tree stays fast and usable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:54:08 -04:00
justin b13fafd624 Merge pull request 'Phase 2: GEDCOM import/export' (#12) from phase2-gedcom into main
build-backend / build (push) Successful in 26s
build-frontend / build (push) Successful in 1m22s
2026-06-06 22:46:50 -04:00
justin 631d050540 Add GEDCOM Import/Export UI (defaults to importing into a new tree)
An Import/Export page (sidebar) that defaults to importing into a NEW tree to avoid duplicating existing people, with an explicit 'append to this tree' option (warned), a mapping-report display (counts + skipped tags), and a one-click .ged export download.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:46:48 -04:00
justin d48029a407 Add GEDCOM import/export
A pragmatic GEDCOM parser + mapper: import reads INDI/FAM/SOUR and creates people, names, life events, partnership + qualified parent-child relationships, marriage events, places (deduped), sources, and citations from SOUR refs — returning a mapping report (counts + unmapped tags). Export serializes the tree back to GEDCOM (families derived from the edge model). Import is additive (no merge) and runs inline for now. Round-trip test passes; 29 tests total.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:46:48 -04:00
justin 18dea507d1 Merge pull request 'Pedigree connector lines + 4 grandparents' (#11) from pedigree-connectors into main
build-frontend / build (push) Successful in 1m22s
2026-06-06 22:32:12 -04:00
justin 99a660485e Pedigree: connector lines + correct 4-grandparent structure
Rebuilds the family view's pedigree as a recursive bracket chart with CSS connector lines — focus links to its two parents (2 lines), and each parent links to its two parents (4 lines to grandparents). Fixes the prior ambiguity where grandparent slots weren't tied to a specific parent: now every parent shows its own two parent slots, so a person clearly has up to four grandparents grouped by lineage. Height-robust connectors (each leaf draws its own spine half + stub).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:32:10 -04:00
justin cf6dcf9ce2 Merge pull request 'Family view + soft-delete/recovery' (#10) from phase1-familyview into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m18s
2026-06-06 22:19:02 -04:00
justin 22bc536978 Rebuild People as a family view (pedigree + family group); add recovery UI
The People page is no longer a flat list: it's a focus-person family view with a pedigree of ancestors (parents + grandparents), a spouse/partner panel, and a children panel — with inline 'add parent/child/spouse' (creates the person + the relationship), click-to-refocus, birth–death years, and a searchable people index. Modeled on how real genealogy tools center on a person and let you walk the graph.

Adds delete/restore UI: a Delete on the person page, per-tree delete + a 'Recently deleted' restore section on the trees list, and a Recovery page (sidebar) for deleted people.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:19:01 -04:00
justin f2205b93f4 Add soft-delete + recovery and tree-wide graph endpoints
Tree and person soft-delete + restore (owner-only for trees, editor for people) with recovery listings (?deleted=true); the worker already purges past the 30-day window. Adds tree-wide GET /relationships and /events so the family/pedigree view loads the whole graph in a few calls. 27 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:19:01 -04:00
justin b0c7c8570b Merge pull request 'App-shell UI overhaul + media stream endpoint' (#9) from ui-shell into main
build-backend / build (push) Successful in 26s
build-frontend / build (push) Successful in 1m20s
2026-06-06 21:56:26 -04:00
justin fe9a95c60d Rebuild the UI as an app shell: left sidebar, media gallery, structured events
Replaces the centered single-column of full-width cards with a proper application layout: a persistent left sidebar (Trees, and per-tree People/Sources/Media, with the tree name and sign-out) and a constrained content column. Marketing landing and auth pages are split out (own header/footer; centered auth with the logo).

Adds a Media gallery (upload + image thumbnails / file tiles, served via the backend content endpoint). Events are no longer free-text: a curated event-type list (+ custom) and a structured date (qualifier + day/month/year) that composes a proper genealogical date. Regenerated the OpenAPI client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 21:56:05 -04:00
justin bd8ee9b647 Stream media through the backend (browser-reachable, privacy-checked)
Presigned URLs point at the internal minio:9000 host a browser can't reach. Add ObjectStore.get_object and a GET /media/{id}/content endpoint that resolves visibility and streams the bytes; MediaRead.url now points there. Keeps the object store private and downloads behind the privacy engine.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 21:56:04 -04:00
justin 660130f007 Merge pull request 'Phase 1: media (object storage) + background worker' (#8) from phase1-media into main
build-backend / build (push) Successful in 30s
2026-06-06 21:46:35 -04:00
justin 34d30e3134 Add media (object storage) and the background worker (Phase 1)
Media model + migration; an ObjectStore interface with an S3/MinIO (boto3) implementation behind the service layer. Upload (multipart) stores bytes in object storage + a metadata row (checksum, size, content-type, optional attach to person/event/source); list returns presigned URLs; delete is soft. Editor-gated, privacy-filtered, audited. 24 tests pass (object store faked).

Introduces the worker container (same image, 'python -m app.worker'): its first job is the scheduled 30-day soft-delete purge across tables + media object cleanup. Compose gains worker + S3 env on backend/worker; dev override builds the worker too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 21:46:09 -04:00
justin 049545fcc8 Merge pull request 'Frontend redesign: real type, hero, depth' (#7) from design-overhaul into main
build-frontend / build (push) Successful in 1m18s
2026-06-06 21:34:48 -04:00
justin 3a14fcc4ca Redesign the frontend: real type, hero landing, depth
Lifts the UI from wireframe to a finished heritage look: Fraunces (display serif) + Inter (sans) via next/font; a proper hero landing with a feature triad and the Origin mark; a warm bronze-tinted background gradient for depth; a sticky branded header and refined footer. Polished button (sizes + bronze focus ring + shadow), card (rounded-xl, soft layered shadow), and input (bronze focus) primitives that carry across every page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 21:34:47 -04:00
justin fc4cb0273e Merge pull request 'Phase 1: sources-first spine (sources + citations)' (#6) from phase1-sources into main
build-backend / build (push) Successful in 24s
build-frontend / build (push) Successful in 1m18s
2026-06-06 13:17:34 -04:00
justin 83f83ab641 Add source manager and inline citing with 'sourced' badges
New /trees/[id]/sources page (list + create sources). Person-detail page now loads tree sources + citations and shows a '✓ N sourced' badge with an inline cite picker (source + page) on each event and on the person. Tree view links to Sources. Regenerated the OpenAPI client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 13:17:33 -04:00
justin 064bb6ea65 Add sources and citations API (Phase 1: sources-first spine)
Source CRUD (reusable, tree-scoped) and Citation create/list/soft-delete linking one source to exactly one fact (person/event/name/relationship). Editor-gated writes, privacy-filtered reads, audit throughout; tenant + existence validation on source and target. list_citations returns all tree citations so the UI can render 'sourced' indicators in one round-trip. 22 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 13:17:33 -04:00
justin fbb9d0195c Merge pull request 'Phase 1: events + relationships + person detail' (#5) from phase1-graph into main
build-backend / build (push) Successful in 27s
build-frontend / build (push) Successful in 1m16s
2026-06-06 12:11:11 -04:00
justin 1f25eb2f21 Add person-detail page with events timeline and relationships
New /trees/[id]/persons/[personId] view: life-events timeline with add/remove, and relationships grouped into parents/children/partners/siblings with an add form (kind + person picker + qualifier). People in the tree list now link here. Regenerated the OpenAPI client for the new endpoints.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 12:10:56 -04:00
justin d6e2df4a61 Add events and relationships API (Phase 1: flesh out the graph)
Events (create/list-per-person/soft-delete) and relationships (create/list-per-person/soft-delete) through the layered stack: editor-gated writes, privacy-engine reads, audit on every change. Events carry exactly one subject (person XOR partnership); relationships are typed qualified edges (parent_child gets a biological/adoptive/step/foster/donor/guardian qualifier). Adds a single-person GET. 18 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 12:10:56 -04:00
justin a799d101b5 Merge pull request 'Use host global Watchtower (drop bundled one)' (#4) from watchtower-use-host into main 2026-06-06 11:58:50 -04:00
justin 0b9d72c878 Drop bundled Watchtower; rely on the host's global Watchtower
ripper already runs a single global nickfedor/watchtower (label-enabled) that watches every stack; the bundled containrrr/watchtower was redundant and crash-looped (its Docker API client is too old for Docker 29). Keep the watchtower.enable labels on backend/frontend so the host instance auto-deploys them; remove the per-stack service and profile.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 11:58:49 -04:00
justin 2d0635e710 Merge pull request 'Add Watchtower auto-deploy (2-min poll)' (#3) from watchtower-autodeploy into main 2026-06-06 11:55:51 -04:00
justin 768d1b23d4 Add Watchtower auto-deploy for app images (2-minute poll)
Watchtower (profile-gated) watches only the label-enabled backend/frontend containers and recreates them when a new :test-main digest lands in the registry, polling every 120s. Scoped by label so it never touches Postgres/MinIO/Caddy/cloudflared. Reads registry creds from the host docker config. Lab host runs COMPOSE_PROFILES=tunnel,watchtower.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 11:55:38 -04:00
justin 11f0f79866 Merge pull request 'Frontend rebrand: ink + bronze + paper' (#2) from frontend-rebrand into main
build-frontend / build (push) Successful in 1m16s
2026-06-06 11:51:13 -04:00
justin b8f5c35045 Apply brand identity to the frontend (ink + bronze + paper)
Replaces the default black/gray with the docs/brand palette: warm ink text on paper surfaces, bronze accent, serif headings and the Origin-mark wordmark in the header, favicon, and the 'where it came from matters' tagline. Light/dark adapt via CSS vars (ink/paper flip); bronze and paper are constant. Tailwind v4 @theme exposes bronze/paper/ink tokens and the serif stack. Buttons/inputs/cards restyled to match; brand SVGs vendored into public/.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 11:49:58 -04:00
justin 9e6cf6e5b7 Merge pull request 'Phase 0 — Foundation: backend, data model, local auth, frontend, deploy + CI' (#1) from phase-0-foundation into main
build-backend / build (push) Failing after 26s
build-frontend / build (push) Failing after 1m16s
2026-06-06 11:32:31 -04:00
justin 828445a6b3 Add Cloudflare Tunnel connector (profile-gated) to the deploy stack
A cloudflared service (opt-in via the 'tunnel' compose profile, token from CLOUDFLARE_TUNNEL_TOKEN) connects the lab to Cloudflare. One public hostname -> http://caddy:80 is sufficient because Caddy does the internal path routing. Mirrors the drawbar tunnel setup.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 11:32:15 -04:00
justin 4921ce0776 Mirror drawbar CI/CD: push to LAN registry, pull via public FQDN
Split the registry endpoints like the drawbar containers. Per-component Gitea Actions workflows (build-backend, build-frontend; runs-on docker, path-filtered) push images to the LAN endpoint 192.168.0.2:1234 over plain HTTP (buildx insecure/http) to bypass Cloudflare's request-body limit, then link each package to the repo via the Gitea API. Auth via the REGISTRY_TOKEN Actions secret (the same token drawbar uses). Tag scheme: test-main / test-sha-<long> / version / latest (v* tags).

The deploy compose now PULLS git.jpaul.io/justin/provenance-{backend,frontend}:${IMAGE_TAG:-test-main} (no host build); docker-compose.dev.yml is a local-build override for dev / pre-CI. Replaces the previous single build.yml. Docs + memory updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 11:19:26 -04:00
justin 6c3a9a89aa Mark Phase 0 complete in CLAUDE.md and ARCHITECTURE
Records the landed frontend scaffold and that Phase 0 (backend, data model, local auth, deploy stack, CI, frontend) is complete and running live; Phase 1 (core tree features) is the new current target.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 11:03:07 -04:00
justin fccc81a6cc Wire the frontend into the deploy stack and CI
Compose gains a frontend service; Caddy now routes / to frontend:3000 (keeping /api/* and /health* on the backend). CI builds and pushes a frontend image alongside the backend. Verified end-to-end on the deploy target: / serves the app, /api and /health still resolve through Caddy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 11:03:07 -04:00
justin a5a79f01a7 Scaffold Next.js frontend with generated OpenAPI client and core views
Next.js (App Router) + React 19 + TypeScript + Tailwind v4, with shadcn-style UI primitives (Button, Input, Card, Label via cva/tailwind-merge). A typed API client is generated from the backend OpenAPI spec with openapi-typescript + openapi-fetch (npm run gen:api); the committed openapi.json/schema.d.ts are the snapshot.

Views: landing, login, register, tree list + create, and tree detail with person list + create. Auth rides the same-origin HttpOnly session cookie the backend sets (Caddy proxies /api/*), so no token handling in JS. Built as a standalone container. Mobile-first; next build is clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 11:03:07 -04:00
justin e2edd4b2f1 Document landed local auth in CLAUDE.md and ARCHITECTURE
Records the auth model (Argon2, opaque sessions, Bearer/cookie, email verify/reset behind AuthProvider/Mailer), supersedes the interim X-User-Id note, and adds integrations/ to the backend layout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:51:51 -04:00
justin 9f8dd960f4 Test auth flows and switch core tests to session auth
New auth suite covers registration, login (incl. wrong-password), email verification, password reset (old sessions + old password rejected), logout revocation, and no-enumeration on reset. Core tenancy tests now authenticate via real sessions. A capturing mailer makes email flows assertable. 13 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:51:51 -04:00
justin 00bfe8bfca Add local auth: AuthProvider, mailer, sessions, /api/v1/auth
Pluggable AuthProvider interface with a local (email+password) implementation, and a Mailer interface (ConsoleMailer for dev, SMTPMailer for operators). The auth service owns registration, login, opaque session issuance, email verification, and password reset (which revokes prior sessions). Endpoints under /api/v1/auth; sessions are returned as a Bearer token and set as an HttpOnly cookie.

Replaces the temporary X-User-Id shim: get_current_user now resolves a real session (Bearer or cookie). The open user-bootstrap endpoint is gone (registration replaces it). App logging is configured so the ConsoleMailer's verification/reset links are visible to self-hosters. Verified end-to-end on the deploy target, including the email-verification flow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:51:51 -04:00
justin 5123c85397 Add auth foundation: sessions/tokens schema, Argon2 hashing, config
Two tables (sessions, user_tokens) + migration; only token *hashes* are stored, so a DB leak yields no usable credential. Argon2id password hashing and token primitives in app/core/security. Config and .env.example gain session/cookie/token TTLs, app base URL, and SMTP settings (twelve-factor). Migration verified reversible (drops the token_purpose enum) and matches the models.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:51:51 -04:00
justin e5a8713293 Document core-model decisions in CLAUDE.md and ARCHITECTURE
Records the landed data model and backend layout, the Phase 0 tree-scoping of Place (vs. the eventual shared gazetteer), and the temporary X-User-Id auth shim.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:40:19 -04:00
justin 64388b75bf Add core API integration tests
End-to-end coverage of the tenancy/people flow and the privacy seam (private-tree isolation, public-tree view-but-not-edit, duplicate-email conflict, auth-required). DB-backed tests run against TEST_DATABASE_URL and skip cleanly when it is unset, so the no-DB suite still runs anywhere.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:40:19 -04:00
justin dffd05d303 Add layered service/API for tenancy and people with the privacy seam
Wires the data model through repository -> service -> API/v1. The privacy engine (app/services/privacy.py) is the single enforcement point: every read resolves visibility there (tree role, tree visibility, per-person override; living-person redaction is a marked Phase 2 TODO). All writes record an attributable AuditEntry.

Endpoints: POST /users (open dev bootstrap until auth), GET /users/me, POST/GET /trees, GET /trees/{id}, and POST/GET /trees/{id}/persons. Authn is a temporary X-User-Id header shim; authz is membership-based (owner/editor/viewer). Domain errors map to 401/403/404/409. Verified on the deploy target: private tree -> 403 for non-members, missing actor -> 401, audit log populated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:40:19 -04:00
justin 297cb797d6 Add core data model (12 tables) and initial Alembic migration
All core entities from ARCHITECTURE §5: tenancy (User, Tree, TreeMembership), people (Person, Name, Relationship), facts (Event, Place, PlaceName), provenance (Source, Citation), and the append-only AuditEntry. Cross-cutting mixins give every row a UUID key, timestamps, soft delete, and (where tree-owned) a tree_id for uniform tenant isolation.

Modeling choices: parentage as qualified edges (biological/adoptive/step/foster/donor/guardian) so non-traditional families are first-class; events keep both a verbatim date string and a normalized start/end range; closed sets are PG enums while GEDCOM-extensible vocabularies (event/name/source type) stay strings; CHECK constraints enforce single-subject events and single-target citations. Place is tree-scoped in Phase 0 (see ARCHITECTURE note). The migration is verified reversible (upgrade/downgrade drops tables and enum types) and matches the models (alembic check clean); applied on the deploy target. Dockerfile now ships migrations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:40:00 -04:00
justin 03124027fe Record Phase 0 repo layout in CLAUDE.md and ARCHITECTURE
Documents the scaffolded tree (/backend, /deploy, /.gitea, pending /frontend), the deploy-first sequencing, and the toolchain choices (uv for backend deps, Alembic for migrations), as CLAUDE.md's layout section requires when code lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:17:12 -04:00
justin 9e4252ba8f Add Gitea Actions CI to build the backend image
Builds and pushes the backend container image to the Gitea registry on git.jpaul.io on push to main and version tags, so servers pull to deploy (no build on the host). Registry credentials come from repo secrets (REGISTRY_USERNAME/REGISTRY_PASSWORD); runner label may need adjusting to the configured Gitea runner.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:17:12 -04:00
justin 0b5c3b260a Add self-host compose stack (Postgres, MinIO, backend, Caddy)
One env-driven compose stack stands up the whole system per ARCHITECTURE §2/§12. Postgres uses the pgvector image (pgvector + pg_trgm in contrib); MinIO is the S3-compatible store; Caddy reverse-proxies /api/* and /health* to the backend with an env-driven site address (':80' local, a domain for auto-HTTPS, or plain HTTP behind a Cloudflare Tunnel). Healthchecks and depends_on gate startup order.

.env.example documents twelve-factor config (DB, S3, SMTP, Caddy, model keys) with placeholders; no secrets in the repo. Verified end-to-end on the deploy target: all services healthy, /health/ready green against real Postgres.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:17:12 -04:00
justin 03aa9a3ca7 Scaffold FastAPI backend skeleton with health probes
Phase 0 foundation. uv-managed FastAPI app (package=false, runs from source via uv run). Layered seams in place: app/api for routers, app/core for config (pydantic-settings, fully env-driven) and the async SQLAlchemy engine; service/repository/domain layers land with the data model.

Exposes /health (liveness) and /health/ready (Postgres reachability via SELECT 1, 503 on failure) so the deploy wiring is verifiable before any data model exists. Includes a liveness test and the resolved uv.lock. Ignore pytest/ruff/mypy caches.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 10:16:58 -04:00
225 changed files with 38345 additions and 30 deletions
+105
View File
@@ -0,0 +1,105 @@
name: build-backend
# Builds + pushes the backend image to justin/provenance-backend's package area
# on Gitea on every merge to main. Servers pull from git.jpaul.io.
#
# Push goes to the LAN registry endpoint 192.168.0.2:1234 (plain HTTP) to bypass
# Cloudflare's request-body limit; pulls use the public git.jpaul.io FQDN. Same
# Gitea registry either way. Mirrors the drawbar setup.
#
# Tag scheme: test-main | test-sha-<long> | <semver from pyproject> | latest (v* tags)
on:
workflow_dispatch:
push:
branches: [main]
tags: ['v*']
paths:
- 'backend/**'
- '.gitea/workflows/build-backend.yml'
concurrency:
group: build-backend-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: docker
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Extract version from pyproject.toml
id: ver
run: |
v=$(grep -oP '^version = "\K[^"]+' backend/pyproject.toml | head -1)
if [ -z "$v" ]; then echo "could not parse version from backend/pyproject.toml"; exit 1; fi
echo "semver=$v" >> "$GITHUB_OUTPUT"
echo "backend semver: $v"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
# LAN registry serves plain HTTP on :1234 (git.jpaul.io is the only TLS
# endpoint, via Cloudflare). Treat the LAN endpoint as insecure so
# buildkit doesn't try to upgrade the push to HTTPS.
config-inline: |
[registry."192.168.0.2:1234"]
http = true
insecure = true
- name: Configure registry credentials for buildx
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
REGISTRY_USER: ${{ github.actor }}
run: |
mkdir -p ~/.docker
AUTH=$(printf '%s:%s' "$REGISTRY_USER" "$REGISTRY_TOKEN" | base64 -w0)
cat > ~/.docker/config.json <<EOF
{
"auths": {
"192.168.0.2:1234": {
"auth": "$AUTH"
}
}
}
EOF
- name: Compute tags
id: meta
uses: docker/metadata-action@v5
with:
images: 192.168.0.2:1234/justin/provenance-backend
tags: |
type=ref,event=branch,prefix=test-
type=sha,prefix=test-sha-,format=long
type=raw,value=${{ steps.ver.outputs.semver }}
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
labels: |
org.opencontainers.image.source=https://git.jpaul.io/justin/provenance
org.opencontainers.image.version=${{ steps.ver.outputs.semver }}
- name: Build and push (amd64)
uses: docker/build-push-action@v6
with:
context: ./backend
platforms: linux/amd64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Link package to the provenance repo
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
code=$(curl -s -o /tmp/link.out -w "%{http_code}" -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \
"https://git.jpaul.io/api/v1/packages/justin/container/provenance-backend/-/link/provenance")
echo "link -> provenance: HTTP $code"
case "$code" in
201) echo "OK — newly linked" ;;
400|409) echo "OK — already linked" ;;
*) cat /tmp/link.out; exit 1 ;;
esac
+102
View File
@@ -0,0 +1,102 @@
name: build-frontend
# Builds + pushes the Next.js image to justin/provenance-frontend's package area
# on Gitea on every merge to main. Servers pull from git.jpaul.io.
#
# Push -> LAN registry 192.168.0.2:1234 (plain HTTP); pull -> git.jpaul.io.
# Mirrors the drawbar setup; see build-backend.yml for the rationale.
#
# Tag scheme: test-main | test-sha-<long> | <version from package.json> | latest (v* tags)
on:
workflow_dispatch:
push:
branches: [main]
tags: ['v*']
paths:
- 'frontend/**'
- '.gitea/workflows/build-frontend.yml'
concurrency:
group: build-frontend-${{ github.ref }}
cancel-in-progress: true
jobs:
build:
runs-on: docker
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Extract version from package.json
id: ver
run: |
v=$(grep -oP '"version"\s*:\s*"\K[^"]+' frontend/package.json | head -1)
if [ -z "$v" ]; then echo "could not parse version from frontend/package.json"; exit 1; fi
echo "semver=$v" >> "$GITHUB_OUTPUT"
echo "frontend version: $v"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
# See build-backend.yml for why these flags are needed.
config-inline: |
[registry."192.168.0.2:1234"]
http = true
insecure = true
- name: Configure registry credentials for buildx
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
REGISTRY_USER: ${{ github.actor }}
run: |
mkdir -p ~/.docker
AUTH=$(printf '%s:%s' "$REGISTRY_USER" "$REGISTRY_TOKEN" | base64 -w0)
cat > ~/.docker/config.json <<EOF
{
"auths": {
"192.168.0.2:1234": {
"auth": "$AUTH"
}
}
}
EOF
- name: Compute tags
id: meta
uses: docker/metadata-action@v5
with:
images: 192.168.0.2:1234/justin/provenance-frontend
tags: |
type=ref,event=branch,prefix=test-
type=sha,prefix=test-sha-,format=long
type=raw,value=${{ steps.ver.outputs.semver }}
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
labels: |
org.opencontainers.image.source=https://git.jpaul.io/justin/provenance
org.opencontainers.image.version=${{ steps.ver.outputs.semver }}
- name: Build and push (amd64)
uses: docker/build-push-action@v6
with:
context: ./frontend
platforms: linux/amd64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
- name: Link package to the provenance repo
env:
GITEA_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
code=$(curl -s -o /tmp/link.out -w "%{http_code}" -X POST \
-H "Authorization: token ${GITEA_TOKEN}" \
"https://git.jpaul.io/api/v1/packages/justin/container/provenance-frontend/-/link/provenance")
echo "link -> provenance: HTTP $code"
case "$code" in
201) echo "OK — newly linked" ;;
400|409) echo "OK — already linked" ;;
*) cat /tmp/link.out; exit 1 ;;
esac
+5
View File
@@ -25,6 +25,11 @@ target/
dist/ dist/
build/ build/
# Tooling caches
.pytest_cache/
.ruff_cache/
.mypy_cache/
# Logs # Logs
*.log *.log
npm-debug.log* npm-debug.log*
+34 -4
View File
@@ -19,6 +19,7 @@ These are product invariants, not preferences. Do not violate them, and flag any
5. **Sources are first-class.** Don't model citations as free-text afterthoughts. A `Source` is a reusable entity; a `Citation` links it to a specific fact. 5. **Sources are first-class.** Don't model citations as free-text afterthoughts. A `Source` is a reusable entity; a `Citation` links it to a specific fact.
6. **Only legal data sources.** Ship scrapers/connectors only for permissible sources (FamilySearch API, Find A Grave, WikiTree, BLM/GLO, USGS, public-domain newspapers, public county records). Never add connectors for paywalled/terms-prohibited sites (Ancestry, MyHeritage, 23andMe). 6. **Only legal data sources.** Ship scrapers/connectors only for permissible sources (FamilySearch API, Find A Grave, WikiTree, BLM/GLO, USGS, public-domain newspapers, public county records). Never add connectors for paywalled/terms-prohibited sites (Ancestry, MyHeritage, 23andMe).
7. **Everything is configurable via environment.** Auth, mail, object storage, database, model providers, scrapers — all twelve-factor. No hard-coded endpoints or keys. 7. **Everything is configurable via environment.** Auth, mail, object storage, database, model providers, scrapers — all twelve-factor. No hard-coded endpoints or keys.
8. **Full CRUD on every object.** Every stored entity (person, name, event, relationship, source, citation, media, tree, …) must support create, read, **update**, and delete — in the API *and* the UI. Historical research is constant correction and new information, so nothing is write-once. Any new feature or data type ships with all four operations; an entity you can create but not edit is a bug.
## Tech stack ## Tech stack
@@ -29,7 +30,8 @@ These are product invariants, not preferences. Do not violate them, and flag any
- **Object storage:** S3-compatible (MinIO for self-host). - **Object storage:** S3-compatible (MinIO for self-host).
- **Edge:** Caddy reverse proxy; optional Cloudflare Tunnel (preferred ingress, never required). - **Edge:** Caddy reverse proxy; optional Cloudflare Tunnel (preferred ingress, never required).
- **Email:** operator-configured SMTP. - **Email:** operator-configured SMTP.
- **CI/CD:** Gitea Actions on `git.jpaul.io` build container images to the Gitea registry; servers pull to deploy. - **Model providers:** pluggable `LLMProvider` + `EmbeddingProvider` abstraction (ABCs) with Null / Anthropic / OpenAI-compatible (OpenAI, xAI, Ollama) implementations; an operator configures one or more via env and they're selectable by name through a registry (per-tree AI policy + `default_llm_provider`/`default_embedding_provider`).
- **CI/CD:** Gitea Actions build per-component images. **Push** to the LAN registry `192.168.0.2:1234` (plain HTTP, bypasses Cloudflare's body limit); **pull** via the public `git.jpaul.io` FQDN. Servers pull to deploy — no host build. Mirrors the drawbar setup; see [[gitea-lan-push-fqdn-pull]].
Pick libraries consistent with this stack. If you introduce a significant dependency or a new service, note it in ARCHITECTURE.md in the same change. Pick libraries consistent with this stack. If you introduce a significant dependency or a new service, note it in ARCHITECTURE.md in the same change.
@@ -38,13 +40,24 @@ Pick libraries consistent with this stack. If you introduce a significant depend
``` ```
/ # docs and project meta (this file, README, LICENSE, COC, CONTRIBUTING) / # docs and project meta (this file, README, LICENSE, COC, CONTRIBUTING)
/docs # PRD.md, ARCHITECTURE.md /docs # PRD.md, ARCHITECTURE.md
/backend # FastAPI service (uv-managed). app/{api/v1, services (+ privacy engine), repositories, models, schemas, integrations (auth, mailer, objectstore, models = pluggable LLM/embedding providers), core}; migrations/ = Alembic
/deploy # docker-compose.yml (+ docker-compose.dev.yml), Caddyfile, .env.example, backup.sh + BACKUP.md (one-command pg_dump + MinIO backup) — the self-host stack
/.gitea/workflows # Gitea Actions CI (build images → Gitea registry)
/frontend # Next.js (App Router, TS, Tailwind, shadcn-style UI). app/ pages, lib/api generated OpenAPI client, components/ui
``` ```
Code does not exist yet — Phase 0 has not landed. When you scaffold it, propose a layout (e.g. `/backend`, `/frontend`, `/deploy` for compose/Caddy) and record it here and in ARCHITECTURE.md. Keep this section current as the tree grows. Phase 0 landed **deploy-first**: the compose stack (Postgres + MinIO + Caddy + FastAPI backend) and CI before the data model and frontend. Backend deps use **uv**; migrations use **Alembic**. Status (keep current as the tree grows):
- **Phase 0 — Foundation: complete** and running live (core data model, local auth behind `AuthProvider`, Next.js frontend).
- **Phase 1 — Core tree: complete.** Media (upload/serve), soft-delete + recovery UI, full CRUD across entities, and the 4-level tree visibility/privacy model (#41#51).
- **Phase 2 — substantially landed.** GEDCOM import (preview→apply, duplicate-aware) and export (citation-preserving, #232); fuzzy name search (pg_trgm) + the public `/explore` directory. Living-person protection is still hardening.
- **Phase 4 — AI assistant foundations landed.** Pluggable `LLMProvider`/`EmbeddingProvider` abstraction + multi-provider registry (Anthropic/OpenAI/xAI/Ollama, #235/#237), the **ChangeProposal** propose-then-confirm flow (#236), and per-tree AI model policy (#238). The assistant's *tool surface that emits proposals* is the remaining piece.
- Also shipped: tree membership management (#233), an **instance owner/operator** role (`OWNER_EMAIL`, #240), a schema-drift readiness guard (#239), and a one-command operator backup (#234).
- **Not built yet:** Phase 3 (Property — parcels/deeds/chain-of-title; no property models exist), Phase 5 (OIDC/social auth — only the `AuthProvider` ABC exists), and cross-tree hints (last; needs multiple populated trees + the embedding provider).
## Where to start ## Where to start
The roadmap is phased in PRD §8. Build in dependency order. **Phase 0 — Foundation** is the current target: The roadmap is phased in PRD §8. Build in dependency order. **Phases 0 and 1 are complete**, Phase 2 is substantially done, and Phase 4's AI foundations have shipped (see the status list above). The biggest unbuilt areas are **Phase 3 (Property)** and **Phase 5 (OIDC/social auth)** — likely current targets. For reference, Phase 0 covered:
1. Backend skeleton (FastAPI, async, layered) + Postgres + migrations 1. Backend skeleton (FastAPI, async, layered) + Postgres + migrations
2. Core data model from ARCHITECTURE §5 — start with User, Tree, TreeMembership, Person, Name, Relationship, Event, Place, Source, Citation, AuditEntry, soft-delete support 2. Core data model from ARCHITECTURE §5 — start with User, Tree, TreeMembership, Person, Name, Relationship, Event, Place, Source, Citation, AuditEntry, soft-delete support
@@ -53,7 +66,7 @@ The roadmap is phased in PRD §8. Build in dependency order. **Phase 0 — Found
5. The deploy stack: `compose` for app + postgres + objectstore, Caddy config, env-driven settings 5. The deploy stack: `compose` for app + postgres + objectstore, Caddy config, env-driven settings
6. CI/CD: Gitea Actions building images to the registry 6. CI/CD: Gitea Actions building images to the registry
Don't get ahead of the phases. GEDCOM lands before the assistant (so AI writes target a stable model); property follows a tested people graph; hints come last because they need multiple populated trees. If you think the order is wrong, raise it rather than reordering silently. Don't get ahead of the phases. GEDCOM and the assistant's propose-diff foundation (provider abstraction + ChangeProposal approval flow) have shipped; the remaining dependency-ordered work is **Property** (Phase 3, on top of the tested people graph), then richer collaboration/audit UI, with **cross-tree hints last** (they need multiple populated trees and the embedding provider). If you think the order is wrong, raise it rather than reordering silently.
## Conventions ## Conventions
@@ -64,6 +77,23 @@ Don't get ahead of the phases. GEDCOM lands before the assistant (so AI writes t
- **Privacy/assistant/hint code gets extra care** — these are the areas where bugs do real harm. Prefer a design note before a large change. - **Privacy/assistant/hint code gets extra care** — these are the areas where bugs do real harm. Prefer a design note before a large change.
- **No secrets in the repo.** Config via env; provide `.env.example` with placeholders. - **No secrets in the repo.** Config via env; provide `.env.example` with placeholders.
## Patched dependencies (family-chart)
The tree view uses **family-chart** (d3-based). Two adjustments live in the repo:
- **CSS is vendored** at `frontend/app/trees/[id]/tree/chart.css` — the package blocks its CSS subpath export, so we copy it in.
- **The library is patched** via `patch-package` (`frontend/patches/family-chart+0.9.0.patch`, applied by the `postinstall` hook; the backend/frontend Dockerfiles `COPY patches` before install). Both hunks touch `dist/family-chart.js` **and** `dist/family-chart.esm.js` (the app loads the `esm` build). Current fixes:
1. **Spouse-centering layout** (`setupSpouses` / `sortChildrenWithSpouses`) — center a person between two spouses with children under the correct pair.
2. **`cardToMiddle` vertical centering** — the lib scaled `datum.x` by the zoom factor `k` but not `datum.y`, so "fly to a node" drifted vertically at any zoom ≠ 1; we add the missing `* k`.
To change a patch: edit the file(s) under `node_modules/family-chart/dist/`, then `cd frontend && npx patch-package family-chart` to regenerate, and verify with `npx patch-package --error-on-fail`.
**Upstreamed.** Both are general library bugfixes, not app-specific, and are submitted upstream:
- `cardToMiddle` vertical centering — **donatso/family-chart#103** (issue **#102**).
- Multi-spouse centered layout — **donatso/family-chart#105** (issue **#104**).
If either is merged + released, bump `family-chart`, drop the corresponding patch hunk, **and** remove any in-app compensation (e.g. the `cardToMiddle` caller in `tree/page.tsx` passes raw `y` precisely because the patch fixes it — pre-scaling there too would double-correct). Until then, keep the patch.
## License & contribution terms ## License & contribution terms
Provenance is **source-available** under **BUSL-1.1** (see [LICENSE](LICENSE)): free for personal/family/non-commercial use, no third-party commercial hosting, and each release converts to **AGPL-3.0** four years after it ships. The DCO sign-off keeps the licensing chain clean so the maintainer can manage that conversion and a possible future hosted offering. Don't add code under an incompatible license, and don't vendor dependencies whose licenses conflict with eventual AGPL distribution. Provenance is **source-available** under **BUSL-1.1** (see [LICENSE](LICENSE)): free for personal/family/non-commercial use, no third-party commercial hosting, and each release converts to **AGPL-3.0** four years after it ships. The DCO sign-off keeps the licensing chain clean so the maintainer can manage that conversion and a possible future hosted offering. Don't add code under an incompatible license, and don't vendor dependencies whose licenses conflict with eventual AGPL distribution.
+5 -4
View File
@@ -19,13 +19,14 @@ Every fact links to its source. Every claim can be traced. Nothing is just asser
## What it does ## What it does
- **Build a tree that holds up.** People, relationships, events, and places — with every fact linked to the document, photo, or record it came from. - **Build a tree that holds up.** People, relationships, events, and places — with every fact linked to the document, photo, or record it came from.
- **Trace the land, not just the family.** Properties are first-class. Record ownership events (grants, deeds, inheritances, sales), reconstruct chain-of-title, and tie parcels to the people who held them.
- **Bring your own archive.** Scans, PDFs, photos, audio recordings — first-class citizens, not afterthoughts. - **Bring your own archive.** Scans, PDFs, photos, audio recordings — first-class citizens, not afterthoughts.
- **A research assistant that proposes, never overwrites.** The built-in AI assistant searches legal sources, lays out what it found, and waits for your approval before anything touches your data. You can point it at the major model providers or a self-hosted model — your keys, your choice. - **A research assistant that proposes, never overwrites.** The built-in AI assistant searches legal sources, lays out what it found, and waits for your approval before anything touches your data. You can point it at the major model providers or a self-hosted model — your keys, your choice.
- **Standards over silos.** Full GEDCOM 7 import and export. Migrate in, migrate out. - **Standards over silos.** GEDCOM import and export (5.5.1 / 7 common subset) — duplicate-aware import, citation-preserving export. Migrate in, migrate out.
- **Privacy you control.** Public, unlisted, or private per tree; any individual can be hidden; living people are protected by default. - **Privacy you control.** Public, members-only (any signed-in user on your instance), unlisted, or private per tree; any individual can be hidden; living people are protected by default.
- **Find your people.** When another user's tree overlaps with yours, Provenance can surface an anonymous "possible match" — and only connects you if you both say yes. - **Find your people.** When another user's tree overlaps with yours, Provenance can surface an anonymous "possible match" — and only connects you if you both say yes.
- **Run it your way.** Container-native. Self-host behind Caddy and, if you like, a Cloudflare Tunnel. Multi-tenant, so your whole extended family — or a whole community of strangers — can coexist on one deployment. - **Run it your way.** Container-native. Self-host behind Caddy and, if you like, a Cloudflare Tunnel. Multi-tenant, so your whole extended family — or a whole community of strangers — can coexist on one deployment. One-command backups (Postgres + object storage) and an instance-owner admin role keep operations in your hands.
**Where it's headed — trace the land, not just the family.** The same source-backed treatment for *property*: parcels, deeds, and ownership events, reconstructing chain-of-title and tying land to the people who held it. The people side ships today; the land half is on the roadmap, not yet built — but it's why Provenance exists, not an afterthought.
## Who it's for ## Who it's for
+10
View File
@@ -0,0 +1,10 @@
.venv/
venv/
__pycache__/
*.py[cod]
.pytest_cache/
.ruff_cache/
.env
.env.*
!.env.example
*.md
+31
View File
@@ -0,0 +1,31 @@
# syntax=docker/dockerfile:1
# uv-managed Python image keeps the toolchain reproducible. Pinned to 3.13 for
# broad wheel availability (asyncpg etc.); bump when 3.14 wheels are ubiquitous.
FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
UV_COMPILE_BYTECODE=1 \
UV_LINK_MODE=copy
WORKDIR /app
# Dependencies first for layer caching. uv.lock is optional on first build;
# `uv sync` resolves and writes it if absent.
COPY pyproject.toml uv.lock* ./
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --no-dev
# Application source + migrations (project is package=false, no install step).
COPY app ./app
COPY alembic.ini ./alembic.ini
COPY migrations ./migrations
COPY docker-entrypoint.sh ./docker-entrypoint.sh
RUN chmod +x ./docker-entrypoint.sh
EXPOSE 8000
# The entrypoint runs migrations first when RUN_MIGRATIONS=1, then the command.
ENTRYPOINT ["./docker-entrypoint.sh"]
CMD ["uv", "run", "--no-dev", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
+41
View File
@@ -0,0 +1,41 @@
# Alembic config. The database URL is injected from DATABASE_URL in
# migrations/env.py (twelve-factor) — intentionally not set here.
[alembic]
script_location = migrations
prepend_sys_path = .
path_separator = os
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console
qualname =
[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
View File
View File
+182
View File
@@ -0,0 +1,182 @@
"""Shared API dependencies: DB session, the authenticated user, and the mailer."""
from typing import Annotated
from fastapi import Depends, HTTPException, Request, status
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings
from app.core.db import get_session
from app.integrations.mailer.base import Mailer
from app.integrations.mailer.console import ConsoleMailer
from app.integrations.mailer.smtp import SMTPMailer
from app.integrations.models.base import EmbeddingProvider, LLMProvider
from app.integrations.models.null import NullEmbeddingProvider, NullLLMProvider
from app.integrations.objectstore.base import ObjectStore
from app.integrations.objectstore.s3 import S3ObjectStore
from app.models.user import User
from app.services import auth_service
SessionDep = Annotated[AsyncSession, Depends(get_session)]
def extract_session_token(request: Request) -> str | None:
"""Bearer header (API clients) takes precedence over the session cookie
(browser)."""
authorization = request.headers.get("authorization")
if authorization and authorization.lower().startswith("bearer "):
return authorization[7:].strip()
return request.cookies.get(get_settings().cookie_name)
async def get_current_user(request: Request, session: SessionDep) -> User:
raw_token = extract_session_token(request)
if raw_token is None:
raise HTTPException(status.HTTP_401_UNAUTHORIZED, "authentication required")
user = await auth_service.resolve_session_user(session, raw_token=raw_token)
if user is None:
raise HTTPException(status.HTTP_401_UNAUTHORIZED, "invalid or expired session")
return user
CurrentUser = Annotated[User, Depends(get_current_user)]
async def get_current_user_or_none(request: Request, session: SessionDep) -> User | None:
"""Optional auth for public read endpoints — never raises. Returns the user
when a valid session is present, else None (anonymous viewer)."""
raw_token = extract_session_token(request)
if raw_token is None:
return None
return await auth_service.resolve_session_user(session, raw_token=raw_token)
CurrentUserOrNone = Annotated[User | None, Depends(get_current_user_or_none)]
def is_instance_owner(user: User) -> bool:
"""Whether this account is an instance owner/operator — i.e. its email is
named in OWNER_EMAIL *and* that email has been verified. Instance ownership
is an operational/config role; it does NOT bypass the privacy engine or grant
access to others' tree data.
The verified-email requirement is load-bearing: registration is open and (by
default) doesn't require verification, so without it an attacker could claim
the owner email by registering it before the operator does — a land-grab to
the highest role with no proof of inbox control. Requiring verification ties
ownership to actual control of the named inbox regardless of the global
REQUIRE_EMAIL_VERIFICATION setting. (Self-hosts without SMTP can verify via
the link the console mailer prints to the operator-controlled logs.)"""
owners = get_settings().owner_emails()
return (
bool(owners)
and user.email_verified_at is not None
and user.email.strip().lower() in owners
)
async def require_instance_owner(current: CurrentUser) -> User:
if not is_instance_owner(current):
raise HTTPException(status.HTTP_403_FORBIDDEN, "instance owner only")
return current
InstanceOwner = Annotated[User, Depends(require_instance_owner)]
def get_mailer() -> Mailer:
settings = get_settings()
if settings.mailer == "smtp" and settings.smtp_host:
return SMTPMailer(settings)
return ConsoleMailer()
MailerDep = Annotated[Mailer, Depends(get_mailer)]
def get_objectstore() -> ObjectStore:
return S3ObjectStore(get_settings())
ObjectStoreDep = Annotated[ObjectStore, Depends(get_objectstore)]
def build_llm_providers() -> dict[str, LLMProvider]:
"""Every LLM provider whose credentials are configured, keyed by name. Run
several at once; pick one with get_llm_provider(name)."""
from app.integrations.models.anthropic_provider import AnthropicLLMProvider
from app.integrations.models.openai_compat import OpenAICompatibleLLMProvider
s = get_settings()
providers: dict[str, LLMProvider] = {}
if s.anthropic_api_key:
providers["anthropic"] = AnthropicLLMProvider(
api_key=s.anthropic_api_key, model=s.anthropic_model, max_tokens=s.llm_max_tokens
)
if s.openai_api_key:
providers["openai"] = OpenAICompatibleLLMProvider(
api_key=s.openai_api_key, base_url=s.openai_base_url, model=s.openai_model,
max_tokens=s.llm_max_tokens,
)
if s.xai_api_key:
providers["xai"] = OpenAICompatibleLLMProvider(
api_key=s.xai_api_key, base_url=s.xai_base_url, model=s.xai_model,
max_tokens=s.llm_max_tokens,
)
if s.ollama_enabled:
providers["ollama"] = OpenAICompatibleLLMProvider(
api_key=None, base_url=s.ollama_base_url, model=s.ollama_model,
max_tokens=s.llm_max_tokens,
)
return providers
def configured_llm_providers() -> list[dict]:
"""Configured LLM providers as {name, model} — for the AI admin view (no
secrets). Mirrors build_llm_providers() without constructing clients."""
s = get_settings()
out: list[dict] = []
if s.anthropic_api_key:
out.append({"name": "anthropic", "model": s.anthropic_model})
if s.openai_api_key:
out.append({"name": "openai", "model": s.openai_model})
if s.xai_api_key:
out.append({"name": "xai", "model": s.xai_model})
if s.ollama_enabled:
out.append({"name": "ollama", "model": s.ollama_model})
return out
def get_llm_provider(name: str | None = None) -> LLMProvider:
"""The named LLM provider, or the configured default, or Null if unconfigured."""
providers = build_llm_providers()
return providers.get(name or get_settings().default_llm_provider) or NullLLMProvider()
LLMProviderDep = Annotated[LLMProvider, Depends(get_llm_provider)]
def build_embedding_providers() -> dict[str, EmbeddingProvider]:
from app.integrations.models.openai_compat import OpenAICompatibleEmbeddingProvider
s = get_settings()
providers: dict[str, EmbeddingProvider] = {}
if s.openai_api_key:
providers["openai"] = OpenAICompatibleEmbeddingProvider(
api_key=s.openai_api_key, base_url=s.openai_base_url,
model=s.openai_embedding_model, dimensions=s.embedding_dimensions,
)
if s.ollama_enabled:
providers["ollama"] = OpenAICompatibleEmbeddingProvider(
api_key=None, base_url=s.ollama_base_url,
model=s.ollama_embedding_model, dimensions=s.embedding_dimensions,
)
return providers
def get_embedding_provider(name: str | None = None) -> EmbeddingProvider:
providers = build_embedding_providers()
return providers.get(name or get_settings().default_embedding_provider) or NullEmbeddingProvider()
EmbeddingProviderDep = Annotated[EmbeddingProvider, Depends(get_embedding_provider)]
+53
View File
@@ -0,0 +1,53 @@
"""Liveness and readiness endpoints.
- ``/health`` — liveness: the process is up. No dependencies touched.
- ``/health/ready`` — readiness: dependencies (Postgres) are reachable.
Orchestrators and Caddy probe these; they are intentionally outside the
versioned ``/api`` surface.
"""
from fastapi import APIRouter, Response, status
from sqlalchemy import text
from app.core.config import get_settings
from app.core.db import get_engine
from app.core.schema_version import schema_is_current
router = APIRouter(tags=["health"])
@router.get("/health")
async def health() -> dict:
settings = get_settings()
return {
"status": "ok",
"service": settings.app_name,
"version": settings.version,
"env": settings.app_env,
}
@router.get("/health/ready")
async def ready(response: Response) -> dict:
checks: dict[str, str] = {}
try:
async with get_engine().connect() as conn:
await conn.execute(text("SELECT 1"))
checks["database"] = "ok"
# Schema drift = code ahead of the DB; queries would 500. Fail
# readiness loudly rather than serve a broken surface.
ok, db, expected = await schema_is_current(conn)
if not ok:
checks["schema"] = (
f"drift: db={sorted(db) or ['none']} expected={sorted(expected)} "
"— run 'alembic upgrade head'"
)
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
return {"status": "not ready", "checks": checks}
checks["schema"] = "ok"
return {"status": "ready", "checks": checks}
except Exception as exc: # noqa: BLE001 — surface any failure as "not ready"
checks.setdefault("database", "error")
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
return {"status": "not ready", "checks": checks, "detail": str(exc)}
+42
View File
@@ -0,0 +1,42 @@
"""Versioned API surface. Mounts under /api/v1."""
from fastapi import APIRouter
from app.api.v1 import (
admin,
ai,
auth,
citations,
cleanup,
events,
gedcom,
media,
members,
names,
persons,
proposals,
public,
relationships,
sources,
trees,
users,
)
api_router = APIRouter(prefix="/api/v1")
api_router.include_router(auth.router)
api_router.include_router(users.router)
api_router.include_router(trees.router)
api_router.include_router(persons.router)
api_router.include_router(names.router)
api_router.include_router(events.router)
api_router.include_router(relationships.router)
api_router.include_router(sources.router)
api_router.include_router(citations.router)
api_router.include_router(media.router)
api_router.include_router(gedcom.router)
api_router.include_router(cleanup.router)
api_router.include_router(public.router)
api_router.include_router(members.router)
api_router.include_router(proposals.router)
api_router.include_router(ai.router)
api_router.include_router(admin.router)
+38
View File
@@ -0,0 +1,38 @@
"""Instance-admin surface — owner-only (OWNER_EMAIL). Operational status and
instance-wide configuration. Deliberately exposes no tree contents or PII:
instance ownership is an operator role, not a privacy bypass."""
from sqlalchemy import func, select
from fastapi import APIRouter
from app.api.deps import InstanceOwner, SessionDep, configured_llm_providers
from app.core.config import get_settings
from app.models.tree import Tree
from app.models.user import User
from app.schemas.admin import InstanceStatus
from app.schemas.ai_policy import ConfiguredProvider
router = APIRouter(prefix="/admin", tags=["admin"])
@router.get("/instance", response_model=InstanceStatus)
async def instance_status(owner: InstanceOwner, session: SessionDep) -> InstanceStatus:
"""Operator dashboard data. Requires the caller to be an instance owner."""
s = get_settings()
user_count = await session.scalar(
select(func.count()).select_from(User).where(User.deleted_at.is_(None))
)
tree_count = await session.scalar(
select(func.count()).select_from(Tree).where(Tree.deleted_at.is_(None))
)
return InstanceStatus(
version=s.version,
env=s.app_env,
owner_emails=sorted(s.owner_emails()),
require_email_verification=s.require_email_verification,
user_count=user_count or 0,
tree_count=tree_count or 0,
default_llm_provider=s.default_llm_provider,
ai_providers=[ConfiguredProvider(**p) for p in configured_llm_providers()],
)
+34
View File
@@ -0,0 +1,34 @@
"""Per-tree AI model policy — owner-only admin view."""
import uuid
from fastapi import APIRouter
from app.api.deps import CurrentUser, SessionDep
from app.schemas.ai_policy import TreeAiPolicyRead, TreeAiPolicyUpdate
from app.services import ai_policy_service, tree_service
router = APIRouter(prefix="/trees", tags=["ai"])
@router.get("/{tree_id}/ai", response_model=TreeAiPolicyRead)
async def get_ai_policy(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> TreeAiPolicyRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
return TreeAiPolicyRead(**await ai_policy_service.get_policy(session, actor=current, tree=tree))
@router.patch("/{tree_id}/ai", response_model=TreeAiPolicyRead)
async def update_ai_policy(
tree_id: uuid.UUID, data: TreeAiPolicyUpdate, session: SessionDep, current: CurrentUser
) -> TreeAiPolicyRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
policy = await ai_policy_service.update_policy(
session,
actor=current,
tree=tree,
member_provider=data.member_provider,
recommender_provider=data.recommender_provider,
)
return TreeAiPolicyRead(**policy)
+94
View File
@@ -0,0 +1,94 @@
from fastapi import APIRouter, HTTPException, Request, Response, status
from app.api.deps import CurrentUser, MailerDep, SessionDep, extract_session_token
from app.core.config import get_settings
from app.schemas.auth import (
LoginRequest,
PasswordChange,
PasswordResetConfirm,
PasswordResetRequest,
RegisterRequest,
SessionRead,
TokenRequest,
)
from app.schemas.user import UserRead
from app.services import auth_service
router = APIRouter(prefix="/auth", tags=["auth"])
def _set_session_cookie(response: Response, token: str) -> None:
settings = get_settings()
response.set_cookie(
settings.cookie_name,
token,
max_age=settings.session_ttl_days * 86400,
httponly=True,
secure=settings.cookie_secure,
samesite="lax",
)
@router.post("/register", response_model=SessionRead, status_code=status.HTTP_201_CREATED)
async def register(
data: RegisterRequest, session: SessionDep, mailer: MailerDep, response: Response
) -> SessionRead:
user, token, expires_at = await auth_service.register(
session,
mailer,
email=data.email,
password=data.password,
display_name=data.display_name,
)
_set_session_cookie(response, token)
return SessionRead(user=UserRead.model_validate(user), token=token, expires_at=expires_at)
@router.post("/login", response_model=SessionRead)
async def login(data: LoginRequest, session: SessionDep, response: Response) -> SessionRead:
result = await auth_service.login(session, email=data.email, password=data.password)
if result is None:
raise HTTPException(status.HTTP_401_UNAUTHORIZED, "invalid credentials")
user, token, expires_at = result
_set_session_cookie(response, token)
return SessionRead(user=UserRead.model_validate(user), token=token, expires_at=expires_at)
@router.post("/logout", status_code=status.HTTP_204_NO_CONTENT)
async def logout(request: Request, session: SessionDep, response: Response) -> None:
raw_token = extract_session_token(request)
if raw_token:
await auth_service.logout(session, raw_token=raw_token)
response.delete_cookie(get_settings().cookie_name)
@router.post("/verify-email", status_code=status.HTTP_204_NO_CONTENT)
async def verify_email(data: TokenRequest, session: SessionDep) -> None:
await auth_service.verify_email(session, raw_token=data.token)
@router.post("/request-password-reset", status_code=status.HTTP_202_ACCEPTED)
async def request_password_reset(
data: PasswordResetRequest, session: SessionDep, mailer: MailerDep
) -> dict:
await auth_service.request_password_reset(session, mailer, email=data.email)
return {"status": "accepted"}
@router.post("/reset-password", status_code=status.HTTP_204_NO_CONTENT)
async def reset_password(data: PasswordResetConfirm, session: SessionDep) -> None:
await auth_service.reset_password(
session, raw_token=data.token, new_password=data.new_password
)
@router.post("/change-password", status_code=status.HTTP_204_NO_CONTENT)
async def change_password(
data: PasswordChange, session: SessionDep, current: CurrentUser
) -> None:
await auth_service.change_password(
session,
user=current,
current_password=data.current_password,
new_password=data.new_password,
)
+60
View File
@@ -0,0 +1,60 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.source import CitationCreate, CitationRead, CitationUpdate
from app.services import citation_service, tree_service
router = APIRouter(prefix="/trees", tags=["citations"])
@router.post(
"/{tree_id}/citations", response_model=CitationRead, status_code=status.HTTP_201_CREATED
)
async def create_citation(
tree_id: uuid.UUID, data: CitationCreate, session: SessionDep, current: CurrentUser
) -> CitationRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
citation = await citation_service.create_citation(
session, actor=current, tree=tree, **data.model_dump()
)
return CitationRead.model_validate(citation)
@router.get("/{tree_id}/citations", response_model=list[CitationRead])
async def list_citations(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[CitationRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
citations = await citation_service.list_citations(session, viewer_id=current.id, tree=tree)
return [CitationRead.model_validate(c) for c in citations]
@router.patch("/{tree_id}/citations/{citation_id}", response_model=CitationRead)
async def update_citation(
tree_id: uuid.UUID,
citation_id: uuid.UUID,
data: CitationUpdate,
session: SessionDep,
current: CurrentUser,
) -> CitationRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
citation = await citation_service.update_citation(
session,
actor=current,
tree=tree,
citation_id=citation_id,
changes=data.model_dump(exclude_unset=True),
)
return CitationRead.model_validate(citation)
@router.delete("/{tree_id}/citations/{citation_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_citation(
tree_id: uuid.UUID, citation_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await citation_service.delete_citation(
session, actor=current, tree=tree, citation_id=citation_id
)
+130
View File
@@ -0,0 +1,130 @@
import uuid
from fastapi import APIRouter, File, UploadFile
from app.api.deps import CurrentUser, SessionDep
from app.schemas.cleanup import (
CleanupResult,
DeceasedApply,
DeceasedByChildCandidate,
DeceasedCandidate,
GenderApply,
GenderProposal,
NameApply,
NameIssue,
)
from app.services import cleanup_service, tree_service
router = APIRouter(prefix="/trees", tags=["cleanup"])
@router.get("/{tree_id}/cleanup/deceased", response_model=list[DeceasedCandidate])
async def preview_deceased(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
born_on_or_before: int = 1930,
) -> list[DeceasedCandidate]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.preview_deceased(
session, actor=current, tree=tree, year=born_on_or_before
)
return [DeceasedCandidate(**r) for r in rows]
@router.get(
"/{tree_id}/cleanup/deceased-by-child", response_model=list[DeceasedByChildCandidate]
)
async def preview_deceased_by_child(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
born_on_or_before: int = 1900,
) -> list[DeceasedByChildCandidate]:
"""People with a child born on/before the cutoff — necessarily deceased even
when their own birth date is missing. Apply via POST .../cleanup/deceased."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.preview_deceased_by_child(
session, actor=current, tree=tree, year=born_on_or_before
)
return [DeceasedByChildCandidate(**r) for r in rows]
@router.post("/{tree_id}/cleanup/deceased", response_model=CleanupResult)
async def apply_deceased(
tree_id: uuid.UUID, data: DeceasedApply, session: SessionDep, current: CurrentUser
) -> CleanupResult:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
n = await cleanup_service.apply_deceased(
session, actor=current, tree=tree, person_ids=data.person_ids
)
return CleanupResult(updated=n)
@router.post("/{tree_id}/cleanup/gender/preview", response_model=list[GenderProposal])
async def preview_gender(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
file: UploadFile = File(...),
) -> list[GenderProposal]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
text = (await file.read()).decode("utf-8", errors="replace")
rows = await cleanup_service.preview_gender(
session, actor=current, tree=tree, gedcom_text=text
)
return [GenderProposal(**r) for r in rows]
@router.get("/{tree_id}/cleanup/gender/guess", response_model=list[GenderProposal])
async def guess_gender(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[GenderProposal]:
"""Best-guess sex from first names (bundled dictionary) for people missing it."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.guess_gender_by_name(session, actor=current, tree=tree)
return [GenderProposal(**r) for r in rows]
@router.get("/{tree_id}/cleanup/gender/from-spouse", response_model=list[GenderProposal])
async def guess_gender_from_spouse(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[GenderProposal]:
"""Infer a missing sex from a partner whose sex is set (opposite-sex couple)."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.guess_gender_by_spouse(session, actor=current, tree=tree)
return [GenderProposal(**r) for r in rows]
@router.post("/{tree_id}/cleanup/gender", response_model=CleanupResult)
async def apply_gender(
tree_id: uuid.UUID, data: GenderApply, session: SessionDep, current: CurrentUser
) -> CleanupResult:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
n = await cleanup_service.apply_gender(
session,
actor=current,
tree=tree,
updates=[u.model_dump() for u in data.updates],
)
return CleanupResult(updated=n)
@router.get("/{tree_id}/cleanup/names", response_model=list[NameIssue])
async def preview_names(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[NameIssue]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.preview_names(session, actor=current, tree=tree)
return [NameIssue(**r) for r in rows]
@router.post("/{tree_id}/cleanup/names", response_model=CleanupResult)
async def apply_names(
tree_id: uuid.UUID, data: NameApply, session: SessionDep, current: CurrentUser
) -> CleanupResult:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
n = await cleanup_service.apply_names(
session, actor=current, tree=tree, edits=[e.model_dump() for e in data.edits]
)
return CleanupResult(updated=n)
+67
View File
@@ -0,0 +1,67 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.event import EventCreate, EventRead, EventUpdate
from app.services import event_service, tree_service
router = APIRouter(prefix="/trees", tags=["events"])
@router.post("/{tree_id}/events", response_model=EventRead, status_code=status.HTTP_201_CREATED)
async def create_event(
tree_id: uuid.UUID, data: EventCreate, session: SessionDep, current: CurrentUser
) -> EventRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
event = await event_service.create_event(
session, actor=current, tree=tree, **data.model_dump()
)
return EventRead.model_validate(event)
@router.get("/{tree_id}/events", response_model=list[EventRead])
async def list_tree_events(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[EventRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
events = await event_service.list_events(session, viewer_id=current.id, tree=tree)
return [EventRead.model_validate(e) for e in events]
@router.get("/{tree_id}/persons/{person_id}/events", response_model=list[EventRead])
async def list_person_events(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[EventRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
events = await event_service.list_events_for_person(
session, viewer_id=current.id, tree=tree, person_id=person_id
)
return [EventRead.model_validate(e) for e in events]
@router.patch("/{tree_id}/events/{event_id}", response_model=EventRead)
async def update_event(
tree_id: uuid.UUID,
event_id: uuid.UUID,
data: EventUpdate,
session: SessionDep,
current: CurrentUser,
) -> EventRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
event = await event_service.update_event(
session,
actor=current,
tree=tree,
event_id=event_id,
changes=data.model_dump(exclude_unset=True),
)
return EventRead.model_validate(event)
@router.delete("/{tree_id}/events/{event_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_event(
tree_id: uuid.UUID, event_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await event_service.delete_event(session, actor=current, tree=tree, event_id=event_id)
+68
View File
@@ -0,0 +1,68 @@
import json
import uuid
from fastapi import APIRouter, File, Form, Response, UploadFile
from app.api.deps import CurrentUser, SessionDep
from app.schemas.gedcom import ImportPreview, ImportReport
from app.services import gedcom, tree_service
router = APIRouter(prefix="/trees", tags=["gedcom"])
@router.post("/{tree_id}/gedcom/preview", response_model=ImportPreview)
async def preview_gedcom(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
file: UploadFile = File(...),
) -> ImportPreview:
"""Dry run: report counts and incoming people that look like duplicates of
existing ones, so the user can choose how to resolve each before importing."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
text = (await file.read()).decode("utf-8", errors="replace")
report = await gedcom.preview_gedcom(session, actor=current, tree=tree, text=text)
return ImportPreview(**report)
@router.post("/{tree_id}/gedcom/import", response_model=ImportReport)
async def import_gedcom(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
file: UploadFile = File(...),
default_action: str = Form("new"),
resolutions: str = Form("{}"),
) -> ImportReport:
"""Import a GEDCOM. ``default_action`` (new|skip|merge|overwrite) applies to
incoming people that match an existing one; ``resolutions`` is a JSON object
{xref: {action, target_id}} overriding it per record."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
text = (await file.read()).decode("utf-8", errors="replace")
try:
parsed = json.loads(resolutions or "{}")
except json.JSONDecodeError:
parsed = {}
report = await gedcom.import_gedcom(
session,
actor=current,
tree=tree,
text=text,
default_action=default_action,
resolutions=parsed,
)
return ImportReport(**report)
@router.get("/{tree_id}/gedcom/export")
async def export_gedcom(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> Response:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
text = await gedcom.export_gedcom(session, viewer_id=current.id, tree=tree)
safe = "".join(c for c in tree.name if c.isalnum() or c in " -_").strip() or "tree"
return Response(
content=text,
media_type="text/plain",
headers={"Content-Disposition": f'attachment; filename="{safe}.ged"'},
)
+109
View File
@@ -0,0 +1,109 @@
import uuid
from fastapi import APIRouter, File, Form, Response, UploadFile, status
from app.api.deps import CurrentUser, ObjectStoreDep, SessionDep
from app.schemas.media import MediaRead, MediaUpdate
from app.services import media_service, tree_service
def _content_url(media) -> str:
return f"/api/v1/trees/{media.tree_id}/media/{media.id}/content"
def _read(media) -> MediaRead:
out = MediaRead.model_validate(media)
# Stream through the backend (privacy-checked, browser-reachable) rather
# than expose the internal object store directly.
out.url = _content_url(media)
return out
router = APIRouter(prefix="/trees", tags=["media"])
@router.post("/{tree_id}/media", response_model=MediaRead, status_code=status.HTTP_201_CREATED)
async def upload_media(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
file: UploadFile = File(...),
title: str | None = Form(None),
person_id: uuid.UUID | None = Form(None),
event_id: uuid.UUID | None = Form(None),
source_id: uuid.UUID | None = Form(None),
) -> MediaRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
data = await file.read()
media = await media_service.upload_media(
session,
store,
actor=current,
tree=tree,
data=data,
filename=file.filename or "upload",
content_type=file.content_type or "application/octet-stream",
title=title,
person_id=person_id,
event_id=event_id,
source_id=source_id,
)
return _read(media)
@router.get("/{tree_id}/media", response_model=list[MediaRead])
async def list_media(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[MediaRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
items = await media_service.list_media(session, viewer_id=current.id, tree=tree)
return [_read(m) for m in items]
@router.get("/{tree_id}/media/{media_id}/content")
async def media_content(
tree_id: uuid.UUID,
media_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
) -> Response:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
media = await media_service.get_media(
session, viewer_id=current.id, tree=tree, media_id=media_id
)
data = await store.get_object(key=media.storage_key)
return Response(
content=data,
media_type=media.content_type,
headers={"Content-Disposition": f'inline; filename="{media.original_filename}"'},
)
@router.patch("/{tree_id}/media/{media_id}", response_model=MediaRead)
async def update_media(
tree_id: uuid.UUID,
media_id: uuid.UUID,
data: MediaUpdate,
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
) -> MediaRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
media = await media_service.update_media(
session,
actor=current,
tree=tree,
media_id=media_id,
changes=data.model_dump(exclude_unset=True),
)
return _read(media)
@router.delete("/{tree_id}/media/{media_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_media(
tree_id: uuid.UUID, media_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await media_service.delete_media(session, actor=current, tree=tree, media_id=media_id)
+61
View File
@@ -0,0 +1,61 @@
"""Tree membership management endpoints (owner-managed; members can list)."""
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.membership import MemberAdd, MemberRoleUpdate, MembershipRead
from app.services import membership_service, tree_service
router = APIRouter(prefix="/trees", tags=["members"])
@router.get("/{tree_id}/members", response_model=list[MembershipRead])
async def list_members(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[MembershipRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await membership_service.list_members(session, viewer_id=current.id, tree=tree)
return [MembershipRead(**r) for r in rows]
@router.post(
"/{tree_id}/members", response_model=MembershipRead, status_code=status.HTTP_201_CREATED
)
async def add_member(
tree_id: uuid.UUID, data: MemberAdd, session: SessionDep, current: CurrentUser
) -> MembershipRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
row = await membership_service.add_member(
session, actor=current, tree=tree, email=data.email, role=data.role
)
return MembershipRead(**row)
@router.patch("/{tree_id}/members/{membership_id}", response_model=MembershipRead)
async def update_member(
tree_id: uuid.UUID,
membership_id: uuid.UUID,
data: MemberRoleUpdate,
session: SessionDep,
current: CurrentUser,
) -> MembershipRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
row = await membership_service.update_member_role(
session, actor=current, tree=tree, membership_id=membership_id, role=data.role
)
return MembershipRead(**row)
@router.delete("/{tree_id}/members/{membership_id}", status_code=status.HTTP_204_NO_CONTENT)
async def remove_member(
tree_id: uuid.UUID,
membership_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await membership_service.remove_member(
session, actor=current, tree=tree, membership_id=membership_id
)
+90
View File
@@ -0,0 +1,90 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.name import NameCreate, NameRead, NameUpdate
from app.services import name_service, tree_service
# Names are nested under their person (which is nested under the tree tenant).
router = APIRouter(prefix="/trees", tags=["names"])
@router.get("/{tree_id}/persons/{person_id}/names", response_model=list[NameRead])
async def list_names(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[NameRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
names = await name_service.list_names(
session, viewer_id=current.id, tree=tree, person_id=person_id
)
return [NameRead.model_validate(n) for n in names]
@router.post(
"/{tree_id}/persons/{person_id}/names",
response_model=NameRead,
status_code=status.HTTP_201_CREATED,
)
async def create_name(
tree_id: uuid.UUID,
person_id: uuid.UUID,
data: NameCreate,
session: SessionDep,
current: CurrentUser,
) -> NameRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
name = await name_service.create_name(
session,
actor=current,
tree=tree,
person_id=person_id,
name_type=data.name_type,
given=data.given,
surname=data.surname,
prefix=data.prefix,
suffix=data.suffix,
nickname=data.nickname,
is_primary=data.is_primary,
)
return NameRead.model_validate(name)
@router.patch(
"/{tree_id}/persons/{person_id}/names/{name_id}", response_model=NameRead
)
async def update_name(
tree_id: uuid.UUID,
person_id: uuid.UUID,
name_id: uuid.UUID,
data: NameUpdate,
session: SessionDep,
current: CurrentUser,
) -> NameRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
name = await name_service.update_name(
session,
actor=current,
tree=tree,
person_id=person_id,
name_id=name_id,
changes=data.model_dump(exclude_unset=True),
)
return NameRead.model_validate(name)
@router.delete(
"/{tree_id}/persons/{person_id}/names/{name_id}",
status_code=status.HTTP_204_NO_CONTENT,
)
async def delete_name(
tree_id: uuid.UUID,
person_id: uuid.UUID,
name_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await name_service.delete_name(
session, actor=current, tree=tree, person_id=person_id, name_id=name_id
)
+123
View File
@@ -0,0 +1,123 @@
import uuid
from fastapi import APIRouter, HTTPException, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.person import PersonCreate, PersonRead, PersonUpdate
from app.services import person_service, tree_service
# Persons are nested under their tree (the tenant boundary).
router = APIRouter(prefix="/trees", tags=["persons"])
@router.post(
"/{tree_id}/persons",
response_model=PersonRead,
status_code=status.HTTP_201_CREATED,
)
async def create_person(
tree_id: uuid.UUID, data: PersonCreate, session: SessionDep, current: CurrentUser
) -> PersonRead:
# get_tree enforces existence + view access; create_person enforces edit rights.
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
person = await person_service.create_person(
session,
actor=current,
tree=tree,
given=data.given,
surname=data.surname,
gender=data.gender,
is_living=data.is_living,
privacy_setting=data.privacy,
notes=data.notes,
)
return PersonRead.model_validate(person)
@router.get("/{tree_id}/persons", response_model=list[PersonRead])
async def list_persons(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
deleted: bool = False,
q: str | None = None,
ids: str | None = None,
) -> list[PersonRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
if ids is not None:
try:
id_list = [uuid.UUID(x) for x in ids.split(",") if x.strip()]
except ValueError as exc:
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "invalid ids") from exc
persons = await person_service.list_persons_by_ids(
session, viewer_id=current.id, tree=tree, ids=id_list
)
elif q:
persons = await person_service.search_persons(
session, viewer_id=current.id, tree=tree, query=q
)
elif deleted:
persons = await person_service.list_deleted_persons(
session, viewer_id=current.id, tree=tree
)
else:
persons = await person_service.list_persons(session, viewer_id=current.id, tree=tree)
return [PersonRead.model_validate(p) for p in persons]
@router.patch("/{tree_id}/persons/{person_id}", response_model=PersonRead)
async def update_person(
tree_id: uuid.UUID,
person_id: uuid.UUID,
data: PersonUpdate,
session: SessionDep,
current: CurrentUser,
) -> PersonRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
person = await person_service.update_person(
session,
actor=current,
tree=tree,
person_id=person_id,
changes=data.model_dump(exclude_unset=True),
)
return PersonRead.model_validate(person)
@router.delete("/{tree_id}/persons/{person_id}")
async def delete_person(
tree_id: uuid.UUID,
person_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
cascade: bool = False,
) -> dict[str, int]:
"""Delete a person. ``cascade=true`` also deletes all descendants. Returns
the number of persons deleted (1 unless cascading)."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
deleted = await person_service.delete_person(
session, actor=current, tree=tree, person_id=person_id, cascade=cascade
)
return {"deleted": deleted}
@router.post("/{tree_id}/persons/{person_id}/restore", response_model=PersonRead)
async def restore_person(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> PersonRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
person = await person_service.restore_person(
session, actor=current, tree=tree, person_id=person_id
)
return PersonRead.model_validate(person)
@router.get("/{tree_id}/persons/{person_id}", response_model=PersonRead)
async def get_person(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> PersonRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
person = await person_service.get_person(
session, viewer_id=current.id, tree=tree, person_id=person_id
)
return PersonRead.model_validate(person)
+116
View File
@@ -0,0 +1,116 @@
"""Change-proposal endpoints: list / create / get / apply / reject / delete.
Applying a proposal is the only way its operations reach the database, and only
an editor can do it (enforced in the service). See docs/design/change-proposal.md.
"""
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.models.enums import ChangeProposalStatus
from app.schemas.change_proposal import (
ChangeProposalCreate,
ChangeProposalRead,
ProposalReview,
)
from app.services import change_proposal_service, tree_service
router = APIRouter(prefix="/trees", tags=["proposals"])
@router.get("/{tree_id}/proposals", response_model=list[ChangeProposalRead])
async def list_proposals(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
status: ChangeProposalStatus | None = None,
) -> list[ChangeProposalRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await change_proposal_service.list_proposals(
session, viewer_id=current.id, tree=tree, status=status
)
return [ChangeProposalRead.model_validate(r) for r in rows]
@router.post(
"/{tree_id}/proposals", response_model=ChangeProposalRead, status_code=status.HTTP_201_CREATED
)
async def create_proposal(
tree_id: uuid.UUID, data: ChangeProposalCreate, session: SessionDep, current: CurrentUser
) -> ChangeProposalRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
operations = [op.model_dump(mode="json") for op in data.operations]
cp = await change_proposal_service.propose(
session,
tree=tree,
origin=data.origin,
created_by=current.id,
summary=data.summary,
rationale=data.rationale,
operations=operations,
)
return ChangeProposalRead.model_validate(cp)
@router.get("/{tree_id}/proposals/{proposal_id}", response_model=ChangeProposalRead)
async def get_proposal(
tree_id: uuid.UUID, proposal_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> ChangeProposalRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
cp = await change_proposal_service.get_proposal(
session, viewer_id=current.id, tree=tree, proposal_id=proposal_id
)
return ChangeProposalRead.model_validate(cp)
@router.post("/{tree_id}/proposals/{proposal_id}/apply", response_model=ChangeProposalRead)
async def apply_proposal(
tree_id: uuid.UUID,
proposal_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
data: ProposalReview | None = None,
) -> ChangeProposalRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
edited = (
[op.model_dump(mode="json") for op in data.operations]
if data and data.operations is not None
else None
)
cp = await change_proposal_service.apply(
session, actor=current, tree=tree, proposal_id=proposal_id, edited_operations=edited
)
return ChangeProposalRead.model_validate(cp)
@router.post("/{tree_id}/proposals/{proposal_id}/reject", response_model=ChangeProposalRead)
async def reject_proposal(
tree_id: uuid.UUID,
proposal_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
data: ProposalReview | None = None,
) -> ChangeProposalRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
cp = await change_proposal_service.reject(
session,
actor=current,
tree=tree,
proposal_id=proposal_id,
note=data.note if data else None,
)
return ChangeProposalRead.model_validate(cp)
@router.delete(
"/{tree_id}/proposals/{proposal_id}", status_code=status.HTTP_204_NO_CONTENT
)
async def delete_proposal(
tree_id: uuid.UUID, proposal_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await change_proposal_service.delete_proposal(
session, actor=current, tree=tree, proposal_id=proposal_id
)
+135
View File
@@ -0,0 +1,135 @@
"""Public, read-only viewing surface.
Optional auth (anonymous allowed). Every response is built by
``public_view_service``, which routes through the privacy engine and redacts
possibly-living people. No create/update/delete here.
"""
import uuid
from fastapi import APIRouter
from app.api.deps import CurrentUserOrNone, SessionDep
from app.schemas.event import EventRead
from app.schemas.name import NameRead
from app.schemas.person import PersonRead
from app.schemas.relationship import RelationshipRead
from app.schemas.tree import PublicTreeRead
from app.services import public_view_service
router = APIRouter(prefix="/public", tags=["public"])
def _vid(viewer: CurrentUserOrNone) -> uuid.UUID | None:
return viewer.id if viewer else None
@router.get("/trees", response_model=list[PublicTreeRead])
async def public_directory(
session: SessionDep,
viewer: CurrentUserOrNone,
q: str | None = None,
limit: int = 50,
offset: int = 0,
) -> list[PublicTreeRead]:
trees = await public_view_service.list_public_trees(
session, viewer_id=_vid(viewer), q=q, limit=limit, offset=offset
)
return [PublicTreeRead.model_validate(t) for t in trees]
@router.get("/trees/{tree_id}", response_model=PublicTreeRead)
async def public_tree(
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
) -> PublicTreeRead:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
return PublicTreeRead.model_validate(tree)
@router.get("/trees/{tree_id}/persons", response_model=list[PersonRead])
async def public_persons(
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
) -> list[PersonRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
persons = await public_view_service.list_public_persons(
session, viewer_id=_vid(viewer), tree=tree
)
return [PersonRead.model_validate(p) for p in persons]
@router.get("/trees/{tree_id}/relationships", response_model=list[RelationshipRead])
async def public_relationships(
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
) -> list[RelationshipRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
rels = await public_view_service.list_public_relationships(
session, viewer_id=_vid(viewer), tree=tree
)
return [RelationshipRead.model_validate(r) for r in rels]
@router.get("/trees/{tree_id}/events", response_model=list[EventRead])
async def public_events(
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
) -> list[EventRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
events = await public_view_service.list_public_events(
session, viewer_id=_vid(viewer), tree=tree
)
return [EventRead.model_validate(e) for e in events]
@router.get("/trees/{tree_id}/persons/{person_id}", response_model=PersonRead)
async def public_person(
tree_id: uuid.UUID,
person_id: uuid.UUID,
session: SessionDep,
viewer: CurrentUserOrNone,
) -> PersonRead:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
person = await public_view_service.get_public_person(
session, viewer_id=_vid(viewer), tree=tree, person_id=person_id
)
return PersonRead.model_validate(person)
@router.get("/trees/{tree_id}/persons/{person_id}/names", response_model=list[NameRead])
async def public_person_names(
tree_id: uuid.UUID,
person_id: uuid.UUID,
session: SessionDep,
viewer: CurrentUserOrNone,
) -> list[NameRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
names = await public_view_service.list_public_person_names(
session, viewer_id=_vid(viewer), tree=tree, person_id=person_id
)
return [NameRead.model_validate(n) for n in names]
@router.get("/trees/{tree_id}/persons/{person_id}/events", response_model=list[EventRead])
async def public_person_events(
tree_id: uuid.UUID,
person_id: uuid.UUID,
session: SessionDep,
viewer: CurrentUserOrNone,
) -> list[EventRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
events = await public_view_service.list_public_person_events(
session, viewer_id=_vid(viewer), tree=tree, person_id=person_id
)
return [EventRead.model_validate(e) for e in events]
+78
View File
@@ -0,0 +1,78 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.relationship import RelationshipCreate, RelationshipRead, RelationshipUpdate
from app.services import relationship_service, tree_service
router = APIRouter(prefix="/trees", tags=["relationships"])
@router.post(
"/{tree_id}/relationships",
response_model=RelationshipRead,
status_code=status.HTTP_201_CREATED,
)
async def create_relationship(
tree_id: uuid.UUID, data: RelationshipCreate, session: SessionDep, current: CurrentUser
) -> RelationshipRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
relationship = await relationship_service.create_relationship(
session, actor=current, tree=tree, **data.model_dump()
)
return RelationshipRead.model_validate(relationship)
@router.get("/{tree_id}/relationships", response_model=list[RelationshipRead])
async def list_relationships(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[RelationshipRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rels = await relationship_service.list_relationships(session, viewer_id=current.id, tree=tree)
return [RelationshipRead.model_validate(r) for r in rels]
@router.get(
"/{tree_id}/persons/{person_id}/relationships",
response_model=list[RelationshipRead],
)
async def list_person_relationships(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[RelationshipRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rels = await relationship_service.list_relationships_for_person(
session, viewer_id=current.id, tree=tree, person_id=person_id
)
return [RelationshipRead.model_validate(r) for r in rels]
@router.patch("/{tree_id}/relationships/{relationship_id}", response_model=RelationshipRead)
async def update_relationship(
tree_id: uuid.UUID,
relationship_id: uuid.UUID,
data: RelationshipUpdate,
session: SessionDep,
current: CurrentUser,
) -> RelationshipRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rel = await relationship_service.update_relationship(
session,
actor=current,
tree=tree,
relationship_id=relationship_id,
changes=data.model_dump(exclude_unset=True),
)
return RelationshipRead.model_validate(rel)
@router.delete(
"/{tree_id}/relationships/{relationship_id}", status_code=status.HTTP_204_NO_CONTENT
)
async def delete_relationship(
tree_id: uuid.UUID, relationship_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await relationship_service.delete_relationship(
session, actor=current, tree=tree, relationship_id=relationship_id
)
+67
View File
@@ -0,0 +1,67 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.source import SourceCreate, SourceRead, SourceUpdate
from app.services import source_service, tree_service
router = APIRouter(prefix="/trees", tags=["sources"])
@router.post("/{tree_id}/sources", response_model=SourceRead, status_code=status.HTTP_201_CREATED)
async def create_source(
tree_id: uuid.UUID, data: SourceCreate, session: SessionDep, current: CurrentUser
) -> SourceRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
source = await source_service.create_source(
session, actor=current, tree=tree, **data.model_dump()
)
return SourceRead.model_validate(source)
@router.get("/{tree_id}/sources", response_model=list[SourceRead])
async def list_sources(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[SourceRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
sources = await source_service.list_sources(session, viewer_id=current.id, tree=tree)
return [SourceRead.model_validate(s) for s in sources]
@router.get("/{tree_id}/sources/{source_id}", response_model=SourceRead)
async def get_source(
tree_id: uuid.UUID, source_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> SourceRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
source = await source_service.get_source(
session, viewer_id=current.id, tree=tree, source_id=source_id
)
return SourceRead.model_validate(source)
@router.patch("/{tree_id}/sources/{source_id}", response_model=SourceRead)
async def update_source(
tree_id: uuid.UUID,
source_id: uuid.UUID,
data: SourceUpdate,
session: SessionDep,
current: CurrentUser,
) -> SourceRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
source = await source_service.update_source(
session,
actor=current,
tree=tree,
source_id=source_id,
changes=data.model_dump(exclude_unset=True),
)
return SourceRead.model_validate(source)
@router.delete("/{tree_id}/sources/{source_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_source(
tree_id: uuid.UUID, source_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await source_service.delete_source(session, actor=current, tree=tree, source_id=source_id)
+74
View File
@@ -0,0 +1,74 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, ObjectStoreDep, SessionDep
from app.schemas.tree import TreeCreate, TreePurge, TreeRead, TreeUpdate
from app.services import tree_service
router = APIRouter(prefix="/trees", tags=["trees"])
@router.post("", response_model=TreeRead, status_code=status.HTTP_201_CREATED)
async def create_tree(data: TreeCreate, session: SessionDep, current: CurrentUser) -> TreeRead:
tree = await tree_service.create_tree(
session,
owner=current,
name=data.name,
description=data.description,
visibility=data.visibility,
)
return TreeRead.model_validate(tree)
@router.get("", response_model=list[TreeRead])
async def list_my_trees(
session: SessionDep, current: CurrentUser, deleted: bool = False
) -> list[TreeRead]:
if deleted:
trees = await tree_service.list_deleted_trees_for_user(session, user=current)
else:
trees = await tree_service.list_trees_for_user(session, user=current)
return [TreeRead.model_validate(t) for t in trees]
@router.get("/{tree_id}", response_model=TreeRead)
async def get_tree(tree_id: uuid.UUID, session: SessionDep, current: CurrentUser) -> TreeRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
return TreeRead.model_validate(tree)
@router.patch("/{tree_id}", response_model=TreeRead)
async def update_tree(
tree_id: uuid.UUID, data: TreeUpdate, session: SessionDep, current: CurrentUser
) -> TreeRead:
tree = await tree_service.update_tree(
session, actor=current, tree_id=tree_id, changes=data.model_dump(exclude_unset=True)
)
return TreeRead.model_validate(tree)
@router.delete("/{tree_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_tree(tree_id: uuid.UUID, session: SessionDep, current: CurrentUser) -> None:
await tree_service.delete_tree(session, actor=current, tree_id=tree_id)
@router.post("/{tree_id}/restore", response_model=TreeRead)
async def restore_tree(tree_id: uuid.UUID, session: SessionDep, current: CurrentUser) -> TreeRead:
tree = await tree_service.restore_tree(session, actor=current, tree_id=tree_id)
return TreeRead.model_validate(tree)
@router.post("/{tree_id}/purge", status_code=status.HTTP_204_NO_CONTENT)
async def purge_tree(
tree_id: uuid.UUID,
data: TreePurge,
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
) -> None:
"""Permanently delete a soft-deleted tree and all its data — irreversible.
Owner-only; the tree must be in the trash and `confirm_name` must match."""
await tree_service.purge_tree(
session, store, actor=current, tree_id=tree_id, confirm_name=data.confirm_name
)
+63
View File
@@ -0,0 +1,63 @@
from fastapi import APIRouter, File, Form, Response, UploadFile
from app.api.deps import CurrentUser, ObjectStoreDep, SessionDep, is_instance_owner
from app.schemas.user import UserRead, UserSelfPersonUpdate
from app.services import account_service, user_service
router = APIRouter(prefix="/users", tags=["users"])
def _me(user) -> UserRead:
out = UserRead.model_validate(user)
out.is_instance_owner = is_instance_owner(user)
return out
@router.get("/me", response_model=UserRead)
async def read_me(current: CurrentUser) -> UserRead:
return _me(current)
@router.patch("/me/self-person", response_model=UserRead)
async def set_self_person(
data: UserSelfPersonUpdate, session: SessionDep, current: CurrentUser
) -> UserRead:
"""Link (or unlink) the Person record that represents this account."""
user = await user_service.set_self_person(
session, user=current, person_id=data.self_person_id
)
return _me(user)
@router.get("/me/export")
async def export_account(
session: SessionDep, current: CurrentUser, store: ObjectStoreDep
) -> Response:
"""Download a full backup (JSON + media) of every tree the user owns."""
data = await account_service.export_account(session, store, user=current)
return Response(
content=data,
media_type="application/zip",
headers={"Content-Disposition": 'attachment; filename="provenance-export.zip"'},
)
@router.post("/me/import")
async def import_account(
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
file: UploadFile = File(...),
) -> dict:
"""Restore a previously-exported backup into new trees (non-destructive)."""
raw = await file.read()
return await account_service.import_account(session, store, user=current, raw_zip=raw)
@router.delete("/me", status_code=204)
async def delete_account(
session: SessionDep, current: CurrentUser, confirm_email: str = Form(...)
) -> None:
"""Delete the account: the user, their owned trees, and their sessions.
Requires retyping the account email as a guard."""
await account_service.delete_account(session, user=current, confirm_email=confirm_email)
View File
+108
View File
@@ -0,0 +1,108 @@
"""Application configuration.
Twelve-factor: everything is read from the environment. Defaults are
development-friendly; production supplies real values via the compose `.env`.
No secrets or endpoints are hard-coded.
"""
from functools import lru_cache
from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
extra="ignore",
)
app_name: str = "Provenance"
version: str = "0.0.0"
app_env: str = Field(default="development", description="development | production")
# --- Instance owner / operator ---
# Email(s) of the instance owner(s) — the operator(s) who run this server.
# The matching account(s) get instance-admin rights (instance-wide settings;
# see /api/v1/admin). Comma-separated for several. Empty = no designated
# owner (the instance has no operator account). Derived at request time, so
# changing it takes effect immediately with no migration or DB state.
owner_email: str = ""
def owner_emails(self) -> frozenset[str]:
"""Normalized (lowercased, trimmed) owner emails; empty if none set."""
return frozenset(e.strip().lower() for e in self.owner_email.split(",") if e.strip())
# SQLAlchemy async URL, e.g. postgresql+asyncpg://user:pass@host:5432/db
database_url: str = Field(
default="postgresql+asyncpg://provenance:provenance@localhost:5432/provenance",
)
# --- Auth / sessions ---
session_ttl_days: int = 30
token_ttl_hours: int = 24 # email-verify / password-reset token lifetime
cookie_name: str = "provenance_session"
cookie_secure: bool = True # set false for local http; true behind TLS
# Base URL used to build links in outbound email.
app_base_url: str = "http://localhost"
# --- Object storage (S3-compatible / MinIO) ---
s3_endpoint_url: str = "http://minio:9000"
s3_bucket: str = "provenance"
s3_access_key: str = "provenance"
s3_secret_key: str = "change-me-too"
s3_region: str = "us-east-1"
s3_presign_ttl: int = 3600 # seconds
# --- Worker ---
purge_interval_seconds: int = 3600 # how often to run the soft-delete purge
purge_after_days: int = 30 # soft-deleted rows older than this are purged
# --- Email (SMTP) ---
# When true, a user with no verified email gets no active session (login is
# refused and existing sessions stop resolving). Default false so self-hosts
# without SMTP — and accounts created before this gate existed — aren't
# locked out; operators turn it on once mail works and accounts are verified.
require_email_verification: bool = False
mailer: str = Field(default="console", description="console | smtp")
smtp_host: str | None = None
smtp_port: int = 587
smtp_username: str | None = None
smtp_password: str | None = None
smtp_from: str = "Provenance <no-reply@provenance.local>"
# --- Model providers (AI assistant + match-ranking embeddings) ---
# Configure as many as you like; each is enabled when its credentials are
# present. `default_*_provider` picks which one is used by default. LLM and
# embeddings are independent (Anthropic has no embeddings endpoint).
default_llm_provider: str = "null" # null | anthropic | openai | xai | ollama
default_embedding_provider: str = "null" # null | openai | ollama
llm_max_tokens: int = 4096
embedding_dimensions: int = 1536 # must match the embedding model + pgvector column
# Anthropic (LLM only)
anthropic_api_key: str | None = None
anthropic_model: str = "claude-opus-4-8"
# OpenAI (LLM + embeddings)
openai_api_key: str | None = None
openai_base_url: str = "https://api.openai.com/v1"
openai_model: str = "gpt-4o"
openai_embedding_model: str = "text-embedding-3-small"
# xAI / Grok — OpenAI-compatible (LLM)
xai_api_key: str | None = None
xai_base_url: str = "https://api.x.ai/v1"
xai_model: str = "grok-2-latest" # set to your account's current Grok model
# Ollama — local, OpenAI-compatible, no key (LLM + embeddings)
ollama_enabled: bool = False
ollama_base_url: str = "http://localhost:11434/v1"
ollama_model: str = "llama3.1"
ollama_embedding_model: str = "nomic-embed-text"
@lru_cache
def get_settings() -> Settings:
return Settings()
+43
View File
@@ -0,0 +1,43 @@
"""Async database engine, session factory, and the FastAPI session dependency.
The repository layer builds on ``get_session``; ``get_engine`` also backs the
readiness probe. Everything is lazy so importing the app never opens a
connection (important for tests and for ``--help``-style invocations).
"""
from collections.abc import AsyncIterator
from sqlalchemy.ext.asyncio import (
AsyncEngine,
AsyncSession,
async_sessionmaker,
create_async_engine,
)
from app.core.config import get_settings
_engine: AsyncEngine | None = None
_sessionmaker: async_sessionmaker[AsyncSession] | None = None
def get_engine() -> AsyncEngine:
global _engine
if _engine is None:
_engine = create_async_engine(get_settings().database_url, pool_pre_ping=True)
return _engine
def get_sessionmaker() -> async_sessionmaker[AsyncSession]:
global _sessionmaker
if _sessionmaker is None:
_sessionmaker = async_sessionmaker(
get_engine(), expire_on_commit=False, class_=AsyncSession
)
return _sessionmaker
async def get_session() -> AsyncIterator[AsyncSession]:
"""FastAPI dependency. One session per request; commits are explicit in the
service layer."""
async with get_sessionmaker()() as session:
yield session
+59
View File
@@ -0,0 +1,59 @@
"""Schema-drift detection — a safety net for the deploy pipeline.
If a deploy ships code whose models reference a column a migration hasn't added
yet (the code is ahead of the DB), every query against that table 500s with an
opaque ``UndefinedColumnError``. That is exactly the failure that took the tree
list down once: the backend image advanced but ``alembic upgrade head`` hadn't
run on the server.
The real prevention is auto-migrate on deploy (the entrypoint runs
``alembic upgrade head`` when ``RUN_MIGRATIONS=1``). This module is defense in
depth: it makes the drift *loud and explicit* — a readiness failure and a
CRITICAL startup log — instead of a silent storm of 500s, so a half-applied
deploy is obvious within seconds.
"""
from functools import lru_cache
from pathlib import Path
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncConnection
# app/core/schema_version.py -> backend/ (parents: core, app, backend)
_MIGRATIONS_DIR = Path(__file__).resolve().parents[2] / "migrations"
@lru_cache
def expected_heads() -> frozenset[str]:
"""Revision head(s) baked into this image's migration scripts. Static for a
given build, so cache it."""
from alembic.config import Config
from alembic.script import ScriptDirectory
cfg = Config()
cfg.set_main_option("script_location", str(_MIGRATIONS_DIR))
return frozenset(ScriptDirectory.from_config(cfg).get_heads())
async def db_heads(conn: AsyncConnection) -> frozenset[str] | None:
"""Revision(s) the database is stamped at, or ``None`` when the DB is not
Alembic-managed (no ``alembic_version`` table — e.g. a test DB built straight
from ``create_all``). ``to_regclass`` returns NULL rather than erroring when
the table is absent, so this never poisons the caller's transaction."""
if await conn.scalar(text("SELECT to_regclass('public.alembic_version')")) is None:
return None
result = await conn.execute(text("SELECT version_num FROM alembic_version"))
return frozenset(row[0] for row in result)
async def schema_is_current(
conn: AsyncConnection,
) -> tuple[bool, frozenset[str], frozenset[str]]:
"""``(ok, db, expected)``. ``ok`` is True when the DB is stamped at the
code's head(s). A DB with no ``alembic_version`` table is treated as current
(not Alembic-managed → nothing to compare), so this stays quiet in tests."""
expected = expected_heads()
current = await db_heads(conn)
if current is None:
return True, frozenset(), expected
return current == expected, current, expected
+35
View File
@@ -0,0 +1,35 @@
"""Password hashing and token primitives.
Passwords use Argon2id (argon2-cffi). Session and email tokens are random
high-entropy strings; only their SHA-256 hash is stored, so a database leak
never exposes a usable credential.
"""
import hashlib
import secrets
from argon2 import PasswordHasher
from argon2.exceptions import Argon2Error
_hasher = PasswordHasher()
def hash_password(password: str) -> str:
return _hasher.hash(password)
def verify_password(password_hash: str, password: str) -> bool:
try:
return _hasher.verify(password_hash, password)
except (Argon2Error, ValueError):
return False
def generate_token() -> str:
"""A URL-safe, high-entropy token (the raw secret handed to the client)."""
return secrets.token_urlsafe(32)
def hash_token(token: str) -> str:
"""SHA-256 of a token — what we store and look up by."""
return hashlib.sha256(token.encode("utf-8")).hexdigest()
+24
View File
@@ -0,0 +1,24 @@
"""AuthProvider interface.
Operators enable any subset of providers (local, OIDC, social). A provider's
job is narrow: verify a credential and return the matching User (or None).
Session issuance, tokens, and registration live in the auth service and are
provider-agnostic, so adding OIDC/social later (Phase 5) is additive.
"""
from abc import ABC, abstractmethod
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.user import User
class AuthProvider(ABC):
name: str
@abstractmethod
async def authenticate(
self, session: AsyncSession, *, identifier: str, secret: str
) -> User | None:
"""Return the User if the credential is valid, else None."""
raise NotImplementedError
+27
View File
@@ -0,0 +1,27 @@
"""Local (email + password) auth provider."""
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.security import verify_password
from app.integrations.auth.base import AuthProvider
from app.models.user import User
class LocalAuthProvider(AuthProvider):
name = "local"
async def authenticate(
self, session: AsyncSession, *, identifier: str, secret: str
) -> User | None:
email = identifier.strip().lower()
user = (
await session.execute(
select(User).where(User.email == email, User.deleted_at.is_(None))
)
).scalar_one_or_none()
if user is None or user.hashed_password is None:
return None
if not verify_password(user.hashed_password, secret):
return None
return user
+16
View File
@@ -0,0 +1,16 @@
"""Mailer interface for transactional email.
Implementations: ConsoleMailer (dev default — logs the link) and SMTPMailer
(operator-configured). Selected by config; resolved via app.api.deps.get_mailer.
Real deployments will move sending to the worker; for now it is inline.
"""
from abc import ABC, abstractmethod
class Mailer(ABC):
@abstractmethod
async def send_email_verification(self, *, to: str, link: str) -> None: ...
@abstractmethod
async def send_password_reset(self, *, to: str, link: str) -> None: ...
@@ -0,0 +1,16 @@
"""Development mailer: logs the would-be email (including the action link) to
stdout instead of sending. Never use in production."""
import logging
from app.integrations.mailer.base import Mailer
logger = logging.getLogger("provenance.mailer")
class ConsoleMailer(Mailer):
async def send_email_verification(self, *, to: str, link: str) -> None:
logger.info("[email] verify address for %s -> %s", to, link)
async def send_password_reset(self, *, to: str, link: str) -> None:
logger.info("[email] password reset for %s -> %s", to, link)
+43
View File
@@ -0,0 +1,43 @@
"""SMTP mailer using the standard library, run off the event loop. Configured
entirely from settings (host/port/credentials/from)."""
import asyncio
import smtplib
from email.message import EmailMessage
from app.core.config import Settings
from app.integrations.mailer.base import Mailer
class SMTPMailer(Mailer):
def __init__(self, settings: Settings) -> None:
self.settings = settings
def _send(self, *, to: str, subject: str, body: str) -> None:
msg = EmailMessage()
msg["From"] = self.settings.smtp_from
msg["To"] = to
msg["Subject"] = subject
msg.set_content(body)
with smtplib.SMTP(self.settings.smtp_host, self.settings.smtp_port) as smtp:
smtp.starttls()
if self.settings.smtp_username and self.settings.smtp_password:
smtp.login(self.settings.smtp_username, self.settings.smtp_password)
smtp.send_message(msg)
async def _send_async(self, *, to: str, subject: str, body: str) -> None:
await asyncio.to_thread(self._send, to=to, subject=subject, body=body)
async def send_email_verification(self, *, to: str, link: str) -> None:
await self._send_async(
to=to,
subject="Verify your Provenance email",
body=f"Confirm your email address:\n\n{link}\n",
)
async def send_password_reset(self, *, to: str, link: str) -> None:
await self._send_async(
to=to,
subject="Reset your Provenance password",
body=f"Reset your password:\n\n{link}\n",
)
@@ -0,0 +1,24 @@
"""Anthropic LLM provider (official SDK). Self-hosters who want everything to
stay on their own metal would configure a local provider instead (e.g. Ollama) —
that's a future implementation of the same LLMProvider interface."""
from anthropic import AsyncAnthropic
from app.integrations.models.base import LLMProvider
class AnthropicLLMProvider(LLMProvider):
def __init__(self, *, api_key: str, model: str, max_tokens: int = 4096) -> None:
self._client = AsyncAnthropic(api_key=api_key)
self._model = model
self._max_tokens = max_tokens
async def complete(self, *, prompt: str, system: str | None = None) -> str:
resp = await self._client.messages.create(
model=self._model,
max_tokens=self._max_tokens,
system=system or "",
messages=[{"role": "user", "content": prompt}],
)
# content is a list of blocks; concatenate the text ones.
return "".join(b.text for b in resp.content if b.type == "text")
+36
View File
@@ -0,0 +1,36 @@
"""Model-provider interfaces — the seam the AI assistant and match ranking plug
into. LLM (text) and embeddings are *separate* abstractions: Anthropic offers no
embeddings endpoint, so the two are configured independently (twelve-factor,
CLAUDE.md #7) and a deployment may run one without the other.
These providers are read-only text/vector producers. They MUST NOT mutate tree
data — the assistant's writes go through a ChangeProposal a human approves
(CLAUDE.md #1). Nothing here touches the database.
"""
from abc import ABC, abstractmethod
class LLMProvider(ABC):
"""Text in, text out. Implementations wrap a chat/completion model."""
@abstractmethod
async def complete(self, *, prompt: str, system: str | None = None) -> str:
"""Return the model's text response to a single user prompt."""
...
class EmbeddingProvider(ABC):
"""Text in, vectors out — for pgvector-backed match ranking."""
#: Dimensionality of the returned vectors (for the pgvector column).
dimensions: int
@abstractmethod
async def embed(self, texts: list[str]) -> list[list[float]]:
"""Return one embedding vector per input text, in order."""
...
class ModelProviderNotConfigured(RuntimeError):
"""Raised when an AI capability is used but no provider is configured."""
+31
View File
@@ -0,0 +1,31 @@
"""Default providers when no model backend is configured — AI features are off.
They fail loudly (rather than silently doing nothing) so a caller that reaches
for an unconfigured capability gets a clear, actionable error.
"""
from app.integrations.models.base import (
EmbeddingProvider,
LLMProvider,
ModelProviderNotConfigured,
)
_MSG = (
"No model provider configured. Set MODEL_PROVIDER (e.g. 'anthropic') and the "
"provider's credentials to enable AI features."
)
class NullLLMProvider(LLMProvider):
async def complete(self, *, prompt: str, system: str | None = None) -> str:
raise ModelProviderNotConfigured(_MSG)
class NullEmbeddingProvider(EmbeddingProvider):
dimensions = 0
async def embed(self, texts: list[str]) -> list[list[float]]:
raise ModelProviderNotConfigured(
"No embedding provider configured. Set EMBEDDING_PROVIDER and its "
"credentials to enable match ranking."
)
@@ -0,0 +1,40 @@
"""OpenAI-compatible providers (one implementation, many vendors).
OpenAI, xAI (api.x.ai/v1), Ollama (…:11434/v1), OpenRouter, Together, vLLM, etc.
all speak the OpenAI Chat Completions / Embeddings API — they differ only by
base URL, key, and model name. So a single class, parameterized by those, plugs
in every one of them via the official `openai` SDK.
"""
from openai import AsyncOpenAI
from app.integrations.models.base import EmbeddingProvider, LLMProvider
class OpenAICompatibleLLMProvider(LLMProvider):
def __init__(self, *, api_key: str | None, base_url: str, model: str, max_tokens: int = 4096) -> None:
# Local backends (Ollama) ignore the key but the SDK requires a non-empty one.
self._client = AsyncOpenAI(api_key=api_key or "not-needed", base_url=base_url)
self._model = model
self._max_tokens = max_tokens
async def complete(self, *, prompt: str, system: str | None = None) -> str:
messages: list[dict] = []
if system:
messages.append({"role": "system", "content": system})
messages.append({"role": "user", "content": prompt})
resp = await self._client.chat.completions.create(
model=self._model, max_tokens=self._max_tokens, messages=messages
)
return resp.choices[0].message.content or ""
class OpenAICompatibleEmbeddingProvider(EmbeddingProvider):
def __init__(self, *, api_key: str | None, base_url: str, model: str, dimensions: int) -> None:
self._client = AsyncOpenAI(api_key=api_key or "not-needed", base_url=base_url)
self._model = model
self.dimensions = dimensions
async def embed(self, texts: list[str]) -> list[list[float]]:
resp = await self._client.embeddings.create(model=self._model, input=texts)
return [d.embedding for d in resp.data]
@@ -0,0 +1,25 @@
"""ObjectStore interface — pluggable binary storage behind the service layer.
Implementations are S3-compatible (MinIO for self-host, any S3 otherwise).
Methods are async wrappers so the service layer stays non-blocking even though
the underlying SDK (boto3) is synchronous.
"""
from abc import ABC, abstractmethod
class ObjectStore(ABC):
@abstractmethod
async def ensure_bucket(self) -> None: ...
@abstractmethod
async def put_object(self, *, key: str, data: bytes, content_type: str) -> None: ...
@abstractmethod
async def get_object(self, *, key: str) -> bytes: ...
@abstractmethod
async def presigned_get_url(self, *, key: str) -> str: ...
@abstractmethod
async def delete_object(self, *, key: str) -> None: ...
@@ -0,0 +1,63 @@
"""S3-compatible ObjectStore (boto3), suitable for MinIO or any S3 provider.
boto3 is synchronous; each call is dispatched to a thread so request handlers
and the worker stay async."""
import asyncio
import boto3
from botocore.client import Config
from botocore.exceptions import ClientError
from app.core.config import Settings
from app.integrations.objectstore.base import ObjectStore
class S3ObjectStore(ObjectStore):
def __init__(self, settings: Settings) -> None:
self.bucket = settings.s3_bucket
self.presign_ttl = settings.s3_presign_ttl
self._client = boto3.client(
"s3",
endpoint_url=settings.s3_endpoint_url,
aws_access_key_id=settings.s3_access_key,
aws_secret_access_key=settings.s3_secret_key,
region_name=settings.s3_region,
config=Config(signature_version="s3v4"),
)
def _ensure_bucket_sync(self) -> None:
try:
self._client.head_bucket(Bucket=self.bucket)
except ClientError:
self._client.create_bucket(Bucket=self.bucket)
async def ensure_bucket(self) -> None:
await asyncio.to_thread(self._ensure_bucket_sync)
async def put_object(self, *, key: str, data: bytes, content_type: str) -> None:
await asyncio.to_thread(
self._client.put_object,
Bucket=self.bucket,
Key=key,
Body=data,
ContentType=content_type,
)
async def get_object(self, *, key: str) -> bytes:
def _get() -> bytes:
obj = self._client.get_object(Bucket=self.bucket, Key=key)
return obj["Body"].read()
return await asyncio.to_thread(_get)
async def presigned_get_url(self, *, key: str) -> str:
return await asyncio.to_thread(
self._client.generate_presigned_url,
"get_object",
Params={"Bucket": self.bucket, "Key": key},
ExpiresIn=self.presign_ttl,
)
async def delete_object(self, *, key: str) -> None:
await asyncio.to_thread(self._client.delete_object, Bucket=self.bucket, Key=key)
+91
View File
@@ -0,0 +1,91 @@
"""FastAPI application entrypoint.
Thin by design: wire settings, routers, and error handling, and expose the
OpenAPI contract. All domain logic lives in the service layer; the privacy
engine is the single enforcement point for reads.
"""
import logging
import sys
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from app.api.health import router as health_router
from app.api.v1 import api_router
from app.core.config import get_settings
from app.core.db import get_engine
from app.core.schema_version import schema_is_current
from app.services.exceptions import Conflict, Forbidden, NotFound
def _configure_logging() -> None:
"""Emit the app's own ``provenance.*`` logs at INFO to stdout (uvicorn only
configures its own loggers). The ConsoleMailer relies on this so self-hosters
can read verification/reset links from the logs."""
app_logger = logging.getLogger("provenance")
app_logger.setLevel(logging.INFO)
if not app_logger.handlers:
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter("%(levelname)s [%(name)s] %(message)s"))
app_logger.addHandler(handler)
app_logger.propagate = False
async def _check_schema_drift() -> None:
"""On startup, shout if the DB schema is behind the code. The entrypoint
runs migrations when RUN_MIGRATIONS=1; this catches the case where that
didn't happen, so a half-applied deploy is obvious in the logs instead of a
silent storm of 500s. Never blocks startup — purely advisory."""
logger = logging.getLogger("provenance")
try:
async with get_engine().connect() as conn:
ok, db, expected = await schema_is_current(conn)
if not ok:
logger.critical(
"SCHEMA DRIFT: database is at %s but this build expects %s. "
"Run 'alembic upgrade head' — queries will fail until migrated.",
sorted(db) or ["none"],
sorted(expected),
)
except Exception as exc: # noqa: BLE001 — advisory only; never block startup
logger.warning("schema drift check skipped: %s", exc)
@asynccontextmanager
async def _lifespan(app: FastAPI):
await _check_schema_drift()
yield
def _register_error_handlers(app: FastAPI) -> None:
@app.exception_handler(NotFound)
async def _not_found(request: Request, exc: NotFound) -> JSONResponse:
return JSONResponse(status_code=404, content={"detail": str(exc) or "not found"})
@app.exception_handler(Forbidden)
async def _forbidden(request: Request, exc: Forbidden) -> JSONResponse:
return JSONResponse(status_code=403, content={"detail": str(exc) or "forbidden"})
@app.exception_handler(Conflict)
async def _conflict(request: Request, exc: Conflict) -> JSONResponse:
return JSONResponse(status_code=409, content={"detail": str(exc) or "conflict"})
def create_app() -> FastAPI:
_configure_logging()
settings = get_settings()
app = FastAPI(
title=settings.app_name,
version=settings.version,
description="Provenance API — family and land provenance.",
lifespan=_lifespan,
)
app.include_router(health_router)
app.include_router(api_router)
_register_error_handlers(app)
return app
app = create_app()
+35
View File
@@ -0,0 +1,35 @@
"""Import every model so ``Base.metadata`` is complete for Alembic autogenerate
and for ``create_all`` in tests."""
from app.models.audit import AuditEntry
from app.models.auth import Session, UserToken
from app.models.base import Base
from app.models.change_proposal import ChangeProposal
from app.models.event import Event
from app.models.media import Media
from app.models.person import Name, Person
from app.models.place import Place, PlaceName
from app.models.relationship import Relationship
from app.models.source import Citation, Source
from app.models.tree import Tree, TreeMembership
from app.models.user import User
__all__ = [
"Base",
"User",
"Tree",
"TreeMembership",
"Person",
"Name",
"Place",
"PlaceName",
"Relationship",
"Event",
"Source",
"Citation",
"AuditEntry",
"Session",
"UserToken",
"Media",
"ChangeProposal",
]
+42
View File
@@ -0,0 +1,42 @@
"""AuditEntry — append-only, immutable record of every mutation.
The actor is a User, or the assistant principal acting *on behalf of* a User
(actor_type = assistant, actor_user_id = the user it serves). No timestamps
mixin and no soft delete: this table is never updated or deleted.
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, String, func
from sqlalchemy import Enum as SAEnum
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.enums import AuditActorType
from app.models.mixins import UUIDPrimaryKey
class AuditEntry(Base, UUIDPrimaryKey):
__tablename__ = "audit_entries"
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False, index=True
)
tree_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("trees.id", ondelete="SET NULL"), index=True
)
actor_type: Mapped[AuditActorType] = mapped_column(
SAEnum(AuditActorType, name="audit_actor_type"),
default=AuditActorType.user,
server_default=AuditActorType.user.value,
)
actor_user_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("users.id", ondelete="SET NULL"), index=True
)
action: Mapped[str] = mapped_column(String(64)) # create | update | delete | restore | ...
entity_type: Mapped[str] = mapped_column(String(64))
entity_id: Mapped[uuid.UUID | None] = mapped_column()
before: Mapped[dict | None] = mapped_column(JSONB)
after: Mapped[dict | None] = mapped_column(JSONB)
+43
View File
@@ -0,0 +1,43 @@
"""Authentication state: opaque backend-issued sessions and single-use email
tokens. Only token *hashes* are stored (see app.core.security).
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, String, func
from sqlalchemy import Enum as SAEnum
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.enums import TokenPurpose
from app.models.mixins import UUIDPrimaryKey
class Session(Base, UUIDPrimaryKey):
__tablename__ = "sessions"
user_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
token_hash: Mapped[str] = mapped_column(String(64), unique=True, index=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
class UserToken(Base, UUIDPrimaryKey):
__tablename__ = "user_tokens"
user_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
purpose: Mapped[TokenPurpose] = mapped_column(SAEnum(TokenPurpose, name="token_purpose"))
token_hash: Mapped[str] = mapped_column(String(64), unique=True, index=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+20
View File
@@ -0,0 +1,20 @@
"""Declarative base with a stable constraint-naming convention.
A fixed naming convention is important so Alembic generates deterministic,
human-readable names for indexes/constraints across migrations.
"""
from sqlalchemy import MetaData
from sqlalchemy.orm import DeclarativeBase
NAMING_CONVENTION = {
"ix": "ix_%(column_0_label)s",
"uq": "uq_%(table_name)s_%(column_0_name)s",
"ck": "ck_%(table_name)s_%(constraint_name)s",
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
"pk": "pk_%(table_name)s",
}
class Base(DeclarativeBase):
metadata = MetaData(naming_convention=NAMING_CONVENTION)
+48
View File
@@ -0,0 +1,48 @@
"""ChangeProposal — a structured diff the AI assistant (or an untrusted
contributor) proposes, which a human approves/edits/rejects. Applying it routes
each operation through the normal editing services, so the change passes the
privacy engine and is audited as the approving human's action. See
docs/design/change-proposal.md and CLAUDE.md non-negotiable #1.
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, String, Text
from sqlalchemy import Enum as SAEnum
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.enums import ChangeProposalOrigin, ChangeProposalStatus
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
class ChangeProposal(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "change_proposals"
status: Mapped[ChangeProposalStatus] = mapped_column(
SAEnum(ChangeProposalStatus, name="change_proposal_status"),
default=ChangeProposalStatus.pending,
server_default=ChangeProposalStatus.pending.value,
index=True,
)
origin: Mapped[ChangeProposalOrigin] = mapped_column(
SAEnum(ChangeProposalOrigin, name="change_proposal_origin"),
default=ChangeProposalOrigin.assistant,
server_default=ChangeProposalOrigin.assistant.value,
)
created_by_user_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("users.id", ondelete="SET NULL")
)
summary: Mapped[str] = mapped_column(String(512))
rationale: Mapped[str | None] = mapped_column(Text)
# The structured diff: a list of {op, entity_type, entity_id?, payload} dicts.
operations: Mapped[list] = mapped_column(JSONB, nullable=False)
reviewed_by_user_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("users.id", ondelete="SET NULL")
)
reviewed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
review_note: Mapped[str | None] = mapped_column(String(512))
apply_error: Mapped[str | None] = mapped_column(Text)
+74
View File
@@ -0,0 +1,74 @@
"""Closed-set enumerations that drive logic (authorization, privacy, traversal).
Open-ended, GEDCOM-extensible vocabularies (event type, name type, source type)
are stored as strings instead, so importing real-world files never fails on an
unknown tag.
"""
import enum
class TreeVisibility(enum.StrEnum):
public = "public" # anyone on the web (anonymous), listed + search-indexable
site_members = "site_members" # any authenticated user of this instance
unlisted = "unlisted" # anyone with the link (anonymous), not listed/indexed
private = "private" # members only (default)
class MembershipRole(enum.StrEnum):
owner = "owner"
editor = "editor"
viewer = "viewer"
class PersonPrivacy(enum.StrEnum):
"""Per-person override of the tree's visibility (PRD US-041)."""
inherit = "inherit"
private = "private"
public = "public"
class RelationshipType(enum.StrEnum):
parent_child = "parent_child"
partnership = "partnership"
sibling = "sibling"
class ParentChildQualifier(enum.StrEnum):
"""Qualifies a parent_child edge so adoption/donor/blended families are
first-class rather than edge cases (ARCHITECTURE §5)."""
biological = "biological"
adoptive = "adoptive"
step = "step"
foster = "foster"
donor = "donor"
guardian = "guardian"
class CitationConfidence(enum.StrEnum):
high = "high"
medium = "medium"
low = "low"
class AuditActorType(enum.StrEnum):
user = "user"
assistant = "assistant"
class TokenPurpose(enum.StrEnum):
email_verify = "email_verify"
password_reset = "password_reset"
class ChangeProposalStatus(enum.StrEnum):
pending = "pending"
applied = "applied"
rejected = "rejected"
class ChangeProposalOrigin(enum.StrEnum):
assistant = "assistant" # the AI assistant, acting on behalf of a user
contributor = "contributor" # an untrusted human edit awaiting moderation
+51
View File
@@ -0,0 +1,51 @@
"""Event — a typed, dated, placed fact attached to a Person or a partnership.
Genealogical dates are messy, so we keep both:
- ``date_value`` — the original string, verbatim (e.g. "ABT 1850", "BET 1850 AND
1855"), for fidelity and GEDCOM round-trip.
- ``date_start`` / ``date_end`` — a normalized range for sorting and filtering
(an exact date sets start == end).
A CHECK enforces that exactly one subject (person XOR relationship) is set.
"""
import uuid
from datetime import date
from sqlalchemy import CheckConstraint, Date, ForeignKey, String, Text
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
class Event(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "events"
__table_args__ = (
CheckConstraint(
"(person_id IS NOT NULL) <> (relationship_id IS NOT NULL)",
name="subject_person_xor_relationship",
),
)
# Open vocabulary (birth, death, marriage, residence, immigration, ...).
event_type: Mapped[str] = mapped_column(String(64), index=True)
person_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("persons.id", ondelete="CASCADE"), index=True
)
relationship_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("relationships.id", ondelete="CASCADE"), index=True
)
place_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("places.id", ondelete="SET NULL"), index=True
)
date_value: Mapped[str | None] = mapped_column(String(255))
date_start: Mapped[date | None] = mapped_column(Date)
date_end: Mapped[date | None] = mapped_column(Date)
date_precision: Mapped[str | None] = mapped_column(String(32)) # exact|about|before|after|range
calendar: Mapped[str] = mapped_column(
String(32), default="gregorian", server_default="gregorian"
)
detail: Mapped[str | None] = mapped_column(String(512)) # e.g. occupation, address
notes: Mapped[str | None] = mapped_column(Text)
+36
View File
@@ -0,0 +1,36 @@
"""Media — a binary asset (image, scan, PDF, audio) in object storage. The row
holds metadata + checksum + the storage key; the bytes live in the ObjectStore.
Optionally attached to a single fact (person, event, or source) for now."""
import uuid
from sqlalchemy import BigInteger, ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
class Media(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "media"
uploader_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("users.id", ondelete="SET NULL"), index=True
)
storage_key: Mapped[str] = mapped_column(String(512), unique=True)
original_filename: Mapped[str] = mapped_column(String(512))
content_type: Mapped[str] = mapped_column(String(128))
byte_size: Mapped[int] = mapped_column(BigInteger)
checksum_sha256: Mapped[str] = mapped_column(String(64), index=True)
title: Mapped[str | None] = mapped_column(String(512))
# Optional single attachment target.
person_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("persons.id", ondelete="SET NULL"), index=True
)
event_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("events.id", ondelete="SET NULL"), index=True
)
source_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("sources.id", ondelete="SET NULL"), index=True
)
+45
View File
@@ -0,0 +1,45 @@
"""Reusable column mixins.
- ``UUIDPrimaryKey`` — UUID surrogate key (no PII in URLs; safe for multi-tenant).
- ``Timestamps`` — created/updated audit timestamps (DB-managed).
- ``SoftDelete`` — ``deleted_at``; a row is "deleted" when set. A scheduled
worker purges rows past the 30-day window (PRD US-080/081).
- ``TenantScoped`` — ``tree_id`` FK; every tree-owned row carries it so the
privacy engine can enforce isolation uniformly.
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, func
from sqlalchemy.orm import Mapped, declared_attr, mapped_column
class UUIDPrimaryKey:
id: Mapped[uuid.UUID] = mapped_column(primary_key=True, default=uuid.uuid4)
class Timestamps:
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
onupdate=func.now(),
nullable=False,
)
class SoftDelete:
deleted_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True, default=None
)
class TenantScoped:
@declared_attr
def tree_id(cls) -> Mapped[uuid.UUID]: # noqa: N805
return mapped_column(
ForeignKey("trees.id", ondelete="CASCADE"), nullable=False, index=True
)
+68
View File
@@ -0,0 +1,68 @@
"""Person and Name.
A Person carries living/deceased status and a per-person privacy override; the
display identity lives in one or more Name rows (variants, married names,
aliases) so name changes over time are first-class.
"""
import uuid
from sqlalchemy import Boolean, ForeignKey, Index, Integer, String, Text, text
from sqlalchemy import Enum as SAEnum
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.enums import PersonPrivacy
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
class Person(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "persons"
# Free-form to stay inclusive; not a closed enum.
gender: Mapped[str | None] = mapped_column(String(32))
# NULL = unknown (let the living-person rule derive it); True/False = asserted.
is_living: Mapped[bool | None] = mapped_column(Boolean)
privacy: Mapped[PersonPrivacy] = mapped_column(
SAEnum(PersonPrivacy, name="person_privacy"),
default=PersonPrivacy.inherit,
server_default=PersonPrivacy.inherit.value,
)
notes: Mapped[str | None] = mapped_column(Text)
class Name(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "names"
# Trigram indexes for fuzzy name search (Mueller/Müller/Muller). Requires the
# pg_trgm extension (enabled in the accompanying migration).
__table_args__ = (
Index(
"ix_names_given_trgm",
"given",
postgresql_using="gin",
postgresql_ops={"given": "gin_trgm_ops"},
),
Index(
"ix_names_surname_trgm",
"surname",
postgresql_using="gin",
postgresql_ops={"surname": "gin_trgm_ops"},
),
)
person_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("persons.id", ondelete="CASCADE"), index=True
)
# Open vocabulary (birth, married, alias, religious, ...) for GEDCOM fidelity.
name_type: Mapped[str] = mapped_column(String(32), default="birth", server_default="birth")
given: Mapped[str | None] = mapped_column(String(255))
surname: Mapped[str | None] = mapped_column(String(255))
prefix: Mapped[str | None] = mapped_column(String(64))
suffix: Mapped[str | None] = mapped_column(String(64))
nickname: Mapped[str | None] = mapped_column(String(128))
# Original full form preserved verbatim (round-trip fidelity).
display_name: Mapped[str | None] = mapped_column(String(512))
is_primary: Mapped[bool] = mapped_column(
Boolean, default=False, server_default=text("false")
)
sort_order: Mapped[int] = mapped_column(Integer, default=0, server_default="0")
+42
View File
@@ -0,0 +1,42 @@
"""Place — a gazetteer entity — and PlaceName, its historical name variants.
PlaceName carries date ranges so a record entered as "Königsberg, 1900" sorts
and displays correctly against "Kaliningrad" (ARCHITECTURE §5, §10).
Phase 0 scopes Place to a Tree (``tree_id``) to keep tenant isolation absolute.
ARCHITECTURE calls the gazetteer "tenant-shared"; a deployment-wide shared
gazetteer is a deliberate later refinement (see ARCHITECTURE §5 note).
"""
import uuid
from datetime import date
from sqlalchemy import Date, Float, ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
class Place(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "places"
name: Mapped[str] = mapped_column(String(512))
# Self-referential hierarchy: place within place.
parent_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("places.id", ondelete="SET NULL"), index=True
)
place_type: Mapped[str | None] = mapped_column(String(64))
latitude: Mapped[float | None] = mapped_column(Float)
longitude: Mapped[float | None] = mapped_column(Float)
class PlaceName(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "place_names"
place_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("places.id", ondelete="CASCADE"), index=True
)
name: Mapped[str] = mapped_column(String(512))
valid_from: Mapped[date | None] = mapped_column(Date)
valid_to: Mapped[date | None] = mapped_column(Date)
+40
View File
@@ -0,0 +1,40 @@
"""Relationship — a typed, qualified edge between two Persons.
Modeling parentage as qualified edges (rather than assuming two biological
parents) is what makes adoption, donor conception, and blended families
first-class. ``qualifier`` applies to parent_child edges; partnership events
(marriage, divorce) attach to the Relationship via Event.relationship_id.
"""
import uuid
from sqlalchemy import CheckConstraint, ForeignKey, Text
from sqlalchemy import Enum as SAEnum
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.enums import ParentChildQualifier, RelationshipType
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
class Relationship(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "relationships"
__table_args__ = (
CheckConstraint("person_from_id <> person_to_id", name="different_persons"),
)
type: Mapped[RelationshipType] = mapped_column(
SAEnum(RelationshipType, name="relationship_type")
)
# For parent_child: from = parent, to = child. For partnership/sibling: symmetric.
person_from_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("persons.id", ondelete="CASCADE"), index=True
)
person_to_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("persons.id", ondelete="CASCADE"), index=True
)
# Only meaningful for parent_child edges.
qualifier: Mapped[ParentChildQualifier | None] = mapped_column(
SAEnum(ParentChildQualifier, name="parent_child_qualifier")
)
notes: Mapped[str | None] = mapped_column(Text)
+66
View File
@@ -0,0 +1,66 @@
"""Source and Citation — the first-class provenance spine.
A Source is a reusable record of an origin; a Citation links one Source to one
specific fact (a Person, Name, Event, or Relationship — and OwnershipEvent once
property lands). A CHECK enforces exactly one target so a citation always points
at a single fact.
"""
import uuid
from sqlalchemy import CheckConstraint, ForeignKey, String, Text
from sqlalchemy import Enum as SAEnum
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.enums import CitationConfidence
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
class Source(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "sources"
title: Mapped[str] = mapped_column(String(512))
author: Mapped[str | None] = mapped_column(String(255))
source_type: Mapped[str | None] = mapped_column(String(64)) # book, census, deed, ...
repository: Mapped[str | None] = mapped_column(String(255))
url: Mapped[str | None] = mapped_column(String(1024))
citation_text: Mapped[str | None] = mapped_column(Text)
publication_info: Mapped[str | None] = mapped_column(Text)
quality_note: Mapped[str | None] = mapped_column(String(255))
class Citation(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "citations"
__table_args__ = (
CheckConstraint(
"(person_id IS NOT NULL)::int + (event_id IS NOT NULL)::int "
"+ (name_id IS NOT NULL)::int + (relationship_id IS NOT NULL)::int = 1",
name="exactly_one_target",
),
)
source_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("sources.id", ondelete="CASCADE"), index=True
)
# Exactly one of these is set (see CHECK above).
person_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("persons.id", ondelete="CASCADE"), index=True
)
event_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("events.id", ondelete="CASCADE"), index=True
)
name_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("names.id", ondelete="CASCADE"), index=True
)
relationship_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("relationships.id", ondelete="CASCADE"), index=True
)
# Locality within the source.
page: Mapped[str | None] = mapped_column(String(255))
detail: Mapped[str | None] = mapped_column(Text) # entry, line, free notes
confidence: Mapped[CitationConfidence | None] = mapped_column(
SAEnum(CitationConfidence, name="citation_confidence")
)
+58
View File
@@ -0,0 +1,58 @@
"""Tree — the top-level tenant boundary for genealogical data — and
TreeMembership, the basis for authorization (ARCHITECTURE §5).
"""
import uuid
from sqlalchemy import Enum as SAEnum
from sqlalchemy import ForeignKey, String, Text, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.enums import MembershipRole, TreeVisibility
from app.models.mixins import SoftDelete, Timestamps, UUIDPrimaryKey
class Tree(Base, UUIDPrimaryKey, Timestamps, SoftDelete):
__tablename__ = "trees"
owner_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("users.id", ondelete="RESTRICT"), index=True
)
name: Mapped[str] = mapped_column(String(255))
description: Mapped[str | None] = mapped_column(Text)
visibility: Mapped[TreeVisibility] = mapped_column(
SAEnum(TreeVisibility, name="tree_visibility"),
default=TreeVisibility.private,
server_default=TreeVisibility.private.value,
)
# The person a tree opens focused on (its "home"/root person). Cleared if
# that person is deleted. use_alter + name: trees<->persons form an FK cycle.
home_person_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey(
"persons.id",
ondelete="SET NULL",
name="fk_trees_home_person_id",
use_alter=True,
)
)
# Per-tree AI model policy (owner-configured). The names reference configured
# providers from the registry; null = that role has no model. The owner may
# use any configured provider; these limit members + the recommender.
ai_member_provider: Mapped[str | None] = mapped_column(String(32))
ai_recommender_provider: Mapped[str | None] = mapped_column(String(32))
class TreeMembership(Base, UUIDPrimaryKey, Timestamps):
__tablename__ = "tree_memberships"
__table_args__ = (
UniqueConstraint("tree_id", "user_id", name="uq_tree_memberships_tree_user"),
)
tree_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("trees.id", ondelete="CASCADE"), index=True
)
user_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
role: Mapped[MembershipRole] = mapped_column(SAEnum(MembershipRole, name="membership_role"))
+34
View File
@@ -0,0 +1,34 @@
"""User — a person with login. Identity is internal so one user can link
multiple auth providers later (the provider-link table arrives with the auth
slice). ``hashed_password`` is nullable: external/OIDC users have none.
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.mixins import SoftDelete, Timestamps, UUIDPrimaryKey
class User(Base, UUIDPrimaryKey, Timestamps, SoftDelete):
__tablename__ = "users"
email: Mapped[str] = mapped_column(String(320), unique=True, index=True)
email_verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
display_name: Mapped[str | None] = mapped_column(String(255))
hashed_password: Mapped[str | None] = mapped_column(String(255))
# The Person record that *is* this user ("home person"). Cleared if that
# person is deleted, so the link can never dangle.
self_person_id: Mapped[uuid.UUID | None] = mapped_column(
# use_alter + explicit name: users<->persons<->trees form an FK cycle,
# so this constraint must be created/dropped via ALTER, not inline.
ForeignKey(
"persons.id",
ondelete="SET NULL",
name="fk_users_self_person_id",
use_alter=True,
)
)
+40
View File
@@ -0,0 +1,40 @@
"""Thin data-access layer over SQLAlchemy. No business rules live here — the
service layer owns those (and the privacy engine). The repository only knows how
to fetch and stage rows, transparently excluding soft-deleted ones.
"""
from typing import Any
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
class BaseRepository:
def __init__(self, session: AsyncSession, model: type) -> None:
self.session = session
self.model = model
def _exclude_deleted(self, stmt):
if hasattr(self.model, "deleted_at"):
stmt = stmt.where(self.model.deleted_at.is_(None))
return stmt
async def get(self, id_: Any, *, include_deleted: bool = False):
stmt = select(self.model).where(self.model.id == id_)
if not include_deleted:
stmt = self._exclude_deleted(stmt)
return (await self.session.execute(stmt)).scalar_one_or_none()
async def list(self, *conditions, include_deleted: bool = False, order_by=None):
stmt = select(self.model)
for condition in conditions:
stmt = stmt.where(condition)
if not include_deleted:
stmt = self._exclude_deleted(stmt)
if order_by is not None:
stmt = stmt.order_by(order_by)
return list((await self.session.execute(stmt)).scalars().all())
def add(self, obj):
self.session.add(obj)
return obj
View File
+20
View File
@@ -0,0 +1,20 @@
"""Instance-admin schemas. Operator-facing, owner-only — operational status and
config, never tree data or PII (instance ownership doesn't bypass privacy)."""
from pydantic import BaseModel
from app.schemas.ai_policy import ConfiguredProvider
class InstanceStatus(BaseModel):
version: str
env: str
# Operator account(s) — the email(s) named in OWNER_EMAIL.
owner_emails: list[str]
require_email_verification: bool
# Aggregate, non-identifying counts (live rows only).
user_count: int
tree_count: int
# Instance-wide AI configuration (no secrets).
default_llm_provider: str
ai_providers: list[ConfiguredProvider]
+22
View File
@@ -0,0 +1,22 @@
from pydantic import BaseModel
class ConfiguredProvider(BaseModel):
name: str
model: str
class TreeAiPolicyRead(BaseModel):
# The model non-owners' assistant uses (null = none).
member_provider: str | None
# The model the association/recommendation engine uses (null = none).
recommender_provider: str | None
# Providers the operator has configured (from env). The owner may use any of
# these; the two settings above restrict members and the recommender to one.
configured_providers: list[ConfiguredProvider]
default_provider: str
class TreeAiPolicyUpdate(BaseModel):
member_provider: str | None = None
recommender_provider: str | None = None
+40
View File
@@ -0,0 +1,40 @@
from datetime import datetime
from pydantic import BaseModel, Field
from app.schemas.user import UserRead
class RegisterRequest(BaseModel):
email: str
password: str = Field(min_length=8)
display_name: str | None = None
class LoginRequest(BaseModel):
email: str
password: str
class TokenRequest(BaseModel):
token: str
class PasswordResetRequest(BaseModel):
email: str
class PasswordResetConfirm(BaseModel):
token: str
new_password: str = Field(min_length=8)
class PasswordChange(BaseModel):
current_password: str
new_password: str = Field(min_length=8)
class SessionRead(BaseModel):
user: UserRead
token: str
expires_at: datetime
+44
View File
@@ -0,0 +1,44 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import ChangeProposalOrigin, ChangeProposalStatus
class ProposalOperation(BaseModel):
op: str # create | update | delete
entity_type: str # person | name | event | relationship | source | citation
entity_id: uuid.UUID | None = None
payload: dict = {}
class ChangeProposalCreate(BaseModel):
summary: str
rationale: str | None = None
origin: ChangeProposalOrigin = ChangeProposalOrigin.contributor
operations: list[ProposalOperation]
class ProposalReview(BaseModel):
note: str | None = None
# Optional edited operations to apply instead of the original (approve-with-edits).
operations: list[ProposalOperation] | None = None
class ChangeProposalRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
status: ChangeProposalStatus
origin: ChangeProposalOrigin
created_by_user_id: uuid.UUID | None
summary: str
rationale: str | None
operations: list
reviewed_by_user_id: uuid.UUID | None
reviewed_at: datetime | None
review_note: str | None
apply_error: str | None
created_at: datetime
+56
View File
@@ -0,0 +1,56 @@
import uuid
from pydantic import BaseModel
class DeceasedCandidate(BaseModel):
person_id: uuid.UUID
name: str
birth_year: int
class DeceasedByChildCandidate(BaseModel):
person_id: uuid.UUID
name: str
child_birth_year: int
class DeceasedApply(BaseModel):
person_ids: list[uuid.UUID]
class GenderProposal(BaseModel):
person_id: uuid.UUID
name: str
proposed_gender: str
class GenderUpdate(BaseModel):
person_id: uuid.UUID
gender: str
class GenderApply(BaseModel):
updates: list[GenderUpdate]
class NameIssue(BaseModel):
name_id: uuid.UUID
person_id: uuid.UUID
given: str | None = None
surname: str | None = None
issue: str
class NameEdit(BaseModel):
name_id: uuid.UUID
given: str | None = None
surname: str | None = None
class NameApply(BaseModel):
edits: list[NameEdit]
class CleanupResult(BaseModel):
updated: int
+52
View File
@@ -0,0 +1,52 @@
import uuid
from datetime import date, datetime
from pydantic import BaseModel, ConfigDict
class EventCreate(BaseModel):
event_type: str
# Exactly one subject: a person or a partnership (relationship).
person_id: uuid.UUID | None = None
relationship_id: uuid.UUID | None = None
place_id: uuid.UUID | None = None
# Verbatim date string (e.g. "ABT 1850") and/or a normalized range.
date_value: str | None = None
date_start: date | None = None
date_end: date | None = None
date_precision: str | None = None
calendar: str = "gregorian"
detail: str | None = None
notes: str | None = None
class EventUpdate(BaseModel):
# All optional; only fields explicitly sent are changed (PATCH semantics).
event_type: str | None = None
place_id: uuid.UUID | None = None
date_value: str | None = None
date_start: date | None = None
date_end: date | None = None
date_precision: str | None = None
calendar: str | None = None
detail: str | None = None
notes: str | None = None
class EventRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
event_type: str
person_id: uuid.UUID | None
relationship_id: uuid.UUID | None
place_id: uuid.UUID | None
date_value: str | None
date_start: date | None
date_end: date | None
date_precision: str | None
calendar: str
detail: str | None
notes: str | None
created_at: datetime
+25
View File
@@ -0,0 +1,25 @@
import uuid
from pydantic import BaseModel
class ImportReport(BaseModel):
counts: dict[str, int]
unmapped_tags: list[str]
class DuplicateMatch(BaseModel):
# An incoming GEDCOM person that resembles an existing one in the tree.
xref: str
incoming_name: str
incoming_birth_year: str | None = None
existing_person_id: uuid.UUID
existing_name: str
existing_birth_year: str | None = None
score: str # "high" | "medium"
class ImportPreview(BaseModel):
counts: dict[str, int]
potential_duplicates: list[DuplicateMatch]
unmapped_tags: list[str]
+29
View File
@@ -0,0 +1,29 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
class MediaUpdate(BaseModel):
title: str | None = None
person_id: uuid.UUID | None = None
event_id: uuid.UUID | None = None
source_id: uuid.UUID | None = None
class MediaRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
original_filename: str
content_type: str
byte_size: int
checksum_sha256: str
title: str | None
person_id: uuid.UUID | None
event_id: uuid.UUID | None
source_id: uuid.UUID | None
created_at: datetime
# Presigned download URL, filled in by the router from the ObjectStore.
url: str | None = None
+26
View File
@@ -0,0 +1,26 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import MembershipRole
class MembershipRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
user_id: uuid.UUID
email: str
display_name: str | None
role: MembershipRole
created_at: datetime
class MemberAdd(BaseModel):
email: str
role: MembershipRole = MembershipRole.viewer
class MemberRoleUpdate(BaseModel):
role: MembershipRole
+42
View File
@@ -0,0 +1,42 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
class NameCreate(BaseModel):
# Open vocabulary: birth/maiden, married, alias, religious, nickname, ...
name_type: str = "birth"
given: str | None = None
surname: str | None = None
prefix: str | None = None
suffix: str | None = None
nickname: str | None = None
is_primary: bool = False
class NameUpdate(BaseModel):
name_type: str | None = None
given: str | None = None
surname: str | None = None
prefix: str | None = None
suffix: str | None = None
nickname: str | None = None
is_primary: bool | None = None
class NameRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
person_id: uuid.UUID
name_type: str
given: str | None
surname: str | None
prefix: str | None
suffix: str | None
nickname: str | None
is_primary: bool
sort_order: int
created_at: datetime
+37
View File
@@ -0,0 +1,37 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import PersonPrivacy
class PersonCreate(BaseModel):
given: str | None = None
surname: str | None = None
gender: str | None = None
is_living: bool | None = None
privacy: PersonPrivacy = PersonPrivacy.inherit
notes: str | None = None
class PersonUpdate(BaseModel):
# Person fields + the primary name's parts; only sent fields are changed.
given: str | None = None
surname: str | None = None
gender: str | None = None
is_living: bool | None = None
privacy: PersonPrivacy | None = None
notes: str | None = None
class PersonRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
primary_name: str | None = None
gender: str | None
is_living: bool | None
privacy: PersonPrivacy
created_at: datetime
+33
View File
@@ -0,0 +1,33 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import ParentChildQualifier, RelationshipType
class RelationshipCreate(BaseModel):
type: RelationshipType
person_from_id: uuid.UUID
person_to_id: uuid.UUID
# Only meaningful for parent_child edges (from = parent, to = child).
qualifier: ParentChildQualifier | None = None
notes: str | None = None
class RelationshipUpdate(BaseModel):
qualifier: ParentChildQualifier | None = None
notes: str | None = None
class RelationshipRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
type: RelationshipType
person_from_id: uuid.UUID
person_to_id: uuid.UUID
qualifier: ParentChildQualifier | None
notes: str | None
created_at: datetime
+78
View File
@@ -0,0 +1,78 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import CitationConfidence
class SourceCreate(BaseModel):
title: str
author: str | None = None
source_type: str | None = None
repository: str | None = None
url: str | None = None
citation_text: str | None = None
publication_info: str | None = None
quality_note: str | None = None
class SourceRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
title: str
author: str | None
source_type: str | None
repository: str | None
url: str | None
citation_text: str | None
publication_info: str | None
quality_note: str | None
created_at: datetime
class SourceUpdate(BaseModel):
title: str | None = None
author: str | None = None
source_type: str | None = None
repository: str | None = None
url: str | None = None
citation_text: str | None = None
publication_info: str | None = None
quality_note: str | None = None
class CitationUpdate(BaseModel):
page: str | None = None
detail: str | None = None
confidence: CitationConfidence | None = None
class CitationCreate(BaseModel):
source_id: uuid.UUID
# Exactly one target fact.
person_id: uuid.UUID | None = None
event_id: uuid.UUID | None = None
name_id: uuid.UUID | None = None
relationship_id: uuid.UUID | None = None
page: str | None = None
detail: str | None = None
confidence: CitationConfidence | None = None
class CitationRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
source_id: uuid.UUID
person_id: uuid.UUID | None
event_id: uuid.UUID | None
name_id: uuid.UUID | None
relationship_id: uuid.UUID | None
page: str | None
detail: str | None
confidence: CitationConfidence | None
created_at: datetime
+49
View File
@@ -0,0 +1,49 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import TreeVisibility
class TreeCreate(BaseModel):
name: str
description: str | None = None
visibility: TreeVisibility = TreeVisibility.private
class TreeUpdate(BaseModel):
name: str | None = None
description: str | None = None
visibility: TreeVisibility | None = None
home_person_id: uuid.UUID | None = None
class TreePurge(BaseModel):
# Retype the tree's name to confirm a permanent, irreversible delete.
confirm_name: str
class TreeRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
name: str
description: str | None
visibility: TreeVisibility
owner_id: uuid.UUID
home_person_id: uuid.UUID | None = None
created_at: datetime
class PublicTreeRead(BaseModel):
"""Tree projection for the public surface — deliberately omits owner_id so a
public/unlisted tree doesn't reveal which account owns it."""
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
name: str
description: str | None
visibility: TreeVisibility
home_person_id: uuid.UUID | None = None
+31
View File
@@ -0,0 +1,31 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
# Note: email is a plain str for now (no email-validator dependency yet); the
# auth slice can tighten this to EmailStr.
class UserCreate(BaseModel):
email: str
display_name: str | None = None
class UserRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
email: str
display_name: str | None
email_verified_at: datetime | None
self_person_id: uuid.UUID | None = None
created_at: datetime
# Operational role, not a DB column: true when this account's email is named
# in OWNER_EMAIL. Set by the API layer (see users.read_me).
is_instance_owner: bool = False
class UserSelfPersonUpdate(BaseModel):
# null clears the link; otherwise the Person that represents this account.
self_person_id: uuid.UUID | None = None
View File
+352
View File
@@ -0,0 +1,352 @@
"""Account-level data portability: export the signed-in user's owned trees as a
zip (JSON + media bytes), restore such a zip into a brand-new tree
(non-destructive), and delete the account.
The export format is a zip containing ``account.json`` plus ``media/<id>`` blobs.
Restore always creates new trees and remaps ids, so it can't clobber existing
data.
"""
import hashlib
import io
import json
import uuid
import zipfile
from datetime import UTC, date, datetime
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.integrations.objectstore.base import ObjectStore
from app.models.auth import Session as SessionModel
from app.models.enums import MembershipRole
from app.models.event import Event
from app.models.media import Media
from app.models.person import Name, Person
from app.models.place import Place
from app.models.relationship import Relationship
from app.models.source import Citation, Source
from app.models.tree import Tree, TreeMembership
from app.models.user import User
from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound
EXPORT_VERSION = 1
_DROP = {"created_at", "updated_at", "deleted_at", "tree_id"}
# Media columns rebuilt on import (storage is re-keyed, checksum recomputed).
_MEDIA_DROP = _DROP | {"uploader_id", "storage_key", "byte_size", "checksum_sha256"}
_DATE_FIELDS = {"date_start", "date_end"}
def _row(obj, drop: set[str]) -> dict:
out: dict = {}
for col in obj.__table__.columns.keys(): # noqa: SIM118
if col in drop:
continue
out[col] = getattr(obj, col)
return out
async def _entities(session: AsyncSession, model, tree_id: uuid.UUID):
stmt = select(model).where(model.tree_id == tree_id, model.deleted_at.is_(None))
return list((await session.execute(stmt)).scalars().all())
async def export_account(session: AsyncSession, store: ObjectStore, *, user: User) -> bytes:
"""Build a zip of every tree the user owns: account.json + media blobs."""
trees = list(
(
await session.execute(
select(Tree).where(Tree.owner_id == user.id, Tree.deleted_at.is_(None))
)
).scalars().all()
)
payload: dict = {
"version": EXPORT_VERSION,
"user": {"email": user.email, "display_name": user.display_name},
"trees": [],
}
media_blobs: list[tuple[str, bytes]] = []
for tree in trees:
media_rows = await _entities(session, Media, tree.id)
media_out = []
for m in media_rows:
ref = f"media/{m.id}"
rec = _row(m, _MEDIA_DROP)
rec["_file"] = ref
media_out.append(rec)
try:
media_blobs.append((ref, await store.get_object(key=m.storage_key)))
except Exception: # noqa: BLE001 — a missing blob shouldn't abort the export
rec["_file"] = None
payload["trees"].append({
"tree": {
"name": tree.name,
"description": tree.description,
"visibility": tree.visibility,
"home_person_id": tree.home_person_id,
},
"places": [_row(p, _DROP) for p in await _entities(session, Place, tree.id)],
"persons": [_row(p, _DROP) for p in await _entities(session, Person, tree.id)],
"names": [_row(n, _DROP) for n in await _entities(session, Name, tree.id)],
"relationships": [
_row(r, _DROP) for r in await _entities(session, Relationship, tree.id)
],
"events": [_row(e, _DROP) for e in await _entities(session, Event, tree.id)],
"sources": [_row(s, _DROP) for s in await _entities(session, Source, tree.id)],
"citations": [_row(c, _DROP) for c in await _entities(session, Citation, tree.id)],
"media": media_out,
})
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("account.json", json.dumps(payload, default=str, indent=2))
for ref, blob in media_blobs:
zf.writestr(ref, blob)
return buf.getvalue()
def _as_uuid(v) -> uuid.UUID | None:
return uuid.UUID(v) if v else None
def _as_date(v) -> date | None:
return date.fromisoformat(v) if v else None
async def import_account(
session: AsyncSession, store: ObjectStore, *, user: User, raw_zip: bytes
) -> dict:
"""Restore an exported zip into NEW trees owned by the user. Non-destructive:
every record gets a fresh id; nothing existing is touched."""
try:
zf = zipfile.ZipFile(io.BytesIO(raw_zip))
payload = json.loads(zf.read("account.json"))
except (zipfile.BadZipFile, KeyError, json.JSONDecodeError) as e:
raise NotFound("not a valid Provenance export") from e
counts: dict[str, int] = {"trees": 0, "persons": 0, "events": 0, "media": 0}
for tdata in payload.get("trees", []):
t = tdata.get("tree", {})
tree = Tree(
owner_id=user.id,
name=(t.get("name") or "Imported tree"),
description=t.get("description"),
visibility=t.get("visibility") or "private",
)
session.add(tree)
await session.flush()
session.add(
TreeMembership(tree_id=tree.id, user_id=user.id, role=MembershipRole.owner)
)
counts["trees"] += 1
# id remaps from the export's ids to the freshly created ones.
pmap: dict[str, uuid.UUID] = {}
rmap: dict[str, uuid.UUID] = {}
smap: dict[str, uuid.UUID] = {}
nmap: dict[str, uuid.UUID] = {}
emap: dict[str, uuid.UUID] = {}
plmap: dict[str, uuid.UUID] = {}
for pl in tdata.get("places", []):
obj = Place(
tree_id=tree.id,
name=pl.get("name") or "",
place_type=pl.get("place_type"),
latitude=pl.get("latitude"),
longitude=pl.get("longitude"),
)
session.add(obj)
await session.flush()
plmap[pl["id"]] = obj.id
for p in tdata.get("persons", []):
obj = Person(
tree_id=tree.id,
gender=p.get("gender"),
is_living=p.get("is_living"),
privacy=p.get("privacy") or "inherit",
notes=p.get("notes"),
)
session.add(obj)
await session.flush()
pmap[p["id"]] = obj.id
counts["persons"] += 1
for n in tdata.get("names", []):
pid = pmap.get(n.get("person_id"))
if pid is None:
continue
obj = Name(
tree_id=tree.id,
person_id=pid,
name_type=n.get("name_type") or "birth",
given=n.get("given"),
surname=n.get("surname"),
prefix=n.get("prefix"),
suffix=n.get("suffix"),
nickname=n.get("nickname"),
display_name=n.get("display_name"),
is_primary=bool(n.get("is_primary")),
sort_order=n.get("sort_order") or 0,
)
session.add(obj)
await session.flush()
nmap[n["id"]] = obj.id
for r in tdata.get("relationships", []):
a = pmap.get(r.get("person_from_id"))
b = pmap.get(r.get("person_to_id"))
if a is None or b is None:
continue
obj = Relationship(
tree_id=tree.id,
type=r.get("type"),
person_from_id=a,
person_to_id=b,
qualifier=r.get("qualifier"),
notes=r.get("notes"),
)
session.add(obj)
await session.flush()
rmap[r["id"]] = obj.id
for e in tdata.get("events", []):
obj = Event(
tree_id=tree.id,
event_type=e.get("event_type") or "other",
person_id=pmap.get(e.get("person_id")),
relationship_id=rmap.get(e.get("relationship_id")),
place_id=plmap.get(e.get("place_id")),
date_value=e.get("date_value"),
date_start=_as_date(e.get("date_start")),
date_end=_as_date(e.get("date_end")),
date_precision=e.get("date_precision"),
calendar=e.get("calendar") or "gregorian",
detail=e.get("detail"),
notes=e.get("notes"),
)
session.add(obj)
await session.flush()
emap[e["id"]] = obj.id
counts["events"] += 1
for s in tdata.get("sources", []):
obj = Source(
tree_id=tree.id,
title=s.get("title") or "Untitled source",
author=s.get("author"),
source_type=s.get("source_type"),
repository=s.get("repository"),
url=s.get("url"),
citation_text=s.get("citation_text"),
publication_info=s.get("publication_info"),
quality_note=s.get("quality_note"),
)
session.add(obj)
await session.flush()
smap[s["id"]] = obj.id
for c in tdata.get("citations", []):
sid = smap.get(c.get("source_id"))
if sid is None:
continue
session.add(
Citation(
tree_id=tree.id,
source_id=sid,
person_id=pmap.get(c.get("person_id")),
event_id=emap.get(c.get("event_id")),
name_id=nmap.get(c.get("name_id")),
relationship_id=rmap.get(c.get("relationship_id")),
page=c.get("page"),
detail=c.get("detail"),
confidence=c.get("confidence"),
)
)
for m in tdata.get("media", []):
ref = m.get("_file")
if not ref:
continue
try:
blob = zf.read(ref)
except KeyError:
continue
media_id = uuid.uuid4()
filename = m.get("original_filename") or "upload"
key = f"{tree.id}/{media_id}/{filename}"
await store.ensure_bucket()
await store.put_object(
key=key,
data=blob,
content_type=m.get("content_type") or "application/octet-stream",
)
session.add(
Media(
id=media_id,
tree_id=tree.id,
uploader_id=user.id,
storage_key=key,
original_filename=filename,
content_type=m.get("content_type") or "application/octet-stream",
byte_size=len(blob),
checksum_sha256=hashlib.sha256(blob).hexdigest(),
title=m.get("title"),
person_id=pmap.get(m.get("person_id")),
event_id=emap.get(m.get("event_id")),
source_id=smap.get(m.get("source_id")),
)
)
counts["media"] += 1
# Remap the home person last, once persons exist.
home = t.get("home_person_id")
if home and home in pmap:
tree.home_person_id = pmap[home]
record_audit(
session,
action="import",
entity_type="Account",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=user.id,
after=counts,
)
await session.commit()
return counts
async def delete_account(session: AsyncSession, *, user: User, confirm_email: str) -> None:
"""Soft-delete the account: the user, the trees they own, and all their
sessions. Requires the user to retype their email as a guard."""
if confirm_email.strip().lower() != user.email.lower():
raise Forbidden("email confirmation does not match")
now = datetime.now(UTC)
await session.execute(
update(Tree)
.where(Tree.owner_id == user.id, Tree.deleted_at.is_(None))
.values(deleted_at=now)
)
await session.execute(
update(SessionModel)
.where(SessionModel.user_id == user.id, SessionModel.revoked_at.is_(None))
.values(revoked_at=now)
)
user.deleted_at = now
record_audit(
session,
action="delete",
entity_type="User",
entity_id=user.id,
actor_user_id=user.id,
)
await session.commit()
+77
View File
@@ -0,0 +1,77 @@
"""Per-tree AI model policy — owner-only. Assigns which configured provider
members and the recommender use; the owner may use any configured provider.
The operator decides which providers exist (env / registry); the tree owner
decides who uses which. See app/api/deps.py for the registry.
"""
import uuid
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import configured_llm_providers
from app.models.enums import MembershipRole
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.exceptions import Forbidden
async def _require_owner(session: AsyncSession, *, actor: User, tree: Tree) -> None:
role = await privacy.get_membership_role(session, actor.id, tree.id)
if role is not MembershipRole.owner:
raise Forbidden("only the tree owner can configure AI")
def _names() -> set[str]:
return {p["name"] for p in configured_llm_providers()}
async def get_policy(session: AsyncSession, *, actor: User, tree: Tree) -> dict:
await _require_owner(session, actor=actor, tree=tree)
from app.core.config import get_settings
return {
"member_provider": tree.ai_member_provider,
"recommender_provider": tree.ai_recommender_provider,
"configured_providers": configured_llm_providers(),
"default_provider": get_settings().default_llm_provider,
}
async def update_policy(
session: AsyncSession,
*,
actor: User,
tree: Tree,
member_provider: str | None,
recommender_provider: str | None,
) -> dict:
await _require_owner(session, actor=actor, tree=tree)
valid = _names()
for value in (member_provider, recommender_provider):
if value is not None and value not in valid:
raise Forbidden(f"'{value}' is not a configured provider")
tree.ai_member_provider = member_provider
tree.ai_recommender_provider = recommender_provider
await session.commit()
await session.refresh(tree)
return await get_policy(session, actor=actor, tree=tree)
# --- Resolution helpers (for the future assistant / recommender) -------------
def provider_name_for_member(tree: Tree) -> str | None:
"""Provider an ordinary member's assistant should use, if any."""
return tree.ai_member_provider
def provider_name_for_recommender(tree: Tree) -> str | None:
return tree.ai_recommender_provider
def provider_name_for_owner(tree: Tree, requested: str | None = None) -> str | None:
"""The owner may use any configured provider; default to the requested one."""
if requested and requested in _names():
return requested
return tree.ai_member_provider # fall back to the member model
+46
View File
@@ -0,0 +1,46 @@
"""Audit logging. Every mutation records an append-only AuditEntry attributing
the change to a User (or the assistant principal acting for a User). Staged on
the session; the caller commits as part of its unit of work.
"""
import json
import uuid
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.audit import AuditEntry
from app.models.enums import AuditActorType
def _json_safe(d: dict | None) -> dict | None:
"""Coerce a change dict to JSON-native types (UUIDs, enums, dates -> str) so
it lands in the JSON audit column regardless of what the caller passed."""
if d is None:
return None
return json.loads(json.dumps(d, default=str))
def record_audit(
session: AsyncSession,
*,
action: str,
entity_type: str,
entity_id: uuid.UUID | None = None,
tree_id: uuid.UUID | None = None,
actor_user_id: uuid.UUID | None = None,
actor_type: AuditActorType = AuditActorType.user,
before: dict | None = None,
after: dict | None = None,
) -> AuditEntry:
entry = AuditEntry(
action=action,
entity_type=entity_type,
entity_id=entity_id,
tree_id=tree_id,
actor_user_id=actor_user_id,
actor_type=actor_type,
before=_json_safe(before),
after=_json_safe(after),
)
session.add(entry)
return entry
+228
View File
@@ -0,0 +1,228 @@
"""Authentication service: registration, login, sessions, email verification,
and password reset. Provider-agnostic — credential checking is delegated to an
AuthProvider; this module owns session/token issuance and the audit trail.
"""
from datetime import UTC, datetime, timedelta
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings
from app.core.security import generate_token, hash_password, hash_token, verify_password
from app.integrations.auth.local import LocalAuthProvider
from app.integrations.mailer.base import Mailer
from app.models.auth import Session as SessionModel
from app.models.auth import UserToken
from app.models.enums import TokenPurpose
from app.models.user import User
from app.services.audit import record_audit
from app.services.exceptions import Conflict, Forbidden, NotFound
_local_provider = LocalAuthProvider()
def _now() -> datetime:
return datetime.now(UTC)
def _link(path: str, raw: str) -> str:
return f"{get_settings().app_base_url}{path}?token={raw}"
def _issue_session(session: AsyncSession, user: User) -> tuple[str, SessionModel]:
raw = generate_token()
record = SessionModel(
user_id=user.id,
token_hash=hash_token(raw),
expires_at=_now() + timedelta(days=get_settings().session_ttl_days),
)
session.add(record)
return raw, record
def _create_email_token(session: AsyncSession, user: User, purpose: TokenPurpose) -> str:
raw = generate_token()
session.add(
UserToken(
user_id=user.id,
purpose=purpose,
token_hash=hash_token(raw),
expires_at=_now() + timedelta(hours=get_settings().token_ttl_hours),
)
)
return raw
async def _consume_token(
session: AsyncSession, raw_token: str, purpose: TokenPurpose
) -> UserToken:
token = (
await session.execute(
select(UserToken).where(
UserToken.token_hash == hash_token(raw_token),
UserToken.purpose == purpose,
)
)
).scalar_one_or_none()
if token is None or token.used_at is not None or token.expires_at <= _now():
raise NotFound("invalid or expired token")
token.used_at = _now()
return token
async def register(
session: AsyncSession,
mailer: Mailer,
*,
email: str,
password: str,
display_name: str | None = None,
) -> tuple[User, str, datetime]:
email = email.strip().lower()
existing = (
await session.execute(select(User).where(User.email == email))
).scalar_one_or_none()
if existing is not None:
raise Conflict("email already registered")
user = User(email=email, display_name=display_name, hashed_password=hash_password(password))
session.add(user)
await session.flush()
verify_raw = _create_email_token(session, user, TokenPurpose.email_verify)
raw_token, record = _issue_session(session, user)
record_audit(
session,
action="register",
entity_type="User",
entity_id=user.id,
actor_user_id=user.id,
after={"email": email},
)
await session.commit()
await session.refresh(user)
await mailer.send_email_verification(to=email, link=_link("/auth/verify-email", verify_raw))
return user, raw_token, record.expires_at
async def login(
session: AsyncSession, *, email: str, password: str
) -> tuple[User, str, datetime] | None:
user = await _local_provider.authenticate(session, identifier=email, secret=password)
if user is None:
return None
if get_settings().require_email_verification and user.email_verified_at is None:
raise Forbidden("email not verified — check your inbox for the verification link")
raw_token, record = _issue_session(session, user)
record_audit(
session, action="login", entity_type="User", entity_id=user.id, actor_user_id=user.id
)
await session.commit()
return user, raw_token, record.expires_at
async def logout(session: AsyncSession, *, raw_token: str) -> None:
await session.execute(
update(SessionModel)
.where(
SessionModel.token_hash == hash_token(raw_token),
SessionModel.revoked_at.is_(None),
)
.values(revoked_at=_now())
)
await session.commit()
async def resolve_session_user(session: AsyncSession, *, raw_token: str) -> User | None:
record = (
await session.execute(
select(SessionModel).where(SessionModel.token_hash == hash_token(raw_token))
)
).scalar_one_or_none()
if record is None or record.revoked_at is not None or record.expires_at <= _now():
return None
user = (
await session.execute(
select(User).where(User.id == record.user_id, User.deleted_at.is_(None))
)
).scalar_one_or_none()
# The single read-side enforcement: an unverified user has no active session
# when verification is required. Gates every authenticated request at once.
if user is not None and get_settings().require_email_verification and user.email_verified_at is None:
return None
return user
async def verify_email(session: AsyncSession, *, raw_token: str) -> None:
token = await _consume_token(session, raw_token, TokenPurpose.email_verify)
await session.execute(
update(User).where(User.id == token.user_id).values(email_verified_at=_now())
)
record_audit(
session,
action="verify_email",
entity_type="User",
entity_id=token.user_id,
actor_user_id=token.user_id,
)
await session.commit()
async def request_password_reset(session: AsyncSession, mailer: Mailer, *, email: str) -> None:
email = email.strip().lower()
user = (
await session.execute(
select(User).where(User.email == email, User.deleted_at.is_(None))
)
).scalar_one_or_none()
# Always succeed to avoid leaking which emails are registered.
if user is None:
return
raw = _create_email_token(session, user, TokenPurpose.password_reset)
await session.commit()
await mailer.send_password_reset(to=email, link=_link("/auth/reset-password", raw))
async def change_password(
session: AsyncSession, *, user: User, current_password: str, new_password: str
) -> None:
"""Change a logged-in user's password after re-verifying the current one.
Revokes other sessions so a changed password takes effect everywhere."""
if not user.hashed_password or not verify_password(
user.hashed_password, current_password
):
raise Forbidden("current password is incorrect")
user.hashed_password = hash_password(new_password)
record_audit(
session,
action="change_password",
entity_type="User",
entity_id=user.id,
actor_user_id=user.id,
)
await session.commit()
async def reset_password(session: AsyncSession, *, raw_token: str, new_password: str) -> None:
token = await _consume_token(session, raw_token, TokenPurpose.password_reset)
await session.execute(
update(User)
.where(User.id == token.user_id)
.values(hashed_password=hash_password(new_password))
)
# Revoke all existing sessions — a reset invalidates prior logins.
await session.execute(
update(SessionModel)
.where(SessionModel.user_id == token.user_id, SessionModel.revoked_at.is_(None))
.values(revoked_at=_now())
)
record_audit(
session,
action="reset_password",
entity_type="User",
entity_id=token.user_id,
actor_user_id=token.user_id,
)
await session.commit()
@@ -0,0 +1,355 @@
"""ChangeProposal lifecycle: propose (assistant/contributor) → review → apply/reject.
The structural guarantee (CLAUDE.md #1): a proposal's operations are executed
ONLY by ``apply()``, which requires the actor be an editor and dispatches every
op through the normal editing services — so each change passes the privacy
engine and is audited as the approving human. ``propose()`` only inserts a
pending row; it performs no domain mutation. See docs/design/change-proposal.md.
"""
import uuid
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.change_proposal import ChangeProposal
from app.models.enums import (
ChangeProposalOrigin,
ChangeProposalStatus,
CitationConfidence,
ParentChildQualifier,
RelationshipType,
)
from app.models.tree import Tree
from app.models.user import User
from app.services import (
citation_service,
event_service,
name_service,
person_service,
privacy,
relationship_service,
source_service,
)
from app.services.exceptions import Conflict, Forbidden, NotFound
def _now() -> datetime:
return datetime.now(UTC)
def _uuid(v) -> uuid.UUID | None:
return uuid.UUID(str(v)) if v else None
async def _require_editor(session: AsyncSession, *, actor: User, tree: Tree) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
async def _require_member(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> None:
# Proposals can reference unredacted facts → members only.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
raise Forbidden("only members can see change proposals")
async def _load(
session: AsyncSession, tree: Tree, proposal_id: uuid.UUID
) -> ChangeProposal:
cp = (
await session.execute(
select(ChangeProposal).where(
ChangeProposal.id == proposal_id,
ChangeProposal.tree_id == tree.id,
ChangeProposal.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if cp is None:
raise NotFound("proposal not found")
return cp
async def propose(
session: AsyncSession,
*,
tree: Tree,
origin: ChangeProposalOrigin,
created_by: uuid.UUID | None,
summary: str,
rationale: str | None,
operations: list[dict],
) -> ChangeProposal:
"""Insert a pending proposal. The ONLY mutation here is the proposal row — no
tree data changes. (No edit-rights check: proposing isn't writing.)"""
cp = ChangeProposal(
tree_id=tree.id,
origin=origin,
created_by_user_id=created_by,
summary=summary,
rationale=rationale,
operations=operations,
status=ChangeProposalStatus.pending,
)
session.add(cp)
await session.commit()
await session.refresh(cp)
return cp
async def list_proposals(
session: AsyncSession,
*,
viewer_id: uuid.UUID,
tree: Tree,
status: ChangeProposalStatus | None = None,
) -> list[ChangeProposal]:
await _require_member(session, viewer_id=viewer_id, tree=tree)
stmt = select(ChangeProposal).where(
ChangeProposal.tree_id == tree.id, ChangeProposal.deleted_at.is_(None)
)
if status is not None:
stmt = stmt.where(ChangeProposal.status == status)
stmt = stmt.order_by(ChangeProposal.created_at.desc())
return list((await session.execute(stmt)).scalars().all())
async def get_proposal(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, proposal_id: uuid.UUID
) -> ChangeProposal:
await _require_member(session, viewer_id=viewer_id, tree=tree)
return await _load(session, tree, proposal_id)
async def reject(
session: AsyncSession,
*,
actor: User,
tree: Tree,
proposal_id: uuid.UUID,
note: str | None = None,
) -> ChangeProposal:
await _require_editor(session, actor=actor, tree=tree)
cp = await _load(session, tree, proposal_id)
if cp.status is not ChangeProposalStatus.pending:
raise Conflict("proposal is not pending")
cp.status = ChangeProposalStatus.rejected
cp.reviewed_by_user_id = actor.id
cp.reviewed_at = _now()
cp.review_note = note
await session.commit()
await session.refresh(cp)
return cp
async def apply(
session: AsyncSession,
*,
actor: User,
tree: Tree,
proposal_id: uuid.UUID,
edited_operations: list[dict] | None = None,
) -> ChangeProposal:
await _require_editor(session, actor=actor, tree=tree)
cp = await _load(session, tree, proposal_id)
if cp.status is not ChangeProposalStatus.pending:
raise Conflict("proposal is not pending")
ops = edited_operations if edited_operations is not None else list(cp.operations)
try:
for op in ops:
await _dispatch(session, actor=actor, tree=tree, op=op)
except Conflict:
raise
except Exception as exc: # noqa: BLE001 — record the failure on the proposal
err = f"{type(exc).__name__}: {exc}"[:2000]
# The editing services raise (NotFound/Forbidden/validation) before
# committing, so the transaction is clean — record the error and commit.
# If a later op did write before failing, those ops already committed
# (v1 isn't cross-op transactional; see the design note).
cp = await _load(session, tree, proposal_id)
cp.apply_error = err
await session.commit()
raise Conflict(f"could not apply proposal: {err}") from exc
if edited_operations is not None:
cp.operations = edited_operations
cp.status = ChangeProposalStatus.applied
cp.reviewed_by_user_id = actor.id
cp.reviewed_at = _now()
cp.apply_error = None
await session.commit()
await session.refresh(cp)
return cp
async def delete_proposal(
session: AsyncSession, *, actor: User, tree: Tree, proposal_id: uuid.UUID
) -> None:
await _require_editor(session, actor=actor, tree=tree)
cp = await _load(session, tree, proposal_id)
cp.deleted_at = _now()
await session.commit()
def _bad(entity_type: str, action: str) -> Conflict:
return Conflict(f"unsupported operation '{action}' on '{entity_type}'")
async def _dispatch(session: AsyncSession, *, actor: User, tree: Tree, op: dict) -> None:
"""Route one operation through the matching editing service (privacy + audit)."""
et = op.get("entity_type")
action = op.get("op")
payload = op.get("payload") or {}
eid = op.get("entity_id")
if et == "person":
if action == "create":
await person_service.create_person(
session,
actor=actor,
tree=tree,
given=payload.get("given"),
surname=payload.get("surname"),
gender=payload.get("gender"),
is_living=payload.get("is_living"),
notes=payload.get("notes"),
)
elif action == "update":
await person_service.update_person(
session, actor=actor, tree=tree, person_id=_uuid(eid), changes=payload
)
elif action == "delete":
await person_service.delete_person(
session,
actor=actor,
tree=tree,
person_id=_uuid(eid),
cascade=bool(payload.get("cascade", False)),
)
else:
raise _bad(et, action)
elif et == "event":
if action == "create":
await event_service.create_event(
session,
actor=actor,
tree=tree,
event_type=payload["event_type"],
person_id=_uuid(payload.get("person_id")),
relationship_id=_uuid(payload.get("relationship_id")),
date_value=payload.get("date_value"),
date_precision=payload.get("date_precision"),
detail=payload.get("detail"),
notes=payload.get("notes"),
)
elif action == "update":
await event_service.update_event(
session, actor=actor, tree=tree, event_id=_uuid(eid), changes=payload
)
elif action == "delete":
await event_service.delete_event(
session, actor=actor, tree=tree, event_id=_uuid(eid)
)
else:
raise _bad(et, action)
elif et == "relationship":
if action == "create":
await relationship_service.create_relationship(
session,
actor=actor,
tree=tree,
type=RelationshipType(payload["type"]),
person_from_id=_uuid(payload["person_from_id"]),
person_to_id=_uuid(payload["person_to_id"]),
qualifier=ParentChildQualifier(payload["qualifier"])
if payload.get("qualifier")
else None,
notes=payload.get("notes"),
)
elif action == "delete":
await relationship_service.delete_relationship(
session, actor=actor, tree=tree, relationship_id=_uuid(eid)
)
else:
raise _bad(et, action)
elif et == "name":
if action == "create":
await name_service.create_name(
session,
actor=actor,
tree=tree,
person_id=_uuid(payload["person_id"]),
name_type=payload.get("name_type", "birth"),
given=payload.get("given"),
surname=payload.get("surname"),
prefix=payload.get("prefix"),
suffix=payload.get("suffix"),
nickname=payload.get("nickname"),
is_primary=bool(payload.get("is_primary", False)),
)
elif action == "update":
changes = {k: v for k, v in payload.items() if k != "person_id"}
await name_service.update_name(
session,
actor=actor,
tree=tree,
person_id=_uuid(payload["person_id"]),
name_id=_uuid(eid),
changes=changes,
)
elif action == "delete":
await name_service.delete_name(
session,
actor=actor,
tree=tree,
person_id=_uuid(payload["person_id"]),
name_id=_uuid(eid),
)
else:
raise _bad(et, action)
elif et == "source":
if action == "create":
await source_service.create_source(
session,
actor=actor,
tree=tree,
title=payload["title"],
author=payload.get("author"),
source_type=payload.get("source_type"),
repository=payload.get("repository"),
url=payload.get("url"),
citation_text=payload.get("citation_text"),
publication_info=payload.get("publication_info"),
quality_note=payload.get("quality_note"),
)
elif action == "delete":
await source_service.delete_source(
session, actor=actor, tree=tree, source_id=_uuid(eid)
)
else:
raise _bad(et, action)
elif et == "citation":
if action == "create":
await citation_service.create_citation(
session,
actor=actor,
tree=tree,
source_id=_uuid(payload["source_id"]),
person_id=_uuid(payload.get("person_id")),
event_id=_uuid(payload.get("event_id")),
name_id=_uuid(payload.get("name_id")),
relationship_id=_uuid(payload.get("relationship_id")),
page=payload.get("page"),
detail=payload.get("detail"),
confidence=CitationConfidence(payload["confidence"])
if payload.get("confidence")
else None,
)
elif action == "delete":
await citation_service.delete_citation(
session, actor=actor, tree=tree, citation_id=_uuid(eid)
)
else:
raise _bad(et, action)
else:
raise Conflict(f"unsupported entity type '{et}'")
+182
View File
@@ -0,0 +1,182 @@
"""Citation service. A citation links one Source to exactly one fact (person,
event, name, or relationship) within a tree — the provenance spine."""
import uuid
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import CitationConfidence
from app.models.event import Event
from app.models.person import Name, Person
from app.models.relationship import Relationship
from app.models.source import Citation, Source
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Conflict, Forbidden, NotFound
# Citation target column -> model, for tenant/existence validation.
_TARGET_MODELS = {
"person_id": Person,
"event_id": Event,
"name_id": Name,
"relationship_id": Relationship,
}
async def _in_tree(session: AsyncSession, model: type, id_: uuid.UUID, tree_id: uuid.UUID) -> bool:
row = (
await session.execute(
select(model.id).where(
model.id == id_, model.tree_id == tree_id, model.deleted_at.is_(None)
)
)
).scalar_one_or_none()
return row is not None
async def create_citation(
session: AsyncSession,
*,
actor: User,
tree: Tree,
source_id: uuid.UUID,
person_id: uuid.UUID | None = None,
event_id: uuid.UUID | None = None,
name_id: uuid.UUID | None = None,
relationship_id: uuid.UUID | None = None,
page: str | None = None,
detail: str | None = None,
confidence: CitationConfidence | None = None,
) -> Citation:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
targets = {
"person_id": person_id,
"event_id": event_id,
"name_id": name_id,
"relationship_id": relationship_id,
}
set_targets = {k: v for k, v in targets.items() if v is not None}
if len(set_targets) != 1:
raise Conflict("a citation must reference exactly one fact")
if not await _in_tree(session, Source, source_id, tree.id):
raise NotFound("source not found in this tree")
(target_col, target_id), = set_targets.items()
if not await _in_tree(session, _TARGET_MODELS[target_col], target_id, tree.id):
raise NotFound("cited fact not found in this tree")
citation = Citation(
tree_id=tree.id,
source_id=source_id,
person_id=person_id,
event_id=event_id,
name_id=name_id,
relationship_id=relationship_id,
page=page,
detail=detail,
confidence=confidence,
)
session.add(citation)
await session.flush()
record_audit(
session,
action="create",
entity_type="Citation",
entity_id=citation.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"source_id": str(source_id), target_col: str(target_id)},
)
await session.commit()
await session.refresh(citation)
return citation
async def list_citations(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree
) -> list[Citation]:
"""All citations in the tree — the UI maps them to facts to show 'sourced'
indicators in a single round-trip."""
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members get only citations whose cited fact resolves to a full-
# visibility person — a citation on a redacted living person's fact would
# otherwise leak that the person has that sourced fact.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_citations(
session, viewer_id=viewer_id, tree=tree
)
stmt = (
select(Citation)
.where(Citation.tree_id == tree.id, Citation.deleted_at.is_(None))
.order_by(Citation.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def update_citation(
session: AsyncSession, *, actor: User, tree: Tree, citation_id: uuid.UUID, changes: dict
) -> Citation:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
citation = (
await session.execute(
select(Citation).where(
Citation.id == citation_id,
Citation.tree_id == tree.id,
Citation.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if citation is None:
raise NotFound("citation not found")
for key in {"page", "detail", "confidence"} & changes.keys():
setattr(citation, key, changes[key])
record_audit(
session,
action="update",
entity_type="Citation",
entity_id=citation.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(citation)
return citation
async def delete_citation(
session: AsyncSession, *, actor: User, tree: Tree, citation_id: uuid.UUID
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
citation = (
await session.execute(
select(Citation).where(
Citation.id == citation_id,
Citation.tree_id == tree.id,
Citation.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if citation is None:
raise NotFound("citation not found")
citation.deleted_at = datetime.now(UTC)
record_audit(
session,
action="delete",
entity_type="Citation",
entity_id=citation.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
+382
View File
@@ -0,0 +1,382 @@
"""Bulk tree cleanup — preview/apply pairs for common import messes.
Per the project's #1 rule (the assistant proposes, humans approve), each fix has
a *preview* that returns the proposed changes and an *apply* that commits only
the ids/edits the user confirmed. Nothing here mutates without an explicit apply
call carrying the user's selections.
"""
import re
import uuid
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import RelationshipType
from app.models.event import Event
from app.models.person import Name, Person
from app.models.relationship import Relationship
from app.models.tree import Tree
from app.models.user import User
from app.services import gedcom, privacy
from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound
from app.services.name_gender_data import guess_sex
async def _require_editor(session: AsyncSession, *, actor: User, tree: Tree) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
async def _persons(session: AsyncSession, tree_id: uuid.UUID) -> list[Person]:
return list(
(
await session.execute(
select(Person).where(Person.tree_id == tree_id, Person.deleted_at.is_(None))
)
).scalars().all()
)
async def _primary_name_by_person(
session: AsyncSession, tree_id: uuid.UUID
) -> dict[uuid.UUID, Name]:
names = (
await session.execute(
select(Name)
.where(Name.tree_id == tree_id, Name.deleted_at.is_(None))
.order_by(Name.is_primary.desc(), Name.sort_order)
)
).scalars().all()
out: dict[uuid.UUID, Name] = {}
for n in names:
out.setdefault(n.person_id, n)
return out
async def _birth_year_by_person(session: AsyncSession, tree_id: uuid.UUID) -> dict[uuid.UUID, int]:
evs = (
await session.execute(
select(Event).where(
Event.tree_id == tree_id,
Event.deleted_at.is_(None),
Event.event_type == "birth",
)
)
).scalars().all()
out: dict[uuid.UUID, int] = {}
for e in evs:
if not e.person_id or e.person_id in out:
continue
y = e.date_start.year if e.date_start else None
if y is None:
ys = gedcom._year(e.date_value)
y = int(ys) if ys else None
if y is not None:
out[e.person_id] = y
return out
def _display(n: Name | None) -> str:
if n is None:
return "Unnamed"
return " ".join(x for x in (n.given, n.surname) if x) or (n.display_name or "Unnamed")
# ---- 1. Mark deceased by birth year -------------------------------------------------
async def preview_deceased(
session: AsyncSession, *, actor: User, tree: Tree, year: int
) -> list[dict]:
await _require_editor(session, actor=actor, tree=tree)
names = await _primary_name_by_person(session, tree.id)
years = await _birth_year_by_person(session, tree.id)
out: list[dict] = []
for p in await _persons(session, tree.id):
if p.is_living is False: # already deceased
continue
by = years.get(p.id)
if by is not None and by <= year:
out.append(
{"person_id": str(p.id), "name": _display(names.get(p.id)), "birth_year": by}
)
out.sort(key=lambda r: r["birth_year"])
return out
async def apply_deceased(
session: AsyncSession, *, actor: User, tree: Tree, person_ids: list[uuid.UUID]
) -> int:
await _require_editor(session, actor=actor, tree=tree)
persons = (
await session.execute(
select(Person).where(
Person.tree_id == tree.id,
Person.deleted_at.is_(None),
Person.id.in_(person_ids),
)
)
).scalars().all()
for p in persons:
p.is_living = False
record_audit(
session,
action="cleanup_deceased",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"count": len(persons)},
)
await session.commit()
return len(persons)
# ---- 1b. Mark deceased by a CHILD's birth year -------------------------------------
# For parents whose own birth date is missing (so the birth-year rule can't reach
# them) but who have a child born long ago — they're necessarily deceased. Applies
# through the same apply_deceased() path.
async def preview_deceased_by_child(
session: AsyncSession, *, actor: User, tree: Tree, year: int
) -> list[dict]:
await _require_editor(session, actor=actor, tree=tree)
names = await _primary_name_by_person(session, tree.id)
years = await _birth_year_by_person(session, tree.id)
rels = (
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
Relationship.type == RelationshipType.parent_child,
)
)
).scalars().all()
# parent id -> earliest child birth year, among children born on/before `year`.
earliest_child: dict[uuid.UUID, int] = {}
for r in rels:
cy = years.get(r.person_to_id) # the child's birth year
if cy is None or cy > year:
continue
if r.person_from_id not in earliest_child or cy < earliest_child[r.person_from_id]:
earliest_child[r.person_from_id] = cy
persons = {p.id: p for p in await _persons(session, tree.id)}
out: list[dict] = []
for parent_id, cy in earliest_child.items():
p = persons.get(parent_id)
if p is None or p.is_living is False: # gone or already deceased
continue
out.append(
{
"person_id": str(parent_id),
"name": _display(names.get(parent_id)),
"child_birth_year": cy,
}
)
out.sort(key=lambda r: r["child_birth_year"])
return out
# ---- 2. Re-derive gender from a source GEDCOM (matches by name) ----------------------
async def preview_gender(
session: AsyncSession, *, actor: User, tree: Tree, gedcom_text: str
) -> list[dict]:
await _require_editor(session, actor=actor, tree=tree)
name2sex: dict[str, str] = {}
for rec in gedcom.parse_records(gedcom_text):
if rec.tag != "INDI":
continue
summ = gedcom._person_summary(rec)
sex = gedcom._sex(rec.text("SEX"))
if sex and summ["norm"]:
name2sex.setdefault(summ["norm"], sex)
names = await _primary_name_by_person(session, tree.id)
out: list[dict] = []
for p in await _persons(session, tree.id):
if p.gender: # only fill in what's missing
continue
nm = names.get(p.id)
if nm is None:
continue
proposed = name2sex.get(gedcom._norm(nm.given, nm.surname))
if proposed:
out.append({"person_id": str(p.id), "name": _display(nm), "proposed_gender": proposed})
out.sort(key=lambda r: r["name"])
return out
async def guess_gender_by_name(
session: AsyncSession, *, actor: User, tree: Tree
) -> list[dict]:
"""Best-guess sex from the first given name for people who don't have it set,
using the bundled name dictionary. Ambiguous/unknown names are skipped."""
await _require_editor(session, actor=actor, tree=tree)
names = await _primary_name_by_person(session, tree.id)
out: list[dict] = []
for p in await _persons(session, tree.id):
if p.gender:
continue
nm = names.get(p.id)
if nm is None:
continue
proposed = guess_sex(nm.given)
if proposed:
out.append({"person_id": str(p.id), "name": _display(nm), "proposed_gender": proposed})
out.sort(key=lambda r: r["name"])
return out
async def guess_gender_by_spouse(
session: AsyncSession, *, actor: User, tree: Tree
) -> list[dict]:
"""Infer the sex of a person who has none set from a partner whose sex IS set
(couples in a tree are opposite-sex in practice — e.g. a confirmed-male
husband implies a female wife). People whose known partners disagree are
ambiguous and skipped; the result is a preview to review, not an auto-write."""
await _require_editor(session, actor=actor, tree=tree)
persons = await _persons(session, tree.id)
gender = {p.id: p.gender for p in persons}
names = await _primary_name_by_person(session, tree.id)
rels = (
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
Relationship.type == RelationshipType.partnership,
)
)
).scalars().all()
opp = {"male": "female", "female": "male"}
proposals: dict[uuid.UUID, set[str]] = {}
for r in rels:
for me_id, other_id in (
(r.person_from_id, r.person_to_id),
(r.person_to_id, r.person_from_id),
):
if gender.get(me_id):
continue # this person already has a sex
other_sex = str(gender.get(other_id) or "")
if other_sex in opp:
proposals.setdefault(me_id, set()).add(opp[other_sex])
out: list[dict] = []
for pid, sexes in proposals.items():
if len(sexes) != 1:
continue # partners of differing known sex → ambiguous
nm = names.get(pid)
if nm is None:
continue
out.append(
{"person_id": str(pid), "name": _display(nm), "proposed_gender": next(iter(sexes))}
)
out.sort(key=lambda r: r["name"])
return out
async def apply_gender(
session: AsyncSession, *, actor: User, tree: Tree, updates: list[dict]
) -> int:
"""updates: [{person_id, gender}]."""
await _require_editor(session, actor=actor, tree=tree)
wanted = {uuid.UUID(str(u["person_id"])): u["gender"] for u in updates if u.get("gender")}
persons = (
await session.execute(
select(Person).where(
Person.tree_id == tree.id,
Person.deleted_at.is_(None),
Person.id.in_(wanted.keys()),
)
)
).scalars().all()
for p in persons:
p.gender = wanted[p.id]
record_audit(
session,
action="cleanup_gender",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"count": len(persons)},
)
await session.commit()
return len(persons)
# ---- 3. Flag malformed names for review --------------------------------------------
_YEAR_RE = re.compile(r"\b\d{3,4}\b")
def _name_issue(n: Name) -> str | None:
given = (n.given or "").strip()
surname = (n.surname or "").strip()
if _YEAR_RE.search(surname) or re.search(r"\d", surname):
return "date_in_surname"
if re.search(r"\d", given):
return "date_in_given"
# A given name with many tokens often means a maiden+married name was packed
# in (e.g. "Mary Smith Jones") — surface it for a human to split.
if surname == "" and len(given.split()) >= 2:
return "no_surname"
if len(given.split()) >= 3:
return "packed_given"
return None
async def preview_names(session: AsyncSession, *, actor: User, tree: Tree) -> list[dict]:
await _require_editor(session, actor=actor, tree=tree)
names = (
await session.execute(
select(Name).where(Name.tree_id == tree.id, Name.deleted_at.is_(None))
)
).scalars().all()
out: list[dict] = []
for n in names:
issue = _name_issue(n)
if issue:
out.append({
"name_id": str(n.id),
"person_id": str(n.person_id),
"given": n.given,
"surname": n.surname,
"issue": issue,
})
return out
async def apply_names(
session: AsyncSession, *, actor: User, tree: Tree, edits: list[dict]
) -> int:
"""edits: [{name_id, given, surname}] — the user's corrected values."""
await _require_editor(session, actor=actor, tree=tree)
by_id = {uuid.UUID(str(e["name_id"])): e for e in edits}
rows = (
await session.execute(
select(Name).where(
Name.tree_id == tree.id,
Name.deleted_at.is_(None),
Name.id.in_(by_id.keys()),
)
)
).scalars().all()
if len(rows) != len(by_id):
raise NotFound("one or more names not found in this tree")
for n in rows:
e = by_id[n.id]
n.given = (e.get("given") or "").strip() or None
n.surname = (e.get("surname") or "").strip() or None
n.display_name = None # rebuild from parts
record_audit(
session,
action="cleanup_names",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"count": len(rows)},
)
await session.commit()
return len(rows)
+221
View File
@@ -0,0 +1,221 @@
"""Event service. Writes require editor rights; reads go through the privacy
engine. Every event has exactly one subject — a Person or a partnership."""
import uuid
from datetime import date
from sqlalchemy import or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import RelationshipType
from app.models.event import Event
from app.models.person import Person
from app.models.place import Place
from app.models.relationship import Relationship
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Conflict, Forbidden, NotFound
async def _belongs_to_tree(
session: AsyncSession, model: type, id_: uuid.UUID, tree_id: uuid.UUID
) -> bool:
row = (
await session.execute(
select(model.id).where(
model.id == id_, model.tree_id == tree_id, model.deleted_at.is_(None)
)
)
).scalar_one_or_none()
return row is not None
async def create_event(
session: AsyncSession,
*,
actor: User,
tree: Tree,
event_type: str,
person_id: uuid.UUID | None = None,
relationship_id: uuid.UUID | None = None,
place_id: uuid.UUID | None = None,
date_value: str | None = None,
date_start: date | None = None,
date_end: date | None = None,
date_precision: str | None = None,
calendar: str = "gregorian",
detail: str | None = None,
notes: str | None = None,
) -> Event:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
if bool(person_id) == bool(relationship_id):
raise Conflict("an event needs exactly one subject: person_id or relationship_id")
if person_id and not await _belongs_to_tree(session, Person, person_id, tree.id):
raise NotFound("person not found in this tree")
if relationship_id and not await _belongs_to_tree(
session, Relationship, relationship_id, tree.id
):
raise NotFound("relationship not found in this tree")
if place_id and not await _belongs_to_tree(session, Place, place_id, tree.id):
raise NotFound("place not found in this tree")
event = Event(
tree_id=tree.id,
event_type=event_type,
person_id=person_id,
relationship_id=relationship_id,
place_id=place_id,
date_value=date_value,
date_start=date_start,
date_end=date_end,
date_precision=date_precision,
calendar=calendar,
detail=detail,
notes=notes,
)
session.add(event)
await session.flush()
record_audit(
session,
action="create",
entity_type="Event",
entity_id=event.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"event_type": event_type, "person_id": str(person_id) if person_id else None},
)
await session.commit()
await session.refresh(event)
return event
async def list_events(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree
) -> list[Event]:
"""All events in the tree — lets the family view compute birth/death years."""
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members get the redacted projection (no living-person dates).
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_events(
session, viewer_id=viewer_id, tree=tree
)
stmt = (
select(Event)
.where(Event.tree_id == tree.id, Event.deleted_at.is_(None))
.order_by(Event.date_start.nulls_last(), Event.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def list_events_for_person(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, person_id: uuid.UUID
) -> list[Event]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members only see a full-visibility person's events (redacted → none).
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_person_events(
session, viewer_id=viewer_id, tree=tree, person_id=person_id
)
# Member view: this person's own events PLUS their partnership events (which
# live on the relationship and show on both partners). Returning both here
# means the person page doesn't have to load every event in the tree.
partner_rel_ids = (
select(Relationship.id)
.where(
Relationship.tree_id == tree.id,
Relationship.type == RelationshipType.partnership,
Relationship.deleted_at.is_(None),
or_(
Relationship.person_from_id == person_id,
Relationship.person_to_id == person_id,
),
)
)
stmt = (
select(Event)
.where(
Event.tree_id == tree.id,
Event.deleted_at.is_(None),
or_(
Event.person_id == person_id,
Event.relationship_id.in_(partner_rel_ids),
),
)
.order_by(Event.date_start.nulls_last(), Event.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def update_event(
session: AsyncSession,
*,
actor: User,
tree: Tree,
event_id: uuid.UUID,
changes: dict,
) -> Event:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
event = (
await session.execute(
select(Event).where(
Event.id == event_id, Event.tree_id == tree.id, Event.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if event is None:
raise NotFound("event not found")
if "place_id" in changes and changes["place_id"] is not None:
if not await _belongs_to_tree(session, Place, changes["place_id"], tree.id):
raise NotFound("place not found in this tree")
for key, value in changes.items():
setattr(event, key, value)
record_audit(
session,
action="update",
entity_type="Event",
entity_id=event.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(event)
return event
async def delete_event(
session: AsyncSession, *, actor: User, tree: Tree, event_id: uuid.UUID
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
event = (
await session.execute(
select(Event).where(
Event.id == event_id, Event.tree_id == tree.id, Event.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if event is None:
raise NotFound("event not found")
from datetime import UTC, datetime
event.deleted_at = datetime.now(UTC)
record_audit(
session,
action="delete",
entity_type="Event",
entity_id=event.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
+18
View File
@@ -0,0 +1,18 @@
"""Domain errors. The API layer maps these to HTTP status codes so services
stay transport-agnostic."""
class DomainError(Exception):
"""Base for domain-level errors."""
class NotFound(DomainError):
"""Requested entity does not exist (or is soft-deleted / not visible)."""
class Forbidden(DomainError):
"""Caller lacks the required role for this action."""
class Conflict(DomainError):
"""Operation conflicts with current state (e.g. duplicate email)."""
+841
View File
@@ -0,0 +1,841 @@
"""GEDCOM import/export.
A pragmatic parser + mapper for the common subset of GEDCOM (5.5.1 / 7 share
the line grammar): INDI, FAM, SOUR. Import maps records into a tree and returns
a mapping report (counts + unmapped tags); export serializes the tree back to
GEDCOM. Runs inline for now — large files should move to the worker later.
Import is duplicate-aware: ``preview_gedcom`` reports incoming people that look
like existing ones, and ``import_gedcom`` applies a per-record resolution
(new / skip / merge / overwrite). Names carry their GEDCOM type (a married name
imports as a typed alternate, not a second primary).
"""
import re
import uuid
from collections import defaultdict
from datetime import UTC, date, datetime
from difflib import SequenceMatcher
from sqlalchemy import or_, select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import ParentChildQualifier, RelationshipType
from app.models.event import Event
from app.models.person import Name, Person
from app.models.place import Place
from app.models.relationship import Relationship
from app.models.source import Citation, Source
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Forbidden
# GEDCOM event tag -> our event_type (INDI-level).
INDI_EVENTS = {
"BIRT": "birth", "DEAT": "death", "BAPM": "baptism", "CHR": "christening",
"BURI": "burial", "CREM": "cremation", "RESI": "residence", "CENS": "census",
"IMMI": "immigration", "EMIG": "emigration", "OCCU": "occupation",
"EDUC": "education", "GRAD": "graduation", "RETI": "retirement",
"NATU": "naturalization", "BAPL": "baptism", "RELI": "religion",
}
# INDI attribute tags whose line VALUE is the fact (no date), stored in detail.
VALUE_EVENTS = {"RELI", "OCCU", "EDUC"}
# INDI sub-tags consumed elsewhere or intentionally ignored (not "unmapped").
INDI_SKIP_TAGS = {
"NAME", "SEX", "SOUR", "FAMC", "FAMS", "CHAN", "OBJE", "_UID", "_MARNM", "NOTE",
}
# FAM-level events.
FAM_EVENTS = {"MARR": "marriage", "DIV": "divorce", "ENGA": "engagement"}
EVENT_TO_GED = {v: k for k, v in {**INDI_EVENTS, **FAM_EVENTS}.items()}
# GEDCOM NAME TYPE (or _MARNM-derived) -> our Name.name_type vocabulary.
NAME_TYPE_MAP = {
"birth": "birth", "maiden": "birth", "married": "married",
"aka": "alias", "also known as": "alias", "nickname": "nickname",
"religious": "religious", "immigrant": "immigration",
"immigration": "immigration", "professional": "alias", "other": "alias",
}
# Our type -> GEDCOM TYPE on export (birth is the default; emit nothing).
EXPORT_TYPE_MAP = {
"married": "married", "alias": "aka", "nickname": "nickname",
"religious": "religious", "immigration": "immigrant",
}
class GedcomNode:
__slots__ = ("level", "tag", "value", "xref", "children")
def __init__(self, level: int, tag: str, value: str = "", xref: str | None = None):
self.level = level
self.tag = tag
self.value = value
self.xref = xref
self.children: list[GedcomNode] = []
def first(self, tag: str) -> "GedcomNode | None":
return next((c for c in self.children if c.tag == tag), None)
def all(self, tag: str) -> list["GedcomNode"]:
return [c for c in self.children if c.tag == tag]
def text(self, tag: str, default: str | None = None) -> str | None:
n = self.first(tag)
return n.value if n is not None else default
def parse_records(text: str) -> list[GedcomNode]:
roots: list[GedcomNode] = []
stack: list[GedcomNode] = []
for raw in text.replace("\r\n", "\n").replace("\r", "\n").split("\n"):
line = raw.lstrip("").rstrip()
if not line.strip():
continue
parts = line.split(" ", 1)
try:
level = int(parts[0])
except ValueError:
continue
rest = parts[1] if len(parts) > 1 else ""
xref: str | None = None
if rest.startswith("@"):
end = rest.find("@", 1)
if end != -1:
xref = rest[: end + 1]
rest = rest[end + 1:].strip()
tparts = rest.split(" ", 1)
tag = tparts[0]
value = tparts[1] if len(tparts) > 1 else ""
while stack and stack[-1].level >= level:
stack.pop()
parent = stack[-1] if stack else None
if tag in ("CONC", "CONT") and parent is not None:
parent.value += ("" if tag == "CONC" else "\n") + value
continue
node = GedcomNode(level, tag, value, xref)
if parent is None:
roots.append(node)
else:
parent.children.append(node)
stack.append(node)
return roots
def _parse_name(value: str) -> tuple[str | None, str | None]:
if "/" in value:
given, _, rest = value.partition("/")
surname = rest.split("/", 1)[0]
return given.strip() or None, surname.strip() or None
return value.strip() or None, None
def _parse_marnm(value: str, base_given: str | None) -> tuple[str | None, str | None]:
"""A _MARNM value is sometimes a full name ("Jane /Smith/") and sometimes
just the married surname ("Smith"). Keep the given name from the base name
in the latter case."""
v = (value or "").strip()
if "/" in v:
g, s = _parse_name(v)
return (g or base_given), s
return base_given, (v or None)
def _extract_names(rec: GedcomNode) -> list[dict]:
"""All names for an INDI, typed. Multiple NAME records (each with an optional
TYPE) plus any _MARNM (married name) subtags become separate Name rows. The
first birth/maiden name is primary."""
out: list[dict] = []
for nm in rec.all("NAME"):
g, s = _parse_name(nm.value)
t = (nm.text("TYPE") or "").strip().lower()
ntype = NAME_TYPE_MAP.get(t, t or "birth")
out.append({"type": ntype, "given": g, "surname": s, "display": nm.value or None,
"nickname": nm.text("NICK")})
for mar in nm.all("_MARNM"):
mg, ms = _parse_marnm(mar.value, g)
out.append({"type": "married", "given": mg, "surname": ms,
"display": mar.value or None, "nickname": None})
for mar in rec.all("_MARNM"):
base_g = out[0]["given"] if out else None
mg, ms = _parse_marnm(mar.value, base_g)
out.append({"type": "married", "given": mg, "surname": ms,
"display": mar.value or None, "nickname": None})
if not out:
return out
primary_idx = next((i for i, n in enumerate(out) if n["type"] == "birth"), 0)
for i, n in enumerate(out):
n["is_primary"] = i == primary_idx
n["sort"] = i
return out
def _norm(given: str | None, surname: str | None) -> str:
return re.sub(r"\s+", " ", f"{given or ''} {surname or ''}".strip().lower())
def _year(date_value: str | None) -> str | None:
if not date_value:
return None
m = re.search(r"\b(\d{3,4})\b", date_value)
return m.group(1) if m else None
def _date_start(date_value: str | None) -> date | None:
y = _year(date_value)
if not y:
return None
try:
return date(int(y), 1, 1)
except ValueError:
return None
def _sex(value: str | None) -> str | None:
if not value:
return None
v = value.strip().upper()
return {"M": "male", "F": "female"}.get(v, value.strip().lower() or None)
def _notes_text(rec: GedcomNode) -> str | None:
"""Join an INDI's NOTE lines (which pack confidence / findagrave / fs_pid /
free text) into the person's notes field."""
vals = [n.value.strip() for n in rec.all("NOTE") if n.value and n.value.strip()]
return "\n".join(vals) or None
def _person_summary(rec: GedcomNode) -> dict:
"""Display name + birth year for an incoming INDI, for duplicate matching."""
names = _extract_names(rec)
primary = next((n for n in names if n.get("is_primary")), names[0] if names else None)
g = primary["given"] if primary else None
s = primary["surname"] if primary else None
disp = " ".join(x for x in (g, s) if x)
if not disp and primary:
disp = primary.get("display") or ""
birth = rec.first("BIRT")
year = _year(birth.text("DATE")) if birth else None
return {"names": names, "norm": _norm(g, s), "name": disp or "(no name)", "year": year}
async def _build_existing_index(session: AsyncSession, tree: Tree) -> list[dict]:
"""Existing (non-deleted) people with a display name + birth year, for
matching incoming records against."""
persons = list(
(
await session.execute(
select(Person).where(Person.tree_id == tree.id, Person.deleted_at.is_(None))
)
).scalars().all()
)
names = list(
(
await session.execute(
select(Name).where(Name.tree_id == tree.id, Name.deleted_at.is_(None))
)
).scalars().all()
)
name_by_person: dict[uuid.UUID, Name] = {}
for n in sorted(names, key=lambda n: (not n.is_primary, n.sort_order)):
name_by_person.setdefault(n.person_id, n)
births = list(
(
await session.execute(
select(Event).where(
Event.tree_id == tree.id,
Event.deleted_at.is_(None),
Event.event_type == "birth",
)
)
).scalars().all()
)
year_by_person: dict[uuid.UUID, str] = {}
for e in births:
if e.person_id and e.person_id not in year_by_person:
y = str(e.date_start.year) if e.date_start else _year(e.date_value)
if y:
year_by_person[e.person_id] = y
index: list[dict] = []
for p in persons:
nm = name_by_person.get(p.id)
g = nm.given if nm else None
s = nm.surname if nm else None
disp = " ".join(x for x in (g, s) if x) or (nm.display_name if nm else None)
index.append({
"id": p.id,
"norm": _norm(g, s),
"name": disp or "(no name)",
"year": year_by_person.get(p.id),
})
return index
def _best_match(norm: str, year: str | None, index: list[dict]) -> tuple[dict | None, str | None]:
"""Closest existing person by name similarity, rejecting clear birth-year
conflicts. Returns (entry, "high"|"medium") or (None, None)."""
if not norm:
return None, None
best: dict | None = None
best_r = 0.0
for e in index:
if not e["norm"]:
continue
r = SequenceMatcher(None, norm, e["norm"]).ratio()
if r < 0.88:
continue
if year and e["year"] and abs(int(year) - int(e["year"])) > 1:
continue # same-ish name but different birth year — not a duplicate
if r > best_r:
best_r = r
best = e
if best is None:
return None, None
year_match = bool(year and best["year"] and abs(int(year) - int(best["year"])) <= 1)
both_unknown = not year and not best["year"]
score = "high" if best_r >= 0.93 and (year_match or both_unknown) else "medium"
return best, score
def _relkey(rtype: RelationshipType, a: uuid.UUID, b: uuid.UUID) -> tuple:
if rtype == RelationshipType.parent_child:
return ("pc", str(a), str(b))
return (rtype.value, *sorted([str(a), str(b)]))
def _count_incoming(roots: list[GedcomNode]) -> tuple[dict, list[str]]:
counts: dict[str, int] = defaultdict(int)
unmapped: set[str] = set()
for rec in roots:
if rec.tag == "INDI" and rec.xref:
counts["persons"] += 1
counts["names"] += len(_extract_names(rec))
for child in rec.children:
if child.tag in INDI_EVENTS:
counts["events"] += 1
elif child.tag not in INDI_SKIP_TAGS:
unmapped.add(child.tag)
elif rec.tag == "FAM":
counts["families"] += 1
for child in rec.children:
if child.tag in FAM_EVENTS:
counts["events"] += 1
elif rec.tag == "SOUR" and rec.xref:
counts["sources"] += 1
return dict(counts), sorted(unmapped)
async def preview_gedcom(session: AsyncSession, *, actor: User, tree: Tree, text: str) -> dict:
"""Dry run: what would import, and which incoming people look like existing
ones. No writes."""
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
roots = parse_records(text)
counts, unmapped = _count_incoming(roots)
index = await _build_existing_index(session, tree)
duplicates: list[dict] = []
for rec in roots:
if rec.tag != "INDI" or not rec.xref:
continue
summ = _person_summary(rec)
entry, score = _best_match(summ["norm"], summ["year"], index)
if entry is None:
continue
duplicates.append({
"xref": rec.xref,
"incoming_name": summ["name"],
"incoming_birth_year": summ["year"],
"existing_person_id": entry["id"],
"existing_name": entry["name"],
"existing_birth_year": entry["year"],
"score": score,
})
return {"counts": counts, "potential_duplicates": duplicates, "unmapped_tags": unmapped}
async def import_gedcom(
session: AsyncSession,
*,
actor: User,
tree: Tree,
text: str,
default_action: str = "new",
resolutions: dict | None = None,
) -> dict:
"""Import records. ``default_action`` (new|skip|merge|overwrite) applies to
incoming people that match an existing one; ``resolutions`` overrides it per
GEDCOM xref ({xref: {action, target_id}}). 'skip' links families to the
existing person but copies nothing; 'merge' also copies the incoming names
(as alternates), events and citations onto them; 'overwrite' deletes the
existing person and imports the incoming one fresh."""
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
resolutions = resolutions or {}
roots = parse_records(text)
counts: dict[str, int] = defaultdict(int)
unmapped: set[str] = set()
place_cache: dict[str, uuid.UUID] = {}
source_map: dict[str, uuid.UUID] = {}
person_map: dict[str, uuid.UUID] = {}
now = datetime.now(UTC)
index = await _build_existing_index(session, tree)
# Pre-load existing relationship keys so a merge doesn't create dup edges.
existing_rels = list(
(
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None)
)
)
).scalars().all()
)
rel_keys = {_relkey(r.type, r.person_from_id, r.person_to_id) for r in existing_rels}
def add_relationship(
rtype: RelationshipType, a: uuid.UUID, b: uuid.UUID, **kw
) -> Relationship | None:
key = _relkey(rtype, a, b)
if key in rel_keys:
return None
rel = Relationship(tree_id=tree.id, type=rtype, person_from_id=a, person_to_id=b, **kw)
session.add(rel)
rel_keys.add(key)
counts["relationships"] += 1
return rel
async def place_id(name: str | None) -> uuid.UUID | None:
if not name:
return None
if name in place_cache:
return place_cache[name]
p = Place(tree_id=tree.id, name=name)
session.add(p)
await session.flush()
place_cache[name] = p.id
counts["places"] += 1
return p.id
# Sources first (so citations can reference them).
for rec in roots:
if rec.tag == "SOUR" and rec.xref:
src = Source(
tree_id=tree.id,
title=rec.text("TITL") or rec.text("ABBR") or "Untitled source",
author=rec.text("AUTH"),
publication_info=rec.text("PUBL"),
citation_text=rec.text("TEXT"),
)
session.add(src)
await session.flush()
source_map[rec.xref] = src.id
counts["sources"] += 1
async def add_citations(holder: GedcomNode, **target) -> None:
for s in holder.all("SOUR"):
sid = source_map.get(s.value.strip())
if sid is None:
continue
session.add(Citation(tree_id=tree.id, source_id=sid, page=s.text("PAGE"), **target))
counts["citations"] += 1
def add_names(person_id: uuid.UUID, names: list[dict], *, set_primary: bool) -> None:
for nd in names:
session.add(
Name(
tree_id=tree.id,
person_id=person_id,
name_type=nd["type"],
given=nd["given"],
surname=nd["surname"],
nickname=nd.get("nickname"),
display_name=nd.get("display"),
is_primary=set_primary and nd.get("is_primary", False),
sort_order=nd.get("sort", 0),
)
)
counts["names"] += 1
async def add_events(rec: GedcomNode, person_id: uuid.UUID) -> None:
for child in rec.children:
if child.tag in INDI_EVENTS:
dv = child.text("DATE")
# Attribute-style facts (RELI, OCCU, EDUC) carry their value on
# the line itself; store it in detail.
detail = child.value.strip() if child.tag in VALUE_EVENTS else None
ev = Event(
tree_id=tree.id,
person_id=person_id,
event_type=INDI_EVENTS[child.tag],
date_value=dv,
date_start=_date_start(dv),
place_id=await place_id(child.text("PLAC")),
detail=detail or None,
notes=child.text("NOTE"),
)
session.add(ev)
await session.flush()
counts["events"] += 1
await add_citations(child, event_id=ev.id)
elif child.tag in INDI_SKIP_TAGS:
continue
else:
unmapped.add(child.tag)
async def soft_delete_existing(person_id: uuid.UUID) -> None:
p = (
await session.execute(
select(Person).where(Person.id == person_id, Person.deleted_at.is_(None))
)
).scalar_one_or_none()
if p is None:
return
p.deleted_at = now
rels = (
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
or_(
Relationship.person_from_id == person_id,
Relationship.person_to_id == person_id,
),
)
)
).scalars().all()
for r in rels:
r.deleted_at = now
await session.execute(
update(User).where(User.self_person_id == person_id).values(self_person_id=None)
)
# Precompute the best match per incoming xref (for default-policy resolution).
matches: dict[str, dict] = {}
for rec in roots:
if rec.tag == "INDI" and rec.xref:
summ = _person_summary(rec)
entry, _score = _best_match(summ["norm"], summ["year"], index)
if entry is not None:
matches[rec.xref] = entry
def resolve(xref: str) -> tuple[str, uuid.UUID | None]:
ov = resolutions.get(xref)
if ov:
action = ov.get("action", "new")
tid = ov.get("target_id")
target = uuid.UUID(tid) if tid else (matches[xref]["id"] if xref in matches else None)
if action in ("skip", "merge", "overwrite") and target is None:
return "new", None
return action, target
if default_action != "new" and xref in matches:
return default_action, matches[xref]["id"]
return "new", None
# Individuals.
for rec in roots:
if rec.tag != "INDI" or not rec.xref:
continue
names = _extract_names(rec)
action, target = resolve(rec.xref)
if action == "skip" and target is not None:
person_map[rec.xref] = target
counts["skipped"] += 1
continue
if action == "merge" and target is not None:
person_map[rec.xref] = target
add_names(target, names, set_primary=False)
await add_events(rec, target)
await add_citations(rec, person_id=target)
note = _notes_text(rec)
if note:
existing = (
await session.execute(select(Person).where(Person.id == target))
).scalar_one_or_none()
if existing is not None:
existing.notes = "\n".join(filter(None, [existing.notes, note]))
counts["merged"] += 1
continue
if action == "overwrite" and target is not None:
await soft_delete_existing(target)
counts["overwritten"] += 1
person = Person(tree_id=tree.id, gender=_sex(rec.text("SEX")), notes=_notes_text(rec))
session.add(person)
await session.flush()
person_map[rec.xref] = person.id
counts["persons"] += 1
add_names(person.id, names, set_primary=True)
await add_citations(rec, person_id=person.id)
await add_events(rec, person.id)
# Families -> partnerships, parent-child edges, marriage events.
for rec in roots:
if rec.tag != "FAM":
continue
counts["families"] += 1
husb = person_map.get((rec.text("HUSB") or "").strip())
wife = person_map.get((rec.text("WIFE") or "").strip())
partnership_id: uuid.UUID | None = None
if husb and wife and husb != wife:
rel = add_relationship(RelationshipType.partnership, husb, wife)
if rel is not None:
await session.flush()
partnership_id = rel.id
if partnership_id is None and husb and wife:
# Edge already existed — find it so marriage events can attach.
existing = next(
(
r for r in existing_rels
if r.type == RelationshipType.partnership
and {r.person_from_id, r.person_to_id} == {husb, wife}
),
None,
)
partnership_id = existing.id if existing else None
for fe in rec.children:
if fe.tag in FAM_EVENTS and partnership_id is not None:
dv = fe.text("DATE")
ev = Event(
tree_id=tree.id,
relationship_id=partnership_id,
event_type=FAM_EVENTS[fe.tag],
date_value=dv,
date_start=_date_start(dv),
place_id=await place_id(fe.text("PLAC")),
)
session.add(ev)
await session.flush()
counts["events"] += 1
for chil in rec.all("CHIL"):
cp = person_map.get(chil.value.strip())
if cp is None:
continue
for parent in (husb, wife):
if parent and parent != cp:
add_relationship(
RelationshipType.parent_child,
parent,
cp,
qualifier=ParentChildQualifier.biological,
)
record_audit(
session,
action="import",
entity_type="Gedcom",
tree_id=tree.id,
actor_user_id=actor.id,
after=dict(counts),
)
await session.commit()
return {"counts": dict(counts), "unmapped_tags": sorted(unmapped)}
def _ged_date(value: str | None) -> str | None:
return value.strip() if value else None
async def export_gedcom(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> str:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
persons = list(
(
await session.execute(
select(Person).where(Person.tree_id == tree.id, Person.deleted_at.is_(None))
)
).scalars().all()
)
names = list(
(
await session.execute(
select(Name).where(Name.tree_id == tree.id, Name.deleted_at.is_(None))
)
).scalars().all()
)
events = list(
(
await session.execute(
select(Event).where(Event.tree_id == tree.id, Event.deleted_at.is_(None))
)
).scalars().all()
)
rels = list(
(
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None)
)
)
).scalars().all()
)
sources = list(
(
await session.execute(
select(Source).where(Source.tree_id == tree.id, Source.deleted_at.is_(None))
)
).scalars().all()
)
places = {
p.id: p
for p in (
await session.execute(select(Place).where(Place.tree_id == tree.id))
).scalars().all()
}
citations = list(
(
await session.execute(
select(Citation).where(
Citation.tree_id == tree.id, Citation.deleted_at.is_(None)
)
)
).scalars().all()
)
pxref = {p.id: f"@I{i + 1}@" for i, p in enumerate(persons)}
gender_by_id = {p.id: p.gender for p in persons}
sxref = {s.id: f"@S{i + 1}@" for i, s in enumerate(sources)}
# Citations grouped by the fact they sit on, so each fact can emit its SOUR
# links (dropping these is the round-trip data loss this fixes). Skip any
# whose source didn't export.
cite_by_person: dict[uuid.UUID, list[Citation]] = defaultdict(list)
cite_by_name: dict[uuid.UUID, list[Citation]] = defaultdict(list)
cite_by_event: dict[uuid.UUID, list[Citation]] = defaultdict(list)
cite_by_rel: dict[uuid.UUID, list[Citation]] = defaultdict(list)
for c in citations:
if c.source_id not in sxref:
continue
if c.person_id:
cite_by_person[c.person_id].append(c)
elif c.event_id:
cite_by_event[c.event_id].append(c)
elif c.name_id:
cite_by_name[c.name_id].append(c)
elif c.relationship_id:
cite_by_rel[c.relationship_id].append(c)
def cite_lines(cites: list[Citation], depth: int) -> list[str]:
lines: list[str] = []
for c in cites:
lines.append(f"{depth} SOUR {sxref[c.source_id]}")
if c.page:
lines.append(f"{depth + 1} PAGE {c.page}")
return lines
names_by_person: dict[uuid.UUID, list[Name]] = defaultdict(list)
for n in sorted(names, key=lambda n: (n.sort_order, not n.is_primary)):
names_by_person[n.person_id].append(n)
events_by_person: dict[uuid.UUID, list[Event]] = defaultdict(list)
events_by_rel: dict[uuid.UUID, list[Event]] = defaultdict(list)
for e in events:
if e.person_id:
events_by_person[e.person_id].append(e)
elif e.relationship_id:
events_by_rel[e.relationship_id].append(e)
# Build families from parent-child + partnership edges (group by parent set).
parents_of: dict[uuid.UUID, set[uuid.UUID]] = defaultdict(set)
for r in rels:
if r.type == RelationshipType.parent_child:
parents_of[r.person_to_id].add(r.person_from_id)
fams: dict[frozenset, dict] = {}
for child, ps in parents_of.items():
key = frozenset(ps)
fams.setdefault(key, {"parents": set(ps), "children": [], "rel_id": None})
fams[key]["children"].append(child)
for r in rels:
if r.type == RelationshipType.partnership:
key = frozenset({r.person_from_id, r.person_to_id})
fam = fams.setdefault(
key,
{"parents": {r.person_from_id, r.person_to_id}, "children": [], "rel_id": None},
)
fam["rel_id"] = r.id
fam_list = list(fams.values())
fxref = {id(f): f"@F{i + 1}@" for i, f in enumerate(fam_list)}
# person -> the families they are a spouse in / a child in
spouse_fams: dict[uuid.UUID, list[str]] = defaultdict(list)
child_fams: dict[uuid.UUID, str] = {}
for f in fam_list:
x = fxref[id(f)]
for pid in f["parents"]:
spouse_fams[pid].append(x)
for cid in f["children"]:
child_fams[cid] = x
out: list[str] = ["0 HEAD", "1 SOUR Provenance", "1 GEDC", "2 VERS 5.5.1", "1 CHAR UTF-8"]
for p in persons:
out.append(f"0 {pxref[p.id]} INDI")
for n in names_by_person.get(p.id, []):
display = n.display_name or f"{n.given or ''} /{n.surname or ''}/".strip()
out.append(f"1 NAME {display}")
ged_type = EXPORT_TYPE_MAP.get(n.name_type)
if ged_type:
out.append(f"2 TYPE {ged_type}")
out += cite_lines(cite_by_name.get(n.id, []), 2)
sex = {"male": "M", "female": "F"}.get(p.gender or "")
if sex:
out.append(f"1 SEX {sex}")
for e in events_by_person.get(p.id, []):
tag = EVENT_TO_GED.get(e.event_type)
if not tag:
continue
out.append(f"1 {tag}")
if _ged_date(e.date_value):
out.append(f"2 DATE {e.date_value}")
if e.place_id and e.place_id in places:
out.append(f"2 PLAC {places[e.place_id].name}")
out += cite_lines(cite_by_event.get(e.id, []), 2)
out += cite_lines(cite_by_person.get(p.id, []), 1)
if p.id in child_fams:
out.append(f"1 FAMC {child_fams[p.id]}")
for x in spouse_fams.get(p.id, []):
out.append(f"1 FAMS {x}")
for f in fam_list:
x = fxref[id(f)]
out.append(f"0 {x} FAM")
ps = list(f["parents"])
# HUSB/WIFE by recorded gender where possible.
males = [pid for pid in ps if gender_by_id.get(pid) == "male"]
females = [pid for pid in ps if gender_by_id.get(pid) == "female"]
husb = males[0] if males else (ps[0] if ps else None)
wife = females[0] if females else next((pid for pid in ps if pid != husb), None)
if husb:
out.append(f"1 HUSB {pxref[husb]}")
if wife:
out.append(f"1 WIFE {pxref[wife]}")
for cid in f["children"]:
out.append(f"1 CHIL {pxref[cid]}")
if f["rel_id"]:
for e in events_by_rel.get(f["rel_id"], []):
tag = EVENT_TO_GED.get(e.event_type)
if not tag:
continue
out.append(f"1 {tag}")
if _ged_date(e.date_value):
out.append(f"2 DATE {e.date_value}")
out += cite_lines(cite_by_event.get(e.id, []), 2)
out += cite_lines(cite_by_rel.get(f["rel_id"], []), 1)
for s in sources:
out.append(f"0 {sxref[s.id]} SOUR")
if s.title:
out.append(f"1 TITL {s.title}")
if s.author:
out.append(f"1 AUTH {s.author}")
if s.publication_info:
out.append(f"1 PUBL {s.publication_info}")
out.append("0 TRLR")
return "\n".join(out) + "\n"
+170
View File
@@ -0,0 +1,170 @@
"""Media service. Bytes go to the ObjectStore; a metadata row goes to the DB.
Writes require editor rights; reads go through the privacy engine."""
import hashlib
import uuid
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.integrations.objectstore.base import ObjectStore
from app.models.media import Media
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound
async def upload_media(
session: AsyncSession,
store: ObjectStore,
*,
actor: User,
tree: Tree,
data: bytes,
filename: str,
content_type: str,
title: str | None = None,
person_id: uuid.UUID | None = None,
event_id: uuid.UUID | None = None,
source_id: uuid.UUID | None = None,
) -> Media:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
media_id = uuid.uuid4()
key = f"{tree.id}/{media_id}/{filename}"
await store.ensure_bucket()
await store.put_object(key=key, data=data, content_type=content_type)
media = Media(
id=media_id,
tree_id=tree.id,
uploader_id=actor.id,
storage_key=key,
original_filename=filename,
content_type=content_type,
byte_size=len(data),
checksum_sha256=hashlib.sha256(data).hexdigest(),
title=title,
person_id=person_id,
event_id=event_id,
source_id=source_id,
)
session.add(media)
await session.flush()
record_audit(
session,
action="create",
entity_type="Media",
entity_id=media.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"filename": filename, "bytes": len(data)},
)
await session.commit()
await session.refresh(media)
return media
async def list_media(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> list[Media]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members only see media of a FULL-visibility person (no living-person photos).
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_media(
session, viewer_id=viewer_id, tree=tree
)
stmt = (
select(Media)
.where(Media.tree_id == tree.id, Media.deleted_at.is_(None))
.order_by(Media.created_at.desc())
)
return list((await session.execute(stmt)).scalars().all())
async def get_media(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, media_id: uuid.UUID
) -> Media:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
media = (
await session.execute(
select(Media).where(
Media.id == media_id, Media.tree_id == tree.id, Media.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if media is None:
raise NotFound("media not found")
# Non-members may only see/download media of a FULL-visibility person. 404
# (not 403) so the item's existence isn't revealed. This gates media_content.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
if not await public_view_service.can_view_media(
session, viewer_id=viewer_id, tree=tree, media=media
):
raise NotFound("media not found")
return media
async def update_media(
session: AsyncSession, *, actor: User, tree: Tree, media_id: uuid.UUID, changes: dict
) -> Media:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
media = (
await session.execute(
select(Media).where(
Media.id == media_id, Media.tree_id == tree.id, Media.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if media is None:
raise NotFound("media not found")
for key in {"title", "person_id", "event_id", "source_id"} & changes.keys():
setattr(media, key, changes[key])
record_audit(
session,
action="update",
entity_type="Media",
entity_id=media.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(media)
return media
async def delete_media(
session: AsyncSession, *, actor: User, tree: Tree, media_id: uuid.UUID
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
media = (
await session.execute(
select(Media).where(
Media.id == media_id, Media.tree_id == tree.id, Media.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if media is None:
raise NotFound("media not found")
# Soft delete the row; the object is removed by the worker's purge job.
media.deleted_at = datetime.now(UTC)
record_audit(
session,
action="delete",
entity_type="Media",
entity_id=media.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
+156
View File
@@ -0,0 +1,156 @@
"""Tree membership management: list / add / change-role / remove.
Only an owner may change membership. A tree must always keep at least one owner.
The member list (which exposes user emails) is visible only to members never
to a non-member viewing a public/unlisted tree.
"""
import uuid
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import MembershipRole
from app.models.tree import Tree, TreeMembership
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Conflict, Forbidden, NotFound
async def _require_owner(session: AsyncSession, *, actor_id: uuid.UUID, tree: Tree) -> None:
if await privacy.get_membership_role(session, actor_id, tree.id) is not MembershipRole.owner:
raise Forbidden("only the owner can manage members")
async def _owner_count(session: AsyncSession, tree_id: uuid.UUID) -> int:
return (
await session.execute(
select(func.count())
.select_from(TreeMembership)
.where(TreeMembership.tree_id == tree_id, TreeMembership.role == MembershipRole.owner)
)
).scalar_one()
def _row(m: TreeMembership, u: User) -> dict:
return {
"id": m.id,
"user_id": u.id,
"email": u.email,
"display_name": u.display_name,
"role": m.role,
"created_at": m.created_at,
}
async def list_members(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> list[dict]:
# Member-only: the list exposes emails, so a non-member (even on a public
# tree) must not see it.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
raise Forbidden("only members can see the member list")
rows = (
await session.execute(
select(TreeMembership, User)
.join(User, User.id == TreeMembership.user_id)
.where(TreeMembership.tree_id == tree.id)
.order_by(TreeMembership.created_at)
)
).all()
return [_row(m, u) for m, u in rows]
async def add_member(
session: AsyncSession, *, actor: User, tree: Tree, email: str, role: MembershipRole
) -> dict:
await _require_owner(session, actor_id=actor.id, tree=tree)
user = (
await session.execute(
select(User).where(User.email == email, User.deleted_at.is_(None))
)
).scalar_one_or_none()
if user is None:
raise NotFound("no user with that email on this instance")
if await privacy.get_membership_role(session, user.id, tree.id) is not None:
raise Conflict("that user is already a member")
m = TreeMembership(tree_id=tree.id, user_id=user.id, role=role)
session.add(m)
record_audit(
session,
action="add_member",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"user_id": str(user.id), "role": role.value},
)
await session.commit()
await session.refresh(m)
return _row(m, user)
async def _get_membership(
session: AsyncSession, tree: Tree, membership_id: uuid.UUID
) -> TreeMembership:
m = (
await session.execute(
select(TreeMembership).where(
TreeMembership.id == membership_id, TreeMembership.tree_id == tree.id
)
)
).scalar_one_or_none()
if m is None:
raise NotFound("member not found")
return m
async def update_member_role(
session: AsyncSession,
*,
actor: User,
tree: Tree,
membership_id: uuid.UUID,
role: MembershipRole,
) -> dict:
await _require_owner(session, actor_id=actor.id, tree=tree)
m = await _get_membership(session, tree, membership_id)
if (
m.role == MembershipRole.owner
and role != MembershipRole.owner
and await _owner_count(session, tree.id) <= 1
):
raise Conflict("a tree must keep at least one owner")
m.role = role
record_audit(
session,
action="update_member",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"membership_id": str(m.id), "role": role.value},
)
await session.commit()
await session.refresh(m)
u = (await session.execute(select(User).where(User.id == m.user_id))).scalar_one()
return _row(m, u)
async def remove_member(
session: AsyncSession, *, actor: User, tree: Tree, membership_id: uuid.UUID
) -> None:
await _require_owner(session, actor_id=actor.id, tree=tree)
m = await _get_membership(session, tree, membership_id)
if m.role == MembershipRole.owner and await _owner_count(session, tree.id) <= 1:
raise Conflict("a tree must keep at least one owner")
await session.delete(m)
record_audit(
session,
action="remove_member",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"membership_id": str(membership_id)},
)
await session.commit()
+69
View File
@@ -0,0 +1,69 @@
"""A curated given-name -> sex lookup for best-guessing a person's sex from
their first name. Weighted toward English + German names (this codebase's first
real tree is a German-American family). Deterministic and offline no model
needed; the Cleanup tool previews every guess before anything is applied.
Genuinely ambiguous names (Marion, Frances/Francis, Jordan, Jamie, Robin, Leslie,
Dana, ) are intentionally left out of BOTH sets so they aren't guessed — better
a human decides those than a coin flip.
"""
MALE_NAMES: set[str] = {
# English / common US
"james", "john", "robert", "michael", "william", "david", "richard", "joseph",
"thomas", "charles", "christopher", "daniel", "matthew", "anthony", "donald",
"mark", "paul", "steven", "andrew", "kenneth", "george", "joshua", "kevin",
"brian", "edward", "ronald", "timothy", "jason", "jeffrey", "gary", "ryan",
"nicholas", "eric", "stephen", "jacob", "larry", "frank", "jonathan", "scott",
"raymond", "gregory", "samuel", "benjamin", "patrick", "jack", "dennis", "jerry",
"alexander", "tyler", "henry", "douglas", "peter", "adam", "harold", "albert",
"arthur", "carl", "ralph", "roy", "eugene", "louis", "philip", "bobby", "walter",
"willie", "wayne", "fred", "howard", "ernest", "earl", "clarence", "leon",
"leonard", "lewis", "floyd", "leroy", "elmer", "homer", "orrin", "josias",
"emerson", "dale", "bernard", "vernon", "virgil", "wilbur", "russell",
"harvey", "herbert", "melvin", "lloyd", "marvin", "norman", "stanley",
# German
"hans", "karl", "wilhelm", "friedrich", "heinrich", "otto", "hermann", "gustav",
"ludwig", "ernst", "fritz", "johann", "conrad", "konrad", "reinhold", "rudolf",
"rudolph", "gerhard", "helmut", "horst", "klaus", "kurt", "dieter", "günther",
"gunther", "manfred", "siegfried", "hilgard", "christian", "august", "wolfgang",
"jürgen", "jurgen", "matthias", "lothar", "bruno", "gottlieb", "reinhard",
}
FEMALE_NAMES: set[str] = {
# English / common US
"mary", "patricia", "jennifer", "linda", "elizabeth", "barbara", "susan",
"jessica", "sarah", "karen", "nancy", "lisa", "betty", "margaret", "sandra",
"ashley", "kimberly", "emily", "donna", "michelle", "carol", "amanda", "dorothy",
"melissa", "deborah", "stephanie", "rebecca", "sharon", "laura", "cynthia",
"kathleen", "amy", "angela", "shirley", "anna", "ruth", "brenda", "pamela",
"nicole", "katherine", "virginia", "catherine", "helen", "debra", "rachel",
"carolyn", "janet", "maria", "heather", "diane", "julie", "joyce", "victoria",
"kelly", "christina", "joan", "evelyn", "judith", "megan", "alice", "frances",
"marie", "florence", "flora", "zella", "thelma", "ellen", "althea", "della",
"beatrice", "pauline", "hedwig", "florentine", "wilhelmina", "augusta", "bertha",
"gladys", "mildred", "lucille", "edith", "esther", "irene", "hazel", "doris",
"rose", "rita", "norma", "june", "lois", "marjorie",
# German
"greta", "ilse", "ursula", "gertrud", "gertrude", "frieda", "frida", "else",
"hilda", "hilde", "hildegard", "ingrid", "helga", "renate", "monika", "sieglinde",
"brigitte", "gisela", "elke", "anneliese", "waltraud", "edeltraud", "johanna",
"katharina", "margarethe", "wilhelmine", "emilie", "auguste",
}
def guess_sex(given: str | None) -> str | None:
"""Best-guess "male"/"female" from the first token of a given name, or None
if unknown/ambiguous."""
if not given:
return None
first = given.strip().split()[0].lower() if given.strip() else ""
# Strip trailing punctuation/initials like "wm." -> "wm".
first = first.strip(".,'\"")
if not first:
return None
if first in MALE_NAMES:
return "male"
if first in FEMALE_NAMES:
return "female"
return None
+215
View File
@@ -0,0 +1,215 @@
"""Name service. A Person carries one or more Name rows — a primary (typically
the birth/maiden name) plus typed alternates (married, alias, religious, ).
Exactly one name is primary at a time; it drives display everywhere. Writes
require editor rights; reads go through the tree's view check.
"""
import uuid
from datetime import UTC, datetime
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.person import Name, Person
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound
async def _get_person(session: AsyncSession, *, tree: Tree, person_id: uuid.UUID) -> Person:
person = (
await session.execute(
select(Person).where(
Person.id == person_id, Person.tree_id == tree.id, Person.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if person is None:
raise NotFound("person not found")
return person
async def _clear_primary(
session: AsyncSession, *, person_id: uuid.UUID, keep: uuid.UUID | None
) -> None:
"""Demote every other name so exactly one stays primary."""
stmt = (
update(Name)
.where(Name.person_id == person_id, Name.deleted_at.is_(None), Name.is_primary.is_(True))
.values(is_primary=False)
)
if keep is not None:
stmt = stmt.where(Name.id != keep)
await session.execute(stmt)
async def list_names(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, person_id: uuid.UUID
) -> list[Name]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
await _get_person(session, tree=tree, person_id=person_id)
# Non-members: a redacted/hidden person's real names must not leak.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_person_names(
session, viewer_id=viewer_id, tree=tree, person_id=person_id
)
stmt = (
select(Name)
.where(Name.person_id == person_id, Name.deleted_at.is_(None))
.order_by(Name.is_primary.desc(), Name.sort_order, Name.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def create_name(
session: AsyncSession,
*,
actor: User,
tree: Tree,
person_id: uuid.UUID,
name_type: str = "birth",
given: str | None = None,
surname: str | None = None,
prefix: str | None = None,
suffix: str | None = None,
nickname: str | None = None,
is_primary: bool = False,
) -> Name:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
await _get_person(session, tree=tree, person_id=person_id)
# First name for a person is always primary; otherwise honor the flag.
existing = (
await session.execute(
select(Name.id).where(Name.person_id == person_id, Name.deleted_at.is_(None))
)
).first()
primary = is_primary or existing is None
if primary:
await _clear_primary(session, person_id=person_id, keep=None)
name = Name(
tree_id=tree.id,
person_id=person_id,
name_type=name_type,
given=given,
surname=surname,
prefix=prefix,
suffix=suffix,
nickname=nickname,
is_primary=primary,
)
session.add(name)
await session.flush()
record_audit(
session,
action="create",
entity_type="Name",
entity_id=name.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"name_type": name_type, "given": given, "surname": surname},
)
await session.commit()
await session.refresh(name)
return name
_NAME_FIELDS = {"name_type", "given", "surname", "prefix", "suffix", "nickname"}
async def update_name(
session: AsyncSession,
*,
actor: User,
tree: Tree,
person_id: uuid.UUID,
name_id: uuid.UUID,
changes: dict,
) -> Name:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
name = (
await session.execute(
select(Name).where(
Name.id == name_id,
Name.person_id == person_id,
Name.tree_id == tree.id,
Name.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if name is None:
raise NotFound("name not found")
for key in _NAME_FIELDS & changes.keys():
setattr(name, key, changes[key])
if changes.get("is_primary") is True:
await _clear_primary(session, person_id=person_id, keep=name.id)
name.is_primary = True
record_audit(
session,
action="update",
entity_type="Name",
entity_id=name.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(name)
return name
async def delete_name(
session: AsyncSession,
*,
actor: User,
tree: Tree,
person_id: uuid.UUID,
name_id: uuid.UUID,
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
name = (
await session.execute(
select(Name).where(
Name.id == name_id,
Name.person_id == person_id,
Name.tree_id == tree.id,
Name.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if name is None:
raise NotFound("name not found")
name.deleted_at = datetime.now(UTC)
was_primary = name.is_primary
name.is_primary = False
record_audit(
session,
action="delete",
entity_type="Name",
entity_id=name.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
# Promote another name to primary so the person never loses their display name.
if was_primary:
nxt = (
await session.execute(
select(Name)
.where(Name.person_id == person_id, Name.deleted_at.is_(None))
.order_by(Name.sort_order, Name.created_at)
)
).scalars().first()
if nxt is not None:
nxt.is_primary = True
await session.commit()

Some files were not shown because too many files have changed in this diff Show More