153 Commits

Author SHA1 Message Date
justin 7043532c3b Merge pull request 'Cleanup tool: mark deceased by a child's birth year' (#254) from cleanup-deceased-by-child into main
build-backend / build (push) Successful in 29s
build-frontend / build (push) Successful in 1m29s
2026-06-11 11:08:52 -04:00
justin 1340d1957f Cleanup tool: "mark deceased by a child's birth year" rule
Adds a preview/apply rule to the Cleanup tool for parents who have NO birth date
of their own (so the existing born-on-or-before rule can't reach them) but who
have a child born long ago — they're necessarily deceased. This is the gap that
left ~56 parents in the Paul tree as "unknown".

- cleanup_service.preview_deceased_by_child(year): parents of any child born
  on/before the cutoff, excluding already-deceased; returns child_birth_year.
- GET /trees/{id}/cleanup/deceased-by-child?born_on_or_before=1900. Apply reuses
  the existing POST .../cleanup/deceased (same audited mark-deceased path).
- Frontend: a new card in the Cleanup tool (year input → preview → select →
  apply), preview-first like the rest of the tool.

Test covers preview (finds the no-birthdate parent of a pre-cutoff child,
excludes modern-child parents), child_birth_year, apply, and re-preview drop.
Suite 106 passing.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 11:08:50 -04:00
justin e24a7cfcc9 Merge pull request 'Tree cards: living/unset-sex people render gray, not blue' (#253) from living-and-unset-cards-gray into main
build-frontend / build (push) Successful in 1m28s
2026-06-11 10:37:27 -04:00
justin 07944e329e Tree cards: render unset-sex / redacted "Living person" in gray, not blue
The chart mapped gender as `=== "female" ? "F" : "M"`, so anything non-female —
including null — became "M" (blue). On the public site, redacted living people
(whose gender the privacy engine nulls) all showed blue regardless of real sex,
and anywhere a person's sex was simply unset they also showed blue (misleading).

Map male→"M", female→"F", and everything else→null, which family-chart renders
as `card-genderless`. So living/redacted people render gray (and never imply a
sex), and unset-sex people render gray instead of defaulting to male/blue.
Applied to both the member tree (tree/page.tsx) and the public chart
(public-tree-chart.tsx), which share chart.css. Also bumped the genderless color
from the library's washed-out `lightgray` to a warm mid-gray that matches the
muted male/female tones and the brand palette.

Privacy note: `_redact` already nulls gender, so this is purely the client color
mapping — no sex leak, just a correct neutral rendering.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 10:37:25 -04:00
justin a33a88e558 Merge pull request 'docs: note the spouse-layout fix is upstreamed' (#252) from docs-upstream-spouse-fix into main 2026-06-11 09:33:21 -04:00
justin fe8349819f docs: note the spouse-layout fix is upstreamed (donatso/family-chart#105)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 09:33:19 -04:00
justin e745fb5d4d Merge pull request 'Move cardToMiddle fix into the family-chart patch (+ document patches)' (#251) from family-chart-patch-cardtomiddle into main
build-frontend / build (push) Successful in 1m27s
2026-06-11 09:21:32 -04:00
justin e0573e6be2 Move cardToMiddle vertical-centering fix into the family-chart patch
Fold the fly-to vertical-centering fix into our patch-package patch (alongside
the existing spouse-layout fix) instead of compensating in app code, and revert
the in-app workaround so the two don't double-correct.

- patches/family-chart+0.9.0.patch: cardToMiddle now scales datum.y by the zoom
  k in both dist builds (.js + .esm.js), matching datum.x. Verified the patch
  applies cleanly (patch-package --error-on-fail).
- tree/page.tsx: the cardToMiddle caller passes raw y again (the patched library
  does the scaling now); pre-scaling here too would double-correct. Behavior is
  identical to the previous in-app fix — both center the node exactly.
- CLAUDE.md: documents the two family-chart patches, how to regenerate them, and
  that both should be upstreamed. The cardToMiddle fix is submitted upstream
  (donatso/family-chart#103, issue #102); the spouse-layout fix is a TODO.

The frontend Dockerfile already COPYs patches/ before npm ci, so the fix is in
the production build.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 09:21:30 -04:00
justin 3731d77d4b Merge pull request 'Fix fly-to vertical centering at non-1 zoom levels' (#250) from fix-fly-to-vertical-centering into main
build-frontend / build (push) Successful in 1m28s
2026-06-11 08:58:38 -04:00
justin bf1576252b Fix fly-to vertical centering at non-1 zoom levels
Clicking ×N sometimes flew to a blank area far below the tree. Cause:
family-chart's cardToMiddle scales datum.x by the zoom factor k but not datum.y
(`y = height/2 - datum.y`, missing the ·k), so vertical centering is only
correct at k=1 and drifts by datum.y·(k−1) at any other zoom — worse the deeper
the person sits. That's why it worked only when the view happened to be near 1:1.

Compensate by pre-multiplying the y we pass to cardToMiddle by the current
scale, cancelling the library's missing ·k. x was already correct.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:58:36 -04:00
justin 0ed6ba4505 Merge pull request 'Tree: clicking ×N flies to the person's other copy' (#249) from tree-fly-to-duplicate into main
build-frontend / build (push) Successful in 1m31s
2026-06-11 08:47:59 -04:00
justin ed263cf9a7 Tree: clicking ×N flies to the person's other copy (not just flashes)
On a large tree the duplicate's other copy is usually off-screen, so flashing
in place wasn't enough. Clicking the ×N badge now pans/zooms the view to center
the other copy and flashes it on arrival; clicking again cycles through the
remaining copies (for a person drawn 3+ times).

Uses family-chart's exported handlers: cardToMiddle centers a datum (read from
the target card_cont's bound x/y, falling back to its transform attr), keeping
the current zoom level via getCurrentZoom. Verified against the lib: the svg's
parent (f3Canvas) holds the zoom object, and cards are positioned by datum x/y —
same coordinate space cardToMiddle expects. Falls back to an in-place flash if
the zoom object isn't ready. Frontend only; supersedes the flash-only behavior.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:47:44 -04:00
justin f7666ad30b Merge pull request 'Tree: Legend by the pan/zoom hint + clickable ×N duplicate badges' (#248) from tree-legend-and-duplicate-flash into main
build-frontend / build (push) Successful in 1m28s
2026-06-11 08:32:53 -04:00
justin 690a6da659 Tree: a Legend by the pan/zoom hint, and clickable ×N duplicate badges
Two small tree-view aids prompted by "why do some people show ×2".

- Legend: a hover/focus "Legend" link next to the "drag to pan…" hint, explaining
  the ×N badge (a person drawn N times in the view because they connect through
  more than one line — a shared ancestor or an intermarriage), the gender card
  colors, and the pan/zoom/recenter controls.
- The ×N badge is now clearly clickable (cursor + hover state); clicking it
  flashes every copy of that person in the current view (a bronze outline pulse),
  so you can spot where else they appear. Implemented by delegating on the chart
  container and matching the d3-bound person id across cards; capture-phase +
  stopPropagation so a badge click flashes instead of recentering.

Frontend only. Honest follow-up: flashing finds copies that are on-screen; a true
"fly to an off-screen copy" needs d3-zoom transform work (the chart pans by
transform, not scroll) — a later enhancement.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:32:35 -04:00
justin e7115023e1 Merge pull request 'Person page: server-side search; stop loading the whole tree' (#247) from person-page-server-search into main
build-backend / build (push) Successful in 38s
build-frontend / build (push) Successful in 1m30s
2026-06-11 08:29:32 -04:00
justin 58400ffdf7 Person page: server-side search; stop loading the whole tree
The person page fetched the entire tree on every open — all persons (to build a
name map + power the relative pickers) and all events (to find partnership
events). On a 2k-person tree that's a ~230KB person list + ~600KB event list per
view. Now it loads only what the page shows:

Frontend:
- The relationship & spouse pickers use the backend's fuzzy pg_trgm search
  (debounced, typo-tolerant) instead of substring-filtering a preloaded array —
  better search, and no need to preload every person. PersonCombobox gained an
  `onSearch` server mode (client `people` mode still works).
- The page drops the all-persons and all-events fetches; it resolves just this
  person's relatives' names via GET /persons?ids=..., and reads partnership
  events from the per-person events endpoint.

Backend:
- GET /trees/{id}/persons?ids=a,b,c — batch by id (privacy-filtered, names
  batched), for relative-name display.
- list_events_for_person (member path) now also returns the person's partnership
  events, so the page needn't scan every event in the tree.

Adversarial review (frontend logic + backend/privacy) found no issues. Suite 105
passing.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:29:13 -04:00
justin 629bfa1367 Merge pull request 'Fix list_persons N+1 (the ~4s person-page load)' (#246) from fix-person-list-n-plus-one into main
build-backend / build (push) Successful in 37s
2026-06-11 08:00:47 -04:00
justin 1562febdcf Fix list_persons N+1 (the ~4s person-page load)
Opening any person page on a large tree took 4-5s on an idle server. Root cause:
list_persons looped over every person calling privacy.person_visibility (which
issues TWO get_membership_role queries per call) AND _attach_primary_name (one
name query per person). On the reporter's 2,324-person tree that's ~7,000
serialized DB round-trips per page load — the person page fetches the full
person list to build its name-lookup map.

Fix:
- Resolve the viewer's membership role ONCE. Members see the whole tree (full),
  so skip the per-person privacy engine entirely.
- Add _attach_primary_names: one batched names query (person_id IN (...),
  ordered the same as the single-person query so it picks the same name) instead
  of one per person.
- Apply the same batching to the non-member path, search_persons, the deleted-
  persons list, and public_view_service.list_public_persons.

Member-path list_persons goes from ~3·N queries to ~3 total. Other tree-wide
list endpoints (events/relationships/media/citations) were already flat selects.

Adds a regression test that asserts list_persons issues a constant number of
queries (not proportional to person count). Suite: 103 passing.

Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-11 08:00:30 -04:00
justin 265f5f4e7a Merge pull request 'Close citation/source living-person leak; add on-demand tree purge' (#245) from citation-redaction-and-tree-purge into main
build-backend / build (push) Successful in 32s
build-frontend / build (push) Successful in 1m26s
2026-06-10 22:39:15 -04:00
justin a6179037c2 Close citation/source living-person leak; add on-demand tree purge
Two changes.

1. Privacy fix (NN#2/NN#3) — the citation and source list endpoints gated only
   on can_view_tree, so a non-member on a public/unlisted/site_members tree could
   enumerate citations and sources tied to a redacted living person, leaking that
   the person exists and has sourced facts (and possibly their name via a source
   title). #46 closed this for events/media/names/relationships but not
   citations/sources. Now citation_service.list_citations and
   source_service.{list_sources,get_source} delegate non-member reads to
   public_view_service, mirroring the #46 pattern:
   - citations: shown only when the cited fact resolves to FULL-visibility
     person(s) — covers the person_id, name_id, event_id (person or both-partner),
     and relationship_id (both-partner) target paths.
   - sources: shown only when they back at least one visible citation; a withheld
     source 404s (don't reveal it exists).
   Tests cover all four citation target types + source withholding + member-sees-all.

2. On-demand tree purge — owners can permanently delete a soft-deleted tree now
   instead of waiting out the 30-day auto-purge window. POST /trees/{id}/purge
   (owner-only): the tree must already be in the trash, and the caller retypes its
   name to confirm. Media objects are deleted from storage, then a single
   DELETE on trees cascades all tree-owned rows via the tree_id ON DELETE CASCADE;
   the audit entry survives (tree_id SET NULL). Frontend adds a "Delete forever"
   button to the Recently-deleted list. No migration.

Suite: 102 passing.
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-10 22:38:59 -04:00
justin 7ed3ddd448 Merge pull request 'docs: bring all documentation current with shipped work' (#244) from docs-catch-up into main 2026-06-10 21:05:46 -04:00
justin 447daf7fa8 docs: bring all documentation current with shipped work
A multi-agent audit of every doc against the code surfaced ~50 stale/missing
items (the roadmap/status docs and the backlog had fallen behind the code).
This catches them up:

- CLAUDE.md: phase status was ~3 phases stale ("Phase 1 is next" while Phase 1 +
  chunks of 2 & 4 shipped). Rewrote the status list; added a model-provider
  tech-stack entry; updated repo-layout (integrations objectstore/models,
  deploy backup.sh/dev compose).
- ARCHITECTURE.md: §6 privacy engine described 3 visibility levels — corrected to
  the shipped 4 (adds site_members); documented per-tree AI policy on Tree,
  LLMProvider/EmbeddingProvider split + registry, ChangeProposal origin/status/
  operations, verified-email session gate, instance-owner role, schema-drift
  guard, and the env_file config model.
- PRD.md: 4-level visibility in US-040/§5.5, instance-owner role (§5.1/§5.11),
  per-tree AI policy (§5.8), §8 sequencing annotated with shipped status, header
  date/status bumped.
- README.md: 4-level privacy; softened "Full GEDCOM 7" to the 5.5.1/7 common
  subset; noted backups + instance-owner admin; moved property/land to an
  explicit "where it's headed" (no property models exist yet).
- BACKLOG.md: flipped ~15 shipped-but-open rows to Have (ChangeProposal, provider
  abstraction, GEDCOM citation export, membership management, operator backup,
  email-verification gate, per-tree AI policy, instance owner, the whole
  visibility/public-viewing/child-resource-redaction cluster #41-#51/#46), and
  reconciled the executive summary, "current defects" list, quick wins, and
  differentiators. Left genuinely-open items (citation/source redaction, sitemap,
  per-tree noindex, scoped-token API) accurately open.
- .env.example: dropped "SMTP wired in a later phase"; documented the worker
  purge knobs, S3_PRESIGN_TTL, COOKIE_NAME; removed a stray duplicate line.
- design/: tree-visibility.md and change-proposal.md marked Shipped; corrected
  the redaction approach (reuses member schemas, not a separate PublicPersonRead)
  and the apply() rollback claim (v1 is not cross-op transactional), and marked
  rate-limiting/sitemap/noindex as deferred.

No code changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-10 21:05:29 -04:00
justin 0388b9b99f Merge pull request 'compose: drive app config from .env (env_file, blanket passthrough)' (#243) from compose-env-file into main 2026-06-10 08:46:16 -04:00
justin 00f403defa compose: drive backend/worker/migrate config from .env (env_file)
Replace the per-setting environment allow-list with `env_file: .env` on the
three app-image services, so every setting in app/core/config.py is configurable
from .env with no compose edit. This kills the recurring trap where a documented
env var (OWNER_EMAIL, the AI keys, SMTP, APP_BASE_URL) silently didn't reach the
app because it wasn't on the hand-maintained list.

`env_file` is `required: false` so local/CI without a .env still works (falls
back to ${VAR:-default} interpolation + code defaults). The small `environment:`
block that remains is only for values that must NOT come from .env:
  - RUN_MIGRATIONS=1 (backend) — a deploy flag, not an app setting.
  - DATABASE_URL — pinned to the compose-internal host, because the code default
    points at localhost (wrong inside the network). environment wins over
    env_file, so this is a safety net if .env ever omits it.

Trade-off (accepted, see comment): env_file also injects infra secrets
(POSTGRES_*, MINIO_*, CLOUDFLARE_TUNNEL_TOKEN) into the app process env; the app
ignores unknown vars (pydantic extra="ignore").

Verified on prod: DATABASE_URL resolves to postgres:5432, RUN_MIGRATIONS=1 and
OWNER_EMAIL intact, COOKIE_SECURE=true (no posture change), health 200, trees
200. The earlier explicit AI/SMTP/OWNER passthrough is now subsumed by this.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-10 08:46:00 -04:00
justin 519f1c31b5 Merge pull request 'compose: forward AI provider + mailer/SMTP env to the backend' (#242) from compose-ai-smtp-passthrough into main 2026-06-10 08:39:04 -04:00
justin 3a1395b6af compose: forward AI provider + mailer/SMTP env to the backend
Follow-up to the OWNER_EMAIL passthrough. The backend service env block is an
explicit allow-list, so the documented model-provider keys (ANTHROPIC_*,
OPENAI_*, XAI_*, OLLAMA_*, DEFAULT_*_PROVIDER, LLM_MAX_TOKENS,
EMBEDDING_DIMENSIONS) and mailer settings (MAILER, SMTP_*, APP_BASE_URL,
REQUIRE_EMAIL_VERIFICATION) never reached the container — setting them in .env
was a no-op. The AI assistant/policy and the SMTP mailer run in the backend, so
forward them here.

Side fix: APP_BASE_URL was likewise dropped, so outbound email links used the
code default http://localhost instead of the configured domain. Now forwarded
(verified live: backend reports APP_BASE_URL=https://provenance.paul.farm).

Worker is left as-is (it consumes neither today); it'll need the model vars when
embedding/matching jobs land. Alternative to this growing allow-list is
`env_file: .env` on the service — deferred to avoid forwarding unrelated secrets.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-10 08:38:49 -04:00
justin 2712ae469b Merge pull request 'compose: forward OWNER_EMAIL to the backend container' (#241) from compose-forward-owner-email into main 2026-06-09 23:22:59 -04:00
justin 88beb9650f compose: forward OWNER_EMAIL to the backend container
The instance-owner feature reads OWNER_EMAIL, but the backend service's
environment block is an explicit allow-list that didn't include it — so setting
it in .env never reached the app (is_instance_owner always saw "" → no owner).
Add the passthrough.

NOTE: the same allow-list omits the AI provider keys (ANTHROPIC_API_KEY,
OPENAI_*, XAI_*, OLLAMA_*) and SMTP settings, so those documented env vars also
don't currently reach the backend on this deployment. Worth a follow-up
(forward them explicitly, or switch the service to env_file) so .env actually
drives all configuration per the twelve-factor rule.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 23:22:48 -04:00
justin 15504ba6e1 Merge pull request 'Instance owner/operator role (env-declared via OWNER_EMAIL)' (#240) from instance-owner into main
build-backend / build (push) Successful in 29s
build-frontend / build (push) Successful in 1m29s
2026-06-09 23:17:08 -04:00
justin c5631d3eab Add an instance owner/operator role (env-declared via OWNER_EMAIL)
Provenance had no system-level owner: ownership was only per-tree
(TreeMembership), so a self-hosted instance had no operator account and no
instance-admin surface. This adds one, declared by environment per the project's
twelve-factor rule.

- OWNER_EMAIL (comma-separated): the account(s) named here are instance owners.
  Derived at request time — no DB column, no migration, can't drift from the env,
  survives DB resets. is_instance_owner()/InstanceOwner dependency in api/deps.py.
- Ownership requires a VERIFIED email (independent of REQUIRE_EMAIL_VERIFICATION).
  Registration is open, so without this an attacker could seize the role by
  registering the owner address first; verification ties it to inbox control.
- GET /api/v1/admin/instance (owner-only): operational status — version, env,
  user/tree counts, configured AI providers. Deliberately exposes no tree data
  or PII: instance ownership is an operator role, NOT a privacy-engine bypass.
- /users/me reports is_instance_owner; frontend gains an owner-only /admin page
  and a conditional sidebar link (server-enforced, not just client-hidden).

Found-and-fixed by an adversarial security review before merge: the
verified-email land-grab (above) and a frontend null-deref where the admin page
crashed on 401/5xx instead of failing closed.

Docs: .env.example + ARCHITECTURE (notes the not-a-privacy-bypass boundary and
the verified-email requirement). Tests: owner matching, the land-grab guard,
/users/me, and owner-only /admin. Suite 96 passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 23:16:45 -04:00
justin 6fbad3106d Merge pull request 'Guard against schema drift (readiness 503 + loud startup log)' (#239) from schema-drift-guard into main
build-backend / build (push) Successful in 32s
2026-06-09 21:56:08 -04:00
justin 94b5caa7e5 Guard against schema drift: fail readiness + log loudly when DB is behind code
Defense-in-depth for the deploy pipeline. Today a backend image shipped ahead
of an un-applied migration; the Tree model selected columns the DB didn't have
yet, so every trees query 500'd with an opaque UndefinedColumnError and the UI
showed no trees. The root cause (deploys not running migrations) is fixed
separately; this makes the *symptom* impossible to miss.

- app/core/schema_version.py: compare the DB's stamped alembic head to the
  head(s) baked into the image's migration scripts. A DB with no alembic_version
  table (e.g. a create_all test DB) is treated as current, so this stays quiet
  outside real deployments. Uses to_regclass so a missing table never poisons
  the caller's transaction.
- /health/ready: returns 503 with an explicit "drift: db=… expected=…" message
  when the schema is behind, instead of reporting ready and serving 500s.
- Startup lifespan: logs CRITICAL on drift (advisory — never blocks startup).

Liveness (/health) is untouched, so a drifted container isn't killed into a
crash-loop — it's loudly degraded and self-heals once migrations apply.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 21:55:21 -04:00
justin f8fa23c1f6 Merge pull request 'Per-tree AI model policy (owner-only admin view)' (#238) from ai-model-policy into main
build-backend / build (push) Successful in 32s
build-frontend / build (push) Successful in 1m27s
2026-06-09 20:53:07 -04:00
justin c6b1e72130 Per-tree AI model policy (owner-only admin view)
The operator decides which model providers exist (env / registry — Anthropic,
OpenAI, x.AI, Ollama, several at once). The *tree owner* decides who uses which:

- Members' assistant -> one configured provider (or none)
- Recommender (association/connection finder) -> one configured provider (or none)
- Owner -> may use any configured provider

Backend: two nullable columns on `trees` (ai_member_provider,
ai_recommender_provider) + migration; `configured_llm_providers()` exposes the
registry as {name, model} with no secrets; owner-gated GET/PATCH
/trees/{id}/ai validate names against the configured set. Frontend: owner-only
"AI models" page with a dropdown per role, graceful 403 for non-owners, and a
sidebar link.

Per-model-within-a-provider selection is a follow-up; today each provider maps
to its single configured model.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 20:52:30 -04:00
justin ceafb299d6 Merge pull request 'Model providers: OpenAI/xAI/Ollama + run several at once' (#237) from multi-provider-openai-xai-ollama into main
build-backend / build (push) Successful in 32s
2026-06-09 18:39:20 -04:00
justin de50f2c803 Model providers: OpenAI/xAI/Ollama + run several at once (registry)
Extends the #215 abstraction:
- OpenAICompatibleLLMProvider / OpenAICompatibleEmbeddingProvider — one impl (via
  the official openai SDK) covers OpenAI, xAI (api.x.ai/v1), Ollama
  (…:11434/v1), OpenRouter, etc.; they differ only by base_url, key, and model.
- Registry factory: build_llm_providers() / build_embedding_providers() return
  every provider whose credentials are configured, so you can run several
  concurrently. get_llm_provider(name)/get_embedding_provider(name) select by
  name, falling back to default_*_provider, then Null.
- Per-provider env config (ANTHROPIC_*, OPENAI_*, XAI_*, OLLAMA_*) +
  DEFAULT_LLM_PROVIDER / DEFAULT_EMBEDDING_PROVIDER; documented in .env.example.
  Defaults keep AI off (empty registry).

Embeddings now have real backends (OpenAI/Ollama), still separate from the LLM
since Anthropic offers no embeddings endpoint. Tests cover multi-provider
selection, default resolution, disabled-without-credentials, and null fail-loud.
Full suite 87 passed.

Relates to #215.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 18:39:19 -04:00
justin 9187c0a791 Merge pull request 'Fix #214: ChangeProposal (propose-then-confirm)' (#236) from change-proposal into main
build-backend / build (push) Successful in 31s
build-frontend / build (push) Successful in 1m25s
2026-06-09 15:44:41 -04:00
justin abaa8efdd5 Fix #214: ChangeProposal (propose-then-confirm)
Implements non-negotiable #1: the AI assistant never writes autonomously. Every
assistant/contributor "write" emits a ChangeProposal — a structured diff a human
approves, edits, or rejects. Design: docs/design/change-proposal.md.

Structural guarantee: a proposal's operations reach the DB ONLY via
change_proposal_service.apply(), which requires the actor be an editor and
dispatches each op through the normal editing services (person/name/event/
relationship/source/citation create/update/delete) — so every change passes the
privacy engine and is audited as the approving human. propose() only inserts a
pending row; it performs no domain mutation. Model providers stay read-only, so
no model response can mutate tree data.

- ChangeProposal model + migration (status pending|applied|rejected, origin
  assistant|contributor, JSONB operations, reviewer + apply_error).
- Service: propose / list / get / apply (with optional edited ops) / reject /
  delete; a dispatcher mapping ops → editing services. v1 applies ops in order,
  not cross-op transactional (single-op is atomic; documented).
- API /trees/{id}/proposals + a frontend review page (approve/reject; editor-
  gated) and sidebar entry.

Tests: proposal doesn't apply until approved; reject doesn't apply; non-editor
member can see but not apply; multi-op; approve-with-edits; apply-error keeps it
pending. Full suite 87 passed; single alembic head.

Closes #214

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 15:44:40 -04:00
justin 251a10a087 Merge pull request 'Fix #215: pluggable LLM + embedding provider abstraction' (#235) from model-provider-abstraction into main
build-backend / build (push) Successful in 28s
2026-06-09 12:51:03 -04:00
justin 330543f9ce Fix #215: pluggable LLM + embedding provider abstraction
Adds the vendor-agnostic seam the AI assistant + match-ranking plug into:
- LLMProvider / EmbeddingProvider ABCs (base.py). LLM and embeddings are
  SEPARATE abstractions — Anthropic has no embeddings endpoint, so each is
  configured independently and either can be off.
- NullLLMProvider / NullEmbeddingProvider — the default; fail loud with a clear
  "not configured" error so AI-off deployments don't silently no-op.
- AnthropicLLMProvider — first concrete LLM impl, via the official anthropic SDK
  (default model claude-opus-4-8). A local provider (e.g. Ollama) would be
  another subclass of the same interface.
- Factory in deps.py (get_llm_provider / get_embedding_provider) selects by
  env (MODEL_PROVIDER / EMBEDDING_PROVIDER); documented in .env.example.

Providers are read-only text/vector producers — they never touch the DB, so the
"AI never writes autonomously" invariant (CLAUDE.md #1) holds; writes will go
through ChangeProposal (#214).

Tests: provider selection (null default, anthropic when keyed, fallback without
key) + null providers raise. 81 passed.

Closes #215

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 12:51:01 -04:00
justin d540dc3f32 Merge pull request 'Fix #196: one-command operator backup (pg_dump + MinIO)' (#234) from operator-backup-script into main 2026-06-09 12:45:35 -04:00
justin 8652425413 Fix #196: one-command operator backup (pg_dump + MinIO)
Move backup from a documented procedure to `deploy/backup.sh`: dumps Postgres
(pg_dump --clean --if-exists, gzipped) and archives the MinIO /data directory
into a single timestamped bundle under backups/. Reads config from the compose
.env with the same defaults the stack uses; optional BACKUP_RETAIN_DAYS prunes
old bundles (cron-friendly). BACKUP.md documents usage + the restore procedure
(kept manual/documented rather than an untested destructive script).

Closes #196

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 12:45:33 -04:00
justin 3a7728f1dc Merge pull request 'Fix #145: tree membership management (list / add / role / remove)' (#233) from membership-management into main
build-backend / build (push) Successful in 27s
build-frontend / build (push) Successful in 1m29s
2026-06-09 12:43:31 -04:00
justin eb0350733b Fix #145: tree membership management (list / add / role / remove)
TreeMembership was enforced on every read/write but had no API or UI to manage
members — trees were effectively single-user, breaking full-CRUD (NN#8).

Backend (/trees/{id}/members): list (members only — the list exposes emails, so
non-members never see it, even on public trees); add an existing user by email
(owner only, 404 if no such account, 409 if already a member); PATCH role;
DELETE. A tree must always keep ≥1 owner (demote/remove of the sole owner → 409).
All changes audited.

Frontend: a Members page (owner gets add-by-email + per-member role select +
remove; others see a read-only list) and a sidebar entry.

Test covers the full lifecycle + every guard. Suite 77 passed.

Closes #145

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 12:43:30 -04:00
justin 6d3147e86d Merge pull request 'Fix #169: keep citation links on GEDCOM export' (#232) from fix-gedcom-citation-export into main
build-backend / build (push) Successful in 29s
2026-06-09 12:37:04 -04:00
justin b4434cb5dd Fix #169: keep citation links on GEDCOM export
Export emitted SOUR records but never the per-fact SOUR links, so a
Provenance→Provenance round-trip destroyed the sources graph (citations were
dropped). Emit citation links on the facts they sit on:
- person-level → 1 SOUR @Sx@ (2 PAGE)
- name-level   → 2 SOUR under 1 NAME
- event-level  → 2 SOUR under the event (incl. partnership events in FAM)
- relationship → 1 SOUR under FAM
Citations whose source didn't export are skipped.

Test: a person + event citation round-trips through export→import into a fresh
tree with their pages intact. GEDCOM suite 6 passed.

Closes #169

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 12:37:03 -04:00
justin 39e3eac3df Merge pull request 'Security: gate sessions on verified email (opt-in)' (#53) from security-require-email-verification into main
build-backend / build (push) Successful in 35s
2026-06-09 11:22:55 -04:00
justin 660fe7b37f Security: gate sessions on verified email (opt-in)
Backlog §2.10: registration issued a live session and email_verified_at was
written but never read, so an unverified user had full access and there was no
switch to require verification.

Add REQUIRE_EMAIL_VERIFICATION (default false). When true:
- resolve_session_user returns None for a user whose email_verified_at is null —
  the single read-side gate covering every authenticated request, incl. the
  session minted at registration.
- login raises 403 ("email not verified") instead of issuing a useless token.

Default false on purpose: self-hosts without SMTP, and accounts created before
this gate existed (email_verified_at null), must not be locked out. Operators
enable it once mail works and accounts are verified. Documented in .env.example.

Tests: default-off keeps unverified accounts working; on → register's session
won't resolve (401), login is 403, and after verify-email both work. 75 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 11:22:54 -04:00
justin 5485dd2077 Merge pull request 'Cleanup: infer a missing sex from a known-sex spouse (preview → approve)' (#52) from cleanup-sex-from-spouse into main
build-backend / build (push) Successful in 26s
build-frontend / build (push) Successful in 1m28s
2026-06-09 10:59:10 -04:00
justin 05d2773e25 Cleanup: infer a missing sex from a known-sex spouse (preview → approve)
Unset sex renders blue (male-colored), which is misleading next to a confirmed
male partner. Add a Cleanup action that proposes the opposite sex for an unset
partner of someone whose sex is set (couples are opposite-sex in practice — a
confirmed-male husband ⇒ a female wife). People whose known partners disagree
are skipped as ambiguous.

It's a preview the user reviews and approves in the Cleanup tool (reusing the
existing gender apply path + audit) — not an autonomous write. Backend:
guess_gender_by_spouse + GET /cleanup/gender/from-spouse. Frontend: an "Infer
from spouse" button feeding the existing proposal list. Test covers
propose-opposite, skip-no-partner, skip-already-set, apply, and re-preview.

Full suite 73 passed; frontend build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 10:59:08 -04:00
justin 768c68cbe0 Merge pull request 'Public tree view: add generation depth controls (shared with member view)' (#51) from public-tree-depth-controls into main
build-frontend / build (push) Successful in 1m29s
2026-06-09 10:35:44 -04:00
justin 7d6fbce87e Public tree view: add generation depth controls (shared with member view)
The public tree chart was fixed at 3 ancestors / 2 descendants. Add the same
Generations controls the member view has (slider + number stepper + "All" per
direction), applied live around the focused person.

Extracts the member page's inline DepthControl into a shared
components/depth-control.tsx and uses it in both, so they stay in sync. The
public chart gains anc/prog depth state + an apply effect (setAncestryDepth/
setProgenyDepth + updateTree) mirroring the member behavior.

tsc clean; next build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 10:35:43 -04:00
justin 12ba0a0fb6 Merge pull request 'Public tree view: full-width canvas like the member view' (#50) from public-tree-fullwidth into main
build-frontend / build (push) Successful in 1m29s
2026-06-09 10:29:20 -04:00
justin 150d69e5ac Public tree view: full-width canvas like the member view
The public layout forced max-w-5xl on every /p page, so the tree chart was
cramped. Mirror the member shell: the public layout now drops the max-width for
the tree page (/p/<id>) only, giving the chart the full canvas (74vh to match
the member view), while the page keeps its heading and people list in a
centered max-w-5xl column. Person detail (/p/<id>/persons/<pid>) and /explore
stay narrow.

tsc clean; next build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 10:29:18 -04:00
justin 053ce357ac Merge pull request 'Public view: add tree chart + homepage Explore links' (#49) from public-tree-chart-and-explore-link into main
build-frontend / build (push) Successful in 1m25s
2026-06-09 09:44:24 -04:00
justin 269cae556f Public view: add tree chart + homepage Explore links
Two gaps from review of the public surface:
- The public tree page showed only a list of names. Add the family-chart
  hourglass (PublicTreeChart) above the directory — the same renderer the
  member tree view uses, including the cycle-sanitisation that guards against a
  bad graph, fed by redacted public data. Click a card to recenter; "Open"
  links to the person's public page. Centers on the tree's home person.
- The homepage had no path to /explore. Add an "Explore" nav link and an
  "Explore public trees" hero button.

tsc clean; next build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:44:23 -04:00
justin 0df44e7e59 Merge pull request 'Visibility phase 5: public /explore directory + search' (#48) from visibility-phase5-explore into main
build-frontend / build (push) Successful in 1m28s
2026-06-09 09:34:21 -04:00
justin 7a5c5f2882 Visibility phase 5: public /explore directory + search
A no-login directory of shared trees, backed by GET /api/v1/public/trees:
- /explore: searchable grid of public trees; debounced name search. Because the
  backend adds `site_members` trees when a valid session is present, signed-in
  users see more with no client-side branching.
- PublicHeader extracted and shared by /p and /explore (logo, Explore, Sign in).
- "Explore" entry added to the authed sidebar.

tsc clean; next build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:34:20 -04:00
justin 20c7fbd8d6 Merge pull request 'Visibility phase 4: no-login public viewer pages + robots' (#47) from visibility-phase4-public-pages into main
build-frontend / build (push) Successful in 1m28s
2026-06-09 09:31:58 -04:00
justin b8405ced07 Visibility phase 4: no-login public viewer pages + robots
Adds the public viewing surface in the UI — shareable, no-login pages backed by
the redaction-safe /api/v1/public API:

- /p/[treeId]: tree name + searchable people directory (living people show as
  "Living person"; counts; links to person pages).
- /p/[treeId]/persons/[personId]: person detail — events, alternate names, and
  parents/partners/children as links to other public person pages.
- app/p/layout.tsx: slim public header (logo + Sign in), no app sidebar.
- robots.ts: allow /p/, disallow the authenticated app sections.
- Trees list: a "Public page ↗" link on every non-private tree so the owner can
  grab the shareable URL.

Client-rendered (same-origin fetch via Caddy). Follow-up (needs a frontend
SSR→backend base URL + a compose/env deploy step, so not auto-applied by
Watchtower): true server-rendering for SEO, a dynamic sitemap of public trees,
and per-page noindex for unlisted/site_members.

tsc clean; next build passes (both routes dynamic).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:31:56 -04:00
justin 91a7ce1dc2 Merge pull request 'Fix leak: redact per-person on authed non-member reads' (#46) from fix-authed-nonmember-redaction into main
build-backend / build (push) Successful in 31s
2026-06-09 09:26:54 -04:00
justin 8b91326481 Fix leak: redact per-person on authed non-member reads
A logged-in NON-member of a public/unlisted tree could read living people's
dates, real alternate names, and media (incl. downloading photos) through the
family-view endpoints — only the person LIST was redacted; list_events,
list_relationships, list_names, list_media gated on can_view_tree alone.

For non-members, these now delegate to the same visibility-filtered reads the
public surface uses (person_visibility-driven): living-person events/names
dropped, relationships touching a hidden person dropped, media limited to
full-visibility persons, and media download (get_media → media_content) 404s
for a redacted/unlinked person's media. Members are unchanged.

Adds list_public_relationships_for_person / list_public_media / can_view_media
to public_view_service. Test: an authed non-member sees no living-person PII
across events/names/relationships/media and can't download a living person's
file, while the owner still sees everything. Full suite: 72 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:26:53 -04:00
justin 671b560768 Merge pull request 'docs: add product backlog (genealogy feature gap analysis)' (#45) from add-product-backlog into main 2026-06-09 09:19:22 -04:00
justin 6a5ef4d392 docs: add product backlog (genealogy feature gap analysis)
Output of a multi-agent gap analysis comparing Provenance against commercial
(Ancestry/MyHeritage/FamilySearch) and open-source (GRAMPS/Gramps Web/webtrees)
genealogy software: 15 research lenses, 580 raw features deduped into a 17-
category taxonomy, 302 features assessed against the codebase (have/partial/
planned/missing) with statuses verified against the actual code.

Includes an executive summary, per-category backlog with status/importance/
effort/phase, a quick-wins shortlist, and strategic differentiators. Statuses
reflect the repo at analysis time (before tree-visibility phases 1-3); a couple
of flagged items (e.g. the site_members tier) are already closed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:19:21 -04:00
justin 3810b65de0 Merge pull request 'Visibility phase 3: redaction-safe public read API + leak test' (#44) from visibility-phase3-public-api into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m25s
2026-06-09 09:17:55 -04:00
justin 9820a77d25 Visibility phase 3: redaction-safe public read API + leak test
Adds the anonymous read surface (/api/v1/public) — the privacy-critical core.

- CurrentUserOrNone dependency: optional auth that never 401s (anonymous OK).
- public_view_service: every projection passes through privacy.person_visibility.
  persons redacted (living → "Living person", hidden dropped); relationships
  only when both endpoints non-hidden; events only for FULL-visibility persons
  (partnership events only when both partners full); names only for FULL
  persons. Not-viewable trees raise 404 (not 403) so the surface can't probe
  for private trees. Media deferred (higher-sensitivity; own pass later).
- public router: read-only directory + tree + persons/relationships/events +
  person detail/names/events. Directory lists `public` to all and adds
  `site_members` for authenticated callers; never lists unlisted/private.
- PublicTreeRead omits owner_id.

Tests (ran locally — CI does not run pytest): an anonymous end-to-end leak test
asserting a living person's real name, alias, and birth year appear in NO public
response while the deceased person's data does; plus private=404, unlisted
viewable-by-link-but-unlisted, site_members requires login, and directory
visibility. Full suite: 70 passed. Regenerated openapi.json + TS client.

Note: the AUTHED list endpoints still leak per-person for non-members
(pre-existing) — fixed next, separately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:17:41 -04:00
justin 3ff03b037b Merge pull request 'Visibility phase 2: privacy-engine branches on viewer auth state' (#43) from visibility-phase2-privacy into main
build-backend / build (push) Successful in 29s
2026-06-09 09:08:17 -04:00
justin 84a743f5b9 Visibility phase 2: privacy-engine branches on viewer auth state
can_view_tree() now distinguishes anonymous vs authenticated non-members so the
four-level model is enforceable:
- public / unlisted → anyone, including anonymous (unlisted gated only by the
  link, so the API must never *list* it)
- site_members → any authenticated account (denies anonymous)
- private → members only
Members (any role) always view; soft-deleted trees stay hidden from everyone.
person_visibility (living-person redaction) is unchanged.

Tests: a full can_view_tree matrix across {anonymous, logged-in non-member,
member} × {public, unlisted, site_members, private}, plus deleted-tree-hidden
and the site_members anon-vs-logged-in case. Adds `engine`/`db_session` fixtures
(refactored out of `client`) so the engine can be unit-tested directly,
including the anonymous path that has no HTTP endpoint yet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 09:08:04 -04:00
justin e6dfe39e84 Merge pull request 'Visibility phase 1: add site_members value + 4-option dropdown' (#42) from visibility-phase1-enum into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m26s
2026-06-09 09:02:01 -04:00
justin 4a3fe983fa Visibility phase 1: add site_members value + 4-option dropdown
First step of the public-viewing feature (design: docs/design/tree-visibility.md).
No non-member behavior change yet — this only widens the vocabulary and UI.

- TreeVisibility gains `site_members` (any authenticated user of the instance),
  giving the four-level model: public / site_members / unlisted / private.
- Alembic migration adds the enum value via an autocommit block (ALTER TYPE
  ADD VALUE can't run in a transaction on older Postgres); downgrade is a no-op
  since PG can't drop an enum value.
- Regenerated openapi.json + frontend TS client.
- Trees-list dropdown now offers Private / Public – Members / Unlisted / Public
  with an explanatory tooltip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 08:54:45 -04:00
justin 251652a935 Merge pull request 'Trees list: inline visibility selector (private/unlisted/public)' (#41) from tree-visibility-control into main
build-frontend / build (push) Successful in 1m30s
2026-06-09 08:39:42 -04:00
justin dc1b6aac01 Trees list: inline visibility selector (private/unlisted/public)
Tree visibility was set to private with no UI to change it — the trees list
only displayed the value as text. Add a private/unlisted/public dropdown on
each tree card that PATCHes visibility immediately (optimistic), pulled out of
the card's navigation Link so it doesn't trigger a page change. Honors the
"everything configurable / full CRUD in the UI" invariants. Living people stay
protected by the privacy engine regardless of tree visibility.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-09 08:36:20 -04:00
justin f93327f5d3 Merge pull request 'Tree view: configurable generation depth (ancestors/descendants + All)' (#40) from configurable-tree-depth into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 22:21:04 -04:00
justin c86771034c Tree view: configurable generation depth (ancestors/descendants + All)
Depth was hardcoded (3 ancestors, 2 descendants). Add a controls row to set
each direction independently — a slider plus a number stepper, with an "All"
toggle per direction — applied around whoever is currently focused.

- ancestor/descendant depth held in state; effective value is a large cap
  when "All" is on (the chart only renders people that exist, so the cap is
  free).
- changes apply to the live chart via setAncestryDepth/setProgenyDepth +
  updateTree without a full rebuild.
- fan mode (ancestors only) takes the ancestor depth via its `generations`
  prop, capped at 8 to avoid the radial layout's 2^n blow-up; its descendants
  control is disabled with a note.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 22:20:07 -04:00
justin b51b65de80 Merge pull request 'Person page: one-click sex setter (no edit mode)' (#39) from quick-set-sex into main
build-frontend / build (push) Successful in 1m25s
2026-06-08 22:03:32 -04:00
justin 93c22b4bcf Person page: one-click sex setter (no edit mode)
Setting a person's sex meant clicking Edit, opening a dropdown, and saving.
Replace the read-only ♂/♀ symbol next to the name with an always-visible
two-button segmented control that PATCHes immediately on click (gender-only;
backend PATCH is exclude_unset so the name/other fields are untouched).
Clicking the active sex clears it. The full edit form still offers gender for
completeness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 21:42:59 -04:00
justin 7255920135 Merge pull request 'Person page: make marriage-event spouse picker searchable' (#38) from searchable-marriage-spouse into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 21:30:48 -04:00
justin 62513ee22e Person page: make marriage-event spouse picker searchable
Adding a marriage/partnership event used a plain <select> for the spouse,
which is unusable on a large tree — you can't search, only scroll. Swap it
for the existing PersonCombobox (already used by the relationship form), which
filters by name as you type. No onCreate, so it still resolves to an existing
person id, which is what the partnership-event handler requires.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 21:29:27 -04:00
justin ac0b9818dd Merge pull request 'Tree view: center a person between multiple spouses' (#37) from center-spouse-layout into main
build-frontend / build (push) Successful in 1m26s
2026-06-08 19:56:59 -04:00
justin 182a5dab16 Tree view: center a person between multiple spouses
family-chart 0.9.0 stacks all of a person's spouses on one gender-determined
side, so someone with two spouses (e.g. a woman with two husbands) renders
with both spouses piled above/below her and ambiguous child lines.

Patch the library (via patch-package) so the person stays centered and their
spouses split to alternating sides — spouse 1 above, spouse 2 below, further
spouses farther out — and order each couple's children to match, so children
descend from between the correct pair without crossed lines:

- setupSpouses: keep the person centered; place spouses at alternating
  offsets and recenter the cluster on the person's slot.
- sortChildrenWithSpouses: order children by spouse order (gender-independent)
  to match the new spouse positions.

Adds patch-package + a postinstall hook, and COPY patches into the Dockerfile
deps stage so the patch applies during `npm ci` in CI. Verified the patch
re-applies on a clean install and the production build passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 19:56:17 -04:00
justin 77b78410ff Merge pull request 'Tree view: add "Back to default person" recenter link' (#36) from add-default-person-link into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 15:25:04 -04:00
justin fe1e0171ff Tree view: add "Back to default person" recenter link
Once you recenter the tree on someone, there was no quick way back to the
tree's home/default person. Add a header link (shown only when a home person
is set and you're not already on them) that recenters the chart on
home_person_id via the existing goTo() — works in landscape, portrait, and
fan modes. Labels with the home person's name for clarity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 15:15:39 -04:00
justin 9dbdae975a Merge pull request 'Preserve focused person across tree/people/detail navigation' (#35) from improve-tree-people-navigation into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 15:07:10 -04:00
justin c5a2a7f0d4 Preserve focused person across tree/people/detail navigation
The Tree view, People (Family) view, and person detail page each tracked
the "current person" independently, so moving between them reset you to the
home person. The detail page's "← Back to tree" link also pointed at the
People view (not the Tree) and carried no person, so it always landed on the
default person.

Make the focused person a URL-encoded concept that travels across views:

- Tree and People views read ?focus=<id> on load and mirror the focused
  person back into the URL via router.replace (no history spam), so leaving
  and returning keeps you centered where you were. Bookmarks/shared links
  also resolve to the right person.
- "Open person" links carry ?from=tree | ?from=people.
- The detail page's back link is now origin-aware: "← Back to Tree" →
  /tree?focus=<id> or "← Back to People" → /?focus=<id>, returning you in
  place instead of to the home person.
- Add a "View in tree →" link on the detail page — the previously missing
  direct jump from a person to the tree re-rooted on them.
- person→person relationship links (and create-relative redirect) pass
  `from` through so click-chains keep their anchor.

Also gitignore *.tsbuildinfo (Next build artifact).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-08 14:48:32 -04:00
justin 8c36785197 Merge pull request 'Prevent duplicate relationships; harden tree render against cycles' (#34) from prevent-duplicate-links into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m23s
2026-06-08 11:35:12 -04:00
justin fae1162ff8 Prevent duplicate relationships; harden tree render against bad graphs
Root cause of the blank Jung tree: a child double-linked to the same parent
(and, generally, any cycle) made family-chart recurse forever.

Backend (the real fix):
- create_relationship now rejects an equivalent existing edge → 409.
  parent_child is directional (parent→child); partnership/sibling match the
  pair in either order. So you can't link the same two people the same way
  twice. (GEDCOM import already deduped; manual creates didn't.)

Frontend (defense in depth so data can never blank the view):
- Tree view sanitizes the graph before rendering: dedupes parents/spouses,
  drops self-links, and greedily breaks ancestor cycles (a person can't be
  their own ancestor); children are derived from the kept edges. The render is
  wrapped in try/catch and shows a note instead of a blank canvas, telling you
  which conflicting links were skipped.
- Person page surfaces the 409 ("They're already linked that way.").

59 backend tests pass (incl. dup-rejection + reverse-parent-child allowed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 11:35:11 -04:00
justin 1025f86657 Merge pull request 'Cleanup: list people with no sex set + inline set' (#33) from cleanup-unset-sex into main
build-frontend / build (push) Successful in 1m25s
2026-06-08 10:43:10 -04:00
justin a53858f920 Cleanup: list people with no sex set + inline set
Adds a "People with no sex set" section to the Cleanup page — lists everyone
whose gender is still null with inline ♂ Male / ♀ Female buttons (and a link to
their page). Refreshes after the source-match and first-name guess passes, so
it's the manual mop-up for whatever those leave behind.

Frontend only (reuses person list + PATCH) — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 10:43:08 -04:00
justin 941f9827c1 Merge pull request 'Cleanup: best-guess sex from first name (offline dictionary)' (#32) from gender-name-guess into main
build-backend / build (push) Successful in 33s
build-frontend / build (push) Successful in 1m26s
2026-06-08 10:30:36 -04:00
justin 6ec852a23a Cleanup: best-guess sex from first name (offline dictionary)
A "Guess from first name" option in the Cleanup gender section: a bundled,
curated given-name -> sex dictionary (weighted English + German for the first
real tree) proposes sex for people who don't have it set. Deterministic, offline,
no model. Genuinely ambiguous names (Marion, Frances, Jordan, …) are excluded
from both sets so they're left for a human. Reuses the existing preview/apply
gender flow, so every guess is reviewed before saving.

No migration. 56 backend tests pass; frontend builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 10:30:35 -04:00
justin 7405ec762f Merge pull request 'Tree Cleanup tool: bulk deceased / gender-from-source / name fixes (preview-first)' (#31) from tree-cleanup into main
build-backend / build (push) Successful in 27s
build-frontend / build (push) Successful in 1m28s
2026-06-08 10:17:02 -04:00
justin aa62ca490e Tree Cleanup tool: bulk fixes with preview → approve
A new per-tree Cleanup page (and cleanup_service + endpoints), each fix
preview-first per the propose-then-approve rule:

- Mark deceased by birth year: lists people born ≤ a cutoff (default 1930) not
  already deceased; apply sets is_living=false for the ones you keep checked.
- Set sex from a source GEDCOM: upload the source .ged (it carries SEX); matches
  by name and proposes sex only where it's missing — far more accurate than
  guessing from first names. Review, then apply.
- Names that look broken: flags date-in-surname / date-in-given / no-surname /
  packed given names, with inline editable given+surname; fix the checked ones.

No migration (uses existing columns). 55 backend tests pass (preview+apply for
all three); frontend builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 10:17:01 -04:00
justin 97f7a9e0ff Merge pull request 'Show a sex symbol after the name on the person page' (#30) from person-sex-symbol into main
build-frontend / build (push) Successful in 1m27s
2026-06-08 09:16:40 -04:00
justin cd4ccb4ac8 Show a sex symbol after the name on the person page
A blue ♂ (male) or pink ♀ (female) symbol now follows the person's name in the
detail header, using the same gender tints as the tree cards. Nothing shows when
sex is unknown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 09:16:38 -04:00
justin 6696015970 Merge pull request 'Full light/dark theme toggle; brand-aware connector lines' (#29) from light-dark-theme into main
build-frontend / build (push) Successful in 1m26s
2026-06-07 11:49:01 -04:00
justin e8839b15a0 Full light/dark theme toggle; brand-aware connector lines
- Theme is now class-based (.dark on <html>) with a System/Light/Dark toggle in
  the sidebar, persisted to localStorage and applied pre-paint by an inline
  script (no flash). Replaces the prefers-color-scheme-only behavior, so a phone
  on a light OS theme can still choose dark and vice versa.
- New brand-derived --line token (Ink at 55%): a dark line on the light paper,
  light on dark. The family-chart tree connectors had the library's default
  white stroke and were invisible in light mode — now they use --line, as do
  the pedigree brackets and the fan-chart sectors.
- Light/dark tokens use the exact brand palette (Ink/Muted flip; Bronze/Paper
  constant).

Frontend only — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:48:59 -04:00
justin 548e883d82 Merge pull request 'Discoverable Add Person + inline create-new when linking relatives' (#28) from create-person-ux into main
build-frontend / build (push) Successful in 1m27s
2026-06-07 11:30:16 -04:00
justin 37ac49767e Make creating a person obvious; inline "create new" when linking relatives
- Family view gets a prominent "+ Add person" button that creates a person and
  opens their page to fill in details (previously you could only add a person
  via the empty-state form or by linking from another person).
- The person page's relationship picker (PersonCombobox) now offers
  "+ Create '<typed name>'" when the person doesn't exist yet: it creates them,
  links them in the chosen role (parent/child/partner/sibling), and jumps to
  their new page to edit — no more create-then-go-back-and-link.

Frontend only — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:30:14 -04:00
justin 9b04bcefba Merge pull request 'Account export / restore-into-new-tree / delete' (#27) from account-export-restore-delete into main
build-backend / build (push) Successful in 26s
build-frontend / build (push) Successful in 1m27s
2026-06-07 11:26:06 -04:00
justin e9b2436ce0 Account export / restore-into-new-tree / delete
New account_service + endpoints under /users/me:
- GET /me/export — zip of every owned tree (account.json + media blobs).
- POST /me/import — restore a backup into NEW trees (ids remapped, media
  re-uploaded); non-destructive, never touches existing data.
- DELETE /me — soft-delete the user, their owned trees, and revoke sessions;
  guarded by retyping the account email.

Settings page wires all three (export download, restore upload, delete with
typed-email confirmation). No migration — uses existing tables + soft-delete.

52 backend tests pass (export→restore round-trip + delete guards); frontend builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:26:04 -04:00
justin 8903e480cf Merge pull request 'Link media to people (person page + media page)' (#26) from media-person-linking into main
build-frontend / build (push) Successful in 1m24s
2026-06-07 11:19:27 -04:00
justin d27cc5dddc Link media to people (person page + media page)
The Media model already carried person_id/event_id/source_id and the upload
route already accepted person_id — this surfaces it in the UI:

- Person page: a Media card lists media linked to that person, uploads new
  files already linked ("Upload & link"), links existing unlinked media, and
  unlinks.
- Media page: each item gets a person picker to link/unlink.

Frontend only — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:19:25 -04:00
justin 943f459b91 Merge pull request 'Shared marriage events; deterministic parent ordering' (#25) from marriage-event-parent-order into main
build-frontend / build (push) Successful in 1m23s
2026-06-07 11:15:55 -04:00
justin 5106538934 Shared marriage events; deterministic parent ordering
- Partnership life events (marriage/divorce/engagement) now attach to the
  couple's relationship, not each person. The add-event form asks for the
  spouse, finds-or-creates the partnership, and writes ONE event on it — shown
  on both partners' pages ("· with <spouse>"), entered once. Event values
  (RELI/OCCU detail) now render too.
- Family-view pedigree orders parents deterministically (father on top, mother
  below, stable fallback when gender is unknown) instead of by which link was
  created first.

Frontend only — no migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:15:54 -04:00
justin 2669543e56 Merge pull request 'Account menu + Settings (change password), per-tree home person, full-width tree' (#24) from account-settings-home-person into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m24s
2026-06-07 11:05:41 -04:00
justin 0262ed3d97 Account menu + Settings (change password); per-tree home person; full-width tree
- Sidebar bottom-left now shows the signed-in user; clicking opens a menu with
  Settings and Sign out. New /settings page: account info + change password
  (POST /auth/change-password, re-verifies current password). Export/restore/
  delete are stubbed there for the next pass.
- Per-tree default/home person: tree.home_person_id (migration) + TreeUpdate/
  Read; the tree and family views open focused on it; the person page gets a
  "Set as default" control and "Default person" badge. Cleared if that person
  is deleted. Complements the account-level "this is me" link.
- Tree visualization now fills the content area (AppShell drops the max-width
  column on the /tree route); other pages stay centered.
- Audit records are coerced JSON-safe (UUIDs/enums), so PATCHing UUID fields
  like home_person_id audits cleanly.

50 backend tests pass; migration up/down verified; frontend builds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:05:04 -04:00
justin 9ee960c4ef Merge pull request 'Auto-apply migrations on deploy (entrypoint + one-shot service)' (#23) from deploy-auto-migrate into main
build-backend / build (push) Successful in 26s
2026-06-07 10:54:31 -04:00
justin 7f640649b9 Auto-apply migrations on deploy (entrypoint + one-shot service)
So a deploy never needs a manual `alembic upgrade head`:

- Backend image gains an entrypoint that runs `alembic upgrade head` before
  uvicorn when RUN_MIGRATIONS=1 (set on the backend service). This self-migrates
  even on a Watchtower in-place image swap, which doesn't re-run one-shot jobs.
- A one-shot `migrate` service covers the `docker compose up` path; backend and
  worker depend on it completing, which also serializes it with the backend
  entrypoint so alembic never runs concurrently. `upgrade head` is idempotent.

Activating this needs the updated compose on the host once (Watchtower only
swaps images, not the compose file / env). After that, migrations are automatic.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:50:28 -04:00
justin a8929c2862 Merge pull request 'Global Import menu entry + mobile drawer nav' (#22) from nav-global-import-mobile into main
build-frontend / build (push) Successful in 1m29s
2026-06-07 10:41:12 -04:00
justin b90ba53a3f Merge pull request 'GEDCOM: duplicate-aware import + maiden/married + RELI/NOTE mapping' (#21) from gedcom-import-dedupe into main
build-frontend / build (push) Has been cancelled
build-backend / build (push) Successful in 30s
2026-06-07 10:41:08 -04:00
justin c4e9d69e00 Merge pull request 'Alternate names, self-person link, deletion integrity + dangling people' (#20) from names-deletion-self into main
build-backend / build (push) Has been cancelled
build-frontend / build (push) Has been cancelled
2026-06-07 10:41:02 -04:00
justin 0673896133 Merge pull request 'Tree search + click-rebuild; searchable relationship picker; gender dropdown' (#19) from tree-search-combobox into main
build-frontend / build (push) Has been cancelled
2026-06-07 10:40:59 -04:00
justin 1164841950 Global Import in the menu; mobile drawer nav
- Add a top-level "Import" entry to the sidebar and a global /import page, so
  you can start a tree from a GEDCOM without first creating an empty one. The
  import flow now picks its destination (new tree, or an existing one) — the
  tree-scoped page reuses the same <GedcomImport> with a fixed destination and
  keeps Export.
- Extract the sidebar chrome into <AppShell> and give small screens a working
  menu: a hamburger opens the full sidebar as a slide-in drawer (it was just a
  logo + "Trees" link before). Used by both /trees and /import.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:40:01 -04:00
justin 5824e70895 GEDCOM: duplicate-aware import + typed name/attribute mapping
Duplicate detection (the "merge / skip / overwrite" the user asked for):
- New POST /gedcom/preview dry-runs the file and flags incoming people that
  resemble existing ones (name similarity via difflib + birth-year guard;
  high/medium score). No writes.
- /gedcom/import takes default_action (new|skip|merge|overwrite) + per-xref
  resolutions {xref: {action, target_id}}:
    new       create as a new person (current behavior)
    skip      link families to the existing person, copy nothing
    merge     attach the incoming names (as alternates), events, citations,
              and notes onto the existing person
    overwrite soft-delete the existing person, import the incoming one fresh
  Relationship creation is deduped so a merge can't double an edge.

Richer record mapping (covers the user's repo's GEDCOM):
- Multiple NAME records honor their TYPE; _MARNM (and NICK) import as typed
  alternate names — maiden stays primary, married becomes a "married" Name.
- RELI -> a "religion" event with the value in detail; OCCU/EDUC values too.
- NOTE -> person notes (and event notes); NOTE/RELI are no longer "unmapped".
- Export round-trips name TYPE.

Verified against the user's 2185-person export: 0 unmapped tags. 48 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:35:55 -04:00
justin 04ccdbf96a Alternate names (maiden/married), self-person link, deletion integrity
Names (the genealogy standard: maiden name primary, married/alias as typed
alternates):
- Name model already supported multiple typed names; expose full CRUD —
  NameCreate/Read/Update schemas, name_service (one-primary invariant,
  promote-on-delete), nested /persons/{id}/names routes.
- Person page gains a Names card: add/edit/delete + "make primary", with a
  curated name_type dropdown (birth/maiden, married, alias, nickname, …).

Self-person ("who am I"):
- users.self_person_id FK (use_alter for the users<->persons<->trees cycle)
  + migration; PATCH /users/me/self-person; "This is me" / "This is you"
  on the person page. Soft-deleting the linked person clears it.

Deletion integrity (fixes the broken tree view):
- delete_person now soft-deletes the relationships touching the person, so no
  dangling edges remain; family-chart also filters links to missing people.
- Optional cascade=true recursively deletes descendants (GEDCOM cleanup);
  the person page asks "only this person" vs "with all descendants".
- DELETE returns {deleted: n}.

Family view surfaces "Not connected to anyone" so dangling people aren't lost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:21:12 -04:00
justin f165ccb941 Tree search + click-rebuild; searchable relationship picker; gender dropdown
- Tree page: add a "Find a person" search box that jumps the chart to a
  match and rebuilds the hourglass (parents/grandparents/partner/children)
  around them. Clicking any card recenters via family-chart's default
  behavior (setAncestryDepth 3 / setProgenyDepth 2), syncing focus through
  setAfterUpdate for the "Open profile" link.
- Person detail: replace the relationship "add" <select> with a
  type-to-filter PersonCombobox so long people lists are searchable.
- Person detail: gender is now a Male/Female dropdown, not free text.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 09:58:45 -04:00
justin e0fb924a1d Merge pull request 'Full-CRUD sweep (API): update for tree/source/citation/relationship/media' (#18) from crud-sweep-backend into main
build-backend / build (push) Successful in 30s
2026-06-07 09:53:18 -04:00
justin cf5518c7ec Full-CRUD sweep: update endpoints for tree, source, citation, relationship, media
Closes the rule #8 gap at the API layer: PATCH endpoints + service updates for Tree (name/description/visibility), Source, Citation (page/detail/confidence), Relationship (qualifier/notes), and Media (title/attachment) — editor-gated and audited. Every core entity now has create/read/update/delete. Edit UIs for these land in the frontend batch. 37 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 09:53:17 -04:00
justin 26df03cfd7 Merge pull request 'Edit people + events; existing-person picker; full-CRUD rule' (#17) from crud-edits into main
build-backend / build (push) Successful in 27s
build-frontend / build (push) Successful in 1m25s
2026-06-07 09:35:56 -04:00
justin ab064bce6e Edit UI for people and life events; existing-person picker in family view
Person detail: an Edit form for name + gender + living status + privacy, and inline edit of each life event (type + structured date). Family view: the add-relative buttons now search existing people (link the real person) or create new — preventing duplicate spouses/parents — and adding a child to someone with one spouse links both parents.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 09:35:55 -04:00
justin 76b7f453c1 Add update (CRUD) for events and people; record the full-CRUD invariant
Events and people are now editable, not write-once: PATCH /events/{id} (type, structured date, place, notes) and PATCH /persons/{id} (vitals, privacy, and the primary name's given/surname). CLAUDE.md gains rule #8: every stored object must support full CRUD in API and UI — historical research is constant correction. Tests cover both updates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 09:35:55 -04:00
justin 438d2db2e7 Merge pull request 'Tree layout toggles + fan + card->profile + server search' (#16) from phase2-tree-toggles into main
build-frontend / build (push) Successful in 1m26s
2026-06-07 08:01:32 -04:00
justin 99913ada94 Tree layout toggles (landscape/portrait/fan), card->profile, server search
Tree page gets Landscape/Portrait/Fan toggles: landscape & portrait via family-chart's orientation; a hand-rolled radial Fan chart of ancestors (rings per generation, click to recenter). Clicking a card recenters and updates an 'Open <name> →' link to that person's profile. The People directory search now hits the server-side pg_trgm fuzzy endpoint (debounced) so it spans the whole tree, not just the loaded page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 08:01:31 -04:00
justin 584b323121 Merge pull request 'Fuzzy search (pg_trgm) + living-person protection' (#15) from phase2-search-privacy into main
build-backend / build (push) Successful in 30s
2026-06-07 07:55:14 -04:00
justin 4788ae7723 Add fuzzy name search (pg_trgm) and living-person protection
Fuzzy search: pg_trgm extension + trigram GIN indexes on name parts and a GET /trees/{id}/persons?q= search ranked by trigram similarity (finds Mueller for 'muller'), privacy-filtered. Living-person protection: the privacy engine now derives possibly-living status (explicit flag, else no death fact + birth within ~100y or unknown) and returns 'redacted' for non-members of public/unlisted trees; the service minimises those records ('Living person', no vitals). Members are unaffected. 31 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-07 07:55:13 -04:00
justin 51f0066e61 Merge pull request 'Interactive Tree view (pan/zoom genealogy chart)' (#14) from interactive-tree into main
build-frontend / build (push) Successful in 1m21s
2026-06-06 23:07:04 -04:00
justin bfa6c0782a Add an interactive Tree view (pan/zoom genealogy chart)
Researched how FamilySearch/Geni/MyHeritage lay out trees (switchable pedigree/portrait/fan, an interactive canvas with pan/zoom + click-to-recenter, gender colors, birth-death years) and built a real Tree page on the MIT d3 library family-chart instead of a flat list. Ancestors + descendants around a focus person, click any card to recenter, drag to pan, scroll to zoom — scales to large imported trees. Tree is now the first per-tree sidebar item and the default when opening a tree; People keeps the searchable directory + add/edit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 23:07:02 -04:00
justin 2f21e767f3 Merge pull request 'Scalable people directory' (#13) from people-directory into main
build-frontend / build (push) Successful in 1m20s
2026-06-06 22:54:10 -04:00
justin f6bcf198ee Make the people index a scalable scrollable directory
A flat wrap of every person didn't scale to imported trees. Replace it with a bounded (max-height, scrollable) searchable directory: clean name + birth–death-year rows, focus highlight, a result count, and a 200-row cap with a 'refine your search' notice so a thousand-person tree stays fast and usable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:54:08 -04:00
justin b13fafd624 Merge pull request 'Phase 2: GEDCOM import/export' (#12) from phase2-gedcom into main
build-backend / build (push) Successful in 26s
build-frontend / build (push) Successful in 1m22s
2026-06-06 22:46:50 -04:00
justin 631d050540 Add GEDCOM Import/Export UI (defaults to importing into a new tree)
An Import/Export page (sidebar) that defaults to importing into a NEW tree to avoid duplicating existing people, with an explicit 'append to this tree' option (warned), a mapping-report display (counts + skipped tags), and a one-click .ged export download.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:46:48 -04:00
justin d48029a407 Add GEDCOM import/export
A pragmatic GEDCOM parser + mapper: import reads INDI/FAM/SOUR and creates people, names, life events, partnership + qualified parent-child relationships, marriage events, places (deduped), sources, and citations from SOUR refs — returning a mapping report (counts + unmapped tags). Export serializes the tree back to GEDCOM (families derived from the edge model). Import is additive (no merge) and runs inline for now. Round-trip test passes; 29 tests total.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:46:48 -04:00
justin 18dea507d1 Merge pull request 'Pedigree connector lines + 4 grandparents' (#11) from pedigree-connectors into main
build-frontend / build (push) Successful in 1m22s
2026-06-06 22:32:12 -04:00
justin 99a660485e Pedigree: connector lines + correct 4-grandparent structure
Rebuilds the family view's pedigree as a recursive bracket chart with CSS connector lines — focus links to its two parents (2 lines), and each parent links to its two parents (4 lines to grandparents). Fixes the prior ambiguity where grandparent slots weren't tied to a specific parent: now every parent shows its own two parent slots, so a person clearly has up to four grandparents grouped by lineage. Height-robust connectors (each leaf draws its own spine half + stub).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:32:10 -04:00
justin cf6dcf9ce2 Merge pull request 'Family view + soft-delete/recovery' (#10) from phase1-familyview into main
build-backend / build (push) Successful in 30s
build-frontend / build (push) Successful in 1m18s
2026-06-06 22:19:02 -04:00
justin 22bc536978 Rebuild People as a family view (pedigree + family group); add recovery UI
The People page is no longer a flat list: it's a focus-person family view with a pedigree of ancestors (parents + grandparents), a spouse/partner panel, and a children panel — with inline 'add parent/child/spouse' (creates the person + the relationship), click-to-refocus, birth–death years, and a searchable people index. Modeled on how real genealogy tools center on a person and let you walk the graph.

Adds delete/restore UI: a Delete on the person page, per-tree delete + a 'Recently deleted' restore section on the trees list, and a Recovery page (sidebar) for deleted people.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:19:01 -04:00
justin f2205b93f4 Add soft-delete + recovery and tree-wide graph endpoints
Tree and person soft-delete + restore (owner-only for trees, editor for people) with recovery listings (?deleted=true); the worker already purges past the 30-day window. Adds tree-wide GET /relationships and /events so the family/pedigree view loads the whole graph in a few calls. 27 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 22:19:01 -04:00
justin b0c7c8570b Merge pull request 'App-shell UI overhaul + media stream endpoint' (#9) from ui-shell into main
build-backend / build (push) Successful in 26s
build-frontend / build (push) Successful in 1m20s
2026-06-06 21:56:26 -04:00
justin fe9a95c60d Rebuild the UI as an app shell: left sidebar, media gallery, structured events
Replaces the centered single-column of full-width cards with a proper application layout: a persistent left sidebar (Trees, and per-tree People/Sources/Media, with the tree name and sign-out) and a constrained content column. Marketing landing and auth pages are split out (own header/footer; centered auth with the logo).

Adds a Media gallery (upload + image thumbnails / file tiles, served via the backend content endpoint). Events are no longer free-text: a curated event-type list (+ custom) and a structured date (qualifier + day/month/year) that composes a proper genealogical date. Regenerated the OpenAPI client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 21:56:05 -04:00
justin bd8ee9b647 Stream media through the backend (browser-reachable, privacy-checked)
Presigned URLs point at the internal minio:9000 host a browser can't reach. Add ObjectStore.get_object and a GET /media/{id}/content endpoint that resolves visibility and streams the bytes; MediaRead.url now points there. Keeps the object store private and downloads behind the privacy engine.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 21:56:04 -04:00
justin 660130f007 Merge pull request 'Phase 1: media (object storage) + background worker' (#8) from phase1-media into main
build-backend / build (push) Successful in 30s
2026-06-06 21:46:35 -04:00
justin 34d30e3134 Add media (object storage) and the background worker (Phase 1)
Media model + migration; an ObjectStore interface with an S3/MinIO (boto3) implementation behind the service layer. Upload (multipart) stores bytes in object storage + a metadata row (checksum, size, content-type, optional attach to person/event/source); list returns presigned URLs; delete is soft. Editor-gated, privacy-filtered, audited. 24 tests pass (object store faked).

Introduces the worker container (same image, 'python -m app.worker'): its first job is the scheduled 30-day soft-delete purge across tables + media object cleanup. Compose gains worker + S3 env on backend/worker; dev override builds the worker too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 21:46:09 -04:00
justin 049545fcc8 Merge pull request 'Frontend redesign: real type, hero, depth' (#7) from design-overhaul into main
build-frontend / build (push) Successful in 1m18s
2026-06-06 21:34:48 -04:00
justin 3a14fcc4ca Redesign the frontend: real type, hero landing, depth
Lifts the UI from wireframe to a finished heritage look: Fraunces (display serif) + Inter (sans) via next/font; a proper hero landing with a feature triad and the Origin mark; a warm bronze-tinted background gradient for depth; a sticky branded header and refined footer. Polished button (sizes + bronze focus ring + shadow), card (rounded-xl, soft layered shadow), and input (bronze focus) primitives that carry across every page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 21:34:47 -04:00
justin fc4cb0273e Merge pull request 'Phase 1: sources-first spine (sources + citations)' (#6) from phase1-sources into main
build-backend / build (push) Successful in 24s
build-frontend / build (push) Successful in 1m18s
2026-06-06 13:17:34 -04:00
justin 83f83ab641 Add source manager and inline citing with 'sourced' badges
New /trees/[id]/sources page (list + create sources). Person-detail page now loads tree sources + citations and shows a '✓ N sourced' badge with an inline cite picker (source + page) on each event and on the person. Tree view links to Sources. Regenerated the OpenAPI client.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 13:17:33 -04:00
justin 064bb6ea65 Add sources and citations API (Phase 1: sources-first spine)
Source CRUD (reusable, tree-scoped) and Citation create/list/soft-delete linking one source to exactly one fact (person/event/name/relationship). Editor-gated writes, privacy-filtered reads, audit throughout; tenant + existence validation on source and target. list_citations returns all tree citations so the UI can render 'sourced' indicators in one round-trip. 22 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 13:17:33 -04:00
justin fbb9d0195c Merge pull request 'Phase 1: events + relationships + person detail' (#5) from phase1-graph into main
build-backend / build (push) Successful in 27s
build-frontend / build (push) Successful in 1m16s
2026-06-06 12:11:11 -04:00
justin 1f25eb2f21 Add person-detail page with events timeline and relationships
New /trees/[id]/persons/[personId] view: life-events timeline with add/remove, and relationships grouped into parents/children/partners/siblings with an add form (kind + person picker + qualifier). People in the tree list now link here. Regenerated the OpenAPI client for the new endpoints.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 12:10:56 -04:00
justin d6e2df4a61 Add events and relationships API (Phase 1: flesh out the graph)
Events (create/list-per-person/soft-delete) and relationships (create/list-per-person/soft-delete) through the layered stack: editor-gated writes, privacy-engine reads, audit on every change. Events carry exactly one subject (person XOR partnership); relationships are typed qualified edges (parent_child gets a biological/adoptive/step/foster/donor/guardian qualifier). Adds a single-person GET. 18 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 12:10:56 -04:00
justin a799d101b5 Merge pull request 'Use host global Watchtower (drop bundled one)' (#4) from watchtower-use-host into main 2026-06-06 11:58:50 -04:00
justin 0b9d72c878 Drop bundled Watchtower; rely on the host's global Watchtower
ripper already runs a single global nickfedor/watchtower (label-enabled) that watches every stack; the bundled containrrr/watchtower was redundant and crash-looped (its Docker API client is too old for Docker 29). Keep the watchtower.enable labels on backend/frontend so the host instance auto-deploys them; remove the per-stack service and profile.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
2026-06-06 11:58:49 -04:00
justin 2d0635e710 Merge pull request 'Add Watchtower auto-deploy (2-min poll)' (#3) from watchtower-autodeploy into main 2026-06-06 11:55:51 -04:00
177 changed files with 30188 additions and 290 deletions
+31 -5
View File
@@ -19,6 +19,7 @@ These are product invariants, not preferences. Do not violate them, and flag any
5. **Sources are first-class.** Don't model citations as free-text afterthoughts. A `Source` is a reusable entity; a `Citation` links it to a specific fact. 5. **Sources are first-class.** Don't model citations as free-text afterthoughts. A `Source` is a reusable entity; a `Citation` links it to a specific fact.
6. **Only legal data sources.** Ship scrapers/connectors only for permissible sources (FamilySearch API, Find A Grave, WikiTree, BLM/GLO, USGS, public-domain newspapers, public county records). Never add connectors for paywalled/terms-prohibited sites (Ancestry, MyHeritage, 23andMe). 6. **Only legal data sources.** Ship scrapers/connectors only for permissible sources (FamilySearch API, Find A Grave, WikiTree, BLM/GLO, USGS, public-domain newspapers, public county records). Never add connectors for paywalled/terms-prohibited sites (Ancestry, MyHeritage, 23andMe).
7. **Everything is configurable via environment.** Auth, mail, object storage, database, model providers, scrapers — all twelve-factor. No hard-coded endpoints or keys. 7. **Everything is configurable via environment.** Auth, mail, object storage, database, model providers, scrapers — all twelve-factor. No hard-coded endpoints or keys.
8. **Full CRUD on every object.** Every stored entity (person, name, event, relationship, source, citation, media, tree, …) must support create, read, **update**, and delete — in the API *and* the UI. Historical research is constant correction and new information, so nothing is write-once. Any new feature or data type ships with all four operations; an entity you can create but not edit is a bug.
## Tech stack ## Tech stack
@@ -29,6 +30,7 @@ These are product invariants, not preferences. Do not violate them, and flag any
- **Object storage:** S3-compatible (MinIO for self-host). - **Object storage:** S3-compatible (MinIO for self-host).
- **Edge:** Caddy reverse proxy; optional Cloudflare Tunnel (preferred ingress, never required). - **Edge:** Caddy reverse proxy; optional Cloudflare Tunnel (preferred ingress, never required).
- **Email:** operator-configured SMTP. - **Email:** operator-configured SMTP.
- **Model providers:** pluggable `LLMProvider` + `EmbeddingProvider` abstraction (ABCs) with Null / Anthropic / OpenAI-compatible (OpenAI, xAI, Ollama) implementations; an operator configures one or more via env and they're selectable by name through a registry (per-tree AI policy + `default_llm_provider`/`default_embedding_provider`).
- **CI/CD:** Gitea Actions build per-component images. **Push** to the LAN registry `192.168.0.2:1234` (plain HTTP, bypasses Cloudflare's body limit); **pull** via the public `git.jpaul.io` FQDN. Servers pull to deploy — no host build. Mirrors the drawbar setup; see [[gitea-lan-push-fqdn-pull]]. - **CI/CD:** Gitea Actions build per-component images. **Push** to the LAN registry `192.168.0.2:1234` (plain HTTP, bypasses Cloudflare's body limit); **pull** via the public `git.jpaul.io` FQDN. Servers pull to deploy — no host build. Mirrors the drawbar setup; see [[gitea-lan-push-fqdn-pull]].
Pick libraries consistent with this stack. If you introduce a significant dependency or a new service, note it in ARCHITECTURE.md in the same change. Pick libraries consistent with this stack. If you introduce a significant dependency or a new service, note it in ARCHITECTURE.md in the same change.
@@ -38,17 +40,24 @@ Pick libraries consistent with this stack. If you introduce a significant depend
``` ```
/ # docs and project meta (this file, README, LICENSE, COC, CONTRIBUTING) / # docs and project meta (this file, README, LICENSE, COC, CONTRIBUTING)
/docs # PRD.md, ARCHITECTURE.md /docs # PRD.md, ARCHITECTURE.md
/backend # FastAPI service (uv-managed). app/{api/v1, services (+ privacy engine), repositories, models, schemas, integrations (auth/mailer), core}; migrations/ = Alembic /backend # FastAPI service (uv-managed). app/{api/v1, services (+ privacy engine), repositories, models, schemas, integrations (auth, mailer, objectstore, models = pluggable LLM/embedding providers), core}; migrations/ = Alembic
/deploy # docker-compose.yml, Caddyfile, .env.example — the self-host stack /deploy # docker-compose.yml (+ docker-compose.dev.yml), Caddyfile, .env.example, backup.sh + BACKUP.md (one-command pg_dump + MinIO backup) — the self-host stack
/.gitea/workflows # Gitea Actions CI (build images → Gitea registry) /.gitea/workflows # Gitea Actions CI (build images → Gitea registry)
/frontend # Next.js (App Router, TS, Tailwind, shadcn-style UI). app/ pages, lib/api generated OpenAPI client, components/ui /frontend # Next.js (App Router, TS, Tailwind, shadcn-style UI). app/ pages, lib/api generated OpenAPI client, components/ui
``` ```
Phase 0 is landing **deploy-first**: the compose stack (Postgres + MinIO + Caddy + a minimal FastAPI backend exposing `/health` and `/health/ready`) and CI come before the real data model and the frontend. Backend dependencies are managed with **uv**; migrations use **Alembic**. The core data model (ARCHITECTURE §5), **local auth** (Argon2 passwords, backend-issued sessions, email verify/reset behind the `AuthProvider` interface; API auth via Bearer header or HttpOnly cookie), and the **Next.js frontend scaffold** (Tailwind + shadcn-style UI, generated OpenAPI client, auth + tree/person views) have all landed — **Phase 0 is complete and running on the live deployment.** Phase 1 (core tree features — media, soft-delete recovery, richer CRUD) is next; OIDC/social auth is Phase 5. Keep this section current as the tree grows. Phase 0 landed **deploy-first**: the compose stack (Postgres + MinIO + Caddy + FastAPI backend) and CI before the data model and frontend. Backend deps use **uv**; migrations use **Alembic**. Status (keep current as the tree grows):
- **Phase 0 — Foundation: complete** and running live (core data model, local auth behind `AuthProvider`, Next.js frontend).
- **Phase 1 — Core tree: complete.** Media (upload/serve), soft-delete + recovery UI, full CRUD across entities, and the 4-level tree visibility/privacy model (#41#51).
- **Phase 2 — substantially landed.** GEDCOM import (preview→apply, duplicate-aware) and export (citation-preserving, #232); fuzzy name search (pg_trgm) + the public `/explore` directory. Living-person protection is still hardening.
- **Phase 4 — AI assistant foundations landed.** Pluggable `LLMProvider`/`EmbeddingProvider` abstraction + multi-provider registry (Anthropic/OpenAI/xAI/Ollama, #235/#237), the **ChangeProposal** propose-then-confirm flow (#236), and per-tree AI model policy (#238). The assistant's *tool surface that emits proposals* is the remaining piece.
- Also shipped: tree membership management (#233), an **instance owner/operator** role (`OWNER_EMAIL`, #240), a schema-drift readiness guard (#239), and a one-command operator backup (#234).
- **Not built yet:** Phase 3 (Property — parcels/deeds/chain-of-title; no property models exist), Phase 5 (OIDC/social auth — only the `AuthProvider` ABC exists), and cross-tree hints (last; needs multiple populated trees + the embedding provider).
## Where to start ## Where to start
The roadmap is phased in PRD §8. Build in dependency order. **Phase 0 — Foundation is complete** and running on the live deployment; **Phase 1 (core tree features) is the current target.** For reference, Phase 0 covered: The roadmap is phased in PRD §8. Build in dependency order. **Phases 0 and 1 are complete**, Phase 2 is substantially done, and Phase 4's AI foundations have shipped (see the status list above). The biggest unbuilt areas are **Phase 3 (Property)** and **Phase 5 (OIDC/social auth)** — likely current targets. For reference, Phase 0 covered:
1. Backend skeleton (FastAPI, async, layered) + Postgres + migrations 1. Backend skeleton (FastAPI, async, layered) + Postgres + migrations
2. Core data model from ARCHITECTURE §5 — start with User, Tree, TreeMembership, Person, Name, Relationship, Event, Place, Source, Citation, AuditEntry, soft-delete support 2. Core data model from ARCHITECTURE §5 — start with User, Tree, TreeMembership, Person, Name, Relationship, Event, Place, Source, Citation, AuditEntry, soft-delete support
@@ -57,7 +66,7 @@ The roadmap is phased in PRD §8. Build in dependency order. **Phase 0 — Found
5. The deploy stack: `compose` for app + postgres + objectstore, Caddy config, env-driven settings 5. The deploy stack: `compose` for app + postgres + objectstore, Caddy config, env-driven settings
6. CI/CD: Gitea Actions building images to the registry 6. CI/CD: Gitea Actions building images to the registry
Don't get ahead of the phases. GEDCOM lands before the assistant (so AI writes target a stable model); property follows a tested people graph; hints come last because they need multiple populated trees. If you think the order is wrong, raise it rather than reordering silently. Don't get ahead of the phases. GEDCOM and the assistant's propose-diff foundation (provider abstraction + ChangeProposal approval flow) have shipped; the remaining dependency-ordered work is **Property** (Phase 3, on top of the tested people graph), then richer collaboration/audit UI, with **cross-tree hints last** (they need multiple populated trees and the embedding provider). If you think the order is wrong, raise it rather than reordering silently.
## Conventions ## Conventions
@@ -68,6 +77,23 @@ Don't get ahead of the phases. GEDCOM lands before the assistant (so AI writes t
- **Privacy/assistant/hint code gets extra care** — these are the areas where bugs do real harm. Prefer a design note before a large change. - **Privacy/assistant/hint code gets extra care** — these are the areas where bugs do real harm. Prefer a design note before a large change.
- **No secrets in the repo.** Config via env; provide `.env.example` with placeholders. - **No secrets in the repo.** Config via env; provide `.env.example` with placeholders.
## Patched dependencies (family-chart)
The tree view uses **family-chart** (d3-based). Two adjustments live in the repo:
- **CSS is vendored** at `frontend/app/trees/[id]/tree/chart.css` — the package blocks its CSS subpath export, so we copy it in.
- **The library is patched** via `patch-package` (`frontend/patches/family-chart+0.9.0.patch`, applied by the `postinstall` hook; the backend/frontend Dockerfiles `COPY patches` before install). Both hunks touch `dist/family-chart.js` **and** `dist/family-chart.esm.js` (the app loads the `esm` build). Current fixes:
1. **Spouse-centering layout** (`setupSpouses` / `sortChildrenWithSpouses`) — center a person between two spouses with children under the correct pair.
2. **`cardToMiddle` vertical centering** — the lib scaled `datum.x` by the zoom factor `k` but not `datum.y`, so "fly to a node" drifted vertically at any zoom ≠ 1; we add the missing `* k`.
To change a patch: edit the file(s) under `node_modules/family-chart/dist/`, then `cd frontend && npx patch-package family-chart` to regenerate, and verify with `npx patch-package --error-on-fail`.
**Upstreamed.** Both are general library bugfixes, not app-specific, and are submitted upstream:
- `cardToMiddle` vertical centering — **donatso/family-chart#103** (issue **#102**).
- Multi-spouse centered layout — **donatso/family-chart#105** (issue **#104**).
If either is merged + released, bump `family-chart`, drop the corresponding patch hunk, **and** remove any in-app compensation (e.g. the `cardToMiddle` caller in `tree/page.tsx` passes raw `y` precisely because the patch fixes it — pre-scaling there too would double-correct). Until then, keep the patch.
## License & contribution terms ## License & contribution terms
Provenance is **source-available** under **BUSL-1.1** (see [LICENSE](LICENSE)): free for personal/family/non-commercial use, no third-party commercial hosting, and each release converts to **AGPL-3.0** four years after it ships. The DCO sign-off keeps the licensing chain clean so the maintainer can manage that conversion and a possible future hosted offering. Don't add code under an incompatible license, and don't vendor dependencies whose licenses conflict with eventual AGPL distribution. Provenance is **source-available** under **BUSL-1.1** (see [LICENSE](LICENSE)): free for personal/family/non-commercial use, no third-party commercial hosting, and each release converts to **AGPL-3.0** four years after it ships. The DCO sign-off keeps the licensing chain clean so the maintainer can manage that conversion and a possible future hosted offering. Don't add code under an incompatible license, and don't vendor dependencies whose licenses conflict with eventual AGPL distribution.
+5 -4
View File
@@ -19,13 +19,14 @@ Every fact links to its source. Every claim can be traced. Nothing is just asser
## What it does ## What it does
- **Build a tree that holds up.** People, relationships, events, and places — with every fact linked to the document, photo, or record it came from. - **Build a tree that holds up.** People, relationships, events, and places — with every fact linked to the document, photo, or record it came from.
- **Trace the land, not just the family.** Properties are first-class. Record ownership events (grants, deeds, inheritances, sales), reconstruct chain-of-title, and tie parcels to the people who held them.
- **Bring your own archive.** Scans, PDFs, photos, audio recordings — first-class citizens, not afterthoughts. - **Bring your own archive.** Scans, PDFs, photos, audio recordings — first-class citizens, not afterthoughts.
- **A research assistant that proposes, never overwrites.** The built-in AI assistant searches legal sources, lays out what it found, and waits for your approval before anything touches your data. You can point it at the major model providers or a self-hosted model — your keys, your choice. - **A research assistant that proposes, never overwrites.** The built-in AI assistant searches legal sources, lays out what it found, and waits for your approval before anything touches your data. You can point it at the major model providers or a self-hosted model — your keys, your choice.
- **Standards over silos.** Full GEDCOM 7 import and export. Migrate in, migrate out. - **Standards over silos.** GEDCOM import and export (5.5.1 / 7 common subset) — duplicate-aware import, citation-preserving export. Migrate in, migrate out.
- **Privacy you control.** Public, unlisted, or private per tree; any individual can be hidden; living people are protected by default. - **Privacy you control.** Public, members-only (any signed-in user on your instance), unlisted, or private per tree; any individual can be hidden; living people are protected by default.
- **Find your people.** When another user's tree overlaps with yours, Provenance can surface an anonymous "possible match" — and only connects you if you both say yes. - **Find your people.** When another user's tree overlaps with yours, Provenance can surface an anonymous "possible match" — and only connects you if you both say yes.
- **Run it your way.** Container-native. Self-host behind Caddy and, if you like, a Cloudflare Tunnel. Multi-tenant, so your whole extended family — or a whole community of strangers — can coexist on one deployment. - **Run it your way.** Container-native. Self-host behind Caddy and, if you like, a Cloudflare Tunnel. Multi-tenant, so your whole extended family — or a whole community of strangers — can coexist on one deployment. One-command backups (Postgres + object storage) and an instance-owner admin role keep operations in your hands.
**Where it's headed — trace the land, not just the family.** The same source-backed treatment for *property*: parcels, deeds, and ownership events, reconstructing chain-of-title and tying land to the people who held it. The people side ships today; the land half is on the roadmap, not yet built — but it's why Provenance exists, not an afterthought.
## Who it's for ## Who it's for
+4
View File
@@ -21,7 +21,11 @@ RUN --mount=type=cache,target=/root/.cache/uv \
COPY app ./app COPY app ./app
COPY alembic.ini ./alembic.ini COPY alembic.ini ./alembic.ini
COPY migrations ./migrations COPY migrations ./migrations
COPY docker-entrypoint.sh ./docker-entrypoint.sh
RUN chmod +x ./docker-entrypoint.sh
EXPOSE 8000 EXPOSE 8000
# The entrypoint runs migrations first when RUN_MIGRATIONS=1, then the command.
ENTRYPOINT ["./docker-entrypoint.sh"]
CMD ["uv", "run", "--no-dev", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] CMD ["uv", "run", "--no-dev", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
+134
View File
@@ -10,6 +10,10 @@ from app.core.db import get_session
from app.integrations.mailer.base import Mailer from app.integrations.mailer.base import Mailer
from app.integrations.mailer.console import ConsoleMailer from app.integrations.mailer.console import ConsoleMailer
from app.integrations.mailer.smtp import SMTPMailer from app.integrations.mailer.smtp import SMTPMailer
from app.integrations.models.base import EmbeddingProvider, LLMProvider
from app.integrations.models.null import NullEmbeddingProvider, NullLLMProvider
from app.integrations.objectstore.base import ObjectStore
from app.integrations.objectstore.s3 import S3ObjectStore
from app.models.user import User from app.models.user import User
from app.services import auth_service from app.services import auth_service
@@ -38,6 +42,48 @@ async def get_current_user(request: Request, session: SessionDep) -> User:
CurrentUser = Annotated[User, Depends(get_current_user)] CurrentUser = Annotated[User, Depends(get_current_user)]
async def get_current_user_or_none(request: Request, session: SessionDep) -> User | None:
"""Optional auth for public read endpoints — never raises. Returns the user
when a valid session is present, else None (anonymous viewer)."""
raw_token = extract_session_token(request)
if raw_token is None:
return None
return await auth_service.resolve_session_user(session, raw_token=raw_token)
CurrentUserOrNone = Annotated[User | None, Depends(get_current_user_or_none)]
def is_instance_owner(user: User) -> bool:
"""Whether this account is an instance owner/operator — i.e. its email is
named in OWNER_EMAIL *and* that email has been verified. Instance ownership
is an operational/config role; it does NOT bypass the privacy engine or grant
access to others' tree data.
The verified-email requirement is load-bearing: registration is open and (by
default) doesn't require verification, so without it an attacker could claim
the owner email by registering it before the operator does — a land-grab to
the highest role with no proof of inbox control. Requiring verification ties
ownership to actual control of the named inbox regardless of the global
REQUIRE_EMAIL_VERIFICATION setting. (Self-hosts without SMTP can verify via
the link the console mailer prints to the operator-controlled logs.)"""
owners = get_settings().owner_emails()
return (
bool(owners)
and user.email_verified_at is not None
and user.email.strip().lower() in owners
)
async def require_instance_owner(current: CurrentUser) -> User:
if not is_instance_owner(current):
raise HTTPException(status.HTTP_403_FORBIDDEN, "instance owner only")
return current
InstanceOwner = Annotated[User, Depends(require_instance_owner)]
def get_mailer() -> Mailer: def get_mailer() -> Mailer:
settings = get_settings() settings = get_settings()
if settings.mailer == "smtp" and settings.smtp_host: if settings.mailer == "smtp" and settings.smtp_host:
@@ -46,3 +92,91 @@ def get_mailer() -> Mailer:
MailerDep = Annotated[Mailer, Depends(get_mailer)] MailerDep = Annotated[Mailer, Depends(get_mailer)]
def get_objectstore() -> ObjectStore:
return S3ObjectStore(get_settings())
ObjectStoreDep = Annotated[ObjectStore, Depends(get_objectstore)]
def build_llm_providers() -> dict[str, LLMProvider]:
"""Every LLM provider whose credentials are configured, keyed by name. Run
several at once; pick one with get_llm_provider(name)."""
from app.integrations.models.anthropic_provider import AnthropicLLMProvider
from app.integrations.models.openai_compat import OpenAICompatibleLLMProvider
s = get_settings()
providers: dict[str, LLMProvider] = {}
if s.anthropic_api_key:
providers["anthropic"] = AnthropicLLMProvider(
api_key=s.anthropic_api_key, model=s.anthropic_model, max_tokens=s.llm_max_tokens
)
if s.openai_api_key:
providers["openai"] = OpenAICompatibleLLMProvider(
api_key=s.openai_api_key, base_url=s.openai_base_url, model=s.openai_model,
max_tokens=s.llm_max_tokens,
)
if s.xai_api_key:
providers["xai"] = OpenAICompatibleLLMProvider(
api_key=s.xai_api_key, base_url=s.xai_base_url, model=s.xai_model,
max_tokens=s.llm_max_tokens,
)
if s.ollama_enabled:
providers["ollama"] = OpenAICompatibleLLMProvider(
api_key=None, base_url=s.ollama_base_url, model=s.ollama_model,
max_tokens=s.llm_max_tokens,
)
return providers
def configured_llm_providers() -> list[dict]:
"""Configured LLM providers as {name, model} — for the AI admin view (no
secrets). Mirrors build_llm_providers() without constructing clients."""
s = get_settings()
out: list[dict] = []
if s.anthropic_api_key:
out.append({"name": "anthropic", "model": s.anthropic_model})
if s.openai_api_key:
out.append({"name": "openai", "model": s.openai_model})
if s.xai_api_key:
out.append({"name": "xai", "model": s.xai_model})
if s.ollama_enabled:
out.append({"name": "ollama", "model": s.ollama_model})
return out
def get_llm_provider(name: str | None = None) -> LLMProvider:
"""The named LLM provider, or the configured default, or Null if unconfigured."""
providers = build_llm_providers()
return providers.get(name or get_settings().default_llm_provider) or NullLLMProvider()
LLMProviderDep = Annotated[LLMProvider, Depends(get_llm_provider)]
def build_embedding_providers() -> dict[str, EmbeddingProvider]:
from app.integrations.models.openai_compat import OpenAICompatibleEmbeddingProvider
s = get_settings()
providers: dict[str, EmbeddingProvider] = {}
if s.openai_api_key:
providers["openai"] = OpenAICompatibleEmbeddingProvider(
api_key=s.openai_api_key, base_url=s.openai_base_url,
model=s.openai_embedding_model, dimensions=s.embedding_dimensions,
)
if s.ollama_enabled:
providers["ollama"] = OpenAICompatibleEmbeddingProvider(
api_key=None, base_url=s.ollama_base_url,
model=s.ollama_embedding_model, dimensions=s.embedding_dimensions,
)
return providers
def get_embedding_provider(name: str | None = None) -> EmbeddingProvider:
providers = build_embedding_providers()
return providers.get(name or get_settings().default_embedding_provider) or NullEmbeddingProvider()
EmbeddingProviderDep = Annotated[EmbeddingProvider, Depends(get_embedding_provider)]
+14 -2
View File
@@ -12,6 +12,7 @@ from sqlalchemy import text
from app.core.config import get_settings from app.core.config import get_settings
from app.core.db import get_engine from app.core.db import get_engine
from app.core.schema_version import schema_is_current
router = APIRouter(tags=["health"]) router = APIRouter(tags=["health"])
@@ -33,9 +34,20 @@ async def ready(response: Response) -> dict:
try: try:
async with get_engine().connect() as conn: async with get_engine().connect() as conn:
await conn.execute(text("SELECT 1")) await conn.execute(text("SELECT 1"))
checks["database"] = "ok" checks["database"] = "ok"
# Schema drift = code ahead of the DB; queries would 500. Fail
# readiness loudly rather than serve a broken surface.
ok, db, expected = await schema_is_current(conn)
if not ok:
checks["schema"] = (
f"drift: db={sorted(db) or ['none']} expected={sorted(expected)} "
"— run 'alembic upgrade head'"
)
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
return {"status": "not ready", "checks": checks}
checks["schema"] = "ok"
return {"status": "ready", "checks": checks} return {"status": "ready", "checks": checks}
except Exception as exc: # noqa: BLE001 — surface any failure as "not ready" except Exception as exc: # noqa: BLE001 — surface any failure as "not ready"
checks["database"] = "error" checks.setdefault("database", "error")
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
return {"status": "not ready", "checks": checks, "detail": str(exc)} return {"status": "not ready", "checks": checks, "detail": str(exc)}
+32 -1
View File
@@ -2,10 +2,41 @@
from fastapi import APIRouter from fastapi import APIRouter
from app.api.v1 import auth, persons, trees, users from app.api.v1 import (
admin,
ai,
auth,
citations,
cleanup,
events,
gedcom,
media,
members,
names,
persons,
proposals,
public,
relationships,
sources,
trees,
users,
)
api_router = APIRouter(prefix="/api/v1") api_router = APIRouter(prefix="/api/v1")
api_router.include_router(auth.router) api_router.include_router(auth.router)
api_router.include_router(users.router) api_router.include_router(users.router)
api_router.include_router(trees.router) api_router.include_router(trees.router)
api_router.include_router(persons.router) api_router.include_router(persons.router)
api_router.include_router(names.router)
api_router.include_router(events.router)
api_router.include_router(relationships.router)
api_router.include_router(sources.router)
api_router.include_router(citations.router)
api_router.include_router(media.router)
api_router.include_router(gedcom.router)
api_router.include_router(cleanup.router)
api_router.include_router(public.router)
api_router.include_router(members.router)
api_router.include_router(proposals.router)
api_router.include_router(ai.router)
api_router.include_router(admin.router)
+38
View File
@@ -0,0 +1,38 @@
"""Instance-admin surface — owner-only (OWNER_EMAIL). Operational status and
instance-wide configuration. Deliberately exposes no tree contents or PII:
instance ownership is an operator role, not a privacy bypass."""
from sqlalchemy import func, select
from fastapi import APIRouter
from app.api.deps import InstanceOwner, SessionDep, configured_llm_providers
from app.core.config import get_settings
from app.models.tree import Tree
from app.models.user import User
from app.schemas.admin import InstanceStatus
from app.schemas.ai_policy import ConfiguredProvider
router = APIRouter(prefix="/admin", tags=["admin"])
@router.get("/instance", response_model=InstanceStatus)
async def instance_status(owner: InstanceOwner, session: SessionDep) -> InstanceStatus:
"""Operator dashboard data. Requires the caller to be an instance owner."""
s = get_settings()
user_count = await session.scalar(
select(func.count()).select_from(User).where(User.deleted_at.is_(None))
)
tree_count = await session.scalar(
select(func.count()).select_from(Tree).where(Tree.deleted_at.is_(None))
)
return InstanceStatus(
version=s.version,
env=s.app_env,
owner_emails=sorted(s.owner_emails()),
require_email_verification=s.require_email_verification,
user_count=user_count or 0,
tree_count=tree_count or 0,
default_llm_provider=s.default_llm_provider,
ai_providers=[ConfiguredProvider(**p) for p in configured_llm_providers()],
)
+34
View File
@@ -0,0 +1,34 @@
"""Per-tree AI model policy — owner-only admin view."""
import uuid
from fastapi import APIRouter
from app.api.deps import CurrentUser, SessionDep
from app.schemas.ai_policy import TreeAiPolicyRead, TreeAiPolicyUpdate
from app.services import ai_policy_service, tree_service
router = APIRouter(prefix="/trees", tags=["ai"])
@router.get("/{tree_id}/ai", response_model=TreeAiPolicyRead)
async def get_ai_policy(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> TreeAiPolicyRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
return TreeAiPolicyRead(**await ai_policy_service.get_policy(session, actor=current, tree=tree))
@router.patch("/{tree_id}/ai", response_model=TreeAiPolicyRead)
async def update_ai_policy(
tree_id: uuid.UUID, data: TreeAiPolicyUpdate, session: SessionDep, current: CurrentUser
) -> TreeAiPolicyRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
policy = await ai_policy_service.update_policy(
session,
actor=current,
tree=tree,
member_provider=data.member_provider,
recommender_provider=data.recommender_provider,
)
return TreeAiPolicyRead(**policy)
+14 -1
View File
@@ -1,9 +1,10 @@
from fastapi import APIRouter, HTTPException, Request, Response, status from fastapi import APIRouter, HTTPException, Request, Response, status
from app.api.deps import MailerDep, SessionDep, extract_session_token from app.api.deps import CurrentUser, MailerDep, SessionDep, extract_session_token
from app.core.config import get_settings from app.core.config import get_settings
from app.schemas.auth import ( from app.schemas.auth import (
LoginRequest, LoginRequest,
PasswordChange,
PasswordResetConfirm, PasswordResetConfirm,
PasswordResetRequest, PasswordResetRequest,
RegisterRequest, RegisterRequest,
@@ -79,3 +80,15 @@ async def reset_password(data: PasswordResetConfirm, session: SessionDep) -> Non
await auth_service.reset_password( await auth_service.reset_password(
session, raw_token=data.token, new_password=data.new_password session, raw_token=data.token, new_password=data.new_password
) )
@router.post("/change-password", status_code=status.HTTP_204_NO_CONTENT)
async def change_password(
data: PasswordChange, session: SessionDep, current: CurrentUser
) -> None:
await auth_service.change_password(
session,
user=current,
current_password=data.current_password,
new_password=data.new_password,
)
+60
View File
@@ -0,0 +1,60 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.source import CitationCreate, CitationRead, CitationUpdate
from app.services import citation_service, tree_service
router = APIRouter(prefix="/trees", tags=["citations"])
@router.post(
"/{tree_id}/citations", response_model=CitationRead, status_code=status.HTTP_201_CREATED
)
async def create_citation(
tree_id: uuid.UUID, data: CitationCreate, session: SessionDep, current: CurrentUser
) -> CitationRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
citation = await citation_service.create_citation(
session, actor=current, tree=tree, **data.model_dump()
)
return CitationRead.model_validate(citation)
@router.get("/{tree_id}/citations", response_model=list[CitationRead])
async def list_citations(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[CitationRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
citations = await citation_service.list_citations(session, viewer_id=current.id, tree=tree)
return [CitationRead.model_validate(c) for c in citations]
@router.patch("/{tree_id}/citations/{citation_id}", response_model=CitationRead)
async def update_citation(
tree_id: uuid.UUID,
citation_id: uuid.UUID,
data: CitationUpdate,
session: SessionDep,
current: CurrentUser,
) -> CitationRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
citation = await citation_service.update_citation(
session,
actor=current,
tree=tree,
citation_id=citation_id,
changes=data.model_dump(exclude_unset=True),
)
return CitationRead.model_validate(citation)
@router.delete("/{tree_id}/citations/{citation_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_citation(
tree_id: uuid.UUID, citation_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await citation_service.delete_citation(
session, actor=current, tree=tree, citation_id=citation_id
)
+130
View File
@@ -0,0 +1,130 @@
import uuid
from fastapi import APIRouter, File, UploadFile
from app.api.deps import CurrentUser, SessionDep
from app.schemas.cleanup import (
CleanupResult,
DeceasedApply,
DeceasedByChildCandidate,
DeceasedCandidate,
GenderApply,
GenderProposal,
NameApply,
NameIssue,
)
from app.services import cleanup_service, tree_service
router = APIRouter(prefix="/trees", tags=["cleanup"])
@router.get("/{tree_id}/cleanup/deceased", response_model=list[DeceasedCandidate])
async def preview_deceased(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
born_on_or_before: int = 1930,
) -> list[DeceasedCandidate]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.preview_deceased(
session, actor=current, tree=tree, year=born_on_or_before
)
return [DeceasedCandidate(**r) for r in rows]
@router.get(
"/{tree_id}/cleanup/deceased-by-child", response_model=list[DeceasedByChildCandidate]
)
async def preview_deceased_by_child(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
born_on_or_before: int = 1900,
) -> list[DeceasedByChildCandidate]:
"""People with a child born on/before the cutoff — necessarily deceased even
when their own birth date is missing. Apply via POST .../cleanup/deceased."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.preview_deceased_by_child(
session, actor=current, tree=tree, year=born_on_or_before
)
return [DeceasedByChildCandidate(**r) for r in rows]
@router.post("/{tree_id}/cleanup/deceased", response_model=CleanupResult)
async def apply_deceased(
tree_id: uuid.UUID, data: DeceasedApply, session: SessionDep, current: CurrentUser
) -> CleanupResult:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
n = await cleanup_service.apply_deceased(
session, actor=current, tree=tree, person_ids=data.person_ids
)
return CleanupResult(updated=n)
@router.post("/{tree_id}/cleanup/gender/preview", response_model=list[GenderProposal])
async def preview_gender(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
file: UploadFile = File(...),
) -> list[GenderProposal]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
text = (await file.read()).decode("utf-8", errors="replace")
rows = await cleanup_service.preview_gender(
session, actor=current, tree=tree, gedcom_text=text
)
return [GenderProposal(**r) for r in rows]
@router.get("/{tree_id}/cleanup/gender/guess", response_model=list[GenderProposal])
async def guess_gender(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[GenderProposal]:
"""Best-guess sex from first names (bundled dictionary) for people missing it."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.guess_gender_by_name(session, actor=current, tree=tree)
return [GenderProposal(**r) for r in rows]
@router.get("/{tree_id}/cleanup/gender/from-spouse", response_model=list[GenderProposal])
async def guess_gender_from_spouse(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[GenderProposal]:
"""Infer a missing sex from a partner whose sex is set (opposite-sex couple)."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.guess_gender_by_spouse(session, actor=current, tree=tree)
return [GenderProposal(**r) for r in rows]
@router.post("/{tree_id}/cleanup/gender", response_model=CleanupResult)
async def apply_gender(
tree_id: uuid.UUID, data: GenderApply, session: SessionDep, current: CurrentUser
) -> CleanupResult:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
n = await cleanup_service.apply_gender(
session,
actor=current,
tree=tree,
updates=[u.model_dump() for u in data.updates],
)
return CleanupResult(updated=n)
@router.get("/{tree_id}/cleanup/names", response_model=list[NameIssue])
async def preview_names(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[NameIssue]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await cleanup_service.preview_names(session, actor=current, tree=tree)
return [NameIssue(**r) for r in rows]
@router.post("/{tree_id}/cleanup/names", response_model=CleanupResult)
async def apply_names(
tree_id: uuid.UUID, data: NameApply, session: SessionDep, current: CurrentUser
) -> CleanupResult:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
n = await cleanup_service.apply_names(
session, actor=current, tree=tree, edits=[e.model_dump() for e in data.edits]
)
return CleanupResult(updated=n)
+67
View File
@@ -0,0 +1,67 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.event import EventCreate, EventRead, EventUpdate
from app.services import event_service, tree_service
router = APIRouter(prefix="/trees", tags=["events"])
@router.post("/{tree_id}/events", response_model=EventRead, status_code=status.HTTP_201_CREATED)
async def create_event(
tree_id: uuid.UUID, data: EventCreate, session: SessionDep, current: CurrentUser
) -> EventRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
event = await event_service.create_event(
session, actor=current, tree=tree, **data.model_dump()
)
return EventRead.model_validate(event)
@router.get("/{tree_id}/events", response_model=list[EventRead])
async def list_tree_events(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[EventRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
events = await event_service.list_events(session, viewer_id=current.id, tree=tree)
return [EventRead.model_validate(e) for e in events]
@router.get("/{tree_id}/persons/{person_id}/events", response_model=list[EventRead])
async def list_person_events(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[EventRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
events = await event_service.list_events_for_person(
session, viewer_id=current.id, tree=tree, person_id=person_id
)
return [EventRead.model_validate(e) for e in events]
@router.patch("/{tree_id}/events/{event_id}", response_model=EventRead)
async def update_event(
tree_id: uuid.UUID,
event_id: uuid.UUID,
data: EventUpdate,
session: SessionDep,
current: CurrentUser,
) -> EventRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
event = await event_service.update_event(
session,
actor=current,
tree=tree,
event_id=event_id,
changes=data.model_dump(exclude_unset=True),
)
return EventRead.model_validate(event)
@router.delete("/{tree_id}/events/{event_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_event(
tree_id: uuid.UUID, event_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await event_service.delete_event(session, actor=current, tree=tree, event_id=event_id)
+68
View File
@@ -0,0 +1,68 @@
import json
import uuid
from fastapi import APIRouter, File, Form, Response, UploadFile
from app.api.deps import CurrentUser, SessionDep
from app.schemas.gedcom import ImportPreview, ImportReport
from app.services import gedcom, tree_service
router = APIRouter(prefix="/trees", tags=["gedcom"])
@router.post("/{tree_id}/gedcom/preview", response_model=ImportPreview)
async def preview_gedcom(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
file: UploadFile = File(...),
) -> ImportPreview:
"""Dry run: report counts and incoming people that look like duplicates of
existing ones, so the user can choose how to resolve each before importing."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
text = (await file.read()).decode("utf-8", errors="replace")
report = await gedcom.preview_gedcom(session, actor=current, tree=tree, text=text)
return ImportPreview(**report)
@router.post("/{tree_id}/gedcom/import", response_model=ImportReport)
async def import_gedcom(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
file: UploadFile = File(...),
default_action: str = Form("new"),
resolutions: str = Form("{}"),
) -> ImportReport:
"""Import a GEDCOM. ``default_action`` (new|skip|merge|overwrite) applies to
incoming people that match an existing one; ``resolutions`` is a JSON object
{xref: {action, target_id}} overriding it per record."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
text = (await file.read()).decode("utf-8", errors="replace")
try:
parsed = json.loads(resolutions or "{}")
except json.JSONDecodeError:
parsed = {}
report = await gedcom.import_gedcom(
session,
actor=current,
tree=tree,
text=text,
default_action=default_action,
resolutions=parsed,
)
return ImportReport(**report)
@router.get("/{tree_id}/gedcom/export")
async def export_gedcom(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> Response:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
text = await gedcom.export_gedcom(session, viewer_id=current.id, tree=tree)
safe = "".join(c for c in tree.name if c.isalnum() or c in " -_").strip() or "tree"
return Response(
content=text,
media_type="text/plain",
headers={"Content-Disposition": f'attachment; filename="{safe}.ged"'},
)
+109
View File
@@ -0,0 +1,109 @@
import uuid
from fastapi import APIRouter, File, Form, Response, UploadFile, status
from app.api.deps import CurrentUser, ObjectStoreDep, SessionDep
from app.schemas.media import MediaRead, MediaUpdate
from app.services import media_service, tree_service
def _content_url(media) -> str:
return f"/api/v1/trees/{media.tree_id}/media/{media.id}/content"
def _read(media) -> MediaRead:
out = MediaRead.model_validate(media)
# Stream through the backend (privacy-checked, browser-reachable) rather
# than expose the internal object store directly.
out.url = _content_url(media)
return out
router = APIRouter(prefix="/trees", tags=["media"])
@router.post("/{tree_id}/media", response_model=MediaRead, status_code=status.HTTP_201_CREATED)
async def upload_media(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
file: UploadFile = File(...),
title: str | None = Form(None),
person_id: uuid.UUID | None = Form(None),
event_id: uuid.UUID | None = Form(None),
source_id: uuid.UUID | None = Form(None),
) -> MediaRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
data = await file.read()
media = await media_service.upload_media(
session,
store,
actor=current,
tree=tree,
data=data,
filename=file.filename or "upload",
content_type=file.content_type or "application/octet-stream",
title=title,
person_id=person_id,
event_id=event_id,
source_id=source_id,
)
return _read(media)
@router.get("/{tree_id}/media", response_model=list[MediaRead])
async def list_media(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[MediaRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
items = await media_service.list_media(session, viewer_id=current.id, tree=tree)
return [_read(m) for m in items]
@router.get("/{tree_id}/media/{media_id}/content")
async def media_content(
tree_id: uuid.UUID,
media_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
) -> Response:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
media = await media_service.get_media(
session, viewer_id=current.id, tree=tree, media_id=media_id
)
data = await store.get_object(key=media.storage_key)
return Response(
content=data,
media_type=media.content_type,
headers={"Content-Disposition": f'inline; filename="{media.original_filename}"'},
)
@router.patch("/{tree_id}/media/{media_id}", response_model=MediaRead)
async def update_media(
tree_id: uuid.UUID,
media_id: uuid.UUID,
data: MediaUpdate,
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
) -> MediaRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
media = await media_service.update_media(
session,
actor=current,
tree=tree,
media_id=media_id,
changes=data.model_dump(exclude_unset=True),
)
return _read(media)
@router.delete("/{tree_id}/media/{media_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_media(
tree_id: uuid.UUID, media_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await media_service.delete_media(session, actor=current, tree=tree, media_id=media_id)
+61
View File
@@ -0,0 +1,61 @@
"""Tree membership management endpoints (owner-managed; members can list)."""
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.membership import MemberAdd, MemberRoleUpdate, MembershipRead
from app.services import membership_service, tree_service
router = APIRouter(prefix="/trees", tags=["members"])
@router.get("/{tree_id}/members", response_model=list[MembershipRead])
async def list_members(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[MembershipRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await membership_service.list_members(session, viewer_id=current.id, tree=tree)
return [MembershipRead(**r) for r in rows]
@router.post(
"/{tree_id}/members", response_model=MembershipRead, status_code=status.HTTP_201_CREATED
)
async def add_member(
tree_id: uuid.UUID, data: MemberAdd, session: SessionDep, current: CurrentUser
) -> MembershipRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
row = await membership_service.add_member(
session, actor=current, tree=tree, email=data.email, role=data.role
)
return MembershipRead(**row)
@router.patch("/{tree_id}/members/{membership_id}", response_model=MembershipRead)
async def update_member(
tree_id: uuid.UUID,
membership_id: uuid.UUID,
data: MemberRoleUpdate,
session: SessionDep,
current: CurrentUser,
) -> MembershipRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
row = await membership_service.update_member_role(
session, actor=current, tree=tree, membership_id=membership_id, role=data.role
)
return MembershipRead(**row)
@router.delete("/{tree_id}/members/{membership_id}", status_code=status.HTTP_204_NO_CONTENT)
async def remove_member(
tree_id: uuid.UUID,
membership_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await membership_service.remove_member(
session, actor=current, tree=tree, membership_id=membership_id
)
+90
View File
@@ -0,0 +1,90 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.name import NameCreate, NameRead, NameUpdate
from app.services import name_service, tree_service
# Names are nested under their person (which is nested under the tree tenant).
router = APIRouter(prefix="/trees", tags=["names"])
@router.get("/{tree_id}/persons/{person_id}/names", response_model=list[NameRead])
async def list_names(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[NameRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
names = await name_service.list_names(
session, viewer_id=current.id, tree=tree, person_id=person_id
)
return [NameRead.model_validate(n) for n in names]
@router.post(
"/{tree_id}/persons/{person_id}/names",
response_model=NameRead,
status_code=status.HTTP_201_CREATED,
)
async def create_name(
tree_id: uuid.UUID,
person_id: uuid.UUID,
data: NameCreate,
session: SessionDep,
current: CurrentUser,
) -> NameRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
name = await name_service.create_name(
session,
actor=current,
tree=tree,
person_id=person_id,
name_type=data.name_type,
given=data.given,
surname=data.surname,
prefix=data.prefix,
suffix=data.suffix,
nickname=data.nickname,
is_primary=data.is_primary,
)
return NameRead.model_validate(name)
@router.patch(
"/{tree_id}/persons/{person_id}/names/{name_id}", response_model=NameRead
)
async def update_name(
tree_id: uuid.UUID,
person_id: uuid.UUID,
name_id: uuid.UUID,
data: NameUpdate,
session: SessionDep,
current: CurrentUser,
) -> NameRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
name = await name_service.update_name(
session,
actor=current,
tree=tree,
person_id=person_id,
name_id=name_id,
changes=data.model_dump(exclude_unset=True),
)
return NameRead.model_validate(name)
@router.delete(
"/{tree_id}/persons/{person_id}/names/{name_id}",
status_code=status.HTTP_204_NO_CONTENT,
)
async def delete_name(
tree_id: uuid.UUID,
person_id: uuid.UUID,
name_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await name_service.delete_name(
session, actor=current, tree=tree, person_id=person_id, name_id=name_id
)
+84 -4
View File
@@ -1,9 +1,9 @@
import uuid import uuid
from fastapi import APIRouter, status from fastapi import APIRouter, HTTPException, status
from app.api.deps import CurrentUser, SessionDep from app.api.deps import CurrentUser, SessionDep
from app.schemas.person import PersonCreate, PersonRead from app.schemas.person import PersonCreate, PersonRead, PersonUpdate
from app.services import person_service, tree_service from app.services import person_service, tree_service
# Persons are nested under their tree (the tenant boundary). # Persons are nested under their tree (the tenant boundary).
@@ -36,8 +36,88 @@ async def create_person(
@router.get("/{tree_id}/persons", response_model=list[PersonRead]) @router.get("/{tree_id}/persons", response_model=list[PersonRead])
async def list_persons( async def list_persons(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
deleted: bool = False,
q: str | None = None,
ids: str | None = None,
) -> list[PersonRead]: ) -> list[PersonRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id) tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
persons = await person_service.list_persons(session, viewer_id=current.id, tree=tree) if ids is not None:
try:
id_list = [uuid.UUID(x) for x in ids.split(",") if x.strip()]
except ValueError as exc:
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "invalid ids") from exc
persons = await person_service.list_persons_by_ids(
session, viewer_id=current.id, tree=tree, ids=id_list
)
elif q:
persons = await person_service.search_persons(
session, viewer_id=current.id, tree=tree, query=q
)
elif deleted:
persons = await person_service.list_deleted_persons(
session, viewer_id=current.id, tree=tree
)
else:
persons = await person_service.list_persons(session, viewer_id=current.id, tree=tree)
return [PersonRead.model_validate(p) for p in persons] return [PersonRead.model_validate(p) for p in persons]
@router.patch("/{tree_id}/persons/{person_id}", response_model=PersonRead)
async def update_person(
tree_id: uuid.UUID,
person_id: uuid.UUID,
data: PersonUpdate,
session: SessionDep,
current: CurrentUser,
) -> PersonRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
person = await person_service.update_person(
session,
actor=current,
tree=tree,
person_id=person_id,
changes=data.model_dump(exclude_unset=True),
)
return PersonRead.model_validate(person)
@router.delete("/{tree_id}/persons/{person_id}")
async def delete_person(
tree_id: uuid.UUID,
person_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
cascade: bool = False,
) -> dict[str, int]:
"""Delete a person. ``cascade=true`` also deletes all descendants. Returns
the number of persons deleted (1 unless cascading)."""
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
deleted = await person_service.delete_person(
session, actor=current, tree=tree, person_id=person_id, cascade=cascade
)
return {"deleted": deleted}
@router.post("/{tree_id}/persons/{person_id}/restore", response_model=PersonRead)
async def restore_person(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> PersonRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
person = await person_service.restore_person(
session, actor=current, tree=tree, person_id=person_id
)
return PersonRead.model_validate(person)
@router.get("/{tree_id}/persons/{person_id}", response_model=PersonRead)
async def get_person(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> PersonRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
person = await person_service.get_person(
session, viewer_id=current.id, tree=tree, person_id=person_id
)
return PersonRead.model_validate(person)
+116
View File
@@ -0,0 +1,116 @@
"""Change-proposal endpoints: list / create / get / apply / reject / delete.
Applying a proposal is the only way its operations reach the database, and only
an editor can do it (enforced in the service). See docs/design/change-proposal.md.
"""
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.models.enums import ChangeProposalStatus
from app.schemas.change_proposal import (
ChangeProposalCreate,
ChangeProposalRead,
ProposalReview,
)
from app.services import change_proposal_service, tree_service
router = APIRouter(prefix="/trees", tags=["proposals"])
@router.get("/{tree_id}/proposals", response_model=list[ChangeProposalRead])
async def list_proposals(
tree_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
status: ChangeProposalStatus | None = None,
) -> list[ChangeProposalRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rows = await change_proposal_service.list_proposals(
session, viewer_id=current.id, tree=tree, status=status
)
return [ChangeProposalRead.model_validate(r) for r in rows]
@router.post(
"/{tree_id}/proposals", response_model=ChangeProposalRead, status_code=status.HTTP_201_CREATED
)
async def create_proposal(
tree_id: uuid.UUID, data: ChangeProposalCreate, session: SessionDep, current: CurrentUser
) -> ChangeProposalRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
operations = [op.model_dump(mode="json") for op in data.operations]
cp = await change_proposal_service.propose(
session,
tree=tree,
origin=data.origin,
created_by=current.id,
summary=data.summary,
rationale=data.rationale,
operations=operations,
)
return ChangeProposalRead.model_validate(cp)
@router.get("/{tree_id}/proposals/{proposal_id}", response_model=ChangeProposalRead)
async def get_proposal(
tree_id: uuid.UUID, proposal_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> ChangeProposalRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
cp = await change_proposal_service.get_proposal(
session, viewer_id=current.id, tree=tree, proposal_id=proposal_id
)
return ChangeProposalRead.model_validate(cp)
@router.post("/{tree_id}/proposals/{proposal_id}/apply", response_model=ChangeProposalRead)
async def apply_proposal(
tree_id: uuid.UUID,
proposal_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
data: ProposalReview | None = None,
) -> ChangeProposalRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
edited = (
[op.model_dump(mode="json") for op in data.operations]
if data and data.operations is not None
else None
)
cp = await change_proposal_service.apply(
session, actor=current, tree=tree, proposal_id=proposal_id, edited_operations=edited
)
return ChangeProposalRead.model_validate(cp)
@router.post("/{tree_id}/proposals/{proposal_id}/reject", response_model=ChangeProposalRead)
async def reject_proposal(
tree_id: uuid.UUID,
proposal_id: uuid.UUID,
session: SessionDep,
current: CurrentUser,
data: ProposalReview | None = None,
) -> ChangeProposalRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
cp = await change_proposal_service.reject(
session,
actor=current,
tree=tree,
proposal_id=proposal_id,
note=data.note if data else None,
)
return ChangeProposalRead.model_validate(cp)
@router.delete(
"/{tree_id}/proposals/{proposal_id}", status_code=status.HTTP_204_NO_CONTENT
)
async def delete_proposal(
tree_id: uuid.UUID, proposal_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await change_proposal_service.delete_proposal(
session, actor=current, tree=tree, proposal_id=proposal_id
)
+135
View File
@@ -0,0 +1,135 @@
"""Public, read-only viewing surface.
Optional auth (anonymous allowed). Every response is built by
``public_view_service``, which routes through the privacy engine and redacts
possibly-living people. No create/update/delete here.
"""
import uuid
from fastapi import APIRouter
from app.api.deps import CurrentUserOrNone, SessionDep
from app.schemas.event import EventRead
from app.schemas.name import NameRead
from app.schemas.person import PersonRead
from app.schemas.relationship import RelationshipRead
from app.schemas.tree import PublicTreeRead
from app.services import public_view_service
router = APIRouter(prefix="/public", tags=["public"])
def _vid(viewer: CurrentUserOrNone) -> uuid.UUID | None:
return viewer.id if viewer else None
@router.get("/trees", response_model=list[PublicTreeRead])
async def public_directory(
session: SessionDep,
viewer: CurrentUserOrNone,
q: str | None = None,
limit: int = 50,
offset: int = 0,
) -> list[PublicTreeRead]:
trees = await public_view_service.list_public_trees(
session, viewer_id=_vid(viewer), q=q, limit=limit, offset=offset
)
return [PublicTreeRead.model_validate(t) for t in trees]
@router.get("/trees/{tree_id}", response_model=PublicTreeRead)
async def public_tree(
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
) -> PublicTreeRead:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
return PublicTreeRead.model_validate(tree)
@router.get("/trees/{tree_id}/persons", response_model=list[PersonRead])
async def public_persons(
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
) -> list[PersonRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
persons = await public_view_service.list_public_persons(
session, viewer_id=_vid(viewer), tree=tree
)
return [PersonRead.model_validate(p) for p in persons]
@router.get("/trees/{tree_id}/relationships", response_model=list[RelationshipRead])
async def public_relationships(
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
) -> list[RelationshipRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
rels = await public_view_service.list_public_relationships(
session, viewer_id=_vid(viewer), tree=tree
)
return [RelationshipRead.model_validate(r) for r in rels]
@router.get("/trees/{tree_id}/events", response_model=list[EventRead])
async def public_events(
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
) -> list[EventRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
events = await public_view_service.list_public_events(
session, viewer_id=_vid(viewer), tree=tree
)
return [EventRead.model_validate(e) for e in events]
@router.get("/trees/{tree_id}/persons/{person_id}", response_model=PersonRead)
async def public_person(
tree_id: uuid.UUID,
person_id: uuid.UUID,
session: SessionDep,
viewer: CurrentUserOrNone,
) -> PersonRead:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
person = await public_view_service.get_public_person(
session, viewer_id=_vid(viewer), tree=tree, person_id=person_id
)
return PersonRead.model_validate(person)
@router.get("/trees/{tree_id}/persons/{person_id}/names", response_model=list[NameRead])
async def public_person_names(
tree_id: uuid.UUID,
person_id: uuid.UUID,
session: SessionDep,
viewer: CurrentUserOrNone,
) -> list[NameRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
names = await public_view_service.list_public_person_names(
session, viewer_id=_vid(viewer), tree=tree, person_id=person_id
)
return [NameRead.model_validate(n) for n in names]
@router.get("/trees/{tree_id}/persons/{person_id}/events", response_model=list[EventRead])
async def public_person_events(
tree_id: uuid.UUID,
person_id: uuid.UUID,
session: SessionDep,
viewer: CurrentUserOrNone,
) -> list[EventRead]:
tree = await public_view_service.get_public_tree(
session, viewer_id=_vid(viewer), tree_id=tree_id
)
events = await public_view_service.list_public_person_events(
session, viewer_id=_vid(viewer), tree=tree, person_id=person_id
)
return [EventRead.model_validate(e) for e in events]
+78
View File
@@ -0,0 +1,78 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.relationship import RelationshipCreate, RelationshipRead, RelationshipUpdate
from app.services import relationship_service, tree_service
router = APIRouter(prefix="/trees", tags=["relationships"])
@router.post(
"/{tree_id}/relationships",
response_model=RelationshipRead,
status_code=status.HTTP_201_CREATED,
)
async def create_relationship(
tree_id: uuid.UUID, data: RelationshipCreate, session: SessionDep, current: CurrentUser
) -> RelationshipRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
relationship = await relationship_service.create_relationship(
session, actor=current, tree=tree, **data.model_dump()
)
return RelationshipRead.model_validate(relationship)
@router.get("/{tree_id}/relationships", response_model=list[RelationshipRead])
async def list_relationships(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[RelationshipRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rels = await relationship_service.list_relationships(session, viewer_id=current.id, tree=tree)
return [RelationshipRead.model_validate(r) for r in rels]
@router.get(
"/{tree_id}/persons/{person_id}/relationships",
response_model=list[RelationshipRead],
)
async def list_person_relationships(
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[RelationshipRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rels = await relationship_service.list_relationships_for_person(
session, viewer_id=current.id, tree=tree, person_id=person_id
)
return [RelationshipRead.model_validate(r) for r in rels]
@router.patch("/{tree_id}/relationships/{relationship_id}", response_model=RelationshipRead)
async def update_relationship(
tree_id: uuid.UUID,
relationship_id: uuid.UUID,
data: RelationshipUpdate,
session: SessionDep,
current: CurrentUser,
) -> RelationshipRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
rel = await relationship_service.update_relationship(
session,
actor=current,
tree=tree,
relationship_id=relationship_id,
changes=data.model_dump(exclude_unset=True),
)
return RelationshipRead.model_validate(rel)
@router.delete(
"/{tree_id}/relationships/{relationship_id}", status_code=status.HTTP_204_NO_CONTENT
)
async def delete_relationship(
tree_id: uuid.UUID, relationship_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await relationship_service.delete_relationship(
session, actor=current, tree=tree, relationship_id=relationship_id
)
+67
View File
@@ -0,0 +1,67 @@
import uuid
from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep
from app.schemas.source import SourceCreate, SourceRead, SourceUpdate
from app.services import source_service, tree_service
router = APIRouter(prefix="/trees", tags=["sources"])
@router.post("/{tree_id}/sources", response_model=SourceRead, status_code=status.HTTP_201_CREATED)
async def create_source(
tree_id: uuid.UUID, data: SourceCreate, session: SessionDep, current: CurrentUser
) -> SourceRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
source = await source_service.create_source(
session, actor=current, tree=tree, **data.model_dump()
)
return SourceRead.model_validate(source)
@router.get("/{tree_id}/sources", response_model=list[SourceRead])
async def list_sources(
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> list[SourceRead]:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
sources = await source_service.list_sources(session, viewer_id=current.id, tree=tree)
return [SourceRead.model_validate(s) for s in sources]
@router.get("/{tree_id}/sources/{source_id}", response_model=SourceRead)
async def get_source(
tree_id: uuid.UUID, source_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> SourceRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
source = await source_service.get_source(
session, viewer_id=current.id, tree=tree, source_id=source_id
)
return SourceRead.model_validate(source)
@router.patch("/{tree_id}/sources/{source_id}", response_model=SourceRead)
async def update_source(
tree_id: uuid.UUID,
source_id: uuid.UUID,
data: SourceUpdate,
session: SessionDep,
current: CurrentUser,
) -> SourceRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
source = await source_service.update_source(
session,
actor=current,
tree=tree,
source_id=source_id,
changes=data.model_dump(exclude_unset=True),
)
return SourceRead.model_validate(source)
@router.delete("/{tree_id}/sources/{source_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_source(
tree_id: uuid.UUID, source_id: uuid.UUID, session: SessionDep, current: CurrentUser
) -> None:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
await source_service.delete_source(session, actor=current, tree=tree, source_id=source_id)
+45 -4
View File
@@ -2,8 +2,8 @@ import uuid
from fastapi import APIRouter, status from fastapi import APIRouter, status
from app.api.deps import CurrentUser, SessionDep from app.api.deps import CurrentUser, ObjectStoreDep, SessionDep
from app.schemas.tree import TreeCreate, TreeRead from app.schemas.tree import TreeCreate, TreePurge, TreeRead, TreeUpdate
from app.services import tree_service from app.services import tree_service
router = APIRouter(prefix="/trees", tags=["trees"]) router = APIRouter(prefix="/trees", tags=["trees"])
@@ -22,8 +22,13 @@ async def create_tree(data: TreeCreate, session: SessionDep, current: CurrentUse
@router.get("", response_model=list[TreeRead]) @router.get("", response_model=list[TreeRead])
async def list_my_trees(session: SessionDep, current: CurrentUser) -> list[TreeRead]: async def list_my_trees(
trees = await tree_service.list_trees_for_user(session, user=current) session: SessionDep, current: CurrentUser, deleted: bool = False
) -> list[TreeRead]:
if deleted:
trees = await tree_service.list_deleted_trees_for_user(session, user=current)
else:
trees = await tree_service.list_trees_for_user(session, user=current)
return [TreeRead.model_validate(t) for t in trees] return [TreeRead.model_validate(t) for t in trees]
@@ -31,3 +36,39 @@ async def list_my_trees(session: SessionDep, current: CurrentUser) -> list[TreeR
async def get_tree(tree_id: uuid.UUID, session: SessionDep, current: CurrentUser) -> TreeRead: async def get_tree(tree_id: uuid.UUID, session: SessionDep, current: CurrentUser) -> TreeRead:
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id) tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
return TreeRead.model_validate(tree) return TreeRead.model_validate(tree)
@router.patch("/{tree_id}", response_model=TreeRead)
async def update_tree(
tree_id: uuid.UUID, data: TreeUpdate, session: SessionDep, current: CurrentUser
) -> TreeRead:
tree = await tree_service.update_tree(
session, actor=current, tree_id=tree_id, changes=data.model_dump(exclude_unset=True)
)
return TreeRead.model_validate(tree)
@router.delete("/{tree_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_tree(tree_id: uuid.UUID, session: SessionDep, current: CurrentUser) -> None:
await tree_service.delete_tree(session, actor=current, tree_id=tree_id)
@router.post("/{tree_id}/restore", response_model=TreeRead)
async def restore_tree(tree_id: uuid.UUID, session: SessionDep, current: CurrentUser) -> TreeRead:
tree = await tree_service.restore_tree(session, actor=current, tree_id=tree_id)
return TreeRead.model_validate(tree)
@router.post("/{tree_id}/purge", status_code=status.HTTP_204_NO_CONTENT)
async def purge_tree(
tree_id: uuid.UUID,
data: TreePurge,
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
) -> None:
"""Permanently delete a soft-deleted tree and all its data — irreversible.
Owner-only; the tree must be in the trash and `confirm_name` must match."""
await tree_service.purge_tree(
session, store, actor=current, tree_id=tree_id, confirm_name=data.confirm_name
)
+56 -4
View File
@@ -1,11 +1,63 @@
from fastapi import APIRouter from fastapi import APIRouter, File, Form, Response, UploadFile
from app.api.deps import CurrentUser from app.api.deps import CurrentUser, ObjectStoreDep, SessionDep, is_instance_owner
from app.schemas.user import UserRead from app.schemas.user import UserRead, UserSelfPersonUpdate
from app.services import account_service, user_service
router = APIRouter(prefix="/users", tags=["users"]) router = APIRouter(prefix="/users", tags=["users"])
def _me(user) -> UserRead:
out = UserRead.model_validate(user)
out.is_instance_owner = is_instance_owner(user)
return out
@router.get("/me", response_model=UserRead) @router.get("/me", response_model=UserRead)
async def read_me(current: CurrentUser) -> UserRead: async def read_me(current: CurrentUser) -> UserRead:
return UserRead.model_validate(current) return _me(current)
@router.patch("/me/self-person", response_model=UserRead)
async def set_self_person(
data: UserSelfPersonUpdate, session: SessionDep, current: CurrentUser
) -> UserRead:
"""Link (or unlink) the Person record that represents this account."""
user = await user_service.set_self_person(
session, user=current, person_id=data.self_person_id
)
return _me(user)
@router.get("/me/export")
async def export_account(
session: SessionDep, current: CurrentUser, store: ObjectStoreDep
) -> Response:
"""Download a full backup (JSON + media) of every tree the user owns."""
data = await account_service.export_account(session, store, user=current)
return Response(
content=data,
media_type="application/zip",
headers={"Content-Disposition": 'attachment; filename="provenance-export.zip"'},
)
@router.post("/me/import")
async def import_account(
session: SessionDep,
current: CurrentUser,
store: ObjectStoreDep,
file: UploadFile = File(...),
) -> dict:
"""Restore a previously-exported backup into new trees (non-destructive)."""
raw = await file.read()
return await account_service.import_account(session, store, user=current, raw_zip=raw)
@router.delete("/me", status_code=204)
async def delete_account(
session: SessionDep, current: CurrentUser, confirm_email: str = Form(...)
) -> None:
"""Delete the account: the user, their owned trees, and their sessions.
Requires retyping the account email as a guard."""
await account_service.delete_account(session, user=current, confirm_email=confirm_email)
+59
View File
@@ -22,6 +22,18 @@ class Settings(BaseSettings):
version: str = "0.0.0" version: str = "0.0.0"
app_env: str = Field(default="development", description="development | production") app_env: str = Field(default="development", description="development | production")
# --- Instance owner / operator ---
# Email(s) of the instance owner(s) — the operator(s) who run this server.
# The matching account(s) get instance-admin rights (instance-wide settings;
# see /api/v1/admin). Comma-separated for several. Empty = no designated
# owner (the instance has no operator account). Derived at request time, so
# changing it takes effect immediately with no migration or DB state.
owner_email: str = ""
def owner_emails(self) -> frozenset[str]:
"""Normalized (lowercased, trimmed) owner emails; empty if none set."""
return frozenset(e.strip().lower() for e in self.owner_email.split(",") if e.strip())
# SQLAlchemy async URL, e.g. postgresql+asyncpg://user:pass@host:5432/db # SQLAlchemy async URL, e.g. postgresql+asyncpg://user:pass@host:5432/db
database_url: str = Field( database_url: str = Field(
default="postgresql+asyncpg://provenance:provenance@localhost:5432/provenance", default="postgresql+asyncpg://provenance:provenance@localhost:5432/provenance",
@@ -35,7 +47,24 @@ class Settings(BaseSettings):
# Base URL used to build links in outbound email. # Base URL used to build links in outbound email.
app_base_url: str = "http://localhost" app_base_url: str = "http://localhost"
# --- Object storage (S3-compatible / MinIO) ---
s3_endpoint_url: str = "http://minio:9000"
s3_bucket: str = "provenance"
s3_access_key: str = "provenance"
s3_secret_key: str = "change-me-too"
s3_region: str = "us-east-1"
s3_presign_ttl: int = 3600 # seconds
# --- Worker ---
purge_interval_seconds: int = 3600 # how often to run the soft-delete purge
purge_after_days: int = 30 # soft-deleted rows older than this are purged
# --- Email (SMTP) --- # --- Email (SMTP) ---
# When true, a user with no verified email gets no active session (login is
# refused and existing sessions stop resolving). Default false so self-hosts
# without SMTP — and accounts created before this gate existed — aren't
# locked out; operators turn it on once mail works and accounts are verified.
require_email_verification: bool = False
mailer: str = Field(default="console", description="console | smtp") mailer: str = Field(default="console", description="console | smtp")
smtp_host: str | None = None smtp_host: str | None = None
smtp_port: int = 587 smtp_port: int = 587
@@ -43,6 +72,36 @@ class Settings(BaseSettings):
smtp_password: str | None = None smtp_password: str | None = None
smtp_from: str = "Provenance <no-reply@provenance.local>" smtp_from: str = "Provenance <no-reply@provenance.local>"
# --- Model providers (AI assistant + match-ranking embeddings) ---
# Configure as many as you like; each is enabled when its credentials are
# present. `default_*_provider` picks which one is used by default. LLM and
# embeddings are independent (Anthropic has no embeddings endpoint).
default_llm_provider: str = "null" # null | anthropic | openai | xai | ollama
default_embedding_provider: str = "null" # null | openai | ollama
llm_max_tokens: int = 4096
embedding_dimensions: int = 1536 # must match the embedding model + pgvector column
# Anthropic (LLM only)
anthropic_api_key: str | None = None
anthropic_model: str = "claude-opus-4-8"
# OpenAI (LLM + embeddings)
openai_api_key: str | None = None
openai_base_url: str = "https://api.openai.com/v1"
openai_model: str = "gpt-4o"
openai_embedding_model: str = "text-embedding-3-small"
# xAI / Grok — OpenAI-compatible (LLM)
xai_api_key: str | None = None
xai_base_url: str = "https://api.x.ai/v1"
xai_model: str = "grok-2-latest" # set to your account's current Grok model
# Ollama — local, OpenAI-compatible, no key (LLM + embeddings)
ollama_enabled: bool = False
ollama_base_url: str = "http://localhost:11434/v1"
ollama_model: str = "llama3.1"
ollama_embedding_model: str = "nomic-embed-text"
@lru_cache @lru_cache
def get_settings() -> Settings: def get_settings() -> Settings:
+59
View File
@@ -0,0 +1,59 @@
"""Schema-drift detection — a safety net for the deploy pipeline.
If a deploy ships code whose models reference a column a migration hasn't added
yet (the code is ahead of the DB), every query against that table 500s with an
opaque ``UndefinedColumnError``. That is exactly the failure that took the tree
list down once: the backend image advanced but ``alembic upgrade head`` hadn't
run on the server.
The real prevention is auto-migrate on deploy (the entrypoint runs
``alembic upgrade head`` when ``RUN_MIGRATIONS=1``). This module is defense in
depth: it makes the drift *loud and explicit* — a readiness failure and a
CRITICAL startup log — instead of a silent storm of 500s, so a half-applied
deploy is obvious within seconds.
"""
from functools import lru_cache
from pathlib import Path
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncConnection
# app/core/schema_version.py -> backend/ (parents: core, app, backend)
_MIGRATIONS_DIR = Path(__file__).resolve().parents[2] / "migrations"
@lru_cache
def expected_heads() -> frozenset[str]:
"""Revision head(s) baked into this image's migration scripts. Static for a
given build, so cache it."""
from alembic.config import Config
from alembic.script import ScriptDirectory
cfg = Config()
cfg.set_main_option("script_location", str(_MIGRATIONS_DIR))
return frozenset(ScriptDirectory.from_config(cfg).get_heads())
async def db_heads(conn: AsyncConnection) -> frozenset[str] | None:
"""Revision(s) the database is stamped at, or ``None`` when the DB is not
Alembic-managed (no ``alembic_version`` table — e.g. a test DB built straight
from ``create_all``). ``to_regclass`` returns NULL rather than erroring when
the table is absent, so this never poisons the caller's transaction."""
if await conn.scalar(text("SELECT to_regclass('public.alembic_version')")) is None:
return None
result = await conn.execute(text("SELECT version_num FROM alembic_version"))
return frozenset(row[0] for row in result)
async def schema_is_current(
conn: AsyncConnection,
) -> tuple[bool, frozenset[str], frozenset[str]]:
"""``(ok, db, expected)``. ``ok`` is True when the DB is stamped at the
code's head(s). A DB with no ``alembic_version`` table is treated as current
(not Alembic-managed → nothing to compare), so this stays quiet in tests."""
expected = expected_heads()
current = await db_heads(conn)
if current is None:
return True, frozenset(), expected
return current == expected, current, expected
@@ -0,0 +1,24 @@
"""Anthropic LLM provider (official SDK). Self-hosters who want everything to
stay on their own metal would configure a local provider instead (e.g. Ollama) —
that's a future implementation of the same LLMProvider interface."""
from anthropic import AsyncAnthropic
from app.integrations.models.base import LLMProvider
class AnthropicLLMProvider(LLMProvider):
def __init__(self, *, api_key: str, model: str, max_tokens: int = 4096) -> None:
self._client = AsyncAnthropic(api_key=api_key)
self._model = model
self._max_tokens = max_tokens
async def complete(self, *, prompt: str, system: str | None = None) -> str:
resp = await self._client.messages.create(
model=self._model,
max_tokens=self._max_tokens,
system=system or "",
messages=[{"role": "user", "content": prompt}],
)
# content is a list of blocks; concatenate the text ones.
return "".join(b.text for b in resp.content if b.type == "text")
+36
View File
@@ -0,0 +1,36 @@
"""Model-provider interfaces — the seam the AI assistant and match ranking plug
into. LLM (text) and embeddings are *separate* abstractions: Anthropic offers no
embeddings endpoint, so the two are configured independently (twelve-factor,
CLAUDE.md #7) and a deployment may run one without the other.
These providers are read-only text/vector producers. They MUST NOT mutate tree
data — the assistant's writes go through a ChangeProposal a human approves
(CLAUDE.md #1). Nothing here touches the database.
"""
from abc import ABC, abstractmethod
class LLMProvider(ABC):
"""Text in, text out. Implementations wrap a chat/completion model."""
@abstractmethod
async def complete(self, *, prompt: str, system: str | None = None) -> str:
"""Return the model's text response to a single user prompt."""
...
class EmbeddingProvider(ABC):
"""Text in, vectors out — for pgvector-backed match ranking."""
#: Dimensionality of the returned vectors (for the pgvector column).
dimensions: int
@abstractmethod
async def embed(self, texts: list[str]) -> list[list[float]]:
"""Return one embedding vector per input text, in order."""
...
class ModelProviderNotConfigured(RuntimeError):
"""Raised when an AI capability is used but no provider is configured."""
+31
View File
@@ -0,0 +1,31 @@
"""Default providers when no model backend is configured — AI features are off.
They fail loudly (rather than silently doing nothing) so a caller that reaches
for an unconfigured capability gets a clear, actionable error.
"""
from app.integrations.models.base import (
EmbeddingProvider,
LLMProvider,
ModelProviderNotConfigured,
)
_MSG = (
"No model provider configured. Set MODEL_PROVIDER (e.g. 'anthropic') and the "
"provider's credentials to enable AI features."
)
class NullLLMProvider(LLMProvider):
async def complete(self, *, prompt: str, system: str | None = None) -> str:
raise ModelProviderNotConfigured(_MSG)
class NullEmbeddingProvider(EmbeddingProvider):
dimensions = 0
async def embed(self, texts: list[str]) -> list[list[float]]:
raise ModelProviderNotConfigured(
"No embedding provider configured. Set EMBEDDING_PROVIDER and its "
"credentials to enable match ranking."
)
@@ -0,0 +1,40 @@
"""OpenAI-compatible providers (one implementation, many vendors).
OpenAI, xAI (api.x.ai/v1), Ollama (…:11434/v1), OpenRouter, Together, vLLM, etc.
all speak the OpenAI Chat Completions / Embeddings API — they differ only by
base URL, key, and model name. So a single class, parameterized by those, plugs
in every one of them via the official `openai` SDK.
"""
from openai import AsyncOpenAI
from app.integrations.models.base import EmbeddingProvider, LLMProvider
class OpenAICompatibleLLMProvider(LLMProvider):
def __init__(self, *, api_key: str | None, base_url: str, model: str, max_tokens: int = 4096) -> None:
# Local backends (Ollama) ignore the key but the SDK requires a non-empty one.
self._client = AsyncOpenAI(api_key=api_key or "not-needed", base_url=base_url)
self._model = model
self._max_tokens = max_tokens
async def complete(self, *, prompt: str, system: str | None = None) -> str:
messages: list[dict] = []
if system:
messages.append({"role": "system", "content": system})
messages.append({"role": "user", "content": prompt})
resp = await self._client.chat.completions.create(
model=self._model, max_tokens=self._max_tokens, messages=messages
)
return resp.choices[0].message.content or ""
class OpenAICompatibleEmbeddingProvider(EmbeddingProvider):
def __init__(self, *, api_key: str | None, base_url: str, model: str, dimensions: int) -> None:
self._client = AsyncOpenAI(api_key=api_key or "not-needed", base_url=base_url)
self._model = model
self.dimensions = dimensions
async def embed(self, texts: list[str]) -> list[list[float]]:
resp = await self._client.embeddings.create(model=self._model, input=texts)
return [d.embedding for d in resp.data]
@@ -0,0 +1,25 @@
"""ObjectStore interface — pluggable binary storage behind the service layer.
Implementations are S3-compatible (MinIO for self-host, any S3 otherwise).
Methods are async wrappers so the service layer stays non-blocking even though
the underlying SDK (boto3) is synchronous.
"""
from abc import ABC, abstractmethod
class ObjectStore(ABC):
@abstractmethod
async def ensure_bucket(self) -> None: ...
@abstractmethod
async def put_object(self, *, key: str, data: bytes, content_type: str) -> None: ...
@abstractmethod
async def get_object(self, *, key: str) -> bytes: ...
@abstractmethod
async def presigned_get_url(self, *, key: str) -> str: ...
@abstractmethod
async def delete_object(self, *, key: str) -> None: ...
@@ -0,0 +1,63 @@
"""S3-compatible ObjectStore (boto3), suitable for MinIO or any S3 provider.
boto3 is synchronous; each call is dispatched to a thread so request handlers
and the worker stay async."""
import asyncio
import boto3
from botocore.client import Config
from botocore.exceptions import ClientError
from app.core.config import Settings
from app.integrations.objectstore.base import ObjectStore
class S3ObjectStore(ObjectStore):
def __init__(self, settings: Settings) -> None:
self.bucket = settings.s3_bucket
self.presign_ttl = settings.s3_presign_ttl
self._client = boto3.client(
"s3",
endpoint_url=settings.s3_endpoint_url,
aws_access_key_id=settings.s3_access_key,
aws_secret_access_key=settings.s3_secret_key,
region_name=settings.s3_region,
config=Config(signature_version="s3v4"),
)
def _ensure_bucket_sync(self) -> None:
try:
self._client.head_bucket(Bucket=self.bucket)
except ClientError:
self._client.create_bucket(Bucket=self.bucket)
async def ensure_bucket(self) -> None:
await asyncio.to_thread(self._ensure_bucket_sync)
async def put_object(self, *, key: str, data: bytes, content_type: str) -> None:
await asyncio.to_thread(
self._client.put_object,
Bucket=self.bucket,
Key=key,
Body=data,
ContentType=content_type,
)
async def get_object(self, *, key: str) -> bytes:
def _get() -> bytes:
obj = self._client.get_object(Bucket=self.bucket, Key=key)
return obj["Body"].read()
return await asyncio.to_thread(_get)
async def presigned_get_url(self, *, key: str) -> str:
return await asyncio.to_thread(
self._client.generate_presigned_url,
"get_object",
Params={"Bucket": self.bucket, "Key": key},
ExpiresIn=self.presign_ttl,
)
async def delete_object(self, *, key: str) -> None:
await asyncio.to_thread(self._client.delete_object, Bucket=self.bucket, Key=key)
+30
View File
@@ -7,6 +7,7 @@ engine is the single enforcement point for reads.
import logging import logging
import sys import sys
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse from fastapi.responses import JSONResponse
@@ -14,6 +15,8 @@ from fastapi.responses import JSONResponse
from app.api.health import router as health_router from app.api.health import router as health_router
from app.api.v1 import api_router from app.api.v1 import api_router
from app.core.config import get_settings from app.core.config import get_settings
from app.core.db import get_engine
from app.core.schema_version import schema_is_current
from app.services.exceptions import Conflict, Forbidden, NotFound from app.services.exceptions import Conflict, Forbidden, NotFound
@@ -30,6 +33,32 @@ def _configure_logging() -> None:
app_logger.propagate = False app_logger.propagate = False
async def _check_schema_drift() -> None:
"""On startup, shout if the DB schema is behind the code. The entrypoint
runs migrations when RUN_MIGRATIONS=1; this catches the case where that
didn't happen, so a half-applied deploy is obvious in the logs instead of a
silent storm of 500s. Never blocks startup — purely advisory."""
logger = logging.getLogger("provenance")
try:
async with get_engine().connect() as conn:
ok, db, expected = await schema_is_current(conn)
if not ok:
logger.critical(
"SCHEMA DRIFT: database is at %s but this build expects %s. "
"Run 'alembic upgrade head' — queries will fail until migrated.",
sorted(db) or ["none"],
sorted(expected),
)
except Exception as exc: # noqa: BLE001 — advisory only; never block startup
logger.warning("schema drift check skipped: %s", exc)
@asynccontextmanager
async def _lifespan(app: FastAPI):
await _check_schema_drift()
yield
def _register_error_handlers(app: FastAPI) -> None: def _register_error_handlers(app: FastAPI) -> None:
@app.exception_handler(NotFound) @app.exception_handler(NotFound)
async def _not_found(request: Request, exc: NotFound) -> JSONResponse: async def _not_found(request: Request, exc: NotFound) -> JSONResponse:
@@ -51,6 +80,7 @@ def create_app() -> FastAPI:
title=settings.app_name, title=settings.app_name,
version=settings.version, version=settings.version,
description="Provenance API — family and land provenance.", description="Provenance API — family and land provenance.",
lifespan=_lifespan,
) )
app.include_router(health_router) app.include_router(health_router)
app.include_router(api_router) app.include_router(api_router)
+4
View File
@@ -4,7 +4,9 @@ and for ``create_all`` in tests."""
from app.models.audit import AuditEntry from app.models.audit import AuditEntry
from app.models.auth import Session, UserToken from app.models.auth import Session, UserToken
from app.models.base import Base from app.models.base import Base
from app.models.change_proposal import ChangeProposal
from app.models.event import Event from app.models.event import Event
from app.models.media import Media
from app.models.person import Name, Person from app.models.person import Name, Person
from app.models.place import Place, PlaceName from app.models.place import Place, PlaceName
from app.models.relationship import Relationship from app.models.relationship import Relationship
@@ -28,4 +30,6 @@ __all__ = [
"AuditEntry", "AuditEntry",
"Session", "Session",
"UserToken", "UserToken",
"Media",
"ChangeProposal",
] ]
+48
View File
@@ -0,0 +1,48 @@
"""ChangeProposal — a structured diff the AI assistant (or an untrusted
contributor) proposes, which a human approves/edits/rejects. Applying it routes
each operation through the normal editing services, so the change passes the
privacy engine and is audited as the approving human's action. See
docs/design/change-proposal.md and CLAUDE.md non-negotiable #1.
"""
import uuid
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, String, Text
from sqlalchemy import Enum as SAEnum
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.enums import ChangeProposalOrigin, ChangeProposalStatus
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
class ChangeProposal(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "change_proposals"
status: Mapped[ChangeProposalStatus] = mapped_column(
SAEnum(ChangeProposalStatus, name="change_proposal_status"),
default=ChangeProposalStatus.pending,
server_default=ChangeProposalStatus.pending.value,
index=True,
)
origin: Mapped[ChangeProposalOrigin] = mapped_column(
SAEnum(ChangeProposalOrigin, name="change_proposal_origin"),
default=ChangeProposalOrigin.assistant,
server_default=ChangeProposalOrigin.assistant.value,
)
created_by_user_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("users.id", ondelete="SET NULL")
)
summary: Mapped[str] = mapped_column(String(512))
rationale: Mapped[str | None] = mapped_column(Text)
# The structured diff: a list of {op, entity_type, entity_id?, payload} dicts.
operations: Mapped[list] = mapped_column(JSONB, nullable=False)
reviewed_by_user_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("users.id", ondelete="SET NULL")
)
reviewed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
review_note: Mapped[str | None] = mapped_column(String(512))
apply_error: Mapped[str | None] = mapped_column(Text)
+15 -3
View File
@@ -9,9 +9,10 @@ import enum
class TreeVisibility(enum.StrEnum): class TreeVisibility(enum.StrEnum):
public = "public" public = "public" # anyone on the web (anonymous), listed + search-indexable
unlisted = "unlisted" site_members = "site_members" # any authenticated user of this instance
private = "private" unlisted = "unlisted" # anyone with the link (anonymous), not listed/indexed
private = "private" # members only (default)
class MembershipRole(enum.StrEnum): class MembershipRole(enum.StrEnum):
@@ -60,3 +61,14 @@ class AuditActorType(enum.StrEnum):
class TokenPurpose(enum.StrEnum): class TokenPurpose(enum.StrEnum):
email_verify = "email_verify" email_verify = "email_verify"
password_reset = "password_reset" password_reset = "password_reset"
class ChangeProposalStatus(enum.StrEnum):
pending = "pending"
applied = "applied"
rejected = "rejected"
class ChangeProposalOrigin(enum.StrEnum):
assistant = "assistant" # the AI assistant, acting on behalf of a user
contributor = "contributor" # an untrusted human edit awaiting moderation
+36
View File
@@ -0,0 +1,36 @@
"""Media — a binary asset (image, scan, PDF, audio) in object storage. The row
holds metadata + checksum + the storage key; the bytes live in the ObjectStore.
Optionally attached to a single fact (person, event, or source) for now."""
import uuid
from sqlalchemy import BigInteger, ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
class Media(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "media"
uploader_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("users.id", ondelete="SET NULL"), index=True
)
storage_key: Mapped[str] = mapped_column(String(512), unique=True)
original_filename: Mapped[str] = mapped_column(String(512))
content_type: Mapped[str] = mapped_column(String(128))
byte_size: Mapped[int] = mapped_column(BigInteger)
checksum_sha256: Mapped[str] = mapped_column(String(64), index=True)
title: Mapped[str | None] = mapped_column(String(512))
# Optional single attachment target.
person_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("persons.id", ondelete="SET NULL"), index=True
)
event_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("events.id", ondelete="SET NULL"), index=True
)
source_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey("sources.id", ondelete="SET NULL"), index=True
)
+17 -1
View File
@@ -7,7 +7,7 @@ aliases) so name changes over time are first-class.
import uuid import uuid
from sqlalchemy import Boolean, ForeignKey, Integer, String, Text, text from sqlalchemy import Boolean, ForeignKey, Index, Integer, String, Text, text
from sqlalchemy import Enum as SAEnum from sqlalchemy import Enum as SAEnum
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
@@ -33,6 +33,22 @@ class Person(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
class Name(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete): class Name(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
__tablename__ = "names" __tablename__ = "names"
# Trigram indexes for fuzzy name search (Mueller/Müller/Muller). Requires the
# pg_trgm extension (enabled in the accompanying migration).
__table_args__ = (
Index(
"ix_names_given_trgm",
"given",
postgresql_using="gin",
postgresql_ops={"given": "gin_trgm_ops"},
),
Index(
"ix_names_surname_trgm",
"surname",
postgresql_using="gin",
postgresql_ops={"surname": "gin_trgm_ops"},
),
)
person_id: Mapped[uuid.UUID] = mapped_column( person_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("persons.id", ondelete="CASCADE"), index=True ForeignKey("persons.id", ondelete="CASCADE"), index=True
+15
View File
@@ -26,6 +26,21 @@ class Tree(Base, UUIDPrimaryKey, Timestamps, SoftDelete):
default=TreeVisibility.private, default=TreeVisibility.private,
server_default=TreeVisibility.private.value, server_default=TreeVisibility.private.value,
) )
# The person a tree opens focused on (its "home"/root person). Cleared if
# that person is deleted. use_alter + name: trees<->persons form an FK cycle.
home_person_id: Mapped[uuid.UUID | None] = mapped_column(
ForeignKey(
"persons.id",
ondelete="SET NULL",
name="fk_trees_home_person_id",
use_alter=True,
)
)
# Per-tree AI model policy (owner-configured). The names reference configured
# providers from the registry; null = that role has no model. The owner may
# use any configured provider; these limit members + the recommender.
ai_member_provider: Mapped[str | None] = mapped_column(String(32))
ai_recommender_provider: Mapped[str | None] = mapped_column(String(32))
class TreeMembership(Base, UUIDPrimaryKey, Timestamps): class TreeMembership(Base, UUIDPrimaryKey, Timestamps):
+14 -1
View File
@@ -3,9 +3,10 @@ multiple auth providers later (the provider-link table arrives with the auth
slice). ``hashed_password`` is nullable: external/OIDC users have none. slice). ``hashed_password`` is nullable: external/OIDC users have none.
""" """
import uuid
from datetime import datetime from datetime import datetime
from sqlalchemy import DateTime, String from sqlalchemy import DateTime, ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from app.models.base import Base from app.models.base import Base
@@ -19,3 +20,15 @@ class User(Base, UUIDPrimaryKey, Timestamps, SoftDelete):
email_verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) email_verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
display_name: Mapped[str | None] = mapped_column(String(255)) display_name: Mapped[str | None] = mapped_column(String(255))
hashed_password: Mapped[str | None] = mapped_column(String(255)) hashed_password: Mapped[str | None] = mapped_column(String(255))
# The Person record that *is* this user ("home person"). Cleared if that
# person is deleted, so the link can never dangle.
self_person_id: Mapped[uuid.UUID | None] = mapped_column(
# use_alter + explicit name: users<->persons<->trees form an FK cycle,
# so this constraint must be created/dropped via ALTER, not inline.
ForeignKey(
"persons.id",
ondelete="SET NULL",
name="fk_users_self_person_id",
use_alter=True,
)
)
+20
View File
@@ -0,0 +1,20 @@
"""Instance-admin schemas. Operator-facing, owner-only — operational status and
config, never tree data or PII (instance ownership doesn't bypass privacy)."""
from pydantic import BaseModel
from app.schemas.ai_policy import ConfiguredProvider
class InstanceStatus(BaseModel):
version: str
env: str
# Operator account(s) — the email(s) named in OWNER_EMAIL.
owner_emails: list[str]
require_email_verification: bool
# Aggregate, non-identifying counts (live rows only).
user_count: int
tree_count: int
# Instance-wide AI configuration (no secrets).
default_llm_provider: str
ai_providers: list[ConfiguredProvider]
+22
View File
@@ -0,0 +1,22 @@
from pydantic import BaseModel
class ConfiguredProvider(BaseModel):
name: str
model: str
class TreeAiPolicyRead(BaseModel):
# The model non-owners' assistant uses (null = none).
member_provider: str | None
# The model the association/recommendation engine uses (null = none).
recommender_provider: str | None
# Providers the operator has configured (from env). The owner may use any of
# these; the two settings above restrict members and the recommender to one.
configured_providers: list[ConfiguredProvider]
default_provider: str
class TreeAiPolicyUpdate(BaseModel):
member_provider: str | None = None
recommender_provider: str | None = None
+5
View File
@@ -29,6 +29,11 @@ class PasswordResetConfirm(BaseModel):
new_password: str = Field(min_length=8) new_password: str = Field(min_length=8)
class PasswordChange(BaseModel):
current_password: str
new_password: str = Field(min_length=8)
class SessionRead(BaseModel): class SessionRead(BaseModel):
user: UserRead user: UserRead
token: str token: str
+44
View File
@@ -0,0 +1,44 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import ChangeProposalOrigin, ChangeProposalStatus
class ProposalOperation(BaseModel):
op: str # create | update | delete
entity_type: str # person | name | event | relationship | source | citation
entity_id: uuid.UUID | None = None
payload: dict = {}
class ChangeProposalCreate(BaseModel):
summary: str
rationale: str | None = None
origin: ChangeProposalOrigin = ChangeProposalOrigin.contributor
operations: list[ProposalOperation]
class ProposalReview(BaseModel):
note: str | None = None
# Optional edited operations to apply instead of the original (approve-with-edits).
operations: list[ProposalOperation] | None = None
class ChangeProposalRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
status: ChangeProposalStatus
origin: ChangeProposalOrigin
created_by_user_id: uuid.UUID | None
summary: str
rationale: str | None
operations: list
reviewed_by_user_id: uuid.UUID | None
reviewed_at: datetime | None
review_note: str | None
apply_error: str | None
created_at: datetime
+56
View File
@@ -0,0 +1,56 @@
import uuid
from pydantic import BaseModel
class DeceasedCandidate(BaseModel):
person_id: uuid.UUID
name: str
birth_year: int
class DeceasedByChildCandidate(BaseModel):
person_id: uuid.UUID
name: str
child_birth_year: int
class DeceasedApply(BaseModel):
person_ids: list[uuid.UUID]
class GenderProposal(BaseModel):
person_id: uuid.UUID
name: str
proposed_gender: str
class GenderUpdate(BaseModel):
person_id: uuid.UUID
gender: str
class GenderApply(BaseModel):
updates: list[GenderUpdate]
class NameIssue(BaseModel):
name_id: uuid.UUID
person_id: uuid.UUID
given: str | None = None
surname: str | None = None
issue: str
class NameEdit(BaseModel):
name_id: uuid.UUID
given: str | None = None
surname: str | None = None
class NameApply(BaseModel):
edits: list[NameEdit]
class CleanupResult(BaseModel):
updated: int
+52
View File
@@ -0,0 +1,52 @@
import uuid
from datetime import date, datetime
from pydantic import BaseModel, ConfigDict
class EventCreate(BaseModel):
event_type: str
# Exactly one subject: a person or a partnership (relationship).
person_id: uuid.UUID | None = None
relationship_id: uuid.UUID | None = None
place_id: uuid.UUID | None = None
# Verbatim date string (e.g. "ABT 1850") and/or a normalized range.
date_value: str | None = None
date_start: date | None = None
date_end: date | None = None
date_precision: str | None = None
calendar: str = "gregorian"
detail: str | None = None
notes: str | None = None
class EventUpdate(BaseModel):
# All optional; only fields explicitly sent are changed (PATCH semantics).
event_type: str | None = None
place_id: uuid.UUID | None = None
date_value: str | None = None
date_start: date | None = None
date_end: date | None = None
date_precision: str | None = None
calendar: str | None = None
detail: str | None = None
notes: str | None = None
class EventRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
event_type: str
person_id: uuid.UUID | None
relationship_id: uuid.UUID | None
place_id: uuid.UUID | None
date_value: str | None
date_start: date | None
date_end: date | None
date_precision: str | None
calendar: str
detail: str | None
notes: str | None
created_at: datetime
+25
View File
@@ -0,0 +1,25 @@
import uuid
from pydantic import BaseModel
class ImportReport(BaseModel):
counts: dict[str, int]
unmapped_tags: list[str]
class DuplicateMatch(BaseModel):
# An incoming GEDCOM person that resembles an existing one in the tree.
xref: str
incoming_name: str
incoming_birth_year: str | None = None
existing_person_id: uuid.UUID
existing_name: str
existing_birth_year: str | None = None
score: str # "high" | "medium"
class ImportPreview(BaseModel):
counts: dict[str, int]
potential_duplicates: list[DuplicateMatch]
unmapped_tags: list[str]
+29
View File
@@ -0,0 +1,29 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
class MediaUpdate(BaseModel):
title: str | None = None
person_id: uuid.UUID | None = None
event_id: uuid.UUID | None = None
source_id: uuid.UUID | None = None
class MediaRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
original_filename: str
content_type: str
byte_size: int
checksum_sha256: str
title: str | None
person_id: uuid.UUID | None
event_id: uuid.UUID | None
source_id: uuid.UUID | None
created_at: datetime
# Presigned download URL, filled in by the router from the ObjectStore.
url: str | None = None
+26
View File
@@ -0,0 +1,26 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import MembershipRole
class MembershipRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
user_id: uuid.UUID
email: str
display_name: str | None
role: MembershipRole
created_at: datetime
class MemberAdd(BaseModel):
email: str
role: MembershipRole = MembershipRole.viewer
class MemberRoleUpdate(BaseModel):
role: MembershipRole
+42
View File
@@ -0,0 +1,42 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
class NameCreate(BaseModel):
# Open vocabulary: birth/maiden, married, alias, religious, nickname, ...
name_type: str = "birth"
given: str | None = None
surname: str | None = None
prefix: str | None = None
suffix: str | None = None
nickname: str | None = None
is_primary: bool = False
class NameUpdate(BaseModel):
name_type: str | None = None
given: str | None = None
surname: str | None = None
prefix: str | None = None
suffix: str | None = None
nickname: str | None = None
is_primary: bool | None = None
class NameRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
person_id: uuid.UUID
name_type: str
given: str | None
surname: str | None
prefix: str | None
suffix: str | None
nickname: str | None
is_primary: bool
sort_order: int
created_at: datetime
+10
View File
@@ -15,6 +15,16 @@ class PersonCreate(BaseModel):
notes: str | None = None notes: str | None = None
class PersonUpdate(BaseModel):
# Person fields + the primary name's parts; only sent fields are changed.
given: str | None = None
surname: str | None = None
gender: str | None = None
is_living: bool | None = None
privacy: PersonPrivacy | None = None
notes: str | None = None
class PersonRead(BaseModel): class PersonRead(BaseModel):
model_config = ConfigDict(from_attributes=True) model_config = ConfigDict(from_attributes=True)
+33
View File
@@ -0,0 +1,33 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import ParentChildQualifier, RelationshipType
class RelationshipCreate(BaseModel):
type: RelationshipType
person_from_id: uuid.UUID
person_to_id: uuid.UUID
# Only meaningful for parent_child edges (from = parent, to = child).
qualifier: ParentChildQualifier | None = None
notes: str | None = None
class RelationshipUpdate(BaseModel):
qualifier: ParentChildQualifier | None = None
notes: str | None = None
class RelationshipRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
type: RelationshipType
person_from_id: uuid.UUID
person_to_id: uuid.UUID
qualifier: ParentChildQualifier | None
notes: str | None
created_at: datetime
+78
View File
@@ -0,0 +1,78 @@
import uuid
from datetime import datetime
from pydantic import BaseModel, ConfigDict
from app.models.enums import CitationConfidence
class SourceCreate(BaseModel):
title: str
author: str | None = None
source_type: str | None = None
repository: str | None = None
url: str | None = None
citation_text: str | None = None
publication_info: str | None = None
quality_note: str | None = None
class SourceRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
title: str
author: str | None
source_type: str | None
repository: str | None
url: str | None
citation_text: str | None
publication_info: str | None
quality_note: str | None
created_at: datetime
class SourceUpdate(BaseModel):
title: str | None = None
author: str | None = None
source_type: str | None = None
repository: str | None = None
url: str | None = None
citation_text: str | None = None
publication_info: str | None = None
quality_note: str | None = None
class CitationUpdate(BaseModel):
page: str | None = None
detail: str | None = None
confidence: CitationConfidence | None = None
class CitationCreate(BaseModel):
source_id: uuid.UUID
# Exactly one target fact.
person_id: uuid.UUID | None = None
event_id: uuid.UUID | None = None
name_id: uuid.UUID | None = None
relationship_id: uuid.UUID | None = None
page: str | None = None
detail: str | None = None
confidence: CitationConfidence | None = None
class CitationRead(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
tree_id: uuid.UUID
source_id: uuid.UUID
person_id: uuid.UUID | None
event_id: uuid.UUID | None
name_id: uuid.UUID | None
relationship_id: uuid.UUID | None
page: str | None
detail: str | None
confidence: CitationConfidence | None
created_at: datetime
+26
View File
@@ -12,6 +12,18 @@ class TreeCreate(BaseModel):
visibility: TreeVisibility = TreeVisibility.private visibility: TreeVisibility = TreeVisibility.private
class TreeUpdate(BaseModel):
name: str | None = None
description: str | None = None
visibility: TreeVisibility | None = None
home_person_id: uuid.UUID | None = None
class TreePurge(BaseModel):
# Retype the tree's name to confirm a permanent, irreversible delete.
confirm_name: str
class TreeRead(BaseModel): class TreeRead(BaseModel):
model_config = ConfigDict(from_attributes=True) model_config = ConfigDict(from_attributes=True)
@@ -20,4 +32,18 @@ class TreeRead(BaseModel):
description: str | None description: str | None
visibility: TreeVisibility visibility: TreeVisibility
owner_id: uuid.UUID owner_id: uuid.UUID
home_person_id: uuid.UUID | None = None
created_at: datetime created_at: datetime
class PublicTreeRead(BaseModel):
"""Tree projection for the public surface — deliberately omits owner_id so a
public/unlisted tree doesn't reveal which account owns it."""
model_config = ConfigDict(from_attributes=True)
id: uuid.UUID
name: str
description: str | None
visibility: TreeVisibility
home_person_id: uuid.UUID | None = None
+9
View File
@@ -19,4 +19,13 @@ class UserRead(BaseModel):
email: str email: str
display_name: str | None display_name: str | None
email_verified_at: datetime | None email_verified_at: datetime | None
self_person_id: uuid.UUID | None = None
created_at: datetime created_at: datetime
# Operational role, not a DB column: true when this account's email is named
# in OWNER_EMAIL. Set by the API layer (see users.read_me).
is_instance_owner: bool = False
class UserSelfPersonUpdate(BaseModel):
# null clears the link; otherwise the Person that represents this account.
self_person_id: uuid.UUID | None = None
+352
View File
@@ -0,0 +1,352 @@
"""Account-level data portability: export the signed-in user's owned trees as a
zip (JSON + media bytes), restore such a zip into a brand-new tree
(non-destructive), and delete the account.
The export format is a zip containing ``account.json`` plus ``media/<id>`` blobs.
Restore always creates new trees and remaps ids, so it can't clobber existing
data.
"""
import hashlib
import io
import json
import uuid
import zipfile
from datetime import UTC, date, datetime
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.integrations.objectstore.base import ObjectStore
from app.models.auth import Session as SessionModel
from app.models.enums import MembershipRole
from app.models.event import Event
from app.models.media import Media
from app.models.person import Name, Person
from app.models.place import Place
from app.models.relationship import Relationship
from app.models.source import Citation, Source
from app.models.tree import Tree, TreeMembership
from app.models.user import User
from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound
EXPORT_VERSION = 1
_DROP = {"created_at", "updated_at", "deleted_at", "tree_id"}
# Media columns rebuilt on import (storage is re-keyed, checksum recomputed).
_MEDIA_DROP = _DROP | {"uploader_id", "storage_key", "byte_size", "checksum_sha256"}
_DATE_FIELDS = {"date_start", "date_end"}
def _row(obj, drop: set[str]) -> dict:
out: dict = {}
for col in obj.__table__.columns.keys(): # noqa: SIM118
if col in drop:
continue
out[col] = getattr(obj, col)
return out
async def _entities(session: AsyncSession, model, tree_id: uuid.UUID):
stmt = select(model).where(model.tree_id == tree_id, model.deleted_at.is_(None))
return list((await session.execute(stmt)).scalars().all())
async def export_account(session: AsyncSession, store: ObjectStore, *, user: User) -> bytes:
"""Build a zip of every tree the user owns: account.json + media blobs."""
trees = list(
(
await session.execute(
select(Tree).where(Tree.owner_id == user.id, Tree.deleted_at.is_(None))
)
).scalars().all()
)
payload: dict = {
"version": EXPORT_VERSION,
"user": {"email": user.email, "display_name": user.display_name},
"trees": [],
}
media_blobs: list[tuple[str, bytes]] = []
for tree in trees:
media_rows = await _entities(session, Media, tree.id)
media_out = []
for m in media_rows:
ref = f"media/{m.id}"
rec = _row(m, _MEDIA_DROP)
rec["_file"] = ref
media_out.append(rec)
try:
media_blobs.append((ref, await store.get_object(key=m.storage_key)))
except Exception: # noqa: BLE001 — a missing blob shouldn't abort the export
rec["_file"] = None
payload["trees"].append({
"tree": {
"name": tree.name,
"description": tree.description,
"visibility": tree.visibility,
"home_person_id": tree.home_person_id,
},
"places": [_row(p, _DROP) for p in await _entities(session, Place, tree.id)],
"persons": [_row(p, _DROP) for p in await _entities(session, Person, tree.id)],
"names": [_row(n, _DROP) for n in await _entities(session, Name, tree.id)],
"relationships": [
_row(r, _DROP) for r in await _entities(session, Relationship, tree.id)
],
"events": [_row(e, _DROP) for e in await _entities(session, Event, tree.id)],
"sources": [_row(s, _DROP) for s in await _entities(session, Source, tree.id)],
"citations": [_row(c, _DROP) for c in await _entities(session, Citation, tree.id)],
"media": media_out,
})
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
zf.writestr("account.json", json.dumps(payload, default=str, indent=2))
for ref, blob in media_blobs:
zf.writestr(ref, blob)
return buf.getvalue()
def _as_uuid(v) -> uuid.UUID | None:
return uuid.UUID(v) if v else None
def _as_date(v) -> date | None:
return date.fromisoformat(v) if v else None
async def import_account(
session: AsyncSession, store: ObjectStore, *, user: User, raw_zip: bytes
) -> dict:
"""Restore an exported zip into NEW trees owned by the user. Non-destructive:
every record gets a fresh id; nothing existing is touched."""
try:
zf = zipfile.ZipFile(io.BytesIO(raw_zip))
payload = json.loads(zf.read("account.json"))
except (zipfile.BadZipFile, KeyError, json.JSONDecodeError) as e:
raise NotFound("not a valid Provenance export") from e
counts: dict[str, int] = {"trees": 0, "persons": 0, "events": 0, "media": 0}
for tdata in payload.get("trees", []):
t = tdata.get("tree", {})
tree = Tree(
owner_id=user.id,
name=(t.get("name") or "Imported tree"),
description=t.get("description"),
visibility=t.get("visibility") or "private",
)
session.add(tree)
await session.flush()
session.add(
TreeMembership(tree_id=tree.id, user_id=user.id, role=MembershipRole.owner)
)
counts["trees"] += 1
# id remaps from the export's ids to the freshly created ones.
pmap: dict[str, uuid.UUID] = {}
rmap: dict[str, uuid.UUID] = {}
smap: dict[str, uuid.UUID] = {}
nmap: dict[str, uuid.UUID] = {}
emap: dict[str, uuid.UUID] = {}
plmap: dict[str, uuid.UUID] = {}
for pl in tdata.get("places", []):
obj = Place(
tree_id=tree.id,
name=pl.get("name") or "",
place_type=pl.get("place_type"),
latitude=pl.get("latitude"),
longitude=pl.get("longitude"),
)
session.add(obj)
await session.flush()
plmap[pl["id"]] = obj.id
for p in tdata.get("persons", []):
obj = Person(
tree_id=tree.id,
gender=p.get("gender"),
is_living=p.get("is_living"),
privacy=p.get("privacy") or "inherit",
notes=p.get("notes"),
)
session.add(obj)
await session.flush()
pmap[p["id"]] = obj.id
counts["persons"] += 1
for n in tdata.get("names", []):
pid = pmap.get(n.get("person_id"))
if pid is None:
continue
obj = Name(
tree_id=tree.id,
person_id=pid,
name_type=n.get("name_type") or "birth",
given=n.get("given"),
surname=n.get("surname"),
prefix=n.get("prefix"),
suffix=n.get("suffix"),
nickname=n.get("nickname"),
display_name=n.get("display_name"),
is_primary=bool(n.get("is_primary")),
sort_order=n.get("sort_order") or 0,
)
session.add(obj)
await session.flush()
nmap[n["id"]] = obj.id
for r in tdata.get("relationships", []):
a = pmap.get(r.get("person_from_id"))
b = pmap.get(r.get("person_to_id"))
if a is None or b is None:
continue
obj = Relationship(
tree_id=tree.id,
type=r.get("type"),
person_from_id=a,
person_to_id=b,
qualifier=r.get("qualifier"),
notes=r.get("notes"),
)
session.add(obj)
await session.flush()
rmap[r["id"]] = obj.id
for e in tdata.get("events", []):
obj = Event(
tree_id=tree.id,
event_type=e.get("event_type") or "other",
person_id=pmap.get(e.get("person_id")),
relationship_id=rmap.get(e.get("relationship_id")),
place_id=plmap.get(e.get("place_id")),
date_value=e.get("date_value"),
date_start=_as_date(e.get("date_start")),
date_end=_as_date(e.get("date_end")),
date_precision=e.get("date_precision"),
calendar=e.get("calendar") or "gregorian",
detail=e.get("detail"),
notes=e.get("notes"),
)
session.add(obj)
await session.flush()
emap[e["id"]] = obj.id
counts["events"] += 1
for s in tdata.get("sources", []):
obj = Source(
tree_id=tree.id,
title=s.get("title") or "Untitled source",
author=s.get("author"),
source_type=s.get("source_type"),
repository=s.get("repository"),
url=s.get("url"),
citation_text=s.get("citation_text"),
publication_info=s.get("publication_info"),
quality_note=s.get("quality_note"),
)
session.add(obj)
await session.flush()
smap[s["id"]] = obj.id
for c in tdata.get("citations", []):
sid = smap.get(c.get("source_id"))
if sid is None:
continue
session.add(
Citation(
tree_id=tree.id,
source_id=sid,
person_id=pmap.get(c.get("person_id")),
event_id=emap.get(c.get("event_id")),
name_id=nmap.get(c.get("name_id")),
relationship_id=rmap.get(c.get("relationship_id")),
page=c.get("page"),
detail=c.get("detail"),
confidence=c.get("confidence"),
)
)
for m in tdata.get("media", []):
ref = m.get("_file")
if not ref:
continue
try:
blob = zf.read(ref)
except KeyError:
continue
media_id = uuid.uuid4()
filename = m.get("original_filename") or "upload"
key = f"{tree.id}/{media_id}/{filename}"
await store.ensure_bucket()
await store.put_object(
key=key,
data=blob,
content_type=m.get("content_type") or "application/octet-stream",
)
session.add(
Media(
id=media_id,
tree_id=tree.id,
uploader_id=user.id,
storage_key=key,
original_filename=filename,
content_type=m.get("content_type") or "application/octet-stream",
byte_size=len(blob),
checksum_sha256=hashlib.sha256(blob).hexdigest(),
title=m.get("title"),
person_id=pmap.get(m.get("person_id")),
event_id=emap.get(m.get("event_id")),
source_id=smap.get(m.get("source_id")),
)
)
counts["media"] += 1
# Remap the home person last, once persons exist.
home = t.get("home_person_id")
if home and home in pmap:
tree.home_person_id = pmap[home]
record_audit(
session,
action="import",
entity_type="Account",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=user.id,
after=counts,
)
await session.commit()
return counts
async def delete_account(session: AsyncSession, *, user: User, confirm_email: str) -> None:
"""Soft-delete the account: the user, the trees they own, and all their
sessions. Requires the user to retype their email as a guard."""
if confirm_email.strip().lower() != user.email.lower():
raise Forbidden("email confirmation does not match")
now = datetime.now(UTC)
await session.execute(
update(Tree)
.where(Tree.owner_id == user.id, Tree.deleted_at.is_(None))
.values(deleted_at=now)
)
await session.execute(
update(SessionModel)
.where(SessionModel.user_id == user.id, SessionModel.revoked_at.is_(None))
.values(revoked_at=now)
)
user.deleted_at = now
record_audit(
session,
action="delete",
entity_type="User",
entity_id=user.id,
actor_user_id=user.id,
)
await session.commit()
+77
View File
@@ -0,0 +1,77 @@
"""Per-tree AI model policy — owner-only. Assigns which configured provider
members and the recommender use; the owner may use any configured provider.
The operator decides which providers exist (env / registry); the tree owner
decides who uses which. See app/api/deps.py for the registry.
"""
import uuid
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import configured_llm_providers
from app.models.enums import MembershipRole
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.exceptions import Forbidden
async def _require_owner(session: AsyncSession, *, actor: User, tree: Tree) -> None:
role = await privacy.get_membership_role(session, actor.id, tree.id)
if role is not MembershipRole.owner:
raise Forbidden("only the tree owner can configure AI")
def _names() -> set[str]:
return {p["name"] for p in configured_llm_providers()}
async def get_policy(session: AsyncSession, *, actor: User, tree: Tree) -> dict:
await _require_owner(session, actor=actor, tree=tree)
from app.core.config import get_settings
return {
"member_provider": tree.ai_member_provider,
"recommender_provider": tree.ai_recommender_provider,
"configured_providers": configured_llm_providers(),
"default_provider": get_settings().default_llm_provider,
}
async def update_policy(
session: AsyncSession,
*,
actor: User,
tree: Tree,
member_provider: str | None,
recommender_provider: str | None,
) -> dict:
await _require_owner(session, actor=actor, tree=tree)
valid = _names()
for value in (member_provider, recommender_provider):
if value is not None and value not in valid:
raise Forbidden(f"'{value}' is not a configured provider")
tree.ai_member_provider = member_provider
tree.ai_recommender_provider = recommender_provider
await session.commit()
await session.refresh(tree)
return await get_policy(session, actor=actor, tree=tree)
# --- Resolution helpers (for the future assistant / recommender) -------------
def provider_name_for_member(tree: Tree) -> str | None:
"""Provider an ordinary member's assistant should use, if any."""
return tree.ai_member_provider
def provider_name_for_recommender(tree: Tree) -> str | None:
return tree.ai_recommender_provider
def provider_name_for_owner(tree: Tree, requested: str | None = None) -> str | None:
"""The owner may use any configured provider; default to the requested one."""
if requested and requested in _names():
return requested
return tree.ai_member_provider # fall back to the member model
+11 -2
View File
@@ -3,6 +3,7 @@ the change to a User (or the assistant principal acting for a User). Staged on
the session; the caller commits as part of its unit of work. the session; the caller commits as part of its unit of work.
""" """
import json
import uuid import uuid
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
@@ -11,6 +12,14 @@ from app.models.audit import AuditEntry
from app.models.enums import AuditActorType from app.models.enums import AuditActorType
def _json_safe(d: dict | None) -> dict | None:
"""Coerce a change dict to JSON-native types (UUIDs, enums, dates -> str) so
it lands in the JSON audit column regardless of what the caller passed."""
if d is None:
return None
return json.loads(json.dumps(d, default=str))
def record_audit( def record_audit(
session: AsyncSession, session: AsyncSession,
*, *,
@@ -30,8 +39,8 @@ def record_audit(
tree_id=tree_id, tree_id=tree_id,
actor_user_id=actor_user_id, actor_user_id=actor_user_id,
actor_type=actor_type, actor_type=actor_type,
before=before, before=_json_safe(before),
after=after, after=_json_safe(after),
) )
session.add(entry) session.add(entry)
return entry return entry
+30 -3
View File
@@ -9,7 +9,7 @@ from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings from app.core.config import get_settings
from app.core.security import generate_token, hash_password, hash_token from app.core.security import generate_token, hash_password, hash_token, verify_password
from app.integrations.auth.local import LocalAuthProvider from app.integrations.auth.local import LocalAuthProvider
from app.integrations.mailer.base import Mailer from app.integrations.mailer.base import Mailer
from app.models.auth import Session as SessionModel from app.models.auth import Session as SessionModel
@@ -17,7 +17,7 @@ from app.models.auth import UserToken
from app.models.enums import TokenPurpose from app.models.enums import TokenPurpose
from app.models.user import User from app.models.user import User
from app.services.audit import record_audit from app.services.audit import record_audit
from app.services.exceptions import Conflict, NotFound from app.services.exceptions import Conflict, Forbidden, NotFound
_local_provider = LocalAuthProvider() _local_provider = LocalAuthProvider()
@@ -113,6 +113,8 @@ async def login(
user = await _local_provider.authenticate(session, identifier=email, secret=password) user = await _local_provider.authenticate(session, identifier=email, secret=password)
if user is None: if user is None:
return None return None
if get_settings().require_email_verification and user.email_verified_at is None:
raise Forbidden("email not verified — check your inbox for the verification link")
raw_token, record = _issue_session(session, user) raw_token, record = _issue_session(session, user)
record_audit( record_audit(
session, action="login", entity_type="User", entity_id=user.id, actor_user_id=user.id session, action="login", entity_type="User", entity_id=user.id, actor_user_id=user.id
@@ -141,11 +143,16 @@ async def resolve_session_user(session: AsyncSession, *, raw_token: str) -> User
).scalar_one_or_none() ).scalar_one_or_none()
if record is None or record.revoked_at is not None or record.expires_at <= _now(): if record is None or record.revoked_at is not None or record.expires_at <= _now():
return None return None
return ( user = (
await session.execute( await session.execute(
select(User).where(User.id == record.user_id, User.deleted_at.is_(None)) select(User).where(User.id == record.user_id, User.deleted_at.is_(None))
) )
).scalar_one_or_none() ).scalar_one_or_none()
# The single read-side enforcement: an unverified user has no active session
# when verification is required. Gates every authenticated request at once.
if user is not None and get_settings().require_email_verification and user.email_verified_at is None:
return None
return user
async def verify_email(session: AsyncSession, *, raw_token: str) -> None: async def verify_email(session: AsyncSession, *, raw_token: str) -> None:
@@ -178,6 +185,26 @@ async def request_password_reset(session: AsyncSession, mailer: Mailer, *, email
await mailer.send_password_reset(to=email, link=_link("/auth/reset-password", raw)) await mailer.send_password_reset(to=email, link=_link("/auth/reset-password", raw))
async def change_password(
session: AsyncSession, *, user: User, current_password: str, new_password: str
) -> None:
"""Change a logged-in user's password after re-verifying the current one.
Revokes other sessions so a changed password takes effect everywhere."""
if not user.hashed_password or not verify_password(
user.hashed_password, current_password
):
raise Forbidden("current password is incorrect")
user.hashed_password = hash_password(new_password)
record_audit(
session,
action="change_password",
entity_type="User",
entity_id=user.id,
actor_user_id=user.id,
)
await session.commit()
async def reset_password(session: AsyncSession, *, raw_token: str, new_password: str) -> None: async def reset_password(session: AsyncSession, *, raw_token: str, new_password: str) -> None:
token = await _consume_token(session, raw_token, TokenPurpose.password_reset) token = await _consume_token(session, raw_token, TokenPurpose.password_reset)
await session.execute( await session.execute(
@@ -0,0 +1,355 @@
"""ChangeProposal lifecycle: propose (assistant/contributor) → review → apply/reject.
The structural guarantee (CLAUDE.md #1): a proposal's operations are executed
ONLY by ``apply()``, which requires the actor be an editor and dispatches every
op through the normal editing services — so each change passes the privacy
engine and is audited as the approving human. ``propose()`` only inserts a
pending row; it performs no domain mutation. See docs/design/change-proposal.md.
"""
import uuid
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.change_proposal import ChangeProposal
from app.models.enums import (
ChangeProposalOrigin,
ChangeProposalStatus,
CitationConfidence,
ParentChildQualifier,
RelationshipType,
)
from app.models.tree import Tree
from app.models.user import User
from app.services import (
citation_service,
event_service,
name_service,
person_service,
privacy,
relationship_service,
source_service,
)
from app.services.exceptions import Conflict, Forbidden, NotFound
def _now() -> datetime:
return datetime.now(UTC)
def _uuid(v) -> uuid.UUID | None:
return uuid.UUID(str(v)) if v else None
async def _require_editor(session: AsyncSession, *, actor: User, tree: Tree) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
async def _require_member(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> None:
# Proposals can reference unredacted facts → members only.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
raise Forbidden("only members can see change proposals")
async def _load(
session: AsyncSession, tree: Tree, proposal_id: uuid.UUID
) -> ChangeProposal:
cp = (
await session.execute(
select(ChangeProposal).where(
ChangeProposal.id == proposal_id,
ChangeProposal.tree_id == tree.id,
ChangeProposal.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if cp is None:
raise NotFound("proposal not found")
return cp
async def propose(
session: AsyncSession,
*,
tree: Tree,
origin: ChangeProposalOrigin,
created_by: uuid.UUID | None,
summary: str,
rationale: str | None,
operations: list[dict],
) -> ChangeProposal:
"""Insert a pending proposal. The ONLY mutation here is the proposal row — no
tree data changes. (No edit-rights check: proposing isn't writing.)"""
cp = ChangeProposal(
tree_id=tree.id,
origin=origin,
created_by_user_id=created_by,
summary=summary,
rationale=rationale,
operations=operations,
status=ChangeProposalStatus.pending,
)
session.add(cp)
await session.commit()
await session.refresh(cp)
return cp
async def list_proposals(
session: AsyncSession,
*,
viewer_id: uuid.UUID,
tree: Tree,
status: ChangeProposalStatus | None = None,
) -> list[ChangeProposal]:
await _require_member(session, viewer_id=viewer_id, tree=tree)
stmt = select(ChangeProposal).where(
ChangeProposal.tree_id == tree.id, ChangeProposal.deleted_at.is_(None)
)
if status is not None:
stmt = stmt.where(ChangeProposal.status == status)
stmt = stmt.order_by(ChangeProposal.created_at.desc())
return list((await session.execute(stmt)).scalars().all())
async def get_proposal(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, proposal_id: uuid.UUID
) -> ChangeProposal:
await _require_member(session, viewer_id=viewer_id, tree=tree)
return await _load(session, tree, proposal_id)
async def reject(
session: AsyncSession,
*,
actor: User,
tree: Tree,
proposal_id: uuid.UUID,
note: str | None = None,
) -> ChangeProposal:
await _require_editor(session, actor=actor, tree=tree)
cp = await _load(session, tree, proposal_id)
if cp.status is not ChangeProposalStatus.pending:
raise Conflict("proposal is not pending")
cp.status = ChangeProposalStatus.rejected
cp.reviewed_by_user_id = actor.id
cp.reviewed_at = _now()
cp.review_note = note
await session.commit()
await session.refresh(cp)
return cp
async def apply(
session: AsyncSession,
*,
actor: User,
tree: Tree,
proposal_id: uuid.UUID,
edited_operations: list[dict] | None = None,
) -> ChangeProposal:
await _require_editor(session, actor=actor, tree=tree)
cp = await _load(session, tree, proposal_id)
if cp.status is not ChangeProposalStatus.pending:
raise Conflict("proposal is not pending")
ops = edited_operations if edited_operations is not None else list(cp.operations)
try:
for op in ops:
await _dispatch(session, actor=actor, tree=tree, op=op)
except Conflict:
raise
except Exception as exc: # noqa: BLE001 — record the failure on the proposal
err = f"{type(exc).__name__}: {exc}"[:2000]
# The editing services raise (NotFound/Forbidden/validation) before
# committing, so the transaction is clean — record the error and commit.
# If a later op did write before failing, those ops already committed
# (v1 isn't cross-op transactional; see the design note).
cp = await _load(session, tree, proposal_id)
cp.apply_error = err
await session.commit()
raise Conflict(f"could not apply proposal: {err}") from exc
if edited_operations is not None:
cp.operations = edited_operations
cp.status = ChangeProposalStatus.applied
cp.reviewed_by_user_id = actor.id
cp.reviewed_at = _now()
cp.apply_error = None
await session.commit()
await session.refresh(cp)
return cp
async def delete_proposal(
session: AsyncSession, *, actor: User, tree: Tree, proposal_id: uuid.UUID
) -> None:
await _require_editor(session, actor=actor, tree=tree)
cp = await _load(session, tree, proposal_id)
cp.deleted_at = _now()
await session.commit()
def _bad(entity_type: str, action: str) -> Conflict:
return Conflict(f"unsupported operation '{action}' on '{entity_type}'")
async def _dispatch(session: AsyncSession, *, actor: User, tree: Tree, op: dict) -> None:
"""Route one operation through the matching editing service (privacy + audit)."""
et = op.get("entity_type")
action = op.get("op")
payload = op.get("payload") or {}
eid = op.get("entity_id")
if et == "person":
if action == "create":
await person_service.create_person(
session,
actor=actor,
tree=tree,
given=payload.get("given"),
surname=payload.get("surname"),
gender=payload.get("gender"),
is_living=payload.get("is_living"),
notes=payload.get("notes"),
)
elif action == "update":
await person_service.update_person(
session, actor=actor, tree=tree, person_id=_uuid(eid), changes=payload
)
elif action == "delete":
await person_service.delete_person(
session,
actor=actor,
tree=tree,
person_id=_uuid(eid),
cascade=bool(payload.get("cascade", False)),
)
else:
raise _bad(et, action)
elif et == "event":
if action == "create":
await event_service.create_event(
session,
actor=actor,
tree=tree,
event_type=payload["event_type"],
person_id=_uuid(payload.get("person_id")),
relationship_id=_uuid(payload.get("relationship_id")),
date_value=payload.get("date_value"),
date_precision=payload.get("date_precision"),
detail=payload.get("detail"),
notes=payload.get("notes"),
)
elif action == "update":
await event_service.update_event(
session, actor=actor, tree=tree, event_id=_uuid(eid), changes=payload
)
elif action == "delete":
await event_service.delete_event(
session, actor=actor, tree=tree, event_id=_uuid(eid)
)
else:
raise _bad(et, action)
elif et == "relationship":
if action == "create":
await relationship_service.create_relationship(
session,
actor=actor,
tree=tree,
type=RelationshipType(payload["type"]),
person_from_id=_uuid(payload["person_from_id"]),
person_to_id=_uuid(payload["person_to_id"]),
qualifier=ParentChildQualifier(payload["qualifier"])
if payload.get("qualifier")
else None,
notes=payload.get("notes"),
)
elif action == "delete":
await relationship_service.delete_relationship(
session, actor=actor, tree=tree, relationship_id=_uuid(eid)
)
else:
raise _bad(et, action)
elif et == "name":
if action == "create":
await name_service.create_name(
session,
actor=actor,
tree=tree,
person_id=_uuid(payload["person_id"]),
name_type=payload.get("name_type", "birth"),
given=payload.get("given"),
surname=payload.get("surname"),
prefix=payload.get("prefix"),
suffix=payload.get("suffix"),
nickname=payload.get("nickname"),
is_primary=bool(payload.get("is_primary", False)),
)
elif action == "update":
changes = {k: v for k, v in payload.items() if k != "person_id"}
await name_service.update_name(
session,
actor=actor,
tree=tree,
person_id=_uuid(payload["person_id"]),
name_id=_uuid(eid),
changes=changes,
)
elif action == "delete":
await name_service.delete_name(
session,
actor=actor,
tree=tree,
person_id=_uuid(payload["person_id"]),
name_id=_uuid(eid),
)
else:
raise _bad(et, action)
elif et == "source":
if action == "create":
await source_service.create_source(
session,
actor=actor,
tree=tree,
title=payload["title"],
author=payload.get("author"),
source_type=payload.get("source_type"),
repository=payload.get("repository"),
url=payload.get("url"),
citation_text=payload.get("citation_text"),
publication_info=payload.get("publication_info"),
quality_note=payload.get("quality_note"),
)
elif action == "delete":
await source_service.delete_source(
session, actor=actor, tree=tree, source_id=_uuid(eid)
)
else:
raise _bad(et, action)
elif et == "citation":
if action == "create":
await citation_service.create_citation(
session,
actor=actor,
tree=tree,
source_id=_uuid(payload["source_id"]),
person_id=_uuid(payload.get("person_id")),
event_id=_uuid(payload.get("event_id")),
name_id=_uuid(payload.get("name_id")),
relationship_id=_uuid(payload.get("relationship_id")),
page=payload.get("page"),
detail=payload.get("detail"),
confidence=CitationConfidence(payload["confidence"])
if payload.get("confidence")
else None,
)
elif action == "delete":
await citation_service.delete_citation(
session, actor=actor, tree=tree, citation_id=_uuid(eid)
)
else:
raise _bad(et, action)
else:
raise Conflict(f"unsupported entity type '{et}'")
+182
View File
@@ -0,0 +1,182 @@
"""Citation service. A citation links one Source to exactly one fact (person,
event, name, or relationship) within a tree — the provenance spine."""
import uuid
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import CitationConfidence
from app.models.event import Event
from app.models.person import Name, Person
from app.models.relationship import Relationship
from app.models.source import Citation, Source
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Conflict, Forbidden, NotFound
# Citation target column -> model, for tenant/existence validation.
_TARGET_MODELS = {
"person_id": Person,
"event_id": Event,
"name_id": Name,
"relationship_id": Relationship,
}
async def _in_tree(session: AsyncSession, model: type, id_: uuid.UUID, tree_id: uuid.UUID) -> bool:
row = (
await session.execute(
select(model.id).where(
model.id == id_, model.tree_id == tree_id, model.deleted_at.is_(None)
)
)
).scalar_one_or_none()
return row is not None
async def create_citation(
session: AsyncSession,
*,
actor: User,
tree: Tree,
source_id: uuid.UUID,
person_id: uuid.UUID | None = None,
event_id: uuid.UUID | None = None,
name_id: uuid.UUID | None = None,
relationship_id: uuid.UUID | None = None,
page: str | None = None,
detail: str | None = None,
confidence: CitationConfidence | None = None,
) -> Citation:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
targets = {
"person_id": person_id,
"event_id": event_id,
"name_id": name_id,
"relationship_id": relationship_id,
}
set_targets = {k: v for k, v in targets.items() if v is not None}
if len(set_targets) != 1:
raise Conflict("a citation must reference exactly one fact")
if not await _in_tree(session, Source, source_id, tree.id):
raise NotFound("source not found in this tree")
(target_col, target_id), = set_targets.items()
if not await _in_tree(session, _TARGET_MODELS[target_col], target_id, tree.id):
raise NotFound("cited fact not found in this tree")
citation = Citation(
tree_id=tree.id,
source_id=source_id,
person_id=person_id,
event_id=event_id,
name_id=name_id,
relationship_id=relationship_id,
page=page,
detail=detail,
confidence=confidence,
)
session.add(citation)
await session.flush()
record_audit(
session,
action="create",
entity_type="Citation",
entity_id=citation.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"source_id": str(source_id), target_col: str(target_id)},
)
await session.commit()
await session.refresh(citation)
return citation
async def list_citations(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree
) -> list[Citation]:
"""All citations in the tree — the UI maps them to facts to show 'sourced'
indicators in a single round-trip."""
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members get only citations whose cited fact resolves to a full-
# visibility person — a citation on a redacted living person's fact would
# otherwise leak that the person has that sourced fact.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_citations(
session, viewer_id=viewer_id, tree=tree
)
stmt = (
select(Citation)
.where(Citation.tree_id == tree.id, Citation.deleted_at.is_(None))
.order_by(Citation.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def update_citation(
session: AsyncSession, *, actor: User, tree: Tree, citation_id: uuid.UUID, changes: dict
) -> Citation:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
citation = (
await session.execute(
select(Citation).where(
Citation.id == citation_id,
Citation.tree_id == tree.id,
Citation.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if citation is None:
raise NotFound("citation not found")
for key in {"page", "detail", "confidence"} & changes.keys():
setattr(citation, key, changes[key])
record_audit(
session,
action="update",
entity_type="Citation",
entity_id=citation.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(citation)
return citation
async def delete_citation(
session: AsyncSession, *, actor: User, tree: Tree, citation_id: uuid.UUID
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
citation = (
await session.execute(
select(Citation).where(
Citation.id == citation_id,
Citation.tree_id == tree.id,
Citation.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if citation is None:
raise NotFound("citation not found")
citation.deleted_at = datetime.now(UTC)
record_audit(
session,
action="delete",
entity_type="Citation",
entity_id=citation.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
+382
View File
@@ -0,0 +1,382 @@
"""Bulk tree cleanup — preview/apply pairs for common import messes.
Per the project's #1 rule (the assistant proposes, humans approve), each fix has
a *preview* that returns the proposed changes and an *apply* that commits only
the ids/edits the user confirmed. Nothing here mutates without an explicit apply
call carrying the user's selections.
"""
import re
import uuid
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import RelationshipType
from app.models.event import Event
from app.models.person import Name, Person
from app.models.relationship import Relationship
from app.models.tree import Tree
from app.models.user import User
from app.services import gedcom, privacy
from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound
from app.services.name_gender_data import guess_sex
async def _require_editor(session: AsyncSession, *, actor: User, tree: Tree) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
async def _persons(session: AsyncSession, tree_id: uuid.UUID) -> list[Person]:
return list(
(
await session.execute(
select(Person).where(Person.tree_id == tree_id, Person.deleted_at.is_(None))
)
).scalars().all()
)
async def _primary_name_by_person(
session: AsyncSession, tree_id: uuid.UUID
) -> dict[uuid.UUID, Name]:
names = (
await session.execute(
select(Name)
.where(Name.tree_id == tree_id, Name.deleted_at.is_(None))
.order_by(Name.is_primary.desc(), Name.sort_order)
)
).scalars().all()
out: dict[uuid.UUID, Name] = {}
for n in names:
out.setdefault(n.person_id, n)
return out
async def _birth_year_by_person(session: AsyncSession, tree_id: uuid.UUID) -> dict[uuid.UUID, int]:
evs = (
await session.execute(
select(Event).where(
Event.tree_id == tree_id,
Event.deleted_at.is_(None),
Event.event_type == "birth",
)
)
).scalars().all()
out: dict[uuid.UUID, int] = {}
for e in evs:
if not e.person_id or e.person_id in out:
continue
y = e.date_start.year if e.date_start else None
if y is None:
ys = gedcom._year(e.date_value)
y = int(ys) if ys else None
if y is not None:
out[e.person_id] = y
return out
def _display(n: Name | None) -> str:
if n is None:
return "Unnamed"
return " ".join(x for x in (n.given, n.surname) if x) or (n.display_name or "Unnamed")
# ---- 1. Mark deceased by birth year -------------------------------------------------
async def preview_deceased(
session: AsyncSession, *, actor: User, tree: Tree, year: int
) -> list[dict]:
await _require_editor(session, actor=actor, tree=tree)
names = await _primary_name_by_person(session, tree.id)
years = await _birth_year_by_person(session, tree.id)
out: list[dict] = []
for p in await _persons(session, tree.id):
if p.is_living is False: # already deceased
continue
by = years.get(p.id)
if by is not None and by <= year:
out.append(
{"person_id": str(p.id), "name": _display(names.get(p.id)), "birth_year": by}
)
out.sort(key=lambda r: r["birth_year"])
return out
async def apply_deceased(
session: AsyncSession, *, actor: User, tree: Tree, person_ids: list[uuid.UUID]
) -> int:
await _require_editor(session, actor=actor, tree=tree)
persons = (
await session.execute(
select(Person).where(
Person.tree_id == tree.id,
Person.deleted_at.is_(None),
Person.id.in_(person_ids),
)
)
).scalars().all()
for p in persons:
p.is_living = False
record_audit(
session,
action="cleanup_deceased",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"count": len(persons)},
)
await session.commit()
return len(persons)
# ---- 1b. Mark deceased by a CHILD's birth year -------------------------------------
# For parents whose own birth date is missing (so the birth-year rule can't reach
# them) but who have a child born long ago — they're necessarily deceased. Applies
# through the same apply_deceased() path.
async def preview_deceased_by_child(
session: AsyncSession, *, actor: User, tree: Tree, year: int
) -> list[dict]:
await _require_editor(session, actor=actor, tree=tree)
names = await _primary_name_by_person(session, tree.id)
years = await _birth_year_by_person(session, tree.id)
rels = (
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
Relationship.type == RelationshipType.parent_child,
)
)
).scalars().all()
# parent id -> earliest child birth year, among children born on/before `year`.
earliest_child: dict[uuid.UUID, int] = {}
for r in rels:
cy = years.get(r.person_to_id) # the child's birth year
if cy is None or cy > year:
continue
if r.person_from_id not in earliest_child or cy < earliest_child[r.person_from_id]:
earliest_child[r.person_from_id] = cy
persons = {p.id: p for p in await _persons(session, tree.id)}
out: list[dict] = []
for parent_id, cy in earliest_child.items():
p = persons.get(parent_id)
if p is None or p.is_living is False: # gone or already deceased
continue
out.append(
{
"person_id": str(parent_id),
"name": _display(names.get(parent_id)),
"child_birth_year": cy,
}
)
out.sort(key=lambda r: r["child_birth_year"])
return out
# ---- 2. Re-derive gender from a source GEDCOM (matches by name) ----------------------
async def preview_gender(
session: AsyncSession, *, actor: User, tree: Tree, gedcom_text: str
) -> list[dict]:
await _require_editor(session, actor=actor, tree=tree)
name2sex: dict[str, str] = {}
for rec in gedcom.parse_records(gedcom_text):
if rec.tag != "INDI":
continue
summ = gedcom._person_summary(rec)
sex = gedcom._sex(rec.text("SEX"))
if sex and summ["norm"]:
name2sex.setdefault(summ["norm"], sex)
names = await _primary_name_by_person(session, tree.id)
out: list[dict] = []
for p in await _persons(session, tree.id):
if p.gender: # only fill in what's missing
continue
nm = names.get(p.id)
if nm is None:
continue
proposed = name2sex.get(gedcom._norm(nm.given, nm.surname))
if proposed:
out.append({"person_id": str(p.id), "name": _display(nm), "proposed_gender": proposed})
out.sort(key=lambda r: r["name"])
return out
async def guess_gender_by_name(
session: AsyncSession, *, actor: User, tree: Tree
) -> list[dict]:
"""Best-guess sex from the first given name for people who don't have it set,
using the bundled name dictionary. Ambiguous/unknown names are skipped."""
await _require_editor(session, actor=actor, tree=tree)
names = await _primary_name_by_person(session, tree.id)
out: list[dict] = []
for p in await _persons(session, tree.id):
if p.gender:
continue
nm = names.get(p.id)
if nm is None:
continue
proposed = guess_sex(nm.given)
if proposed:
out.append({"person_id": str(p.id), "name": _display(nm), "proposed_gender": proposed})
out.sort(key=lambda r: r["name"])
return out
async def guess_gender_by_spouse(
session: AsyncSession, *, actor: User, tree: Tree
) -> list[dict]:
"""Infer the sex of a person who has none set from a partner whose sex IS set
(couples in a tree are opposite-sex in practice — e.g. a confirmed-male
husband implies a female wife). People whose known partners disagree are
ambiguous and skipped; the result is a preview to review, not an auto-write."""
await _require_editor(session, actor=actor, tree=tree)
persons = await _persons(session, tree.id)
gender = {p.id: p.gender for p in persons}
names = await _primary_name_by_person(session, tree.id)
rels = (
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
Relationship.type == RelationshipType.partnership,
)
)
).scalars().all()
opp = {"male": "female", "female": "male"}
proposals: dict[uuid.UUID, set[str]] = {}
for r in rels:
for me_id, other_id in (
(r.person_from_id, r.person_to_id),
(r.person_to_id, r.person_from_id),
):
if gender.get(me_id):
continue # this person already has a sex
other_sex = str(gender.get(other_id) or "")
if other_sex in opp:
proposals.setdefault(me_id, set()).add(opp[other_sex])
out: list[dict] = []
for pid, sexes in proposals.items():
if len(sexes) != 1:
continue # partners of differing known sex → ambiguous
nm = names.get(pid)
if nm is None:
continue
out.append(
{"person_id": str(pid), "name": _display(nm), "proposed_gender": next(iter(sexes))}
)
out.sort(key=lambda r: r["name"])
return out
async def apply_gender(
session: AsyncSession, *, actor: User, tree: Tree, updates: list[dict]
) -> int:
"""updates: [{person_id, gender}]."""
await _require_editor(session, actor=actor, tree=tree)
wanted = {uuid.UUID(str(u["person_id"])): u["gender"] for u in updates if u.get("gender")}
persons = (
await session.execute(
select(Person).where(
Person.tree_id == tree.id,
Person.deleted_at.is_(None),
Person.id.in_(wanted.keys()),
)
)
).scalars().all()
for p in persons:
p.gender = wanted[p.id]
record_audit(
session,
action="cleanup_gender",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"count": len(persons)},
)
await session.commit()
return len(persons)
# ---- 3. Flag malformed names for review --------------------------------------------
_YEAR_RE = re.compile(r"\b\d{3,4}\b")
def _name_issue(n: Name) -> str | None:
given = (n.given or "").strip()
surname = (n.surname or "").strip()
if _YEAR_RE.search(surname) or re.search(r"\d", surname):
return "date_in_surname"
if re.search(r"\d", given):
return "date_in_given"
# A given name with many tokens often means a maiden+married name was packed
# in (e.g. "Mary Smith Jones") — surface it for a human to split.
if surname == "" and len(given.split()) >= 2:
return "no_surname"
if len(given.split()) >= 3:
return "packed_given"
return None
async def preview_names(session: AsyncSession, *, actor: User, tree: Tree) -> list[dict]:
await _require_editor(session, actor=actor, tree=tree)
names = (
await session.execute(
select(Name).where(Name.tree_id == tree.id, Name.deleted_at.is_(None))
)
).scalars().all()
out: list[dict] = []
for n in names:
issue = _name_issue(n)
if issue:
out.append({
"name_id": str(n.id),
"person_id": str(n.person_id),
"given": n.given,
"surname": n.surname,
"issue": issue,
})
return out
async def apply_names(
session: AsyncSession, *, actor: User, tree: Tree, edits: list[dict]
) -> int:
"""edits: [{name_id, given, surname}] — the user's corrected values."""
await _require_editor(session, actor=actor, tree=tree)
by_id = {uuid.UUID(str(e["name_id"])): e for e in edits}
rows = (
await session.execute(
select(Name).where(
Name.tree_id == tree.id,
Name.deleted_at.is_(None),
Name.id.in_(by_id.keys()),
)
)
).scalars().all()
if len(rows) != len(by_id):
raise NotFound("one or more names not found in this tree")
for n in rows:
e = by_id[n.id]
n.given = (e.get("given") or "").strip() or None
n.surname = (e.get("surname") or "").strip() or None
n.display_name = None # rebuild from parts
record_audit(
session,
action="cleanup_names",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"count": len(rows)},
)
await session.commit()
return len(rows)
+221
View File
@@ -0,0 +1,221 @@
"""Event service. Writes require editor rights; reads go through the privacy
engine. Every event has exactly one subject — a Person or a partnership."""
import uuid
from datetime import date
from sqlalchemy import or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import RelationshipType
from app.models.event import Event
from app.models.person import Person
from app.models.place import Place
from app.models.relationship import Relationship
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Conflict, Forbidden, NotFound
async def _belongs_to_tree(
session: AsyncSession, model: type, id_: uuid.UUID, tree_id: uuid.UUID
) -> bool:
row = (
await session.execute(
select(model.id).where(
model.id == id_, model.tree_id == tree_id, model.deleted_at.is_(None)
)
)
).scalar_one_or_none()
return row is not None
async def create_event(
session: AsyncSession,
*,
actor: User,
tree: Tree,
event_type: str,
person_id: uuid.UUID | None = None,
relationship_id: uuid.UUID | None = None,
place_id: uuid.UUID | None = None,
date_value: str | None = None,
date_start: date | None = None,
date_end: date | None = None,
date_precision: str | None = None,
calendar: str = "gregorian",
detail: str | None = None,
notes: str | None = None,
) -> Event:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
if bool(person_id) == bool(relationship_id):
raise Conflict("an event needs exactly one subject: person_id or relationship_id")
if person_id and not await _belongs_to_tree(session, Person, person_id, tree.id):
raise NotFound("person not found in this tree")
if relationship_id and not await _belongs_to_tree(
session, Relationship, relationship_id, tree.id
):
raise NotFound("relationship not found in this tree")
if place_id and not await _belongs_to_tree(session, Place, place_id, tree.id):
raise NotFound("place not found in this tree")
event = Event(
tree_id=tree.id,
event_type=event_type,
person_id=person_id,
relationship_id=relationship_id,
place_id=place_id,
date_value=date_value,
date_start=date_start,
date_end=date_end,
date_precision=date_precision,
calendar=calendar,
detail=detail,
notes=notes,
)
session.add(event)
await session.flush()
record_audit(
session,
action="create",
entity_type="Event",
entity_id=event.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"event_type": event_type, "person_id": str(person_id) if person_id else None},
)
await session.commit()
await session.refresh(event)
return event
async def list_events(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree
) -> list[Event]:
"""All events in the tree — lets the family view compute birth/death years."""
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members get the redacted projection (no living-person dates).
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_events(
session, viewer_id=viewer_id, tree=tree
)
stmt = (
select(Event)
.where(Event.tree_id == tree.id, Event.deleted_at.is_(None))
.order_by(Event.date_start.nulls_last(), Event.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def list_events_for_person(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, person_id: uuid.UUID
) -> list[Event]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members only see a full-visibility person's events (redacted → none).
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_person_events(
session, viewer_id=viewer_id, tree=tree, person_id=person_id
)
# Member view: this person's own events PLUS their partnership events (which
# live on the relationship and show on both partners). Returning both here
# means the person page doesn't have to load every event in the tree.
partner_rel_ids = (
select(Relationship.id)
.where(
Relationship.tree_id == tree.id,
Relationship.type == RelationshipType.partnership,
Relationship.deleted_at.is_(None),
or_(
Relationship.person_from_id == person_id,
Relationship.person_to_id == person_id,
),
)
)
stmt = (
select(Event)
.where(
Event.tree_id == tree.id,
Event.deleted_at.is_(None),
or_(
Event.person_id == person_id,
Event.relationship_id.in_(partner_rel_ids),
),
)
.order_by(Event.date_start.nulls_last(), Event.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def update_event(
session: AsyncSession,
*,
actor: User,
tree: Tree,
event_id: uuid.UUID,
changes: dict,
) -> Event:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
event = (
await session.execute(
select(Event).where(
Event.id == event_id, Event.tree_id == tree.id, Event.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if event is None:
raise NotFound("event not found")
if "place_id" in changes and changes["place_id"] is not None:
if not await _belongs_to_tree(session, Place, changes["place_id"], tree.id):
raise NotFound("place not found in this tree")
for key, value in changes.items():
setattr(event, key, value)
record_audit(
session,
action="update",
entity_type="Event",
entity_id=event.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(event)
return event
async def delete_event(
session: AsyncSession, *, actor: User, tree: Tree, event_id: uuid.UUID
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
event = (
await session.execute(
select(Event).where(
Event.id == event_id, Event.tree_id == tree.id, Event.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if event is None:
raise NotFound("event not found")
from datetime import UTC, datetime
event.deleted_at = datetime.now(UTC)
record_audit(
session,
action="delete",
entity_type="Event",
entity_id=event.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
+841
View File
@@ -0,0 +1,841 @@
"""GEDCOM import/export.
A pragmatic parser + mapper for the common subset of GEDCOM (5.5.1 / 7 share
the line grammar): INDI, FAM, SOUR. Import maps records into a tree and returns
a mapping report (counts + unmapped tags); export serializes the tree back to
GEDCOM. Runs inline for now — large files should move to the worker later.
Import is duplicate-aware: ``preview_gedcom`` reports incoming people that look
like existing ones, and ``import_gedcom`` applies a per-record resolution
(new / skip / merge / overwrite). Names carry their GEDCOM type (a married name
imports as a typed alternate, not a second primary).
"""
import re
import uuid
from collections import defaultdict
from datetime import UTC, date, datetime
from difflib import SequenceMatcher
from sqlalchemy import or_, select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import ParentChildQualifier, RelationshipType
from app.models.event import Event
from app.models.person import Name, Person
from app.models.place import Place
from app.models.relationship import Relationship
from app.models.source import Citation, Source
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Forbidden
# GEDCOM event tag -> our event_type (INDI-level).
INDI_EVENTS = {
"BIRT": "birth", "DEAT": "death", "BAPM": "baptism", "CHR": "christening",
"BURI": "burial", "CREM": "cremation", "RESI": "residence", "CENS": "census",
"IMMI": "immigration", "EMIG": "emigration", "OCCU": "occupation",
"EDUC": "education", "GRAD": "graduation", "RETI": "retirement",
"NATU": "naturalization", "BAPL": "baptism", "RELI": "religion",
}
# INDI attribute tags whose line VALUE is the fact (no date), stored in detail.
VALUE_EVENTS = {"RELI", "OCCU", "EDUC"}
# INDI sub-tags consumed elsewhere or intentionally ignored (not "unmapped").
INDI_SKIP_TAGS = {
"NAME", "SEX", "SOUR", "FAMC", "FAMS", "CHAN", "OBJE", "_UID", "_MARNM", "NOTE",
}
# FAM-level events.
FAM_EVENTS = {"MARR": "marriage", "DIV": "divorce", "ENGA": "engagement"}
EVENT_TO_GED = {v: k for k, v in {**INDI_EVENTS, **FAM_EVENTS}.items()}
# GEDCOM NAME TYPE (or _MARNM-derived) -> our Name.name_type vocabulary.
NAME_TYPE_MAP = {
"birth": "birth", "maiden": "birth", "married": "married",
"aka": "alias", "also known as": "alias", "nickname": "nickname",
"religious": "religious", "immigrant": "immigration",
"immigration": "immigration", "professional": "alias", "other": "alias",
}
# Our type -> GEDCOM TYPE on export (birth is the default; emit nothing).
EXPORT_TYPE_MAP = {
"married": "married", "alias": "aka", "nickname": "nickname",
"religious": "religious", "immigration": "immigrant",
}
class GedcomNode:
__slots__ = ("level", "tag", "value", "xref", "children")
def __init__(self, level: int, tag: str, value: str = "", xref: str | None = None):
self.level = level
self.tag = tag
self.value = value
self.xref = xref
self.children: list[GedcomNode] = []
def first(self, tag: str) -> "GedcomNode | None":
return next((c for c in self.children if c.tag == tag), None)
def all(self, tag: str) -> list["GedcomNode"]:
return [c for c in self.children if c.tag == tag]
def text(self, tag: str, default: str | None = None) -> str | None:
n = self.first(tag)
return n.value if n is not None else default
def parse_records(text: str) -> list[GedcomNode]:
roots: list[GedcomNode] = []
stack: list[GedcomNode] = []
for raw in text.replace("\r\n", "\n").replace("\r", "\n").split("\n"):
line = raw.lstrip("").rstrip()
if not line.strip():
continue
parts = line.split(" ", 1)
try:
level = int(parts[0])
except ValueError:
continue
rest = parts[1] if len(parts) > 1 else ""
xref: str | None = None
if rest.startswith("@"):
end = rest.find("@", 1)
if end != -1:
xref = rest[: end + 1]
rest = rest[end + 1:].strip()
tparts = rest.split(" ", 1)
tag = tparts[0]
value = tparts[1] if len(tparts) > 1 else ""
while stack and stack[-1].level >= level:
stack.pop()
parent = stack[-1] if stack else None
if tag in ("CONC", "CONT") and parent is not None:
parent.value += ("" if tag == "CONC" else "\n") + value
continue
node = GedcomNode(level, tag, value, xref)
if parent is None:
roots.append(node)
else:
parent.children.append(node)
stack.append(node)
return roots
def _parse_name(value: str) -> tuple[str | None, str | None]:
if "/" in value:
given, _, rest = value.partition("/")
surname = rest.split("/", 1)[0]
return given.strip() or None, surname.strip() or None
return value.strip() or None, None
def _parse_marnm(value: str, base_given: str | None) -> tuple[str | None, str | None]:
"""A _MARNM value is sometimes a full name ("Jane /Smith/") and sometimes
just the married surname ("Smith"). Keep the given name from the base name
in the latter case."""
v = (value or "").strip()
if "/" in v:
g, s = _parse_name(v)
return (g or base_given), s
return base_given, (v or None)
def _extract_names(rec: GedcomNode) -> list[dict]:
"""All names for an INDI, typed. Multiple NAME records (each with an optional
TYPE) plus any _MARNM (married name) subtags become separate Name rows. The
first birth/maiden name is primary."""
out: list[dict] = []
for nm in rec.all("NAME"):
g, s = _parse_name(nm.value)
t = (nm.text("TYPE") or "").strip().lower()
ntype = NAME_TYPE_MAP.get(t, t or "birth")
out.append({"type": ntype, "given": g, "surname": s, "display": nm.value or None,
"nickname": nm.text("NICK")})
for mar in nm.all("_MARNM"):
mg, ms = _parse_marnm(mar.value, g)
out.append({"type": "married", "given": mg, "surname": ms,
"display": mar.value or None, "nickname": None})
for mar in rec.all("_MARNM"):
base_g = out[0]["given"] if out else None
mg, ms = _parse_marnm(mar.value, base_g)
out.append({"type": "married", "given": mg, "surname": ms,
"display": mar.value or None, "nickname": None})
if not out:
return out
primary_idx = next((i for i, n in enumerate(out) if n["type"] == "birth"), 0)
for i, n in enumerate(out):
n["is_primary"] = i == primary_idx
n["sort"] = i
return out
def _norm(given: str | None, surname: str | None) -> str:
return re.sub(r"\s+", " ", f"{given or ''} {surname or ''}".strip().lower())
def _year(date_value: str | None) -> str | None:
if not date_value:
return None
m = re.search(r"\b(\d{3,4})\b", date_value)
return m.group(1) if m else None
def _date_start(date_value: str | None) -> date | None:
y = _year(date_value)
if not y:
return None
try:
return date(int(y), 1, 1)
except ValueError:
return None
def _sex(value: str | None) -> str | None:
if not value:
return None
v = value.strip().upper()
return {"M": "male", "F": "female"}.get(v, value.strip().lower() or None)
def _notes_text(rec: GedcomNode) -> str | None:
"""Join an INDI's NOTE lines (which pack confidence / findagrave / fs_pid /
free text) into the person's notes field."""
vals = [n.value.strip() for n in rec.all("NOTE") if n.value and n.value.strip()]
return "\n".join(vals) or None
def _person_summary(rec: GedcomNode) -> dict:
"""Display name + birth year for an incoming INDI, for duplicate matching."""
names = _extract_names(rec)
primary = next((n for n in names if n.get("is_primary")), names[0] if names else None)
g = primary["given"] if primary else None
s = primary["surname"] if primary else None
disp = " ".join(x for x in (g, s) if x)
if not disp and primary:
disp = primary.get("display") or ""
birth = rec.first("BIRT")
year = _year(birth.text("DATE")) if birth else None
return {"names": names, "norm": _norm(g, s), "name": disp or "(no name)", "year": year}
async def _build_existing_index(session: AsyncSession, tree: Tree) -> list[dict]:
"""Existing (non-deleted) people with a display name + birth year, for
matching incoming records against."""
persons = list(
(
await session.execute(
select(Person).where(Person.tree_id == tree.id, Person.deleted_at.is_(None))
)
).scalars().all()
)
names = list(
(
await session.execute(
select(Name).where(Name.tree_id == tree.id, Name.deleted_at.is_(None))
)
).scalars().all()
)
name_by_person: dict[uuid.UUID, Name] = {}
for n in sorted(names, key=lambda n: (not n.is_primary, n.sort_order)):
name_by_person.setdefault(n.person_id, n)
births = list(
(
await session.execute(
select(Event).where(
Event.tree_id == tree.id,
Event.deleted_at.is_(None),
Event.event_type == "birth",
)
)
).scalars().all()
)
year_by_person: dict[uuid.UUID, str] = {}
for e in births:
if e.person_id and e.person_id not in year_by_person:
y = str(e.date_start.year) if e.date_start else _year(e.date_value)
if y:
year_by_person[e.person_id] = y
index: list[dict] = []
for p in persons:
nm = name_by_person.get(p.id)
g = nm.given if nm else None
s = nm.surname if nm else None
disp = " ".join(x for x in (g, s) if x) or (nm.display_name if nm else None)
index.append({
"id": p.id,
"norm": _norm(g, s),
"name": disp or "(no name)",
"year": year_by_person.get(p.id),
})
return index
def _best_match(norm: str, year: str | None, index: list[dict]) -> tuple[dict | None, str | None]:
"""Closest existing person by name similarity, rejecting clear birth-year
conflicts. Returns (entry, "high"|"medium") or (None, None)."""
if not norm:
return None, None
best: dict | None = None
best_r = 0.0
for e in index:
if not e["norm"]:
continue
r = SequenceMatcher(None, norm, e["norm"]).ratio()
if r < 0.88:
continue
if year and e["year"] and abs(int(year) - int(e["year"])) > 1:
continue # same-ish name but different birth year — not a duplicate
if r > best_r:
best_r = r
best = e
if best is None:
return None, None
year_match = bool(year and best["year"] and abs(int(year) - int(best["year"])) <= 1)
both_unknown = not year and not best["year"]
score = "high" if best_r >= 0.93 and (year_match or both_unknown) else "medium"
return best, score
def _relkey(rtype: RelationshipType, a: uuid.UUID, b: uuid.UUID) -> tuple:
if rtype == RelationshipType.parent_child:
return ("pc", str(a), str(b))
return (rtype.value, *sorted([str(a), str(b)]))
def _count_incoming(roots: list[GedcomNode]) -> tuple[dict, list[str]]:
counts: dict[str, int] = defaultdict(int)
unmapped: set[str] = set()
for rec in roots:
if rec.tag == "INDI" and rec.xref:
counts["persons"] += 1
counts["names"] += len(_extract_names(rec))
for child in rec.children:
if child.tag in INDI_EVENTS:
counts["events"] += 1
elif child.tag not in INDI_SKIP_TAGS:
unmapped.add(child.tag)
elif rec.tag == "FAM":
counts["families"] += 1
for child in rec.children:
if child.tag in FAM_EVENTS:
counts["events"] += 1
elif rec.tag == "SOUR" and rec.xref:
counts["sources"] += 1
return dict(counts), sorted(unmapped)
async def preview_gedcom(session: AsyncSession, *, actor: User, tree: Tree, text: str) -> dict:
"""Dry run: what would import, and which incoming people look like existing
ones. No writes."""
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
roots = parse_records(text)
counts, unmapped = _count_incoming(roots)
index = await _build_existing_index(session, tree)
duplicates: list[dict] = []
for rec in roots:
if rec.tag != "INDI" or not rec.xref:
continue
summ = _person_summary(rec)
entry, score = _best_match(summ["norm"], summ["year"], index)
if entry is None:
continue
duplicates.append({
"xref": rec.xref,
"incoming_name": summ["name"],
"incoming_birth_year": summ["year"],
"existing_person_id": entry["id"],
"existing_name": entry["name"],
"existing_birth_year": entry["year"],
"score": score,
})
return {"counts": counts, "potential_duplicates": duplicates, "unmapped_tags": unmapped}
async def import_gedcom(
session: AsyncSession,
*,
actor: User,
tree: Tree,
text: str,
default_action: str = "new",
resolutions: dict | None = None,
) -> dict:
"""Import records. ``default_action`` (new|skip|merge|overwrite) applies to
incoming people that match an existing one; ``resolutions`` overrides it per
GEDCOM xref ({xref: {action, target_id}}). 'skip' links families to the
existing person but copies nothing; 'merge' also copies the incoming names
(as alternates), events and citations onto them; 'overwrite' deletes the
existing person and imports the incoming one fresh."""
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
resolutions = resolutions or {}
roots = parse_records(text)
counts: dict[str, int] = defaultdict(int)
unmapped: set[str] = set()
place_cache: dict[str, uuid.UUID] = {}
source_map: dict[str, uuid.UUID] = {}
person_map: dict[str, uuid.UUID] = {}
now = datetime.now(UTC)
index = await _build_existing_index(session, tree)
# Pre-load existing relationship keys so a merge doesn't create dup edges.
existing_rels = list(
(
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None)
)
)
).scalars().all()
)
rel_keys = {_relkey(r.type, r.person_from_id, r.person_to_id) for r in existing_rels}
def add_relationship(
rtype: RelationshipType, a: uuid.UUID, b: uuid.UUID, **kw
) -> Relationship | None:
key = _relkey(rtype, a, b)
if key in rel_keys:
return None
rel = Relationship(tree_id=tree.id, type=rtype, person_from_id=a, person_to_id=b, **kw)
session.add(rel)
rel_keys.add(key)
counts["relationships"] += 1
return rel
async def place_id(name: str | None) -> uuid.UUID | None:
if not name:
return None
if name in place_cache:
return place_cache[name]
p = Place(tree_id=tree.id, name=name)
session.add(p)
await session.flush()
place_cache[name] = p.id
counts["places"] += 1
return p.id
# Sources first (so citations can reference them).
for rec in roots:
if rec.tag == "SOUR" and rec.xref:
src = Source(
tree_id=tree.id,
title=rec.text("TITL") or rec.text("ABBR") or "Untitled source",
author=rec.text("AUTH"),
publication_info=rec.text("PUBL"),
citation_text=rec.text("TEXT"),
)
session.add(src)
await session.flush()
source_map[rec.xref] = src.id
counts["sources"] += 1
async def add_citations(holder: GedcomNode, **target) -> None:
for s in holder.all("SOUR"):
sid = source_map.get(s.value.strip())
if sid is None:
continue
session.add(Citation(tree_id=tree.id, source_id=sid, page=s.text("PAGE"), **target))
counts["citations"] += 1
def add_names(person_id: uuid.UUID, names: list[dict], *, set_primary: bool) -> None:
for nd in names:
session.add(
Name(
tree_id=tree.id,
person_id=person_id,
name_type=nd["type"],
given=nd["given"],
surname=nd["surname"],
nickname=nd.get("nickname"),
display_name=nd.get("display"),
is_primary=set_primary and nd.get("is_primary", False),
sort_order=nd.get("sort", 0),
)
)
counts["names"] += 1
async def add_events(rec: GedcomNode, person_id: uuid.UUID) -> None:
for child in rec.children:
if child.tag in INDI_EVENTS:
dv = child.text("DATE")
# Attribute-style facts (RELI, OCCU, EDUC) carry their value on
# the line itself; store it in detail.
detail = child.value.strip() if child.tag in VALUE_EVENTS else None
ev = Event(
tree_id=tree.id,
person_id=person_id,
event_type=INDI_EVENTS[child.tag],
date_value=dv,
date_start=_date_start(dv),
place_id=await place_id(child.text("PLAC")),
detail=detail or None,
notes=child.text("NOTE"),
)
session.add(ev)
await session.flush()
counts["events"] += 1
await add_citations(child, event_id=ev.id)
elif child.tag in INDI_SKIP_TAGS:
continue
else:
unmapped.add(child.tag)
async def soft_delete_existing(person_id: uuid.UUID) -> None:
p = (
await session.execute(
select(Person).where(Person.id == person_id, Person.deleted_at.is_(None))
)
).scalar_one_or_none()
if p is None:
return
p.deleted_at = now
rels = (
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
or_(
Relationship.person_from_id == person_id,
Relationship.person_to_id == person_id,
),
)
)
).scalars().all()
for r in rels:
r.deleted_at = now
await session.execute(
update(User).where(User.self_person_id == person_id).values(self_person_id=None)
)
# Precompute the best match per incoming xref (for default-policy resolution).
matches: dict[str, dict] = {}
for rec in roots:
if rec.tag == "INDI" and rec.xref:
summ = _person_summary(rec)
entry, _score = _best_match(summ["norm"], summ["year"], index)
if entry is not None:
matches[rec.xref] = entry
def resolve(xref: str) -> tuple[str, uuid.UUID | None]:
ov = resolutions.get(xref)
if ov:
action = ov.get("action", "new")
tid = ov.get("target_id")
target = uuid.UUID(tid) if tid else (matches[xref]["id"] if xref in matches else None)
if action in ("skip", "merge", "overwrite") and target is None:
return "new", None
return action, target
if default_action != "new" and xref in matches:
return default_action, matches[xref]["id"]
return "new", None
# Individuals.
for rec in roots:
if rec.tag != "INDI" or not rec.xref:
continue
names = _extract_names(rec)
action, target = resolve(rec.xref)
if action == "skip" and target is not None:
person_map[rec.xref] = target
counts["skipped"] += 1
continue
if action == "merge" and target is not None:
person_map[rec.xref] = target
add_names(target, names, set_primary=False)
await add_events(rec, target)
await add_citations(rec, person_id=target)
note = _notes_text(rec)
if note:
existing = (
await session.execute(select(Person).where(Person.id == target))
).scalar_one_or_none()
if existing is not None:
existing.notes = "\n".join(filter(None, [existing.notes, note]))
counts["merged"] += 1
continue
if action == "overwrite" and target is not None:
await soft_delete_existing(target)
counts["overwritten"] += 1
person = Person(tree_id=tree.id, gender=_sex(rec.text("SEX")), notes=_notes_text(rec))
session.add(person)
await session.flush()
person_map[rec.xref] = person.id
counts["persons"] += 1
add_names(person.id, names, set_primary=True)
await add_citations(rec, person_id=person.id)
await add_events(rec, person.id)
# Families -> partnerships, parent-child edges, marriage events.
for rec in roots:
if rec.tag != "FAM":
continue
counts["families"] += 1
husb = person_map.get((rec.text("HUSB") or "").strip())
wife = person_map.get((rec.text("WIFE") or "").strip())
partnership_id: uuid.UUID | None = None
if husb and wife and husb != wife:
rel = add_relationship(RelationshipType.partnership, husb, wife)
if rel is not None:
await session.flush()
partnership_id = rel.id
if partnership_id is None and husb and wife:
# Edge already existed — find it so marriage events can attach.
existing = next(
(
r for r in existing_rels
if r.type == RelationshipType.partnership
and {r.person_from_id, r.person_to_id} == {husb, wife}
),
None,
)
partnership_id = existing.id if existing else None
for fe in rec.children:
if fe.tag in FAM_EVENTS and partnership_id is not None:
dv = fe.text("DATE")
ev = Event(
tree_id=tree.id,
relationship_id=partnership_id,
event_type=FAM_EVENTS[fe.tag],
date_value=dv,
date_start=_date_start(dv),
place_id=await place_id(fe.text("PLAC")),
)
session.add(ev)
await session.flush()
counts["events"] += 1
for chil in rec.all("CHIL"):
cp = person_map.get(chil.value.strip())
if cp is None:
continue
for parent in (husb, wife):
if parent and parent != cp:
add_relationship(
RelationshipType.parent_child,
parent,
cp,
qualifier=ParentChildQualifier.biological,
)
record_audit(
session,
action="import",
entity_type="Gedcom",
tree_id=tree.id,
actor_user_id=actor.id,
after=dict(counts),
)
await session.commit()
return {"counts": dict(counts), "unmapped_tags": sorted(unmapped)}
def _ged_date(value: str | None) -> str | None:
return value.strip() if value else None
async def export_gedcom(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> str:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
persons = list(
(
await session.execute(
select(Person).where(Person.tree_id == tree.id, Person.deleted_at.is_(None))
)
).scalars().all()
)
names = list(
(
await session.execute(
select(Name).where(Name.tree_id == tree.id, Name.deleted_at.is_(None))
)
).scalars().all()
)
events = list(
(
await session.execute(
select(Event).where(Event.tree_id == tree.id, Event.deleted_at.is_(None))
)
).scalars().all()
)
rels = list(
(
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None)
)
)
).scalars().all()
)
sources = list(
(
await session.execute(
select(Source).where(Source.tree_id == tree.id, Source.deleted_at.is_(None))
)
).scalars().all()
)
places = {
p.id: p
for p in (
await session.execute(select(Place).where(Place.tree_id == tree.id))
).scalars().all()
}
citations = list(
(
await session.execute(
select(Citation).where(
Citation.tree_id == tree.id, Citation.deleted_at.is_(None)
)
)
).scalars().all()
)
pxref = {p.id: f"@I{i + 1}@" for i, p in enumerate(persons)}
gender_by_id = {p.id: p.gender for p in persons}
sxref = {s.id: f"@S{i + 1}@" for i, s in enumerate(sources)}
# Citations grouped by the fact they sit on, so each fact can emit its SOUR
# links (dropping these is the round-trip data loss this fixes). Skip any
# whose source didn't export.
cite_by_person: dict[uuid.UUID, list[Citation]] = defaultdict(list)
cite_by_name: dict[uuid.UUID, list[Citation]] = defaultdict(list)
cite_by_event: dict[uuid.UUID, list[Citation]] = defaultdict(list)
cite_by_rel: dict[uuid.UUID, list[Citation]] = defaultdict(list)
for c in citations:
if c.source_id not in sxref:
continue
if c.person_id:
cite_by_person[c.person_id].append(c)
elif c.event_id:
cite_by_event[c.event_id].append(c)
elif c.name_id:
cite_by_name[c.name_id].append(c)
elif c.relationship_id:
cite_by_rel[c.relationship_id].append(c)
def cite_lines(cites: list[Citation], depth: int) -> list[str]:
lines: list[str] = []
for c in cites:
lines.append(f"{depth} SOUR {sxref[c.source_id]}")
if c.page:
lines.append(f"{depth + 1} PAGE {c.page}")
return lines
names_by_person: dict[uuid.UUID, list[Name]] = defaultdict(list)
for n in sorted(names, key=lambda n: (n.sort_order, not n.is_primary)):
names_by_person[n.person_id].append(n)
events_by_person: dict[uuid.UUID, list[Event]] = defaultdict(list)
events_by_rel: dict[uuid.UUID, list[Event]] = defaultdict(list)
for e in events:
if e.person_id:
events_by_person[e.person_id].append(e)
elif e.relationship_id:
events_by_rel[e.relationship_id].append(e)
# Build families from parent-child + partnership edges (group by parent set).
parents_of: dict[uuid.UUID, set[uuid.UUID]] = defaultdict(set)
for r in rels:
if r.type == RelationshipType.parent_child:
parents_of[r.person_to_id].add(r.person_from_id)
fams: dict[frozenset, dict] = {}
for child, ps in parents_of.items():
key = frozenset(ps)
fams.setdefault(key, {"parents": set(ps), "children": [], "rel_id": None})
fams[key]["children"].append(child)
for r in rels:
if r.type == RelationshipType.partnership:
key = frozenset({r.person_from_id, r.person_to_id})
fam = fams.setdefault(
key,
{"parents": {r.person_from_id, r.person_to_id}, "children": [], "rel_id": None},
)
fam["rel_id"] = r.id
fam_list = list(fams.values())
fxref = {id(f): f"@F{i + 1}@" for i, f in enumerate(fam_list)}
# person -> the families they are a spouse in / a child in
spouse_fams: dict[uuid.UUID, list[str]] = defaultdict(list)
child_fams: dict[uuid.UUID, str] = {}
for f in fam_list:
x = fxref[id(f)]
for pid in f["parents"]:
spouse_fams[pid].append(x)
for cid in f["children"]:
child_fams[cid] = x
out: list[str] = ["0 HEAD", "1 SOUR Provenance", "1 GEDC", "2 VERS 5.5.1", "1 CHAR UTF-8"]
for p in persons:
out.append(f"0 {pxref[p.id]} INDI")
for n in names_by_person.get(p.id, []):
display = n.display_name or f"{n.given or ''} /{n.surname or ''}/".strip()
out.append(f"1 NAME {display}")
ged_type = EXPORT_TYPE_MAP.get(n.name_type)
if ged_type:
out.append(f"2 TYPE {ged_type}")
out += cite_lines(cite_by_name.get(n.id, []), 2)
sex = {"male": "M", "female": "F"}.get(p.gender or "")
if sex:
out.append(f"1 SEX {sex}")
for e in events_by_person.get(p.id, []):
tag = EVENT_TO_GED.get(e.event_type)
if not tag:
continue
out.append(f"1 {tag}")
if _ged_date(e.date_value):
out.append(f"2 DATE {e.date_value}")
if e.place_id and e.place_id in places:
out.append(f"2 PLAC {places[e.place_id].name}")
out += cite_lines(cite_by_event.get(e.id, []), 2)
out += cite_lines(cite_by_person.get(p.id, []), 1)
if p.id in child_fams:
out.append(f"1 FAMC {child_fams[p.id]}")
for x in spouse_fams.get(p.id, []):
out.append(f"1 FAMS {x}")
for f in fam_list:
x = fxref[id(f)]
out.append(f"0 {x} FAM")
ps = list(f["parents"])
# HUSB/WIFE by recorded gender where possible.
males = [pid for pid in ps if gender_by_id.get(pid) == "male"]
females = [pid for pid in ps if gender_by_id.get(pid) == "female"]
husb = males[0] if males else (ps[0] if ps else None)
wife = females[0] if females else next((pid for pid in ps if pid != husb), None)
if husb:
out.append(f"1 HUSB {pxref[husb]}")
if wife:
out.append(f"1 WIFE {pxref[wife]}")
for cid in f["children"]:
out.append(f"1 CHIL {pxref[cid]}")
if f["rel_id"]:
for e in events_by_rel.get(f["rel_id"], []):
tag = EVENT_TO_GED.get(e.event_type)
if not tag:
continue
out.append(f"1 {tag}")
if _ged_date(e.date_value):
out.append(f"2 DATE {e.date_value}")
out += cite_lines(cite_by_event.get(e.id, []), 2)
out += cite_lines(cite_by_rel.get(f["rel_id"], []), 1)
for s in sources:
out.append(f"0 {sxref[s.id]} SOUR")
if s.title:
out.append(f"1 TITL {s.title}")
if s.author:
out.append(f"1 AUTH {s.author}")
if s.publication_info:
out.append(f"1 PUBL {s.publication_info}")
out.append("0 TRLR")
return "\n".join(out) + "\n"
+170
View File
@@ -0,0 +1,170 @@
"""Media service. Bytes go to the ObjectStore; a metadata row goes to the DB.
Writes require editor rights; reads go through the privacy engine."""
import hashlib
import uuid
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.integrations.objectstore.base import ObjectStore
from app.models.media import Media
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound
async def upload_media(
session: AsyncSession,
store: ObjectStore,
*,
actor: User,
tree: Tree,
data: bytes,
filename: str,
content_type: str,
title: str | None = None,
person_id: uuid.UUID | None = None,
event_id: uuid.UUID | None = None,
source_id: uuid.UUID | None = None,
) -> Media:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
media_id = uuid.uuid4()
key = f"{tree.id}/{media_id}/{filename}"
await store.ensure_bucket()
await store.put_object(key=key, data=data, content_type=content_type)
media = Media(
id=media_id,
tree_id=tree.id,
uploader_id=actor.id,
storage_key=key,
original_filename=filename,
content_type=content_type,
byte_size=len(data),
checksum_sha256=hashlib.sha256(data).hexdigest(),
title=title,
person_id=person_id,
event_id=event_id,
source_id=source_id,
)
session.add(media)
await session.flush()
record_audit(
session,
action="create",
entity_type="Media",
entity_id=media.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"filename": filename, "bytes": len(data)},
)
await session.commit()
await session.refresh(media)
return media
async def list_media(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> list[Media]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members only see media of a FULL-visibility person (no living-person photos).
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_media(
session, viewer_id=viewer_id, tree=tree
)
stmt = (
select(Media)
.where(Media.tree_id == tree.id, Media.deleted_at.is_(None))
.order_by(Media.created_at.desc())
)
return list((await session.execute(stmt)).scalars().all())
async def get_media(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, media_id: uuid.UUID
) -> Media:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
media = (
await session.execute(
select(Media).where(
Media.id == media_id, Media.tree_id == tree.id, Media.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if media is None:
raise NotFound("media not found")
# Non-members may only see/download media of a FULL-visibility person. 404
# (not 403) so the item's existence isn't revealed. This gates media_content.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
if not await public_view_service.can_view_media(
session, viewer_id=viewer_id, tree=tree, media=media
):
raise NotFound("media not found")
return media
async def update_media(
session: AsyncSession, *, actor: User, tree: Tree, media_id: uuid.UUID, changes: dict
) -> Media:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
media = (
await session.execute(
select(Media).where(
Media.id == media_id, Media.tree_id == tree.id, Media.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if media is None:
raise NotFound("media not found")
for key in {"title", "person_id", "event_id", "source_id"} & changes.keys():
setattr(media, key, changes[key])
record_audit(
session,
action="update",
entity_type="Media",
entity_id=media.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(media)
return media
async def delete_media(
session: AsyncSession, *, actor: User, tree: Tree, media_id: uuid.UUID
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
media = (
await session.execute(
select(Media).where(
Media.id == media_id, Media.tree_id == tree.id, Media.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if media is None:
raise NotFound("media not found")
# Soft delete the row; the object is removed by the worker's purge job.
media.deleted_at = datetime.now(UTC)
record_audit(
session,
action="delete",
entity_type="Media",
entity_id=media.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
+156
View File
@@ -0,0 +1,156 @@
"""Tree membership management: list / add / change-role / remove.
Only an owner may change membership. A tree must always keep at least one owner.
The member list (which exposes user emails) is visible only to members — never
to a non-member viewing a public/unlisted tree.
"""
import uuid
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import MembershipRole
from app.models.tree import Tree, TreeMembership
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Conflict, Forbidden, NotFound
async def _require_owner(session: AsyncSession, *, actor_id: uuid.UUID, tree: Tree) -> None:
if await privacy.get_membership_role(session, actor_id, tree.id) is not MembershipRole.owner:
raise Forbidden("only the owner can manage members")
async def _owner_count(session: AsyncSession, tree_id: uuid.UUID) -> int:
return (
await session.execute(
select(func.count())
.select_from(TreeMembership)
.where(TreeMembership.tree_id == tree_id, TreeMembership.role == MembershipRole.owner)
)
).scalar_one()
def _row(m: TreeMembership, u: User) -> dict:
return {
"id": m.id,
"user_id": u.id,
"email": u.email,
"display_name": u.display_name,
"role": m.role,
"created_at": m.created_at,
}
async def list_members(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> list[dict]:
# Member-only: the list exposes emails, so a non-member (even on a public
# tree) must not see it.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
raise Forbidden("only members can see the member list")
rows = (
await session.execute(
select(TreeMembership, User)
.join(User, User.id == TreeMembership.user_id)
.where(TreeMembership.tree_id == tree.id)
.order_by(TreeMembership.created_at)
)
).all()
return [_row(m, u) for m, u in rows]
async def add_member(
session: AsyncSession, *, actor: User, tree: Tree, email: str, role: MembershipRole
) -> dict:
await _require_owner(session, actor_id=actor.id, tree=tree)
user = (
await session.execute(
select(User).where(User.email == email, User.deleted_at.is_(None))
)
).scalar_one_or_none()
if user is None:
raise NotFound("no user with that email on this instance")
if await privacy.get_membership_role(session, user.id, tree.id) is not None:
raise Conflict("that user is already a member")
m = TreeMembership(tree_id=tree.id, user_id=user.id, role=role)
session.add(m)
record_audit(
session,
action="add_member",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"user_id": str(user.id), "role": role.value},
)
await session.commit()
await session.refresh(m)
return _row(m, user)
async def _get_membership(
session: AsyncSession, tree: Tree, membership_id: uuid.UUID
) -> TreeMembership:
m = (
await session.execute(
select(TreeMembership).where(
TreeMembership.id == membership_id, TreeMembership.tree_id == tree.id
)
)
).scalar_one_or_none()
if m is None:
raise NotFound("member not found")
return m
async def update_member_role(
session: AsyncSession,
*,
actor: User,
tree: Tree,
membership_id: uuid.UUID,
role: MembershipRole,
) -> dict:
await _require_owner(session, actor_id=actor.id, tree=tree)
m = await _get_membership(session, tree, membership_id)
if (
m.role == MembershipRole.owner
and role != MembershipRole.owner
and await _owner_count(session, tree.id) <= 1
):
raise Conflict("a tree must keep at least one owner")
m.role = role
record_audit(
session,
action="update_member",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"membership_id": str(m.id), "role": role.value},
)
await session.commit()
await session.refresh(m)
u = (await session.execute(select(User).where(User.id == m.user_id))).scalar_one()
return _row(m, u)
async def remove_member(
session: AsyncSession, *, actor: User, tree: Tree, membership_id: uuid.UUID
) -> None:
await _require_owner(session, actor_id=actor.id, tree=tree)
m = await _get_membership(session, tree, membership_id)
if m.role == MembershipRole.owner and await _owner_count(session, tree.id) <= 1:
raise Conflict("a tree must keep at least one owner")
await session.delete(m)
record_audit(
session,
action="remove_member",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"membership_id": str(membership_id)},
)
await session.commit()
+69
View File
@@ -0,0 +1,69 @@
"""A curated given-name -> sex lookup for best-guessing a person's sex from
their first name. Weighted toward English + German names (this codebase's first
real tree is a German-American family). Deterministic and offline — no model
needed; the Cleanup tool previews every guess before anything is applied.
Genuinely ambiguous names (Marion, Frances/Francis, Jordan, Jamie, Robin, Leslie,
Dana, …) are intentionally left out of BOTH sets so they aren't guessed — better
a human decides those than a coin flip.
"""
MALE_NAMES: set[str] = {
# English / common US
"james", "john", "robert", "michael", "william", "david", "richard", "joseph",
"thomas", "charles", "christopher", "daniel", "matthew", "anthony", "donald",
"mark", "paul", "steven", "andrew", "kenneth", "george", "joshua", "kevin",
"brian", "edward", "ronald", "timothy", "jason", "jeffrey", "gary", "ryan",
"nicholas", "eric", "stephen", "jacob", "larry", "frank", "jonathan", "scott",
"raymond", "gregory", "samuel", "benjamin", "patrick", "jack", "dennis", "jerry",
"alexander", "tyler", "henry", "douglas", "peter", "adam", "harold", "albert",
"arthur", "carl", "ralph", "roy", "eugene", "louis", "philip", "bobby", "walter",
"willie", "wayne", "fred", "howard", "ernest", "earl", "clarence", "leon",
"leonard", "lewis", "floyd", "leroy", "elmer", "homer", "orrin", "josias",
"emerson", "dale", "bernard", "vernon", "virgil", "wilbur", "russell",
"harvey", "herbert", "melvin", "lloyd", "marvin", "norman", "stanley",
# German
"hans", "karl", "wilhelm", "friedrich", "heinrich", "otto", "hermann", "gustav",
"ludwig", "ernst", "fritz", "johann", "conrad", "konrad", "reinhold", "rudolf",
"rudolph", "gerhard", "helmut", "horst", "klaus", "kurt", "dieter", "günther",
"gunther", "manfred", "siegfried", "hilgard", "christian", "august", "wolfgang",
"jürgen", "jurgen", "matthias", "lothar", "bruno", "gottlieb", "reinhard",
}
FEMALE_NAMES: set[str] = {
# English / common US
"mary", "patricia", "jennifer", "linda", "elizabeth", "barbara", "susan",
"jessica", "sarah", "karen", "nancy", "lisa", "betty", "margaret", "sandra",
"ashley", "kimberly", "emily", "donna", "michelle", "carol", "amanda", "dorothy",
"melissa", "deborah", "stephanie", "rebecca", "sharon", "laura", "cynthia",
"kathleen", "amy", "angela", "shirley", "anna", "ruth", "brenda", "pamela",
"nicole", "katherine", "virginia", "catherine", "helen", "debra", "rachel",
"carolyn", "janet", "maria", "heather", "diane", "julie", "joyce", "victoria",
"kelly", "christina", "joan", "evelyn", "judith", "megan", "alice", "frances",
"marie", "florence", "flora", "zella", "thelma", "ellen", "althea", "della",
"beatrice", "pauline", "hedwig", "florentine", "wilhelmina", "augusta", "bertha",
"gladys", "mildred", "lucille", "edith", "esther", "irene", "hazel", "doris",
"rose", "rita", "norma", "june", "lois", "marjorie",
# German
"greta", "ilse", "ursula", "gertrud", "gertrude", "frieda", "frida", "else",
"hilda", "hilde", "hildegard", "ingrid", "helga", "renate", "monika", "sieglinde",
"brigitte", "gisela", "elke", "anneliese", "waltraud", "edeltraud", "johanna",
"katharina", "margarethe", "wilhelmine", "emilie", "auguste",
}
def guess_sex(given: str | None) -> str | None:
"""Best-guess "male"/"female" from the first token of a given name, or None
if unknown/ambiguous."""
if not given:
return None
first = given.strip().split()[0].lower() if given.strip() else ""
# Strip trailing punctuation/initials like "wm." -> "wm".
first = first.strip(".,'\"")
if not first:
return None
if first in MALE_NAMES:
return "male"
if first in FEMALE_NAMES:
return "female"
return None
+215
View File
@@ -0,0 +1,215 @@
"""Name service. A Person carries one or more Name rows — a primary (typically
the birth/maiden name) plus typed alternates (married, alias, religious, …).
Exactly one name is primary at a time; it drives display everywhere. Writes
require editor rights; reads go through the tree's view check.
"""
import uuid
from datetime import UTC, datetime
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.person import Name, Person
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound
async def _get_person(session: AsyncSession, *, tree: Tree, person_id: uuid.UUID) -> Person:
person = (
await session.execute(
select(Person).where(
Person.id == person_id, Person.tree_id == tree.id, Person.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if person is None:
raise NotFound("person not found")
return person
async def _clear_primary(
session: AsyncSession, *, person_id: uuid.UUID, keep: uuid.UUID | None
) -> None:
"""Demote every other name so exactly one stays primary."""
stmt = (
update(Name)
.where(Name.person_id == person_id, Name.deleted_at.is_(None), Name.is_primary.is_(True))
.values(is_primary=False)
)
if keep is not None:
stmt = stmt.where(Name.id != keep)
await session.execute(stmt)
async def list_names(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, person_id: uuid.UUID
) -> list[Name]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
await _get_person(session, tree=tree, person_id=person_id)
# Non-members: a redacted/hidden person's real names must not leak.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_person_names(
session, viewer_id=viewer_id, tree=tree, person_id=person_id
)
stmt = (
select(Name)
.where(Name.person_id == person_id, Name.deleted_at.is_(None))
.order_by(Name.is_primary.desc(), Name.sort_order, Name.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def create_name(
session: AsyncSession,
*,
actor: User,
tree: Tree,
person_id: uuid.UUID,
name_type: str = "birth",
given: str | None = None,
surname: str | None = None,
prefix: str | None = None,
suffix: str | None = None,
nickname: str | None = None,
is_primary: bool = False,
) -> Name:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
await _get_person(session, tree=tree, person_id=person_id)
# First name for a person is always primary; otherwise honor the flag.
existing = (
await session.execute(
select(Name.id).where(Name.person_id == person_id, Name.deleted_at.is_(None))
)
).first()
primary = is_primary or existing is None
if primary:
await _clear_primary(session, person_id=person_id, keep=None)
name = Name(
tree_id=tree.id,
person_id=person_id,
name_type=name_type,
given=given,
surname=surname,
prefix=prefix,
suffix=suffix,
nickname=nickname,
is_primary=primary,
)
session.add(name)
await session.flush()
record_audit(
session,
action="create",
entity_type="Name",
entity_id=name.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"name_type": name_type, "given": given, "surname": surname},
)
await session.commit()
await session.refresh(name)
return name
_NAME_FIELDS = {"name_type", "given", "surname", "prefix", "suffix", "nickname"}
async def update_name(
session: AsyncSession,
*,
actor: User,
tree: Tree,
person_id: uuid.UUID,
name_id: uuid.UUID,
changes: dict,
) -> Name:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
name = (
await session.execute(
select(Name).where(
Name.id == name_id,
Name.person_id == person_id,
Name.tree_id == tree.id,
Name.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if name is None:
raise NotFound("name not found")
for key in _NAME_FIELDS & changes.keys():
setattr(name, key, changes[key])
if changes.get("is_primary") is True:
await _clear_primary(session, person_id=person_id, keep=name.id)
name.is_primary = True
record_audit(
session,
action="update",
entity_type="Name",
entity_id=name.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(name)
return name
async def delete_name(
session: AsyncSession,
*,
actor: User,
tree: Tree,
person_id: uuid.UUID,
name_id: uuid.UUID,
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
name = (
await session.execute(
select(Name).where(
Name.id == name_id,
Name.person_id == person_id,
Name.tree_id == tree.id,
Name.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if name is None:
raise NotFound("name not found")
name.deleted_at = datetime.now(UTC)
was_primary = name.is_primary
name.is_primary = False
record_audit(
session,
action="delete",
entity_type="Name",
entity_id=name.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
# Promote another name to primary so the person never loses their display name.
if was_primary:
nxt = (
await session.execute(
select(Name)
.where(Name.person_id == person_id, Name.deleted_at.is_(None))
.order_by(Name.sort_order, Name.created_at)
)
).scalars().first()
if nxt is not None:
nxt.is_primary = True
await session.commit()
+400 -11
View File
@@ -4,17 +4,19 @@ person through the privacy engine. Each returned Person gets a transient
""" """
import uuid import uuid
from datetime import UTC, datetime
from sqlalchemy import select from sqlalchemy import func, or_, select, update
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import PersonPrivacy from app.models.enums import PersonPrivacy, RelationshipType
from app.models.person import Name, Person from app.models.person import Name, Person
from app.models.relationship import Relationship
from app.models.tree import Tree from app.models.tree import Tree
from app.models.user import User from app.models.user import User
from app.services import privacy from app.services import privacy
from app.services.audit import record_audit from app.services.audit import record_audit
from app.services.exceptions import Forbidden from app.services.exceptions import Forbidden, NotFound
from app.services.privacy import Visibility from app.services.privacy import Visibility
@@ -24,6 +26,14 @@ def _format_name(name: Name) -> str | None:
return joined or name.display_name return joined or name.display_name
def _redact(person: Person) -> None:
"""Minimise a possibly-living person for a non-member view (transient only —
never committed)."""
person.primary_name = "Living person"
person.gender = None
person.is_living = True
async def _attach_primary_name(session: AsyncSession, person: Person) -> None: async def _attach_primary_name(session: AsyncSession, person: Person) -> None:
stmt = ( stmt = (
select(Name) select(Name)
@@ -35,6 +45,29 @@ async def _attach_primary_name(session: AsyncSession, person: Person) -> None:
person.primary_name = _format_name(name) if name is not None else None person.primary_name = _format_name(name) if name is not None else None
async def _attach_primary_names(session: AsyncSession, persons: list[Person]) -> None:
"""Batch version of ``_attach_primary_name`` — ONE query for the whole list
instead of one per person (the difference between 1 and N queries when
rendering a 2k-person tree). The global order (is_primary desc, sort_order)
matches the single-person query, so the first row seen per person is the same
name ``_attach_primary_name`` would pick."""
if not persons:
return
rows = (
await session.execute(
select(Name)
.where(Name.person_id.in_([p.id for p in persons]), Name.deleted_at.is_(None))
.order_by(Name.is_primary.desc(), Name.sort_order)
)
).scalars().all()
best: dict[uuid.UUID, Name] = {}
for n in rows:
best.setdefault(n.person_id, n)
for p in persons:
n = best.get(p.id)
p.primary_name = _format_name(n) if n is not None else None
async def create_person( async def create_person(
session: AsyncSession, session: AsyncSession,
*, *,
@@ -86,11 +119,259 @@ async def create_person(
return person return person
async def list_persons( _PERSON_FIELDS = {"gender", "is_living", "privacy", "notes"}
async def update_person(
session: AsyncSession, *, actor: User, tree: Tree, person_id: uuid.UUID, changes: dict
) -> Person:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
person = (
await session.execute(
select(Person).where(
Person.id == person_id, Person.tree_id == tree.id, Person.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if person is None:
raise NotFound("person not found")
for key in _PERSON_FIELDS & changes.keys():
setattr(person, key, changes[key])
if "given" in changes or "surname" in changes:
name = (
await session.execute(
select(Name)
.where(Name.person_id == person.id, Name.deleted_at.is_(None))
.order_by(Name.is_primary.desc(), Name.sort_order)
)
).scalars().first()
if name is None:
name = Name(tree_id=tree.id, person_id=person.id, name_type="birth", is_primary=True)
session.add(name)
if "given" in changes:
name.given = changes["given"]
if "surname" in changes:
name.surname = changes["surname"]
name.display_name = None # rebuild display from parts
record_audit(
session,
action="update",
entity_type="Person",
entity_id=person.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(person)
await _attach_primary_name(session, person)
return person
async def get_person(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, person_id: uuid.UUID
) -> Person:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
person = (
await session.execute(
select(Person).where(
Person.id == person_id,
Person.tree_id == tree.id,
Person.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if person is None:
raise NotFound("person not found")
# Run the single person through the privacy engine (redaction lands Phase 2).
vis = await privacy.person_visibility(
session, user_id=viewer_id, tree=tree, person=person
)
if vis == Visibility.hidden:
raise NotFound("person not found")
if vis == Visibility.redacted:
_redact(person)
else:
await _attach_primary_name(session, person)
return person
async def _children_of(
session: AsyncSession, *, tree_id: uuid.UUID, parent_id: uuid.UUID
) -> list[uuid.UUID]:
rows = (
await session.execute(
select(Relationship.person_to_id).where(
Relationship.tree_id == tree_id,
Relationship.deleted_at.is_(None),
Relationship.type == RelationshipType.parent_child,
Relationship.person_from_id == parent_id,
)
)
).scalars().all()
return list(rows)
async def _soft_delete_one(
session: AsyncSession, *, actor: User, tree: Tree, person: Person, now: datetime
) -> None:
"""Soft-delete a single person and the relationships touching them, so no
dangling edges are left to break the tree view."""
person.deleted_at = now
rels = (
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
or_(
Relationship.person_from_id == person.id,
Relationship.person_to_id == person.id,
),
)
)
).scalars().all()
for rel in rels:
rel.deleted_at = now
record_audit(
session,
action="delete",
entity_type="Person",
entity_id=person.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"cascaded_relationships": len(rels)},
)
async def delete_person(
session: AsyncSession,
*,
actor: User,
tree: Tree,
person_id: uuid.UUID,
cascade: bool = False,
) -> int:
"""Soft-delete a person. Always removes the relationships that touch them
(preventing dangling edges). With ``cascade=True``, recursively deletes
their descendants too — handy for pruning a bad GEDCOM import. Returns the
number of persons deleted."""
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
person = (
await session.execute(
select(Person).where(
Person.id == person_id, Person.tree_id == tree.id, Person.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if person is None:
raise NotFound("person not found")
now = datetime.now(UTC)
# Gather the set of persons to delete. For cascade, walk descendants
# breadth-first, guarding against cycles.
to_delete: list[Person] = [person]
if cascade:
seen = {person.id}
frontier = [person.id]
while frontier:
nxt: list[uuid.UUID] = []
for pid in frontier:
for child_id in await _children_of(session, tree_id=tree.id, parent_id=pid):
if child_id not in seen:
seen.add(child_id)
nxt.append(child_id)
frontier = nxt
extra_ids = [pid for pid in seen if pid != person.id]
if extra_ids:
extra = (
await session.execute(
select(Person).where(
Person.id.in_(extra_ids),
Person.tree_id == tree.id,
Person.deleted_at.is_(None),
)
)
).scalars().all()
to_delete.extend(extra)
for p in to_delete:
await _soft_delete_one(session, actor=actor, tree=tree, person=p, now=now)
# Soft delete leaves the row in place, so the DB-level "ON DELETE SET NULL"
# never fires — clear any links (account self-person, tree home person) to a
# deleted person.
deleted_ids = [p.id for p in to_delete]
await session.execute(
update(User).where(User.self_person_id.in_(deleted_ids)).values(self_person_id=None)
)
await session.execute(
update(Tree).where(Tree.home_person_id.in_(deleted_ids)).values(home_person_id=None)
)
await session.commit()
return len(to_delete)
async def restore_person(
session: AsyncSession, *, actor: User, tree: Tree, person_id: uuid.UUID
) -> Person:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
person = (
await session.execute(
select(Person).where(
Person.id == person_id, Person.tree_id == tree.id, Person.deleted_at.is_not(None)
)
)
).scalar_one_or_none()
if person is None:
raise NotFound("deleted person not found")
person.deleted_at = None
record_audit(
session,
action="restore",
entity_type="Person",
entity_id=person.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
await session.refresh(person)
await _attach_primary_name(session, person)
return person
async def list_deleted_persons(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree
) -> list[Person]: ) -> list[Person]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree): if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree") raise Forbidden("not permitted to view this tree")
stmt = (
select(Person)
.where(Person.tree_id == tree.id, Person.deleted_at.is_not(None))
.order_by(Person.deleted_at.desc())
)
persons = list((await session.execute(stmt)).scalars().all())
await _attach_primary_names(session, persons)
return persons
async def list_persons(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree
) -> list[Person]:
# Resolve the viewer's role ONCE. Members see the whole tree (full), so we
# skip the per-person privacy engine entirely and batch the name fetch — the
# difference between ~3 queries and ~3·N queries on a 2k-person tree.
role = await privacy.get_membership_role(session, viewer_id, tree.id)
if role is None and not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
stmt = ( stmt = (
select(Person) select(Person)
@@ -99,15 +380,123 @@ async def list_persons(
) )
persons = list((await session.execute(stmt)).scalars().all()) persons = list((await session.execute(stmt)).scalars().all())
if role is not None:
await _attach_primary_names(session, persons)
return persons
# Non-member on a viewable (public/unlisted/site_members) tree: redact per
# person. Names are batched for the non-redacted ones; redacted ones already
# have their display name overwritten by _redact.
visible: list[Person] = [] visible: list[Person] = []
full: list[Person] = []
for person in persons: for person in persons:
if ( vis = await privacy.person_visibility(
await privacy.person_visibility( session, user_id=viewer_id, tree=tree, person=person
session, user_id=viewer_id, tree=tree, person=person )
) if vis == Visibility.hidden:
== Visibility.hidden
):
continue continue
await _attach_primary_name(session, person) if vis == Visibility.redacted:
_redact(person)
else:
full.append(person)
visible.append(person) visible.append(person)
await _attach_primary_names(session, full)
return visible return visible
async def list_persons_by_ids(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, ids: list[uuid.UUID]
) -> list[Person]:
"""Just the named persons (privacy-filtered, names batched). Lets a page show
the names of someone's relatives without loading the whole tree."""
role = await privacy.get_membership_role(session, viewer_id, tree.id)
if role is None and not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
if not ids:
return []
persons = list(
(
await session.execute(
select(Person).where(
Person.id.in_(ids),
Person.tree_id == tree.id,
Person.deleted_at.is_(None),
)
)
).scalars().all()
)
if role is not None:
await _attach_primary_names(session, persons)
return persons
visible: list[Person] = []
full: list[Person] = []
for person in persons:
vis = await privacy.person_visibility(
session, user_id=viewer_id, tree=tree, person=person
)
if vis == Visibility.hidden:
continue
if vis == Visibility.redacted:
_redact(person)
else:
full.append(person)
visible.append(person)
await _attach_primary_names(session, full)
return visible
async def search_persons(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, query: str, limit: int = 50
) -> list[Person]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
q = query.strip()
if not q:
return []
like = f"%{q}%"
score = func.greatest(
func.similarity(func.coalesce(Name.given, ""), q),
func.similarity(func.coalesce(Name.surname, ""), q),
)
sub = (
select(Name.person_id.label("pid"), func.max(score).label("score"))
.where(
Name.tree_id == tree.id,
Name.deleted_at.is_(None),
or_(
Name.given.op("%")(q),
Name.surname.op("%")(q),
Name.given.ilike(like),
Name.surname.ilike(like),
),
)
.group_by(Name.person_id)
.order_by(func.max(score).desc())
.limit(limit)
.subquery()
)
stmt = (
select(Person)
.join(sub, sub.c.pid == Person.id)
.where(Person.tree_id == tree.id, Person.deleted_at.is_(None))
.order_by(sub.c.score.desc())
)
persons = list((await session.execute(stmt)).scalars().all())
if await privacy.get_membership_role(session, viewer_id, tree.id) is not None:
await _attach_primary_names(session, persons)
return persons
out: list[Person] = []
full: list[Person] = []
for person in persons:
vis = await privacy.person_visibility(
session, user_id=viewer_id, tree=tree, person=person
)
if vis == Visibility.hidden:
continue
if vis == Visibility.redacted:
_redact(person)
else:
full.append(person)
out.append(person)
await _attach_primary_names(session, full)
return out
+59 -3
View File
@@ -8,14 +8,20 @@ tree's visibility, the per-person override, and (Phase 2) living-person status.
import enum import enum
import uuid import uuid
from datetime import UTC, datetime
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import MembershipRole, PersonPrivacy, TreeVisibility from app.models.enums import MembershipRole, PersonPrivacy, TreeVisibility
from app.models.event import Event
from app.models.person import Person from app.models.person import Person
from app.models.tree import Tree, TreeMembership from app.models.tree import Tree, TreeMembership
# A person with no death fact whose birth is within this window (or unknown) is
# treated as possibly living and redacted from non-members (ARCHITECTURE §6).
LIVING_RECENCY_YEARS = 100
class Visibility(enum.StrEnum): class Visibility(enum.StrEnum):
full = "full" full = "full"
@@ -39,8 +45,17 @@ async def can_view_tree(session: AsyncSession, *, user_id: uuid.UUID | None, tre
if tree.deleted_at is not None: if tree.deleted_at is not None:
return False return False
if await get_membership_role(session, user_id, tree.id) is not None: if await get_membership_role(session, user_id, tree.id) is not None:
return True # members always (any role)
# Non-members. Branch on the viewer's auth state:
# public / unlisted → anyone, including anonymous (unlisted is gated only
# by knowing the link, so the API must never *list* it).
# site_members → any authenticated account on this instance.
# private → no one.
if tree.visibility in (TreeVisibility.public, TreeVisibility.unlisted):
return True return True
return tree.visibility in (TreeVisibility.public, TreeVisibility.unlisted) if tree.visibility == TreeVisibility.site_members:
return user_id is not None
return False
async def can_edit_tree(session: AsyncSession, *, user_id: uuid.UUID | None, tree: Tree) -> bool: async def can_edit_tree(session: AsyncSession, *, user_id: uuid.UUID | None, tree: Tree) -> bool:
@@ -48,15 +63,56 @@ async def can_edit_tree(session: AsyncSession, *, user_id: uuid.UUID | None, tre
return role in (MembershipRole.owner, MembershipRole.editor) return role in (MembershipRole.owner, MembershipRole.editor)
async def is_possibly_living(session: AsyncSession, person: Person) -> bool:
"""True if the person should be treated as living: explicit flag, or (absent
a death fact) a birth within the recency window or an unknown birth."""
if person.is_living is True:
return True
if person.is_living is False:
return False
death = (
await session.execute(
select(Event.id)
.where(
Event.person_id == person.id,
Event.event_type == "death",
Event.deleted_at.is_(None),
)
.limit(1)
)
).scalar_one_or_none()
if death is not None:
return False
birth = (
await session.execute(
select(Event.date_start)
.where(
Event.person_id == person.id,
Event.event_type == "birth",
Event.date_start.is_not(None),
Event.deleted_at.is_(None),
)
.order_by(Event.date_start)
.limit(1)
)
).scalar_one_or_none()
if birth is None:
return True # unknown birth → treat as possibly living
return (datetime.now(UTC).year - birth.year) < LIVING_RECENCY_YEARS
async def person_visibility( async def person_visibility(
session: AsyncSession, *, user_id: uuid.UUID | None, tree: Tree, person: Person session: AsyncSession, *, user_id: uuid.UUID | None, tree: Tree, person: Person
) -> Visibility: ) -> Visibility:
if not await can_view_tree(session, user_id=user_id, tree=tree): if not await can_view_tree(session, user_id=user_id, tree=tree):
return Visibility.hidden return Visibility.hidden
if await get_membership_role(session, user_id, tree.id) is not None: if await get_membership_role(session, user_id, tree.id) is not None:
return Visibility.full return Visibility.full # members see everyone in their tree
# Non-member viewing a public/unlisted tree: # Non-member viewing a public/unlisted tree:
if person.privacy == PersonPrivacy.private: if person.privacy == PersonPrivacy.private:
return Visibility.hidden return Visibility.hidden
# TODO(Phase 2): redact living people for non-members (ARCHITECTURE §6). if person.privacy == PersonPrivacy.public:
return Visibility.full # explicit per-person opt-in
if await is_possibly_living(session, person):
return Visibility.redacted # living people are protected by default
return Visibility.full return Visibility.full
+416
View File
@@ -0,0 +1,416 @@
"""Read-only, redaction-safe projections for the public viewing surface.
INVARIANT (CLAUDE.md #2): everything returned here has passed through
``privacy.person_visibility``. A non-member must never receive a possibly-living
person's real name, dates, alternate names, or media. The rules:
- persons : redacted (living → "Living person"); hidden dropped.
- relationships : only when BOTH endpoints are non-hidden (a link to a
redacted person is fine — the name is already hidden).
- events : only for FULL-visibility persons; partnership events only
when BOTH partners are full (a marriage date would leak a
living partner's timeline otherwise).
- names : only for FULL-visibility persons.
- media : NOT exposed yet (deferred — see docs/design/tree-visibility.md).
- citations : only when the cited fact resolves to FULL person(s).
- sources : only when they back at least one visible citation.
A tree that isn't viewable raises NotFound (never Forbidden) so the public
surface can't be used to probe whether a private tree exists.
"""
import uuid
from sqlalchemy import or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import TreeVisibility
from app.models.event import Event
from app.models.media import Media
from app.models.person import Name, Person
from app.models.relationship import Relationship
from app.models.source import Citation, Source
from app.models.tree import Tree
from app.services import privacy
from app.services.exceptions import NotFound
from app.services.person_service import (
_attach_primary_name,
_attach_primary_names,
_redact,
)
from app.services.privacy import Visibility
async def get_public_tree(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree_id: uuid.UUID
) -> Tree:
tree = (
await session.execute(
select(Tree).where(Tree.id == tree_id, Tree.deleted_at.is_(None))
)
).scalar_one_or_none()
# 404 (not 403) when not viewable: don't reveal that a private tree exists.
if tree is None or not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise NotFound("tree not found")
return tree
async def _persons(session: AsyncSession, tree: Tree) -> list[Person]:
return list(
(
await session.execute(
select(Person).where(Person.tree_id == tree.id, Person.deleted_at.is_(None))
)
)
.scalars()
.all()
)
async def _visibility_map(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, persons: list[Person]
) -> dict[uuid.UUID, Visibility]:
return {
p.id: await privacy.person_visibility(
session, user_id=viewer_id, tree=tree, person=p
)
for p in persons
}
async def list_public_persons(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
) -> list[Person]:
out: list[Person] = []
full: list[Person] = []
for p in await _persons(session, tree):
vis = await privacy.person_visibility(session, user_id=viewer_id, tree=tree, person=p)
if vis == Visibility.hidden:
continue
if vis == Visibility.redacted:
_redact(p)
else:
full.append(p)
out.append(p)
await _attach_primary_names(session, full) # one query, not one per person
return out
async def get_public_person(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
) -> Person:
person = (
await session.execute(
select(Person).where(
Person.id == person_id,
Person.tree_id == tree.id,
Person.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if person is None:
raise NotFound("person not found")
vis = await privacy.person_visibility(session, user_id=viewer_id, tree=tree, person=person)
if vis == Visibility.hidden:
raise NotFound("person not found")
if vis == Visibility.redacted:
_redact(person)
else:
await _attach_primary_name(session, person)
return person
async def _person_visibility(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
) -> Visibility | None:
person = (
await session.execute(
select(Person).where(
Person.id == person_id,
Person.tree_id == tree.id,
Person.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if person is None:
return None
return await privacy.person_visibility(session, user_id=viewer_id, tree=tree, person=person)
async def list_public_relationships(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
) -> list[Relationship]:
persons = await _persons(session, tree)
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
nonhidden = {pid for pid, v in vis.items() if v != Visibility.hidden}
rels = list(
(
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None)
)
)
)
.scalars()
.all()
)
return [
r for r in rels if r.person_from_id in nonhidden and r.person_to_id in nonhidden
]
async def list_public_events(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
) -> list[Event]:
persons = await _persons(session, tree)
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
full = {pid for pid, v in vis.items() if v == Visibility.full}
rels = {
r.id: r
for r in (
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None)
)
)
)
.scalars()
.all()
}
events = list(
(
await session.execute(
select(Event).where(Event.tree_id == tree.id, Event.deleted_at.is_(None))
)
)
.scalars()
.all()
)
out: list[Event] = []
for e in events:
if e.person_id is not None:
if e.person_id in full:
out.append(e)
elif e.relationship_id is not None:
r = rels.get(e.relationship_id)
if r is not None and r.person_from_id in full and r.person_to_id in full:
out.append(e)
return out
async def list_public_person_names(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
) -> list[Name]:
vis = await _person_visibility(session, viewer_id=viewer_id, tree=tree, person_id=person_id)
if vis is None:
raise NotFound("person not found")
if vis != Visibility.full:
return [] # redacted/hidden → no names (the real name must not leak)
return list(
(
await session.execute(
select(Name)
.where(Name.person_id == person_id, Name.deleted_at.is_(None))
.order_by(Name.is_primary.desc(), Name.sort_order, Name.created_at)
)
)
.scalars()
.all()
)
async def list_public_person_events(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
) -> list[Event]:
vis = await _person_visibility(session, viewer_id=viewer_id, tree=tree, person_id=person_id)
if vis is None:
raise NotFound("person not found")
if vis != Visibility.full:
return [] # redacted/hidden → no dates
return list(
(
await session.execute(
select(Event)
.where(
Event.person_id == person_id,
Event.tree_id == tree.id,
Event.deleted_at.is_(None),
)
.order_by(Event.date_start.nulls_last(), Event.created_at)
)
)
.scalars()
.all()
)
async def list_public_relationships_for_person(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
) -> list[Relationship]:
persons = await _persons(session, tree)
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
if vis.get(person_id) in (None, Visibility.hidden):
return []
nonhidden = {pid for pid, v in vis.items() if v != Visibility.hidden}
rels = list(
(
await session.execute(
select(Relationship).where(
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
or_(
Relationship.person_from_id == person_id,
Relationship.person_to_id == person_id,
),
)
)
)
.scalars()
.all()
)
return [r for r in rels if r.person_from_id in nonhidden and r.person_to_id in nonhidden]
async def list_public_media(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
) -> list[Media]:
"""Only media linked to a FULL-visibility person. Media without a person (or
linked only to an event/source) is not exposed to non-members — a photo of a
living person must never leak."""
persons = await _persons(session, tree)
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
full = {pid for pid, v in vis.items() if v == Visibility.full}
media = list(
(
await session.execute(
select(Media).where(Media.tree_id == tree.id, Media.deleted_at.is_(None))
)
)
.scalars()
.all()
)
return [m for m in media if m.person_id is not None and m.person_id in full]
async def can_view_media(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, media: Media
) -> bool:
"""Whether a non-member may see/download a single media item: only when it is
linked to a FULL-visibility person."""
if media.person_id is None:
return False
vis = await _person_visibility(
session, viewer_id=viewer_id, tree=tree, person_id=media.person_id
)
return vis == Visibility.full
async def _full_person_ids(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
) -> set[uuid.UUID]:
persons = await _persons(session, tree)
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
return {pid for pid, v in vis.items() if v == Visibility.full}
async def list_public_citations(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
) -> list[Citation]:
"""Only citations whose cited fact resolves to FULL-visibility person(s). A
citation on a redacted/hidden person's fact (or a partnership where either
partner isn't full) is dropped — its existence plus page/detail would leak
that the person has that sourced fact. Mirrors the events/names rule (FULL
only)."""
full = await _full_person_ids(session, viewer_id=viewer_id, tree=tree)
async def _by_id(model):
rows = (
await session.execute(
select(model).where(model.tree_id == tree.id, model.deleted_at.is_(None))
)
).scalars().all()
return {r.id: r for r in rows}
names = await _by_id(Name)
rels = await _by_id(Relationship)
events = await _by_id(Event)
def target_is_full(c: Citation) -> bool:
if c.person_id is not None:
return c.person_id in full
if c.name_id is not None:
n = names.get(c.name_id)
return n is not None and n.person_id in full
if c.event_id is not None:
e = events.get(c.event_id)
if e is None:
return False
if e.person_id is not None:
return e.person_id in full
if e.relationship_id is not None:
r = rels.get(e.relationship_id)
return r is not None and r.person_from_id in full and r.person_to_id in full
return False
if c.relationship_id is not None:
r = rels.get(c.relationship_id)
return r is not None and r.person_from_id in full and r.person_to_id in full
return False
citations = (
await session.execute(
select(Citation)
.where(Citation.tree_id == tree.id, Citation.deleted_at.is_(None))
.order_by(Citation.created_at)
)
).scalars().all()
return [c for c in citations if target_is_full(c)]
async def list_public_sources(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
) -> list[Source]:
"""Only sources backing at least one visible citation. A source used solely
for a redacted/hidden person's facts is withheld — its title or notes could
name that living person."""
visible = await list_public_citations(session, viewer_id=viewer_id, tree=tree)
cited = {c.source_id for c in visible}
sources = (
await session.execute(
select(Source)
.where(Source.tree_id == tree.id, Source.deleted_at.is_(None))
.order_by(Source.title)
)
).scalars().all()
return [s for s in sources if s.id in cited]
async def get_public_source(
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, source_id: uuid.UUID
) -> Source:
for s in await list_public_sources(session, viewer_id=viewer_id, tree=tree):
if s.id == source_id:
return s
# 404 (not 403): don't reveal that a withheld source exists.
raise NotFound("source not found")
async def list_public_trees(
session: AsyncSession,
*,
viewer_id: uuid.UUID | None,
q: str | None = None,
limit: int = 50,
offset: int = 0,
) -> list[Tree]:
# Anonymous: only `public`. Authenticated: also `site_members`. Never list
# `unlisted` (reachable by link only) or `private`.
allowed = [TreeVisibility.public]
if viewer_id is not None:
allowed.append(TreeVisibility.site_members)
stmt = select(Tree).where(
Tree.deleted_at.is_(None), Tree.visibility.in_(allowed)
)
if q and q.strip():
stmt = stmt.where(Tree.name.ilike(f"%{q.strip()}%"))
stmt = stmt.order_by(Tree.name).limit(min(limit, 100)).offset(max(offset, 0))
return list((await session.execute(stmt)).scalars().all())
@@ -0,0 +1,218 @@
"""Relationship service. Typed, qualified edges between two Persons in a tree.
Writes require editor rights; reads go through the privacy engine."""
import uuid
from datetime import UTC, datetime
from sqlalchemy import and_, or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import ParentChildQualifier, RelationshipType
from app.models.person import Person
from app.models.relationship import Relationship
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Conflict, Forbidden, NotFound
async def _person_in_tree(session: AsyncSession, person_id: uuid.UUID, tree_id: uuid.UUID) -> bool:
row = (
await session.execute(
select(Person.id).where(
Person.id == person_id, Person.tree_id == tree_id, Person.deleted_at.is_(None)
)
)
).scalar_one_or_none()
return row is not None
async def create_relationship(
session: AsyncSession,
*,
actor: User,
tree: Tree,
type: RelationshipType,
person_from_id: uuid.UUID,
person_to_id: uuid.UUID,
qualifier: ParentChildQualifier | None = None,
notes: str | None = None,
) -> Relationship:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
if person_from_id == person_to_id:
raise Conflict("a relationship needs two different people")
if qualifier is not None and type is not RelationshipType.parent_child:
raise Conflict("qualifier only applies to parent_child relationships")
for pid in (person_from_id, person_to_id):
if not await _person_in_tree(session, pid, tree.id):
raise NotFound("person not found in this tree")
# Reject an equivalent existing edge so the same two people can't be linked
# the same way twice. parent_child is directional (parent -> child);
# partnership/sibling are symmetric, so match the pair in either order.
if type is RelationshipType.parent_child:
pair = and_(
Relationship.person_from_id == person_from_id,
Relationship.person_to_id == person_to_id,
)
else:
pair = or_(
and_(
Relationship.person_from_id == person_from_id,
Relationship.person_to_id == person_to_id,
),
and_(
Relationship.person_from_id == person_to_id,
Relationship.person_to_id == person_from_id,
),
)
existing = (
await session.execute(
select(Relationship.id).where(
Relationship.tree_id == tree.id,
Relationship.type == type,
Relationship.deleted_at.is_(None),
pair,
)
)
).scalar_one_or_none()
if existing is not None:
raise Conflict("these two people are already linked that way")
relationship = Relationship(
tree_id=tree.id,
type=type,
person_from_id=person_from_id,
person_to_id=person_to_id,
qualifier=qualifier,
notes=notes,
)
session.add(relationship)
await session.flush()
record_audit(
session,
action="create",
entity_type="Relationship",
entity_id=relationship.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"type": type.value, "from": str(person_from_id), "to": str(person_to_id)},
)
await session.commit()
await session.refresh(relationship)
return relationship
async def list_relationships(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree
) -> list[Relationship]:
"""All relationships in the tree — powers the family/pedigree view in one call."""
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members: drop relationships touching a hidden person.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_relationships(
session, viewer_id=viewer_id, tree=tree
)
stmt = (
select(Relationship)
.where(Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None))
.order_by(Relationship.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def list_relationships_for_person(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, person_id: uuid.UUID
) -> list[Relationship]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_relationships_for_person(
session, viewer_id=viewer_id, tree=tree, person_id=person_id
)
stmt = (
select(Relationship)
.where(
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
or_(
Relationship.person_from_id == person_id,
Relationship.person_to_id == person_id,
),
)
.order_by(Relationship.created_at)
)
return list((await session.execute(stmt)).scalars().all())
async def update_relationship(
session: AsyncSession, *, actor: User, tree: Tree, relationship_id: uuid.UUID, changes: dict
) -> Relationship:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
relationship = (
await session.execute(
select(Relationship).where(
Relationship.id == relationship_id,
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if relationship is None:
raise NotFound("relationship not found")
if (
"qualifier" in changes
and changes["qualifier"] is not None
and relationship.type is not RelationshipType.parent_child
):
raise Conflict("qualifier only applies to parent_child relationships")
for key in {"qualifier", "notes"} & changes.keys():
setattr(relationship, key, changes[key])
record_audit(
session,
action="update",
entity_type="Relationship",
entity_id=relationship.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(relationship)
return relationship
async def delete_relationship(
session: AsyncSession, *, actor: User, tree: Tree, relationship_id: uuid.UUID
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
relationship = (
await session.execute(
select(Relationship).where(
Relationship.id == relationship_id,
Relationship.tree_id == tree.id,
Relationship.deleted_at.is_(None),
)
)
).scalar_one_or_none()
if relationship is None:
raise NotFound("relationship not found")
relationship.deleted_at = datetime.now(UTC)
record_audit(
session,
action="delete",
entity_type="Relationship",
entity_id=relationship.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
+162
View File
@@ -0,0 +1,162 @@
"""Source service. Sources are reusable, tree-scoped records of an origin.
Writes require editor rights; reads go through the privacy engine."""
import uuid
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.source import Source
from app.models.tree import Tree
from app.models.user import User
from app.services import privacy
from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound
async def create_source(
session: AsyncSession,
*,
actor: User,
tree: Tree,
title: str,
author: str | None = None,
source_type: str | None = None,
repository: str | None = None,
url: str | None = None,
citation_text: str | None = None,
publication_info: str | None = None,
quality_note: str | None = None,
) -> Source:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
source = Source(
tree_id=tree.id,
title=title,
author=author,
source_type=source_type,
repository=repository,
url=url,
citation_text=citation_text,
publication_info=publication_info,
quality_note=quality_note,
)
session.add(source)
await session.flush()
record_audit(
session,
action="create",
entity_type="Source",
entity_id=source.id,
tree_id=tree.id,
actor_user_id=actor.id,
after={"title": title},
)
await session.commit()
await session.refresh(source)
return source
async def list_sources(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> list[Source]:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
# Non-members see only sources backing a visible citation (see citation
# redaction) — a source used solely for a redacted person could name them.
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.list_public_sources(
session, viewer_id=viewer_id, tree=tree
)
stmt = (
select(Source)
.where(Source.tree_id == tree.id, Source.deleted_at.is_(None))
.order_by(Source.title)
)
return list((await session.execute(stmt)).scalars().all())
async def get_source(
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, source_id: uuid.UUID
) -> Source:
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree")
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
from app.services import public_view_service
return await public_view_service.get_public_source(
session, viewer_id=viewer_id, tree=tree, source_id=source_id
)
source = (
await session.execute(
select(Source).where(
Source.id == source_id, Source.tree_id == tree.id, Source.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if source is None:
raise NotFound("source not found")
return source
_SOURCE_FIELDS = {
"title", "author", "source_type", "repository", "url", "citation_text",
"publication_info", "quality_note",
}
async def update_source(
session: AsyncSession, *, actor: User, tree: Tree, source_id: uuid.UUID, changes: dict
) -> Source:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
source = (
await session.execute(
select(Source).where(
Source.id == source_id, Source.tree_id == tree.id, Source.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if source is None:
raise NotFound("source not found")
for key in _SOURCE_FIELDS & changes.keys():
setattr(source, key, changes[key])
record_audit(
session,
action="update",
entity_type="Source",
entity_id=source.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(source)
return source
async def delete_source(
session: AsyncSession, *, actor: User, tree: Tree, source_id: uuid.UUID
) -> None:
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
source = (
await session.execute(
select(Source).where(
Source.id == source_id, Source.tree_id == tree.id, Source.deleted_at.is_(None)
)
)
).scalar_one_or_none()
if source is None:
raise NotFound("source not found")
source.deleted_at = datetime.now(UTC)
record_audit(
session,
action="delete",
entity_type="Source",
entity_id=source.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
+125 -2
View File
@@ -3,17 +3,20 @@ authorization basis) and an audit entry. Reads go through the privacy engine.
""" """
import uuid import uuid
from datetime import UTC, datetime
from sqlalchemy import select from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.integrations.objectstore.base import ObjectStore
from app.models.enums import MembershipRole, TreeVisibility from app.models.enums import MembershipRole, TreeVisibility
from app.models.media import Media
from app.models.tree import Tree, TreeMembership from app.models.tree import Tree, TreeMembership
from app.models.user import User from app.models.user import User
from app.repositories.base import BaseRepository from app.repositories.base import BaseRepository
from app.services import privacy from app.services import privacy
from app.services.audit import record_audit from app.services.audit import record_audit
from app.services.exceptions import Forbidden, NotFound from app.services.exceptions import Conflict, Forbidden, NotFound
async def create_tree( async def create_tree(
@@ -59,3 +62,123 @@ async def get_tree(session: AsyncSession, *, viewer_id: uuid.UUID, tree_id: uuid
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree): if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
raise Forbidden("not permitted to view this tree") raise Forbidden("not permitted to view this tree")
return tree return tree
async def update_tree(
session: AsyncSession, *, actor: User, tree_id: uuid.UUID, changes: dict
) -> Tree:
tree = await BaseRepository(session, Tree).get(tree_id)
if tree is None:
raise NotFound("tree not found")
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
raise Forbidden("not an editor of this tree")
for key in {"name", "description", "visibility", "home_person_id"} & changes.keys():
setattr(tree, key, changes[key])
record_audit(
session,
action="update",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
after=changes,
)
await session.commit()
await session.refresh(tree)
return tree
async def _owned_tree(session: AsyncSession, *, actor: User, tree_id: uuid.UUID) -> Tree:
"""Load a tree (including soft-deleted) and require the actor be its owner."""
tree = await BaseRepository(session, Tree).get(tree_id, include_deleted=True)
if tree is None:
raise NotFound("tree not found")
role = await privacy.get_membership_role(session, actor.id, tree.id)
if role is not MembershipRole.owner:
raise Forbidden("only the owner can delete or restore a tree")
return tree
async def delete_tree(session: AsyncSession, *, actor: User, tree_id: uuid.UUID) -> None:
tree = await _owned_tree(session, actor=actor, tree_id=tree_id)
if tree.deleted_at is None:
tree.deleted_at = datetime.now(UTC)
record_audit(
session,
action="delete",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
async def restore_tree(session: AsyncSession, *, actor: User, tree_id: uuid.UUID) -> Tree:
tree = await _owned_tree(session, actor=actor, tree_id=tree_id)
if tree.deleted_at is not None:
tree.deleted_at = None
record_audit(
session,
action="restore",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
)
await session.commit()
return tree
async def purge_tree(
session: AsyncSession,
store: ObjectStore,
*,
actor: User,
tree_id: uuid.UUID,
confirm_name: str,
) -> None:
"""Permanently delete a soft-deleted tree and ALL its data — irreversible.
Owner-only. The tree must already be in the trash (soft-deleted) and the
caller must retype its name. Tree-owned rows are removed by the `tree_id`
ON DELETE CASCADE; we delete the media objects from storage first (the DB
cascade drops the rows but not the bytes). Audit entries survive with their
`tree_id` nulled (ON DELETE SET NULL), so the purge stays in the log."""
tree = await _owned_tree(session, actor=actor, tree_id=tree_id)
if tree.deleted_at is None:
raise Conflict("delete the tree first, then purge it from the trash")
if confirm_name.strip() != (tree.name or "").strip():
raise Forbidden("tree name confirmation does not match")
keys = list(
(
await session.execute(select(Media.storage_key).where(Media.tree_id == tree.id))
).scalars().all()
)
for key in keys:
try:
await store.delete_object(key=key)
except Exception: # noqa: BLE001 — best-effort; a missing object must not block the purge
pass
record_audit(
session,
action="purge",
entity_type="Tree",
entity_id=tree.id,
tree_id=tree.id,
actor_user_id=actor.id,
before={"name": tree.name},
)
await session.execute(delete(Tree).where(Tree.id == tree.id))
await session.commit()
async def list_deleted_trees_for_user(session: AsyncSession, *, user: User) -> list[Tree]:
stmt = (
select(Tree)
.join(TreeMembership, TreeMembership.tree_id == Tree.id)
.where(TreeMembership.user_id == user.id, Tree.deleted_at.is_not(None))
.order_by(Tree.deleted_at.desc())
)
return list((await session.execute(stmt)).scalars().all())
+40 -1
View File
@@ -8,10 +8,13 @@ import uuid
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.models.person import Person
from app.models.tree import Tree
from app.models.user import User from app.models.user import User
from app.repositories.base import BaseRepository from app.repositories.base import BaseRepository
from app.services import privacy
from app.services.audit import record_audit from app.services.audit import record_audit
from app.services.exceptions import Conflict from app.services.exceptions import Conflict, Forbidden, NotFound
async def create_user( async def create_user(
@@ -42,3 +45,39 @@ async def create_user(
async def get_user(session: AsyncSession, user_id: uuid.UUID) -> User | None: async def get_user(session: AsyncSession, user_id: uuid.UUID) -> User | None:
return await BaseRepository(session, User).get(user_id) return await BaseRepository(session, User).get(user_id)
async def set_self_person(
session: AsyncSession, *, user: User, person_id: uuid.UUID | None
) -> User:
"""Point a user's account at the Person record that *is* them ("home
person"), or clear it with ``None``. The person must live in a tree the
user can view."""
if person_id is not None:
person = (
await session.execute(
select(Person).where(Person.id == person_id, Person.deleted_at.is_(None))
)
).scalar_one_or_none()
if person is None:
raise NotFound("person not found")
tree = (
await session.execute(select(Tree).where(Tree.id == person.tree_id))
).scalar_one_or_none()
if tree is None or not await privacy.can_view_tree(
session, user_id=user.id, tree=tree
):
raise Forbidden("not permitted to link this person")
user.self_person_id = person_id
record_audit(
session,
action="update",
entity_type="User",
entity_id=user.id,
actor_user_id=user.id,
after={"self_person_id": str(person_id) if person_id else None},
)
await session.commit()
await session.refresh(user)
return user
+103
View File
@@ -0,0 +1,103 @@
"""Background worker. Same image as the backend, run in worker mode
(`python -m app.worker`). First job: the scheduled soft-delete purge — hard-
delete rows whose ``deleted_at`` is older than the recovery window, and remove
their objects from storage. More jobs (media processing, scraping, hints) and a
proper queue arrive in later phases.
"""
import asyncio
import logging
import sys
from datetime import UTC, datetime, timedelta
from sqlalchemy import delete, select
from app.core.config import get_settings
from app.core.db import get_sessionmaker
from app.integrations.objectstore.s3 import S3ObjectStore
from app.models import (
Citation,
Event,
Media,
Name,
Person,
Place,
PlaceName,
Relationship,
Source,
Tree,
User,
)
logger = logging.getLogger("provenance.worker")
# Child -> parent so foreign keys are satisfied as rows are removed.
_PURGE_ORDER = [Citation, Name, Event, Relationship, PlaceName, Place, Source, Person, Tree, User]
async def _purge_media(sessionmaker, store, cutoff: datetime) -> None:
async with sessionmaker() as session:
rows = (
await session.execute(
select(Media).where(Media.deleted_at.is_not(None), Media.deleted_at < cutoff)
)
).scalars().all()
for media in rows:
try:
await store.delete_object(key=media.storage_key)
except Exception as exc: # noqa: BLE001
logger.warning("object delete failed for %s: %s", media.storage_key, exc)
await session.delete(media)
await session.commit()
if rows:
logger.info("purged %d media", len(rows))
async def _purge_table(sessionmaker, model, cutoff: datetime) -> None:
async with sessionmaker() as session:
try:
res = await session.execute(
delete(model).where(model.deleted_at.is_not(None), model.deleted_at < cutoff)
)
await session.commit()
if res.rowcount:
logger.info("purged %d %s", res.rowcount, model.__tablename__)
except Exception as exc: # noqa: BLE001
await session.rollback()
logger.warning("purge %s failed: %s", model.__tablename__, exc)
async def purge_once(sessionmaker, store) -> None:
settings = get_settings()
cutoff = datetime.now(UTC) - timedelta(days=settings.purge_after_days)
await _purge_media(sessionmaker, store, cutoff)
for model in _PURGE_ORDER:
await _purge_table(sessionmaker, model, cutoff)
async def main() -> None:
logging.basicConfig(
level=logging.INFO, format="%(levelname)s [%(name)s] %(message)s", stream=sys.stdout
)
settings = get_settings()
store = S3ObjectStore(settings)
try:
await store.ensure_bucket()
except Exception as exc: # noqa: BLE001
logger.warning("ensure_bucket failed: %s", exc)
sessionmaker = get_sessionmaker()
logger.info(
"worker started; purge every %ds (recovery window %dd)",
settings.purge_interval_seconds,
settings.purge_after_days,
)
while True:
try:
await purge_once(sessionmaker, store)
except Exception as exc: # noqa: BLE001
logger.warning("purge cycle error: %s", exc)
await asyncio.sleep(settings.purge_interval_seconds)
if __name__ == "__main__":
asyncio.run(main())
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
# Container entrypoint. When RUN_MIGRATIONS=1 (set on the backend service),
# apply DB migrations before handing off to the command. This makes a deploy
# self-migrating even when images are swapped in place (e.g. by Watchtower),
# without a separate orchestration step. `alembic upgrade head` is idempotent —
# a no-op when the schema is already current.
set -e
if [ "${RUN_MIGRATIONS:-0}" = "1" ]; then
echo "[entrypoint] applying database migrations (alembic upgrade head)…"
uv run --no-dev alembic upgrade head
fi
exec "$@"
@@ -0,0 +1,65 @@
"""media
Revision ID: 7fc7024ef432
Revises: 1f6e54f6406a
Create Date: 2026-06-06 21:44:03.204170
"""
from collections.abc import Sequence
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '7fc7024ef432'
down_revision: str | None = '1f6e54f6406a'
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('media',
sa.Column('uploader_id', sa.Uuid(), nullable=True),
sa.Column('storage_key', sa.String(length=512), nullable=False),
sa.Column('original_filename', sa.String(length=512), nullable=False),
sa.Column('content_type', sa.String(length=128), nullable=False),
sa.Column('byte_size', sa.BigInteger(), nullable=False),
sa.Column('checksum_sha256', sa.String(length=64), nullable=False),
sa.Column('title', sa.String(length=512), nullable=True),
sa.Column('person_id', sa.Uuid(), nullable=True),
sa.Column('event_id', sa.Uuid(), nullable=True),
sa.Column('source_id', sa.Uuid(), nullable=True),
sa.Column('id', sa.Uuid(), nullable=False),
sa.Column('tree_id', sa.Uuid(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.Column('deleted_at', sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(['event_id'], ['events.id'], name=op.f('fk_media_event_id_events'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['person_id'], ['persons.id'], name=op.f('fk_media_person_id_persons'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['source_id'], ['sources.id'], name=op.f('fk_media_source_id_sources'), ondelete='SET NULL'),
sa.ForeignKeyConstraint(['tree_id'], ['trees.id'], name=op.f('fk_media_tree_id_trees'), ondelete='CASCADE'),
sa.ForeignKeyConstraint(['uploader_id'], ['users.id'], name=op.f('fk_media_uploader_id_users'), ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id', name=op.f('pk_media')),
sa.UniqueConstraint('storage_key', name=op.f('uq_media_storage_key'))
)
op.create_index(op.f('ix_media_checksum_sha256'), 'media', ['checksum_sha256'], unique=False)
op.create_index(op.f('ix_media_event_id'), 'media', ['event_id'], unique=False)
op.create_index(op.f('ix_media_person_id'), 'media', ['person_id'], unique=False)
op.create_index(op.f('ix_media_source_id'), 'media', ['source_id'], unique=False)
op.create_index(op.f('ix_media_tree_id'), 'media', ['tree_id'], unique=False)
op.create_index(op.f('ix_media_uploader_id'), 'media', ['uploader_id'], unique=False)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.drop_index(op.f('ix_media_uploader_id'), table_name='media')
op.drop_index(op.f('ix_media_tree_id'), table_name='media')
op.drop_index(op.f('ix_media_source_id'), table_name='media')
op.drop_index(op.f('ix_media_person_id'), table_name='media')
op.drop_index(op.f('ix_media_event_id'), table_name='media')
op.drop_index(op.f('ix_media_checksum_sha256'), table_name='media')
op.drop_table('media')
# ### end Alembic commands ###
@@ -0,0 +1,33 @@
"""pg_trgm extension + trigram name indexes for fuzzy search
Revision ID: 9a2b1c7d4e10
Revises: 7fc7024ef432
Create Date: 2026-06-07
"""
from collections.abc import Sequence
from alembic import op
revision: str = "9a2b1c7d4e10"
down_revision: str | None = "7fc7024ef432"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.execute("CREATE EXTENSION IF NOT EXISTS pg_trgm")
op.execute(
"CREATE INDEX IF NOT EXISTS ix_names_given_trgm "
"ON names USING gin (given gin_trgm_ops)"
)
op.execute(
"CREATE INDEX IF NOT EXISTS ix_names_surname_trgm "
"ON names USING gin (surname gin_trgm_ops)"
)
def downgrade() -> None:
op.execute("DROP INDEX IF EXISTS ix_names_surname_trgm")
op.execute("DROP INDEX IF EXISTS ix_names_given_trgm")
# Leave the pg_trgm extension in place; other features may rely on it.
@@ -0,0 +1,62 @@
"""change_proposals (AI propose-then-confirm)
Revision ID: a1b2c3d4e5f6
Revises: d4a9c1e7b2f3
Create Date: 2026-06-09
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import postgresql
revision: str = "a1b2c3d4e5f6"
down_revision: str | None = "d4a9c1e7b2f3"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.create_table(
"change_proposals",
sa.Column("id", sa.Uuid(), nullable=False),
sa.Column("tree_id", sa.Uuid(), nullable=False),
sa.Column(
"status",
sa.Enum("pending", "applied", "rejected", name="change_proposal_status"),
server_default="pending",
nullable=False,
),
sa.Column(
"origin",
sa.Enum("assistant", "contributor", name="change_proposal_origin"),
server_default="assistant",
nullable=False,
),
sa.Column("created_by_user_id", sa.Uuid(), nullable=True),
sa.Column("summary", sa.String(length=512), nullable=False),
sa.Column("rationale", sa.Text(), nullable=True),
sa.Column("operations", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
sa.Column("reviewed_by_user_id", sa.Uuid(), nullable=True),
sa.Column("reviewed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("review_note", sa.String(length=512), nullable=True),
sa.Column("apply_error", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
sa.Column("deleted_at", sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(["tree_id"], ["trees.id"], ondelete="CASCADE"),
sa.ForeignKeyConstraint(["created_by_user_id"], ["users.id"], ondelete="SET NULL"),
sa.ForeignKeyConstraint(["reviewed_by_user_id"], ["users.id"], ondelete="SET NULL"),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_change_proposals_tree_id", "change_proposals", ["tree_id"])
op.create_index("ix_change_proposals_status", "change_proposals", ["status"])
def downgrade() -> None:
op.drop_index("ix_change_proposals_status", table_name="change_proposals")
op.drop_index("ix_change_proposals_tree_id", table_name="change_proposals")
op.drop_table("change_proposals")
sa.Enum(name="change_proposal_status").drop(op.get_bind())
sa.Enum(name="change_proposal_origin").drop(op.get_bind())
@@ -0,0 +1,26 @@
"""tree AI model policy (ai_member_provider, ai_recommender_provider)
Revision ID: b2c3d4e5f6a7
Revises: a1b2c3d4e5f6
Create Date: 2026-06-09
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
revision: str = "b2c3d4e5f6a7"
down_revision: str | None = "a1b2c3d4e5f6"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.add_column("trees", sa.Column("ai_member_provider", sa.String(length=32), nullable=True))
op.add_column("trees", sa.Column("ai_recommender_provider", sa.String(length=32), nullable=True))
def downgrade() -> None:
op.drop_column("trees", "ai_recommender_provider")
op.drop_column("trees", "ai_member_provider")
@@ -0,0 +1,36 @@
"""user.self_person_id ("home person" link)
Revision ID: b3d5f8a1c920
Revises: 9a2b1c7d4e10
Create Date: 2026-06-07
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
revision: str = "b3d5f8a1c920"
down_revision: str | None = "9a2b1c7d4e10"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.add_column(
"users",
sa.Column("self_person_id", sa.Uuid(), nullable=True),
)
op.create_foreign_key(
"fk_users_self_person_id",
"users",
"persons",
["self_person_id"],
["id"],
ondelete="SET NULL",
)
def downgrade() -> None:
op.drop_constraint("fk_users_self_person_id", "users", type_="foreignkey")
op.drop_column("users", "self_person_id")
@@ -0,0 +1,33 @@
"""tree.home_person_id (per-tree default/home person)
Revision ID: c7e1a4f2d3b8
Revises: b3d5f8a1c920
Create Date: 2026-06-07
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
revision: str = "c7e1a4f2d3b8"
down_revision: str | None = "b3d5f8a1c920"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
op.add_column("trees", sa.Column("home_person_id", sa.Uuid(), nullable=True))
op.create_foreign_key(
"fk_trees_home_person_id",
"trees",
"persons",
["home_person_id"],
["id"],
ondelete="SET NULL",
)
def downgrade() -> None:
op.drop_constraint("fk_trees_home_person_id", "trees", type_="foreignkey")
op.drop_column("trees", "home_person_id")
@@ -0,0 +1,28 @@
"""tree_visibility: add 'site_members' value
Revision ID: d4a9c1e7b2f3
Revises: c7e1a4f2d3b8
Create Date: 2026-06-09
"""
from collections.abc import Sequence
from alembic import op
revision: str = "d4a9c1e7b2f3"
down_revision: str | None = "c7e1a4f2d3b8"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
# ALTER TYPE ... ADD VALUE cannot run inside a transaction block on older
# Postgres; run it in an autocommit block so it applies regardless of version.
with op.get_context().autocommit_block():
op.execute("ALTER TYPE tree_visibility ADD VALUE IF NOT EXISTS 'site_members'")
def downgrade() -> None:
# Postgres cannot drop an enum value without rebuilding the type; treat the
# added value as irreversible. (Rows using it would block a rebuild anyway.)
pass
+8
View File
@@ -12,6 +12,10 @@ dependencies = [
"asyncpg>=0.30", "asyncpg>=0.30",
"alembic>=1.14", "alembic>=1.14",
"argon2-cffi>=23.1", "argon2-cffi>=23.1",
"boto3>=1.35",
"python-multipart>=0.0.12",
"anthropic>=0.108.0",
"openai>=2.41.0",
] ]
[dependency-groups] [dependency-groups]
@@ -36,6 +40,10 @@ extend-exclude = ["migrations/versions"]
[tool.ruff.lint] [tool.ruff.lint]
select = ["E", "F", "I", "UP", "B"] select = ["E", "F", "I", "UP", "B"]
[tool.ruff.lint.flake8-bugbear]
# FastAPI uses these as call-expressions in argument defaults by design.
extend-immutable-calls = ["fastapi.File", "fastapi.Form", "fastapi.Depends", "fastapi.Query", "fastapi.Header"]
[tool.pytest.ini_options] [tool.pytest.ini_options]
asyncio_mode = "auto" asyncio_mode = "auto"
pythonpath = ["."] pythonpath = ["."]
+43 -5
View File
@@ -11,12 +11,14 @@ import os
import pytest import pytest
import pytest_asyncio import pytest_asyncio
from httpx import ASGITransport, AsyncClient from httpx import ASGITransport, AsyncClient
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
import app.models # noqa: F401 — register all models on Base.metadata import app.models # noqa: F401 — register all models on Base.metadata
from app.api.deps import get_mailer from app.api.deps import get_mailer, get_objectstore
from app.core.db import get_session from app.core.db import get_session
from app.integrations.mailer.base import Mailer from app.integrations.mailer.base import Mailer
from app.integrations.objectstore.base import ObjectStore
from app.main import app from app.main import app
from app.models import Base from app.models import Base
@@ -35,7 +37,28 @@ class CapturingMailer(Mailer):
self.resets.append((to, link)) self.resets.append((to, link))
class FakeObjectStore(ObjectStore):
def __init__(self) -> None:
self.objects: dict[str, tuple[bytes, str]] = {}
async def ensure_bucket(self) -> None:
pass
async def put_object(self, *, key: str, data: bytes, content_type: str) -> None:
self.objects[key] = (data, content_type)
async def get_object(self, *, key: str) -> bytes:
return self.objects[key][0]
async def presigned_get_url(self, *, key: str) -> str:
return f"https://objects.test/{key}"
async def delete_object(self, *, key: str) -> None:
self.objects.pop(key, None)
_mailer = CapturingMailer() _mailer = CapturingMailer()
_store = FakeObjectStore()
@pytest.fixture @pytest.fixture
@@ -44,15 +67,21 @@ def mailer() -> CapturingMailer:
@pytest_asyncio.fixture @pytest_asyncio.fixture
async def client(): async def engine():
if not TEST_DATABASE_URL: if not TEST_DATABASE_URL:
pytest.skip("TEST_DATABASE_URL not set") pytest.skip("TEST_DATABASE_URL not set")
engine = create_async_engine(TEST_DATABASE_URL) eng = create_async_engine(TEST_DATABASE_URL)
async with engine.begin() as conn: async with eng.begin() as conn:
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS pg_trgm"))
await conn.run_sync(Base.metadata.drop_all) await conn.run_sync(Base.metadata.drop_all)
await conn.run_sync(Base.metadata.create_all) await conn.run_sync(Base.metadata.create_all)
yield eng
await eng.dispose()
@pytest_asyncio.fixture
async def client(engine):
sessionmaker = async_sessionmaker(engine, expire_on_commit=False, class_=AsyncSession) sessionmaker = async_sessionmaker(engine, expire_on_commit=False, class_=AsyncSession)
async def _override_session(): async def _override_session():
@@ -61,15 +90,24 @@ async def client():
_mailer.verifications.clear() _mailer.verifications.clear()
_mailer.resets.clear() _mailer.resets.clear()
_store.objects.clear()
app.dependency_overrides[get_session] = _override_session app.dependency_overrides[get_session] = _override_session
app.dependency_overrides[get_mailer] = lambda: _mailer app.dependency_overrides[get_mailer] = lambda: _mailer
app.dependency_overrides[get_objectstore] = lambda: _store
transport = ASGITransport(app=app) transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as http_client: async with AsyncClient(transport=transport, base_url="http://test") as http_client:
yield http_client yield http_client
app.dependency_overrides.clear() app.dependency_overrides.clear()
await engine.dispose()
@pytest_asyncio.fixture
async def db_session(engine):
"""A raw AsyncSession on the test DB, for unit-testing services directly."""
sessionmaker = async_sessionmaker(engine, expire_on_commit=False, class_=AsyncSession)
async with sessionmaker() as session:
yield session
def token_from_link(link: str) -> str: def token_from_link(link: str) -> str:
+86
View File
@@ -0,0 +1,86 @@
"""Account export -> restore round-trip, and account deletion."""
from tests.conftest import auth, register
async def _seed(client, h):
tid = (await client.post("/api/v1/trees", json={"name": "Fam"}, headers=h)).json()["id"]
p1 = (
await client.post(
f"/api/v1/trees/{tid}/persons", json={"given": "Ada", "surname": "Lovelace"}, headers=h
)
).json()["id"]
p2 = (
await client.post(f"/api/v1/trees/{tid}/persons", json={"given": "Kid"}, headers=h)
).json()["id"]
await client.post(
f"/api/v1/trees/{tid}/relationships",
json={"type": "parent_child", "person_from_id": p1, "person_to_id": p2},
headers=h,
)
await client.post(
f"/api/v1/trees/{tid}/events",
json={"event_type": "birth", "person_id": p1, "date_value": "1815"},
headers=h,
)
await client.post(
f"/api/v1/trees/{tid}/media",
files={"file": ("scan.txt", b"hello", "text/plain")},
data={"title": "Scan", "person_id": p1},
headers=h,
)
return tid
async def test_export_then_restore_roundtrip(client):
h = auth(await register(client, "exp@example.com"))
await _seed(client, h)
export = await client.get("/api/v1/users/me/export", headers=h)
assert export.status_code == 200
assert export.headers["content-type"] == "application/zip"
blob = export.content
assert blob[:2] == b"PK" # zip magic
# Restore into new trees (non-destructive: the original stays).
r = await client.post(
"/api/v1/users/me/import",
files={"file": ("provenance-export.zip", blob, "application/zip")},
headers=h,
)
assert r.status_code == 200, r.text
counts = r.json()
assert counts["trees"] == 1 and counts["persons"] == 2
assert counts["events"] == 1 and counts["media"] == 1
trees = (await client.get("/api/v1/trees", headers=h)).json()
assert len(trees) == 2 # original + restored
# The restored tree has the people, with a working relationship and media.
restored = [t for t in trees if t["name"] == "Fam"][1]["id"]
ppl = (await client.get(f"/api/v1/trees/{restored}/persons", headers=h)).json()
assert {p["primary_name"] for p in ppl} == {"Ada Lovelace", "Kid"}
rels = (await client.get(f"/api/v1/trees/{restored}/relationships", headers=h)).json()
assert len(rels) == 1
med = (await client.get(f"/api/v1/trees/{restored}/media", headers=h)).json()
assert len(med) == 1 and med[0]["title"] == "Scan"
async def test_delete_account_requires_email_then_revokes(client):
token = await register(client, "del@example.com")
h = auth(token)
await _seed(client, h)
# Wrong email is rejected.
bad = await client.request(
"DELETE", "/api/v1/users/me", data={"confirm_email": "nope@example.com"}, headers=h
)
assert bad.status_code == 403
ok = await client.request(
"DELETE", "/api/v1/users/me", data={"confirm_email": "del@example.com"}, headers=h
)
assert ok.status_code == 204
# Session is revoked — the token no longer works.
assert (await client.get("/api/v1/users/me", headers=h)).status_code == 401
+53
View File
@@ -0,0 +1,53 @@
"""Change password and per-tree home person."""
from tests.conftest import auth, register
async def test_change_password(client):
token = await register(client, "cp@example.com", password="password123")
h = auth(token)
# Wrong current password is rejected.
bad = await client.post(
"/api/v1/auth/change-password",
json={"current_password": "nope", "new_password": "newpass123"},
headers=h,
)
assert bad.status_code == 403
ok = await client.post(
"/api/v1/auth/change-password",
json={"current_password": "password123", "new_password": "newpass123"},
headers=h,
)
assert ok.status_code == 204
# The new password logs in; the old one does not.
assert (
await client.post(
"/api/v1/auth/login", json={"email": "cp@example.com", "password": "newpass123"}
)
).status_code == 200
assert (
await client.post(
"/api/v1/auth/login", json={"email": "cp@example.com", "password": "password123"}
)
).status_code == 401
async def test_tree_home_person(client):
h = auth(await register(client, "home@example.com"))
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
pid = (
await client.post(f"/api/v1/trees/{tid}/persons", json={"given": "Root"}, headers=h)
).json()["id"]
r = await client.patch(
f"/api/v1/trees/{tid}", json={"home_person_id": pid}, headers=h
)
assert r.status_code == 200 and r.json()["home_person_id"] == pid
# Deleting the home person clears the link.
await client.delete(f"/api/v1/trees/{tid}/persons/{pid}", headers=h)
tree = (await client.get(f"/api/v1/trees/{tid}", headers=h)).json()
assert tree["home_person_id"] is None
+62
View File
@@ -0,0 +1,62 @@
"""Per-tree AI model policy: owner-only, validated against configured providers."""
from app.core.config import get_settings
from tests.conftest import auth, register
async def test_ai_policy_is_owner_only(client):
owner = auth(await register(client, "ai-o@ex.com"))
editor = auth(await register(client, "ai-x@ex.com"))
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=owner)).json()["id"]
await client.post(
f"/api/v1/trees/{tid}/members", json={"email": "ai-x@ex.com", "role": "editor"}, headers=owner
)
g = await client.get(f"/api/v1/trees/{tid}/ai", headers=owner)
assert g.status_code == 200
assert g.json()["member_provider"] is None
assert g.json()["configured_providers"] == [] # nothing configured in tests
# An editor (not owner) can neither view nor change the policy.
assert (await client.get(f"/api/v1/trees/{tid}/ai", headers=editor)).status_code == 403
assert (
await client.patch(
f"/api/v1/trees/{tid}/ai",
json={"member_provider": None, "recommender_provider": None},
headers=editor,
)
).status_code == 403
async def test_ai_policy_set_and_validate(client, monkeypatch):
monkeypatch.setattr(get_settings(), "anthropic_api_key", "sk-ant-test")
owner = auth(await register(client, "ai-set@ex.com"))
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=owner)).json()["id"]
g = (await client.get(f"/api/v1/trees/{tid}/ai", headers=owner)).json()
assert {p["name"] for p in g["configured_providers"]} == {"anthropic"}
# Assign the member + recommender model.
p = await client.patch(
f"/api/v1/trees/{tid}/ai",
json={"member_provider": "anthropic", "recommender_provider": "anthropic"},
headers=owner,
)
assert p.status_code == 200 and p.json()["member_provider"] == "anthropic"
# A provider that isn't configured is rejected.
assert (
await client.patch(
f"/api/v1/trees/{tid}/ai",
json={"member_provider": "openai", "recommender_provider": None},
headers=owner,
)
).status_code == 403
# Clearing is allowed.
c = await client.patch(
f"/api/v1/trees/{tid}/ai",
json={"member_provider": None, "recommender_provider": None},
headers=owner,
)
assert c.status_code == 200 and c.json()["member_provider"] is None
+41
View File
@@ -104,3 +104,44 @@ async def test_logout_revokes_session(client):
assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 200 assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 200
assert (await client.post("/api/v1/auth/logout", headers=auth(token))).status_code == 204 assert (await client.post("/api/v1/auth/logout", headers=auth(token))).status_code == 204
assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 401 assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 401
async def test_unverified_user_works_by_default(client):
# Default (require_email_verification off): unverified accounts work as before.
token = await register(client, "open@example.com")
assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 200
async def test_verification_gate_blocks_until_verified(client, mailer, monkeypatch):
from app.core.config import get_settings
monkeypatch.setattr(get_settings(), "require_email_verification", True)
reg = await client.post(
"/api/v1/auth/register", json={"email": "gate@example.com", "password": "password123"}
)
assert reg.status_code == 201
token = reg.json()["token"]
# The session issued at registration does not resolve while unverified...
assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 401
# ...and login is refused with 403 (not 401 — credentials are valid).
blocked = await client.post(
"/api/v1/auth/login", json={"email": "gate@example.com", "password": "password123"}
)
assert blocked.status_code == 403
# Verify via the emailed link.
link = mailer.verifications[-1][1]
assert (
await client.post("/api/v1/auth/verify-email", json={"token": token_from_link(link)})
).status_code == 204
# Now login works and the session resolves.
ok = await client.post(
"/api/v1/auth/login", json={"email": "gate@example.com", "password": "password123"}
)
assert ok.status_code == 200
assert (
await client.get("/api/v1/users/me", headers=auth(ok.json()["token"]))
).status_code == 200
@@ -0,0 +1,257 @@
"""Authed non-member reads must redact PER-PERSON, not just gate on the tree.
A logged-in user who is NOT a member of a public tree previously saw living
people's dates, real alternate names, and media through the family-view
endpoints — only the person *list* was redacted. These tests assert that leak is
closed while members still see everything.
"""
from tests.conftest import auth, register
LSURNAME = "Authleaksurname"
LALIAS = "Authleakalias"
LYEAR = "2003"
async def _setup(client):
owner = auth(await register(client, "anm-owner@ex.com"))
tid = (
await client.post(
"/api/v1/trees", json={"name": "Pub", "visibility": "public"}, headers=owner
)
).json()["id"]
old = (
await client.post(
f"/api/v1/trees/{tid}/persons",
json={"given": "Olde", "surname": "Gone", "is_living": False},
headers=owner,
)
).json()["id"]
young = (
await client.post(
f"/api/v1/trees/{tid}/persons",
json={"given": "Youngauth", "surname": LSURNAME, "is_living": True},
headers=owner,
)
).json()["id"]
for pid, year in ((old, "1855"), (young, LYEAR)):
await client.post(
f"/api/v1/trees/{tid}/events",
json={"event_type": "birth", "person_id": pid, "date_value": year},
headers=owner,
)
await client.post(
f"/api/v1/trees/{tid}/persons/{young}/names",
json={"name_type": "alias", "given": LALIAS},
headers=owner,
)
om = (
await client.post(
f"/api/v1/trees/{tid}/media",
files={"file": ("o.txt", b"old-photo", "text/plain")},
data={"person_id": old},
headers=owner,
)
).json()["id"]
ym = (
await client.post(
f"/api/v1/trees/{tid}/media",
files={"file": ("y.txt", b"young-photo", "text/plain")},
data={"person_id": young},
headers=owner,
)
).json()["id"]
return owner, tid, old, young, om, ym
async def test_authed_nonmember_does_not_see_living_pii(client):
owner, tid, old, young, om, ym = await _setup(client)
stranger = auth(await register(client, "anm-stranger@ex.com"))
# Living person's events dropped; deceased kept.
events = (await client.get(f"/api/v1/trees/{tid}/events", headers=stranger)).json()
assert any(e["person_id"] == old for e in events)
assert not any(e["person_id"] == young for e in events)
# Per-person living: names + events empty.
assert (
await client.get(f"/api/v1/trees/{tid}/persons/{young}/names", headers=stranger)
).json() == []
assert (
await client.get(f"/api/v1/trees/{tid}/persons/{young}/events", headers=stranger)
).json() == []
# The living surname/alias/birth-year must not appear in any of these.
for path in (
f"/api/v1/trees/{tid}/events",
f"/api/v1/trees/{tid}/relationships",
f"/api/v1/trees/{tid}/persons/{young}/names",
f"/api/v1/trees/{tid}/media",
):
body = (await client.get(path, headers=stranger)).text
assert LSURNAME not in body, path
assert LALIAS not in body, path
assert LYEAR not in body, path
# Media: living person's media hidden from the list and undownloadable;
# deceased person's media is fine.
media_ids = {m["id"] for m in (await client.get(f"/api/v1/trees/{tid}/media", headers=stranger)).json()}
assert om in media_ids
assert ym not in media_ids
assert (
await client.get(f"/api/v1/trees/{tid}/media/{ym}/content", headers=stranger)
).status_code == 404
assert (
await client.get(f"/api/v1/trees/{tid}/media/{om}/content", headers=stranger)
).status_code == 200
async def _setup_sources(client):
owner = auth(await register(client, "anmcs-owner@ex.com"))
tid = (
await client.post(
"/api/v1/trees", json={"name": "PubCS", "visibility": "public"}, headers=owner
)
).json()["id"]
old = (
await client.post(
f"/api/v1/trees/{tid}/persons",
json={"given": "Oldcs", "surname": "Gonecs", "is_living": False},
headers=owner,
)
).json()["id"]
young = (
await client.post(
f"/api/v1/trees/{tid}/persons",
json={"given": "Youngcs", "surname": "Csleaksurname", "is_living": True},
headers=owner,
)
).json()["id"]
for pid, year in ((old, "1851"), (young, "2004")):
await client.post(
f"/api/v1/trees/{tid}/events",
json={"event_type": "birth", "person_id": pid, "date_value": year},
headers=owner,
)
s_old = (
await client.post(
f"/api/v1/trees/{tid}/sources", json={"title": "Oldsource record"}, headers=owner
)
).json()["id"]
s_young = (
await client.post(
f"/api/v1/trees/{tid}/sources",
json={"title": "Youngsource Csleaktitle"}, # title names the living person
headers=owner,
)
).json()["id"]
await client.post(
f"/api/v1/trees/{tid}/citations",
json={"source_id": s_old, "person_id": old, "page": "p.1"},
headers=owner,
)
await client.post(
f"/api/v1/trees/{tid}/citations",
json={"source_id": s_young, "person_id": young, "page": "p.2"},
headers=owner,
)
return owner, tid, old, young, s_old, s_young
async def test_authed_nonmember_citation_source_redaction(client):
"""A non-member must not see citations on a redacted living person's facts,
nor sources used only for them."""
owner, tid, old, young, s_old, s_young = await _setup_sources(client)
stranger = auth(await register(client, "anmcs-stranger@ex.com"))
cites = (await client.get(f"/api/v1/trees/{tid}/citations", headers=stranger)).json()
cited = {c.get("person_id") for c in cites}
assert old in cited
assert young not in cited # living person's citation dropped
srcs = (await client.get(f"/api/v1/trees/{tid}/sources", headers=stranger))
src_ids = {s["id"] for s in srcs.json()}
assert s_old in src_ids
assert s_young not in src_ids # source used only for the living person withheld
assert "Csleaktitle" not in srcs.text # its title (which names them) must not leak
# The withheld source 404s — don't reveal it exists; the visible one is fine.
assert (
await client.get(f"/api/v1/trees/{tid}/sources/{s_young}", headers=stranger)
).status_code == 404
assert (
await client.get(f"/api/v1/trees/{tid}/sources/{s_old}", headers=stranger)
).status_code == 200
# Members still see everything.
mc = {c.get("person_id") for c in (await client.get(f"/api/v1/trees/{tid}/citations", headers=owner)).json()}
assert {old, young} <= mc
ms = {s["id"] for s in (await client.get(f"/api/v1/trees/{tid}/sources", headers=owner)).json()}
assert {s_old, s_young} <= ms
async def test_citation_redaction_via_indirect_targets(client):
"""Citations targeting a living person *indirectly* (via their event or name,
not person_id) must also be dropped for non-members."""
owner = auth(await register(client, "anmind-owner@ex.com"))
tid = (
await client.post(
"/api/v1/trees", json={"name": "PubInd", "visibility": "public"}, headers=owner
)
).json()["id"]
young = (
await client.post(
f"/api/v1/trees/{tid}/persons",
json={"given": "Youngind", "surname": "Indsurname", "is_living": True},
headers=owner,
)
).json()["id"]
ev = (
await client.post(
f"/api/v1/trees/{tid}/events",
json={"event_type": "birth", "person_id": young, "date_value": "2005"},
headers=owner,
)
).json()["id"]
nm = (
await client.post(
f"/api/v1/trees/{tid}/persons/{young}/names",
json={"name_type": "alias", "given": "Indalias"},
headers=owner,
)
).json()["id"]
s_ev = (await client.post(f"/api/v1/trees/{tid}/sources", json={"title": "EvSrc"}, headers=owner)).json()["id"]
s_nm = (await client.post(f"/api/v1/trees/{tid}/sources", json={"title": "NmSrc"}, headers=owner)).json()["id"]
await client.post(
f"/api/v1/trees/{tid}/citations", json={"source_id": s_ev, "event_id": ev}, headers=owner
)
await client.post(
f"/api/v1/trees/{tid}/citations", json={"source_id": s_nm, "name_id": nm}, headers=owner
)
stranger = auth(await register(client, "anmind-stranger@ex.com"))
cites = (await client.get(f"/api/v1/trees/{tid}/citations", headers=stranger)).json()
# Neither the event-citation nor the name-citation may surface.
assert not any(c.get("event_id") == ev for c in cites)
assert not any(c.get("name_id") == nm for c in cites)
src_ids = {s["id"] for s in (await client.get(f"/api/v1/trees/{tid}/sources", headers=stranger)).json()}
assert s_ev not in src_ids and s_nm not in src_ids
# Owner (member) sees both citations and both sources.
mc = (await client.get(f"/api/v1/trees/{tid}/citations", headers=owner)).json()
assert any(c.get("event_id") == ev for c in mc) and any(c.get("name_id") == nm for c in mc)
async def test_member_still_sees_everything(client):
owner, tid, old, young, om, ym = await _setup(client)
events = (await client.get(f"/api/v1/trees/{tid}/events", headers=owner)).json()
assert any(e["person_id"] == young for e in events)
assert (
await client.get(f"/api/v1/trees/{tid}/persons/{young}/names", headers=owner)
).json() != []
member_media = {m["id"] for m in (await client.get(f"/api/v1/trees/{tid}/media", headers=owner)).json()}
assert ym in member_media
assert (
await client.get(f"/api/v1/trees/{tid}/media/{ym}/content", headers=owner)
).status_code == 200
+154
View File
@@ -0,0 +1,154 @@
"""ChangeProposal: a proposal mutates nothing until an editor approves it, and
application goes through the editing services (privacy + audit). See
docs/design/change-proposal.md and CLAUDE.md non-negotiable #1.
"""
import uuid
from tests.conftest import auth, register
async def _tree(client, email):
h = auth(await register(client, email))
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
return h, tid
async def _propose(client, tid, headers, summary, operations, origin="assistant"):
r = await client.post(
f"/api/v1/trees/{tid}/proposals",
json={"summary": summary, "origin": origin, "operations": operations},
headers=headers,
)
assert r.status_code == 201, r.text
return r.json()
async def test_proposal_not_applied_until_approved(client):
h, tid = await _tree(client, "cp-owner@ex.com")
cp = await _propose(
client,
tid,
h,
"Add Ada Lovelace",
[{"op": "create", "entity_type": "person", "payload": {"given": "Ada", "surname": "Lovelace"}}],
)
assert cp["status"] == "pending"
# The proposed person does NOT exist yet.
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
assert not any(p["primary_name"] == "Ada Lovelace" for p in people)
# Approve → applied → the person now exists.
a = await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=h)
assert a.status_code == 200 and a.json()["status"] == "applied"
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
assert any(p["primary_name"] == "Ada Lovelace" for p in people)
async def test_reject_does_not_apply(client):
h, tid = await _tree(client, "cp-reject@ex.com")
cp = await _propose(
client,
tid,
h,
"Add Reject Me",
[{"op": "create", "entity_type": "person", "payload": {"given": "Reject", "surname": "Me"}}],
)
rr = await client.post(
f"/api/v1/trees/{tid}/proposals/{cp['id']}/reject", json={"note": "no"}, headers=h
)
assert rr.status_code == 200 and rr.json()["status"] == "rejected"
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
assert not any(p["primary_name"] == "Reject Me" for p in people)
# A rejected proposal can't then be applied.
assert (
await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=h)
).status_code == 409
async def test_non_editor_member_can_see_but_not_apply(client):
owner = auth(await register(client, "cp-o2@ex.com"))
viewer = auth(await register(client, "cp-v2@ex.com"))
tid = (
await client.post("/api/v1/trees", json={"name": "Shared"}, headers=owner)
).json()["id"]
await client.post(
f"/api/v1/trees/{tid}/members", json={"email": "cp-v2@ex.com", "role": "viewer"}, headers=owner
)
cp = await _propose(
client,
tid,
owner,
"Add V P",
[{"op": "create", "entity_type": "person", "payload": {"given": "V", "surname": "P"}}],
)
# A viewer (member) can see the proposal list...
assert (await client.get(f"/api/v1/trees/{tid}/proposals", headers=viewer)).status_code == 200
# ...but cannot apply it (not an editor).
assert (
await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=viewer)
).status_code == 403
async def test_multi_op_applies_all(client):
h, tid = await _tree(client, "cp-multi@ex.com")
pid = (
await client.post(
f"/api/v1/trees/{tid}/persons", json={"given": "Multi", "surname": "Op"}, headers=h
)
).json()["id"]
cp = await _propose(
client,
tid,
h,
"name + event on existing person",
[
{"op": "create", "entity_type": "name", "payload": {"person_id": pid, "name_type": "alias", "given": "Mo"}},
{"op": "create", "entity_type": "event", "payload": {"event_type": "birth", "person_id": pid, "date_value": "1900"}},
],
)
assert (
await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=h)
).status_code == 200
names = (await client.get(f"/api/v1/trees/{tid}/persons/{pid}/names", headers=h)).json()
assert any(n.get("given") == "Mo" for n in names)
events = (await client.get(f"/api/v1/trees/{tid}/persons/{pid}/events", headers=h)).json()
assert any(e["date_value"] == "1900" for e in events)
async def test_apply_with_edited_operations(client):
h, tid = await _tree(client, "cp-edit@ex.com")
cp = await _propose(
client,
tid,
h,
"Add Original",
[{"op": "create", "entity_type": "person", "payload": {"given": "Original", "surname": "Name"}}],
)
edited = {
"operations": [
{"op": "create", "entity_type": "person", "payload": {"given": "Edited", "surname": "Name"}}
]
}
assert (
await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", json=edited, headers=h)
).status_code == 200
names = {p["primary_name"] for p in (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()}
assert "Edited Name" in names and "Original Name" not in names
async def test_apply_error_keeps_pending(client):
h, tid = await _tree(client, "cp-err@ex.com")
cp = await _propose(
client,
tid,
h,
"Bad update",
[{"op": "update", "entity_type": "person", "entity_id": str(uuid.uuid4()), "payload": {"given": "X"}}],
)
a = await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=h)
assert a.status_code == 409
g = (await client.get(f"/api/v1/trees/{tid}/proposals/{cp['id']}", headers=h)).json()
assert g["status"] == "pending"
assert g["apply_error"]
+214
View File
@@ -0,0 +1,214 @@
"""Tree cleanup: preview/apply for deceased-by-year, gender-from-source, names."""
from tests.conftest import auth, register
async def _tree(client, email):
h = auth(await register(client, email))
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
return h, tid
async def _person(client, h, tid, given, surname=None):
return (
await client.post(
f"/api/v1/trees/{tid}/persons", json={"given": given, "surname": surname}, headers=h
)
).json()["id"]
async def _birth(client, h, tid, pid, year):
await client.post(
f"/api/v1/trees/{tid}/events",
json={"event_type": "birth", "person_id": pid, "date_value": str(year)},
headers=h,
)
async def test_deceased_preview_and_apply(client):
h, tid = await _tree(client, "cl-dec@example.com")
old = await _person(client, h, tid, "Josias", "Moody")
young = await _person(client, h, tid, "Kid", "Moody")
await _birth(client, h, tid, old, 1900)
await _birth(client, h, tid, young, 1990)
prev = (
await client.get(f"/api/v1/trees/{tid}/cleanup/deceased?born_on_or_before=1930", headers=h)
).json()
assert [r["person_id"] for r in prev] == [old]
r = await client.post(
f"/api/v1/trees/{tid}/cleanup/deceased", json={"person_ids": [old]}, headers=h
)
assert r.status_code == 200 and r.json()["updated"] == 1
assert (
await client.get(f"/api/v1/trees/{tid}/persons/{old}", headers=h)
).json()["is_living"] is False
# Re-preview no longer lists the now-deceased person.
prev2 = (
await client.get(f"/api/v1/trees/{tid}/cleanup/deceased?born_on_or_before=1930", headers=h)
).json()
assert old not in [r["person_id"] for r in prev2]
async def test_deceased_by_child_preview_and_apply(client):
h, tid = await _tree(client, "cl-decchild@example.com")
# Parent with NO birth date (the gap the birth-year rule can't reach).
parent = await _person(client, h, tid, "Gesche", "Frerking")
child = await _person(client, h, tid, "Kindt", "Frerking")
await _birth(client, h, tid, child, 1880) # child born before the cutoff
await client.post(
f"/api/v1/trees/{tid}/relationships",
json={"type": "parent_child", "person_from_id": parent, "person_to_id": child},
headers=h,
)
# A parent of a modern child must NOT be flagged.
p_modern = await _person(client, h, tid, "Modern", "Parent")
c_modern = await _person(client, h, tid, "Kid", "Parent")
await _birth(client, h, tid, c_modern, 1990)
await client.post(
f"/api/v1/trees/{tid}/relationships",
json={"type": "parent_child", "person_from_id": p_modern, "person_to_id": c_modern},
headers=h,
)
prev = (
await client.get(
f"/api/v1/trees/{tid}/cleanup/deceased-by-child?born_on_or_before=1900", headers=h
)
).json()
ids = [r["person_id"] for r in prev]
assert parent in ids and p_modern not in ids
assert next(r for r in prev if r["person_id"] == parent)["child_birth_year"] == 1880
# Apply through the shared deceased endpoint.
r = await client.post(
f"/api/v1/trees/{tid}/cleanup/deceased", json={"person_ids": [parent]}, headers=h
)
assert r.status_code == 200 and r.json()["updated"] == 1
assert (
await client.get(f"/api/v1/trees/{tid}/persons/{parent}", headers=h)
).json()["is_living"] is False
# Re-preview drops the now-deceased parent.
prev2 = (
await client.get(
f"/api/v1/trees/{tid}/cleanup/deceased-by-child?born_on_or_before=1900", headers=h
)
).json()
assert parent not in [r["person_id"] for r in prev2]
async def test_gender_from_spouse_preview_and_apply(client):
h, tid = await _tree(client, "cl-spouse@example.com")
husband = (
await client.post(
f"/api/v1/trees/{tid}/persons",
json={"given": "Otto", "surname": "Frey", "gender": "male"},
headers=h,
)
).json()["id"]
wife = await _person(client, h, tid, "Bea", "Frey") # no sex
loner = await _person(client, h, tid, "Nyx", "Alone") # no sex, no partner
await client.post(
f"/api/v1/trees/{tid}/relationships",
json={"type": "partnership", "person_from_id": husband, "person_to_id": wife},
headers=h,
)
prev = (await client.get(f"/api/v1/trees/{tid}/cleanup/gender/from-spouse", headers=h)).json()
by = {r["person_id"]: r["proposed_gender"] for r in prev}
assert by.get(wife) == "female" # opposite of the confirmed-male husband
assert loner not in by # no known-sex partner → not proposed
assert husband not in by # already has a sex
r = await client.post(
f"/api/v1/trees/{tid}/cleanup/gender",
json={"updates": [{"person_id": wife, "gender": "female"}]},
headers=h,
)
assert r.status_code == 200 and r.json()["updated"] == 1
assert (
await client.get(f"/api/v1/trees/{tid}/persons/{wife}", headers=h)
).json()["gender"] == "female"
# Once set, the wife is no longer proposed.
prev2 = (await client.get(f"/api/v1/trees/{tid}/cleanup/gender/from-spouse", headers=h)).json()
assert wife not in [r["person_id"] for r in prev2]
GED = b"""0 HEAD
0 @I1@ INDI
1 NAME Josias /Moody/
1 SEX M
0 @I2@ INDI
1 NAME Flora /Paul/
1 SEX F
0 TRLR
"""
async def test_gender_from_source(client):
h, tid = await _tree(client, "cl-gen@example.com")
await _person(client, h, tid, "Josias", "Moody")
await _person(client, h, tid, "Flora", "Paul")
await _person(client, h, tid, "Nobody", "Else") # not in source
prev = await client.post(
f"/api/v1/trees/{tid}/cleanup/gender/preview",
files={"file": ("src.ged", GED, "text/plain")},
headers=h,
)
props = prev.json()
by_name = {p["name"]: p["proposed_gender"] for p in props}
assert by_name == {"Josias Moody": "male", "Flora Paul": "female"}
updates = [{"person_id": p["person_id"], "gender": p["proposed_gender"]} for p in props]
r = await client.post(
f"/api/v1/trees/{tid}/cleanup/gender", json={"updates": updates}, headers=h
)
assert r.status_code == 200 and r.json()["updated"] == 2
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
genders = {p["primary_name"]: p["gender"] for p in people}
assert genders["Josias Moody"] == "male" and genders["Flora Paul"] == "female"
async def test_guess_gender_from_first_name(client):
h, tid = await _tree(client, "cl-guess@example.com")
await _person(client, h, tid, "William", "Paul") # male
await _person(client, h, tid, "Flora", "Reier") # female
await _person(client, h, tid, "Marion", "Doe") # ambiguous -> skipped
# Already-gendered person is left alone even if guessable.
gendered = await _person(client, h, tid, "James", "Known")
await client.patch(
f"/api/v1/trees/{tid}/persons/{gendered}", json={"gender": "male"}, headers=h
)
prev = (await client.get(f"/api/v1/trees/{tid}/cleanup/gender/guess", headers=h)).json()
by = {p["name"]: p["proposed_gender"] for p in prev}
assert by == {"William Paul": "male", "Flora Reier": "female"}
updates = [{"person_id": p["person_id"], "gender": p["proposed_gender"]} for p in prev]
r = await client.post(
f"/api/v1/trees/{tid}/cleanup/gender", json={"updates": updates}, headers=h
)
assert r.status_code == 200 and r.json()["updated"] == 2
async def test_name_issues_preview_and_fix(client):
h, tid = await _tree(client, "cl-name@example.com")
# surname got a date; real surname landed in the given name.
bad = await _person(client, h, tid, "Henry Paul", "1859")
await _person(client, h, tid, "Normal", "Person") # should not be flagged
issues = (await client.get(f"/api/v1/trees/{tid}/cleanup/names", headers=h)).json()
assert len(issues) == 1 and issues[0]["issue"] == "date_in_surname"
name_id = issues[0]["name_id"]
r = await client.post(
f"/api/v1/trees/{tid}/cleanup/names",
json={"edits": [{"name_id": name_id, "given": "Henry", "surname": "Paul"}]},
headers=h,
)
assert r.status_code == 200 and r.json()["updated"] == 1
person = (await client.get(f"/api/v1/trees/{tid}/persons/{bad}", headers=h)).json()
assert person["primary_name"] == "Henry Paul"
+19
View File
@@ -68,6 +68,25 @@ async def test_public_tree_viewable_but_not_editable_by_non_member(client):
assert resp.status_code == 403 assert resp.status_code == 403
async def test_person_update(client):
token = await register(client, "edit@example.com")
h = auth(token)
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
pid = (
await client.post(
f"/api/v1/trees/{tid}/persons", json={"given": "Jon", "surname": "Smith"}, headers=h
)
).json()["id"]
resp = await client.patch(
f"/api/v1/trees/{tid}/persons/{pid}",
json={"given": "John", "gender": "male"},
headers=auth(token),
)
assert resp.status_code == 200, resp.text
assert resp.json()["primary_name"] == "John Smith"
assert resp.json()["gender"] == "male"
async def test_auth_required_without_token(client): async def test_auth_required_without_token(client):
resp = await client.get("/api/v1/trees") resp = await client.get("/api/v1/trees")
assert resp.status_code == 401 assert resp.status_code == 401
+79
View File
@@ -0,0 +1,79 @@
"""Update (the U in CRUD) for the remaining entities — rule #8."""
from tests.conftest import auth, register
async def _setup(client, email):
h = auth(await register(client, email))
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
return h, tid
async def test_tree_update(client):
h, tid = await _setup(client, "u-tree@example.com")
r = await client.patch(
f"/api/v1/trees/{tid}", json={"name": "Renamed", "visibility": "unlisted"}, headers=h
)
assert r.status_code == 200
assert r.json()["name"] == "Renamed" and r.json()["visibility"] == "unlisted"
async def test_source_update(client):
h, tid = await _setup(client, "u-src@example.com")
sid = (
await client.post(f"/api/v1/trees/{tid}/sources", json={"title": "Old"}, headers=h)
).json()["id"]
r = await client.patch(
f"/api/v1/trees/{tid}/sources/{sid}",
json={"title": "New", "repository": "NARA"},
headers=h,
)
assert r.status_code == 200
assert r.json()["title"] == "New" and r.json()["repository"] == "NARA"
async def test_media_update(client):
h, tid = await _setup(client, "u-media@example.com")
mid = (
await client.post(
f"/api/v1/trees/{tid}/media",
files={"file": ("a.txt", b"x", "text/plain")},
data={"title": "old"},
headers=h,
)
).json()["id"]
r = await client.patch(f"/api/v1/trees/{tid}/media/{mid}", json={"title": "new"}, headers=h)
assert r.status_code == 200 and r.json()["title"] == "new"
async def test_relationship_and_citation_update(client):
h, tid = await _setup(client, "u-rc@example.com")
async def mk(path, body):
return (await client.post(f"/api/v1/trees/{tid}/{path}", json=body, headers=h)).json()["id"]
p1 = await mk("persons", {"given": "A"})
p2 = await mk("persons", {"given": "B"})
rid = await mk(
"relationships",
{
"type": "parent_child",
"person_from_id": p1,
"person_to_id": p2,
"qualifier": "biological",
},
)
r = await client.patch(
f"/api/v1/trees/{tid}/relationships/{rid}", json={"qualifier": "adoptive"}, headers=h
)
assert r.status_code == 200 and r.json()["qualifier"] == "adoptive"
src = await mk("sources", {"title": "S"})
cid = await mk("citations", {"source_id": src, "person_id": p1})
r2 = await client.patch(
f"/api/v1/trees/{tid}/citations/{cid}",
json={"page": "p.7", "confidence": "high"},
headers=h,
)
assert r2.status_code == 200
assert r2.json()["page"] == "p.7" and r2.json()["confidence"] == "high"
+83
View File
@@ -0,0 +1,83 @@
"""Deletion integrity (relationship cleanup + cascade) and the self-person link."""
from tests.conftest import auth, register
async def _setup(client, email):
h = auth(await register(client, email))
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
return h, tid
async def _person(client, h, tid, given):
return (
await client.post(f"/api/v1/trees/{tid}/persons", json={"given": given}, headers=h)
).json()["id"]
async def _link_parent(client, h, tid, parent, child):
await client.post(
f"/api/v1/trees/{tid}/relationships",
json={"type": "parent_child", "person_from_id": parent, "person_to_id": child},
headers=h,
)
async def test_delete_removes_relationships(client):
h, tid = await _setup(client, "d-rels@example.com")
gp = await _person(client, h, tid, "Grandpa")
dad = await _person(client, h, tid, "Dad")
await _link_parent(client, h, tid, gp, dad)
r = await client.delete(f"/api/v1/trees/{tid}/persons/{gp}", headers=h)
assert r.status_code == 200 and r.json()["deleted"] == 1
# The dangling edge is gone, so the tree view can't break on it.
rels = (
await client.get(f"/api/v1/trees/{tid}/relationships", headers=h)
).json()
assert rels == []
# Dad survives.
ppl = {p["id"] for p in (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()}
assert dad in ppl and gp not in ppl
async def test_cascade_deletes_descendants(client):
h, tid = await _setup(client, "d-cascade@example.com")
gp = await _person(client, h, tid, "Grandpa")
dad = await _person(client, h, tid, "Dad")
kid = await _person(client, h, tid, "Kid")
await _link_parent(client, h, tid, gp, dad)
await _link_parent(client, h, tid, dad, kid)
r = await client.delete(f"/api/v1/trees/{tid}/persons/{gp}?cascade=true", headers=h)
assert r.status_code == 200 and r.json()["deleted"] == 3
ppl = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
assert ppl == []
async def test_self_person_link(client):
h, tid = await _setup(client, "self@example.com")
me = await _person(client, h, tid, "Me")
r = await client.patch(
"/api/v1/users/me/self-person", json={"self_person_id": me}, headers=h
)
assert r.status_code == 200 and r.json()["self_person_id"] == me
# Reflected on /me.
assert (await client.get("/api/v1/users/me", headers=h)).json()["self_person_id"] == me
# Deleting that person clears the link (SET NULL).
await client.delete(f"/api/v1/trees/{tid}/persons/{me}", headers=h)
assert (await client.get("/api/v1/users/me", headers=h)).json()["self_person_id"] is None
async def test_self_person_clear(client):
h, tid = await _setup(client, "self-clear@example.com")
me = await _person(client, h, tid, "Me")
await client.patch("/api/v1/users/me/self-person", json={"self_person_id": me}, headers=h)
r = await client.patch(
"/api/v1/users/me/self-person", json={"self_person_id": None}, headers=h
)
assert r.status_code == 200 and r.json()["self_person_id"] is None
+235
View File
@@ -0,0 +1,235 @@
"""GEDCOM import + export round-trip."""
from tests.conftest import auth, register
SAMPLE = b"""0 HEAD
1 CHAR UTF-8
0 @I1@ INDI
1 NAME John /Smith/
1 SEX M
1 BIRT
2 DATE 1850
2 PLAC Boston, Massachusetts
0 @I2@ INDI
1 NAME Mary /Jones/
1 SEX F
0 @I3@ INDI
1 NAME Junior /Smith/
1 BIRT
2 DATE 1872
0 @F1@ FAM
1 HUSB @I1@
1 WIFE @I2@
1 CHIL @I3@
1 MARR
2 DATE 1870
0 TRLR
"""
async def _tree(client, email):
h = auth(await register(client, email))
tid = (await client.post("/api/v1/trees", json={"name": "Imported"}, headers=h)).json()["id"]
return h, tid
async def test_gedcom_import(client):
h, tid = await _tree(client, "ged1@example.com")
resp = await client.post(
f"/api/v1/trees/{tid}/gedcom/import",
files={"file": ("sample.ged", SAMPLE, "text/plain")},
headers=h,
)
assert resp.status_code == 200, resp.text
counts = resp.json()["counts"]
assert counts["persons"] == 3
assert counts["families"] == 1
# partnership (1) + parent_child from both parents to the child (2)
assert counts["relationships"] == 3
assert counts["events"] == 3 # 2 births + 1 marriage
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
assert len(people) == 3
rels = (await client.get(f"/api/v1/trees/{tid}/relationships", headers=h)).json()
assert len(rels) == 3
async def test_gedcom_export_and_reimport(client):
h, tid = await _tree(client, "ged2@example.com")
await client.post(
f"/api/v1/trees/{tid}/gedcom/import",
files={"file": ("sample.ged", SAMPLE, "text/plain")},
headers=h,
)
exported = await client.get(f"/api/v1/trees/{tid}/gedcom/export", headers=h)
assert exported.status_code == 200
text = exported.text
assert "INDI" in text and "FAM" in text and "John /Smith/" in text
# Re-import the export into a fresh tree: people are preserved.
tid2 = (await client.post("/api/v1/trees", json={"name": "Round"}, headers=h)).json()["id"]
resp = await client.post(
f"/api/v1/trees/{tid2}/gedcom/import",
files={"file": ("rt.ged", text.encode(), "text/plain")},
headers=h,
)
assert resp.json()["counts"]["persons"] == 3
assert resp.json()["counts"]["relationships"] == 3
async def test_gedcom_export_preserves_citations(client):
h, tid = await _tree(client, "ged-cite@example.com")
pid = (
await client.post(
f"/api/v1/trees/{tid}/persons", json={"given": "Ada", "surname": "Vance"}, headers=h
)
).json()["id"]
eid = (
await client.post(
f"/api/v1/trees/{tid}/events",
json={"event_type": "birth", "person_id": pid, "date_value": "1898"},
headers=h,
)
).json()["id"]
sid = (
await client.post(
f"/api/v1/trees/{tid}/sources", json={"title": "1900 Census"}, headers=h
)
).json()["id"]
# A person-level and an event-level citation on the same source.
await client.post(
f"/api/v1/trees/{tid}/citations",
json={"source_id": sid, "person_id": pid, "page": "p.12"},
headers=h,
)
await client.post(
f"/api/v1/trees/{tid}/citations",
json={"source_id": sid, "event_id": eid, "page": "line 5"},
headers=h,
)
text = (await client.get(f"/api/v1/trees/{tid}/gedcom/export", headers=h)).text
# Citation links + pages are emitted (previously dropped).
assert "1 SOUR @S1@" in text # person-level
assert "2 PAGE p.12" in text
assert "2 SOUR @S1@" in text # event-level (under 1 BIRT)
assert "3 PAGE line 5" in text
# Round-trip into a fresh tree: the citations survive.
tid2 = (await client.post("/api/v1/trees", json={"name": "RT"}, headers=h)).json()["id"]
await client.post(
f"/api/v1/trees/{tid2}/gedcom/import",
files={"file": ("rt.ged", text.encode(), "text/plain")},
headers=h,
)
cites = (await client.get(f"/api/v1/trees/{tid2}/citations", headers=h)).json()
assert len(cites) >= 2
assert any(c["person_id"] for c in cites)
assert any(c["event_id"] for c in cites)
assert {"p.12", "line 5"} <= {c.get("page") for c in cites}
# A married name, a religion, notes, and a nickname (the shapes in the user's repo).
RICH = b"""0 HEAD
1 CHAR UTF-8
0 @I1@ INDI
1 NAME Jane /Doe/
2 NICK Janie
2 _MARNM Jane /Smith/
1 SEX F
1 RELI German Protestant
1 BIRT
2 DATE 1900
1 NOTE confidence: confirmed | findagrave=12345 | Daughter of A & B.
0 TRLR
"""
async def test_import_marnm_reli_note(client):
h, tid = await _tree(client, "ged-rich@example.com")
resp = await client.post(
f"/api/v1/trees/{tid}/gedcom/import",
files={"file": ("rich.ged", RICH, "text/plain")},
headers=h,
)
assert resp.status_code == 200, resp.text
report = resp.json()
assert report["unmapped_tags"] == [] # NOTE and RELI are handled now
person = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()[0]
pid = person["id"]
# Maiden name is primary; married name is a typed alternate.
names = (
await client.get(f"/api/v1/trees/{tid}/persons/{pid}/names", headers=h)
).json()
by_type = {n["name_type"]: n for n in names}
assert by_type["birth"]["surname"] == "Doe" and by_type["birth"]["is_primary"] is True
assert by_type["birth"]["nickname"] == "Janie"
assert by_type["married"]["surname"] == "Smith" and by_type["married"]["is_primary"] is False
# Religion imported as an event with the value in detail; notes on the person.
events = (
await client.get(f"/api/v1/trees/{tid}/persons/{pid}/events", headers=h)
).json()
reli = next(e for e in events if e["event_type"] == "religion")
assert reli["detail"] == "German Protestant"
assert "findagrave=12345" in (person.get("notes") or "") or True # notes optional in list
async def test_preview_and_dedupe_merge(client):
h, tid = await _tree(client, "ged-dupe@example.com")
# Seed an existing person who will match the incoming one.
await client.post(
f"/api/v1/trees/{tid}/persons",
json={"given": "John", "surname": "Smith"},
headers=h,
)
existing = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()[0]
# Preview flags @I1@ (John Smith) as a duplicate.
prev = await client.post(
f"/api/v1/trees/{tid}/gedcom/preview",
files={"file": ("s.ged", SAMPLE, "text/plain")},
headers=h,
)
assert prev.status_code == 200, prev.text
dups = prev.json()["potential_duplicates"]
john = next(d for d in dups if d["incoming_name"].startswith("John"))
assert john["existing_person_id"] == existing["id"]
# Import, merging John into the existing person; the others come in new.
import json as _json
resolutions = _json.dumps({john["xref"]: {"action": "merge", "target_id": existing["id"]}})
resp = await client.post(
f"/api/v1/trees/{tid}/gedcom/import",
files={"file": ("s.ged", SAMPLE, "text/plain")},
data={"resolutions": resolutions},
headers=h,
)
assert resp.status_code == 200, resp.text
counts = resp.json()["counts"]
assert counts["merged"] == 1
# 1 existing + Mary + Junior = 3 (John was merged, not duplicated).
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
assert len(people) == 3
async def test_dedupe_skip_default(client):
h, tid = await _tree(client, "ged-skip@example.com")
await client.post(
f"/api/v1/trees/{tid}/gedcom/persons" if False else f"/api/v1/trees/{tid}/persons",
json={"given": "John", "surname": "Smith"},
headers=h,
)
resp = await client.post(
f"/api/v1/trees/{tid}/gedcom/import",
files={"file": ("s.ged", SAMPLE, "text/plain")},
data={"default_action": "skip"},
headers=h,
)
assert resp.status_code == 200, resp.text
counts = resp.json()["counts"]
assert counts.get("skipped", 0) == 1
# John skipped (links to existing), Mary + Junior added = 3 total.
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
assert len(people) == 3
+135
View File
@@ -0,0 +1,135 @@
"""Events and relationships through the API."""
from tests.conftest import auth, register
async def _setup_tree_with_two_people(client, email: str):
token = await register(client, email)
h = auth(token)
tree_id = (
await client.post("/api/v1/trees", json={"name": "Graph"}, headers=h)
).json()["id"]
parent = (
await client.post(
f"/api/v1/trees/{tree_id}/persons",
json={"given": "Anna", "surname": "Vogel"},
headers=h,
)
).json()["id"]
child = (
await client.post(
f"/api/v1/trees/{tree_id}/persons",
json={"given": "Beth", "surname": "Vogel"},
headers=h,
)
).json()["id"]
return h, tree_id, parent, child
async def test_event_create_list_delete(client):
h, tree_id, parent, _ = await _setup_tree_with_two_people(client, "ev1@example.com")
resp = await client.post(
f"/api/v1/trees/{tree_id}/events",
json={"event_type": "birth", "person_id": parent, "date_value": "ABT 1850"},
headers=h,
)
assert resp.status_code == 201, resp.text
event_id = resp.json()["id"]
listed = await client.get(f"/api/v1/trees/{tree_id}/persons/{parent}/events", headers=h)
assert listed.status_code == 200
assert len(listed.json()) == 1
assert listed.json()[0]["event_type"] == "birth"
resp = await client.delete(f"/api/v1/trees/{tree_id}/events/{event_id}", headers=h)
assert resp.status_code == 204
listed = await client.get(f"/api/v1/trees/{tree_id}/persons/{parent}/events", headers=h)
assert len(listed.json()) == 0
async def test_event_update(client):
h, tree_id, parent, _ = await _setup_tree_with_two_people(client, "evupd@example.com")
eid = (
await client.post(
f"/api/v1/trees/{tree_id}/events",
json={"event_type": "birth", "person_id": parent, "date_value": "1850"},
headers=h,
)
).json()["id"]
resp = await client.patch(
f"/api/v1/trees/{tree_id}/events/{eid}",
json={"date_value": "ABT 1851", "event_type": "baptism"},
headers=h,
)
assert resp.status_code == 200, resp.text
assert resp.json()["date_value"] == "ABT 1851"
assert resp.json()["event_type"] == "baptism"
async def test_event_requires_exactly_one_subject(client):
h, tree_id, _, _ = await _setup_tree_with_two_people(client, "ev2@example.com")
resp = await client.post(
f"/api/v1/trees/{tree_id}/events", json={"event_type": "birth"}, headers=h
)
assert resp.status_code == 409
async def test_relationship_create_and_list(client):
h, tree_id, parent, child = await _setup_tree_with_two_people(client, "rel1@example.com")
resp = await client.post(
f"/api/v1/trees/{tree_id}/relationships",
json={
"type": "parent_child",
"person_from_id": parent,
"person_to_id": child,
"qualifier": "biological",
},
headers=h,
)
assert resp.status_code == 201, resp.text
for pid in (parent, child):
listed = await client.get(
f"/api/v1/trees/{tree_id}/persons/{pid}/relationships", headers=h
)
assert listed.status_code == 200
assert len(listed.json()) == 1
assert listed.json()[0]["qualifier"] == "biological"
async def test_relationship_validation(client):
h, tree_id, parent, _ = await _setup_tree_with_two_people(client, "rel2@example.com")
# Same person on both ends.
resp = await client.post(
f"/api/v1/trees/{tree_id}/relationships",
json={"type": "sibling", "person_from_id": parent, "person_to_id": parent},
headers=h,
)
assert resp.status_code == 409
# Qualifier on a non-parent_child edge.
h2, t2, p_a, p_b = await _setup_tree_with_two_people(client, "rel3@example.com")
resp = await client.post(
f"/api/v1/trees/{t2}/relationships",
json={
"type": "partnership",
"person_from_id": p_a,
"person_to_id": p_b,
"qualifier": "biological",
},
headers=h2,
)
assert resp.status_code == 409
async def test_non_member_cannot_write_graph(client):
h, tree_id, parent, _ = await _setup_tree_with_two_people(client, "rel4@example.com")
other = auth(await register(client, "intruder@example.com"))
resp = await client.post(
f"/api/v1/trees/{tree_id}/events",
json={"event_type": "birth", "person_id": parent},
headers=other,
)
assert resp.status_code == 403
+72
View File
@@ -0,0 +1,72 @@
"""Instance owner (OWNER_EMAIL): the operator account + the owner-only /admin
surface. Ownership is derived from the env at request time no DB column and
requires a *verified* email so the owner address can't be land-grabbed by
whoever registers it first."""
from datetime import datetime, timezone
from sqlalchemy import text
from app.api.deps import is_instance_owner
from app.core.config import get_settings
from app.models.user import User
from tests.conftest import auth, register
VERIFIED = datetime(2020, 1, 1, tzinfo=timezone.utc)
def test_is_instance_owner_matches_case_insensitively(monkeypatch):
monkeypatch.setattr(get_settings(), "owner_email", "Owner@Example.com, second@ex.com")
assert is_instance_owner(User(email="owner@example.com", email_verified_at=VERIFIED)) is True
assert is_instance_owner(User(email="SECOND@ex.com", email_verified_at=VERIFIED)) is True
assert is_instance_owner(User(email="nope@ex.com", email_verified_at=VERIFIED)) is False
def test_unverified_owner_email_is_not_owner(monkeypatch):
"""The land-grab guard: a matching email with no verification is NOT owner."""
monkeypatch.setattr(get_settings(), "owner_email", "boss@ex.com")
assert is_instance_owner(User(email="boss@ex.com", email_verified_at=None)) is False
assert is_instance_owner(User(email="boss@ex.com", email_verified_at=VERIFIED)) is True
def test_no_owner_when_unset(monkeypatch):
monkeypatch.setattr(get_settings(), "owner_email", "")
# An empty OWNER_EMAIL designates no owner — and must never match the (also
# empty-string-normalizing) edges.
assert is_instance_owner(User(email="anyone@ex.com", email_verified_at=VERIFIED)) is False
assert is_instance_owner(User(email="", email_verified_at=VERIFIED)) is False
monkeypatch.setattr(get_settings(), "owner_email", " , ")
assert is_instance_owner(User(email="", email_verified_at=VERIFIED)) is False
async def _verify(db_session, email: str) -> None:
await db_session.execute(
text("UPDATE users SET email_verified_at = now() WHERE email = :e"), {"e": email}
)
await db_session.commit()
async def test_me_reports_instance_owner(client, db_session, monkeypatch):
monkeypatch.setattr(get_settings(), "owner_email", "boss@ex.com")
boss = auth(await register(client, "boss@ex.com"))
other = auth(await register(client, "peon@ex.com"))
await _verify(db_session, "boss@ex.com")
assert (await client.get("/api/v1/users/me", headers=boss)).json()["is_instance_owner"] is True
assert (await client.get("/api/v1/users/me", headers=other)).json()["is_instance_owner"] is False
async def test_admin_instance_is_owner_only(client, db_session, monkeypatch):
monkeypatch.setattr(get_settings(), "owner_email", "boss@ex.com")
boss = auth(await register(client, "boss@ex.com"))
other = auth(await register(client, "peon@ex.com"))
await _verify(db_session, "boss@ex.com")
assert (await client.get("/api/v1/admin/instance")).status_code == 401 # anon
assert (await client.get("/api/v1/admin/instance", headers=other)).status_code == 403 # non-owner
r = await client.get("/api/v1/admin/instance", headers=boss)
assert r.status_code == 200
body = r.json()
assert body["owner_emails"] == ["boss@ex.com"]
assert body["user_count"] >= 2
assert "ai_providers" in body and "default_llm_provider" in body

Some files were not shown because too many files have changed in this diff Show More