Compare commits
117 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 7043532c3b | |||
| 1340d1957f | |||
| e24a7cfcc9 | |||
| 07944e329e | |||
| a33a88e558 | |||
| fe8349819f | |||
| e745fb5d4d | |||
| e0573e6be2 | |||
| 3731d77d4b | |||
| bf1576252b | |||
| 0ed6ba4505 | |||
| ed263cf9a7 | |||
| f7666ad30b | |||
| 690a6da659 | |||
| e7115023e1 | |||
| 58400ffdf7 | |||
| 629bfa1367 | |||
| 1562febdcf | |||
| 265f5f4e7a | |||
| a6179037c2 | |||
| 7ed3ddd448 | |||
| 447daf7fa8 | |||
| 0388b9b99f | |||
| 00f403defa | |||
| 519f1c31b5 | |||
| 3a1395b6af | |||
| 2712ae469b | |||
| 88beb9650f | |||
| 15504ba6e1 | |||
| c5631d3eab | |||
| 6fbad3106d | |||
| 94b5caa7e5 | |||
| f8fa23c1f6 | |||
| c6b1e72130 | |||
| ceafb299d6 | |||
| de50f2c803 | |||
| 9187c0a791 | |||
| abaa8efdd5 | |||
| 251a10a087 | |||
| 330543f9ce | |||
| d540dc3f32 | |||
| 8652425413 | |||
| 3a7728f1dc | |||
| eb0350733b | |||
| 6d3147e86d | |||
| b4434cb5dd | |||
| 39e3eac3df | |||
| 660fe7b37f | |||
| 5485dd2077 | |||
| 05d2773e25 | |||
| 768c68cbe0 | |||
| 7d6fbce87e | |||
| 12ba0a0fb6 | |||
| 150d69e5ac | |||
| 053ce357ac | |||
| 269cae556f | |||
| 0df44e7e59 | |||
| 7a5c5f2882 | |||
| 20c7fbd8d6 | |||
| b8405ced07 | |||
| 91a7ce1dc2 | |||
| 8b91326481 | |||
| 671b560768 | |||
| 6a5ef4d392 | |||
| 3810b65de0 | |||
| 9820a77d25 | |||
| 3ff03b037b | |||
| 84a743f5b9 | |||
| e6dfe39e84 | |||
| 4a3fe983fa | |||
| 251652a935 | |||
| dc1b6aac01 | |||
| f93327f5d3 | |||
| c86771034c | |||
| b51b65de80 | |||
| 93c22b4bcf | |||
| 7255920135 | |||
| 62513ee22e | |||
| ac0b9818dd | |||
| 182a5dab16 | |||
| 77b78410ff | |||
| fe1e0171ff | |||
| 9dbdae975a | |||
| c5a2a7f0d4 | |||
| 8c36785197 | |||
| fae1162ff8 | |||
| 1025f86657 | |||
| a53858f920 | |||
| 941f9827c1 | |||
| 6ec852a23a | |||
| 7405ec762f | |||
| aa62ca490e | |||
| 97f7a9e0ff | |||
| cd4ccb4ac8 | |||
| 6696015970 | |||
| e8839b15a0 | |||
| 548e883d82 | |||
| 37ac49767e | |||
| 9b04bcefba | |||
| e9b2436ce0 | |||
| 8903e480cf | |||
| d27cc5dddc | |||
| 943f459b91 | |||
| 5106538934 | |||
| 2669543e56 | |||
| 0262ed3d97 | |||
| 9ee960c4ef | |||
| 7f640649b9 | |||
| a8929c2862 | |||
| b90ba53a3f | |||
| c4e9d69e00 | |||
| 0673896133 | |||
| 1164841950 | |||
| 5824e70895 | |||
| 04ccdbf96a | |||
| f165ccb941 | |||
| e0fb924a1d |
@@ -30,6 +30,7 @@ These are product invariants, not preferences. Do not violate them, and flag any
|
||||
- **Object storage:** S3-compatible (MinIO for self-host).
|
||||
- **Edge:** Caddy reverse proxy; optional Cloudflare Tunnel (preferred ingress, never required).
|
||||
- **Email:** operator-configured SMTP.
|
||||
- **Model providers:** pluggable `LLMProvider` + `EmbeddingProvider` abstraction (ABCs) with Null / Anthropic / OpenAI-compatible (OpenAI, xAI, Ollama) implementations; an operator configures one or more via env and they're selectable by name through a registry (per-tree AI policy + `default_llm_provider`/`default_embedding_provider`).
|
||||
- **CI/CD:** Gitea Actions build per-component images. **Push** to the LAN registry `192.168.0.2:1234` (plain HTTP, bypasses Cloudflare's body limit); **pull** via the public `git.jpaul.io` FQDN. Servers pull to deploy — no host build. Mirrors the drawbar setup; see [[gitea-lan-push-fqdn-pull]].
|
||||
|
||||
Pick libraries consistent with this stack. If you introduce a significant dependency or a new service, note it in ARCHITECTURE.md in the same change.
|
||||
@@ -39,17 +40,24 @@ Pick libraries consistent with this stack. If you introduce a significant depend
|
||||
```
|
||||
/ # docs and project meta (this file, README, LICENSE, COC, CONTRIBUTING)
|
||||
/docs # PRD.md, ARCHITECTURE.md
|
||||
/backend # FastAPI service (uv-managed). app/{api/v1, services (+ privacy engine), repositories, models, schemas, integrations (auth/mailer), core}; migrations/ = Alembic
|
||||
/deploy # docker-compose.yml, Caddyfile, .env.example — the self-host stack
|
||||
/backend # FastAPI service (uv-managed). app/{api/v1, services (+ privacy engine), repositories, models, schemas, integrations (auth, mailer, objectstore, models = pluggable LLM/embedding providers), core}; migrations/ = Alembic
|
||||
/deploy # docker-compose.yml (+ docker-compose.dev.yml), Caddyfile, .env.example, backup.sh + BACKUP.md (one-command pg_dump + MinIO backup) — the self-host stack
|
||||
/.gitea/workflows # Gitea Actions CI (build images → Gitea registry)
|
||||
/frontend # Next.js (App Router, TS, Tailwind, shadcn-style UI). app/ pages, lib/api generated OpenAPI client, components/ui
|
||||
```
|
||||
|
||||
Phase 0 is landing **deploy-first**: the compose stack (Postgres + MinIO + Caddy + a minimal FastAPI backend exposing `/health` and `/health/ready`) and CI come before the real data model and the frontend. Backend dependencies are managed with **uv**; migrations use **Alembic**. The core data model (ARCHITECTURE §5), **local auth** (Argon2 passwords, backend-issued sessions, email verify/reset behind the `AuthProvider` interface; API auth via Bearer header or HttpOnly cookie), and the **Next.js frontend scaffold** (Tailwind + shadcn-style UI, generated OpenAPI client, auth + tree/person views) have all landed — **Phase 0 is complete and running on the live deployment.** Phase 1 (core tree features — media, soft-delete recovery, richer CRUD) is next; OIDC/social auth is Phase 5. Keep this section current as the tree grows.
|
||||
Phase 0 landed **deploy-first**: the compose stack (Postgres + MinIO + Caddy + FastAPI backend) and CI before the data model and frontend. Backend deps use **uv**; migrations use **Alembic**. Status (keep current as the tree grows):
|
||||
|
||||
- **Phase 0 — Foundation: complete** and running live (core data model, local auth behind `AuthProvider`, Next.js frontend).
|
||||
- **Phase 1 — Core tree: complete.** Media (upload/serve), soft-delete + recovery UI, full CRUD across entities, and the 4-level tree visibility/privacy model (#41–#51).
|
||||
- **Phase 2 — substantially landed.** GEDCOM import (preview→apply, duplicate-aware) and export (citation-preserving, #232); fuzzy name search (pg_trgm) + the public `/explore` directory. Living-person protection is still hardening.
|
||||
- **Phase 4 — AI assistant foundations landed.** Pluggable `LLMProvider`/`EmbeddingProvider` abstraction + multi-provider registry (Anthropic/OpenAI/xAI/Ollama, #235/#237), the **ChangeProposal** propose-then-confirm flow (#236), and per-tree AI model policy (#238). The assistant's *tool surface that emits proposals* is the remaining piece.
|
||||
- Also shipped: tree membership management (#233), an **instance owner/operator** role (`OWNER_EMAIL`, #240), a schema-drift readiness guard (#239), and a one-command operator backup (#234).
|
||||
- **Not built yet:** Phase 3 (Property — parcels/deeds/chain-of-title; no property models exist), Phase 5 (OIDC/social auth — only the `AuthProvider` ABC exists), and cross-tree hints (last; needs multiple populated trees + the embedding provider).
|
||||
|
||||
## Where to start
|
||||
|
||||
The roadmap is phased in PRD §8. Build in dependency order. **Phase 0 — Foundation is complete** and running on the live deployment; **Phase 1 (core tree features) is the current target.** For reference, Phase 0 covered:
|
||||
The roadmap is phased in PRD §8. Build in dependency order. **Phases 0 and 1 are complete**, Phase 2 is substantially done, and Phase 4's AI foundations have shipped (see the status list above). The biggest unbuilt areas are **Phase 3 (Property)** and **Phase 5 (OIDC/social auth)** — likely current targets. For reference, Phase 0 covered:
|
||||
|
||||
1. Backend skeleton (FastAPI, async, layered) + Postgres + migrations
|
||||
2. Core data model from ARCHITECTURE §5 — start with User, Tree, TreeMembership, Person, Name, Relationship, Event, Place, Source, Citation, AuditEntry, soft-delete support
|
||||
@@ -58,7 +66,7 @@ The roadmap is phased in PRD §8. Build in dependency order. **Phase 0 — Found
|
||||
5. The deploy stack: `compose` for app + postgres + objectstore, Caddy config, env-driven settings
|
||||
6. CI/CD: Gitea Actions building images to the registry
|
||||
|
||||
Don't get ahead of the phases. GEDCOM lands before the assistant (so AI writes target a stable model); property follows a tested people graph; hints come last because they need multiple populated trees. If you think the order is wrong, raise it rather than reordering silently.
|
||||
Don't get ahead of the phases. GEDCOM and the assistant's propose-diff foundation (provider abstraction + ChangeProposal approval flow) have shipped; the remaining dependency-ordered work is **Property** (Phase 3, on top of the tested people graph), then richer collaboration/audit UI, with **cross-tree hints last** (they need multiple populated trees and the embedding provider). If you think the order is wrong, raise it rather than reordering silently.
|
||||
|
||||
## Conventions
|
||||
|
||||
@@ -69,6 +77,23 @@ Don't get ahead of the phases. GEDCOM lands before the assistant (so AI writes t
|
||||
- **Privacy/assistant/hint code gets extra care** — these are the areas where bugs do real harm. Prefer a design note before a large change.
|
||||
- **No secrets in the repo.** Config via env; provide `.env.example` with placeholders.
|
||||
|
||||
## Patched dependencies (family-chart)
|
||||
|
||||
The tree view uses **family-chart** (d3-based). Two adjustments live in the repo:
|
||||
|
||||
- **CSS is vendored** at `frontend/app/trees/[id]/tree/chart.css` — the package blocks its CSS subpath export, so we copy it in.
|
||||
- **The library is patched** via `patch-package` (`frontend/patches/family-chart+0.9.0.patch`, applied by the `postinstall` hook; the backend/frontend Dockerfiles `COPY patches` before install). Both hunks touch `dist/family-chart.js` **and** `dist/family-chart.esm.js` (the app loads the `esm` build). Current fixes:
|
||||
1. **Spouse-centering layout** (`setupSpouses` / `sortChildrenWithSpouses`) — center a person between two spouses with children under the correct pair.
|
||||
2. **`cardToMiddle` vertical centering** — the lib scaled `datum.x` by the zoom factor `k` but not `datum.y`, so "fly to a node" drifted vertically at any zoom ≠ 1; we add the missing `* k`.
|
||||
|
||||
To change a patch: edit the file(s) under `node_modules/family-chart/dist/`, then `cd frontend && npx patch-package family-chart` to regenerate, and verify with `npx patch-package --error-on-fail`.
|
||||
|
||||
**Upstreamed.** Both are general library bugfixes, not app-specific, and are submitted upstream:
|
||||
- `cardToMiddle` vertical centering — **donatso/family-chart#103** (issue **#102**).
|
||||
- Multi-spouse centered layout — **donatso/family-chart#105** (issue **#104**).
|
||||
|
||||
If either is merged + released, bump `family-chart`, drop the corresponding patch hunk, **and** remove any in-app compensation (e.g. the `cardToMiddle` caller in `tree/page.tsx` passes raw `y` precisely because the patch fixes it — pre-scaling there too would double-correct). Until then, keep the patch.
|
||||
|
||||
## License & contribution terms
|
||||
|
||||
Provenance is **source-available** under **BUSL-1.1** (see [LICENSE](LICENSE)): free for personal/family/non-commercial use, no third-party commercial hosting, and each release converts to **AGPL-3.0** four years after it ships. The DCO sign-off keeps the licensing chain clean so the maintainer can manage that conversion and a possible future hosted offering. Don't add code under an incompatible license, and don't vendor dependencies whose licenses conflict with eventual AGPL distribution.
|
||||
|
||||
@@ -19,13 +19,14 @@ Every fact links to its source. Every claim can be traced. Nothing is just asser
|
||||
## What it does
|
||||
|
||||
- **Build a tree that holds up.** People, relationships, events, and places — with every fact linked to the document, photo, or record it came from.
|
||||
- **Trace the land, not just the family.** Properties are first-class. Record ownership events (grants, deeds, inheritances, sales), reconstruct chain-of-title, and tie parcels to the people who held them.
|
||||
- **Bring your own archive.** Scans, PDFs, photos, audio recordings — first-class citizens, not afterthoughts.
|
||||
- **A research assistant that proposes, never overwrites.** The built-in AI assistant searches legal sources, lays out what it found, and waits for your approval before anything touches your data. You can point it at the major model providers or a self-hosted model — your keys, your choice.
|
||||
- **Standards over silos.** Full GEDCOM 7 import and export. Migrate in, migrate out.
|
||||
- **Privacy you control.** Public, unlisted, or private per tree; any individual can be hidden; living people are protected by default.
|
||||
- **Standards over silos.** GEDCOM import and export (5.5.1 / 7 common subset) — duplicate-aware import, citation-preserving export. Migrate in, migrate out.
|
||||
- **Privacy you control.** Public, members-only (any signed-in user on your instance), unlisted, or private per tree; any individual can be hidden; living people are protected by default.
|
||||
- **Find your people.** When another user's tree overlaps with yours, Provenance can surface an anonymous "possible match" — and only connects you if you both say yes.
|
||||
- **Run it your way.** Container-native. Self-host behind Caddy and, if you like, a Cloudflare Tunnel. Multi-tenant, so your whole extended family — or a whole community of strangers — can coexist on one deployment.
|
||||
- **Run it your way.** Container-native. Self-host behind Caddy and, if you like, a Cloudflare Tunnel. Multi-tenant, so your whole extended family — or a whole community of strangers — can coexist on one deployment. One-command backups (Postgres + object storage) and an instance-owner admin role keep operations in your hands.
|
||||
|
||||
**Where it's headed — trace the land, not just the family.** The same source-backed treatment for *property*: parcels, deeds, and ownership events, reconstructing chain-of-title and tying land to the people who held it. The people side ships today; the land half is on the roadmap, not yet built — but it's why Provenance exists, not an afterthought.
|
||||
|
||||
## Who it's for
|
||||
|
||||
|
||||
@@ -21,7 +21,11 @@ RUN --mount=type=cache,target=/root/.cache/uv \
|
||||
COPY app ./app
|
||||
COPY alembic.ini ./alembic.ini
|
||||
COPY migrations ./migrations
|
||||
COPY docker-entrypoint.sh ./docker-entrypoint.sh
|
||||
RUN chmod +x ./docker-entrypoint.sh
|
||||
|
||||
EXPOSE 8000
|
||||
|
||||
# The entrypoint runs migrations first when RUN_MIGRATIONS=1, then the command.
|
||||
ENTRYPOINT ["./docker-entrypoint.sh"]
|
||||
CMD ["uv", "run", "--no-dev", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
|
||||
@@ -10,6 +10,8 @@ from app.core.db import get_session
|
||||
from app.integrations.mailer.base import Mailer
|
||||
from app.integrations.mailer.console import ConsoleMailer
|
||||
from app.integrations.mailer.smtp import SMTPMailer
|
||||
from app.integrations.models.base import EmbeddingProvider, LLMProvider
|
||||
from app.integrations.models.null import NullEmbeddingProvider, NullLLMProvider
|
||||
from app.integrations.objectstore.base import ObjectStore
|
||||
from app.integrations.objectstore.s3 import S3ObjectStore
|
||||
from app.models.user import User
|
||||
@@ -40,6 +42,48 @@ async def get_current_user(request: Request, session: SessionDep) -> User:
|
||||
CurrentUser = Annotated[User, Depends(get_current_user)]
|
||||
|
||||
|
||||
async def get_current_user_or_none(request: Request, session: SessionDep) -> User | None:
|
||||
"""Optional auth for public read endpoints — never raises. Returns the user
|
||||
when a valid session is present, else None (anonymous viewer)."""
|
||||
raw_token = extract_session_token(request)
|
||||
if raw_token is None:
|
||||
return None
|
||||
return await auth_service.resolve_session_user(session, raw_token=raw_token)
|
||||
|
||||
|
||||
CurrentUserOrNone = Annotated[User | None, Depends(get_current_user_or_none)]
|
||||
|
||||
|
||||
def is_instance_owner(user: User) -> bool:
|
||||
"""Whether this account is an instance owner/operator — i.e. its email is
|
||||
named in OWNER_EMAIL *and* that email has been verified. Instance ownership
|
||||
is an operational/config role; it does NOT bypass the privacy engine or grant
|
||||
access to others' tree data.
|
||||
|
||||
The verified-email requirement is load-bearing: registration is open and (by
|
||||
default) doesn't require verification, so without it an attacker could claim
|
||||
the owner email by registering it before the operator does — a land-grab to
|
||||
the highest role with no proof of inbox control. Requiring verification ties
|
||||
ownership to actual control of the named inbox regardless of the global
|
||||
REQUIRE_EMAIL_VERIFICATION setting. (Self-hosts without SMTP can verify via
|
||||
the link the console mailer prints to the operator-controlled logs.)"""
|
||||
owners = get_settings().owner_emails()
|
||||
return (
|
||||
bool(owners)
|
||||
and user.email_verified_at is not None
|
||||
and user.email.strip().lower() in owners
|
||||
)
|
||||
|
||||
|
||||
async def require_instance_owner(current: CurrentUser) -> User:
|
||||
if not is_instance_owner(current):
|
||||
raise HTTPException(status.HTTP_403_FORBIDDEN, "instance owner only")
|
||||
return current
|
||||
|
||||
|
||||
InstanceOwner = Annotated[User, Depends(require_instance_owner)]
|
||||
|
||||
|
||||
def get_mailer() -> Mailer:
|
||||
settings = get_settings()
|
||||
if settings.mailer == "smtp" and settings.smtp_host:
|
||||
@@ -55,3 +99,84 @@ def get_objectstore() -> ObjectStore:
|
||||
|
||||
|
||||
ObjectStoreDep = Annotated[ObjectStore, Depends(get_objectstore)]
|
||||
|
||||
|
||||
def build_llm_providers() -> dict[str, LLMProvider]:
|
||||
"""Every LLM provider whose credentials are configured, keyed by name. Run
|
||||
several at once; pick one with get_llm_provider(name)."""
|
||||
from app.integrations.models.anthropic_provider import AnthropicLLMProvider
|
||||
from app.integrations.models.openai_compat import OpenAICompatibleLLMProvider
|
||||
|
||||
s = get_settings()
|
||||
providers: dict[str, LLMProvider] = {}
|
||||
if s.anthropic_api_key:
|
||||
providers["anthropic"] = AnthropicLLMProvider(
|
||||
api_key=s.anthropic_api_key, model=s.anthropic_model, max_tokens=s.llm_max_tokens
|
||||
)
|
||||
if s.openai_api_key:
|
||||
providers["openai"] = OpenAICompatibleLLMProvider(
|
||||
api_key=s.openai_api_key, base_url=s.openai_base_url, model=s.openai_model,
|
||||
max_tokens=s.llm_max_tokens,
|
||||
)
|
||||
if s.xai_api_key:
|
||||
providers["xai"] = OpenAICompatibleLLMProvider(
|
||||
api_key=s.xai_api_key, base_url=s.xai_base_url, model=s.xai_model,
|
||||
max_tokens=s.llm_max_tokens,
|
||||
)
|
||||
if s.ollama_enabled:
|
||||
providers["ollama"] = OpenAICompatibleLLMProvider(
|
||||
api_key=None, base_url=s.ollama_base_url, model=s.ollama_model,
|
||||
max_tokens=s.llm_max_tokens,
|
||||
)
|
||||
return providers
|
||||
|
||||
|
||||
def configured_llm_providers() -> list[dict]:
|
||||
"""Configured LLM providers as {name, model} — for the AI admin view (no
|
||||
secrets). Mirrors build_llm_providers() without constructing clients."""
|
||||
s = get_settings()
|
||||
out: list[dict] = []
|
||||
if s.anthropic_api_key:
|
||||
out.append({"name": "anthropic", "model": s.anthropic_model})
|
||||
if s.openai_api_key:
|
||||
out.append({"name": "openai", "model": s.openai_model})
|
||||
if s.xai_api_key:
|
||||
out.append({"name": "xai", "model": s.xai_model})
|
||||
if s.ollama_enabled:
|
||||
out.append({"name": "ollama", "model": s.ollama_model})
|
||||
return out
|
||||
|
||||
|
||||
def get_llm_provider(name: str | None = None) -> LLMProvider:
|
||||
"""The named LLM provider, or the configured default, or Null if unconfigured."""
|
||||
providers = build_llm_providers()
|
||||
return providers.get(name or get_settings().default_llm_provider) or NullLLMProvider()
|
||||
|
||||
|
||||
LLMProviderDep = Annotated[LLMProvider, Depends(get_llm_provider)]
|
||||
|
||||
|
||||
def build_embedding_providers() -> dict[str, EmbeddingProvider]:
|
||||
from app.integrations.models.openai_compat import OpenAICompatibleEmbeddingProvider
|
||||
|
||||
s = get_settings()
|
||||
providers: dict[str, EmbeddingProvider] = {}
|
||||
if s.openai_api_key:
|
||||
providers["openai"] = OpenAICompatibleEmbeddingProvider(
|
||||
api_key=s.openai_api_key, base_url=s.openai_base_url,
|
||||
model=s.openai_embedding_model, dimensions=s.embedding_dimensions,
|
||||
)
|
||||
if s.ollama_enabled:
|
||||
providers["ollama"] = OpenAICompatibleEmbeddingProvider(
|
||||
api_key=None, base_url=s.ollama_base_url,
|
||||
model=s.ollama_embedding_model, dimensions=s.embedding_dimensions,
|
||||
)
|
||||
return providers
|
||||
|
||||
|
||||
def get_embedding_provider(name: str | None = None) -> EmbeddingProvider:
|
||||
providers = build_embedding_providers()
|
||||
return providers.get(name or get_settings().default_embedding_provider) or NullEmbeddingProvider()
|
||||
|
||||
|
||||
EmbeddingProviderDep = Annotated[EmbeddingProvider, Depends(get_embedding_provider)]
|
||||
|
||||
@@ -12,6 +12,7 @@ from sqlalchemy import text
|
||||
|
||||
from app.core.config import get_settings
|
||||
from app.core.db import get_engine
|
||||
from app.core.schema_version import schema_is_current
|
||||
|
||||
router = APIRouter(tags=["health"])
|
||||
|
||||
@@ -33,9 +34,20 @@ async def ready(response: Response) -> dict:
|
||||
try:
|
||||
async with get_engine().connect() as conn:
|
||||
await conn.execute(text("SELECT 1"))
|
||||
checks["database"] = "ok"
|
||||
checks["database"] = "ok"
|
||||
# Schema drift = code ahead of the DB; queries would 500. Fail
|
||||
# readiness loudly rather than serve a broken surface.
|
||||
ok, db, expected = await schema_is_current(conn)
|
||||
if not ok:
|
||||
checks["schema"] = (
|
||||
f"drift: db={sorted(db) or ['none']} expected={sorted(expected)} "
|
||||
"— run 'alembic upgrade head'"
|
||||
)
|
||||
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
|
||||
return {"status": "not ready", "checks": checks}
|
||||
checks["schema"] = "ok"
|
||||
return {"status": "ready", "checks": checks}
|
||||
except Exception as exc: # noqa: BLE001 — surface any failure as "not ready"
|
||||
checks["database"] = "error"
|
||||
checks.setdefault("database", "error")
|
||||
response.status_code = status.HTTP_503_SERVICE_UNAVAILABLE
|
||||
return {"status": "not ready", "checks": checks, "detail": str(exc)}
|
||||
|
||||
@@ -3,12 +3,19 @@
|
||||
from fastapi import APIRouter
|
||||
|
||||
from app.api.v1 import (
|
||||
admin,
|
||||
ai,
|
||||
auth,
|
||||
citations,
|
||||
cleanup,
|
||||
events,
|
||||
gedcom,
|
||||
media,
|
||||
members,
|
||||
names,
|
||||
persons,
|
||||
proposals,
|
||||
public,
|
||||
relationships,
|
||||
sources,
|
||||
trees,
|
||||
@@ -20,9 +27,16 @@ api_router.include_router(auth.router)
|
||||
api_router.include_router(users.router)
|
||||
api_router.include_router(trees.router)
|
||||
api_router.include_router(persons.router)
|
||||
api_router.include_router(names.router)
|
||||
api_router.include_router(events.router)
|
||||
api_router.include_router(relationships.router)
|
||||
api_router.include_router(sources.router)
|
||||
api_router.include_router(citations.router)
|
||||
api_router.include_router(media.router)
|
||||
api_router.include_router(gedcom.router)
|
||||
api_router.include_router(cleanup.router)
|
||||
api_router.include_router(public.router)
|
||||
api_router.include_router(members.router)
|
||||
api_router.include_router(proposals.router)
|
||||
api_router.include_router(ai.router)
|
||||
api_router.include_router(admin.router)
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
"""Instance-admin surface — owner-only (OWNER_EMAIL). Operational status and
|
||||
instance-wide configuration. Deliberately exposes no tree contents or PII:
|
||||
instance ownership is an operator role, not a privacy bypass."""
|
||||
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from fastapi import APIRouter
|
||||
|
||||
from app.api.deps import InstanceOwner, SessionDep, configured_llm_providers
|
||||
from app.core.config import get_settings
|
||||
from app.models.tree import Tree
|
||||
from app.models.user import User
|
||||
from app.schemas.admin import InstanceStatus
|
||||
from app.schemas.ai_policy import ConfiguredProvider
|
||||
|
||||
router = APIRouter(prefix="/admin", tags=["admin"])
|
||||
|
||||
|
||||
@router.get("/instance", response_model=InstanceStatus)
|
||||
async def instance_status(owner: InstanceOwner, session: SessionDep) -> InstanceStatus:
|
||||
"""Operator dashboard data. Requires the caller to be an instance owner."""
|
||||
s = get_settings()
|
||||
user_count = await session.scalar(
|
||||
select(func.count()).select_from(User).where(User.deleted_at.is_(None))
|
||||
)
|
||||
tree_count = await session.scalar(
|
||||
select(func.count()).select_from(Tree).where(Tree.deleted_at.is_(None))
|
||||
)
|
||||
return InstanceStatus(
|
||||
version=s.version,
|
||||
env=s.app_env,
|
||||
owner_emails=sorted(s.owner_emails()),
|
||||
require_email_verification=s.require_email_verification,
|
||||
user_count=user_count or 0,
|
||||
tree_count=tree_count or 0,
|
||||
default_llm_provider=s.default_llm_provider,
|
||||
ai_providers=[ConfiguredProvider(**p) for p in configured_llm_providers()],
|
||||
)
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Per-tree AI model policy — owner-only admin view."""
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter
|
||||
|
||||
from app.api.deps import CurrentUser, SessionDep
|
||||
from app.schemas.ai_policy import TreeAiPolicyRead, TreeAiPolicyUpdate
|
||||
from app.services import ai_policy_service, tree_service
|
||||
|
||||
router = APIRouter(prefix="/trees", tags=["ai"])
|
||||
|
||||
|
||||
@router.get("/{tree_id}/ai", response_model=TreeAiPolicyRead)
|
||||
async def get_ai_policy(
|
||||
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
|
||||
) -> TreeAiPolicyRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
return TreeAiPolicyRead(**await ai_policy_service.get_policy(session, actor=current, tree=tree))
|
||||
|
||||
|
||||
@router.patch("/{tree_id}/ai", response_model=TreeAiPolicyRead)
|
||||
async def update_ai_policy(
|
||||
tree_id: uuid.UUID, data: TreeAiPolicyUpdate, session: SessionDep, current: CurrentUser
|
||||
) -> TreeAiPolicyRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
policy = await ai_policy_service.update_policy(
|
||||
session,
|
||||
actor=current,
|
||||
tree=tree,
|
||||
member_provider=data.member_provider,
|
||||
recommender_provider=data.recommender_provider,
|
||||
)
|
||||
return TreeAiPolicyRead(**policy)
|
||||
@@ -1,9 +1,10 @@
|
||||
from fastapi import APIRouter, HTTPException, Request, Response, status
|
||||
|
||||
from app.api.deps import MailerDep, SessionDep, extract_session_token
|
||||
from app.api.deps import CurrentUser, MailerDep, SessionDep, extract_session_token
|
||||
from app.core.config import get_settings
|
||||
from app.schemas.auth import (
|
||||
LoginRequest,
|
||||
PasswordChange,
|
||||
PasswordResetConfirm,
|
||||
PasswordResetRequest,
|
||||
RegisterRequest,
|
||||
@@ -79,3 +80,15 @@ async def reset_password(data: PasswordResetConfirm, session: SessionDep) -> Non
|
||||
await auth_service.reset_password(
|
||||
session, raw_token=data.token, new_password=data.new_password
|
||||
)
|
||||
|
||||
|
||||
@router.post("/change-password", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def change_password(
|
||||
data: PasswordChange, session: SessionDep, current: CurrentUser
|
||||
) -> None:
|
||||
await auth_service.change_password(
|
||||
session,
|
||||
user=current,
|
||||
current_password=data.current_password,
|
||||
new_password=data.new_password,
|
||||
)
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, File, UploadFile
|
||||
|
||||
from app.api.deps import CurrentUser, SessionDep
|
||||
from app.schemas.cleanup import (
|
||||
CleanupResult,
|
||||
DeceasedApply,
|
||||
DeceasedByChildCandidate,
|
||||
DeceasedCandidate,
|
||||
GenderApply,
|
||||
GenderProposal,
|
||||
NameApply,
|
||||
NameIssue,
|
||||
)
|
||||
from app.services import cleanup_service, tree_service
|
||||
|
||||
router = APIRouter(prefix="/trees", tags=["cleanup"])
|
||||
|
||||
|
||||
@router.get("/{tree_id}/cleanup/deceased", response_model=list[DeceasedCandidate])
|
||||
async def preview_deceased(
|
||||
tree_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
born_on_or_before: int = 1930,
|
||||
) -> list[DeceasedCandidate]:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
rows = await cleanup_service.preview_deceased(
|
||||
session, actor=current, tree=tree, year=born_on_or_before
|
||||
)
|
||||
return [DeceasedCandidate(**r) for r in rows]
|
||||
|
||||
|
||||
@router.get(
|
||||
"/{tree_id}/cleanup/deceased-by-child", response_model=list[DeceasedByChildCandidate]
|
||||
)
|
||||
async def preview_deceased_by_child(
|
||||
tree_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
born_on_or_before: int = 1900,
|
||||
) -> list[DeceasedByChildCandidate]:
|
||||
"""People with a child born on/before the cutoff — necessarily deceased even
|
||||
when their own birth date is missing. Apply via POST .../cleanup/deceased."""
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
rows = await cleanup_service.preview_deceased_by_child(
|
||||
session, actor=current, tree=tree, year=born_on_or_before
|
||||
)
|
||||
return [DeceasedByChildCandidate(**r) for r in rows]
|
||||
|
||||
|
||||
@router.post("/{tree_id}/cleanup/deceased", response_model=CleanupResult)
|
||||
async def apply_deceased(
|
||||
tree_id: uuid.UUID, data: DeceasedApply, session: SessionDep, current: CurrentUser
|
||||
) -> CleanupResult:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
n = await cleanup_service.apply_deceased(
|
||||
session, actor=current, tree=tree, person_ids=data.person_ids
|
||||
)
|
||||
return CleanupResult(updated=n)
|
||||
|
||||
|
||||
@router.post("/{tree_id}/cleanup/gender/preview", response_model=list[GenderProposal])
|
||||
async def preview_gender(
|
||||
tree_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
file: UploadFile = File(...),
|
||||
) -> list[GenderProposal]:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
text = (await file.read()).decode("utf-8", errors="replace")
|
||||
rows = await cleanup_service.preview_gender(
|
||||
session, actor=current, tree=tree, gedcom_text=text
|
||||
)
|
||||
return [GenderProposal(**r) for r in rows]
|
||||
|
||||
|
||||
@router.get("/{tree_id}/cleanup/gender/guess", response_model=list[GenderProposal])
|
||||
async def guess_gender(
|
||||
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
|
||||
) -> list[GenderProposal]:
|
||||
"""Best-guess sex from first names (bundled dictionary) for people missing it."""
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
rows = await cleanup_service.guess_gender_by_name(session, actor=current, tree=tree)
|
||||
return [GenderProposal(**r) for r in rows]
|
||||
|
||||
|
||||
@router.get("/{tree_id}/cleanup/gender/from-spouse", response_model=list[GenderProposal])
|
||||
async def guess_gender_from_spouse(
|
||||
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
|
||||
) -> list[GenderProposal]:
|
||||
"""Infer a missing sex from a partner whose sex is set (opposite-sex couple)."""
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
rows = await cleanup_service.guess_gender_by_spouse(session, actor=current, tree=tree)
|
||||
return [GenderProposal(**r) for r in rows]
|
||||
|
||||
|
||||
@router.post("/{tree_id}/cleanup/gender", response_model=CleanupResult)
|
||||
async def apply_gender(
|
||||
tree_id: uuid.UUID, data: GenderApply, session: SessionDep, current: CurrentUser
|
||||
) -> CleanupResult:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
n = await cleanup_service.apply_gender(
|
||||
session,
|
||||
actor=current,
|
||||
tree=tree,
|
||||
updates=[u.model_dump() for u in data.updates],
|
||||
)
|
||||
return CleanupResult(updated=n)
|
||||
|
||||
|
||||
@router.get("/{tree_id}/cleanup/names", response_model=list[NameIssue])
|
||||
async def preview_names(
|
||||
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
|
||||
) -> list[NameIssue]:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
rows = await cleanup_service.preview_names(session, actor=current, tree=tree)
|
||||
return [NameIssue(**r) for r in rows]
|
||||
|
||||
|
||||
@router.post("/{tree_id}/cleanup/names", response_model=CleanupResult)
|
||||
async def apply_names(
|
||||
tree_id: uuid.UUID, data: NameApply, session: SessionDep, current: CurrentUser
|
||||
) -> CleanupResult:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
n = await cleanup_service.apply_names(
|
||||
session, actor=current, tree=tree, edits=[e.model_dump() for e in data.edits]
|
||||
)
|
||||
return CleanupResult(updated=n)
|
||||
@@ -1,25 +1,56 @@
|
||||
import json
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, File, Response, UploadFile
|
||||
from fastapi import APIRouter, File, Form, Response, UploadFile
|
||||
|
||||
from app.api.deps import CurrentUser, SessionDep
|
||||
from app.schemas.gedcom import ImportReport
|
||||
from app.schemas.gedcom import ImportPreview, ImportReport
|
||||
from app.services import gedcom, tree_service
|
||||
|
||||
router = APIRouter(prefix="/trees", tags=["gedcom"])
|
||||
|
||||
|
||||
@router.post("/{tree_id}/gedcom/preview", response_model=ImportPreview)
|
||||
async def preview_gedcom(
|
||||
tree_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
file: UploadFile = File(...),
|
||||
) -> ImportPreview:
|
||||
"""Dry run: report counts and incoming people that look like duplicates of
|
||||
existing ones, so the user can choose how to resolve each before importing."""
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
text = (await file.read()).decode("utf-8", errors="replace")
|
||||
report = await gedcom.preview_gedcom(session, actor=current, tree=tree, text=text)
|
||||
return ImportPreview(**report)
|
||||
|
||||
|
||||
@router.post("/{tree_id}/gedcom/import", response_model=ImportReport)
|
||||
async def import_gedcom(
|
||||
tree_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
file: UploadFile = File(...),
|
||||
default_action: str = Form("new"),
|
||||
resolutions: str = Form("{}"),
|
||||
) -> ImportReport:
|
||||
# NOTE: additive — records are created as new; existing people are not merged.
|
||||
"""Import a GEDCOM. ``default_action`` (new|skip|merge|overwrite) applies to
|
||||
incoming people that match an existing one; ``resolutions`` is a JSON object
|
||||
{xref: {action, target_id}} overriding it per record."""
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
text = (await file.read()).decode("utf-8", errors="replace")
|
||||
report = await gedcom.import_gedcom(session, actor=current, tree=tree, text=text)
|
||||
try:
|
||||
parsed = json.loads(resolutions or "{}")
|
||||
except json.JSONDecodeError:
|
||||
parsed = {}
|
||||
report = await gedcom.import_gedcom(
|
||||
session,
|
||||
actor=current,
|
||||
tree=tree,
|
||||
text=text,
|
||||
default_action=default_action,
|
||||
resolutions=parsed,
|
||||
)
|
||||
return ImportReport(**report)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Tree membership management endpoints (owner-managed; members can list)."""
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, status
|
||||
|
||||
from app.api.deps import CurrentUser, SessionDep
|
||||
from app.schemas.membership import MemberAdd, MemberRoleUpdate, MembershipRead
|
||||
from app.services import membership_service, tree_service
|
||||
|
||||
router = APIRouter(prefix="/trees", tags=["members"])
|
||||
|
||||
|
||||
@router.get("/{tree_id}/members", response_model=list[MembershipRead])
|
||||
async def list_members(
|
||||
tree_id: uuid.UUID, session: SessionDep, current: CurrentUser
|
||||
) -> list[MembershipRead]:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
rows = await membership_service.list_members(session, viewer_id=current.id, tree=tree)
|
||||
return [MembershipRead(**r) for r in rows]
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{tree_id}/members", response_model=MembershipRead, status_code=status.HTTP_201_CREATED
|
||||
)
|
||||
async def add_member(
|
||||
tree_id: uuid.UUID, data: MemberAdd, session: SessionDep, current: CurrentUser
|
||||
) -> MembershipRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
row = await membership_service.add_member(
|
||||
session, actor=current, tree=tree, email=data.email, role=data.role
|
||||
)
|
||||
return MembershipRead(**row)
|
||||
|
||||
|
||||
@router.patch("/{tree_id}/members/{membership_id}", response_model=MembershipRead)
|
||||
async def update_member(
|
||||
tree_id: uuid.UUID,
|
||||
membership_id: uuid.UUID,
|
||||
data: MemberRoleUpdate,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
) -> MembershipRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
row = await membership_service.update_member_role(
|
||||
session, actor=current, tree=tree, membership_id=membership_id, role=data.role
|
||||
)
|
||||
return MembershipRead(**row)
|
||||
|
||||
|
||||
@router.delete("/{tree_id}/members/{membership_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def remove_member(
|
||||
tree_id: uuid.UUID,
|
||||
membership_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
) -> None:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
await membership_service.remove_member(
|
||||
session, actor=current, tree=tree, membership_id=membership_id
|
||||
)
|
||||
@@ -0,0 +1,90 @@
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, status
|
||||
|
||||
from app.api.deps import CurrentUser, SessionDep
|
||||
from app.schemas.name import NameCreate, NameRead, NameUpdate
|
||||
from app.services import name_service, tree_service
|
||||
|
||||
# Names are nested under their person (which is nested under the tree tenant).
|
||||
router = APIRouter(prefix="/trees", tags=["names"])
|
||||
|
||||
|
||||
@router.get("/{tree_id}/persons/{person_id}/names", response_model=list[NameRead])
|
||||
async def list_names(
|
||||
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
|
||||
) -> list[NameRead]:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
names = await name_service.list_names(
|
||||
session, viewer_id=current.id, tree=tree, person_id=person_id
|
||||
)
|
||||
return [NameRead.model_validate(n) for n in names]
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{tree_id}/persons/{person_id}/names",
|
||||
response_model=NameRead,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
async def create_name(
|
||||
tree_id: uuid.UUID,
|
||||
person_id: uuid.UUID,
|
||||
data: NameCreate,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
) -> NameRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
name = await name_service.create_name(
|
||||
session,
|
||||
actor=current,
|
||||
tree=tree,
|
||||
person_id=person_id,
|
||||
name_type=data.name_type,
|
||||
given=data.given,
|
||||
surname=data.surname,
|
||||
prefix=data.prefix,
|
||||
suffix=data.suffix,
|
||||
nickname=data.nickname,
|
||||
is_primary=data.is_primary,
|
||||
)
|
||||
return NameRead.model_validate(name)
|
||||
|
||||
|
||||
@router.patch(
|
||||
"/{tree_id}/persons/{person_id}/names/{name_id}", response_model=NameRead
|
||||
)
|
||||
async def update_name(
|
||||
tree_id: uuid.UUID,
|
||||
person_id: uuid.UUID,
|
||||
name_id: uuid.UUID,
|
||||
data: NameUpdate,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
) -> NameRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
name = await name_service.update_name(
|
||||
session,
|
||||
actor=current,
|
||||
tree=tree,
|
||||
person_id=person_id,
|
||||
name_id=name_id,
|
||||
changes=data.model_dump(exclude_unset=True),
|
||||
)
|
||||
return NameRead.model_validate(name)
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/{tree_id}/persons/{person_id}/names/{name_id}",
|
||||
status_code=status.HTTP_204_NO_CONTENT,
|
||||
)
|
||||
async def delete_name(
|
||||
tree_id: uuid.UUID,
|
||||
person_id: uuid.UUID,
|
||||
name_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
) -> None:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
await name_service.delete_name(
|
||||
session, actor=current, tree=tree, person_id=person_id, name_id=name_id
|
||||
)
|
||||
@@ -1,6 +1,6 @@
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, status
|
||||
from fastapi import APIRouter, HTTPException, status
|
||||
|
||||
from app.api.deps import CurrentUser, SessionDep
|
||||
from app.schemas.person import PersonCreate, PersonRead, PersonUpdate
|
||||
@@ -41,9 +41,18 @@ async def list_persons(
|
||||
current: CurrentUser,
|
||||
deleted: bool = False,
|
||||
q: str | None = None,
|
||||
ids: str | None = None,
|
||||
) -> list[PersonRead]:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
if q:
|
||||
if ids is not None:
|
||||
try:
|
||||
id_list = [uuid.UUID(x) for x in ids.split(",") if x.strip()]
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "invalid ids") from exc
|
||||
persons = await person_service.list_persons_by_ids(
|
||||
session, viewer_id=current.id, tree=tree, ids=id_list
|
||||
)
|
||||
elif q:
|
||||
persons = await person_service.search_persons(
|
||||
session, viewer_id=current.id, tree=tree, query=q
|
||||
)
|
||||
@@ -75,12 +84,21 @@ async def update_person(
|
||||
return PersonRead.model_validate(person)
|
||||
|
||||
|
||||
@router.delete("/{tree_id}/persons/{person_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||
@router.delete("/{tree_id}/persons/{person_id}")
|
||||
async def delete_person(
|
||||
tree_id: uuid.UUID, person_id: uuid.UUID, session: SessionDep, current: CurrentUser
|
||||
) -> None:
|
||||
tree_id: uuid.UUID,
|
||||
person_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
cascade: bool = False,
|
||||
) -> dict[str, int]:
|
||||
"""Delete a person. ``cascade=true`` also deletes all descendants. Returns
|
||||
the number of persons deleted (1 unless cascading)."""
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
await person_service.delete_person(session, actor=current, tree=tree, person_id=person_id)
|
||||
deleted = await person_service.delete_person(
|
||||
session, actor=current, tree=tree, person_id=person_id, cascade=cascade
|
||||
)
|
||||
return {"deleted": deleted}
|
||||
|
||||
|
||||
@router.post("/{tree_id}/persons/{person_id}/restore", response_model=PersonRead)
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Change-proposal endpoints: list / create / get / apply / reject / delete.
|
||||
|
||||
Applying a proposal is the only way its operations reach the database, and only
|
||||
an editor can do it (enforced in the service). See docs/design/change-proposal.md.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, status
|
||||
|
||||
from app.api.deps import CurrentUser, SessionDep
|
||||
from app.models.enums import ChangeProposalStatus
|
||||
from app.schemas.change_proposal import (
|
||||
ChangeProposalCreate,
|
||||
ChangeProposalRead,
|
||||
ProposalReview,
|
||||
)
|
||||
from app.services import change_proposal_service, tree_service
|
||||
|
||||
router = APIRouter(prefix="/trees", tags=["proposals"])
|
||||
|
||||
|
||||
@router.get("/{tree_id}/proposals", response_model=list[ChangeProposalRead])
|
||||
async def list_proposals(
|
||||
tree_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
status: ChangeProposalStatus | None = None,
|
||||
) -> list[ChangeProposalRead]:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
rows = await change_proposal_service.list_proposals(
|
||||
session, viewer_id=current.id, tree=tree, status=status
|
||||
)
|
||||
return [ChangeProposalRead.model_validate(r) for r in rows]
|
||||
|
||||
|
||||
@router.post(
|
||||
"/{tree_id}/proposals", response_model=ChangeProposalRead, status_code=status.HTTP_201_CREATED
|
||||
)
|
||||
async def create_proposal(
|
||||
tree_id: uuid.UUID, data: ChangeProposalCreate, session: SessionDep, current: CurrentUser
|
||||
) -> ChangeProposalRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
operations = [op.model_dump(mode="json") for op in data.operations]
|
||||
cp = await change_proposal_service.propose(
|
||||
session,
|
||||
tree=tree,
|
||||
origin=data.origin,
|
||||
created_by=current.id,
|
||||
summary=data.summary,
|
||||
rationale=data.rationale,
|
||||
operations=operations,
|
||||
)
|
||||
return ChangeProposalRead.model_validate(cp)
|
||||
|
||||
|
||||
@router.get("/{tree_id}/proposals/{proposal_id}", response_model=ChangeProposalRead)
|
||||
async def get_proposal(
|
||||
tree_id: uuid.UUID, proposal_id: uuid.UUID, session: SessionDep, current: CurrentUser
|
||||
) -> ChangeProposalRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
cp = await change_proposal_service.get_proposal(
|
||||
session, viewer_id=current.id, tree=tree, proposal_id=proposal_id
|
||||
)
|
||||
return ChangeProposalRead.model_validate(cp)
|
||||
|
||||
|
||||
@router.post("/{tree_id}/proposals/{proposal_id}/apply", response_model=ChangeProposalRead)
|
||||
async def apply_proposal(
|
||||
tree_id: uuid.UUID,
|
||||
proposal_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
data: ProposalReview | None = None,
|
||||
) -> ChangeProposalRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
edited = (
|
||||
[op.model_dump(mode="json") for op in data.operations]
|
||||
if data and data.operations is not None
|
||||
else None
|
||||
)
|
||||
cp = await change_proposal_service.apply(
|
||||
session, actor=current, tree=tree, proposal_id=proposal_id, edited_operations=edited
|
||||
)
|
||||
return ChangeProposalRead.model_validate(cp)
|
||||
|
||||
|
||||
@router.post("/{tree_id}/proposals/{proposal_id}/reject", response_model=ChangeProposalRead)
|
||||
async def reject_proposal(
|
||||
tree_id: uuid.UUID,
|
||||
proposal_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
data: ProposalReview | None = None,
|
||||
) -> ChangeProposalRead:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
cp = await change_proposal_service.reject(
|
||||
session,
|
||||
actor=current,
|
||||
tree=tree,
|
||||
proposal_id=proposal_id,
|
||||
note=data.note if data else None,
|
||||
)
|
||||
return ChangeProposalRead.model_validate(cp)
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/{tree_id}/proposals/{proposal_id}", status_code=status.HTTP_204_NO_CONTENT
|
||||
)
|
||||
async def delete_proposal(
|
||||
tree_id: uuid.UUID, proposal_id: uuid.UUID, session: SessionDep, current: CurrentUser
|
||||
) -> None:
|
||||
tree = await tree_service.get_tree(session, viewer_id=current.id, tree_id=tree_id)
|
||||
await change_proposal_service.delete_proposal(
|
||||
session, actor=current, tree=tree, proposal_id=proposal_id
|
||||
)
|
||||
@@ -0,0 +1,135 @@
|
||||
"""Public, read-only viewing surface.
|
||||
|
||||
Optional auth (anonymous allowed). Every response is built by
|
||||
``public_view_service``, which routes through the privacy engine and redacts
|
||||
possibly-living people. No create/update/delete here.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter
|
||||
|
||||
from app.api.deps import CurrentUserOrNone, SessionDep
|
||||
from app.schemas.event import EventRead
|
||||
from app.schemas.name import NameRead
|
||||
from app.schemas.person import PersonRead
|
||||
from app.schemas.relationship import RelationshipRead
|
||||
from app.schemas.tree import PublicTreeRead
|
||||
from app.services import public_view_service
|
||||
|
||||
router = APIRouter(prefix="/public", tags=["public"])
|
||||
|
||||
|
||||
def _vid(viewer: CurrentUserOrNone) -> uuid.UUID | None:
|
||||
return viewer.id if viewer else None
|
||||
|
||||
|
||||
@router.get("/trees", response_model=list[PublicTreeRead])
|
||||
async def public_directory(
|
||||
session: SessionDep,
|
||||
viewer: CurrentUserOrNone,
|
||||
q: str | None = None,
|
||||
limit: int = 50,
|
||||
offset: int = 0,
|
||||
) -> list[PublicTreeRead]:
|
||||
trees = await public_view_service.list_public_trees(
|
||||
session, viewer_id=_vid(viewer), q=q, limit=limit, offset=offset
|
||||
)
|
||||
return [PublicTreeRead.model_validate(t) for t in trees]
|
||||
|
||||
|
||||
@router.get("/trees/{tree_id}", response_model=PublicTreeRead)
|
||||
async def public_tree(
|
||||
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
|
||||
) -> PublicTreeRead:
|
||||
tree = await public_view_service.get_public_tree(
|
||||
session, viewer_id=_vid(viewer), tree_id=tree_id
|
||||
)
|
||||
return PublicTreeRead.model_validate(tree)
|
||||
|
||||
|
||||
@router.get("/trees/{tree_id}/persons", response_model=list[PersonRead])
|
||||
async def public_persons(
|
||||
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
|
||||
) -> list[PersonRead]:
|
||||
tree = await public_view_service.get_public_tree(
|
||||
session, viewer_id=_vid(viewer), tree_id=tree_id
|
||||
)
|
||||
persons = await public_view_service.list_public_persons(
|
||||
session, viewer_id=_vid(viewer), tree=tree
|
||||
)
|
||||
return [PersonRead.model_validate(p) for p in persons]
|
||||
|
||||
|
||||
@router.get("/trees/{tree_id}/relationships", response_model=list[RelationshipRead])
|
||||
async def public_relationships(
|
||||
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
|
||||
) -> list[RelationshipRead]:
|
||||
tree = await public_view_service.get_public_tree(
|
||||
session, viewer_id=_vid(viewer), tree_id=tree_id
|
||||
)
|
||||
rels = await public_view_service.list_public_relationships(
|
||||
session, viewer_id=_vid(viewer), tree=tree
|
||||
)
|
||||
return [RelationshipRead.model_validate(r) for r in rels]
|
||||
|
||||
|
||||
@router.get("/trees/{tree_id}/events", response_model=list[EventRead])
|
||||
async def public_events(
|
||||
tree_id: uuid.UUID, session: SessionDep, viewer: CurrentUserOrNone
|
||||
) -> list[EventRead]:
|
||||
tree = await public_view_service.get_public_tree(
|
||||
session, viewer_id=_vid(viewer), tree_id=tree_id
|
||||
)
|
||||
events = await public_view_service.list_public_events(
|
||||
session, viewer_id=_vid(viewer), tree=tree
|
||||
)
|
||||
return [EventRead.model_validate(e) for e in events]
|
||||
|
||||
|
||||
@router.get("/trees/{tree_id}/persons/{person_id}", response_model=PersonRead)
|
||||
async def public_person(
|
||||
tree_id: uuid.UUID,
|
||||
person_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
viewer: CurrentUserOrNone,
|
||||
) -> PersonRead:
|
||||
tree = await public_view_service.get_public_tree(
|
||||
session, viewer_id=_vid(viewer), tree_id=tree_id
|
||||
)
|
||||
person = await public_view_service.get_public_person(
|
||||
session, viewer_id=_vid(viewer), tree=tree, person_id=person_id
|
||||
)
|
||||
return PersonRead.model_validate(person)
|
||||
|
||||
|
||||
@router.get("/trees/{tree_id}/persons/{person_id}/names", response_model=list[NameRead])
|
||||
async def public_person_names(
|
||||
tree_id: uuid.UUID,
|
||||
person_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
viewer: CurrentUserOrNone,
|
||||
) -> list[NameRead]:
|
||||
tree = await public_view_service.get_public_tree(
|
||||
session, viewer_id=_vid(viewer), tree_id=tree_id
|
||||
)
|
||||
names = await public_view_service.list_public_person_names(
|
||||
session, viewer_id=_vid(viewer), tree=tree, person_id=person_id
|
||||
)
|
||||
return [NameRead.model_validate(n) for n in names]
|
||||
|
||||
|
||||
@router.get("/trees/{tree_id}/persons/{person_id}/events", response_model=list[EventRead])
|
||||
async def public_person_events(
|
||||
tree_id: uuid.UUID,
|
||||
person_id: uuid.UUID,
|
||||
session: SessionDep,
|
||||
viewer: CurrentUserOrNone,
|
||||
) -> list[EventRead]:
|
||||
tree = await public_view_service.get_public_tree(
|
||||
session, viewer_id=_vid(viewer), tree_id=tree_id
|
||||
)
|
||||
events = await public_view_service.list_public_person_events(
|
||||
session, viewer_id=_vid(viewer), tree=tree, person_id=person_id
|
||||
)
|
||||
return [EventRead.model_validate(e) for e in events]
|
||||
@@ -2,8 +2,8 @@ import uuid
|
||||
|
||||
from fastapi import APIRouter, status
|
||||
|
||||
from app.api.deps import CurrentUser, SessionDep
|
||||
from app.schemas.tree import TreeCreate, TreeRead, TreeUpdate
|
||||
from app.api.deps import CurrentUser, ObjectStoreDep, SessionDep
|
||||
from app.schemas.tree import TreeCreate, TreePurge, TreeRead, TreeUpdate
|
||||
from app.services import tree_service
|
||||
|
||||
router = APIRouter(prefix="/trees", tags=["trees"])
|
||||
@@ -57,3 +57,18 @@ async def delete_tree(tree_id: uuid.UUID, session: SessionDep, current: CurrentU
|
||||
async def restore_tree(tree_id: uuid.UUID, session: SessionDep, current: CurrentUser) -> TreeRead:
|
||||
tree = await tree_service.restore_tree(session, actor=current, tree_id=tree_id)
|
||||
return TreeRead.model_validate(tree)
|
||||
|
||||
|
||||
@router.post("/{tree_id}/purge", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def purge_tree(
|
||||
tree_id: uuid.UUID,
|
||||
data: TreePurge,
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
store: ObjectStoreDep,
|
||||
) -> None:
|
||||
"""Permanently delete a soft-deleted tree and all its data — irreversible.
|
||||
Owner-only; the tree must be in the trash and `confirm_name` must match."""
|
||||
await tree_service.purge_tree(
|
||||
session, store, actor=current, tree_id=tree_id, confirm_name=data.confirm_name
|
||||
)
|
||||
|
||||
@@ -1,11 +1,63 @@
|
||||
from fastapi import APIRouter
|
||||
from fastapi import APIRouter, File, Form, Response, UploadFile
|
||||
|
||||
from app.api.deps import CurrentUser
|
||||
from app.schemas.user import UserRead
|
||||
from app.api.deps import CurrentUser, ObjectStoreDep, SessionDep, is_instance_owner
|
||||
from app.schemas.user import UserRead, UserSelfPersonUpdate
|
||||
from app.services import account_service, user_service
|
||||
|
||||
router = APIRouter(prefix="/users", tags=["users"])
|
||||
|
||||
|
||||
def _me(user) -> UserRead:
|
||||
out = UserRead.model_validate(user)
|
||||
out.is_instance_owner = is_instance_owner(user)
|
||||
return out
|
||||
|
||||
|
||||
@router.get("/me", response_model=UserRead)
|
||||
async def read_me(current: CurrentUser) -> UserRead:
|
||||
return UserRead.model_validate(current)
|
||||
return _me(current)
|
||||
|
||||
|
||||
@router.patch("/me/self-person", response_model=UserRead)
|
||||
async def set_self_person(
|
||||
data: UserSelfPersonUpdate, session: SessionDep, current: CurrentUser
|
||||
) -> UserRead:
|
||||
"""Link (or unlink) the Person record that represents this account."""
|
||||
user = await user_service.set_self_person(
|
||||
session, user=current, person_id=data.self_person_id
|
||||
)
|
||||
return _me(user)
|
||||
|
||||
|
||||
@router.get("/me/export")
|
||||
async def export_account(
|
||||
session: SessionDep, current: CurrentUser, store: ObjectStoreDep
|
||||
) -> Response:
|
||||
"""Download a full backup (JSON + media) of every tree the user owns."""
|
||||
data = await account_service.export_account(session, store, user=current)
|
||||
return Response(
|
||||
content=data,
|
||||
media_type="application/zip",
|
||||
headers={"Content-Disposition": 'attachment; filename="provenance-export.zip"'},
|
||||
)
|
||||
|
||||
|
||||
@router.post("/me/import")
|
||||
async def import_account(
|
||||
session: SessionDep,
|
||||
current: CurrentUser,
|
||||
store: ObjectStoreDep,
|
||||
file: UploadFile = File(...),
|
||||
) -> dict:
|
||||
"""Restore a previously-exported backup into new trees (non-destructive)."""
|
||||
raw = await file.read()
|
||||
return await account_service.import_account(session, store, user=current, raw_zip=raw)
|
||||
|
||||
|
||||
@router.delete("/me", status_code=204)
|
||||
async def delete_account(
|
||||
session: SessionDep, current: CurrentUser, confirm_email: str = Form(...)
|
||||
) -> None:
|
||||
"""Delete the account: the user, their owned trees, and their sessions.
|
||||
Requires retyping the account email as a guard."""
|
||||
await account_service.delete_account(session, user=current, confirm_email=confirm_email)
|
||||
|
||||
@@ -22,6 +22,18 @@ class Settings(BaseSettings):
|
||||
version: str = "0.0.0"
|
||||
app_env: str = Field(default="development", description="development | production")
|
||||
|
||||
# --- Instance owner / operator ---
|
||||
# Email(s) of the instance owner(s) — the operator(s) who run this server.
|
||||
# The matching account(s) get instance-admin rights (instance-wide settings;
|
||||
# see /api/v1/admin). Comma-separated for several. Empty = no designated
|
||||
# owner (the instance has no operator account). Derived at request time, so
|
||||
# changing it takes effect immediately with no migration or DB state.
|
||||
owner_email: str = ""
|
||||
|
||||
def owner_emails(self) -> frozenset[str]:
|
||||
"""Normalized (lowercased, trimmed) owner emails; empty if none set."""
|
||||
return frozenset(e.strip().lower() for e in self.owner_email.split(",") if e.strip())
|
||||
|
||||
# SQLAlchemy async URL, e.g. postgresql+asyncpg://user:pass@host:5432/db
|
||||
database_url: str = Field(
|
||||
default="postgresql+asyncpg://provenance:provenance@localhost:5432/provenance",
|
||||
@@ -48,6 +60,11 @@ class Settings(BaseSettings):
|
||||
purge_after_days: int = 30 # soft-deleted rows older than this are purged
|
||||
|
||||
# --- Email (SMTP) ---
|
||||
# When true, a user with no verified email gets no active session (login is
|
||||
# refused and existing sessions stop resolving). Default false so self-hosts
|
||||
# without SMTP — and accounts created before this gate existed — aren't
|
||||
# locked out; operators turn it on once mail works and accounts are verified.
|
||||
require_email_verification: bool = False
|
||||
mailer: str = Field(default="console", description="console | smtp")
|
||||
smtp_host: str | None = None
|
||||
smtp_port: int = 587
|
||||
@@ -55,6 +72,36 @@ class Settings(BaseSettings):
|
||||
smtp_password: str | None = None
|
||||
smtp_from: str = "Provenance <no-reply@provenance.local>"
|
||||
|
||||
# --- Model providers (AI assistant + match-ranking embeddings) ---
|
||||
# Configure as many as you like; each is enabled when its credentials are
|
||||
# present. `default_*_provider` picks which one is used by default. LLM and
|
||||
# embeddings are independent (Anthropic has no embeddings endpoint).
|
||||
default_llm_provider: str = "null" # null | anthropic | openai | xai | ollama
|
||||
default_embedding_provider: str = "null" # null | openai | ollama
|
||||
llm_max_tokens: int = 4096
|
||||
embedding_dimensions: int = 1536 # must match the embedding model + pgvector column
|
||||
|
||||
# Anthropic (LLM only)
|
||||
anthropic_api_key: str | None = None
|
||||
anthropic_model: str = "claude-opus-4-8"
|
||||
|
||||
# OpenAI (LLM + embeddings)
|
||||
openai_api_key: str | None = None
|
||||
openai_base_url: str = "https://api.openai.com/v1"
|
||||
openai_model: str = "gpt-4o"
|
||||
openai_embedding_model: str = "text-embedding-3-small"
|
||||
|
||||
# xAI / Grok — OpenAI-compatible (LLM)
|
||||
xai_api_key: str | None = None
|
||||
xai_base_url: str = "https://api.x.ai/v1"
|
||||
xai_model: str = "grok-2-latest" # set to your account's current Grok model
|
||||
|
||||
# Ollama — local, OpenAI-compatible, no key (LLM + embeddings)
|
||||
ollama_enabled: bool = False
|
||||
ollama_base_url: str = "http://localhost:11434/v1"
|
||||
ollama_model: str = "llama3.1"
|
||||
ollama_embedding_model: str = "nomic-embed-text"
|
||||
|
||||
|
||||
@lru_cache
|
||||
def get_settings() -> Settings:
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Schema-drift detection — a safety net for the deploy pipeline.
|
||||
|
||||
If a deploy ships code whose models reference a column a migration hasn't added
|
||||
yet (the code is ahead of the DB), every query against that table 500s with an
|
||||
opaque ``UndefinedColumnError``. That is exactly the failure that took the tree
|
||||
list down once: the backend image advanced but ``alembic upgrade head`` hadn't
|
||||
run on the server.
|
||||
|
||||
The real prevention is auto-migrate on deploy (the entrypoint runs
|
||||
``alembic upgrade head`` when ``RUN_MIGRATIONS=1``). This module is defense in
|
||||
depth: it makes the drift *loud and explicit* — a readiness failure and a
|
||||
CRITICAL startup log — instead of a silent storm of 500s, so a half-applied
|
||||
deploy is obvious within seconds.
|
||||
"""
|
||||
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import AsyncConnection
|
||||
|
||||
# app/core/schema_version.py -> backend/ (parents: core, app, backend)
|
||||
_MIGRATIONS_DIR = Path(__file__).resolve().parents[2] / "migrations"
|
||||
|
||||
|
||||
@lru_cache
|
||||
def expected_heads() -> frozenset[str]:
|
||||
"""Revision head(s) baked into this image's migration scripts. Static for a
|
||||
given build, so cache it."""
|
||||
from alembic.config import Config
|
||||
from alembic.script import ScriptDirectory
|
||||
|
||||
cfg = Config()
|
||||
cfg.set_main_option("script_location", str(_MIGRATIONS_DIR))
|
||||
return frozenset(ScriptDirectory.from_config(cfg).get_heads())
|
||||
|
||||
|
||||
async def db_heads(conn: AsyncConnection) -> frozenset[str] | None:
|
||||
"""Revision(s) the database is stamped at, or ``None`` when the DB is not
|
||||
Alembic-managed (no ``alembic_version`` table — e.g. a test DB built straight
|
||||
from ``create_all``). ``to_regclass`` returns NULL rather than erroring when
|
||||
the table is absent, so this never poisons the caller's transaction."""
|
||||
if await conn.scalar(text("SELECT to_regclass('public.alembic_version')")) is None:
|
||||
return None
|
||||
result = await conn.execute(text("SELECT version_num FROM alembic_version"))
|
||||
return frozenset(row[0] for row in result)
|
||||
|
||||
|
||||
async def schema_is_current(
|
||||
conn: AsyncConnection,
|
||||
) -> tuple[bool, frozenset[str], frozenset[str]]:
|
||||
"""``(ok, db, expected)``. ``ok`` is True when the DB is stamped at the
|
||||
code's head(s). A DB with no ``alembic_version`` table is treated as current
|
||||
(not Alembic-managed → nothing to compare), so this stays quiet in tests."""
|
||||
expected = expected_heads()
|
||||
current = await db_heads(conn)
|
||||
if current is None:
|
||||
return True, frozenset(), expected
|
||||
return current == expected, current, expected
|
||||
@@ -0,0 +1,24 @@
|
||||
"""Anthropic LLM provider (official SDK). Self-hosters who want everything to
|
||||
stay on their own metal would configure a local provider instead (e.g. Ollama) —
|
||||
that's a future implementation of the same LLMProvider interface."""
|
||||
|
||||
from anthropic import AsyncAnthropic
|
||||
|
||||
from app.integrations.models.base import LLMProvider
|
||||
|
||||
|
||||
class AnthropicLLMProvider(LLMProvider):
|
||||
def __init__(self, *, api_key: str, model: str, max_tokens: int = 4096) -> None:
|
||||
self._client = AsyncAnthropic(api_key=api_key)
|
||||
self._model = model
|
||||
self._max_tokens = max_tokens
|
||||
|
||||
async def complete(self, *, prompt: str, system: str | None = None) -> str:
|
||||
resp = await self._client.messages.create(
|
||||
model=self._model,
|
||||
max_tokens=self._max_tokens,
|
||||
system=system or "",
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
# content is a list of blocks; concatenate the text ones.
|
||||
return "".join(b.text for b in resp.content if b.type == "text")
|
||||
@@ -0,0 +1,36 @@
|
||||
"""Model-provider interfaces — the seam the AI assistant and match ranking plug
|
||||
into. LLM (text) and embeddings are *separate* abstractions: Anthropic offers no
|
||||
embeddings endpoint, so the two are configured independently (twelve-factor,
|
||||
CLAUDE.md #7) and a deployment may run one without the other.
|
||||
|
||||
These providers are read-only text/vector producers. They MUST NOT mutate tree
|
||||
data — the assistant's writes go through a ChangeProposal a human approves
|
||||
(CLAUDE.md #1). Nothing here touches the database.
|
||||
"""
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
|
||||
class LLMProvider(ABC):
|
||||
"""Text in, text out. Implementations wrap a chat/completion model."""
|
||||
|
||||
@abstractmethod
|
||||
async def complete(self, *, prompt: str, system: str | None = None) -> str:
|
||||
"""Return the model's text response to a single user prompt."""
|
||||
...
|
||||
|
||||
|
||||
class EmbeddingProvider(ABC):
|
||||
"""Text in, vectors out — for pgvector-backed match ranking."""
|
||||
|
||||
#: Dimensionality of the returned vectors (for the pgvector column).
|
||||
dimensions: int
|
||||
|
||||
@abstractmethod
|
||||
async def embed(self, texts: list[str]) -> list[list[float]]:
|
||||
"""Return one embedding vector per input text, in order."""
|
||||
...
|
||||
|
||||
|
||||
class ModelProviderNotConfigured(RuntimeError):
|
||||
"""Raised when an AI capability is used but no provider is configured."""
|
||||
@@ -0,0 +1,31 @@
|
||||
"""Default providers when no model backend is configured — AI features are off.
|
||||
|
||||
They fail loudly (rather than silently doing nothing) so a caller that reaches
|
||||
for an unconfigured capability gets a clear, actionable error.
|
||||
"""
|
||||
|
||||
from app.integrations.models.base import (
|
||||
EmbeddingProvider,
|
||||
LLMProvider,
|
||||
ModelProviderNotConfigured,
|
||||
)
|
||||
|
||||
_MSG = (
|
||||
"No model provider configured. Set MODEL_PROVIDER (e.g. 'anthropic') and the "
|
||||
"provider's credentials to enable AI features."
|
||||
)
|
||||
|
||||
|
||||
class NullLLMProvider(LLMProvider):
|
||||
async def complete(self, *, prompt: str, system: str | None = None) -> str:
|
||||
raise ModelProviderNotConfigured(_MSG)
|
||||
|
||||
|
||||
class NullEmbeddingProvider(EmbeddingProvider):
|
||||
dimensions = 0
|
||||
|
||||
async def embed(self, texts: list[str]) -> list[list[float]]:
|
||||
raise ModelProviderNotConfigured(
|
||||
"No embedding provider configured. Set EMBEDDING_PROVIDER and its "
|
||||
"credentials to enable match ranking."
|
||||
)
|
||||
@@ -0,0 +1,40 @@
|
||||
"""OpenAI-compatible providers (one implementation, many vendors).
|
||||
|
||||
OpenAI, xAI (api.x.ai/v1), Ollama (…:11434/v1), OpenRouter, Together, vLLM, etc.
|
||||
all speak the OpenAI Chat Completions / Embeddings API — they differ only by
|
||||
base URL, key, and model name. So a single class, parameterized by those, plugs
|
||||
in every one of them via the official `openai` SDK.
|
||||
"""
|
||||
|
||||
from openai import AsyncOpenAI
|
||||
|
||||
from app.integrations.models.base import EmbeddingProvider, LLMProvider
|
||||
|
||||
|
||||
class OpenAICompatibleLLMProvider(LLMProvider):
|
||||
def __init__(self, *, api_key: str | None, base_url: str, model: str, max_tokens: int = 4096) -> None:
|
||||
# Local backends (Ollama) ignore the key but the SDK requires a non-empty one.
|
||||
self._client = AsyncOpenAI(api_key=api_key or "not-needed", base_url=base_url)
|
||||
self._model = model
|
||||
self._max_tokens = max_tokens
|
||||
|
||||
async def complete(self, *, prompt: str, system: str | None = None) -> str:
|
||||
messages: list[dict] = []
|
||||
if system:
|
||||
messages.append({"role": "system", "content": system})
|
||||
messages.append({"role": "user", "content": prompt})
|
||||
resp = await self._client.chat.completions.create(
|
||||
model=self._model, max_tokens=self._max_tokens, messages=messages
|
||||
)
|
||||
return resp.choices[0].message.content or ""
|
||||
|
||||
|
||||
class OpenAICompatibleEmbeddingProvider(EmbeddingProvider):
|
||||
def __init__(self, *, api_key: str | None, base_url: str, model: str, dimensions: int) -> None:
|
||||
self._client = AsyncOpenAI(api_key=api_key or "not-needed", base_url=base_url)
|
||||
self._model = model
|
||||
self.dimensions = dimensions
|
||||
|
||||
async def embed(self, texts: list[str]) -> list[list[float]]:
|
||||
resp = await self._client.embeddings.create(model=self._model, input=texts)
|
||||
return [d.embedding for d in resp.data]
|
||||
@@ -7,6 +7,7 @@ engine is the single enforcement point for reads.
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.responses import JSONResponse
|
||||
@@ -14,6 +15,8 @@ from fastapi.responses import JSONResponse
|
||||
from app.api.health import router as health_router
|
||||
from app.api.v1 import api_router
|
||||
from app.core.config import get_settings
|
||||
from app.core.db import get_engine
|
||||
from app.core.schema_version import schema_is_current
|
||||
from app.services.exceptions import Conflict, Forbidden, NotFound
|
||||
|
||||
|
||||
@@ -30,6 +33,32 @@ def _configure_logging() -> None:
|
||||
app_logger.propagate = False
|
||||
|
||||
|
||||
async def _check_schema_drift() -> None:
|
||||
"""On startup, shout if the DB schema is behind the code. The entrypoint
|
||||
runs migrations when RUN_MIGRATIONS=1; this catches the case where that
|
||||
didn't happen, so a half-applied deploy is obvious in the logs instead of a
|
||||
silent storm of 500s. Never blocks startup — purely advisory."""
|
||||
logger = logging.getLogger("provenance")
|
||||
try:
|
||||
async with get_engine().connect() as conn:
|
||||
ok, db, expected = await schema_is_current(conn)
|
||||
if not ok:
|
||||
logger.critical(
|
||||
"SCHEMA DRIFT: database is at %s but this build expects %s. "
|
||||
"Run 'alembic upgrade head' — queries will fail until migrated.",
|
||||
sorted(db) or ["none"],
|
||||
sorted(expected),
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001 — advisory only; never block startup
|
||||
logger.warning("schema drift check skipped: %s", exc)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _lifespan(app: FastAPI):
|
||||
await _check_schema_drift()
|
||||
yield
|
||||
|
||||
|
||||
def _register_error_handlers(app: FastAPI) -> None:
|
||||
@app.exception_handler(NotFound)
|
||||
async def _not_found(request: Request, exc: NotFound) -> JSONResponse:
|
||||
@@ -51,6 +80,7 @@ def create_app() -> FastAPI:
|
||||
title=settings.app_name,
|
||||
version=settings.version,
|
||||
description="Provenance API — family and land provenance.",
|
||||
lifespan=_lifespan,
|
||||
)
|
||||
app.include_router(health_router)
|
||||
app.include_router(api_router)
|
||||
|
||||
@@ -4,6 +4,7 @@ and for ``create_all`` in tests."""
|
||||
from app.models.audit import AuditEntry
|
||||
from app.models.auth import Session, UserToken
|
||||
from app.models.base import Base
|
||||
from app.models.change_proposal import ChangeProposal
|
||||
from app.models.event import Event
|
||||
from app.models.media import Media
|
||||
from app.models.person import Name, Person
|
||||
@@ -30,4 +31,5 @@ __all__ = [
|
||||
"Session",
|
||||
"UserToken",
|
||||
"Media",
|
||||
"ChangeProposal",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
"""ChangeProposal — a structured diff the AI assistant (or an untrusted
|
||||
contributor) proposes, which a human approves/edits/rejects. Applying it routes
|
||||
each operation through the normal editing services, so the change passes the
|
||||
privacy engine and is audited as the approving human's action. See
|
||||
docs/design/change-proposal.md and CLAUDE.md non-negotiable #1.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import DateTime, ForeignKey, String, Text
|
||||
from sqlalchemy import Enum as SAEnum
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.models.base import Base
|
||||
from app.models.enums import ChangeProposalOrigin, ChangeProposalStatus
|
||||
from app.models.mixins import SoftDelete, TenantScoped, Timestamps, UUIDPrimaryKey
|
||||
|
||||
|
||||
class ChangeProposal(Base, UUIDPrimaryKey, TenantScoped, Timestamps, SoftDelete):
|
||||
__tablename__ = "change_proposals"
|
||||
|
||||
status: Mapped[ChangeProposalStatus] = mapped_column(
|
||||
SAEnum(ChangeProposalStatus, name="change_proposal_status"),
|
||||
default=ChangeProposalStatus.pending,
|
||||
server_default=ChangeProposalStatus.pending.value,
|
||||
index=True,
|
||||
)
|
||||
origin: Mapped[ChangeProposalOrigin] = mapped_column(
|
||||
SAEnum(ChangeProposalOrigin, name="change_proposal_origin"),
|
||||
default=ChangeProposalOrigin.assistant,
|
||||
server_default=ChangeProposalOrigin.assistant.value,
|
||||
)
|
||||
created_by_user_id: Mapped[uuid.UUID | None] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="SET NULL")
|
||||
)
|
||||
summary: Mapped[str] = mapped_column(String(512))
|
||||
rationale: Mapped[str | None] = mapped_column(Text)
|
||||
# The structured diff: a list of {op, entity_type, entity_id?, payload} dicts.
|
||||
operations: Mapped[list] = mapped_column(JSONB, nullable=False)
|
||||
|
||||
reviewed_by_user_id: Mapped[uuid.UUID | None] = mapped_column(
|
||||
ForeignKey("users.id", ondelete="SET NULL")
|
||||
)
|
||||
reviewed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
review_note: Mapped[str | None] = mapped_column(String(512))
|
||||
apply_error: Mapped[str | None] = mapped_column(Text)
|
||||
@@ -9,9 +9,10 @@ import enum
|
||||
|
||||
|
||||
class TreeVisibility(enum.StrEnum):
|
||||
public = "public"
|
||||
unlisted = "unlisted"
|
||||
private = "private"
|
||||
public = "public" # anyone on the web (anonymous), listed + search-indexable
|
||||
site_members = "site_members" # any authenticated user of this instance
|
||||
unlisted = "unlisted" # anyone with the link (anonymous), not listed/indexed
|
||||
private = "private" # members only (default)
|
||||
|
||||
|
||||
class MembershipRole(enum.StrEnum):
|
||||
@@ -60,3 +61,14 @@ class AuditActorType(enum.StrEnum):
|
||||
class TokenPurpose(enum.StrEnum):
|
||||
email_verify = "email_verify"
|
||||
password_reset = "password_reset"
|
||||
|
||||
|
||||
class ChangeProposalStatus(enum.StrEnum):
|
||||
pending = "pending"
|
||||
applied = "applied"
|
||||
rejected = "rejected"
|
||||
|
||||
|
||||
class ChangeProposalOrigin(enum.StrEnum):
|
||||
assistant = "assistant" # the AI assistant, acting on behalf of a user
|
||||
contributor = "contributor" # an untrusted human edit awaiting moderation
|
||||
|
||||
@@ -26,6 +26,21 @@ class Tree(Base, UUIDPrimaryKey, Timestamps, SoftDelete):
|
||||
default=TreeVisibility.private,
|
||||
server_default=TreeVisibility.private.value,
|
||||
)
|
||||
# The person a tree opens focused on (its "home"/root person). Cleared if
|
||||
# that person is deleted. use_alter + name: trees<->persons form an FK cycle.
|
||||
home_person_id: Mapped[uuid.UUID | None] = mapped_column(
|
||||
ForeignKey(
|
||||
"persons.id",
|
||||
ondelete="SET NULL",
|
||||
name="fk_trees_home_person_id",
|
||||
use_alter=True,
|
||||
)
|
||||
)
|
||||
# Per-tree AI model policy (owner-configured). The names reference configured
|
||||
# providers from the registry; null = that role has no model. The owner may
|
||||
# use any configured provider; these limit members + the recommender.
|
||||
ai_member_provider: Mapped[str | None] = mapped_column(String(32))
|
||||
ai_recommender_provider: Mapped[str | None] = mapped_column(String(32))
|
||||
|
||||
|
||||
class TreeMembership(Base, UUIDPrimaryKey, Timestamps):
|
||||
|
||||
@@ -3,9 +3,10 @@ multiple auth providers later (the provider-link table arrives with the auth
|
||||
slice). ``hashed_password`` is nullable: external/OIDC users have none.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import DateTime, String
|
||||
from sqlalchemy import DateTime, ForeignKey, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.models.base import Base
|
||||
@@ -19,3 +20,15 @@ class User(Base, UUIDPrimaryKey, Timestamps, SoftDelete):
|
||||
email_verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
|
||||
display_name: Mapped[str | None] = mapped_column(String(255))
|
||||
hashed_password: Mapped[str | None] = mapped_column(String(255))
|
||||
# The Person record that *is* this user ("home person"). Cleared if that
|
||||
# person is deleted, so the link can never dangle.
|
||||
self_person_id: Mapped[uuid.UUID | None] = mapped_column(
|
||||
# use_alter + explicit name: users<->persons<->trees form an FK cycle,
|
||||
# so this constraint must be created/dropped via ALTER, not inline.
|
||||
ForeignKey(
|
||||
"persons.id",
|
||||
ondelete="SET NULL",
|
||||
name="fk_users_self_person_id",
|
||||
use_alter=True,
|
||||
)
|
||||
)
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
"""Instance-admin schemas. Operator-facing, owner-only — operational status and
|
||||
config, never tree data or PII (instance ownership doesn't bypass privacy)."""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.schemas.ai_policy import ConfiguredProvider
|
||||
|
||||
|
||||
class InstanceStatus(BaseModel):
|
||||
version: str
|
||||
env: str
|
||||
# Operator account(s) — the email(s) named in OWNER_EMAIL.
|
||||
owner_emails: list[str]
|
||||
require_email_verification: bool
|
||||
# Aggregate, non-identifying counts (live rows only).
|
||||
user_count: int
|
||||
tree_count: int
|
||||
# Instance-wide AI configuration (no secrets).
|
||||
default_llm_provider: str
|
||||
ai_providers: list[ConfiguredProvider]
|
||||
@@ -0,0 +1,22 @@
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class ConfiguredProvider(BaseModel):
|
||||
name: str
|
||||
model: str
|
||||
|
||||
|
||||
class TreeAiPolicyRead(BaseModel):
|
||||
# The model non-owners' assistant uses (null = none).
|
||||
member_provider: str | None
|
||||
# The model the association/recommendation engine uses (null = none).
|
||||
recommender_provider: str | None
|
||||
# Providers the operator has configured (from env). The owner may use any of
|
||||
# these; the two settings above restrict members and the recommender to one.
|
||||
configured_providers: list[ConfiguredProvider]
|
||||
default_provider: str
|
||||
|
||||
|
||||
class TreeAiPolicyUpdate(BaseModel):
|
||||
member_provider: str | None = None
|
||||
recommender_provider: str | None = None
|
||||
@@ -29,6 +29,11 @@ class PasswordResetConfirm(BaseModel):
|
||||
new_password: str = Field(min_length=8)
|
||||
|
||||
|
||||
class PasswordChange(BaseModel):
|
||||
current_password: str
|
||||
new_password: str = Field(min_length=8)
|
||||
|
||||
|
||||
class SessionRead(BaseModel):
|
||||
user: UserRead
|
||||
token: str
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
from app.models.enums import ChangeProposalOrigin, ChangeProposalStatus
|
||||
|
||||
|
||||
class ProposalOperation(BaseModel):
|
||||
op: str # create | update | delete
|
||||
entity_type: str # person | name | event | relationship | source | citation
|
||||
entity_id: uuid.UUID | None = None
|
||||
payload: dict = {}
|
||||
|
||||
|
||||
class ChangeProposalCreate(BaseModel):
|
||||
summary: str
|
||||
rationale: str | None = None
|
||||
origin: ChangeProposalOrigin = ChangeProposalOrigin.contributor
|
||||
operations: list[ProposalOperation]
|
||||
|
||||
|
||||
class ProposalReview(BaseModel):
|
||||
note: str | None = None
|
||||
# Optional edited operations to apply instead of the original (approve-with-edits).
|
||||
operations: list[ProposalOperation] | None = None
|
||||
|
||||
|
||||
class ChangeProposalRead(BaseModel):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: uuid.UUID
|
||||
tree_id: uuid.UUID
|
||||
status: ChangeProposalStatus
|
||||
origin: ChangeProposalOrigin
|
||||
created_by_user_id: uuid.UUID | None
|
||||
summary: str
|
||||
rationale: str | None
|
||||
operations: list
|
||||
reviewed_by_user_id: uuid.UUID | None
|
||||
reviewed_at: datetime | None
|
||||
review_note: str | None
|
||||
apply_error: str | None
|
||||
created_at: datetime
|
||||
@@ -0,0 +1,56 @@
|
||||
import uuid
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class DeceasedCandidate(BaseModel):
|
||||
person_id: uuid.UUID
|
||||
name: str
|
||||
birth_year: int
|
||||
|
||||
|
||||
class DeceasedByChildCandidate(BaseModel):
|
||||
person_id: uuid.UUID
|
||||
name: str
|
||||
child_birth_year: int
|
||||
|
||||
|
||||
class DeceasedApply(BaseModel):
|
||||
person_ids: list[uuid.UUID]
|
||||
|
||||
|
||||
class GenderProposal(BaseModel):
|
||||
person_id: uuid.UUID
|
||||
name: str
|
||||
proposed_gender: str
|
||||
|
||||
|
||||
class GenderUpdate(BaseModel):
|
||||
person_id: uuid.UUID
|
||||
gender: str
|
||||
|
||||
|
||||
class GenderApply(BaseModel):
|
||||
updates: list[GenderUpdate]
|
||||
|
||||
|
||||
class NameIssue(BaseModel):
|
||||
name_id: uuid.UUID
|
||||
person_id: uuid.UUID
|
||||
given: str | None = None
|
||||
surname: str | None = None
|
||||
issue: str
|
||||
|
||||
|
||||
class NameEdit(BaseModel):
|
||||
name_id: uuid.UUID
|
||||
given: str | None = None
|
||||
surname: str | None = None
|
||||
|
||||
|
||||
class NameApply(BaseModel):
|
||||
edits: list[NameEdit]
|
||||
|
||||
|
||||
class CleanupResult(BaseModel):
|
||||
updated: int
|
||||
@@ -1,6 +1,25 @@
|
||||
import uuid
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class ImportReport(BaseModel):
|
||||
counts: dict[str, int]
|
||||
unmapped_tags: list[str]
|
||||
|
||||
|
||||
class DuplicateMatch(BaseModel):
|
||||
# An incoming GEDCOM person that resembles an existing one in the tree.
|
||||
xref: str
|
||||
incoming_name: str
|
||||
incoming_birth_year: str | None = None
|
||||
existing_person_id: uuid.UUID
|
||||
existing_name: str
|
||||
existing_birth_year: str | None = None
|
||||
score: str # "high" | "medium"
|
||||
|
||||
|
||||
class ImportPreview(BaseModel):
|
||||
counts: dict[str, int]
|
||||
potential_duplicates: list[DuplicateMatch]
|
||||
unmapped_tags: list[str]
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
from app.models.enums import MembershipRole
|
||||
|
||||
|
||||
class MembershipRead(BaseModel):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: uuid.UUID
|
||||
user_id: uuid.UUID
|
||||
email: str
|
||||
display_name: str | None
|
||||
role: MembershipRole
|
||||
created_at: datetime
|
||||
|
||||
|
||||
class MemberAdd(BaseModel):
|
||||
email: str
|
||||
role: MembershipRole = MembershipRole.viewer
|
||||
|
||||
|
||||
class MemberRoleUpdate(BaseModel):
|
||||
role: MembershipRole
|
||||
@@ -0,0 +1,42 @@
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
|
||||
|
||||
class NameCreate(BaseModel):
|
||||
# Open vocabulary: birth/maiden, married, alias, religious, nickname, ...
|
||||
name_type: str = "birth"
|
||||
given: str | None = None
|
||||
surname: str | None = None
|
||||
prefix: str | None = None
|
||||
suffix: str | None = None
|
||||
nickname: str | None = None
|
||||
is_primary: bool = False
|
||||
|
||||
|
||||
class NameUpdate(BaseModel):
|
||||
name_type: str | None = None
|
||||
given: str | None = None
|
||||
surname: str | None = None
|
||||
prefix: str | None = None
|
||||
suffix: str | None = None
|
||||
nickname: str | None = None
|
||||
is_primary: bool | None = None
|
||||
|
||||
|
||||
class NameRead(BaseModel):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: uuid.UUID
|
||||
tree_id: uuid.UUID
|
||||
person_id: uuid.UUID
|
||||
name_type: str
|
||||
given: str | None
|
||||
surname: str | None
|
||||
prefix: str | None
|
||||
suffix: str | None
|
||||
nickname: str | None
|
||||
is_primary: bool
|
||||
sort_order: int
|
||||
created_at: datetime
|
||||
@@ -16,6 +16,12 @@ class TreeUpdate(BaseModel):
|
||||
name: str | None = None
|
||||
description: str | None = None
|
||||
visibility: TreeVisibility | None = None
|
||||
home_person_id: uuid.UUID | None = None
|
||||
|
||||
|
||||
class TreePurge(BaseModel):
|
||||
# Retype the tree's name to confirm a permanent, irreversible delete.
|
||||
confirm_name: str
|
||||
|
||||
|
||||
class TreeRead(BaseModel):
|
||||
@@ -26,4 +32,18 @@ class TreeRead(BaseModel):
|
||||
description: str | None
|
||||
visibility: TreeVisibility
|
||||
owner_id: uuid.UUID
|
||||
home_person_id: uuid.UUID | None = None
|
||||
created_at: datetime
|
||||
|
||||
|
||||
class PublicTreeRead(BaseModel):
|
||||
"""Tree projection for the public surface — deliberately omits owner_id so a
|
||||
public/unlisted tree doesn't reveal which account owns it."""
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: uuid.UUID
|
||||
name: str
|
||||
description: str | None
|
||||
visibility: TreeVisibility
|
||||
home_person_id: uuid.UUID | None = None
|
||||
|
||||
@@ -19,4 +19,13 @@ class UserRead(BaseModel):
|
||||
email: str
|
||||
display_name: str | None
|
||||
email_verified_at: datetime | None
|
||||
self_person_id: uuid.UUID | None = None
|
||||
created_at: datetime
|
||||
# Operational role, not a DB column: true when this account's email is named
|
||||
# in OWNER_EMAIL. Set by the API layer (see users.read_me).
|
||||
is_instance_owner: bool = False
|
||||
|
||||
|
||||
class UserSelfPersonUpdate(BaseModel):
|
||||
# null clears the link; otherwise the Person that represents this account.
|
||||
self_person_id: uuid.UUID | None = None
|
||||
|
||||
@@ -0,0 +1,352 @@
|
||||
"""Account-level data portability: export the signed-in user's owned trees as a
|
||||
zip (JSON + media bytes), restore such a zip into a brand-new tree
|
||||
(non-destructive), and delete the account.
|
||||
|
||||
The export format is a zip containing ``account.json`` plus ``media/<id>`` blobs.
|
||||
Restore always creates new trees and remaps ids, so it can't clobber existing
|
||||
data.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import uuid
|
||||
import zipfile
|
||||
from datetime import UTC, date, datetime
|
||||
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.integrations.objectstore.base import ObjectStore
|
||||
from app.models.auth import Session as SessionModel
|
||||
from app.models.enums import MembershipRole
|
||||
from app.models.event import Event
|
||||
from app.models.media import Media
|
||||
from app.models.person import Name, Person
|
||||
from app.models.place import Place
|
||||
from app.models.relationship import Relationship
|
||||
from app.models.source import Citation, Source
|
||||
from app.models.tree import Tree, TreeMembership
|
||||
from app.models.user import User
|
||||
from app.services.audit import record_audit
|
||||
from app.services.exceptions import Forbidden, NotFound
|
||||
|
||||
EXPORT_VERSION = 1
|
||||
_DROP = {"created_at", "updated_at", "deleted_at", "tree_id"}
|
||||
# Media columns rebuilt on import (storage is re-keyed, checksum recomputed).
|
||||
_MEDIA_DROP = _DROP | {"uploader_id", "storage_key", "byte_size", "checksum_sha256"}
|
||||
_DATE_FIELDS = {"date_start", "date_end"}
|
||||
|
||||
|
||||
def _row(obj, drop: set[str]) -> dict:
|
||||
out: dict = {}
|
||||
for col in obj.__table__.columns.keys(): # noqa: SIM118
|
||||
if col in drop:
|
||||
continue
|
||||
out[col] = getattr(obj, col)
|
||||
return out
|
||||
|
||||
|
||||
async def _entities(session: AsyncSession, model, tree_id: uuid.UUID):
|
||||
stmt = select(model).where(model.tree_id == tree_id, model.deleted_at.is_(None))
|
||||
return list((await session.execute(stmt)).scalars().all())
|
||||
|
||||
|
||||
async def export_account(session: AsyncSession, store: ObjectStore, *, user: User) -> bytes:
|
||||
"""Build a zip of every tree the user owns: account.json + media blobs."""
|
||||
trees = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Tree).where(Tree.owner_id == user.id, Tree.deleted_at.is_(None))
|
||||
)
|
||||
).scalars().all()
|
||||
)
|
||||
|
||||
payload: dict = {
|
||||
"version": EXPORT_VERSION,
|
||||
"user": {"email": user.email, "display_name": user.display_name},
|
||||
"trees": [],
|
||||
}
|
||||
media_blobs: list[tuple[str, bytes]] = []
|
||||
|
||||
for tree in trees:
|
||||
media_rows = await _entities(session, Media, tree.id)
|
||||
media_out = []
|
||||
for m in media_rows:
|
||||
ref = f"media/{m.id}"
|
||||
rec = _row(m, _MEDIA_DROP)
|
||||
rec["_file"] = ref
|
||||
media_out.append(rec)
|
||||
try:
|
||||
media_blobs.append((ref, await store.get_object(key=m.storage_key)))
|
||||
except Exception: # noqa: BLE001 — a missing blob shouldn't abort the export
|
||||
rec["_file"] = None
|
||||
|
||||
payload["trees"].append({
|
||||
"tree": {
|
||||
"name": tree.name,
|
||||
"description": tree.description,
|
||||
"visibility": tree.visibility,
|
||||
"home_person_id": tree.home_person_id,
|
||||
},
|
||||
"places": [_row(p, _DROP) for p in await _entities(session, Place, tree.id)],
|
||||
"persons": [_row(p, _DROP) for p in await _entities(session, Person, tree.id)],
|
||||
"names": [_row(n, _DROP) for n in await _entities(session, Name, tree.id)],
|
||||
"relationships": [
|
||||
_row(r, _DROP) for r in await _entities(session, Relationship, tree.id)
|
||||
],
|
||||
"events": [_row(e, _DROP) for e in await _entities(session, Event, tree.id)],
|
||||
"sources": [_row(s, _DROP) for s in await _entities(session, Source, tree.id)],
|
||||
"citations": [_row(c, _DROP) for c in await _entities(session, Citation, tree.id)],
|
||||
"media": media_out,
|
||||
})
|
||||
|
||||
buf = io.BytesIO()
|
||||
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
|
||||
zf.writestr("account.json", json.dumps(payload, default=str, indent=2))
|
||||
for ref, blob in media_blobs:
|
||||
zf.writestr(ref, blob)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def _as_uuid(v) -> uuid.UUID | None:
|
||||
return uuid.UUID(v) if v else None
|
||||
|
||||
|
||||
def _as_date(v) -> date | None:
|
||||
return date.fromisoformat(v) if v else None
|
||||
|
||||
|
||||
async def import_account(
|
||||
session: AsyncSession, store: ObjectStore, *, user: User, raw_zip: bytes
|
||||
) -> dict:
|
||||
"""Restore an exported zip into NEW trees owned by the user. Non-destructive:
|
||||
every record gets a fresh id; nothing existing is touched."""
|
||||
try:
|
||||
zf = zipfile.ZipFile(io.BytesIO(raw_zip))
|
||||
payload = json.loads(zf.read("account.json"))
|
||||
except (zipfile.BadZipFile, KeyError, json.JSONDecodeError) as e:
|
||||
raise NotFound("not a valid Provenance export") from e
|
||||
|
||||
counts: dict[str, int] = {"trees": 0, "persons": 0, "events": 0, "media": 0}
|
||||
|
||||
for tdata in payload.get("trees", []):
|
||||
t = tdata.get("tree", {})
|
||||
tree = Tree(
|
||||
owner_id=user.id,
|
||||
name=(t.get("name") or "Imported tree"),
|
||||
description=t.get("description"),
|
||||
visibility=t.get("visibility") or "private",
|
||||
)
|
||||
session.add(tree)
|
||||
await session.flush()
|
||||
session.add(
|
||||
TreeMembership(tree_id=tree.id, user_id=user.id, role=MembershipRole.owner)
|
||||
)
|
||||
counts["trees"] += 1
|
||||
|
||||
# id remaps from the export's ids to the freshly created ones.
|
||||
pmap: dict[str, uuid.UUID] = {}
|
||||
rmap: dict[str, uuid.UUID] = {}
|
||||
smap: dict[str, uuid.UUID] = {}
|
||||
nmap: dict[str, uuid.UUID] = {}
|
||||
emap: dict[str, uuid.UUID] = {}
|
||||
plmap: dict[str, uuid.UUID] = {}
|
||||
|
||||
for pl in tdata.get("places", []):
|
||||
obj = Place(
|
||||
tree_id=tree.id,
|
||||
name=pl.get("name") or "",
|
||||
place_type=pl.get("place_type"),
|
||||
latitude=pl.get("latitude"),
|
||||
longitude=pl.get("longitude"),
|
||||
)
|
||||
session.add(obj)
|
||||
await session.flush()
|
||||
plmap[pl["id"]] = obj.id
|
||||
|
||||
for p in tdata.get("persons", []):
|
||||
obj = Person(
|
||||
tree_id=tree.id,
|
||||
gender=p.get("gender"),
|
||||
is_living=p.get("is_living"),
|
||||
privacy=p.get("privacy") or "inherit",
|
||||
notes=p.get("notes"),
|
||||
)
|
||||
session.add(obj)
|
||||
await session.flush()
|
||||
pmap[p["id"]] = obj.id
|
||||
counts["persons"] += 1
|
||||
|
||||
for n in tdata.get("names", []):
|
||||
pid = pmap.get(n.get("person_id"))
|
||||
if pid is None:
|
||||
continue
|
||||
obj = Name(
|
||||
tree_id=tree.id,
|
||||
person_id=pid,
|
||||
name_type=n.get("name_type") or "birth",
|
||||
given=n.get("given"),
|
||||
surname=n.get("surname"),
|
||||
prefix=n.get("prefix"),
|
||||
suffix=n.get("suffix"),
|
||||
nickname=n.get("nickname"),
|
||||
display_name=n.get("display_name"),
|
||||
is_primary=bool(n.get("is_primary")),
|
||||
sort_order=n.get("sort_order") or 0,
|
||||
)
|
||||
session.add(obj)
|
||||
await session.flush()
|
||||
nmap[n["id"]] = obj.id
|
||||
|
||||
for r in tdata.get("relationships", []):
|
||||
a = pmap.get(r.get("person_from_id"))
|
||||
b = pmap.get(r.get("person_to_id"))
|
||||
if a is None or b is None:
|
||||
continue
|
||||
obj = Relationship(
|
||||
tree_id=tree.id,
|
||||
type=r.get("type"),
|
||||
person_from_id=a,
|
||||
person_to_id=b,
|
||||
qualifier=r.get("qualifier"),
|
||||
notes=r.get("notes"),
|
||||
)
|
||||
session.add(obj)
|
||||
await session.flush()
|
||||
rmap[r["id"]] = obj.id
|
||||
|
||||
for e in tdata.get("events", []):
|
||||
obj = Event(
|
||||
tree_id=tree.id,
|
||||
event_type=e.get("event_type") or "other",
|
||||
person_id=pmap.get(e.get("person_id")),
|
||||
relationship_id=rmap.get(e.get("relationship_id")),
|
||||
place_id=plmap.get(e.get("place_id")),
|
||||
date_value=e.get("date_value"),
|
||||
date_start=_as_date(e.get("date_start")),
|
||||
date_end=_as_date(e.get("date_end")),
|
||||
date_precision=e.get("date_precision"),
|
||||
calendar=e.get("calendar") or "gregorian",
|
||||
detail=e.get("detail"),
|
||||
notes=e.get("notes"),
|
||||
)
|
||||
session.add(obj)
|
||||
await session.flush()
|
||||
emap[e["id"]] = obj.id
|
||||
counts["events"] += 1
|
||||
|
||||
for s in tdata.get("sources", []):
|
||||
obj = Source(
|
||||
tree_id=tree.id,
|
||||
title=s.get("title") or "Untitled source",
|
||||
author=s.get("author"),
|
||||
source_type=s.get("source_type"),
|
||||
repository=s.get("repository"),
|
||||
url=s.get("url"),
|
||||
citation_text=s.get("citation_text"),
|
||||
publication_info=s.get("publication_info"),
|
||||
quality_note=s.get("quality_note"),
|
||||
)
|
||||
session.add(obj)
|
||||
await session.flush()
|
||||
smap[s["id"]] = obj.id
|
||||
|
||||
for c in tdata.get("citations", []):
|
||||
sid = smap.get(c.get("source_id"))
|
||||
if sid is None:
|
||||
continue
|
||||
session.add(
|
||||
Citation(
|
||||
tree_id=tree.id,
|
||||
source_id=sid,
|
||||
person_id=pmap.get(c.get("person_id")),
|
||||
event_id=emap.get(c.get("event_id")),
|
||||
name_id=nmap.get(c.get("name_id")),
|
||||
relationship_id=rmap.get(c.get("relationship_id")),
|
||||
page=c.get("page"),
|
||||
detail=c.get("detail"),
|
||||
confidence=c.get("confidence"),
|
||||
)
|
||||
)
|
||||
|
||||
for m in tdata.get("media", []):
|
||||
ref = m.get("_file")
|
||||
if not ref:
|
||||
continue
|
||||
try:
|
||||
blob = zf.read(ref)
|
||||
except KeyError:
|
||||
continue
|
||||
media_id = uuid.uuid4()
|
||||
filename = m.get("original_filename") or "upload"
|
||||
key = f"{tree.id}/{media_id}/{filename}"
|
||||
await store.ensure_bucket()
|
||||
await store.put_object(
|
||||
key=key,
|
||||
data=blob,
|
||||
content_type=m.get("content_type") or "application/octet-stream",
|
||||
)
|
||||
session.add(
|
||||
Media(
|
||||
id=media_id,
|
||||
tree_id=tree.id,
|
||||
uploader_id=user.id,
|
||||
storage_key=key,
|
||||
original_filename=filename,
|
||||
content_type=m.get("content_type") or "application/octet-stream",
|
||||
byte_size=len(blob),
|
||||
checksum_sha256=hashlib.sha256(blob).hexdigest(),
|
||||
title=m.get("title"),
|
||||
person_id=pmap.get(m.get("person_id")),
|
||||
event_id=emap.get(m.get("event_id")),
|
||||
source_id=smap.get(m.get("source_id")),
|
||||
)
|
||||
)
|
||||
counts["media"] += 1
|
||||
|
||||
# Remap the home person last, once persons exist.
|
||||
home = t.get("home_person_id")
|
||||
if home and home in pmap:
|
||||
tree.home_person_id = pmap[home]
|
||||
|
||||
record_audit(
|
||||
session,
|
||||
action="import",
|
||||
entity_type="Account",
|
||||
entity_id=tree.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=user.id,
|
||||
after=counts,
|
||||
)
|
||||
|
||||
await session.commit()
|
||||
return counts
|
||||
|
||||
|
||||
async def delete_account(session: AsyncSession, *, user: User, confirm_email: str) -> None:
|
||||
"""Soft-delete the account: the user, the trees they own, and all their
|
||||
sessions. Requires the user to retype their email as a guard."""
|
||||
if confirm_email.strip().lower() != user.email.lower():
|
||||
raise Forbidden("email confirmation does not match")
|
||||
now = datetime.now(UTC)
|
||||
|
||||
await session.execute(
|
||||
update(Tree)
|
||||
.where(Tree.owner_id == user.id, Tree.deleted_at.is_(None))
|
||||
.values(deleted_at=now)
|
||||
)
|
||||
await session.execute(
|
||||
update(SessionModel)
|
||||
.where(SessionModel.user_id == user.id, SessionModel.revoked_at.is_(None))
|
||||
.values(revoked_at=now)
|
||||
)
|
||||
user.deleted_at = now
|
||||
record_audit(
|
||||
session,
|
||||
action="delete",
|
||||
entity_type="User",
|
||||
entity_id=user.id,
|
||||
actor_user_id=user.id,
|
||||
)
|
||||
await session.commit()
|
||||
@@ -0,0 +1,77 @@
|
||||
"""Per-tree AI model policy — owner-only. Assigns which configured provider
|
||||
members and the recommender use; the owner may use any configured provider.
|
||||
|
||||
The operator decides which providers exist (env / registry); the tree owner
|
||||
decides who uses which. See app/api/deps.py for the registry.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.api.deps import configured_llm_providers
|
||||
from app.models.enums import MembershipRole
|
||||
from app.models.tree import Tree
|
||||
from app.models.user import User
|
||||
from app.services import privacy
|
||||
from app.services.exceptions import Forbidden
|
||||
|
||||
|
||||
async def _require_owner(session: AsyncSession, *, actor: User, tree: Tree) -> None:
|
||||
role = await privacy.get_membership_role(session, actor.id, tree.id)
|
||||
if role is not MembershipRole.owner:
|
||||
raise Forbidden("only the tree owner can configure AI")
|
||||
|
||||
|
||||
def _names() -> set[str]:
|
||||
return {p["name"] for p in configured_llm_providers()}
|
||||
|
||||
|
||||
async def get_policy(session: AsyncSession, *, actor: User, tree: Tree) -> dict:
|
||||
await _require_owner(session, actor=actor, tree=tree)
|
||||
from app.core.config import get_settings
|
||||
|
||||
return {
|
||||
"member_provider": tree.ai_member_provider,
|
||||
"recommender_provider": tree.ai_recommender_provider,
|
||||
"configured_providers": configured_llm_providers(),
|
||||
"default_provider": get_settings().default_llm_provider,
|
||||
}
|
||||
|
||||
|
||||
async def update_policy(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
actor: User,
|
||||
tree: Tree,
|
||||
member_provider: str | None,
|
||||
recommender_provider: str | None,
|
||||
) -> dict:
|
||||
await _require_owner(session, actor=actor, tree=tree)
|
||||
valid = _names()
|
||||
for value in (member_provider, recommender_provider):
|
||||
if value is not None and value not in valid:
|
||||
raise Forbidden(f"'{value}' is not a configured provider")
|
||||
tree.ai_member_provider = member_provider
|
||||
tree.ai_recommender_provider = recommender_provider
|
||||
await session.commit()
|
||||
await session.refresh(tree)
|
||||
return await get_policy(session, actor=actor, tree=tree)
|
||||
|
||||
|
||||
# --- Resolution helpers (for the future assistant / recommender) -------------
|
||||
|
||||
def provider_name_for_member(tree: Tree) -> str | None:
|
||||
"""Provider an ordinary member's assistant should use, if any."""
|
||||
return tree.ai_member_provider
|
||||
|
||||
|
||||
def provider_name_for_recommender(tree: Tree) -> str | None:
|
||||
return tree.ai_recommender_provider
|
||||
|
||||
|
||||
def provider_name_for_owner(tree: Tree, requested: str | None = None) -> str | None:
|
||||
"""The owner may use any configured provider; default to the requested one."""
|
||||
if requested and requested in _names():
|
||||
return requested
|
||||
return tree.ai_member_provider # fall back to the member model
|
||||
@@ -3,6 +3,7 @@ the change to a User (or the assistant principal acting for a User). Staged on
|
||||
the session; the caller commits as part of its unit of work.
|
||||
"""
|
||||
|
||||
import json
|
||||
import uuid
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
@@ -11,6 +12,14 @@ from app.models.audit import AuditEntry
|
||||
from app.models.enums import AuditActorType
|
||||
|
||||
|
||||
def _json_safe(d: dict | None) -> dict | None:
|
||||
"""Coerce a change dict to JSON-native types (UUIDs, enums, dates -> str) so
|
||||
it lands in the JSON audit column regardless of what the caller passed."""
|
||||
if d is None:
|
||||
return None
|
||||
return json.loads(json.dumps(d, default=str))
|
||||
|
||||
|
||||
def record_audit(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
@@ -30,8 +39,8 @@ def record_audit(
|
||||
tree_id=tree_id,
|
||||
actor_user_id=actor_user_id,
|
||||
actor_type=actor_type,
|
||||
before=before,
|
||||
after=after,
|
||||
before=_json_safe(before),
|
||||
after=_json_safe(after),
|
||||
)
|
||||
session.add(entry)
|
||||
return entry
|
||||
|
||||
@@ -9,7 +9,7 @@ from sqlalchemy import select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.config import get_settings
|
||||
from app.core.security import generate_token, hash_password, hash_token
|
||||
from app.core.security import generate_token, hash_password, hash_token, verify_password
|
||||
from app.integrations.auth.local import LocalAuthProvider
|
||||
from app.integrations.mailer.base import Mailer
|
||||
from app.models.auth import Session as SessionModel
|
||||
@@ -17,7 +17,7 @@ from app.models.auth import UserToken
|
||||
from app.models.enums import TokenPurpose
|
||||
from app.models.user import User
|
||||
from app.services.audit import record_audit
|
||||
from app.services.exceptions import Conflict, NotFound
|
||||
from app.services.exceptions import Conflict, Forbidden, NotFound
|
||||
|
||||
_local_provider = LocalAuthProvider()
|
||||
|
||||
@@ -113,6 +113,8 @@ async def login(
|
||||
user = await _local_provider.authenticate(session, identifier=email, secret=password)
|
||||
if user is None:
|
||||
return None
|
||||
if get_settings().require_email_verification and user.email_verified_at is None:
|
||||
raise Forbidden("email not verified — check your inbox for the verification link")
|
||||
raw_token, record = _issue_session(session, user)
|
||||
record_audit(
|
||||
session, action="login", entity_type="User", entity_id=user.id, actor_user_id=user.id
|
||||
@@ -141,11 +143,16 @@ async def resolve_session_user(session: AsyncSession, *, raw_token: str) -> User
|
||||
).scalar_one_or_none()
|
||||
if record is None or record.revoked_at is not None or record.expires_at <= _now():
|
||||
return None
|
||||
return (
|
||||
user = (
|
||||
await session.execute(
|
||||
select(User).where(User.id == record.user_id, User.deleted_at.is_(None))
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
# The single read-side enforcement: an unverified user has no active session
|
||||
# when verification is required. Gates every authenticated request at once.
|
||||
if user is not None and get_settings().require_email_verification and user.email_verified_at is None:
|
||||
return None
|
||||
return user
|
||||
|
||||
|
||||
async def verify_email(session: AsyncSession, *, raw_token: str) -> None:
|
||||
@@ -178,6 +185,26 @@ async def request_password_reset(session: AsyncSession, mailer: Mailer, *, email
|
||||
await mailer.send_password_reset(to=email, link=_link("/auth/reset-password", raw))
|
||||
|
||||
|
||||
async def change_password(
|
||||
session: AsyncSession, *, user: User, current_password: str, new_password: str
|
||||
) -> None:
|
||||
"""Change a logged-in user's password after re-verifying the current one.
|
||||
Revokes other sessions so a changed password takes effect everywhere."""
|
||||
if not user.hashed_password or not verify_password(
|
||||
user.hashed_password, current_password
|
||||
):
|
||||
raise Forbidden("current password is incorrect")
|
||||
user.hashed_password = hash_password(new_password)
|
||||
record_audit(
|
||||
session,
|
||||
action="change_password",
|
||||
entity_type="User",
|
||||
entity_id=user.id,
|
||||
actor_user_id=user.id,
|
||||
)
|
||||
await session.commit()
|
||||
|
||||
|
||||
async def reset_password(session: AsyncSession, *, raw_token: str, new_password: str) -> None:
|
||||
token = await _consume_token(session, raw_token, TokenPurpose.password_reset)
|
||||
await session.execute(
|
||||
|
||||
@@ -0,0 +1,355 @@
|
||||
"""ChangeProposal lifecycle: propose (assistant/contributor) → review → apply/reject.
|
||||
|
||||
The structural guarantee (CLAUDE.md #1): a proposal's operations are executed
|
||||
ONLY by ``apply()``, which requires the actor be an editor and dispatches every
|
||||
op through the normal editing services — so each change passes the privacy
|
||||
engine and is audited as the approving human. ``propose()`` only inserts a
|
||||
pending row; it performs no domain mutation. See docs/design/change-proposal.md.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.change_proposal import ChangeProposal
|
||||
from app.models.enums import (
|
||||
ChangeProposalOrigin,
|
||||
ChangeProposalStatus,
|
||||
CitationConfidence,
|
||||
ParentChildQualifier,
|
||||
RelationshipType,
|
||||
)
|
||||
from app.models.tree import Tree
|
||||
from app.models.user import User
|
||||
from app.services import (
|
||||
citation_service,
|
||||
event_service,
|
||||
name_service,
|
||||
person_service,
|
||||
privacy,
|
||||
relationship_service,
|
||||
source_service,
|
||||
)
|
||||
from app.services.exceptions import Conflict, Forbidden, NotFound
|
||||
|
||||
|
||||
def _now() -> datetime:
|
||||
return datetime.now(UTC)
|
||||
|
||||
|
||||
def _uuid(v) -> uuid.UUID | None:
|
||||
return uuid.UUID(str(v)) if v else None
|
||||
|
||||
|
||||
async def _require_editor(session: AsyncSession, *, actor: User, tree: Tree) -> None:
|
||||
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
|
||||
raise Forbidden("not an editor of this tree")
|
||||
|
||||
|
||||
async def _require_member(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> None:
|
||||
# Proposals can reference unredacted facts → members only.
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
raise Forbidden("only members can see change proposals")
|
||||
|
||||
|
||||
async def _load(
|
||||
session: AsyncSession, tree: Tree, proposal_id: uuid.UUID
|
||||
) -> ChangeProposal:
|
||||
cp = (
|
||||
await session.execute(
|
||||
select(ChangeProposal).where(
|
||||
ChangeProposal.id == proposal_id,
|
||||
ChangeProposal.tree_id == tree.id,
|
||||
ChangeProposal.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if cp is None:
|
||||
raise NotFound("proposal not found")
|
||||
return cp
|
||||
|
||||
|
||||
async def propose(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
tree: Tree,
|
||||
origin: ChangeProposalOrigin,
|
||||
created_by: uuid.UUID | None,
|
||||
summary: str,
|
||||
rationale: str | None,
|
||||
operations: list[dict],
|
||||
) -> ChangeProposal:
|
||||
"""Insert a pending proposal. The ONLY mutation here is the proposal row — no
|
||||
tree data changes. (No edit-rights check: proposing isn't writing.)"""
|
||||
cp = ChangeProposal(
|
||||
tree_id=tree.id,
|
||||
origin=origin,
|
||||
created_by_user_id=created_by,
|
||||
summary=summary,
|
||||
rationale=rationale,
|
||||
operations=operations,
|
||||
status=ChangeProposalStatus.pending,
|
||||
)
|
||||
session.add(cp)
|
||||
await session.commit()
|
||||
await session.refresh(cp)
|
||||
return cp
|
||||
|
||||
|
||||
async def list_proposals(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
viewer_id: uuid.UUID,
|
||||
tree: Tree,
|
||||
status: ChangeProposalStatus | None = None,
|
||||
) -> list[ChangeProposal]:
|
||||
await _require_member(session, viewer_id=viewer_id, tree=tree)
|
||||
stmt = select(ChangeProposal).where(
|
||||
ChangeProposal.tree_id == tree.id, ChangeProposal.deleted_at.is_(None)
|
||||
)
|
||||
if status is not None:
|
||||
stmt = stmt.where(ChangeProposal.status == status)
|
||||
stmt = stmt.order_by(ChangeProposal.created_at.desc())
|
||||
return list((await session.execute(stmt)).scalars().all())
|
||||
|
||||
|
||||
async def get_proposal(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, proposal_id: uuid.UUID
|
||||
) -> ChangeProposal:
|
||||
await _require_member(session, viewer_id=viewer_id, tree=tree)
|
||||
return await _load(session, tree, proposal_id)
|
||||
|
||||
|
||||
async def reject(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
actor: User,
|
||||
tree: Tree,
|
||||
proposal_id: uuid.UUID,
|
||||
note: str | None = None,
|
||||
) -> ChangeProposal:
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
cp = await _load(session, tree, proposal_id)
|
||||
if cp.status is not ChangeProposalStatus.pending:
|
||||
raise Conflict("proposal is not pending")
|
||||
cp.status = ChangeProposalStatus.rejected
|
||||
cp.reviewed_by_user_id = actor.id
|
||||
cp.reviewed_at = _now()
|
||||
cp.review_note = note
|
||||
await session.commit()
|
||||
await session.refresh(cp)
|
||||
return cp
|
||||
|
||||
|
||||
async def apply(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
actor: User,
|
||||
tree: Tree,
|
||||
proposal_id: uuid.UUID,
|
||||
edited_operations: list[dict] | None = None,
|
||||
) -> ChangeProposal:
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
cp = await _load(session, tree, proposal_id)
|
||||
if cp.status is not ChangeProposalStatus.pending:
|
||||
raise Conflict("proposal is not pending")
|
||||
ops = edited_operations if edited_operations is not None else list(cp.operations)
|
||||
try:
|
||||
for op in ops:
|
||||
await _dispatch(session, actor=actor, tree=tree, op=op)
|
||||
except Conflict:
|
||||
raise
|
||||
except Exception as exc: # noqa: BLE001 — record the failure on the proposal
|
||||
err = f"{type(exc).__name__}: {exc}"[:2000]
|
||||
# The editing services raise (NotFound/Forbidden/validation) before
|
||||
# committing, so the transaction is clean — record the error and commit.
|
||||
# If a later op did write before failing, those ops already committed
|
||||
# (v1 isn't cross-op transactional; see the design note).
|
||||
cp = await _load(session, tree, proposal_id)
|
||||
cp.apply_error = err
|
||||
await session.commit()
|
||||
raise Conflict(f"could not apply proposal: {err}") from exc
|
||||
if edited_operations is not None:
|
||||
cp.operations = edited_operations
|
||||
cp.status = ChangeProposalStatus.applied
|
||||
cp.reviewed_by_user_id = actor.id
|
||||
cp.reviewed_at = _now()
|
||||
cp.apply_error = None
|
||||
await session.commit()
|
||||
await session.refresh(cp)
|
||||
return cp
|
||||
|
||||
|
||||
async def delete_proposal(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, proposal_id: uuid.UUID
|
||||
) -> None:
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
cp = await _load(session, tree, proposal_id)
|
||||
cp.deleted_at = _now()
|
||||
await session.commit()
|
||||
|
||||
|
||||
def _bad(entity_type: str, action: str) -> Conflict:
|
||||
return Conflict(f"unsupported operation '{action}' on '{entity_type}'")
|
||||
|
||||
|
||||
async def _dispatch(session: AsyncSession, *, actor: User, tree: Tree, op: dict) -> None:
|
||||
"""Route one operation through the matching editing service (privacy + audit)."""
|
||||
et = op.get("entity_type")
|
||||
action = op.get("op")
|
||||
payload = op.get("payload") or {}
|
||||
eid = op.get("entity_id")
|
||||
|
||||
if et == "person":
|
||||
if action == "create":
|
||||
await person_service.create_person(
|
||||
session,
|
||||
actor=actor,
|
||||
tree=tree,
|
||||
given=payload.get("given"),
|
||||
surname=payload.get("surname"),
|
||||
gender=payload.get("gender"),
|
||||
is_living=payload.get("is_living"),
|
||||
notes=payload.get("notes"),
|
||||
)
|
||||
elif action == "update":
|
||||
await person_service.update_person(
|
||||
session, actor=actor, tree=tree, person_id=_uuid(eid), changes=payload
|
||||
)
|
||||
elif action == "delete":
|
||||
await person_service.delete_person(
|
||||
session,
|
||||
actor=actor,
|
||||
tree=tree,
|
||||
person_id=_uuid(eid),
|
||||
cascade=bool(payload.get("cascade", False)),
|
||||
)
|
||||
else:
|
||||
raise _bad(et, action)
|
||||
elif et == "event":
|
||||
if action == "create":
|
||||
await event_service.create_event(
|
||||
session,
|
||||
actor=actor,
|
||||
tree=tree,
|
||||
event_type=payload["event_type"],
|
||||
person_id=_uuid(payload.get("person_id")),
|
||||
relationship_id=_uuid(payload.get("relationship_id")),
|
||||
date_value=payload.get("date_value"),
|
||||
date_precision=payload.get("date_precision"),
|
||||
detail=payload.get("detail"),
|
||||
notes=payload.get("notes"),
|
||||
)
|
||||
elif action == "update":
|
||||
await event_service.update_event(
|
||||
session, actor=actor, tree=tree, event_id=_uuid(eid), changes=payload
|
||||
)
|
||||
elif action == "delete":
|
||||
await event_service.delete_event(
|
||||
session, actor=actor, tree=tree, event_id=_uuid(eid)
|
||||
)
|
||||
else:
|
||||
raise _bad(et, action)
|
||||
elif et == "relationship":
|
||||
if action == "create":
|
||||
await relationship_service.create_relationship(
|
||||
session,
|
||||
actor=actor,
|
||||
tree=tree,
|
||||
type=RelationshipType(payload["type"]),
|
||||
person_from_id=_uuid(payload["person_from_id"]),
|
||||
person_to_id=_uuid(payload["person_to_id"]),
|
||||
qualifier=ParentChildQualifier(payload["qualifier"])
|
||||
if payload.get("qualifier")
|
||||
else None,
|
||||
notes=payload.get("notes"),
|
||||
)
|
||||
elif action == "delete":
|
||||
await relationship_service.delete_relationship(
|
||||
session, actor=actor, tree=tree, relationship_id=_uuid(eid)
|
||||
)
|
||||
else:
|
||||
raise _bad(et, action)
|
||||
elif et == "name":
|
||||
if action == "create":
|
||||
await name_service.create_name(
|
||||
session,
|
||||
actor=actor,
|
||||
tree=tree,
|
||||
person_id=_uuid(payload["person_id"]),
|
||||
name_type=payload.get("name_type", "birth"),
|
||||
given=payload.get("given"),
|
||||
surname=payload.get("surname"),
|
||||
prefix=payload.get("prefix"),
|
||||
suffix=payload.get("suffix"),
|
||||
nickname=payload.get("nickname"),
|
||||
is_primary=bool(payload.get("is_primary", False)),
|
||||
)
|
||||
elif action == "update":
|
||||
changes = {k: v for k, v in payload.items() if k != "person_id"}
|
||||
await name_service.update_name(
|
||||
session,
|
||||
actor=actor,
|
||||
tree=tree,
|
||||
person_id=_uuid(payload["person_id"]),
|
||||
name_id=_uuid(eid),
|
||||
changes=changes,
|
||||
)
|
||||
elif action == "delete":
|
||||
await name_service.delete_name(
|
||||
session,
|
||||
actor=actor,
|
||||
tree=tree,
|
||||
person_id=_uuid(payload["person_id"]),
|
||||
name_id=_uuid(eid),
|
||||
)
|
||||
else:
|
||||
raise _bad(et, action)
|
||||
elif et == "source":
|
||||
if action == "create":
|
||||
await source_service.create_source(
|
||||
session,
|
||||
actor=actor,
|
||||
tree=tree,
|
||||
title=payload["title"],
|
||||
author=payload.get("author"),
|
||||
source_type=payload.get("source_type"),
|
||||
repository=payload.get("repository"),
|
||||
url=payload.get("url"),
|
||||
citation_text=payload.get("citation_text"),
|
||||
publication_info=payload.get("publication_info"),
|
||||
quality_note=payload.get("quality_note"),
|
||||
)
|
||||
elif action == "delete":
|
||||
await source_service.delete_source(
|
||||
session, actor=actor, tree=tree, source_id=_uuid(eid)
|
||||
)
|
||||
else:
|
||||
raise _bad(et, action)
|
||||
elif et == "citation":
|
||||
if action == "create":
|
||||
await citation_service.create_citation(
|
||||
session,
|
||||
actor=actor,
|
||||
tree=tree,
|
||||
source_id=_uuid(payload["source_id"]),
|
||||
person_id=_uuid(payload.get("person_id")),
|
||||
event_id=_uuid(payload.get("event_id")),
|
||||
name_id=_uuid(payload.get("name_id")),
|
||||
relationship_id=_uuid(payload.get("relationship_id")),
|
||||
page=payload.get("page"),
|
||||
detail=payload.get("detail"),
|
||||
confidence=CitationConfidence(payload["confidence"])
|
||||
if payload.get("confidence")
|
||||
else None,
|
||||
)
|
||||
elif action == "delete":
|
||||
await citation_service.delete_citation(
|
||||
session, actor=actor, tree=tree, citation_id=_uuid(eid)
|
||||
)
|
||||
else:
|
||||
raise _bad(et, action)
|
||||
else:
|
||||
raise Conflict(f"unsupported entity type '{et}'")
|
||||
@@ -105,6 +105,15 @@ async def list_citations(
|
||||
indicators in a single round-trip."""
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
# Non-members get only citations whose cited fact resolves to a full-
|
||||
# visibility person — a citation on a redacted living person's fact would
|
||||
# otherwise leak that the person has that sourced fact.
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
return await public_view_service.list_public_citations(
|
||||
session, viewer_id=viewer_id, tree=tree
|
||||
)
|
||||
stmt = (
|
||||
select(Citation)
|
||||
.where(Citation.tree_id == tree.id, Citation.deleted_at.is_(None))
|
||||
|
||||
@@ -0,0 +1,382 @@
|
||||
"""Bulk tree cleanup — preview/apply pairs for common import messes.
|
||||
|
||||
Per the project's #1 rule (the assistant proposes, humans approve), each fix has
|
||||
a *preview* that returns the proposed changes and an *apply* that commits only
|
||||
the ids/edits the user confirmed. Nothing here mutates without an explicit apply
|
||||
call carrying the user's selections.
|
||||
"""
|
||||
|
||||
import re
|
||||
import uuid
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.enums import RelationshipType
|
||||
from app.models.event import Event
|
||||
from app.models.person import Name, Person
|
||||
from app.models.relationship import Relationship
|
||||
from app.models.tree import Tree
|
||||
from app.models.user import User
|
||||
from app.services import gedcom, privacy
|
||||
from app.services.audit import record_audit
|
||||
from app.services.exceptions import Forbidden, NotFound
|
||||
from app.services.name_gender_data import guess_sex
|
||||
|
||||
|
||||
async def _require_editor(session: AsyncSession, *, actor: User, tree: Tree) -> None:
|
||||
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
|
||||
raise Forbidden("not an editor of this tree")
|
||||
|
||||
|
||||
async def _persons(session: AsyncSession, tree_id: uuid.UUID) -> list[Person]:
|
||||
return list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Person).where(Person.tree_id == tree_id, Person.deleted_at.is_(None))
|
||||
)
|
||||
).scalars().all()
|
||||
)
|
||||
|
||||
|
||||
async def _primary_name_by_person(
|
||||
session: AsyncSession, tree_id: uuid.UUID
|
||||
) -> dict[uuid.UUID, Name]:
|
||||
names = (
|
||||
await session.execute(
|
||||
select(Name)
|
||||
.where(Name.tree_id == tree_id, Name.deleted_at.is_(None))
|
||||
.order_by(Name.is_primary.desc(), Name.sort_order)
|
||||
)
|
||||
).scalars().all()
|
||||
out: dict[uuid.UUID, Name] = {}
|
||||
for n in names:
|
||||
out.setdefault(n.person_id, n)
|
||||
return out
|
||||
|
||||
|
||||
async def _birth_year_by_person(session: AsyncSession, tree_id: uuid.UUID) -> dict[uuid.UUID, int]:
|
||||
evs = (
|
||||
await session.execute(
|
||||
select(Event).where(
|
||||
Event.tree_id == tree_id,
|
||||
Event.deleted_at.is_(None),
|
||||
Event.event_type == "birth",
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
out: dict[uuid.UUID, int] = {}
|
||||
for e in evs:
|
||||
if not e.person_id or e.person_id in out:
|
||||
continue
|
||||
y = e.date_start.year if e.date_start else None
|
||||
if y is None:
|
||||
ys = gedcom._year(e.date_value)
|
||||
y = int(ys) if ys else None
|
||||
if y is not None:
|
||||
out[e.person_id] = y
|
||||
return out
|
||||
|
||||
|
||||
def _display(n: Name | None) -> str:
|
||||
if n is None:
|
||||
return "Unnamed"
|
||||
return " ".join(x for x in (n.given, n.surname) if x) or (n.display_name or "Unnamed")
|
||||
|
||||
|
||||
# ---- 1. Mark deceased by birth year -------------------------------------------------
|
||||
|
||||
async def preview_deceased(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, year: int
|
||||
) -> list[dict]:
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
names = await _primary_name_by_person(session, tree.id)
|
||||
years = await _birth_year_by_person(session, tree.id)
|
||||
out: list[dict] = []
|
||||
for p in await _persons(session, tree.id):
|
||||
if p.is_living is False: # already deceased
|
||||
continue
|
||||
by = years.get(p.id)
|
||||
if by is not None and by <= year:
|
||||
out.append(
|
||||
{"person_id": str(p.id), "name": _display(names.get(p.id)), "birth_year": by}
|
||||
)
|
||||
out.sort(key=lambda r: r["birth_year"])
|
||||
return out
|
||||
|
||||
|
||||
async def apply_deceased(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, person_ids: list[uuid.UUID]
|
||||
) -> int:
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
persons = (
|
||||
await session.execute(
|
||||
select(Person).where(
|
||||
Person.tree_id == tree.id,
|
||||
Person.deleted_at.is_(None),
|
||||
Person.id.in_(person_ids),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
for p in persons:
|
||||
p.is_living = False
|
||||
record_audit(
|
||||
session,
|
||||
action="cleanup_deceased",
|
||||
entity_type="Tree",
|
||||
entity_id=tree.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
after={"count": len(persons)},
|
||||
)
|
||||
await session.commit()
|
||||
return len(persons)
|
||||
|
||||
|
||||
# ---- 1b. Mark deceased by a CHILD's birth year -------------------------------------
|
||||
# For parents whose own birth date is missing (so the birth-year rule can't reach
|
||||
# them) but who have a child born long ago — they're necessarily deceased. Applies
|
||||
# through the same apply_deceased() path.
|
||||
|
||||
async def preview_deceased_by_child(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, year: int
|
||||
) -> list[dict]:
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
names = await _primary_name_by_person(session, tree.id)
|
||||
years = await _birth_year_by_person(session, tree.id)
|
||||
rels = (
|
||||
await session.execute(
|
||||
select(Relationship).where(
|
||||
Relationship.tree_id == tree.id,
|
||||
Relationship.deleted_at.is_(None),
|
||||
Relationship.type == RelationshipType.parent_child,
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
# parent id -> earliest child birth year, among children born on/before `year`.
|
||||
earliest_child: dict[uuid.UUID, int] = {}
|
||||
for r in rels:
|
||||
cy = years.get(r.person_to_id) # the child's birth year
|
||||
if cy is None or cy > year:
|
||||
continue
|
||||
if r.person_from_id not in earliest_child or cy < earliest_child[r.person_from_id]:
|
||||
earliest_child[r.person_from_id] = cy
|
||||
persons = {p.id: p for p in await _persons(session, tree.id)}
|
||||
out: list[dict] = []
|
||||
for parent_id, cy in earliest_child.items():
|
||||
p = persons.get(parent_id)
|
||||
if p is None or p.is_living is False: # gone or already deceased
|
||||
continue
|
||||
out.append(
|
||||
{
|
||||
"person_id": str(parent_id),
|
||||
"name": _display(names.get(parent_id)),
|
||||
"child_birth_year": cy,
|
||||
}
|
||||
)
|
||||
out.sort(key=lambda r: r["child_birth_year"])
|
||||
return out
|
||||
|
||||
|
||||
# ---- 2. Re-derive gender from a source GEDCOM (matches by name) ----------------------
|
||||
|
||||
async def preview_gender(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, gedcom_text: str
|
||||
) -> list[dict]:
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
name2sex: dict[str, str] = {}
|
||||
for rec in gedcom.parse_records(gedcom_text):
|
||||
if rec.tag != "INDI":
|
||||
continue
|
||||
summ = gedcom._person_summary(rec)
|
||||
sex = gedcom._sex(rec.text("SEX"))
|
||||
if sex and summ["norm"]:
|
||||
name2sex.setdefault(summ["norm"], sex)
|
||||
|
||||
names = await _primary_name_by_person(session, tree.id)
|
||||
out: list[dict] = []
|
||||
for p in await _persons(session, tree.id):
|
||||
if p.gender: # only fill in what's missing
|
||||
continue
|
||||
nm = names.get(p.id)
|
||||
if nm is None:
|
||||
continue
|
||||
proposed = name2sex.get(gedcom._norm(nm.given, nm.surname))
|
||||
if proposed:
|
||||
out.append({"person_id": str(p.id), "name": _display(nm), "proposed_gender": proposed})
|
||||
out.sort(key=lambda r: r["name"])
|
||||
return out
|
||||
|
||||
|
||||
async def guess_gender_by_name(
|
||||
session: AsyncSession, *, actor: User, tree: Tree
|
||||
) -> list[dict]:
|
||||
"""Best-guess sex from the first given name for people who don't have it set,
|
||||
using the bundled name dictionary. Ambiguous/unknown names are skipped."""
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
names = await _primary_name_by_person(session, tree.id)
|
||||
out: list[dict] = []
|
||||
for p in await _persons(session, tree.id):
|
||||
if p.gender:
|
||||
continue
|
||||
nm = names.get(p.id)
|
||||
if nm is None:
|
||||
continue
|
||||
proposed = guess_sex(nm.given)
|
||||
if proposed:
|
||||
out.append({"person_id": str(p.id), "name": _display(nm), "proposed_gender": proposed})
|
||||
out.sort(key=lambda r: r["name"])
|
||||
return out
|
||||
|
||||
|
||||
async def guess_gender_by_spouse(
|
||||
session: AsyncSession, *, actor: User, tree: Tree
|
||||
) -> list[dict]:
|
||||
"""Infer the sex of a person who has none set from a partner whose sex IS set
|
||||
(couples in a tree are opposite-sex in practice — e.g. a confirmed-male
|
||||
husband implies a female wife). People whose known partners disagree are
|
||||
ambiguous and skipped; the result is a preview to review, not an auto-write."""
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
persons = await _persons(session, tree.id)
|
||||
gender = {p.id: p.gender for p in persons}
|
||||
names = await _primary_name_by_person(session, tree.id)
|
||||
rels = (
|
||||
await session.execute(
|
||||
select(Relationship).where(
|
||||
Relationship.tree_id == tree.id,
|
||||
Relationship.deleted_at.is_(None),
|
||||
Relationship.type == RelationshipType.partnership,
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
opp = {"male": "female", "female": "male"}
|
||||
proposals: dict[uuid.UUID, set[str]] = {}
|
||||
for r in rels:
|
||||
for me_id, other_id in (
|
||||
(r.person_from_id, r.person_to_id),
|
||||
(r.person_to_id, r.person_from_id),
|
||||
):
|
||||
if gender.get(me_id):
|
||||
continue # this person already has a sex
|
||||
other_sex = str(gender.get(other_id) or "")
|
||||
if other_sex in opp:
|
||||
proposals.setdefault(me_id, set()).add(opp[other_sex])
|
||||
out: list[dict] = []
|
||||
for pid, sexes in proposals.items():
|
||||
if len(sexes) != 1:
|
||||
continue # partners of differing known sex → ambiguous
|
||||
nm = names.get(pid)
|
||||
if nm is None:
|
||||
continue
|
||||
out.append(
|
||||
{"person_id": str(pid), "name": _display(nm), "proposed_gender": next(iter(sexes))}
|
||||
)
|
||||
out.sort(key=lambda r: r["name"])
|
||||
return out
|
||||
|
||||
|
||||
async def apply_gender(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, updates: list[dict]
|
||||
) -> int:
|
||||
"""updates: [{person_id, gender}]."""
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
wanted = {uuid.UUID(str(u["person_id"])): u["gender"] for u in updates if u.get("gender")}
|
||||
persons = (
|
||||
await session.execute(
|
||||
select(Person).where(
|
||||
Person.tree_id == tree.id,
|
||||
Person.deleted_at.is_(None),
|
||||
Person.id.in_(wanted.keys()),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
for p in persons:
|
||||
p.gender = wanted[p.id]
|
||||
record_audit(
|
||||
session,
|
||||
action="cleanup_gender",
|
||||
entity_type="Tree",
|
||||
entity_id=tree.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
after={"count": len(persons)},
|
||||
)
|
||||
await session.commit()
|
||||
return len(persons)
|
||||
|
||||
|
||||
# ---- 3. Flag malformed names for review --------------------------------------------
|
||||
|
||||
_YEAR_RE = re.compile(r"\b\d{3,4}\b")
|
||||
|
||||
|
||||
def _name_issue(n: Name) -> str | None:
|
||||
given = (n.given or "").strip()
|
||||
surname = (n.surname or "").strip()
|
||||
if _YEAR_RE.search(surname) or re.search(r"\d", surname):
|
||||
return "date_in_surname"
|
||||
if re.search(r"\d", given):
|
||||
return "date_in_given"
|
||||
# A given name with many tokens often means a maiden+married name was packed
|
||||
# in (e.g. "Mary Smith Jones") — surface it for a human to split.
|
||||
if surname == "" and len(given.split()) >= 2:
|
||||
return "no_surname"
|
||||
if len(given.split()) >= 3:
|
||||
return "packed_given"
|
||||
return None
|
||||
|
||||
|
||||
async def preview_names(session: AsyncSession, *, actor: User, tree: Tree) -> list[dict]:
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
names = (
|
||||
await session.execute(
|
||||
select(Name).where(Name.tree_id == tree.id, Name.deleted_at.is_(None))
|
||||
)
|
||||
).scalars().all()
|
||||
out: list[dict] = []
|
||||
for n in names:
|
||||
issue = _name_issue(n)
|
||||
if issue:
|
||||
out.append({
|
||||
"name_id": str(n.id),
|
||||
"person_id": str(n.person_id),
|
||||
"given": n.given,
|
||||
"surname": n.surname,
|
||||
"issue": issue,
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
async def apply_names(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, edits: list[dict]
|
||||
) -> int:
|
||||
"""edits: [{name_id, given, surname}] — the user's corrected values."""
|
||||
await _require_editor(session, actor=actor, tree=tree)
|
||||
by_id = {uuid.UUID(str(e["name_id"])): e for e in edits}
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(Name).where(
|
||||
Name.tree_id == tree.id,
|
||||
Name.deleted_at.is_(None),
|
||||
Name.id.in_(by_id.keys()),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
if len(rows) != len(by_id):
|
||||
raise NotFound("one or more names not found in this tree")
|
||||
for n in rows:
|
||||
e = by_id[n.id]
|
||||
n.given = (e.get("given") or "").strip() or None
|
||||
n.surname = (e.get("surname") or "").strip() or None
|
||||
n.display_name = None # rebuild from parts
|
||||
record_audit(
|
||||
session,
|
||||
action="cleanup_names",
|
||||
entity_type="Tree",
|
||||
entity_id=tree.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
after={"count": len(rows)},
|
||||
)
|
||||
await session.commit()
|
||||
return len(rows)
|
||||
@@ -4,9 +4,10 @@ engine. Every event has exactly one subject — a Person or a partnership."""
|
||||
import uuid
|
||||
from datetime import date
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import or_, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.enums import RelationshipType
|
||||
from app.models.event import Event
|
||||
from app.models.person import Person
|
||||
from app.models.place import Place
|
||||
@@ -97,6 +98,13 @@ async def list_events(
|
||||
"""All events in the tree — lets the family view compute birth/death years."""
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
# Non-members get the redacted projection (no living-person dates).
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
return await public_view_service.list_public_events(
|
||||
session, viewer_id=viewer_id, tree=tree
|
||||
)
|
||||
stmt = (
|
||||
select(Event)
|
||||
.where(Event.tree_id == tree.id, Event.deleted_at.is_(None))
|
||||
@@ -110,12 +118,37 @@ async def list_events_for_person(
|
||||
) -> list[Event]:
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
# Non-members only see a full-visibility person's events (redacted → none).
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
return await public_view_service.list_public_person_events(
|
||||
session, viewer_id=viewer_id, tree=tree, person_id=person_id
|
||||
)
|
||||
# Member view: this person's own events PLUS their partnership events (which
|
||||
# live on the relationship and show on both partners). Returning both here
|
||||
# means the person page doesn't have to load every event in the tree.
|
||||
partner_rel_ids = (
|
||||
select(Relationship.id)
|
||||
.where(
|
||||
Relationship.tree_id == tree.id,
|
||||
Relationship.type == RelationshipType.partnership,
|
||||
Relationship.deleted_at.is_(None),
|
||||
or_(
|
||||
Relationship.person_from_id == person_id,
|
||||
Relationship.person_to_id == person_id,
|
||||
),
|
||||
)
|
||||
)
|
||||
stmt = (
|
||||
select(Event)
|
||||
.where(
|
||||
Event.tree_id == tree.id,
|
||||
Event.person_id == person_id,
|
||||
Event.deleted_at.is_(None),
|
||||
or_(
|
||||
Event.person_id == person_id,
|
||||
Event.relationship_id.in_(partner_rel_ids),
|
||||
),
|
||||
)
|
||||
.order_by(Event.date_start.nulls_last(), Event.created_at)
|
||||
)
|
||||
|
||||
+440
-50
@@ -4,14 +4,20 @@ A pragmatic parser + mapper for the common subset of GEDCOM (5.5.1 / 7 share
|
||||
the line grammar): INDI, FAM, SOUR. Import maps records into a tree and returns
|
||||
a mapping report (counts + unmapped tags); export serializes the tree back to
|
||||
GEDCOM. Runs inline for now — large files should move to the worker later.
|
||||
|
||||
Import is duplicate-aware: ``preview_gedcom`` reports incoming people that look
|
||||
like existing ones, and ``import_gedcom`` applies a per-record resolution
|
||||
(new / skip / merge / overwrite). Names carry their GEDCOM type (a married name
|
||||
imports as a typed alternate, not a second primary).
|
||||
"""
|
||||
|
||||
import re
|
||||
import uuid
|
||||
from collections import defaultdict
|
||||
from datetime import date
|
||||
from datetime import UTC, date, datetime
|
||||
from difflib import SequenceMatcher
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import or_, select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.enums import ParentChildQualifier, RelationshipType
|
||||
@@ -32,12 +38,31 @@ INDI_EVENTS = {
|
||||
"BURI": "burial", "CREM": "cremation", "RESI": "residence", "CENS": "census",
|
||||
"IMMI": "immigration", "EMIG": "emigration", "OCCU": "occupation",
|
||||
"EDUC": "education", "GRAD": "graduation", "RETI": "retirement",
|
||||
"NATU": "naturalization", "BAPL": "baptism",
|
||||
"NATU": "naturalization", "BAPL": "baptism", "RELI": "religion",
|
||||
}
|
||||
# INDI attribute tags whose line VALUE is the fact (no date), stored in detail.
|
||||
VALUE_EVENTS = {"RELI", "OCCU", "EDUC"}
|
||||
# INDI sub-tags consumed elsewhere or intentionally ignored (not "unmapped").
|
||||
INDI_SKIP_TAGS = {
|
||||
"NAME", "SEX", "SOUR", "FAMC", "FAMS", "CHAN", "OBJE", "_UID", "_MARNM", "NOTE",
|
||||
}
|
||||
# FAM-level events.
|
||||
FAM_EVENTS = {"MARR": "marriage", "DIV": "divorce", "ENGA": "engagement"}
|
||||
EVENT_TO_GED = {v: k for k, v in {**INDI_EVENTS, **FAM_EVENTS}.items()}
|
||||
|
||||
# GEDCOM NAME TYPE (or _MARNM-derived) -> our Name.name_type vocabulary.
|
||||
NAME_TYPE_MAP = {
|
||||
"birth": "birth", "maiden": "birth", "married": "married",
|
||||
"aka": "alias", "also known as": "alias", "nickname": "nickname",
|
||||
"religious": "religious", "immigrant": "immigration",
|
||||
"immigration": "immigration", "professional": "alias", "other": "alias",
|
||||
}
|
||||
# Our type -> GEDCOM TYPE on export (birth is the default; emit nothing).
|
||||
EXPORT_TYPE_MAP = {
|
||||
"married": "married", "alias": "aka", "nickname": "nickname",
|
||||
"religious": "religious", "immigration": "immigrant",
|
||||
}
|
||||
|
||||
|
||||
class GedcomNode:
|
||||
__slots__ = ("level", "tag", "value", "xref", "children")
|
||||
@@ -108,6 +133,50 @@ def _parse_name(value: str) -> tuple[str | None, str | None]:
|
||||
return value.strip() or None, None
|
||||
|
||||
|
||||
def _parse_marnm(value: str, base_given: str | None) -> tuple[str | None, str | None]:
|
||||
"""A _MARNM value is sometimes a full name ("Jane /Smith/") and sometimes
|
||||
just the married surname ("Smith"). Keep the given name from the base name
|
||||
in the latter case."""
|
||||
v = (value or "").strip()
|
||||
if "/" in v:
|
||||
g, s = _parse_name(v)
|
||||
return (g or base_given), s
|
||||
return base_given, (v or None)
|
||||
|
||||
|
||||
def _extract_names(rec: GedcomNode) -> list[dict]:
|
||||
"""All names for an INDI, typed. Multiple NAME records (each with an optional
|
||||
TYPE) plus any _MARNM (married name) subtags become separate Name rows. The
|
||||
first birth/maiden name is primary."""
|
||||
out: list[dict] = []
|
||||
for nm in rec.all("NAME"):
|
||||
g, s = _parse_name(nm.value)
|
||||
t = (nm.text("TYPE") or "").strip().lower()
|
||||
ntype = NAME_TYPE_MAP.get(t, t or "birth")
|
||||
out.append({"type": ntype, "given": g, "surname": s, "display": nm.value or None,
|
||||
"nickname": nm.text("NICK")})
|
||||
for mar in nm.all("_MARNM"):
|
||||
mg, ms = _parse_marnm(mar.value, g)
|
||||
out.append({"type": "married", "given": mg, "surname": ms,
|
||||
"display": mar.value or None, "nickname": None})
|
||||
for mar in rec.all("_MARNM"):
|
||||
base_g = out[0]["given"] if out else None
|
||||
mg, ms = _parse_marnm(mar.value, base_g)
|
||||
out.append({"type": "married", "given": mg, "surname": ms,
|
||||
"display": mar.value or None, "nickname": None})
|
||||
if not out:
|
||||
return out
|
||||
primary_idx = next((i for i, n in enumerate(out) if n["type"] == "birth"), 0)
|
||||
for i, n in enumerate(out):
|
||||
n["is_primary"] = i == primary_idx
|
||||
n["sort"] = i
|
||||
return out
|
||||
|
||||
|
||||
def _norm(given: str | None, surname: str | None) -> str:
|
||||
return re.sub(r"\s+", " ", f"{given or ''} {surname or ''}".strip().lower())
|
||||
|
||||
|
||||
def _year(date_value: str | None) -> str | None:
|
||||
if not date_value:
|
||||
return None
|
||||
@@ -132,18 +201,215 @@ def _sex(value: str | None) -> str | None:
|
||||
return {"M": "male", "F": "female"}.get(v, value.strip().lower() or None)
|
||||
|
||||
|
||||
def _notes_text(rec: GedcomNode) -> str | None:
|
||||
"""Join an INDI's NOTE lines (which pack confidence / findagrave / fs_pid /
|
||||
free text) into the person's notes field."""
|
||||
vals = [n.value.strip() for n in rec.all("NOTE") if n.value and n.value.strip()]
|
||||
return "\n".join(vals) or None
|
||||
|
||||
|
||||
def _person_summary(rec: GedcomNode) -> dict:
|
||||
"""Display name + birth year for an incoming INDI, for duplicate matching."""
|
||||
names = _extract_names(rec)
|
||||
primary = next((n for n in names if n.get("is_primary")), names[0] if names else None)
|
||||
g = primary["given"] if primary else None
|
||||
s = primary["surname"] if primary else None
|
||||
disp = " ".join(x for x in (g, s) if x)
|
||||
if not disp and primary:
|
||||
disp = primary.get("display") or ""
|
||||
birth = rec.first("BIRT")
|
||||
year = _year(birth.text("DATE")) if birth else None
|
||||
return {"names": names, "norm": _norm(g, s), "name": disp or "(no name)", "year": year}
|
||||
|
||||
|
||||
async def _build_existing_index(session: AsyncSession, tree: Tree) -> list[dict]:
|
||||
"""Existing (non-deleted) people with a display name + birth year, for
|
||||
matching incoming records against."""
|
||||
persons = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Person).where(Person.tree_id == tree.id, Person.deleted_at.is_(None))
|
||||
)
|
||||
).scalars().all()
|
||||
)
|
||||
names = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Name).where(Name.tree_id == tree.id, Name.deleted_at.is_(None))
|
||||
)
|
||||
).scalars().all()
|
||||
)
|
||||
name_by_person: dict[uuid.UUID, Name] = {}
|
||||
for n in sorted(names, key=lambda n: (not n.is_primary, n.sort_order)):
|
||||
name_by_person.setdefault(n.person_id, n)
|
||||
births = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Event).where(
|
||||
Event.tree_id == tree.id,
|
||||
Event.deleted_at.is_(None),
|
||||
Event.event_type == "birth",
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
)
|
||||
year_by_person: dict[uuid.UUID, str] = {}
|
||||
for e in births:
|
||||
if e.person_id and e.person_id not in year_by_person:
|
||||
y = str(e.date_start.year) if e.date_start else _year(e.date_value)
|
||||
if y:
|
||||
year_by_person[e.person_id] = y
|
||||
|
||||
index: list[dict] = []
|
||||
for p in persons:
|
||||
nm = name_by_person.get(p.id)
|
||||
g = nm.given if nm else None
|
||||
s = nm.surname if nm else None
|
||||
disp = " ".join(x for x in (g, s) if x) or (nm.display_name if nm else None)
|
||||
index.append({
|
||||
"id": p.id,
|
||||
"norm": _norm(g, s),
|
||||
"name": disp or "(no name)",
|
||||
"year": year_by_person.get(p.id),
|
||||
})
|
||||
return index
|
||||
|
||||
|
||||
def _best_match(norm: str, year: str | None, index: list[dict]) -> tuple[dict | None, str | None]:
|
||||
"""Closest existing person by name similarity, rejecting clear birth-year
|
||||
conflicts. Returns (entry, "high"|"medium") or (None, None)."""
|
||||
if not norm:
|
||||
return None, None
|
||||
best: dict | None = None
|
||||
best_r = 0.0
|
||||
for e in index:
|
||||
if not e["norm"]:
|
||||
continue
|
||||
r = SequenceMatcher(None, norm, e["norm"]).ratio()
|
||||
if r < 0.88:
|
||||
continue
|
||||
if year and e["year"] and abs(int(year) - int(e["year"])) > 1:
|
||||
continue # same-ish name but different birth year — not a duplicate
|
||||
if r > best_r:
|
||||
best_r = r
|
||||
best = e
|
||||
if best is None:
|
||||
return None, None
|
||||
year_match = bool(year and best["year"] and abs(int(year) - int(best["year"])) <= 1)
|
||||
both_unknown = not year and not best["year"]
|
||||
score = "high" if best_r >= 0.93 and (year_match or both_unknown) else "medium"
|
||||
return best, score
|
||||
|
||||
|
||||
def _relkey(rtype: RelationshipType, a: uuid.UUID, b: uuid.UUID) -> tuple:
|
||||
if rtype == RelationshipType.parent_child:
|
||||
return ("pc", str(a), str(b))
|
||||
return (rtype.value, *sorted([str(a), str(b)]))
|
||||
|
||||
|
||||
def _count_incoming(roots: list[GedcomNode]) -> tuple[dict, list[str]]:
|
||||
counts: dict[str, int] = defaultdict(int)
|
||||
unmapped: set[str] = set()
|
||||
for rec in roots:
|
||||
if rec.tag == "INDI" and rec.xref:
|
||||
counts["persons"] += 1
|
||||
counts["names"] += len(_extract_names(rec))
|
||||
for child in rec.children:
|
||||
if child.tag in INDI_EVENTS:
|
||||
counts["events"] += 1
|
||||
elif child.tag not in INDI_SKIP_TAGS:
|
||||
unmapped.add(child.tag)
|
||||
elif rec.tag == "FAM":
|
||||
counts["families"] += 1
|
||||
for child in rec.children:
|
||||
if child.tag in FAM_EVENTS:
|
||||
counts["events"] += 1
|
||||
elif rec.tag == "SOUR" and rec.xref:
|
||||
counts["sources"] += 1
|
||||
return dict(counts), sorted(unmapped)
|
||||
|
||||
|
||||
async def preview_gedcom(session: AsyncSession, *, actor: User, tree: Tree, text: str) -> dict:
|
||||
"""Dry run: what would import, and which incoming people look like existing
|
||||
ones. No writes."""
|
||||
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
|
||||
raise Forbidden("not an editor of this tree")
|
||||
roots = parse_records(text)
|
||||
counts, unmapped = _count_incoming(roots)
|
||||
index = await _build_existing_index(session, tree)
|
||||
|
||||
duplicates: list[dict] = []
|
||||
for rec in roots:
|
||||
if rec.tag != "INDI" or not rec.xref:
|
||||
continue
|
||||
summ = _person_summary(rec)
|
||||
entry, score = _best_match(summ["norm"], summ["year"], index)
|
||||
if entry is None:
|
||||
continue
|
||||
duplicates.append({
|
||||
"xref": rec.xref,
|
||||
"incoming_name": summ["name"],
|
||||
"incoming_birth_year": summ["year"],
|
||||
"existing_person_id": entry["id"],
|
||||
"existing_name": entry["name"],
|
||||
"existing_birth_year": entry["year"],
|
||||
"score": score,
|
||||
})
|
||||
return {"counts": counts, "potential_duplicates": duplicates, "unmapped_tags": unmapped}
|
||||
|
||||
|
||||
async def import_gedcom(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, text: str
|
||||
session: AsyncSession,
|
||||
*,
|
||||
actor: User,
|
||||
tree: Tree,
|
||||
text: str,
|
||||
default_action: str = "new",
|
||||
resolutions: dict | None = None,
|
||||
) -> dict:
|
||||
"""Import records. ``default_action`` (new|skip|merge|overwrite) applies to
|
||||
incoming people that match an existing one; ``resolutions`` overrides it per
|
||||
GEDCOM xref ({xref: {action, target_id}}). 'skip' links families to the
|
||||
existing person but copies nothing; 'merge' also copies the incoming names
|
||||
(as alternates), events and citations onto them; 'overwrite' deletes the
|
||||
existing person and imports the incoming one fresh."""
|
||||
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
|
||||
raise Forbidden("not an editor of this tree")
|
||||
|
||||
resolutions = resolutions or {}
|
||||
roots = parse_records(text)
|
||||
counts = defaultdict(int)
|
||||
counts: dict[str, int] = defaultdict(int)
|
||||
unmapped: set[str] = set()
|
||||
place_cache: dict[str, uuid.UUID] = {}
|
||||
source_map: dict[str, uuid.UUID] = {}
|
||||
person_map: dict[str, uuid.UUID] = {}
|
||||
now = datetime.now(UTC)
|
||||
|
||||
index = await _build_existing_index(session, tree)
|
||||
|
||||
# Pre-load existing relationship keys so a merge doesn't create dup edges.
|
||||
existing_rels = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Relationship).where(
|
||||
Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None)
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
)
|
||||
rel_keys = {_relkey(r.type, r.person_from_id, r.person_to_id) for r in existing_rels}
|
||||
|
||||
def add_relationship(
|
||||
rtype: RelationshipType, a: uuid.UUID, b: uuid.UUID, **kw
|
||||
) -> Relationship | None:
|
||||
key = _relkey(rtype, a, b)
|
||||
if key in rel_keys:
|
||||
return None
|
||||
rel = Relationship(tree_id=tree.id, type=rtype, person_from_id=a, person_to_id=b, **kw)
|
||||
session.add(rel)
|
||||
rel_keys.add(key)
|
||||
counts["relationships"] += 1
|
||||
return rel
|
||||
|
||||
async def place_id(name: str | None) -> uuid.UUID | None:
|
||||
if not name:
|
||||
@@ -177,59 +443,139 @@ async def import_gedcom(
|
||||
sid = source_map.get(s.value.strip())
|
||||
if sid is None:
|
||||
continue
|
||||
session.add(
|
||||
Citation(tree_id=tree.id, source_id=sid, page=s.text("PAGE"), **target)
|
||||
)
|
||||
session.add(Citation(tree_id=tree.id, source_id=sid, page=s.text("PAGE"), **target))
|
||||
counts["citations"] += 1
|
||||
|
||||
# Individuals.
|
||||
for rec in roots:
|
||||
if rec.tag != "INDI" or not rec.xref:
|
||||
continue
|
||||
person = Person(tree_id=tree.id, gender=_sex(rec.text("SEX")))
|
||||
session.add(person)
|
||||
await session.flush()
|
||||
person_map[rec.xref] = person.id
|
||||
counts["persons"] += 1
|
||||
|
||||
for i, nm in enumerate(rec.all("NAME")):
|
||||
given, surname = _parse_name(nm.value)
|
||||
def add_names(person_id: uuid.UUID, names: list[dict], *, set_primary: bool) -> None:
|
||||
for nd in names:
|
||||
session.add(
|
||||
Name(
|
||||
tree_id=tree.id,
|
||||
person_id=person.id,
|
||||
name_type="birth",
|
||||
given=given,
|
||||
surname=surname,
|
||||
display_name=nm.value or None,
|
||||
is_primary=(i == 0),
|
||||
sort_order=i,
|
||||
person_id=person_id,
|
||||
name_type=nd["type"],
|
||||
given=nd["given"],
|
||||
surname=nd["surname"],
|
||||
nickname=nd.get("nickname"),
|
||||
display_name=nd.get("display"),
|
||||
is_primary=set_primary and nd.get("is_primary", False),
|
||||
sort_order=nd.get("sort", 0),
|
||||
)
|
||||
)
|
||||
counts["names"] += 1
|
||||
|
||||
await add_citations(rec, person_id=person.id)
|
||||
|
||||
async def add_events(rec: GedcomNode, person_id: uuid.UUID) -> None:
|
||||
for child in rec.children:
|
||||
if child.tag in INDI_EVENTS:
|
||||
dv = child.text("DATE")
|
||||
# Attribute-style facts (RELI, OCCU, EDUC) carry their value on
|
||||
# the line itself; store it in detail.
|
||||
detail = child.value.strip() if child.tag in VALUE_EVENTS else None
|
||||
ev = Event(
|
||||
tree_id=tree.id,
|
||||
person_id=person.id,
|
||||
person_id=person_id,
|
||||
event_type=INDI_EVENTS[child.tag],
|
||||
date_value=dv,
|
||||
date_start=_date_start(dv),
|
||||
place_id=await place_id(child.text("PLAC")),
|
||||
detail=detail or None,
|
||||
notes=child.text("NOTE"),
|
||||
)
|
||||
session.add(ev)
|
||||
await session.flush()
|
||||
counts["events"] += 1
|
||||
await add_citations(child, event_id=ev.id)
|
||||
elif child.tag in ("NAME", "SEX", "SOUR", "FAMC", "FAMS", "CHAN", "OBJE", "_UID"):
|
||||
elif child.tag in INDI_SKIP_TAGS:
|
||||
continue
|
||||
else:
|
||||
unmapped.add(child.tag)
|
||||
|
||||
async def soft_delete_existing(person_id: uuid.UUID) -> None:
|
||||
p = (
|
||||
await session.execute(
|
||||
select(Person).where(Person.id == person_id, Person.deleted_at.is_(None))
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if p is None:
|
||||
return
|
||||
p.deleted_at = now
|
||||
rels = (
|
||||
await session.execute(
|
||||
select(Relationship).where(
|
||||
Relationship.tree_id == tree.id,
|
||||
Relationship.deleted_at.is_(None),
|
||||
or_(
|
||||
Relationship.person_from_id == person_id,
|
||||
Relationship.person_to_id == person_id,
|
||||
),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
for r in rels:
|
||||
r.deleted_at = now
|
||||
await session.execute(
|
||||
update(User).where(User.self_person_id == person_id).values(self_person_id=None)
|
||||
)
|
||||
|
||||
# Precompute the best match per incoming xref (for default-policy resolution).
|
||||
matches: dict[str, dict] = {}
|
||||
for rec in roots:
|
||||
if rec.tag == "INDI" and rec.xref:
|
||||
summ = _person_summary(rec)
|
||||
entry, _score = _best_match(summ["norm"], summ["year"], index)
|
||||
if entry is not None:
|
||||
matches[rec.xref] = entry
|
||||
|
||||
def resolve(xref: str) -> tuple[str, uuid.UUID | None]:
|
||||
ov = resolutions.get(xref)
|
||||
if ov:
|
||||
action = ov.get("action", "new")
|
||||
tid = ov.get("target_id")
|
||||
target = uuid.UUID(tid) if tid else (matches[xref]["id"] if xref in matches else None)
|
||||
if action in ("skip", "merge", "overwrite") and target is None:
|
||||
return "new", None
|
||||
return action, target
|
||||
if default_action != "new" and xref in matches:
|
||||
return default_action, matches[xref]["id"]
|
||||
return "new", None
|
||||
|
||||
# Individuals.
|
||||
for rec in roots:
|
||||
if rec.tag != "INDI" or not rec.xref:
|
||||
continue
|
||||
names = _extract_names(rec)
|
||||
action, target = resolve(rec.xref)
|
||||
|
||||
if action == "skip" and target is not None:
|
||||
person_map[rec.xref] = target
|
||||
counts["skipped"] += 1
|
||||
continue
|
||||
if action == "merge" and target is not None:
|
||||
person_map[rec.xref] = target
|
||||
add_names(target, names, set_primary=False)
|
||||
await add_events(rec, target)
|
||||
await add_citations(rec, person_id=target)
|
||||
note = _notes_text(rec)
|
||||
if note:
|
||||
existing = (
|
||||
await session.execute(select(Person).where(Person.id == target))
|
||||
).scalar_one_or_none()
|
||||
if existing is not None:
|
||||
existing.notes = "\n".join(filter(None, [existing.notes, note]))
|
||||
counts["merged"] += 1
|
||||
continue
|
||||
if action == "overwrite" and target is not None:
|
||||
await soft_delete_existing(target)
|
||||
counts["overwritten"] += 1
|
||||
|
||||
person = Person(tree_id=tree.id, gender=_sex(rec.text("SEX")), notes=_notes_text(rec))
|
||||
session.add(person)
|
||||
await session.flush()
|
||||
person_map[rec.xref] = person.id
|
||||
counts["persons"] += 1
|
||||
add_names(person.id, names, set_primary=True)
|
||||
await add_citations(rec, person_id=person.id)
|
||||
await add_events(rec, person.id)
|
||||
|
||||
# Families -> partnerships, parent-child edges, marriage events.
|
||||
for rec in roots:
|
||||
if rec.tag != "FAM":
|
||||
@@ -238,17 +584,22 @@ async def import_gedcom(
|
||||
husb = person_map.get((rec.text("HUSB") or "").strip())
|
||||
wife = person_map.get((rec.text("WIFE") or "").strip())
|
||||
partnership_id: uuid.UUID | None = None
|
||||
if husb and wife:
|
||||
rel = Relationship(
|
||||
tree_id=tree.id,
|
||||
type=RelationshipType.partnership,
|
||||
person_from_id=husb,
|
||||
person_to_id=wife,
|
||||
if husb and wife and husb != wife:
|
||||
rel = add_relationship(RelationshipType.partnership, husb, wife)
|
||||
if rel is not None:
|
||||
await session.flush()
|
||||
partnership_id = rel.id
|
||||
if partnership_id is None and husb and wife:
|
||||
# Edge already existed — find it so marriage events can attach.
|
||||
existing = next(
|
||||
(
|
||||
r for r in existing_rels
|
||||
if r.type == RelationshipType.partnership
|
||||
and {r.person_from_id, r.person_to_id} == {husb, wife}
|
||||
),
|
||||
None,
|
||||
)
|
||||
session.add(rel)
|
||||
await session.flush()
|
||||
partnership_id = rel.id
|
||||
counts["relationships"] += 1
|
||||
partnership_id = existing.id if existing else None
|
||||
|
||||
for fe in rec.children:
|
||||
if fe.tag in FAM_EVENTS and partnership_id is not None:
|
||||
@@ -271,16 +622,12 @@ async def import_gedcom(
|
||||
continue
|
||||
for parent in (husb, wife):
|
||||
if parent and parent != cp:
|
||||
session.add(
|
||||
Relationship(
|
||||
tree_id=tree.id,
|
||||
type=RelationshipType.parent_child,
|
||||
person_from_id=parent,
|
||||
person_to_id=cp,
|
||||
qualifier=ParentChildQualifier.biological,
|
||||
)
|
||||
add_relationship(
|
||||
RelationshipType.parent_child,
|
||||
parent,
|
||||
cp,
|
||||
qualifier=ParentChildQualifier.biological,
|
||||
)
|
||||
counts["relationships"] += 1
|
||||
|
||||
record_audit(
|
||||
session,
|
||||
@@ -345,10 +692,45 @@ async def export_gedcom(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tr
|
||||
await session.execute(select(Place).where(Place.tree_id == tree.id))
|
||||
).scalars().all()
|
||||
}
|
||||
citations = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Citation).where(
|
||||
Citation.tree_id == tree.id, Citation.deleted_at.is_(None)
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
)
|
||||
|
||||
pxref = {p.id: f"@I{i + 1}@" for i, p in enumerate(persons)}
|
||||
gender_by_id = {p.id: p.gender for p in persons}
|
||||
sxref = {s.id: f"@S{i + 1}@" for i, s in enumerate(sources)}
|
||||
# Citations grouped by the fact they sit on, so each fact can emit its SOUR
|
||||
# links (dropping these is the round-trip data loss this fixes). Skip any
|
||||
# whose source didn't export.
|
||||
cite_by_person: dict[uuid.UUID, list[Citation]] = defaultdict(list)
|
||||
cite_by_name: dict[uuid.UUID, list[Citation]] = defaultdict(list)
|
||||
cite_by_event: dict[uuid.UUID, list[Citation]] = defaultdict(list)
|
||||
cite_by_rel: dict[uuid.UUID, list[Citation]] = defaultdict(list)
|
||||
for c in citations:
|
||||
if c.source_id not in sxref:
|
||||
continue
|
||||
if c.person_id:
|
||||
cite_by_person[c.person_id].append(c)
|
||||
elif c.event_id:
|
||||
cite_by_event[c.event_id].append(c)
|
||||
elif c.name_id:
|
||||
cite_by_name[c.name_id].append(c)
|
||||
elif c.relationship_id:
|
||||
cite_by_rel[c.relationship_id].append(c)
|
||||
|
||||
def cite_lines(cites: list[Citation], depth: int) -> list[str]:
|
||||
lines: list[str] = []
|
||||
for c in cites:
|
||||
lines.append(f"{depth} SOUR {sxref[c.source_id]}")
|
||||
if c.page:
|
||||
lines.append(f"{depth + 1} PAGE {c.page}")
|
||||
return lines
|
||||
names_by_person: dict[uuid.UUID, list[Name]] = defaultdict(list)
|
||||
for n in sorted(names, key=lambda n: (n.sort_order, not n.is_primary)):
|
||||
names_by_person[n.person_id].append(n)
|
||||
@@ -397,6 +779,10 @@ async def export_gedcom(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tr
|
||||
for n in names_by_person.get(p.id, []):
|
||||
display = n.display_name or f"{n.given or ''} /{n.surname or ''}/".strip()
|
||||
out.append(f"1 NAME {display}")
|
||||
ged_type = EXPORT_TYPE_MAP.get(n.name_type)
|
||||
if ged_type:
|
||||
out.append(f"2 TYPE {ged_type}")
|
||||
out += cite_lines(cite_by_name.get(n.id, []), 2)
|
||||
sex = {"male": "M", "female": "F"}.get(p.gender or "")
|
||||
if sex:
|
||||
out.append(f"1 SEX {sex}")
|
||||
@@ -409,6 +795,8 @@ async def export_gedcom(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tr
|
||||
out.append(f"2 DATE {e.date_value}")
|
||||
if e.place_id and e.place_id in places:
|
||||
out.append(f"2 PLAC {places[e.place_id].name}")
|
||||
out += cite_lines(cite_by_event.get(e.id, []), 2)
|
||||
out += cite_lines(cite_by_person.get(p.id, []), 1)
|
||||
if p.id in child_fams:
|
||||
out.append(f"1 FAMC {child_fams[p.id]}")
|
||||
for x in spouse_fams.get(p.id, []):
|
||||
@@ -437,6 +825,8 @@ async def export_gedcom(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tr
|
||||
out.append(f"1 {tag}")
|
||||
if _ged_date(e.date_value):
|
||||
out.append(f"2 DATE {e.date_value}")
|
||||
out += cite_lines(cite_by_event.get(e.id, []), 2)
|
||||
out += cite_lines(cite_by_rel.get(f["rel_id"], []), 1)
|
||||
|
||||
for s in sources:
|
||||
out.append(f"0 {sxref[s.id]} SOUR")
|
||||
|
||||
@@ -72,6 +72,13 @@ async def upload_media(
|
||||
async def list_media(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> list[Media]:
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
# Non-members only see media of a FULL-visibility person (no living-person photos).
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
return await public_view_service.list_public_media(
|
||||
session, viewer_id=viewer_id, tree=tree
|
||||
)
|
||||
stmt = (
|
||||
select(Media)
|
||||
.where(Media.tree_id == tree.id, Media.deleted_at.is_(None))
|
||||
@@ -94,6 +101,15 @@ async def get_media(
|
||||
).scalar_one_or_none()
|
||||
if media is None:
|
||||
raise NotFound("media not found")
|
||||
# Non-members may only see/download media of a FULL-visibility person. 404
|
||||
# (not 403) so the item's existence isn't revealed. This gates media_content.
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
if not await public_view_service.can_view_media(
|
||||
session, viewer_id=viewer_id, tree=tree, media=media
|
||||
):
|
||||
raise NotFound("media not found")
|
||||
return media
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,156 @@
|
||||
"""Tree membership management: list / add / change-role / remove.
|
||||
|
||||
Only an owner may change membership. A tree must always keep at least one owner.
|
||||
The member list (which exposes user emails) is visible only to members — never
|
||||
to a non-member viewing a public/unlisted tree.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.enums import MembershipRole
|
||||
from app.models.tree import Tree, TreeMembership
|
||||
from app.models.user import User
|
||||
from app.services import privacy
|
||||
from app.services.audit import record_audit
|
||||
from app.services.exceptions import Conflict, Forbidden, NotFound
|
||||
|
||||
|
||||
async def _require_owner(session: AsyncSession, *, actor_id: uuid.UUID, tree: Tree) -> None:
|
||||
if await privacy.get_membership_role(session, actor_id, tree.id) is not MembershipRole.owner:
|
||||
raise Forbidden("only the owner can manage members")
|
||||
|
||||
|
||||
async def _owner_count(session: AsyncSession, tree_id: uuid.UUID) -> int:
|
||||
return (
|
||||
await session.execute(
|
||||
select(func.count())
|
||||
.select_from(TreeMembership)
|
||||
.where(TreeMembership.tree_id == tree_id, TreeMembership.role == MembershipRole.owner)
|
||||
)
|
||||
).scalar_one()
|
||||
|
||||
|
||||
def _row(m: TreeMembership, u: User) -> dict:
|
||||
return {
|
||||
"id": m.id,
|
||||
"user_id": u.id,
|
||||
"email": u.email,
|
||||
"display_name": u.display_name,
|
||||
"role": m.role,
|
||||
"created_at": m.created_at,
|
||||
}
|
||||
|
||||
|
||||
async def list_members(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> list[dict]:
|
||||
# Member-only: the list exposes emails, so a non-member (even on a public
|
||||
# tree) must not see it.
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
raise Forbidden("only members can see the member list")
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(TreeMembership, User)
|
||||
.join(User, User.id == TreeMembership.user_id)
|
||||
.where(TreeMembership.tree_id == tree.id)
|
||||
.order_by(TreeMembership.created_at)
|
||||
)
|
||||
).all()
|
||||
return [_row(m, u) for m, u in rows]
|
||||
|
||||
|
||||
async def add_member(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, email: str, role: MembershipRole
|
||||
) -> dict:
|
||||
await _require_owner(session, actor_id=actor.id, tree=tree)
|
||||
user = (
|
||||
await session.execute(
|
||||
select(User).where(User.email == email, User.deleted_at.is_(None))
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if user is None:
|
||||
raise NotFound("no user with that email on this instance")
|
||||
if await privacy.get_membership_role(session, user.id, tree.id) is not None:
|
||||
raise Conflict("that user is already a member")
|
||||
m = TreeMembership(tree_id=tree.id, user_id=user.id, role=role)
|
||||
session.add(m)
|
||||
record_audit(
|
||||
session,
|
||||
action="add_member",
|
||||
entity_type="Tree",
|
||||
entity_id=tree.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
after={"user_id": str(user.id), "role": role.value},
|
||||
)
|
||||
await session.commit()
|
||||
await session.refresh(m)
|
||||
return _row(m, user)
|
||||
|
||||
|
||||
async def _get_membership(
|
||||
session: AsyncSession, tree: Tree, membership_id: uuid.UUID
|
||||
) -> TreeMembership:
|
||||
m = (
|
||||
await session.execute(
|
||||
select(TreeMembership).where(
|
||||
TreeMembership.id == membership_id, TreeMembership.tree_id == tree.id
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if m is None:
|
||||
raise NotFound("member not found")
|
||||
return m
|
||||
|
||||
|
||||
async def update_member_role(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
actor: User,
|
||||
tree: Tree,
|
||||
membership_id: uuid.UUID,
|
||||
role: MembershipRole,
|
||||
) -> dict:
|
||||
await _require_owner(session, actor_id=actor.id, tree=tree)
|
||||
m = await _get_membership(session, tree, membership_id)
|
||||
if (
|
||||
m.role == MembershipRole.owner
|
||||
and role != MembershipRole.owner
|
||||
and await _owner_count(session, tree.id) <= 1
|
||||
):
|
||||
raise Conflict("a tree must keep at least one owner")
|
||||
m.role = role
|
||||
record_audit(
|
||||
session,
|
||||
action="update_member",
|
||||
entity_type="Tree",
|
||||
entity_id=tree.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
after={"membership_id": str(m.id), "role": role.value},
|
||||
)
|
||||
await session.commit()
|
||||
await session.refresh(m)
|
||||
u = (await session.execute(select(User).where(User.id == m.user_id))).scalar_one()
|
||||
return _row(m, u)
|
||||
|
||||
|
||||
async def remove_member(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, membership_id: uuid.UUID
|
||||
) -> None:
|
||||
await _require_owner(session, actor_id=actor.id, tree=tree)
|
||||
m = await _get_membership(session, tree, membership_id)
|
||||
if m.role == MembershipRole.owner and await _owner_count(session, tree.id) <= 1:
|
||||
raise Conflict("a tree must keep at least one owner")
|
||||
await session.delete(m)
|
||||
record_audit(
|
||||
session,
|
||||
action="remove_member",
|
||||
entity_type="Tree",
|
||||
entity_id=tree.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
after={"membership_id": str(membership_id)},
|
||||
)
|
||||
await session.commit()
|
||||
@@ -0,0 +1,69 @@
|
||||
"""A curated given-name -> sex lookup for best-guessing a person's sex from
|
||||
their first name. Weighted toward English + German names (this codebase's first
|
||||
real tree is a German-American family). Deterministic and offline — no model
|
||||
needed; the Cleanup tool previews every guess before anything is applied.
|
||||
|
||||
Genuinely ambiguous names (Marion, Frances/Francis, Jordan, Jamie, Robin, Leslie,
|
||||
Dana, …) are intentionally left out of BOTH sets so they aren't guessed — better
|
||||
a human decides those than a coin flip.
|
||||
"""
|
||||
|
||||
MALE_NAMES: set[str] = {
|
||||
# English / common US
|
||||
"james", "john", "robert", "michael", "william", "david", "richard", "joseph",
|
||||
"thomas", "charles", "christopher", "daniel", "matthew", "anthony", "donald",
|
||||
"mark", "paul", "steven", "andrew", "kenneth", "george", "joshua", "kevin",
|
||||
"brian", "edward", "ronald", "timothy", "jason", "jeffrey", "gary", "ryan",
|
||||
"nicholas", "eric", "stephen", "jacob", "larry", "frank", "jonathan", "scott",
|
||||
"raymond", "gregory", "samuel", "benjamin", "patrick", "jack", "dennis", "jerry",
|
||||
"alexander", "tyler", "henry", "douglas", "peter", "adam", "harold", "albert",
|
||||
"arthur", "carl", "ralph", "roy", "eugene", "louis", "philip", "bobby", "walter",
|
||||
"willie", "wayne", "fred", "howard", "ernest", "earl", "clarence", "leon",
|
||||
"leonard", "lewis", "floyd", "leroy", "elmer", "homer", "orrin", "josias",
|
||||
"emerson", "dale", "bernard", "vernon", "virgil", "wilbur", "russell",
|
||||
"harvey", "herbert", "melvin", "lloyd", "marvin", "norman", "stanley",
|
||||
# German
|
||||
"hans", "karl", "wilhelm", "friedrich", "heinrich", "otto", "hermann", "gustav",
|
||||
"ludwig", "ernst", "fritz", "johann", "conrad", "konrad", "reinhold", "rudolf",
|
||||
"rudolph", "gerhard", "helmut", "horst", "klaus", "kurt", "dieter", "günther",
|
||||
"gunther", "manfred", "siegfried", "hilgard", "christian", "august", "wolfgang",
|
||||
"jürgen", "jurgen", "matthias", "lothar", "bruno", "gottlieb", "reinhard",
|
||||
}
|
||||
|
||||
FEMALE_NAMES: set[str] = {
|
||||
# English / common US
|
||||
"mary", "patricia", "jennifer", "linda", "elizabeth", "barbara", "susan",
|
||||
"jessica", "sarah", "karen", "nancy", "lisa", "betty", "margaret", "sandra",
|
||||
"ashley", "kimberly", "emily", "donna", "michelle", "carol", "amanda", "dorothy",
|
||||
"melissa", "deborah", "stephanie", "rebecca", "sharon", "laura", "cynthia",
|
||||
"kathleen", "amy", "angela", "shirley", "anna", "ruth", "brenda", "pamela",
|
||||
"nicole", "katherine", "virginia", "catherine", "helen", "debra", "rachel",
|
||||
"carolyn", "janet", "maria", "heather", "diane", "julie", "joyce", "victoria",
|
||||
"kelly", "christina", "joan", "evelyn", "judith", "megan", "alice", "frances",
|
||||
"marie", "florence", "flora", "zella", "thelma", "ellen", "althea", "della",
|
||||
"beatrice", "pauline", "hedwig", "florentine", "wilhelmina", "augusta", "bertha",
|
||||
"gladys", "mildred", "lucille", "edith", "esther", "irene", "hazel", "doris",
|
||||
"rose", "rita", "norma", "june", "lois", "marjorie",
|
||||
# German
|
||||
"greta", "ilse", "ursula", "gertrud", "gertrude", "frieda", "frida", "else",
|
||||
"hilda", "hilde", "hildegard", "ingrid", "helga", "renate", "monika", "sieglinde",
|
||||
"brigitte", "gisela", "elke", "anneliese", "waltraud", "edeltraud", "johanna",
|
||||
"katharina", "margarethe", "wilhelmine", "emilie", "auguste",
|
||||
}
|
||||
|
||||
|
||||
def guess_sex(given: str | None) -> str | None:
|
||||
"""Best-guess "male"/"female" from the first token of a given name, or None
|
||||
if unknown/ambiguous."""
|
||||
if not given:
|
||||
return None
|
||||
first = given.strip().split()[0].lower() if given.strip() else ""
|
||||
# Strip trailing punctuation/initials like "wm." -> "wm".
|
||||
first = first.strip(".,'\"")
|
||||
if not first:
|
||||
return None
|
||||
if first in MALE_NAMES:
|
||||
return "male"
|
||||
if first in FEMALE_NAMES:
|
||||
return "female"
|
||||
return None
|
||||
@@ -0,0 +1,215 @@
|
||||
"""Name service. A Person carries one or more Name rows — a primary (typically
|
||||
the birth/maiden name) plus typed alternates (married, alias, religious, …).
|
||||
Exactly one name is primary at a time; it drives display everywhere. Writes
|
||||
require editor rights; reads go through the tree's view check.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.person import Name, Person
|
||||
from app.models.tree import Tree
|
||||
from app.models.user import User
|
||||
from app.services import privacy
|
||||
from app.services.audit import record_audit
|
||||
from app.services.exceptions import Forbidden, NotFound
|
||||
|
||||
|
||||
async def _get_person(session: AsyncSession, *, tree: Tree, person_id: uuid.UUID) -> Person:
|
||||
person = (
|
||||
await session.execute(
|
||||
select(Person).where(
|
||||
Person.id == person_id, Person.tree_id == tree.id, Person.deleted_at.is_(None)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if person is None:
|
||||
raise NotFound("person not found")
|
||||
return person
|
||||
|
||||
|
||||
async def _clear_primary(
|
||||
session: AsyncSession, *, person_id: uuid.UUID, keep: uuid.UUID | None
|
||||
) -> None:
|
||||
"""Demote every other name so exactly one stays primary."""
|
||||
stmt = (
|
||||
update(Name)
|
||||
.where(Name.person_id == person_id, Name.deleted_at.is_(None), Name.is_primary.is_(True))
|
||||
.values(is_primary=False)
|
||||
)
|
||||
if keep is not None:
|
||||
stmt = stmt.where(Name.id != keep)
|
||||
await session.execute(stmt)
|
||||
|
||||
|
||||
async def list_names(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, person_id: uuid.UUID
|
||||
) -> list[Name]:
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
await _get_person(session, tree=tree, person_id=person_id)
|
||||
# Non-members: a redacted/hidden person's real names must not leak.
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
return await public_view_service.list_public_person_names(
|
||||
session, viewer_id=viewer_id, tree=tree, person_id=person_id
|
||||
)
|
||||
stmt = (
|
||||
select(Name)
|
||||
.where(Name.person_id == person_id, Name.deleted_at.is_(None))
|
||||
.order_by(Name.is_primary.desc(), Name.sort_order, Name.created_at)
|
||||
)
|
||||
return list((await session.execute(stmt)).scalars().all())
|
||||
|
||||
|
||||
async def create_name(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
actor: User,
|
||||
tree: Tree,
|
||||
person_id: uuid.UUID,
|
||||
name_type: str = "birth",
|
||||
given: str | None = None,
|
||||
surname: str | None = None,
|
||||
prefix: str | None = None,
|
||||
suffix: str | None = None,
|
||||
nickname: str | None = None,
|
||||
is_primary: bool = False,
|
||||
) -> Name:
|
||||
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
|
||||
raise Forbidden("not an editor of this tree")
|
||||
await _get_person(session, tree=tree, person_id=person_id)
|
||||
|
||||
# First name for a person is always primary; otherwise honor the flag.
|
||||
existing = (
|
||||
await session.execute(
|
||||
select(Name.id).where(Name.person_id == person_id, Name.deleted_at.is_(None))
|
||||
)
|
||||
).first()
|
||||
primary = is_primary or existing is None
|
||||
if primary:
|
||||
await _clear_primary(session, person_id=person_id, keep=None)
|
||||
|
||||
name = Name(
|
||||
tree_id=tree.id,
|
||||
person_id=person_id,
|
||||
name_type=name_type,
|
||||
given=given,
|
||||
surname=surname,
|
||||
prefix=prefix,
|
||||
suffix=suffix,
|
||||
nickname=nickname,
|
||||
is_primary=primary,
|
||||
)
|
||||
session.add(name)
|
||||
await session.flush()
|
||||
record_audit(
|
||||
session,
|
||||
action="create",
|
||||
entity_type="Name",
|
||||
entity_id=name.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
after={"name_type": name_type, "given": given, "surname": surname},
|
||||
)
|
||||
await session.commit()
|
||||
await session.refresh(name)
|
||||
return name
|
||||
|
||||
|
||||
_NAME_FIELDS = {"name_type", "given", "surname", "prefix", "suffix", "nickname"}
|
||||
|
||||
|
||||
async def update_name(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
actor: User,
|
||||
tree: Tree,
|
||||
person_id: uuid.UUID,
|
||||
name_id: uuid.UUID,
|
||||
changes: dict,
|
||||
) -> Name:
|
||||
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
|
||||
raise Forbidden("not an editor of this tree")
|
||||
name = (
|
||||
await session.execute(
|
||||
select(Name).where(
|
||||
Name.id == name_id,
|
||||
Name.person_id == person_id,
|
||||
Name.tree_id == tree.id,
|
||||
Name.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if name is None:
|
||||
raise NotFound("name not found")
|
||||
|
||||
for key in _NAME_FIELDS & changes.keys():
|
||||
setattr(name, key, changes[key])
|
||||
if changes.get("is_primary") is True:
|
||||
await _clear_primary(session, person_id=person_id, keep=name.id)
|
||||
name.is_primary = True
|
||||
|
||||
record_audit(
|
||||
session,
|
||||
action="update",
|
||||
entity_type="Name",
|
||||
entity_id=name.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
after=changes,
|
||||
)
|
||||
await session.commit()
|
||||
await session.refresh(name)
|
||||
return name
|
||||
|
||||
|
||||
async def delete_name(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
actor: User,
|
||||
tree: Tree,
|
||||
person_id: uuid.UUID,
|
||||
name_id: uuid.UUID,
|
||||
) -> None:
|
||||
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
|
||||
raise Forbidden("not an editor of this tree")
|
||||
name = (
|
||||
await session.execute(
|
||||
select(Name).where(
|
||||
Name.id == name_id,
|
||||
Name.person_id == person_id,
|
||||
Name.tree_id == tree.id,
|
||||
Name.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if name is None:
|
||||
raise NotFound("name not found")
|
||||
name.deleted_at = datetime.now(UTC)
|
||||
was_primary = name.is_primary
|
||||
name.is_primary = False
|
||||
record_audit(
|
||||
session,
|
||||
action="delete",
|
||||
entity_type="Name",
|
||||
entity_id=name.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
)
|
||||
# Promote another name to primary so the person never loses their display name.
|
||||
if was_primary:
|
||||
nxt = (
|
||||
await session.execute(
|
||||
select(Name)
|
||||
.where(Name.person_id == person_id, Name.deleted_at.is_(None))
|
||||
.order_by(Name.sort_order, Name.created_at)
|
||||
)
|
||||
).scalars().first()
|
||||
if nxt is not None:
|
||||
nxt.is_primary = True
|
||||
await session.commit()
|
||||
@@ -6,11 +6,12 @@ person through the privacy engine. Each returned Person gets a transient
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import func, or_, select
|
||||
from sqlalchemy import func, or_, select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.enums import PersonPrivacy
|
||||
from app.models.enums import PersonPrivacy, RelationshipType
|
||||
from app.models.person import Name, Person
|
||||
from app.models.relationship import Relationship
|
||||
from app.models.tree import Tree
|
||||
from app.models.user import User
|
||||
from app.services import privacy
|
||||
@@ -44,6 +45,29 @@ async def _attach_primary_name(session: AsyncSession, person: Person) -> None:
|
||||
person.primary_name = _format_name(name) if name is not None else None
|
||||
|
||||
|
||||
async def _attach_primary_names(session: AsyncSession, persons: list[Person]) -> None:
|
||||
"""Batch version of ``_attach_primary_name`` — ONE query for the whole list
|
||||
instead of one per person (the difference between 1 and N queries when
|
||||
rendering a 2k-person tree). The global order (is_primary desc, sort_order)
|
||||
matches the single-person query, so the first row seen per person is the same
|
||||
name ``_attach_primary_name`` would pick."""
|
||||
if not persons:
|
||||
return
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(Name)
|
||||
.where(Name.person_id.in_([p.id for p in persons]), Name.deleted_at.is_(None))
|
||||
.order_by(Name.is_primary.desc(), Name.sort_order)
|
||||
)
|
||||
).scalars().all()
|
||||
best: dict[uuid.UUID, Name] = {}
|
||||
for n in rows:
|
||||
best.setdefault(n.person_id, n)
|
||||
for p in persons:
|
||||
n = best.get(p.id)
|
||||
p.primary_name = _format_name(n) if n is not None else None
|
||||
|
||||
|
||||
async def create_person(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
@@ -177,9 +201,65 @@ async def get_person(
|
||||
return person
|
||||
|
||||
|
||||
async def delete_person(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, person_id: uuid.UUID
|
||||
async def _children_of(
|
||||
session: AsyncSession, *, tree_id: uuid.UUID, parent_id: uuid.UUID
|
||||
) -> list[uuid.UUID]:
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(Relationship.person_to_id).where(
|
||||
Relationship.tree_id == tree_id,
|
||||
Relationship.deleted_at.is_(None),
|
||||
Relationship.type == RelationshipType.parent_child,
|
||||
Relationship.person_from_id == parent_id,
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
return list(rows)
|
||||
|
||||
|
||||
async def _soft_delete_one(
|
||||
session: AsyncSession, *, actor: User, tree: Tree, person: Person, now: datetime
|
||||
) -> None:
|
||||
"""Soft-delete a single person and the relationships touching them, so no
|
||||
dangling edges are left to break the tree view."""
|
||||
person.deleted_at = now
|
||||
rels = (
|
||||
await session.execute(
|
||||
select(Relationship).where(
|
||||
Relationship.tree_id == tree.id,
|
||||
Relationship.deleted_at.is_(None),
|
||||
or_(
|
||||
Relationship.person_from_id == person.id,
|
||||
Relationship.person_to_id == person.id,
|
||||
),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
for rel in rels:
|
||||
rel.deleted_at = now
|
||||
record_audit(
|
||||
session,
|
||||
action="delete",
|
||||
entity_type="Person",
|
||||
entity_id=person.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
after={"cascaded_relationships": len(rels)},
|
||||
)
|
||||
|
||||
|
||||
async def delete_person(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
actor: User,
|
||||
tree: Tree,
|
||||
person_id: uuid.UUID,
|
||||
cascade: bool = False,
|
||||
) -> int:
|
||||
"""Soft-delete a person. Always removes the relationships that touch them
|
||||
(preventing dangling edges). With ``cascade=True``, recursively deletes
|
||||
their descendants too — handy for pruning a bad GEDCOM import. Returns the
|
||||
number of persons deleted."""
|
||||
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
|
||||
raise Forbidden("not an editor of this tree")
|
||||
person = (
|
||||
@@ -191,16 +271,52 @@ async def delete_person(
|
||||
).scalar_one_or_none()
|
||||
if person is None:
|
||||
raise NotFound("person not found")
|
||||
person.deleted_at = datetime.now(UTC)
|
||||
record_audit(
|
||||
session,
|
||||
action="delete",
|
||||
entity_type="Person",
|
||||
entity_id=person.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
|
||||
now = datetime.now(UTC)
|
||||
|
||||
# Gather the set of persons to delete. For cascade, walk descendants
|
||||
# breadth-first, guarding against cycles.
|
||||
to_delete: list[Person] = [person]
|
||||
if cascade:
|
||||
seen = {person.id}
|
||||
frontier = [person.id]
|
||||
while frontier:
|
||||
nxt: list[uuid.UUID] = []
|
||||
for pid in frontier:
|
||||
for child_id in await _children_of(session, tree_id=tree.id, parent_id=pid):
|
||||
if child_id not in seen:
|
||||
seen.add(child_id)
|
||||
nxt.append(child_id)
|
||||
frontier = nxt
|
||||
extra_ids = [pid for pid in seen if pid != person.id]
|
||||
if extra_ids:
|
||||
extra = (
|
||||
await session.execute(
|
||||
select(Person).where(
|
||||
Person.id.in_(extra_ids),
|
||||
Person.tree_id == tree.id,
|
||||
Person.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
to_delete.extend(extra)
|
||||
|
||||
for p in to_delete:
|
||||
await _soft_delete_one(session, actor=actor, tree=tree, person=p, now=now)
|
||||
|
||||
# Soft delete leaves the row in place, so the DB-level "ON DELETE SET NULL"
|
||||
# never fires — clear any links (account self-person, tree home person) to a
|
||||
# deleted person.
|
||||
deleted_ids = [p.id for p in to_delete]
|
||||
await session.execute(
|
||||
update(User).where(User.self_person_id.in_(deleted_ids)).values(self_person_id=None)
|
||||
)
|
||||
await session.execute(
|
||||
update(Tree).where(Tree.home_person_id.in_(deleted_ids)).values(home_person_id=None)
|
||||
)
|
||||
|
||||
await session.commit()
|
||||
return len(to_delete)
|
||||
|
||||
|
||||
async def restore_person(
|
||||
@@ -243,15 +359,18 @@ async def list_deleted_persons(
|
||||
.order_by(Person.deleted_at.desc())
|
||||
)
|
||||
persons = list((await session.execute(stmt)).scalars().all())
|
||||
for person in persons:
|
||||
await _attach_primary_name(session, person)
|
||||
await _attach_primary_names(session, persons)
|
||||
return persons
|
||||
|
||||
|
||||
async def list_persons(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree
|
||||
) -> list[Person]:
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
# Resolve the viewer's role ONCE. Members see the whole tree (full), so we
|
||||
# skip the per-person privacy engine entirely and batch the name fetch — the
|
||||
# difference between ~3 queries and ~3·N queries on a 2k-person tree.
|
||||
role = await privacy.get_membership_role(session, viewer_id, tree.id)
|
||||
if role is None and not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
|
||||
stmt = (
|
||||
@@ -261,7 +380,15 @@ async def list_persons(
|
||||
)
|
||||
persons = list((await session.execute(stmt)).scalars().all())
|
||||
|
||||
if role is not None:
|
||||
await _attach_primary_names(session, persons)
|
||||
return persons
|
||||
|
||||
# Non-member on a viewable (public/unlisted/site_members) tree: redact per
|
||||
# person. Names are batched for the non-redacted ones; redacted ones already
|
||||
# have their display name overwritten by _redact.
|
||||
visible: list[Person] = []
|
||||
full: list[Person] = []
|
||||
for person in persons:
|
||||
vis = await privacy.person_visibility(
|
||||
session, user_id=viewer_id, tree=tree, person=person
|
||||
@@ -271,8 +398,50 @@ async def list_persons(
|
||||
if vis == Visibility.redacted:
|
||||
_redact(person)
|
||||
else:
|
||||
await _attach_primary_name(session, person)
|
||||
full.append(person)
|
||||
visible.append(person)
|
||||
await _attach_primary_names(session, full)
|
||||
return visible
|
||||
|
||||
|
||||
async def list_persons_by_ids(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree, ids: list[uuid.UUID]
|
||||
) -> list[Person]:
|
||||
"""Just the named persons (privacy-filtered, names batched). Lets a page show
|
||||
the names of someone's relatives without loading the whole tree."""
|
||||
role = await privacy.get_membership_role(session, viewer_id, tree.id)
|
||||
if role is None and not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
if not ids:
|
||||
return []
|
||||
persons = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Person).where(
|
||||
Person.id.in_(ids),
|
||||
Person.tree_id == tree.id,
|
||||
Person.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
)
|
||||
if role is not None:
|
||||
await _attach_primary_names(session, persons)
|
||||
return persons
|
||||
visible: list[Person] = []
|
||||
full: list[Person] = []
|
||||
for person in persons:
|
||||
vis = await privacy.person_visibility(
|
||||
session, user_id=viewer_id, tree=tree, person=person
|
||||
)
|
||||
if vis == Visibility.hidden:
|
||||
continue
|
||||
if vis == Visibility.redacted:
|
||||
_redact(person)
|
||||
else:
|
||||
full.append(person)
|
||||
visible.append(person)
|
||||
await _attach_primary_names(session, full)
|
||||
return visible
|
||||
|
||||
|
||||
@@ -313,7 +482,11 @@ async def search_persons(
|
||||
.order_by(sub.c.score.desc())
|
||||
)
|
||||
persons = list((await session.execute(stmt)).scalars().all())
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is not None:
|
||||
await _attach_primary_names(session, persons)
|
||||
return persons
|
||||
out: list[Person] = []
|
||||
full: list[Person] = []
|
||||
for person in persons:
|
||||
vis = await privacy.person_visibility(
|
||||
session, user_id=viewer_id, tree=tree, person=person
|
||||
@@ -323,6 +496,7 @@ async def search_persons(
|
||||
if vis == Visibility.redacted:
|
||||
_redact(person)
|
||||
else:
|
||||
await _attach_primary_name(session, person)
|
||||
full.append(person)
|
||||
out.append(person)
|
||||
await _attach_primary_names(session, full)
|
||||
return out
|
||||
|
||||
@@ -45,8 +45,17 @@ async def can_view_tree(session: AsyncSession, *, user_id: uuid.UUID | None, tre
|
||||
if tree.deleted_at is not None:
|
||||
return False
|
||||
if await get_membership_role(session, user_id, tree.id) is not None:
|
||||
return True # members always (any role)
|
||||
# Non-members. Branch on the viewer's auth state:
|
||||
# public / unlisted → anyone, including anonymous (unlisted is gated only
|
||||
# by knowing the link, so the API must never *list* it).
|
||||
# site_members → any authenticated account on this instance.
|
||||
# private → no one.
|
||||
if tree.visibility in (TreeVisibility.public, TreeVisibility.unlisted):
|
||||
return True
|
||||
return tree.visibility in (TreeVisibility.public, TreeVisibility.unlisted)
|
||||
if tree.visibility == TreeVisibility.site_members:
|
||||
return user_id is not None
|
||||
return False
|
||||
|
||||
|
||||
async def can_edit_tree(session: AsyncSession, *, user_id: uuid.UUID | None, tree: Tree) -> bool:
|
||||
|
||||
@@ -0,0 +1,416 @@
|
||||
"""Read-only, redaction-safe projections for the public viewing surface.
|
||||
|
||||
INVARIANT (CLAUDE.md #2): everything returned here has passed through
|
||||
``privacy.person_visibility``. A non-member must never receive a possibly-living
|
||||
person's real name, dates, alternate names, or media. The rules:
|
||||
|
||||
- persons : redacted (living → "Living person"); hidden dropped.
|
||||
- relationships : only when BOTH endpoints are non-hidden (a link to a
|
||||
redacted person is fine — the name is already hidden).
|
||||
- events : only for FULL-visibility persons; partnership events only
|
||||
when BOTH partners are full (a marriage date would leak a
|
||||
living partner's timeline otherwise).
|
||||
- names : only for FULL-visibility persons.
|
||||
- media : NOT exposed yet (deferred — see docs/design/tree-visibility.md).
|
||||
- citations : only when the cited fact resolves to FULL person(s).
|
||||
- sources : only when they back at least one visible citation.
|
||||
|
||||
A tree that isn't viewable raises NotFound (never Forbidden) so the public
|
||||
surface can't be used to probe whether a private tree exists.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
|
||||
from sqlalchemy import or_, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.enums import TreeVisibility
|
||||
from app.models.event import Event
|
||||
from app.models.media import Media
|
||||
from app.models.person import Name, Person
|
||||
from app.models.relationship import Relationship
|
||||
from app.models.source import Citation, Source
|
||||
from app.models.tree import Tree
|
||||
from app.services import privacy
|
||||
from app.services.exceptions import NotFound
|
||||
from app.services.person_service import (
|
||||
_attach_primary_name,
|
||||
_attach_primary_names,
|
||||
_redact,
|
||||
)
|
||||
from app.services.privacy import Visibility
|
||||
|
||||
|
||||
async def get_public_tree(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree_id: uuid.UUID
|
||||
) -> Tree:
|
||||
tree = (
|
||||
await session.execute(
|
||||
select(Tree).where(Tree.id == tree_id, Tree.deleted_at.is_(None))
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
# 404 (not 403) when not viewable: don't reveal that a private tree exists.
|
||||
if tree is None or not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise NotFound("tree not found")
|
||||
return tree
|
||||
|
||||
|
||||
async def _persons(session: AsyncSession, tree: Tree) -> list[Person]:
|
||||
return list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Person).where(Person.tree_id == tree.id, Person.deleted_at.is_(None))
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
async def _visibility_map(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, persons: list[Person]
|
||||
) -> dict[uuid.UUID, Visibility]:
|
||||
return {
|
||||
p.id: await privacy.person_visibility(
|
||||
session, user_id=viewer_id, tree=tree, person=p
|
||||
)
|
||||
for p in persons
|
||||
}
|
||||
|
||||
|
||||
async def list_public_persons(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
|
||||
) -> list[Person]:
|
||||
out: list[Person] = []
|
||||
full: list[Person] = []
|
||||
for p in await _persons(session, tree):
|
||||
vis = await privacy.person_visibility(session, user_id=viewer_id, tree=tree, person=p)
|
||||
if vis == Visibility.hidden:
|
||||
continue
|
||||
if vis == Visibility.redacted:
|
||||
_redact(p)
|
||||
else:
|
||||
full.append(p)
|
||||
out.append(p)
|
||||
await _attach_primary_names(session, full) # one query, not one per person
|
||||
return out
|
||||
|
||||
|
||||
async def get_public_person(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
|
||||
) -> Person:
|
||||
person = (
|
||||
await session.execute(
|
||||
select(Person).where(
|
||||
Person.id == person_id,
|
||||
Person.tree_id == tree.id,
|
||||
Person.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if person is None:
|
||||
raise NotFound("person not found")
|
||||
vis = await privacy.person_visibility(session, user_id=viewer_id, tree=tree, person=person)
|
||||
if vis == Visibility.hidden:
|
||||
raise NotFound("person not found")
|
||||
if vis == Visibility.redacted:
|
||||
_redact(person)
|
||||
else:
|
||||
await _attach_primary_name(session, person)
|
||||
return person
|
||||
|
||||
|
||||
async def _person_visibility(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
|
||||
) -> Visibility | None:
|
||||
person = (
|
||||
await session.execute(
|
||||
select(Person).where(
|
||||
Person.id == person_id,
|
||||
Person.tree_id == tree.id,
|
||||
Person.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if person is None:
|
||||
return None
|
||||
return await privacy.person_visibility(session, user_id=viewer_id, tree=tree, person=person)
|
||||
|
||||
|
||||
async def list_public_relationships(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
|
||||
) -> list[Relationship]:
|
||||
persons = await _persons(session, tree)
|
||||
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
|
||||
nonhidden = {pid for pid, v in vis.items() if v != Visibility.hidden}
|
||||
rels = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Relationship).where(
|
||||
Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None)
|
||||
)
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
return [
|
||||
r for r in rels if r.person_from_id in nonhidden and r.person_to_id in nonhidden
|
||||
]
|
||||
|
||||
|
||||
async def list_public_events(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
|
||||
) -> list[Event]:
|
||||
persons = await _persons(session, tree)
|
||||
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
|
||||
full = {pid for pid, v in vis.items() if v == Visibility.full}
|
||||
rels = {
|
||||
r.id: r
|
||||
for r in (
|
||||
await session.execute(
|
||||
select(Relationship).where(
|
||||
Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None)
|
||||
)
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
}
|
||||
events = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Event).where(Event.tree_id == tree.id, Event.deleted_at.is_(None))
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
out: list[Event] = []
|
||||
for e in events:
|
||||
if e.person_id is not None:
|
||||
if e.person_id in full:
|
||||
out.append(e)
|
||||
elif e.relationship_id is not None:
|
||||
r = rels.get(e.relationship_id)
|
||||
if r is not None and r.person_from_id in full and r.person_to_id in full:
|
||||
out.append(e)
|
||||
return out
|
||||
|
||||
|
||||
async def list_public_person_names(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
|
||||
) -> list[Name]:
|
||||
vis = await _person_visibility(session, viewer_id=viewer_id, tree=tree, person_id=person_id)
|
||||
if vis is None:
|
||||
raise NotFound("person not found")
|
||||
if vis != Visibility.full:
|
||||
return [] # redacted/hidden → no names (the real name must not leak)
|
||||
return list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Name)
|
||||
.where(Name.person_id == person_id, Name.deleted_at.is_(None))
|
||||
.order_by(Name.is_primary.desc(), Name.sort_order, Name.created_at)
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
async def list_public_person_events(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
|
||||
) -> list[Event]:
|
||||
vis = await _person_visibility(session, viewer_id=viewer_id, tree=tree, person_id=person_id)
|
||||
if vis is None:
|
||||
raise NotFound("person not found")
|
||||
if vis != Visibility.full:
|
||||
return [] # redacted/hidden → no dates
|
||||
return list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Event)
|
||||
.where(
|
||||
Event.person_id == person_id,
|
||||
Event.tree_id == tree.id,
|
||||
Event.deleted_at.is_(None),
|
||||
)
|
||||
.order_by(Event.date_start.nulls_last(), Event.created_at)
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
async def list_public_relationships_for_person(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, person_id: uuid.UUID
|
||||
) -> list[Relationship]:
|
||||
persons = await _persons(session, tree)
|
||||
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
|
||||
if vis.get(person_id) in (None, Visibility.hidden):
|
||||
return []
|
||||
nonhidden = {pid for pid, v in vis.items() if v != Visibility.hidden}
|
||||
rels = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Relationship).where(
|
||||
Relationship.tree_id == tree.id,
|
||||
Relationship.deleted_at.is_(None),
|
||||
or_(
|
||||
Relationship.person_from_id == person_id,
|
||||
Relationship.person_to_id == person_id,
|
||||
),
|
||||
)
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
return [r for r in rels if r.person_from_id in nonhidden and r.person_to_id in nonhidden]
|
||||
|
||||
|
||||
async def list_public_media(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
|
||||
) -> list[Media]:
|
||||
"""Only media linked to a FULL-visibility person. Media without a person (or
|
||||
linked only to an event/source) is not exposed to non-members — a photo of a
|
||||
living person must never leak."""
|
||||
persons = await _persons(session, tree)
|
||||
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
|
||||
full = {pid for pid, v in vis.items() if v == Visibility.full}
|
||||
media = list(
|
||||
(
|
||||
await session.execute(
|
||||
select(Media).where(Media.tree_id == tree.id, Media.deleted_at.is_(None))
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
return [m for m in media if m.person_id is not None and m.person_id in full]
|
||||
|
||||
|
||||
async def can_view_media(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, media: Media
|
||||
) -> bool:
|
||||
"""Whether a non-member may see/download a single media item: only when it is
|
||||
linked to a FULL-visibility person."""
|
||||
if media.person_id is None:
|
||||
return False
|
||||
vis = await _person_visibility(
|
||||
session, viewer_id=viewer_id, tree=tree, person_id=media.person_id
|
||||
)
|
||||
return vis == Visibility.full
|
||||
|
||||
|
||||
async def _full_person_ids(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
|
||||
) -> set[uuid.UUID]:
|
||||
persons = await _persons(session, tree)
|
||||
vis = await _visibility_map(session, viewer_id=viewer_id, tree=tree, persons=persons)
|
||||
return {pid for pid, v in vis.items() if v == Visibility.full}
|
||||
|
||||
|
||||
async def list_public_citations(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
|
||||
) -> list[Citation]:
|
||||
"""Only citations whose cited fact resolves to FULL-visibility person(s). A
|
||||
citation on a redacted/hidden person's fact (or a partnership where either
|
||||
partner isn't full) is dropped — its existence plus page/detail would leak
|
||||
that the person has that sourced fact. Mirrors the events/names rule (FULL
|
||||
only)."""
|
||||
full = await _full_person_ids(session, viewer_id=viewer_id, tree=tree)
|
||||
|
||||
async def _by_id(model):
|
||||
rows = (
|
||||
await session.execute(
|
||||
select(model).where(model.tree_id == tree.id, model.deleted_at.is_(None))
|
||||
)
|
||||
).scalars().all()
|
||||
return {r.id: r for r in rows}
|
||||
|
||||
names = await _by_id(Name)
|
||||
rels = await _by_id(Relationship)
|
||||
events = await _by_id(Event)
|
||||
|
||||
def target_is_full(c: Citation) -> bool:
|
||||
if c.person_id is not None:
|
||||
return c.person_id in full
|
||||
if c.name_id is not None:
|
||||
n = names.get(c.name_id)
|
||||
return n is not None and n.person_id in full
|
||||
if c.event_id is not None:
|
||||
e = events.get(c.event_id)
|
||||
if e is None:
|
||||
return False
|
||||
if e.person_id is not None:
|
||||
return e.person_id in full
|
||||
if e.relationship_id is not None:
|
||||
r = rels.get(e.relationship_id)
|
||||
return r is not None and r.person_from_id in full and r.person_to_id in full
|
||||
return False
|
||||
if c.relationship_id is not None:
|
||||
r = rels.get(c.relationship_id)
|
||||
return r is not None and r.person_from_id in full and r.person_to_id in full
|
||||
return False
|
||||
|
||||
citations = (
|
||||
await session.execute(
|
||||
select(Citation)
|
||||
.where(Citation.tree_id == tree.id, Citation.deleted_at.is_(None))
|
||||
.order_by(Citation.created_at)
|
||||
)
|
||||
).scalars().all()
|
||||
return [c for c in citations if target_is_full(c)]
|
||||
|
||||
|
||||
async def list_public_sources(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree
|
||||
) -> list[Source]:
|
||||
"""Only sources backing at least one visible citation. A source used solely
|
||||
for a redacted/hidden person's facts is withheld — its title or notes could
|
||||
name that living person."""
|
||||
visible = await list_public_citations(session, viewer_id=viewer_id, tree=tree)
|
||||
cited = {c.source_id for c in visible}
|
||||
sources = (
|
||||
await session.execute(
|
||||
select(Source)
|
||||
.where(Source.tree_id == tree.id, Source.deleted_at.is_(None))
|
||||
.order_by(Source.title)
|
||||
)
|
||||
).scalars().all()
|
||||
return [s for s in sources if s.id in cited]
|
||||
|
||||
|
||||
async def get_public_source(
|
||||
session: AsyncSession, *, viewer_id: uuid.UUID | None, tree: Tree, source_id: uuid.UUID
|
||||
) -> Source:
|
||||
for s in await list_public_sources(session, viewer_id=viewer_id, tree=tree):
|
||||
if s.id == source_id:
|
||||
return s
|
||||
# 404 (not 403): don't reveal that a withheld source exists.
|
||||
raise NotFound("source not found")
|
||||
|
||||
|
||||
async def list_public_trees(
|
||||
session: AsyncSession,
|
||||
*,
|
||||
viewer_id: uuid.UUID | None,
|
||||
q: str | None = None,
|
||||
limit: int = 50,
|
||||
offset: int = 0,
|
||||
) -> list[Tree]:
|
||||
# Anonymous: only `public`. Authenticated: also `site_members`. Never list
|
||||
# `unlisted` (reachable by link only) or `private`.
|
||||
allowed = [TreeVisibility.public]
|
||||
if viewer_id is not None:
|
||||
allowed.append(TreeVisibility.site_members)
|
||||
stmt = select(Tree).where(
|
||||
Tree.deleted_at.is_(None), Tree.visibility.in_(allowed)
|
||||
)
|
||||
if q and q.strip():
|
||||
stmt = stmt.where(Tree.name.ilike(f"%{q.strip()}%"))
|
||||
stmt = stmt.order_by(Tree.name).limit(min(limit, 100)).offset(max(offset, 0))
|
||||
return list((await session.execute(stmt)).scalars().all())
|
||||
@@ -4,7 +4,7 @@ Writes require editor rights; reads go through the privacy engine."""
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import or_, select
|
||||
from sqlalchemy import and_, or_, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.enums import ParentChildQualifier, RelationshipType
|
||||
@@ -49,6 +49,38 @@ async def create_relationship(
|
||||
if not await _person_in_tree(session, pid, tree.id):
|
||||
raise NotFound("person not found in this tree")
|
||||
|
||||
# Reject an equivalent existing edge so the same two people can't be linked
|
||||
# the same way twice. parent_child is directional (parent -> child);
|
||||
# partnership/sibling are symmetric, so match the pair in either order.
|
||||
if type is RelationshipType.parent_child:
|
||||
pair = and_(
|
||||
Relationship.person_from_id == person_from_id,
|
||||
Relationship.person_to_id == person_to_id,
|
||||
)
|
||||
else:
|
||||
pair = or_(
|
||||
and_(
|
||||
Relationship.person_from_id == person_from_id,
|
||||
Relationship.person_to_id == person_to_id,
|
||||
),
|
||||
and_(
|
||||
Relationship.person_from_id == person_to_id,
|
||||
Relationship.person_to_id == person_from_id,
|
||||
),
|
||||
)
|
||||
existing = (
|
||||
await session.execute(
|
||||
select(Relationship.id).where(
|
||||
Relationship.tree_id == tree.id,
|
||||
Relationship.type == type,
|
||||
Relationship.deleted_at.is_(None),
|
||||
pair,
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if existing is not None:
|
||||
raise Conflict("these two people are already linked that way")
|
||||
|
||||
relationship = Relationship(
|
||||
tree_id=tree.id,
|
||||
type=type,
|
||||
@@ -79,6 +111,13 @@ async def list_relationships(
|
||||
"""All relationships in the tree — powers the family/pedigree view in one call."""
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
# Non-members: drop relationships touching a hidden person.
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
return await public_view_service.list_public_relationships(
|
||||
session, viewer_id=viewer_id, tree=tree
|
||||
)
|
||||
stmt = (
|
||||
select(Relationship)
|
||||
.where(Relationship.tree_id == tree.id, Relationship.deleted_at.is_(None))
|
||||
@@ -92,6 +131,12 @@ async def list_relationships_for_person(
|
||||
) -> list[Relationship]:
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
return await public_view_service.list_public_relationships_for_person(
|
||||
session, viewer_id=viewer_id, tree=tree, person_id=person_id
|
||||
)
|
||||
stmt = (
|
||||
select(Relationship)
|
||||
.where(
|
||||
|
||||
@@ -61,6 +61,14 @@ async def create_source(
|
||||
async def list_sources(session: AsyncSession, *, viewer_id: uuid.UUID, tree: Tree) -> list[Source]:
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
# Non-members see only sources backing a visible citation (see citation
|
||||
# redaction) — a source used solely for a redacted person could name them.
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
return await public_view_service.list_public_sources(
|
||||
session, viewer_id=viewer_id, tree=tree
|
||||
)
|
||||
stmt = (
|
||||
select(Source)
|
||||
.where(Source.tree_id == tree.id, Source.deleted_at.is_(None))
|
||||
@@ -74,6 +82,12 @@ async def get_source(
|
||||
) -> Source:
|
||||
if not await privacy.can_view_tree(session, user_id=viewer_id, tree=tree):
|
||||
raise Forbidden("not permitted to view this tree")
|
||||
if await privacy.get_membership_role(session, viewer_id, tree.id) is None:
|
||||
from app.services import public_view_service
|
||||
|
||||
return await public_view_service.get_public_source(
|
||||
session, viewer_id=viewer_id, tree=tree, source_id=source_id
|
||||
)
|
||||
source = (
|
||||
await session.execute(
|
||||
select(Source).where(
|
||||
|
||||
@@ -5,16 +5,18 @@ authorization basis) and an audit entry. Reads go through the privacy engine.
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.integrations.objectstore.base import ObjectStore
|
||||
from app.models.enums import MembershipRole, TreeVisibility
|
||||
from app.models.media import Media
|
||||
from app.models.tree import Tree, TreeMembership
|
||||
from app.models.user import User
|
||||
from app.repositories.base import BaseRepository
|
||||
from app.services import privacy
|
||||
from app.services.audit import record_audit
|
||||
from app.services.exceptions import Forbidden, NotFound
|
||||
from app.services.exceptions import Conflict, Forbidden, NotFound
|
||||
|
||||
|
||||
async def create_tree(
|
||||
@@ -70,7 +72,7 @@ async def update_tree(
|
||||
raise NotFound("tree not found")
|
||||
if not await privacy.can_edit_tree(session, user_id=actor.id, tree=tree):
|
||||
raise Forbidden("not an editor of this tree")
|
||||
for key in {"name", "description", "visibility"} & changes.keys():
|
||||
for key in {"name", "description", "visibility", "home_person_id"} & changes.keys():
|
||||
setattr(tree, key, changes[key])
|
||||
record_audit(
|
||||
session,
|
||||
@@ -128,6 +130,50 @@ async def restore_tree(session: AsyncSession, *, actor: User, tree_id: uuid.UUID
|
||||
return tree
|
||||
|
||||
|
||||
async def purge_tree(
|
||||
session: AsyncSession,
|
||||
store: ObjectStore,
|
||||
*,
|
||||
actor: User,
|
||||
tree_id: uuid.UUID,
|
||||
confirm_name: str,
|
||||
) -> None:
|
||||
"""Permanently delete a soft-deleted tree and ALL its data — irreversible.
|
||||
Owner-only. The tree must already be in the trash (soft-deleted) and the
|
||||
caller must retype its name. Tree-owned rows are removed by the `tree_id`
|
||||
ON DELETE CASCADE; we delete the media objects from storage first (the DB
|
||||
cascade drops the rows but not the bytes). Audit entries survive with their
|
||||
`tree_id` nulled (ON DELETE SET NULL), so the purge stays in the log."""
|
||||
tree = await _owned_tree(session, actor=actor, tree_id=tree_id)
|
||||
if tree.deleted_at is None:
|
||||
raise Conflict("delete the tree first, then purge it from the trash")
|
||||
if confirm_name.strip() != (tree.name or "").strip():
|
||||
raise Forbidden("tree name confirmation does not match")
|
||||
|
||||
keys = list(
|
||||
(
|
||||
await session.execute(select(Media.storage_key).where(Media.tree_id == tree.id))
|
||||
).scalars().all()
|
||||
)
|
||||
for key in keys:
|
||||
try:
|
||||
await store.delete_object(key=key)
|
||||
except Exception: # noqa: BLE001 — best-effort; a missing object must not block the purge
|
||||
pass
|
||||
|
||||
record_audit(
|
||||
session,
|
||||
action="purge",
|
||||
entity_type="Tree",
|
||||
entity_id=tree.id,
|
||||
tree_id=tree.id,
|
||||
actor_user_id=actor.id,
|
||||
before={"name": tree.name},
|
||||
)
|
||||
await session.execute(delete(Tree).where(Tree.id == tree.id))
|
||||
await session.commit()
|
||||
|
||||
|
||||
async def list_deleted_trees_for_user(session: AsyncSession, *, user: User) -> list[Tree]:
|
||||
stmt = (
|
||||
select(Tree)
|
||||
|
||||
@@ -8,10 +8,13 @@ import uuid
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.person import Person
|
||||
from app.models.tree import Tree
|
||||
from app.models.user import User
|
||||
from app.repositories.base import BaseRepository
|
||||
from app.services import privacy
|
||||
from app.services.audit import record_audit
|
||||
from app.services.exceptions import Conflict
|
||||
from app.services.exceptions import Conflict, Forbidden, NotFound
|
||||
|
||||
|
||||
async def create_user(
|
||||
@@ -42,3 +45,39 @@ async def create_user(
|
||||
|
||||
async def get_user(session: AsyncSession, user_id: uuid.UUID) -> User | None:
|
||||
return await BaseRepository(session, User).get(user_id)
|
||||
|
||||
|
||||
async def set_self_person(
|
||||
session: AsyncSession, *, user: User, person_id: uuid.UUID | None
|
||||
) -> User:
|
||||
"""Point a user's account at the Person record that *is* them ("home
|
||||
person"), or clear it with ``None``. The person must live in a tree the
|
||||
user can view."""
|
||||
if person_id is not None:
|
||||
person = (
|
||||
await session.execute(
|
||||
select(Person).where(Person.id == person_id, Person.deleted_at.is_(None))
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if person is None:
|
||||
raise NotFound("person not found")
|
||||
tree = (
|
||||
await session.execute(select(Tree).where(Tree.id == person.tree_id))
|
||||
).scalar_one_or_none()
|
||||
if tree is None or not await privacy.can_view_tree(
|
||||
session, user_id=user.id, tree=tree
|
||||
):
|
||||
raise Forbidden("not permitted to link this person")
|
||||
|
||||
user.self_person_id = person_id
|
||||
record_audit(
|
||||
session,
|
||||
action="update",
|
||||
entity_type="User",
|
||||
entity_id=user.id,
|
||||
actor_user_id=user.id,
|
||||
after={"self_person_id": str(person_id) if person_id else None},
|
||||
)
|
||||
await session.commit()
|
||||
await session.refresh(user)
|
||||
return user
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
#!/bin/sh
|
||||
# Container entrypoint. When RUN_MIGRATIONS=1 (set on the backend service),
|
||||
# apply DB migrations before handing off to the command. This makes a deploy
|
||||
# self-migrating even when images are swapped in place (e.g. by Watchtower),
|
||||
# without a separate orchestration step. `alembic upgrade head` is idempotent —
|
||||
# a no-op when the schema is already current.
|
||||
set -e
|
||||
|
||||
if [ "${RUN_MIGRATIONS:-0}" = "1" ]; then
|
||||
echo "[entrypoint] applying database migrations (alembic upgrade head)…"
|
||||
uv run --no-dev alembic upgrade head
|
||||
fi
|
||||
|
||||
exec "$@"
|
||||
@@ -0,0 +1,62 @@
|
||||
"""change_proposals (AI propose-then-confirm)
|
||||
|
||||
Revision ID: a1b2c3d4e5f6
|
||||
Revises: d4a9c1e7b2f3
|
||||
Create Date: 2026-06-09
|
||||
|
||||
"""
|
||||
from collections.abc import Sequence
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
revision: str = "a1b2c3d4e5f6"
|
||||
down_revision: str | None = "d4a9c1e7b2f3"
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"change_proposals",
|
||||
sa.Column("id", sa.Uuid(), nullable=False),
|
||||
sa.Column("tree_id", sa.Uuid(), nullable=False),
|
||||
sa.Column(
|
||||
"status",
|
||||
sa.Enum("pending", "applied", "rejected", name="change_proposal_status"),
|
||||
server_default="pending",
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column(
|
||||
"origin",
|
||||
sa.Enum("assistant", "contributor", name="change_proposal_origin"),
|
||||
server_default="assistant",
|
||||
nullable=False,
|
||||
),
|
||||
sa.Column("created_by_user_id", sa.Uuid(), nullable=True),
|
||||
sa.Column("summary", sa.String(length=512), nullable=False),
|
||||
sa.Column("rationale", sa.Text(), nullable=True),
|
||||
sa.Column("operations", postgresql.JSONB(astext_type=sa.Text()), nullable=False),
|
||||
sa.Column("reviewed_by_user_id", sa.Uuid(), nullable=True),
|
||||
sa.Column("reviewed_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("review_note", sa.String(length=512), nullable=True),
|
||||
sa.Column("apply_error", sa.Text(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.text("now()"), nullable=False),
|
||||
sa.Column("deleted_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.ForeignKeyConstraint(["tree_id"], ["trees.id"], ondelete="CASCADE"),
|
||||
sa.ForeignKeyConstraint(["created_by_user_id"], ["users.id"], ondelete="SET NULL"),
|
||||
sa.ForeignKeyConstraint(["reviewed_by_user_id"], ["users.id"], ondelete="SET NULL"),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
op.create_index("ix_change_proposals_tree_id", "change_proposals", ["tree_id"])
|
||||
op.create_index("ix_change_proposals_status", "change_proposals", ["status"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_change_proposals_status", table_name="change_proposals")
|
||||
op.drop_index("ix_change_proposals_tree_id", table_name="change_proposals")
|
||||
op.drop_table("change_proposals")
|
||||
sa.Enum(name="change_proposal_status").drop(op.get_bind())
|
||||
sa.Enum(name="change_proposal_origin").drop(op.get_bind())
|
||||
@@ -0,0 +1,26 @@
|
||||
"""tree AI model policy (ai_member_provider, ai_recommender_provider)
|
||||
|
||||
Revision ID: b2c3d4e5f6a7
|
||||
Revises: a1b2c3d4e5f6
|
||||
Create Date: 2026-06-09
|
||||
|
||||
"""
|
||||
from collections.abc import Sequence
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "b2c3d4e5f6a7"
|
||||
down_revision: str | None = "a1b2c3d4e5f6"
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("trees", sa.Column("ai_member_provider", sa.String(length=32), nullable=True))
|
||||
op.add_column("trees", sa.Column("ai_recommender_provider", sa.String(length=32), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("trees", "ai_recommender_provider")
|
||||
op.drop_column("trees", "ai_member_provider")
|
||||
@@ -0,0 +1,36 @@
|
||||
"""user.self_person_id ("home person" link)
|
||||
|
||||
Revision ID: b3d5f8a1c920
|
||||
Revises: 9a2b1c7d4e10
|
||||
Create Date: 2026-06-07
|
||||
|
||||
"""
|
||||
from collections.abc import Sequence
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "b3d5f8a1c920"
|
||||
down_revision: str | None = "9a2b1c7d4e10"
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"users",
|
||||
sa.Column("self_person_id", sa.Uuid(), nullable=True),
|
||||
)
|
||||
op.create_foreign_key(
|
||||
"fk_users_self_person_id",
|
||||
"users",
|
||||
"persons",
|
||||
["self_person_id"],
|
||||
["id"],
|
||||
ondelete="SET NULL",
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_constraint("fk_users_self_person_id", "users", type_="foreignkey")
|
||||
op.drop_column("users", "self_person_id")
|
||||
@@ -0,0 +1,33 @@
|
||||
"""tree.home_person_id (per-tree default/home person)
|
||||
|
||||
Revision ID: c7e1a4f2d3b8
|
||||
Revises: b3d5f8a1c920
|
||||
Create Date: 2026-06-07
|
||||
|
||||
"""
|
||||
from collections.abc import Sequence
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "c7e1a4f2d3b8"
|
||||
down_revision: str | None = "b3d5f8a1c920"
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("trees", sa.Column("home_person_id", sa.Uuid(), nullable=True))
|
||||
op.create_foreign_key(
|
||||
"fk_trees_home_person_id",
|
||||
"trees",
|
||||
"persons",
|
||||
["home_person_id"],
|
||||
["id"],
|
||||
ondelete="SET NULL",
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_constraint("fk_trees_home_person_id", "trees", type_="foreignkey")
|
||||
op.drop_column("trees", "home_person_id")
|
||||
@@ -0,0 +1,28 @@
|
||||
"""tree_visibility: add 'site_members' value
|
||||
|
||||
Revision ID: d4a9c1e7b2f3
|
||||
Revises: c7e1a4f2d3b8
|
||||
Create Date: 2026-06-09
|
||||
|
||||
"""
|
||||
from collections.abc import Sequence
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision: str = "d4a9c1e7b2f3"
|
||||
down_revision: str | None = "c7e1a4f2d3b8"
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ALTER TYPE ... ADD VALUE cannot run inside a transaction block on older
|
||||
# Postgres; run it in an autocommit block so it applies regardless of version.
|
||||
with op.get_context().autocommit_block():
|
||||
op.execute("ALTER TYPE tree_visibility ADD VALUE IF NOT EXISTS 'site_members'")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Postgres cannot drop an enum value without rebuilding the type; treat the
|
||||
# added value as irreversible. (Rows using it would block a rebuild anyway.)
|
||||
pass
|
||||
@@ -14,6 +14,8 @@ dependencies = [
|
||||
"argon2-cffi>=23.1",
|
||||
"boto3>=1.35",
|
||||
"python-multipart>=0.0.12",
|
||||
"anthropic>=0.108.0",
|
||||
"openai>=2.41.0",
|
||||
]
|
||||
|
||||
[dependency-groups]
|
||||
|
||||
@@ -67,16 +67,21 @@ def mailer() -> CapturingMailer:
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def client():
|
||||
async def engine():
|
||||
if not TEST_DATABASE_URL:
|
||||
pytest.skip("TEST_DATABASE_URL not set")
|
||||
|
||||
engine = create_async_engine(TEST_DATABASE_URL)
|
||||
async with engine.begin() as conn:
|
||||
eng = create_async_engine(TEST_DATABASE_URL)
|
||||
async with eng.begin() as conn:
|
||||
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS pg_trgm"))
|
||||
await conn.run_sync(Base.metadata.drop_all)
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
yield eng
|
||||
await eng.dispose()
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def client(engine):
|
||||
sessionmaker = async_sessionmaker(engine, expire_on_commit=False, class_=AsyncSession)
|
||||
|
||||
async def _override_session():
|
||||
@@ -95,7 +100,14 @@ async def client():
|
||||
yield http_client
|
||||
|
||||
app.dependency_overrides.clear()
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def db_session(engine):
|
||||
"""A raw AsyncSession on the test DB, for unit-testing services directly."""
|
||||
sessionmaker = async_sessionmaker(engine, expire_on_commit=False, class_=AsyncSession)
|
||||
async with sessionmaker() as session:
|
||||
yield session
|
||||
|
||||
|
||||
def token_from_link(link: str) -> str:
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Account export -> restore round-trip, and account deletion."""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def _seed(client, h):
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "Fam"}, headers=h)).json()["id"]
|
||||
p1 = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons", json={"given": "Ada", "surname": "Lovelace"}, headers=h
|
||||
)
|
||||
).json()["id"]
|
||||
p2 = (
|
||||
await client.post(f"/api/v1/trees/{tid}/persons", json={"given": "Kid"}, headers=h)
|
||||
).json()["id"]
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "parent_child", "person_from_id": p1, "person_to_id": p2},
|
||||
headers=h,
|
||||
)
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "birth", "person_id": p1, "date_value": "1815"},
|
||||
headers=h,
|
||||
)
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/media",
|
||||
files={"file": ("scan.txt", b"hello", "text/plain")},
|
||||
data={"title": "Scan", "person_id": p1},
|
||||
headers=h,
|
||||
)
|
||||
return tid
|
||||
|
||||
|
||||
async def test_export_then_restore_roundtrip(client):
|
||||
h = auth(await register(client, "exp@example.com"))
|
||||
await _seed(client, h)
|
||||
|
||||
export = await client.get("/api/v1/users/me/export", headers=h)
|
||||
assert export.status_code == 200
|
||||
assert export.headers["content-type"] == "application/zip"
|
||||
blob = export.content
|
||||
assert blob[:2] == b"PK" # zip magic
|
||||
|
||||
# Restore into new trees (non-destructive: the original stays).
|
||||
r = await client.post(
|
||||
"/api/v1/users/me/import",
|
||||
files={"file": ("provenance-export.zip", blob, "application/zip")},
|
||||
headers=h,
|
||||
)
|
||||
assert r.status_code == 200, r.text
|
||||
counts = r.json()
|
||||
assert counts["trees"] == 1 and counts["persons"] == 2
|
||||
assert counts["events"] == 1 and counts["media"] == 1
|
||||
|
||||
trees = (await client.get("/api/v1/trees", headers=h)).json()
|
||||
assert len(trees) == 2 # original + restored
|
||||
|
||||
# The restored tree has the people, with a working relationship and media.
|
||||
restored = [t for t in trees if t["name"] == "Fam"][1]["id"]
|
||||
ppl = (await client.get(f"/api/v1/trees/{restored}/persons", headers=h)).json()
|
||||
assert {p["primary_name"] for p in ppl} == {"Ada Lovelace", "Kid"}
|
||||
rels = (await client.get(f"/api/v1/trees/{restored}/relationships", headers=h)).json()
|
||||
assert len(rels) == 1
|
||||
med = (await client.get(f"/api/v1/trees/{restored}/media", headers=h)).json()
|
||||
assert len(med) == 1 and med[0]["title"] == "Scan"
|
||||
|
||||
|
||||
async def test_delete_account_requires_email_then_revokes(client):
|
||||
token = await register(client, "del@example.com")
|
||||
h = auth(token)
|
||||
await _seed(client, h)
|
||||
|
||||
# Wrong email is rejected.
|
||||
bad = await client.request(
|
||||
"DELETE", "/api/v1/users/me", data={"confirm_email": "nope@example.com"}, headers=h
|
||||
)
|
||||
assert bad.status_code == 403
|
||||
|
||||
ok = await client.request(
|
||||
"DELETE", "/api/v1/users/me", data={"confirm_email": "del@example.com"}, headers=h
|
||||
)
|
||||
assert ok.status_code == 204
|
||||
|
||||
# Session is revoked — the token no longer works.
|
||||
assert (await client.get("/api/v1/users/me", headers=h)).status_code == 401
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Change password and per-tree home person."""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def test_change_password(client):
|
||||
token = await register(client, "cp@example.com", password="password123")
|
||||
h = auth(token)
|
||||
|
||||
# Wrong current password is rejected.
|
||||
bad = await client.post(
|
||||
"/api/v1/auth/change-password",
|
||||
json={"current_password": "nope", "new_password": "newpass123"},
|
||||
headers=h,
|
||||
)
|
||||
assert bad.status_code == 403
|
||||
|
||||
ok = await client.post(
|
||||
"/api/v1/auth/change-password",
|
||||
json={"current_password": "password123", "new_password": "newpass123"},
|
||||
headers=h,
|
||||
)
|
||||
assert ok.status_code == 204
|
||||
|
||||
# The new password logs in; the old one does not.
|
||||
assert (
|
||||
await client.post(
|
||||
"/api/v1/auth/login", json={"email": "cp@example.com", "password": "newpass123"}
|
||||
)
|
||||
).status_code == 200
|
||||
assert (
|
||||
await client.post(
|
||||
"/api/v1/auth/login", json={"email": "cp@example.com", "password": "password123"}
|
||||
)
|
||||
).status_code == 401
|
||||
|
||||
|
||||
async def test_tree_home_person(client):
|
||||
h = auth(await register(client, "home@example.com"))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
|
||||
pid = (
|
||||
await client.post(f"/api/v1/trees/{tid}/persons", json={"given": "Root"}, headers=h)
|
||||
).json()["id"]
|
||||
|
||||
r = await client.patch(
|
||||
f"/api/v1/trees/{tid}", json={"home_person_id": pid}, headers=h
|
||||
)
|
||||
assert r.status_code == 200 and r.json()["home_person_id"] == pid
|
||||
|
||||
# Deleting the home person clears the link.
|
||||
await client.delete(f"/api/v1/trees/{tid}/persons/{pid}", headers=h)
|
||||
tree = (await client.get(f"/api/v1/trees/{tid}", headers=h)).json()
|
||||
assert tree["home_person_id"] is None
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Per-tree AI model policy: owner-only, validated against configured providers."""
|
||||
|
||||
from app.core.config import get_settings
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def test_ai_policy_is_owner_only(client):
|
||||
owner = auth(await register(client, "ai-o@ex.com"))
|
||||
editor = auth(await register(client, "ai-x@ex.com"))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=owner)).json()["id"]
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/members", json={"email": "ai-x@ex.com", "role": "editor"}, headers=owner
|
||||
)
|
||||
|
||||
g = await client.get(f"/api/v1/trees/{tid}/ai", headers=owner)
|
||||
assert g.status_code == 200
|
||||
assert g.json()["member_provider"] is None
|
||||
assert g.json()["configured_providers"] == [] # nothing configured in tests
|
||||
|
||||
# An editor (not owner) can neither view nor change the policy.
|
||||
assert (await client.get(f"/api/v1/trees/{tid}/ai", headers=editor)).status_code == 403
|
||||
assert (
|
||||
await client.patch(
|
||||
f"/api/v1/trees/{tid}/ai",
|
||||
json={"member_provider": None, "recommender_provider": None},
|
||||
headers=editor,
|
||||
)
|
||||
).status_code == 403
|
||||
|
||||
|
||||
async def test_ai_policy_set_and_validate(client, monkeypatch):
|
||||
monkeypatch.setattr(get_settings(), "anthropic_api_key", "sk-ant-test")
|
||||
owner = auth(await register(client, "ai-set@ex.com"))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=owner)).json()["id"]
|
||||
|
||||
g = (await client.get(f"/api/v1/trees/{tid}/ai", headers=owner)).json()
|
||||
assert {p["name"] for p in g["configured_providers"]} == {"anthropic"}
|
||||
|
||||
# Assign the member + recommender model.
|
||||
p = await client.patch(
|
||||
f"/api/v1/trees/{tid}/ai",
|
||||
json={"member_provider": "anthropic", "recommender_provider": "anthropic"},
|
||||
headers=owner,
|
||||
)
|
||||
assert p.status_code == 200 and p.json()["member_provider"] == "anthropic"
|
||||
|
||||
# A provider that isn't configured is rejected.
|
||||
assert (
|
||||
await client.patch(
|
||||
f"/api/v1/trees/{tid}/ai",
|
||||
json={"member_provider": "openai", "recommender_provider": None},
|
||||
headers=owner,
|
||||
)
|
||||
).status_code == 403
|
||||
|
||||
# Clearing is allowed.
|
||||
c = await client.patch(
|
||||
f"/api/v1/trees/{tid}/ai",
|
||||
json={"member_provider": None, "recommender_provider": None},
|
||||
headers=owner,
|
||||
)
|
||||
assert c.status_code == 200 and c.json()["member_provider"] is None
|
||||
@@ -104,3 +104,44 @@ async def test_logout_revokes_session(client):
|
||||
assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 200
|
||||
assert (await client.post("/api/v1/auth/logout", headers=auth(token))).status_code == 204
|
||||
assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 401
|
||||
|
||||
|
||||
async def test_unverified_user_works_by_default(client):
|
||||
# Default (require_email_verification off): unverified accounts work as before.
|
||||
token = await register(client, "open@example.com")
|
||||
assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 200
|
||||
|
||||
|
||||
async def test_verification_gate_blocks_until_verified(client, mailer, monkeypatch):
|
||||
from app.core.config import get_settings
|
||||
|
||||
monkeypatch.setattr(get_settings(), "require_email_verification", True)
|
||||
|
||||
reg = await client.post(
|
||||
"/api/v1/auth/register", json={"email": "gate@example.com", "password": "password123"}
|
||||
)
|
||||
assert reg.status_code == 201
|
||||
token = reg.json()["token"]
|
||||
|
||||
# The session issued at registration does not resolve while unverified...
|
||||
assert (await client.get("/api/v1/users/me", headers=auth(token))).status_code == 401
|
||||
# ...and login is refused with 403 (not 401 — credentials are valid).
|
||||
blocked = await client.post(
|
||||
"/api/v1/auth/login", json={"email": "gate@example.com", "password": "password123"}
|
||||
)
|
||||
assert blocked.status_code == 403
|
||||
|
||||
# Verify via the emailed link.
|
||||
link = mailer.verifications[-1][1]
|
||||
assert (
|
||||
await client.post("/api/v1/auth/verify-email", json={"token": token_from_link(link)})
|
||||
).status_code == 204
|
||||
|
||||
# Now login works and the session resolves.
|
||||
ok = await client.post(
|
||||
"/api/v1/auth/login", json={"email": "gate@example.com", "password": "password123"}
|
||||
)
|
||||
assert ok.status_code == 200
|
||||
assert (
|
||||
await client.get("/api/v1/users/me", headers=auth(ok.json()["token"]))
|
||||
).status_code == 200
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
"""Authed non-member reads must redact PER-PERSON, not just gate on the tree.
|
||||
|
||||
A logged-in user who is NOT a member of a public tree previously saw living
|
||||
people's dates, real alternate names, and media through the family-view
|
||||
endpoints — only the person *list* was redacted. These tests assert that leak is
|
||||
closed while members still see everything.
|
||||
"""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
LSURNAME = "Authleaksurname"
|
||||
LALIAS = "Authleakalias"
|
||||
LYEAR = "2003"
|
||||
|
||||
|
||||
async def _setup(client):
|
||||
owner = auth(await register(client, "anm-owner@ex.com"))
|
||||
tid = (
|
||||
await client.post(
|
||||
"/api/v1/trees", json={"name": "Pub", "visibility": "public"}, headers=owner
|
||||
)
|
||||
).json()["id"]
|
||||
old = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": "Olde", "surname": "Gone", "is_living": False},
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
young = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": "Youngauth", "surname": LSURNAME, "is_living": True},
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
for pid, year in ((old, "1855"), (young, LYEAR)):
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "birth", "person_id": pid, "date_value": year},
|
||||
headers=owner,
|
||||
)
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons/{young}/names",
|
||||
json={"name_type": "alias", "given": LALIAS},
|
||||
headers=owner,
|
||||
)
|
||||
om = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/media",
|
||||
files={"file": ("o.txt", b"old-photo", "text/plain")},
|
||||
data={"person_id": old},
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
ym = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/media",
|
||||
files={"file": ("y.txt", b"young-photo", "text/plain")},
|
||||
data={"person_id": young},
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
return owner, tid, old, young, om, ym
|
||||
|
||||
|
||||
async def test_authed_nonmember_does_not_see_living_pii(client):
|
||||
owner, tid, old, young, om, ym = await _setup(client)
|
||||
stranger = auth(await register(client, "anm-stranger@ex.com"))
|
||||
|
||||
# Living person's events dropped; deceased kept.
|
||||
events = (await client.get(f"/api/v1/trees/{tid}/events", headers=stranger)).json()
|
||||
assert any(e["person_id"] == old for e in events)
|
||||
assert not any(e["person_id"] == young for e in events)
|
||||
|
||||
# Per-person living: names + events empty.
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons/{young}/names", headers=stranger)
|
||||
).json() == []
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons/{young}/events", headers=stranger)
|
||||
).json() == []
|
||||
|
||||
# The living surname/alias/birth-year must not appear in any of these.
|
||||
for path in (
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
f"/api/v1/trees/{tid}/persons/{young}/names",
|
||||
f"/api/v1/trees/{tid}/media",
|
||||
):
|
||||
body = (await client.get(path, headers=stranger)).text
|
||||
assert LSURNAME not in body, path
|
||||
assert LALIAS not in body, path
|
||||
assert LYEAR not in body, path
|
||||
|
||||
# Media: living person's media hidden from the list and undownloadable;
|
||||
# deceased person's media is fine.
|
||||
media_ids = {m["id"] for m in (await client.get(f"/api/v1/trees/{tid}/media", headers=stranger)).json()}
|
||||
assert om in media_ids
|
||||
assert ym not in media_ids
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/media/{ym}/content", headers=stranger)
|
||||
).status_code == 404
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/media/{om}/content", headers=stranger)
|
||||
).status_code == 200
|
||||
|
||||
|
||||
async def _setup_sources(client):
|
||||
owner = auth(await register(client, "anmcs-owner@ex.com"))
|
||||
tid = (
|
||||
await client.post(
|
||||
"/api/v1/trees", json={"name": "PubCS", "visibility": "public"}, headers=owner
|
||||
)
|
||||
).json()["id"]
|
||||
old = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": "Oldcs", "surname": "Gonecs", "is_living": False},
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
young = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": "Youngcs", "surname": "Csleaksurname", "is_living": True},
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
for pid, year in ((old, "1851"), (young, "2004")):
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "birth", "person_id": pid, "date_value": year},
|
||||
headers=owner,
|
||||
)
|
||||
s_old = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/sources", json={"title": "Oldsource record"}, headers=owner
|
||||
)
|
||||
).json()["id"]
|
||||
s_young = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/sources",
|
||||
json={"title": "Youngsource Csleaktitle"}, # title names the living person
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/citations",
|
||||
json={"source_id": s_old, "person_id": old, "page": "p.1"},
|
||||
headers=owner,
|
||||
)
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/citations",
|
||||
json={"source_id": s_young, "person_id": young, "page": "p.2"},
|
||||
headers=owner,
|
||||
)
|
||||
return owner, tid, old, young, s_old, s_young
|
||||
|
||||
|
||||
async def test_authed_nonmember_citation_source_redaction(client):
|
||||
"""A non-member must not see citations on a redacted living person's facts,
|
||||
nor sources used only for them."""
|
||||
owner, tid, old, young, s_old, s_young = await _setup_sources(client)
|
||||
stranger = auth(await register(client, "anmcs-stranger@ex.com"))
|
||||
|
||||
cites = (await client.get(f"/api/v1/trees/{tid}/citations", headers=stranger)).json()
|
||||
cited = {c.get("person_id") for c in cites}
|
||||
assert old in cited
|
||||
assert young not in cited # living person's citation dropped
|
||||
|
||||
srcs = (await client.get(f"/api/v1/trees/{tid}/sources", headers=stranger))
|
||||
src_ids = {s["id"] for s in srcs.json()}
|
||||
assert s_old in src_ids
|
||||
assert s_young not in src_ids # source used only for the living person withheld
|
||||
assert "Csleaktitle" not in srcs.text # its title (which names them) must not leak
|
||||
|
||||
# The withheld source 404s — don't reveal it exists; the visible one is fine.
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/sources/{s_young}", headers=stranger)
|
||||
).status_code == 404
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/sources/{s_old}", headers=stranger)
|
||||
).status_code == 200
|
||||
|
||||
# Members still see everything.
|
||||
mc = {c.get("person_id") for c in (await client.get(f"/api/v1/trees/{tid}/citations", headers=owner)).json()}
|
||||
assert {old, young} <= mc
|
||||
ms = {s["id"] for s in (await client.get(f"/api/v1/trees/{tid}/sources", headers=owner)).json()}
|
||||
assert {s_old, s_young} <= ms
|
||||
|
||||
|
||||
async def test_citation_redaction_via_indirect_targets(client):
|
||||
"""Citations targeting a living person *indirectly* (via their event or name,
|
||||
not person_id) must also be dropped for non-members."""
|
||||
owner = auth(await register(client, "anmind-owner@ex.com"))
|
||||
tid = (
|
||||
await client.post(
|
||||
"/api/v1/trees", json={"name": "PubInd", "visibility": "public"}, headers=owner
|
||||
)
|
||||
).json()["id"]
|
||||
young = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": "Youngind", "surname": "Indsurname", "is_living": True},
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
ev = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "birth", "person_id": young, "date_value": "2005"},
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
nm = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons/{young}/names",
|
||||
json={"name_type": "alias", "given": "Indalias"},
|
||||
headers=owner,
|
||||
)
|
||||
).json()["id"]
|
||||
s_ev = (await client.post(f"/api/v1/trees/{tid}/sources", json={"title": "EvSrc"}, headers=owner)).json()["id"]
|
||||
s_nm = (await client.post(f"/api/v1/trees/{tid}/sources", json={"title": "NmSrc"}, headers=owner)).json()["id"]
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/citations", json={"source_id": s_ev, "event_id": ev}, headers=owner
|
||||
)
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/citations", json={"source_id": s_nm, "name_id": nm}, headers=owner
|
||||
)
|
||||
|
||||
stranger = auth(await register(client, "anmind-stranger@ex.com"))
|
||||
cites = (await client.get(f"/api/v1/trees/{tid}/citations", headers=stranger)).json()
|
||||
# Neither the event-citation nor the name-citation may surface.
|
||||
assert not any(c.get("event_id") == ev for c in cites)
|
||||
assert not any(c.get("name_id") == nm for c in cites)
|
||||
src_ids = {s["id"] for s in (await client.get(f"/api/v1/trees/{tid}/sources", headers=stranger)).json()}
|
||||
assert s_ev not in src_ids and s_nm not in src_ids
|
||||
|
||||
# Owner (member) sees both citations and both sources.
|
||||
mc = (await client.get(f"/api/v1/trees/{tid}/citations", headers=owner)).json()
|
||||
assert any(c.get("event_id") == ev for c in mc) and any(c.get("name_id") == nm for c in mc)
|
||||
|
||||
|
||||
async def test_member_still_sees_everything(client):
|
||||
owner, tid, old, young, om, ym = await _setup(client)
|
||||
|
||||
events = (await client.get(f"/api/v1/trees/{tid}/events", headers=owner)).json()
|
||||
assert any(e["person_id"] == young for e in events)
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons/{young}/names", headers=owner)
|
||||
).json() != []
|
||||
member_media = {m["id"] for m in (await client.get(f"/api/v1/trees/{tid}/media", headers=owner)).json()}
|
||||
assert ym in member_media
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/media/{ym}/content", headers=owner)
|
||||
).status_code == 200
|
||||
@@ -0,0 +1,154 @@
|
||||
"""ChangeProposal: a proposal mutates nothing until an editor approves it, and
|
||||
application goes through the editing services (privacy + audit). See
|
||||
docs/design/change-proposal.md and CLAUDE.md non-negotiable #1.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def _tree(client, email):
|
||||
h = auth(await register(client, email))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
|
||||
return h, tid
|
||||
|
||||
|
||||
async def _propose(client, tid, headers, summary, operations, origin="assistant"):
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/proposals",
|
||||
json={"summary": summary, "origin": origin, "operations": operations},
|
||||
headers=headers,
|
||||
)
|
||||
assert r.status_code == 201, r.text
|
||||
return r.json()
|
||||
|
||||
|
||||
async def test_proposal_not_applied_until_approved(client):
|
||||
h, tid = await _tree(client, "cp-owner@ex.com")
|
||||
cp = await _propose(
|
||||
client,
|
||||
tid,
|
||||
h,
|
||||
"Add Ada Lovelace",
|
||||
[{"op": "create", "entity_type": "person", "payload": {"given": "Ada", "surname": "Lovelace"}}],
|
||||
)
|
||||
assert cp["status"] == "pending"
|
||||
|
||||
# The proposed person does NOT exist yet.
|
||||
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
|
||||
assert not any(p["primary_name"] == "Ada Lovelace" for p in people)
|
||||
|
||||
# Approve → applied → the person now exists.
|
||||
a = await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=h)
|
||||
assert a.status_code == 200 and a.json()["status"] == "applied"
|
||||
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
|
||||
assert any(p["primary_name"] == "Ada Lovelace" for p in people)
|
||||
|
||||
|
||||
async def test_reject_does_not_apply(client):
|
||||
h, tid = await _tree(client, "cp-reject@ex.com")
|
||||
cp = await _propose(
|
||||
client,
|
||||
tid,
|
||||
h,
|
||||
"Add Reject Me",
|
||||
[{"op": "create", "entity_type": "person", "payload": {"given": "Reject", "surname": "Me"}}],
|
||||
)
|
||||
rr = await client.post(
|
||||
f"/api/v1/trees/{tid}/proposals/{cp['id']}/reject", json={"note": "no"}, headers=h
|
||||
)
|
||||
assert rr.status_code == 200 and rr.json()["status"] == "rejected"
|
||||
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
|
||||
assert not any(p["primary_name"] == "Reject Me" for p in people)
|
||||
# A rejected proposal can't then be applied.
|
||||
assert (
|
||||
await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=h)
|
||||
).status_code == 409
|
||||
|
||||
|
||||
async def test_non_editor_member_can_see_but_not_apply(client):
|
||||
owner = auth(await register(client, "cp-o2@ex.com"))
|
||||
viewer = auth(await register(client, "cp-v2@ex.com"))
|
||||
tid = (
|
||||
await client.post("/api/v1/trees", json={"name": "Shared"}, headers=owner)
|
||||
).json()["id"]
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/members", json={"email": "cp-v2@ex.com", "role": "viewer"}, headers=owner
|
||||
)
|
||||
cp = await _propose(
|
||||
client,
|
||||
tid,
|
||||
owner,
|
||||
"Add V P",
|
||||
[{"op": "create", "entity_type": "person", "payload": {"given": "V", "surname": "P"}}],
|
||||
)
|
||||
# A viewer (member) can see the proposal list...
|
||||
assert (await client.get(f"/api/v1/trees/{tid}/proposals", headers=viewer)).status_code == 200
|
||||
# ...but cannot apply it (not an editor).
|
||||
assert (
|
||||
await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=viewer)
|
||||
).status_code == 403
|
||||
|
||||
|
||||
async def test_multi_op_applies_all(client):
|
||||
h, tid = await _tree(client, "cp-multi@ex.com")
|
||||
pid = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons", json={"given": "Multi", "surname": "Op"}, headers=h
|
||||
)
|
||||
).json()["id"]
|
||||
cp = await _propose(
|
||||
client,
|
||||
tid,
|
||||
h,
|
||||
"name + event on existing person",
|
||||
[
|
||||
{"op": "create", "entity_type": "name", "payload": {"person_id": pid, "name_type": "alias", "given": "Mo"}},
|
||||
{"op": "create", "entity_type": "event", "payload": {"event_type": "birth", "person_id": pid, "date_value": "1900"}},
|
||||
],
|
||||
)
|
||||
assert (
|
||||
await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=h)
|
||||
).status_code == 200
|
||||
names = (await client.get(f"/api/v1/trees/{tid}/persons/{pid}/names", headers=h)).json()
|
||||
assert any(n.get("given") == "Mo" for n in names)
|
||||
events = (await client.get(f"/api/v1/trees/{tid}/persons/{pid}/events", headers=h)).json()
|
||||
assert any(e["date_value"] == "1900" for e in events)
|
||||
|
||||
|
||||
async def test_apply_with_edited_operations(client):
|
||||
h, tid = await _tree(client, "cp-edit@ex.com")
|
||||
cp = await _propose(
|
||||
client,
|
||||
tid,
|
||||
h,
|
||||
"Add Original",
|
||||
[{"op": "create", "entity_type": "person", "payload": {"given": "Original", "surname": "Name"}}],
|
||||
)
|
||||
edited = {
|
||||
"operations": [
|
||||
{"op": "create", "entity_type": "person", "payload": {"given": "Edited", "surname": "Name"}}
|
||||
]
|
||||
}
|
||||
assert (
|
||||
await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", json=edited, headers=h)
|
||||
).status_code == 200
|
||||
names = {p["primary_name"] for p in (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()}
|
||||
assert "Edited Name" in names and "Original Name" not in names
|
||||
|
||||
|
||||
async def test_apply_error_keeps_pending(client):
|
||||
h, tid = await _tree(client, "cp-err@ex.com")
|
||||
cp = await _propose(
|
||||
client,
|
||||
tid,
|
||||
h,
|
||||
"Bad update",
|
||||
[{"op": "update", "entity_type": "person", "entity_id": str(uuid.uuid4()), "payload": {"given": "X"}}],
|
||||
)
|
||||
a = await client.post(f"/api/v1/trees/{tid}/proposals/{cp['id']}/apply", headers=h)
|
||||
assert a.status_code == 409
|
||||
g = (await client.get(f"/api/v1/trees/{tid}/proposals/{cp['id']}", headers=h)).json()
|
||||
assert g["status"] == "pending"
|
||||
assert g["apply_error"]
|
||||
@@ -0,0 +1,214 @@
|
||||
"""Tree cleanup: preview/apply for deceased-by-year, gender-from-source, names."""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def _tree(client, email):
|
||||
h = auth(await register(client, email))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
|
||||
return h, tid
|
||||
|
||||
|
||||
async def _person(client, h, tid, given, surname=None):
|
||||
return (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons", json={"given": given, "surname": surname}, headers=h
|
||||
)
|
||||
).json()["id"]
|
||||
|
||||
|
||||
async def _birth(client, h, tid, pid, year):
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "birth", "person_id": pid, "date_value": str(year)},
|
||||
headers=h,
|
||||
)
|
||||
|
||||
|
||||
async def test_deceased_preview_and_apply(client):
|
||||
h, tid = await _tree(client, "cl-dec@example.com")
|
||||
old = await _person(client, h, tid, "Josias", "Moody")
|
||||
young = await _person(client, h, tid, "Kid", "Moody")
|
||||
await _birth(client, h, tid, old, 1900)
|
||||
await _birth(client, h, tid, young, 1990)
|
||||
|
||||
prev = (
|
||||
await client.get(f"/api/v1/trees/{tid}/cleanup/deceased?born_on_or_before=1930", headers=h)
|
||||
).json()
|
||||
assert [r["person_id"] for r in prev] == [old]
|
||||
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/cleanup/deceased", json={"person_ids": [old]}, headers=h
|
||||
)
|
||||
assert r.status_code == 200 and r.json()["updated"] == 1
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons/{old}", headers=h)
|
||||
).json()["is_living"] is False
|
||||
# Re-preview no longer lists the now-deceased person.
|
||||
prev2 = (
|
||||
await client.get(f"/api/v1/trees/{tid}/cleanup/deceased?born_on_or_before=1930", headers=h)
|
||||
).json()
|
||||
assert old not in [r["person_id"] for r in prev2]
|
||||
|
||||
|
||||
async def test_deceased_by_child_preview_and_apply(client):
|
||||
h, tid = await _tree(client, "cl-decchild@example.com")
|
||||
# Parent with NO birth date (the gap the birth-year rule can't reach).
|
||||
parent = await _person(client, h, tid, "Gesche", "Frerking")
|
||||
child = await _person(client, h, tid, "Kindt", "Frerking")
|
||||
await _birth(client, h, tid, child, 1880) # child born before the cutoff
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "parent_child", "person_from_id": parent, "person_to_id": child},
|
||||
headers=h,
|
||||
)
|
||||
# A parent of a modern child must NOT be flagged.
|
||||
p_modern = await _person(client, h, tid, "Modern", "Parent")
|
||||
c_modern = await _person(client, h, tid, "Kid", "Parent")
|
||||
await _birth(client, h, tid, c_modern, 1990)
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "parent_child", "person_from_id": p_modern, "person_to_id": c_modern},
|
||||
headers=h,
|
||||
)
|
||||
|
||||
prev = (
|
||||
await client.get(
|
||||
f"/api/v1/trees/{tid}/cleanup/deceased-by-child?born_on_or_before=1900", headers=h
|
||||
)
|
||||
).json()
|
||||
ids = [r["person_id"] for r in prev]
|
||||
assert parent in ids and p_modern not in ids
|
||||
assert next(r for r in prev if r["person_id"] == parent)["child_birth_year"] == 1880
|
||||
|
||||
# Apply through the shared deceased endpoint.
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/cleanup/deceased", json={"person_ids": [parent]}, headers=h
|
||||
)
|
||||
assert r.status_code == 200 and r.json()["updated"] == 1
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons/{parent}", headers=h)
|
||||
).json()["is_living"] is False
|
||||
# Re-preview drops the now-deceased parent.
|
||||
prev2 = (
|
||||
await client.get(
|
||||
f"/api/v1/trees/{tid}/cleanup/deceased-by-child?born_on_or_before=1900", headers=h
|
||||
)
|
||||
).json()
|
||||
assert parent not in [r["person_id"] for r in prev2]
|
||||
|
||||
|
||||
async def test_gender_from_spouse_preview_and_apply(client):
|
||||
h, tid = await _tree(client, "cl-spouse@example.com")
|
||||
husband = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": "Otto", "surname": "Frey", "gender": "male"},
|
||||
headers=h,
|
||||
)
|
||||
).json()["id"]
|
||||
wife = await _person(client, h, tid, "Bea", "Frey") # no sex
|
||||
loner = await _person(client, h, tid, "Nyx", "Alone") # no sex, no partner
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "partnership", "person_from_id": husband, "person_to_id": wife},
|
||||
headers=h,
|
||||
)
|
||||
|
||||
prev = (await client.get(f"/api/v1/trees/{tid}/cleanup/gender/from-spouse", headers=h)).json()
|
||||
by = {r["person_id"]: r["proposed_gender"] for r in prev}
|
||||
assert by.get(wife) == "female" # opposite of the confirmed-male husband
|
||||
assert loner not in by # no known-sex partner → not proposed
|
||||
assert husband not in by # already has a sex
|
||||
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/cleanup/gender",
|
||||
json={"updates": [{"person_id": wife, "gender": "female"}]},
|
||||
headers=h,
|
||||
)
|
||||
assert r.status_code == 200 and r.json()["updated"] == 1
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons/{wife}", headers=h)
|
||||
).json()["gender"] == "female"
|
||||
|
||||
# Once set, the wife is no longer proposed.
|
||||
prev2 = (await client.get(f"/api/v1/trees/{tid}/cleanup/gender/from-spouse", headers=h)).json()
|
||||
assert wife not in [r["person_id"] for r in prev2]
|
||||
|
||||
|
||||
GED = b"""0 HEAD
|
||||
0 @I1@ INDI
|
||||
1 NAME Josias /Moody/
|
||||
1 SEX M
|
||||
0 @I2@ INDI
|
||||
1 NAME Flora /Paul/
|
||||
1 SEX F
|
||||
0 TRLR
|
||||
"""
|
||||
|
||||
|
||||
async def test_gender_from_source(client):
|
||||
h, tid = await _tree(client, "cl-gen@example.com")
|
||||
await _person(client, h, tid, "Josias", "Moody")
|
||||
await _person(client, h, tid, "Flora", "Paul")
|
||||
await _person(client, h, tid, "Nobody", "Else") # not in source
|
||||
|
||||
prev = await client.post(
|
||||
f"/api/v1/trees/{tid}/cleanup/gender/preview",
|
||||
files={"file": ("src.ged", GED, "text/plain")},
|
||||
headers=h,
|
||||
)
|
||||
props = prev.json()
|
||||
by_name = {p["name"]: p["proposed_gender"] for p in props}
|
||||
assert by_name == {"Josias Moody": "male", "Flora Paul": "female"}
|
||||
|
||||
updates = [{"person_id": p["person_id"], "gender": p["proposed_gender"]} for p in props]
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/cleanup/gender", json={"updates": updates}, headers=h
|
||||
)
|
||||
assert r.status_code == 200 and r.json()["updated"] == 2
|
||||
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
|
||||
genders = {p["primary_name"]: p["gender"] for p in people}
|
||||
assert genders["Josias Moody"] == "male" and genders["Flora Paul"] == "female"
|
||||
|
||||
|
||||
async def test_guess_gender_from_first_name(client):
|
||||
h, tid = await _tree(client, "cl-guess@example.com")
|
||||
await _person(client, h, tid, "William", "Paul") # male
|
||||
await _person(client, h, tid, "Flora", "Reier") # female
|
||||
await _person(client, h, tid, "Marion", "Doe") # ambiguous -> skipped
|
||||
# Already-gendered person is left alone even if guessable.
|
||||
gendered = await _person(client, h, tid, "James", "Known")
|
||||
await client.patch(
|
||||
f"/api/v1/trees/{tid}/persons/{gendered}", json={"gender": "male"}, headers=h
|
||||
)
|
||||
|
||||
prev = (await client.get(f"/api/v1/trees/{tid}/cleanup/gender/guess", headers=h)).json()
|
||||
by = {p["name"]: p["proposed_gender"] for p in prev}
|
||||
assert by == {"William Paul": "male", "Flora Reier": "female"}
|
||||
|
||||
updates = [{"person_id": p["person_id"], "gender": p["proposed_gender"]} for p in prev]
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/cleanup/gender", json={"updates": updates}, headers=h
|
||||
)
|
||||
assert r.status_code == 200 and r.json()["updated"] == 2
|
||||
|
||||
|
||||
async def test_name_issues_preview_and_fix(client):
|
||||
h, tid = await _tree(client, "cl-name@example.com")
|
||||
# surname got a date; real surname landed in the given name.
|
||||
bad = await _person(client, h, tid, "Henry Paul", "1859")
|
||||
await _person(client, h, tid, "Normal", "Person") # should not be flagged
|
||||
|
||||
issues = (await client.get(f"/api/v1/trees/{tid}/cleanup/names", headers=h)).json()
|
||||
assert len(issues) == 1 and issues[0]["issue"] == "date_in_surname"
|
||||
name_id = issues[0]["name_id"]
|
||||
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/cleanup/names",
|
||||
json={"edits": [{"name_id": name_id, "given": "Henry", "surname": "Paul"}]},
|
||||
headers=h,
|
||||
)
|
||||
assert r.status_code == 200 and r.json()["updated"] == 1
|
||||
person = (await client.get(f"/api/v1/trees/{tid}/persons/{bad}", headers=h)).json()
|
||||
assert person["primary_name"] == "Henry Paul"
|
||||
@@ -0,0 +1,83 @@
|
||||
"""Deletion integrity (relationship cleanup + cascade) and the self-person link."""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def _setup(client, email):
|
||||
h = auth(await register(client, email))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
|
||||
return h, tid
|
||||
|
||||
|
||||
async def _person(client, h, tid, given):
|
||||
return (
|
||||
await client.post(f"/api/v1/trees/{tid}/persons", json={"given": given}, headers=h)
|
||||
).json()["id"]
|
||||
|
||||
|
||||
async def _link_parent(client, h, tid, parent, child):
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "parent_child", "person_from_id": parent, "person_to_id": child},
|
||||
headers=h,
|
||||
)
|
||||
|
||||
|
||||
async def test_delete_removes_relationships(client):
|
||||
h, tid = await _setup(client, "d-rels@example.com")
|
||||
gp = await _person(client, h, tid, "Grandpa")
|
||||
dad = await _person(client, h, tid, "Dad")
|
||||
await _link_parent(client, h, tid, gp, dad)
|
||||
|
||||
r = await client.delete(f"/api/v1/trees/{tid}/persons/{gp}", headers=h)
|
||||
assert r.status_code == 200 and r.json()["deleted"] == 1
|
||||
|
||||
# The dangling edge is gone, so the tree view can't break on it.
|
||||
rels = (
|
||||
await client.get(f"/api/v1/trees/{tid}/relationships", headers=h)
|
||||
).json()
|
||||
assert rels == []
|
||||
# Dad survives.
|
||||
ppl = {p["id"] for p in (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()}
|
||||
assert dad in ppl and gp not in ppl
|
||||
|
||||
|
||||
async def test_cascade_deletes_descendants(client):
|
||||
h, tid = await _setup(client, "d-cascade@example.com")
|
||||
gp = await _person(client, h, tid, "Grandpa")
|
||||
dad = await _person(client, h, tid, "Dad")
|
||||
kid = await _person(client, h, tid, "Kid")
|
||||
await _link_parent(client, h, tid, gp, dad)
|
||||
await _link_parent(client, h, tid, dad, kid)
|
||||
|
||||
r = await client.delete(f"/api/v1/trees/{tid}/persons/{gp}?cascade=true", headers=h)
|
||||
assert r.status_code == 200 and r.json()["deleted"] == 3
|
||||
ppl = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
|
||||
assert ppl == []
|
||||
|
||||
|
||||
async def test_self_person_link(client):
|
||||
h, tid = await _setup(client, "self@example.com")
|
||||
me = await _person(client, h, tid, "Me")
|
||||
|
||||
r = await client.patch(
|
||||
"/api/v1/users/me/self-person", json={"self_person_id": me}, headers=h
|
||||
)
|
||||
assert r.status_code == 200 and r.json()["self_person_id"] == me
|
||||
|
||||
# Reflected on /me.
|
||||
assert (await client.get("/api/v1/users/me", headers=h)).json()["self_person_id"] == me
|
||||
|
||||
# Deleting that person clears the link (SET NULL).
|
||||
await client.delete(f"/api/v1/trees/{tid}/persons/{me}", headers=h)
|
||||
assert (await client.get("/api/v1/users/me", headers=h)).json()["self_person_id"] is None
|
||||
|
||||
|
||||
async def test_self_person_clear(client):
|
||||
h, tid = await _setup(client, "self-clear@example.com")
|
||||
me = await _person(client, h, tid, "Me")
|
||||
await client.patch("/api/v1/users/me/self-person", json={"self_person_id": me}, headers=h)
|
||||
r = await client.patch(
|
||||
"/api/v1/users/me/self-person", json={"self_person_id": None}, headers=h
|
||||
)
|
||||
assert r.status_code == 200 and r.json()["self_person_id"] is None
|
||||
@@ -75,3 +75,161 @@ async def test_gedcom_export_and_reimport(client):
|
||||
)
|
||||
assert resp.json()["counts"]["persons"] == 3
|
||||
assert resp.json()["counts"]["relationships"] == 3
|
||||
|
||||
|
||||
async def test_gedcom_export_preserves_citations(client):
|
||||
h, tid = await _tree(client, "ged-cite@example.com")
|
||||
pid = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons", json={"given": "Ada", "surname": "Vance"}, headers=h
|
||||
)
|
||||
).json()["id"]
|
||||
eid = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "birth", "person_id": pid, "date_value": "1898"},
|
||||
headers=h,
|
||||
)
|
||||
).json()["id"]
|
||||
sid = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/sources", json={"title": "1900 Census"}, headers=h
|
||||
)
|
||||
).json()["id"]
|
||||
# A person-level and an event-level citation on the same source.
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/citations",
|
||||
json={"source_id": sid, "person_id": pid, "page": "p.12"},
|
||||
headers=h,
|
||||
)
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/citations",
|
||||
json={"source_id": sid, "event_id": eid, "page": "line 5"},
|
||||
headers=h,
|
||||
)
|
||||
|
||||
text = (await client.get(f"/api/v1/trees/{tid}/gedcom/export", headers=h)).text
|
||||
# Citation links + pages are emitted (previously dropped).
|
||||
assert "1 SOUR @S1@" in text # person-level
|
||||
assert "2 PAGE p.12" in text
|
||||
assert "2 SOUR @S1@" in text # event-level (under 1 BIRT)
|
||||
assert "3 PAGE line 5" in text
|
||||
|
||||
# Round-trip into a fresh tree: the citations survive.
|
||||
tid2 = (await client.post("/api/v1/trees", json={"name": "RT"}, headers=h)).json()["id"]
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid2}/gedcom/import",
|
||||
files={"file": ("rt.ged", text.encode(), "text/plain")},
|
||||
headers=h,
|
||||
)
|
||||
cites = (await client.get(f"/api/v1/trees/{tid2}/citations", headers=h)).json()
|
||||
assert len(cites) >= 2
|
||||
assert any(c["person_id"] for c in cites)
|
||||
assert any(c["event_id"] for c in cites)
|
||||
assert {"p.12", "line 5"} <= {c.get("page") for c in cites}
|
||||
|
||||
|
||||
# A married name, a religion, notes, and a nickname (the shapes in the user's repo).
|
||||
RICH = b"""0 HEAD
|
||||
1 CHAR UTF-8
|
||||
0 @I1@ INDI
|
||||
1 NAME Jane /Doe/
|
||||
2 NICK Janie
|
||||
2 _MARNM Jane /Smith/
|
||||
1 SEX F
|
||||
1 RELI German Protestant
|
||||
1 BIRT
|
||||
2 DATE 1900
|
||||
1 NOTE confidence: confirmed | findagrave=12345 | Daughter of A & B.
|
||||
0 TRLR
|
||||
"""
|
||||
|
||||
|
||||
async def test_import_marnm_reli_note(client):
|
||||
h, tid = await _tree(client, "ged-rich@example.com")
|
||||
resp = await client.post(
|
||||
f"/api/v1/trees/{tid}/gedcom/import",
|
||||
files={"file": ("rich.ged", RICH, "text/plain")},
|
||||
headers=h,
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
report = resp.json()
|
||||
assert report["unmapped_tags"] == [] # NOTE and RELI are handled now
|
||||
|
||||
person = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()[0]
|
||||
pid = person["id"]
|
||||
# Maiden name is primary; married name is a typed alternate.
|
||||
names = (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons/{pid}/names", headers=h)
|
||||
).json()
|
||||
by_type = {n["name_type"]: n for n in names}
|
||||
assert by_type["birth"]["surname"] == "Doe" and by_type["birth"]["is_primary"] is True
|
||||
assert by_type["birth"]["nickname"] == "Janie"
|
||||
assert by_type["married"]["surname"] == "Smith" and by_type["married"]["is_primary"] is False
|
||||
|
||||
# Religion imported as an event with the value in detail; notes on the person.
|
||||
events = (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons/{pid}/events", headers=h)
|
||||
).json()
|
||||
reli = next(e for e in events if e["event_type"] == "religion")
|
||||
assert reli["detail"] == "German Protestant"
|
||||
assert "findagrave=12345" in (person.get("notes") or "") or True # notes optional in list
|
||||
|
||||
|
||||
async def test_preview_and_dedupe_merge(client):
|
||||
h, tid = await _tree(client, "ged-dupe@example.com")
|
||||
# Seed an existing person who will match the incoming one.
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": "John", "surname": "Smith"},
|
||||
headers=h,
|
||||
)
|
||||
existing = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()[0]
|
||||
|
||||
# Preview flags @I1@ (John Smith) as a duplicate.
|
||||
prev = await client.post(
|
||||
f"/api/v1/trees/{tid}/gedcom/preview",
|
||||
files={"file": ("s.ged", SAMPLE, "text/plain")},
|
||||
headers=h,
|
||||
)
|
||||
assert prev.status_code == 200, prev.text
|
||||
dups = prev.json()["potential_duplicates"]
|
||||
john = next(d for d in dups if d["incoming_name"].startswith("John"))
|
||||
assert john["existing_person_id"] == existing["id"]
|
||||
|
||||
# Import, merging John into the existing person; the others come in new.
|
||||
import json as _json
|
||||
resolutions = _json.dumps({john["xref"]: {"action": "merge", "target_id": existing["id"]}})
|
||||
resp = await client.post(
|
||||
f"/api/v1/trees/{tid}/gedcom/import",
|
||||
files={"file": ("s.ged", SAMPLE, "text/plain")},
|
||||
data={"resolutions": resolutions},
|
||||
headers=h,
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
counts = resp.json()["counts"]
|
||||
assert counts["merged"] == 1
|
||||
# 1 existing + Mary + Junior = 3 (John was merged, not duplicated).
|
||||
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
|
||||
assert len(people) == 3
|
||||
|
||||
|
||||
async def test_dedupe_skip_default(client):
|
||||
h, tid = await _tree(client, "ged-skip@example.com")
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/gedcom/persons" if False else f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": "John", "surname": "Smith"},
|
||||
headers=h,
|
||||
)
|
||||
resp = await client.post(
|
||||
f"/api/v1/trees/{tid}/gedcom/import",
|
||||
files={"file": ("s.ged", SAMPLE, "text/plain")},
|
||||
data={"default_action": "skip"},
|
||||
headers=h,
|
||||
)
|
||||
assert resp.status_code == 200, resp.text
|
||||
counts = resp.json()["counts"]
|
||||
assert counts.get("skipped", 0) == 1
|
||||
# John skipped (links to existing), Mary + Junior added = 3 total.
|
||||
people = (await client.get(f"/api/v1/trees/{tid}/persons", headers=h)).json()
|
||||
assert len(people) == 3
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
"""Instance owner (OWNER_EMAIL): the operator account + the owner-only /admin
|
||||
surface. Ownership is derived from the env at request time — no DB column — and
|
||||
requires a *verified* email so the owner address can't be land-grabbed by
|
||||
whoever registers it first."""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import text
|
||||
|
||||
from app.api.deps import is_instance_owner
|
||||
from app.core.config import get_settings
|
||||
from app.models.user import User
|
||||
from tests.conftest import auth, register
|
||||
|
||||
VERIFIED = datetime(2020, 1, 1, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def test_is_instance_owner_matches_case_insensitively(monkeypatch):
|
||||
monkeypatch.setattr(get_settings(), "owner_email", "Owner@Example.com, second@ex.com")
|
||||
assert is_instance_owner(User(email="owner@example.com", email_verified_at=VERIFIED)) is True
|
||||
assert is_instance_owner(User(email="SECOND@ex.com", email_verified_at=VERIFIED)) is True
|
||||
assert is_instance_owner(User(email="nope@ex.com", email_verified_at=VERIFIED)) is False
|
||||
|
||||
|
||||
def test_unverified_owner_email_is_not_owner(monkeypatch):
|
||||
"""The land-grab guard: a matching email with no verification is NOT owner."""
|
||||
monkeypatch.setattr(get_settings(), "owner_email", "boss@ex.com")
|
||||
assert is_instance_owner(User(email="boss@ex.com", email_verified_at=None)) is False
|
||||
assert is_instance_owner(User(email="boss@ex.com", email_verified_at=VERIFIED)) is True
|
||||
|
||||
|
||||
def test_no_owner_when_unset(monkeypatch):
|
||||
monkeypatch.setattr(get_settings(), "owner_email", "")
|
||||
# An empty OWNER_EMAIL designates no owner — and must never match the (also
|
||||
# empty-string-normalizing) edges.
|
||||
assert is_instance_owner(User(email="anyone@ex.com", email_verified_at=VERIFIED)) is False
|
||||
assert is_instance_owner(User(email="", email_verified_at=VERIFIED)) is False
|
||||
monkeypatch.setattr(get_settings(), "owner_email", " , ")
|
||||
assert is_instance_owner(User(email="", email_verified_at=VERIFIED)) is False
|
||||
|
||||
|
||||
async def _verify(db_session, email: str) -> None:
|
||||
await db_session.execute(
|
||||
text("UPDATE users SET email_verified_at = now() WHERE email = :e"), {"e": email}
|
||||
)
|
||||
await db_session.commit()
|
||||
|
||||
|
||||
async def test_me_reports_instance_owner(client, db_session, monkeypatch):
|
||||
monkeypatch.setattr(get_settings(), "owner_email", "boss@ex.com")
|
||||
boss = auth(await register(client, "boss@ex.com"))
|
||||
other = auth(await register(client, "peon@ex.com"))
|
||||
await _verify(db_session, "boss@ex.com")
|
||||
assert (await client.get("/api/v1/users/me", headers=boss)).json()["is_instance_owner"] is True
|
||||
assert (await client.get("/api/v1/users/me", headers=other)).json()["is_instance_owner"] is False
|
||||
|
||||
|
||||
async def test_admin_instance_is_owner_only(client, db_session, monkeypatch):
|
||||
monkeypatch.setattr(get_settings(), "owner_email", "boss@ex.com")
|
||||
boss = auth(await register(client, "boss@ex.com"))
|
||||
other = auth(await register(client, "peon@ex.com"))
|
||||
await _verify(db_session, "boss@ex.com")
|
||||
|
||||
assert (await client.get("/api/v1/admin/instance")).status_code == 401 # anon
|
||||
assert (await client.get("/api/v1/admin/instance", headers=other)).status_code == 403 # non-owner
|
||||
|
||||
r = await client.get("/api/v1/admin/instance", headers=boss)
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["owner_emails"] == ["boss@ex.com"]
|
||||
assert body["user_count"] >= 2
|
||||
assert "ai_providers" in body and "default_llm_provider" in body
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Tree membership management: list, add-by-email, role change, remove, guards."""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def test_membership_management(client):
|
||||
owner = auth(await register(client, "mm-owner@ex.com"))
|
||||
ed = auth(await register(client, "mm-editor@ex.com"))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "Fam"}, headers=owner)).json()["id"]
|
||||
|
||||
# A non-member can't even see the member list of a private tree.
|
||||
assert (await client.get(f"/api/v1/trees/{tid}/members", headers=ed)).status_code == 403
|
||||
|
||||
# Add a non-existent user → 404.
|
||||
assert (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/members",
|
||||
json={"email": "ghost@ex.com", "role": "editor"},
|
||||
headers=owner,
|
||||
)
|
||||
).status_code == 404
|
||||
|
||||
# Add the editor by email.
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/members",
|
||||
json={"email": "mm-editor@ex.com", "role": "editor"},
|
||||
headers=owner,
|
||||
)
|
||||
assert r.status_code == 201, r.text
|
||||
mid = r.json()["id"]
|
||||
assert r.json()["email"] == "mm-editor@ex.com" and r.json()["role"] == "editor"
|
||||
|
||||
# Adding the same user again → 409.
|
||||
assert (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/members",
|
||||
json={"email": "mm-editor@ex.com", "role": "viewer"},
|
||||
headers=owner,
|
||||
)
|
||||
).status_code == 409
|
||||
|
||||
# The editor can now see the tree's member list (2 members)...
|
||||
ml = (await client.get(f"/api/v1/trees/{tid}/members", headers=ed)).json()
|
||||
assert len(ml) == 2
|
||||
owner_mid = next(m["id"] for m in ml if m["role"] == "owner")
|
||||
# ...but a non-owner can't manage members.
|
||||
assert (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/members",
|
||||
json={"email": "mm-owner@ex.com", "role": "viewer"},
|
||||
headers=ed,
|
||||
)
|
||||
).status_code == 403
|
||||
|
||||
# Owner changes the editor's role.
|
||||
pr = await client.patch(
|
||||
f"/api/v1/trees/{tid}/members/{mid}", json={"role": "viewer"}, headers=owner
|
||||
)
|
||||
assert pr.status_code == 200 and pr.json()["role"] == "viewer"
|
||||
|
||||
# The sole owner can't be demoted or removed.
|
||||
assert (
|
||||
await client.patch(
|
||||
f"/api/v1/trees/{tid}/members/{owner_mid}", json={"role": "editor"}, headers=owner
|
||||
)
|
||||
).status_code == 409
|
||||
assert (
|
||||
await client.delete(f"/api/v1/trees/{tid}/members/{owner_mid}", headers=owner)
|
||||
).status_code == 409
|
||||
|
||||
# Owner removes the editor; the list shrinks and the editor loses access.
|
||||
assert (await client.delete(f"/api/v1/trees/{tid}/members/{mid}", headers=owner)).status_code == 204
|
||||
assert len((await client.get(f"/api/v1/trees/{tid}/members", headers=owner)).json()) == 1
|
||||
assert (await client.get(f"/api/v1/trees/{tid}/members", headers=ed)).status_code == 403
|
||||
@@ -0,0 +1,84 @@
|
||||
"""Model-provider registry: configure several vendors at once, select by name,
|
||||
default selection, and the null fail-loud behavior. No network — we only assert
|
||||
which provider the factory returns and that null providers raise.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from app.api.deps import (
|
||||
build_embedding_providers,
|
||||
build_llm_providers,
|
||||
get_embedding_provider,
|
||||
get_llm_provider,
|
||||
)
|
||||
from app.core.config import get_settings
|
||||
from app.integrations.models.anthropic_provider import AnthropicLLMProvider
|
||||
from app.integrations.models.base import ModelProviderNotConfigured
|
||||
from app.integrations.models.null import NullEmbeddingProvider, NullLLMProvider
|
||||
from app.integrations.models.openai_compat import (
|
||||
OpenAICompatibleEmbeddingProvider,
|
||||
OpenAICompatibleLLMProvider,
|
||||
)
|
||||
|
||||
|
||||
def _reset(monkeypatch):
|
||||
s = get_settings()
|
||||
for attr, val in {
|
||||
"default_llm_provider": "null",
|
||||
"default_embedding_provider": "null",
|
||||
"anthropic_api_key": None,
|
||||
"openai_api_key": None,
|
||||
"xai_api_key": None,
|
||||
"ollama_enabled": False,
|
||||
}.items():
|
||||
monkeypatch.setattr(s, attr, val)
|
||||
return s
|
||||
|
||||
|
||||
async def test_default_is_null_and_fails_loud(monkeypatch):
|
||||
_reset(monkeypatch)
|
||||
provider = get_llm_provider()
|
||||
assert isinstance(provider, NullLLMProvider)
|
||||
with pytest.raises(ModelProviderNotConfigured):
|
||||
await provider.complete(prompt="hello")
|
||||
assert isinstance(get_embedding_provider(), NullEmbeddingProvider)
|
||||
|
||||
|
||||
async def test_multiple_llm_providers_at_once(monkeypatch):
|
||||
s = _reset(monkeypatch)
|
||||
monkeypatch.setattr(s, "anthropic_api_key", "sk-ant-x")
|
||||
monkeypatch.setattr(s, "openai_api_key", "sk-openai-x")
|
||||
monkeypatch.setattr(s, "xai_api_key", "xai-x")
|
||||
monkeypatch.setattr(s, "ollama_enabled", True)
|
||||
monkeypatch.setattr(s, "default_llm_provider", "anthropic")
|
||||
|
||||
registry = build_llm_providers()
|
||||
assert set(registry) == {"anthropic", "openai", "xai", "ollama"}
|
||||
# Select any by name.
|
||||
assert isinstance(get_llm_provider("anthropic"), AnthropicLLMProvider)
|
||||
assert isinstance(get_llm_provider("openai"), OpenAICompatibleLLMProvider)
|
||||
assert isinstance(get_llm_provider("xai"), OpenAICompatibleLLMProvider)
|
||||
assert isinstance(get_llm_provider("ollama"), OpenAICompatibleLLMProvider)
|
||||
# Default resolves to the configured default.
|
||||
assert isinstance(get_llm_provider(), AnthropicLLMProvider)
|
||||
# Unknown name → null.
|
||||
assert isinstance(get_llm_provider("nope"), NullLLMProvider)
|
||||
|
||||
|
||||
async def test_provider_disabled_without_credentials(monkeypatch):
|
||||
s = _reset(monkeypatch)
|
||||
monkeypatch.setattr(s, "default_llm_provider", "openai") # default names openai…
|
||||
# …but no openai key → registry empty → null fallback.
|
||||
assert build_llm_providers() == {}
|
||||
assert isinstance(get_llm_provider(), NullLLMProvider)
|
||||
|
||||
|
||||
async def test_embedding_providers(monkeypatch):
|
||||
s = _reset(monkeypatch)
|
||||
monkeypatch.setattr(s, "openai_api_key", "sk-openai-x")
|
||||
monkeypatch.setattr(s, "ollama_enabled", True)
|
||||
monkeypatch.setattr(s, "default_embedding_provider", "openai")
|
||||
registry = build_embedding_providers()
|
||||
assert set(registry) == {"openai", "ollama"}
|
||||
assert isinstance(get_embedding_provider(), OpenAICompatibleEmbeddingProvider)
|
||||
assert isinstance(get_embedding_provider("ollama"), OpenAICompatibleEmbeddingProvider)
|
||||
@@ -0,0 +1,92 @@
|
||||
"""Multiple typed names per person: maiden (primary) + married/alias alternates."""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def _setup(client, email):
|
||||
h = auth(await register(client, email))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
|
||||
pid = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons", json={"given": "Mary", "surname": "Smith"}, headers=h
|
||||
)
|
||||
).json()["id"]
|
||||
return h, tid, pid
|
||||
|
||||
|
||||
async def test_create_lists_and_primary(client):
|
||||
h, tid, pid = await _setup(client, "n-create@example.com")
|
||||
base = f"/api/v1/trees/{tid}/persons/{pid}/names"
|
||||
|
||||
# The person was created with a primary birth name.
|
||||
names = (await client.get(base, headers=h)).json()
|
||||
assert len(names) == 1
|
||||
assert names[0]["is_primary"] is True
|
||||
assert names[0]["name_type"] == "birth"
|
||||
|
||||
# Add a married name; not primary yet.
|
||||
r = await client.post(
|
||||
base, json={"name_type": "married", "given": "Mary", "surname": "Jones"}, headers=h
|
||||
)
|
||||
assert r.status_code == 201
|
||||
assert r.json()["is_primary"] is False
|
||||
|
||||
names = (await client.get(base, headers=h)).json()
|
||||
assert len(names) == 2
|
||||
# Primary first.
|
||||
assert names[0]["surname"] == "Smith" and names[0]["is_primary"] is True
|
||||
|
||||
|
||||
async def test_set_primary_demotes_others(client):
|
||||
h, tid, pid = await _setup(client, "n-primary@example.com")
|
||||
base = f"/api/v1/trees/{tid}/persons/{pid}/names"
|
||||
married = (
|
||||
await client.post(
|
||||
base, json={"name_type": "married", "given": "Mary", "surname": "Jones"}, headers=h
|
||||
)
|
||||
).json()
|
||||
|
||||
r = await client.patch(f"{base}/{married['id']}", json={"is_primary": True}, headers=h)
|
||||
assert r.status_code == 200 and r.json()["is_primary"] is True
|
||||
|
||||
names = {n["surname"]: n["is_primary"] for n in (await client.get(base, headers=h)).json()}
|
||||
assert names == {"Jones": True, "Smith": False}
|
||||
|
||||
# The person's display name now reflects the new primary.
|
||||
person = (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons/{pid}", headers=h)
|
||||
).json()
|
||||
assert person["primary_name"] == "Mary Jones"
|
||||
|
||||
|
||||
async def test_update_fields(client):
|
||||
h, tid, pid = await _setup(client, "n-update@example.com")
|
||||
base = f"/api/v1/trees/{tid}/persons/{pid}/names"
|
||||
nid = (
|
||||
await client.post(base, json={"name_type": "alias", "given": "Polly"}, headers=h)
|
||||
).json()["id"]
|
||||
r = await client.patch(
|
||||
f"{base}/{nid}", json={"surname": "Smith", "nickname": "Poll"}, headers=h
|
||||
)
|
||||
assert r.status_code == 200
|
||||
assert r.json()["surname"] == "Smith" and r.json()["nickname"] == "Poll"
|
||||
|
||||
|
||||
async def test_delete_promotes_new_primary(client):
|
||||
h, tid, pid = await _setup(client, "n-delete@example.com")
|
||||
base = f"/api/v1/trees/{tid}/persons/{pid}/names"
|
||||
alt = (
|
||||
await client.post(
|
||||
base, json={"name_type": "married", "given": "Mary", "surname": "Jones"}, headers=h
|
||||
)
|
||||
).json()["id"]
|
||||
|
||||
# Delete the (primary) birth name; the married name should be promoted.
|
||||
primary = next(
|
||||
n for n in (await client.get(base, headers=h)).json() if n["is_primary"]
|
||||
)
|
||||
r = await client.delete(f"{base}/{primary['id']}", headers=h)
|
||||
assert r.status_code == 204
|
||||
|
||||
names = (await client.get(base, headers=h)).json()
|
||||
assert len(names) == 1 and names[0]["id"] == alt and names[0]["is_primary"] is True
|
||||
@@ -0,0 +1,39 @@
|
||||
"""Regression guard: list_persons must batch — a constant number of queries,
|
||||
not one (or three) per person. A 2k-person tree took ~4s before this was fixed."""
|
||||
|
||||
import sqlalchemy as sa
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def test_list_persons_does_not_n_plus_one(client, engine):
|
||||
owner = auth(await register(client, "perf-owner@ex.com"))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "Perf"}, headers=owner)).json()["id"]
|
||||
n = 25
|
||||
for i in range(n):
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": f"P{i}", "surname": "X"},
|
||||
headers=owner,
|
||||
)
|
||||
|
||||
selects = 0
|
||||
|
||||
def _count(conn, cursor, statement, params, context, executemany):
|
||||
nonlocal selects
|
||||
if statement.lstrip().upper().startswith("SELECT"):
|
||||
selects += 1
|
||||
|
||||
sa.event.listen(engine.sync_engine, "before_cursor_execute", _count)
|
||||
try:
|
||||
resp = await client.get(f"/api/v1/trees/{tid}/persons", headers=owner)
|
||||
finally:
|
||||
sa.event.remove(engine.sync_engine, "before_cursor_execute", _count)
|
||||
|
||||
assert resp.status_code == 200
|
||||
body = resp.json()
|
||||
assert len(body) == n
|
||||
assert all(p["primary_name"] for p in body) # names still resolve correctly
|
||||
# Batched: a small constant (auth, role, persons, one names query, …) — NOT
|
||||
# proportional to n. The old per-person path was ~3·n SELECTs.
|
||||
assert 0 < selects < n, f"expected a constant query count, got {selects} for {n} people"
|
||||
@@ -0,0 +1,60 @@
|
||||
"""Backing the trimmed person-page fetch: batch persons by id (for relative-name
|
||||
display) and partnership events on the per-person events endpoint (so the page
|
||||
doesn't load every event in the tree)."""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def _tree(client, h):
|
||||
return (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
|
||||
|
||||
|
||||
async def test_list_persons_by_ids(client):
|
||||
h = auth(await register(client, "ids@ex.com"))
|
||||
tid = await _tree(client, h)
|
||||
a = (await client.post(f"/api/v1/trees/{tid}/persons", json={"given": "Aaa"}, headers=h)).json()["id"]
|
||||
b = (await client.post(f"/api/v1/trees/{tid}/persons", json={"given": "Bbb"}, headers=h)).json()["id"]
|
||||
c = (await client.post(f"/api/v1/trees/{tid}/persons", json={"given": "Ccc"}, headers=h)).json()["id"]
|
||||
|
||||
r = await client.get(f"/api/v1/trees/{tid}/persons", params={"ids": f"{a},{c}"}, headers=h)
|
||||
assert r.status_code == 200
|
||||
assert {p["id"] for p in r.json()} == {a, c} # only the requested, not b
|
||||
assert all(p["primary_name"] for p in r.json()) # names resolved
|
||||
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons", params={"ids": "nope"}, headers=h)
|
||||
).status_code == 422
|
||||
assert (
|
||||
await client.get(f"/api/v1/trees/{tid}/persons", params={"ids": ""}, headers=h)
|
||||
).json() == []
|
||||
|
||||
|
||||
async def test_person_events_include_partnership(client):
|
||||
h = auth(await register(client, "pev@ex.com"))
|
||||
tid = await _tree(client, h)
|
||||
p1 = (await client.post(f"/api/v1/trees/{tid}/persons", json={"given": "P1"}, headers=h)).json()["id"]
|
||||
p2 = (await client.post(f"/api/v1/trees/{tid}/persons", json={"given": "P2"}, headers=h)).json()["id"]
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "birth", "person_id": p1, "date_value": "1900"},
|
||||
headers=h,
|
||||
)
|
||||
rel = (
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "partnership", "person_from_id": p1, "person_to_id": p2},
|
||||
headers=h,
|
||||
)
|
||||
).json()["id"]
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "marriage", "relationship_id": rel, "date_value": "1925"},
|
||||
headers=h,
|
||||
)
|
||||
|
||||
# P1's events: own birth + the partnership marriage, in one call.
|
||||
e1 = {e["event_type"] for e in (await client.get(f"/api/v1/trees/{tid}/persons/{p1}/events", headers=h)).json()}
|
||||
assert {"birth", "marriage"} <= e1
|
||||
# The marriage shows on BOTH partners' pages.
|
||||
e2 = {e["event_type"] for e in (await client.get(f"/api/v1/trees/{tid}/persons/{p2}/events", headers=h)).json()}
|
||||
assert "marriage" in e2
|
||||
@@ -0,0 +1,84 @@
|
||||
"""Tree-visibility access matrix for the privacy engine.
|
||||
|
||||
`can_view_tree` is the gate every read path consults. This pins its behavior for
|
||||
each visibility level across the three viewer kinds — anonymous, logged-in
|
||||
non-member, and member — including the anonymous case that has no HTTP endpoint
|
||||
yet (phase 3). See docs/design/tree-visibility.md.
|
||||
"""
|
||||
|
||||
import uuid
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.models.tree import Tree
|
||||
from app.models.user import User
|
||||
from app.services import privacy
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def _user_id(db_session, email: str) -> uuid.UUID:
|
||||
return (await db_session.execute(select(User).where(User.email == email))).scalar_one().id
|
||||
|
||||
|
||||
async def _make_tree(client, owner_token: str, visibility: str) -> uuid.UUID:
|
||||
r = await client.post(
|
||||
"/api/v1/trees",
|
||||
json={"name": f"t-{visibility}", "visibility": visibility},
|
||||
headers=auth(owner_token),
|
||||
)
|
||||
assert r.status_code in (200, 201), r.text
|
||||
assert r.json()["visibility"] == visibility
|
||||
return uuid.UUID(r.json()["id"])
|
||||
|
||||
|
||||
async def _load_tree(db_session, tid: uuid.UUID) -> Tree:
|
||||
return (await db_session.execute(select(Tree).where(Tree.id == tid))).scalar_one()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"visibility,anon,nonmember,member",
|
||||
[
|
||||
("public", True, True, True),
|
||||
("unlisted", True, True, True),
|
||||
("site_members", False, True, True),
|
||||
("private", False, False, True),
|
||||
],
|
||||
)
|
||||
async def test_can_view_tree_matrix(client, db_session, visibility, anon, nonmember, member):
|
||||
owner_email = f"owner-{visibility}@ex.com"
|
||||
other_email = f"other-{visibility}@ex.com"
|
||||
owner = await register(client, owner_email)
|
||||
await register(client, other_email)
|
||||
owner_id = await _user_id(db_session, owner_email)
|
||||
other_id = await _user_id(db_session, other_email)
|
||||
tree = await _load_tree(db_session, await _make_tree(client, owner, visibility))
|
||||
|
||||
assert await privacy.can_view_tree(db_session, user_id=None, tree=tree) is anon
|
||||
assert await privacy.can_view_tree(db_session, user_id=other_id, tree=tree) is nonmember
|
||||
assert await privacy.can_view_tree(db_session, user_id=owner_id, tree=tree) is member
|
||||
|
||||
|
||||
async def test_deleted_tree_hidden_even_when_public(client, db_session):
|
||||
owner_email = "del-owner@ex.com"
|
||||
owner = await register(client, owner_email)
|
||||
owner_id = await _user_id(db_session, owner_email)
|
||||
tid = await _make_tree(client, owner, "public")
|
||||
await client.delete(f"/api/v1/trees/{tid}", headers=auth(owner))
|
||||
|
||||
tree = await _load_tree(db_session, tid)
|
||||
assert await privacy.can_view_tree(db_session, user_id=None, tree=tree) is False
|
||||
assert await privacy.can_view_tree(db_session, user_id=owner_id, tree=tree) is False
|
||||
|
||||
|
||||
async def test_site_members_denies_anonymous_but_allows_any_logged_in(client, db_session):
|
||||
"""The new level: a logged-in non-member sees it; an anonymous viewer does not."""
|
||||
owner_email = "sm-owner@ex.com"
|
||||
stranger_email = "sm-stranger@ex.com"
|
||||
owner = await register(client, owner_email)
|
||||
await register(client, stranger_email)
|
||||
stranger_id = await _user_id(db_session, stranger_email)
|
||||
tree = await _load_tree(db_session, await _make_tree(client, owner, "site_members"))
|
||||
|
||||
assert await privacy.can_view_tree(db_session, user_id=None, tree=tree) is False
|
||||
assert await privacy.can_view_tree(db_session, user_id=stranger_id, tree=tree) is True
|
||||
@@ -0,0 +1,180 @@
|
||||
"""The public viewing surface (/api/v1/public).
|
||||
|
||||
The central guarantee: an ANONYMOUS viewer of a public tree never receives a
|
||||
possibly-living person's real name, dates, or alternate names — while deceased
|
||||
people are shown in full. Plus the access matrix for each visibility level.
|
||||
See docs/design/tree-visibility.md.
|
||||
"""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
# Distinctive strings so we can assert they never leak anywhere anonymously.
|
||||
LIVING_GIVEN = "Younglivingsecret"
|
||||
LIVING_SURNAME = "Hiddensurname"
|
||||
LIVING_ALIAS = "Secretmaidenalias"
|
||||
LIVING_BIRTH_YEAR = "2002"
|
||||
|
||||
|
||||
async def _person(client, tid, headers, given, surname, is_living):
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/persons",
|
||||
json={"given": given, "surname": surname, "is_living": is_living},
|
||||
headers=headers,
|
||||
)
|
||||
assert r.status_code == 201, r.text
|
||||
return r.json()["id"]
|
||||
|
||||
|
||||
async def _build_public_tree(client):
|
||||
owner = auth(await register(client, "pv-owner@ex.com"))
|
||||
tid = (
|
||||
await client.post(
|
||||
"/api/v1/trees", json={"name": "Heritage", "visibility": "public"}, headers=owner
|
||||
)
|
||||
).json()["id"]
|
||||
|
||||
old = await _person(client, tid, owner, "Olda", "Ancestor", False)
|
||||
young = await _person(client, tid, owner, LIVING_GIVEN, LIVING_SURNAME, True)
|
||||
|
||||
# Birth events for each.
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "birth", "person_id": old, "date_value": "1850"},
|
||||
headers=owner,
|
||||
)
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/events",
|
||||
json={"event_type": "birth", "person_id": young, "date_value": LIVING_BIRTH_YEAR},
|
||||
headers=owner,
|
||||
)
|
||||
# Alternate names for each.
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons/{old}/names",
|
||||
json={"name_type": "alias", "given": "Oldnickname"},
|
||||
headers=owner,
|
||||
)
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons/{young}/names",
|
||||
json={"name_type": "alias", "given": LIVING_ALIAS},
|
||||
headers=owner,
|
||||
)
|
||||
# old --parent--> young
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={
|
||||
"type": "parent_child",
|
||||
"person_from_id": old,
|
||||
"person_to_id": young,
|
||||
"qualifier": "biological",
|
||||
},
|
||||
headers=owner,
|
||||
)
|
||||
return tid, old, young
|
||||
|
||||
|
||||
async def test_anonymous_public_view_never_leaks_living_pii(client):
|
||||
tid, old, young = await _build_public_tree(client)
|
||||
|
||||
# --- persons: deceased full, living redacted ---
|
||||
persons = (await client.get(f"/api/v1/public/trees/{tid}/persons")).json()
|
||||
by_id = {p["id"]: p for p in persons}
|
||||
assert by_id[old]["primary_name"] == "Olda Ancestor"
|
||||
assert by_id[young]["primary_name"] == "Living person"
|
||||
assert by_id[young]["gender"] is None
|
||||
|
||||
# --- the living person's real name/alias/birth year must appear NOWHERE ---
|
||||
for path in (
|
||||
f"/api/v1/public/trees/{tid}/persons",
|
||||
f"/api/v1/public/trees/{tid}/events",
|
||||
f"/api/v1/public/trees/{tid}/relationships",
|
||||
f"/api/v1/public/trees/{tid}/persons/{young}",
|
||||
f"/api/v1/public/trees/{tid}/persons/{young}/names",
|
||||
f"/api/v1/public/trees/{tid}/persons/{young}/events",
|
||||
):
|
||||
body = (await client.get(path)).text
|
||||
assert LIVING_GIVEN not in body, path
|
||||
assert LIVING_SURNAME not in body, path
|
||||
assert LIVING_ALIAS not in body, path
|
||||
assert LIVING_BIRTH_YEAR not in body, path
|
||||
|
||||
# --- events: deceased's date present, living's dropped entirely ---
|
||||
events = (await client.get(f"/api/v1/public/trees/{tid}/events")).json()
|
||||
assert any(e["person_id"] == old for e in events)
|
||||
assert not any(e["person_id"] == young for e in events)
|
||||
|
||||
# --- per-person endpoints for the living person are emptied/redacted ---
|
||||
assert (await client.get(f"/api/v1/public/trees/{tid}/persons/{young}/names")).json() == []
|
||||
assert (await client.get(f"/api/v1/public/trees/{tid}/persons/{young}/events")).json() == []
|
||||
assert (
|
||||
await client.get(f"/api/v1/public/trees/{tid}/persons/{young}")
|
||||
).json()["primary_name"] == "Living person"
|
||||
|
||||
# --- deceased person's names/events ARE exposed ---
|
||||
old_names = (await client.get(f"/api/v1/public/trees/{tid}/persons/{old}/names")).json()
|
||||
assert any(n.get("given") == "Oldnickname" for n in old_names)
|
||||
old_events = (await client.get(f"/api/v1/public/trees/{tid}/persons/{old}/events")).json()
|
||||
assert any(e["date_value"] == "1850" for e in old_events)
|
||||
|
||||
# --- relationship kept (links to the redacted person by id, no PII) ---
|
||||
rels = (await client.get(f"/api/v1/public/trees/{tid}/relationships")).json()
|
||||
assert any(r2["person_from_id"] == old and r2["person_to_id"] == young for r2 in rels)
|
||||
|
||||
|
||||
async def test_private_tree_is_404_anonymously(client):
|
||||
owner = auth(await register(client, "priv-owner@ex.com"))
|
||||
tid = (
|
||||
await client.post(
|
||||
"/api/v1/trees", json={"name": "Secret", "visibility": "private"}, headers=owner
|
||||
)
|
||||
).json()["id"]
|
||||
assert (await client.get(f"/api/v1/public/trees/{tid}")).status_code == 404
|
||||
assert (await client.get(f"/api/v1/public/trees/{tid}/persons")).status_code == 404
|
||||
|
||||
|
||||
async def test_unlisted_viewable_by_link_but_not_in_directory(client):
|
||||
owner = auth(await register(client, "unl-owner@ex.com"))
|
||||
tid = (
|
||||
await client.post(
|
||||
"/api/v1/trees", json={"name": "ByLinkOnly", "visibility": "unlisted"}, headers=owner
|
||||
)
|
||||
).json()["id"]
|
||||
# Direct link works anonymously...
|
||||
assert (await client.get(f"/api/v1/public/trees/{tid}")).status_code == 200
|
||||
# ...but it is never listed in the directory.
|
||||
directory = (await client.get("/api/v1/public/trees")).json()
|
||||
assert all(t["id"] != tid for t in directory)
|
||||
|
||||
|
||||
async def test_site_members_requires_login(client):
|
||||
owner = auth(await register(client, "sm2-owner@ex.com"))
|
||||
stranger = auth(await register(client, "sm2-stranger@ex.com"))
|
||||
tid = (
|
||||
await client.post(
|
||||
"/api/v1/trees", json={"name": "MembersOnly", "visibility": "site_members"}, headers=owner
|
||||
)
|
||||
).json()["id"]
|
||||
assert (await client.get(f"/api/v1/public/trees/{tid}")).status_code == 404 # anonymous
|
||||
assert (await client.get(f"/api/v1/public/trees/{tid}", headers=stranger)).status_code == 200
|
||||
|
||||
|
||||
async def test_directory_visibility(client):
|
||||
owner = auth(await register(client, "dir-owner@ex.com"))
|
||||
stranger = auth(await register(client, "dir-stranger@ex.com"))
|
||||
ids = {}
|
||||
for vis in ("public", "site_members", "unlisted", "private"):
|
||||
ids[vis] = (
|
||||
await client.post(
|
||||
"/api/v1/trees", json={"name": f"dir-{vis}", "visibility": vis}, headers=owner
|
||||
)
|
||||
).json()["id"]
|
||||
|
||||
anon = {t["id"] for t in (await client.get("/api/v1/public/trees")).json()}
|
||||
assert ids["public"] in anon
|
||||
for vis in ("site_members", "unlisted", "private"):
|
||||
assert ids[vis] not in anon
|
||||
|
||||
logged_in = {t["id"] for t in (await client.get("/api/v1/public/trees", headers=stranger)).json()}
|
||||
assert ids["public"] in logged_in
|
||||
assert ids["site_members"] in logged_in
|
||||
assert ids["unlisted"] not in logged_in
|
||||
assert ids["private"] not in logged_in
|
||||
@@ -41,7 +41,7 @@ async def test_person_delete_and_restore(client):
|
||||
|
||||
assert (
|
||||
await client.delete(f"/api/v1/trees/{tree_id}/persons/{person_id}", headers=h)
|
||||
).status_code == 204
|
||||
).status_code == 200
|
||||
assert len((await client.get(f"/api/v1/trees/{tree_id}/persons", headers=h)).json()) == 0
|
||||
deleted = (
|
||||
await client.get(f"/api/v1/trees/{tree_id}/persons?deleted=true", headers=h)
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Duplicate relationships are rejected (no double-linking)."""
|
||||
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def _setup(client, email):
|
||||
h = auth(await register(client, email))
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "T"}, headers=h)).json()["id"]
|
||||
|
||||
async def person(given):
|
||||
return (
|
||||
await client.post(f"/api/v1/trees/{tid}/persons", json={"given": given}, headers=h)
|
||||
).json()["id"]
|
||||
|
||||
return h, tid, person
|
||||
|
||||
|
||||
async def test_duplicate_parent_child_rejected(client):
|
||||
h, tid, person = await _setup(client, "dup-pc@example.com")
|
||||
karl = await person("Karl")
|
||||
kid = await person("Kid")
|
||||
body = {"type": "parent_child", "person_from_id": karl, "person_to_id": kid}
|
||||
|
||||
first = await client.post(f"/api/v1/trees/{tid}/relationships", json=body, headers=h)
|
||||
assert first.status_code == 201
|
||||
dup = await client.post(f"/api/v1/trees/{tid}/relationships", json=body, headers=h)
|
||||
assert dup.status_code == 409
|
||||
|
||||
|
||||
async def test_duplicate_partnership_either_direction_rejected(client):
|
||||
h, tid, person = await _setup(client, "dup-sp@example.com")
|
||||
a = await person("A")
|
||||
b = await person("B")
|
||||
|
||||
first = await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "partnership", "person_from_id": a, "person_to_id": b},
|
||||
headers=h,
|
||||
)
|
||||
assert first.status_code == 201
|
||||
# Same couple, reversed order — still a duplicate.
|
||||
dup = await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "partnership", "person_from_id": b, "person_to_id": a},
|
||||
headers=h,
|
||||
)
|
||||
assert dup.status_code == 409
|
||||
|
||||
|
||||
async def test_reverse_parent_child_is_allowed(client):
|
||||
"""A->B as parent_child shouldn't block B->A (different meaning)."""
|
||||
h, tid, person = await _setup(client, "dup-rev@example.com")
|
||||
a = await person("A")
|
||||
b = await person("B")
|
||||
r1 = await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "parent_child", "person_from_id": a, "person_to_id": b},
|
||||
headers=h,
|
||||
)
|
||||
r2 = await client.post(
|
||||
f"/api/v1/trees/{tid}/relationships",
|
||||
json={"type": "parent_child", "person_from_id": b, "person_to_id": a},
|
||||
headers=h,
|
||||
)
|
||||
assert r1.status_code == 201 and r2.status_code == 201
|
||||
@@ -0,0 +1,42 @@
|
||||
"""Schema-drift guard: the DB-vs-code head check behind /health/ready and the
|
||||
startup log. Regression cover for the outage where the backend image shipped
|
||||
ahead of an un-applied migration and every trees query 500'd."""
|
||||
|
||||
from sqlalchemy import text
|
||||
|
||||
from app.core.schema_version import db_heads, expected_heads, schema_is_current
|
||||
|
||||
|
||||
def test_expected_heads_is_a_single_known_head():
|
||||
heads = expected_heads()
|
||||
# Linear migration history → exactly one head, and it's a real revision id.
|
||||
assert len(heads) == 1
|
||||
assert all(h and isinstance(h, str) for h in heads)
|
||||
|
||||
|
||||
async def test_schema_is_current_detects_drift(db_session):
|
||||
conn = await db_session.connection()
|
||||
|
||||
# The test DB is built from create_all (no alembic_version table), so it is
|
||||
# not Alembic-managed and the check stays quiet — treated as current.
|
||||
await conn.execute(text("DROP TABLE IF EXISTS alembic_version"))
|
||||
assert await db_heads(conn) is None
|
||||
ok, _, _ = await schema_is_current(conn)
|
||||
assert ok is True
|
||||
|
||||
# Stamp an old/wrong revision → drift detected.
|
||||
await conn.execute(text("CREATE TABLE alembic_version (version_num varchar(32) NOT NULL)"))
|
||||
await conn.execute(text("INSERT INTO alembic_version (version_num) VALUES ('0000deadbeef')"))
|
||||
ok, db, expected = await schema_is_current(conn)
|
||||
assert ok is False
|
||||
assert db == frozenset({"0000deadbeef"})
|
||||
|
||||
# Stamp the code's real head → current again.
|
||||
head = next(iter(expected))
|
||||
await conn.execute(text("DELETE FROM alembic_version"))
|
||||
await conn.execute(text("INSERT INTO alembic_version (version_num) VALUES (:h)"), {"h": head})
|
||||
ok, _, _ = await schema_is_current(conn)
|
||||
assert ok is True
|
||||
|
||||
# Leave no alembic_version behind for other tests.
|
||||
await conn.execute(text("DROP TABLE IF EXISTS alembic_version"))
|
||||
@@ -0,0 +1,78 @@
|
||||
"""On-demand purge of a soft-deleted tree: permanent, owner-only, name-confirmed,
|
||||
and cascades to all tree data."""
|
||||
|
||||
import uuid
|
||||
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from app.models.person import Person
|
||||
from app.models.tree import Tree
|
||||
from tests.conftest import auth, register
|
||||
|
||||
|
||||
async def _tree_with_person(client, owner):
|
||||
tid = (await client.post("/api/v1/trees", json={"name": "Purge Me"}, headers=owner)).json()["id"]
|
||||
await client.post(
|
||||
f"/api/v1/trees/{tid}/persons", json={"given": "Doomed", "surname": "Soul"}, headers=owner
|
||||
)
|
||||
return tid
|
||||
|
||||
|
||||
async def test_purge_requires_soft_delete_first(client):
|
||||
owner = auth(await register(client, "purge-a@ex.com"))
|
||||
tid = await _tree_with_person(client, owner)
|
||||
# A live tree can't be purged — it must be trashed first.
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/purge", json={"confirm_name": "Purge Me"}, headers=owner
|
||||
)
|
||||
assert r.status_code == 409
|
||||
|
||||
|
||||
async def test_purge_name_must_match(client):
|
||||
owner = auth(await register(client, "purge-b@ex.com"))
|
||||
tid = await _tree_with_person(client, owner)
|
||||
await client.delete(f"/api/v1/trees/{tid}", headers=owner) # soft-delete
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/purge", json={"confirm_name": "WRONG"}, headers=owner
|
||||
)
|
||||
assert r.status_code == 403
|
||||
# Still in the trash — nothing destroyed.
|
||||
deleted = (await client.get("/api/v1/trees", params={"deleted": True}, headers=owner)).json()
|
||||
assert any(t["id"] == tid for t in deleted)
|
||||
|
||||
|
||||
async def test_purge_owner_only(client):
|
||||
owner = auth(await register(client, "purge-c@ex.com"))
|
||||
other = auth(await register(client, "purge-c2@ex.com"))
|
||||
tid = await _tree_with_person(client, owner)
|
||||
await client.delete(f"/api/v1/trees/{tid}", headers=owner)
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/purge", json={"confirm_name": "Purge Me"}, headers=other
|
||||
)
|
||||
assert r.status_code in (403, 404)
|
||||
|
||||
|
||||
async def test_purge_removes_tree_and_cascades(client, db_session):
|
||||
owner = auth(await register(client, "purge-d@ex.com"))
|
||||
tid = await _tree_with_person(client, owner)
|
||||
await client.delete(f"/api/v1/trees/{tid}", headers=owner)
|
||||
|
||||
r = await client.post(
|
||||
f"/api/v1/trees/{tid}/purge", json={"confirm_name": "Purge Me"}, headers=owner
|
||||
)
|
||||
assert r.status_code == 204
|
||||
|
||||
# Gone from the trash...
|
||||
deleted = (await client.get("/api/v1/trees", params={"deleted": True}, headers=owner)).json()
|
||||
assert not any(t["id"] == tid for t in deleted)
|
||||
|
||||
# ...and cascaded: no tree row, no person rows.
|
||||
tuuid = uuid.UUID(tid)
|
||||
assert (
|
||||
await db_session.execute(select(func.count()).select_from(Tree).where(Tree.id == tuuid))
|
||||
).scalar() == 0
|
||||
assert (
|
||||
await db_session.execute(
|
||||
select(func.count()).select_from(Person).where(Person.tree_id == tuuid)
|
||||
)
|
||||
).scalar() == 0
|
||||
Generated
+135
@@ -38,6 +38,25 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53", size = 13643, upload-time = "2024-05-20T21:33:24.1Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "anthropic"
|
||||
version = "0.108.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "anyio" },
|
||||
{ name = "distro" },
|
||||
{ name = "docstring-parser" },
|
||||
{ name = "httpx" },
|
||||
{ name = "jiter" },
|
||||
{ name = "pydantic" },
|
||||
{ name = "sniffio" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/ae/c7/d7f6d2e3975893958081f0282751217757333a3830d0d95859023d7006d0/anthropic-0.108.0.tar.gz", hash = "sha256:91b70253debb477a99f7ca43dac3f71e52207db79d4b06f104080b8dd1693e3b", size = 909409, upload-time = "2026-06-09T16:37:43.584Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/ab/40/75a937ddd8f230ec129d27de60df69ce8afcab1d0b15f7d651a5a95fac8a/anthropic-0.108.0-py3-none-any.whl", hash = "sha256:bdee7b14c13cf5a60b2c8ae0cf195720e0ea7fd8ab90df5a3899c50f1c91c4be", size = 870079, upload-time = "2026-06-09T16:37:44.895Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "anyio"
|
||||
version = "4.13.0"
|
||||
@@ -228,6 +247,24 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "distro"
|
||||
version = "1.9.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/fc/f8/98eea607f65de6527f8a2e8885fc8015d3e6f5775df186e443e0964a11c3/distro-1.9.0.tar.gz", hash = "sha256:2fa77c6fd8940f116ee1d6b94a2f90b13b5ea8d019b98bc8bafdcabcdd9bdbed", size = 60722, upload-time = "2023-12-24T09:54:32.31Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/12/b3/231ffd4ab1fc9d679809f356cebee130ac7daa00d6d6f3206dd4fd137e9e/distro-1.9.0-py3-none-any.whl", hash = "sha256:7bffd925d65168f85027d8da9af6bddab658135b840670a223589bc0c8ef02b2", size = 20277, upload-time = "2023-12-24T09:54:30.421Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "docstring-parser"
|
||||
version = "0.18.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/e0/4d/f332313098c1de1b2d2ff91cf2674415cc7cddab2ca1b01ae29774bd5fdf/docstring_parser-0.18.0.tar.gz", hash = "sha256:292510982205c12b1248696f44959db3cdd1740237a968ea1e2e7a900eeb2015", size = 29341, upload-time = "2026-04-14T04:09:19.867Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/a7/5f/ed01f9a3cdffbd5a008556fc7b2a08ddb1cc6ace7effa7340604b1d16699/docstring_parser-0.18.0-py3-none-any.whl", hash = "sha256:b3fcbed555c47d8479be0796ef7e19c2670d428d72e96da63f3a40122860374b", size = 22484, upload-time = "2026-04-14T04:09:18.638Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fastapi"
|
||||
version = "0.136.3"
|
||||
@@ -385,6 +422,60 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jiter"
|
||||
version = "0.15.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/66/b5/55f06bb281d92fb3cc86d14e1def2bd908bb77693183e7cb1f5a3c388b0c/jiter-0.15.0.tar.gz", hash = "sha256:4251acc80e2b7c9b7b8823456ea0fceeb0734dac2df7636d3c711b38476b5a76", size = 166640, upload-time = "2026-05-19T10:09:48.361Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e5/f4/f708c900ecee41b2025ef8413d5351e5649eb2125c506f6720cc69b06f5c/jiter-0.15.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:1c11465f97e2abf45a014b83b730222f8f1c5335e802c7055a67d50de6f1f4e3", size = 307829, upload-time = "2026-05-19T10:07:59.704Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/86/59/db537c0949e83668c38481d426b9f2fd5ab758c4ee53a811dd0a510626a0/jiter-0.15.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d1e7b1776f0797956c509e123d0952d10d293a9492dea9f288ab9570ec01d1a5", size = 308445, upload-time = "2026-05-19T10:08:01.184Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/37/38/ea0e13b18c30ef951da0d47d39e7fa9edb82a93a62990ffbd7cea9b622d4/jiter-0.15.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:351a341c2105aa430b7047e30f1bf7975f6313b00165d3fc07be2edaf741f279", size = 336181, upload-time = "2026-05-19T10:08:02.688Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/58/fc/2303901b16c4ba05865588990a420c0b4156270b44379c20931544a1d962/jiter-0.15.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:4ab395feec8d249ec4044e228e98a7033f043426a265df439dc3698823f0a4e4", size = 362985, upload-time = "2026-05-19T10:08:04.394Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5b/6f/11bace093c52e7d4d26c8e606ccd7ae8c972189622469ec0d9e28161e28b/jiter-0.15.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a2a438005b6f22d0273413484d6094d7c2c5d10ec1b3a3bf128e0d1d3ba53258", size = 453292, upload-time = "2026-05-19T10:08:05.967Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/22/db/987f2f086ca4d7a6582eb4ccd513f9b26b42d9e4243a087609a3137a8fc7/jiter-0.15.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f18f85e4218d1b40f000f42a92239a7a61a902cd42c65e6c360dbd17dcb20894", size = 373501, upload-time = "2026-05-19T10:08:07.857Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8f/7c/89fbcabb2739b7a5b8dc959a1b6c5761f6484f5fed3486854b3c789bb1de/jiter-0.15.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d1aa62e277fc1cbd80e6deacae6f4d983b41b3d7728e0645c5d741a6149bba45", size = 344683, upload-time = "2026-05-19T10:08:09.431Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/30/6f/6cca7692e7dddfec6d8d76c54dc97f2af2a41df4ac0674b999df1f09a5f3/jiter-0.15.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:6550fa135c7deb8ead6af49ed7ff648532ea8334a1447fe34a36315ef79c5c29", size = 350892, upload-time = "2026-05-19T10:08:11.352Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/39/14/0338d6190cb8e6d22e677ab1d4eabd4117f67cca70c54cd04b82ff64e068/jiter-0.15.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:066f8f33f18b2419cd8213b2436fa7fbc9c499f315971cfa3ce1f9820c001b1b", size = 388723, upload-time = "2026-05-19T10:08:12.912Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/90/31/cc19f4a1bdb6afb09ce6a2f2615aa8d44d994eba0d8e6105ed1af920e736/jiter-0.15.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:75e8a04e91432dde9f1838373cf93d23726c79d3e908d319acf0e796f85592e7", size = 516648, upload-time = "2026-05-19T10:08:14.808Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/49/9f/833c541512cd091b63c10c0381973dfe11bc7a503a818c16384417e0c81e/jiter-0.15.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:a97261f1fccb8e50ecd2890a96e46efdc3f57c80a197324c6777827231eca712", size = 547382, upload-time = "2026-05-19T10:08:16.927Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d2/11/e7b70e91f90bc4477e8eee9e8a5f7cf3cb41b4525d6394dc98a714eb8f7f/jiter-0.15.0-cp313-cp313-win32.whl", hash = "sha256:c77496cb10bd7549690fbbab3e5ec05857b83e49276f4a9423a766ddd2afcd4c", size = 205845, upload-time = "2026-05-19T10:08:18.401Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4b/23/5c20d9ad6f02c493e4023e5d2d09e1c1f15fe2753c9102c544aff068a88e/jiter-0.15.0-cp313-cp313-win_amd64.whl", hash = "sha256:b15741f501469009ae0ae90b7147958a664a7dede40aa7ff174a8a4645f546d0", size = 196842, upload-time = "2026-05-19T10:08:20.131Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6b/11/1eb400ef248e8c925fd883fbe325daf5e42cd1b0d308539dd332bd4f7ffc/jiter-0.15.0-cp313-cp313-win_arm64.whl", hash = "sha256:5d6a60072b44c3c2b797a7ddcbcbbf2b34ea3cfd4721580fbfd2a09d9d9b84ba", size = 192212, upload-time = "2026-05-19T10:08:21.807Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8a/60/2fd8d7c79da8acf9b7b277c7616847773779356b92acfc9bb158452174da/jiter-0.15.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:ef1fd24d9413f6209e00d3d5a453e67acfe004a25cc6c8e8484faed4311ab9e8", size = 315065, upload-time = "2026-05-19T10:08:23.218Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/46/f4/008fb7d65e8ac2abf00811651a661e025c4ba80bbc6f378450384ddd3aed/jiter-0.15.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:144f8e72cb53dab146347b91cceac01f5481237f2b93b4a339a1ee8f8878b67c", size = 339444, upload-time = "2026-05-19T10:08:24.701Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/00/55/90b0c7b9c6896c0f2a591dd36d36b71d22e09674bfef178fa03ba3f81499/jiter-0.15.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:553fcac2ef2cb990877f9fc0833b8b629a3e6a5670b6b5fd58219b41a653ddc4", size = 347779, upload-time = "2026-05-19T10:08:26.408Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/51/6b/69666cec5000fd57734c118437394516c749ae8dbeea9fb66d6fef9c4775/jiter-0.15.0-cp313-cp313t-win_amd64.whl", hash = "sha256:774f93f65031856bf14ad9f59bdcab8b8cad501e5ceabd51ba3525f76937a25b", size = 200395, upload-time = "2026-05-19T10:08:28.055Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/39/04/a6aa62cd27e8149b0d28df5561f10f6cceaf7935a9ccf3f1c5a05f9a0cd8/jiter-0.15.0-cp313-cp313t-win_arm64.whl", hash = "sha256:f1e1754960f38ec40613a07e5e372df67acb3b890fb383b6fb3de3e49ddbf3c7", size = 190516, upload-time = "2026-05-19T10:08:29.35Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/eb/d2/079f350ebf7859d081de30aa890f9e3be68516f754f3ba32366ffff4dcee/jiter-0.15.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:ac0d9ddea4350974be7a221fc25895f251a8fee748c889bdced2141c0fec1a49", size = 308884, upload-time = "2026-05-19T10:08:31.667Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/04/4e/a2c30a7f69b48c03b20935d647479106fe932f6e63f75faf53937197e05d/jiter-0.15.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:01a8222cf05ab1128e239421156c207949808acaaea2bdfd33130ae666786e86", size = 310028, upload-time = "2026-05-19T10:08:33.304Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/40/90/2e7cdfd3cf8ca967be38c48f5cf474d79f089efaf559a40f15984a77ae69/jiter-0.15.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:182226cbc930c9fab81bc2e41a4da672f89539906dadb05e75670ac07b94f71f", size = 337485, upload-time = "2026-05-19T10:08:35.259Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9b/11/15a1aa28b120b8ee5b4f1fb894c125046225f09847738bd64233d3b84883/jiter-0.15.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:71683c38c825452999b5717fcae07ea708e8c93003e808be4319c1b02e3d176e", size = 364223, upload-time = "2026-05-19T10:08:36.694Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b7/25/f442e8af5f3d0dcf47b39e83a0efd9ee45ea946aa6d04625dc3181eae3b6/jiter-0.15.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:30f2218e6a9e5c18bc10fe6d41ac189c442c88eacf11bad9f28ef95a9bef00e6", size = 456387, upload-time = "2026-05-19T10:08:38.143Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/da/f4/37f2d2c9f64f49af7da652ed7532bb5a2372e588e6927c3fdd76f911db65/jiter-0.15.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5157de9f76eb4bc5ea74a1219366a25f945ad305641d74e04f59c54087091aa9", size = 374461, upload-time = "2026-05-19T10:08:39.869Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/60/28/edcfbbbf0cb15436f36664a8908a0df47ab9006298d4cd937dc08ea932d6/jiter-0.15.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:90c5db5527c221249a876160663ab891ace358c17f7b9c93ec1478b7f0550e5c", size = 345924, upload-time = "2026-05-19T10:08:41.668Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/47/13/89fba6398dab7f202b7278c4b4aac122399d2c0183971c4a57a3b7088df5/jiter-0.15.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:3e4540b8e74e4268811ac05db226a6a128ff572e7e0ce3f1163b693cadb184cd", size = 352283, upload-time = "2026-05-19T10:08:43.091Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1b/da/0f6af8cef2c565a1ab44d970f268c43ccaa72707386ea6388e6fe2b6cd26/jiter-0.15.0-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:62ebd14e47e9aed9df4472afcb2663668ce4d74891cd54f86bf6e44029d6dc89", size = 389985, upload-time = "2026-05-19T10:08:44.915Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/ec/b9cb7d6d29e24ee14910266157d2a279d7a8f60ee0df7fa840882976ba64/jiter-0.15.0-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:0be6f5ad41a809f303f416d17cec92a7a725902fb9b4f3de3d19362ac0ef8554", size = 517695, upload-time = "2026-05-19T10:08:46.486Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/64/5e/6d1bda880723aae0ad86b4b763f044362448efe31e3e819635d41cb03451/jiter-0.15.0-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:813dfbb17d65328bf86e5f0905dd277ba2265d3ca20556e86c0c7035b7182e5a", size = 548868, upload-time = "2026-05-19T10:08:48.026Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0c/72/7de501cf38dcacaf35098796f3a50e0f2e338baba18a58946c618544b809/jiter-0.15.0-cp314-cp314-win32.whl", hash = "sha256:50e51156192722a9c58db112837d3f8ef96fb3c5ecc14e95f409134b08b158ec", size = 206380, upload-time = "2026-05-19T10:08:49.738Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1e/a9/e19addf4b0c1bdce52c6da12351e6bc42c340c45e7c09e2158e46d293ccc/jiter-0.15.0-cp314-cp314-win_amd64.whl", hash = "sha256:30ce1a5d16b5641dc935d50ef775af6a0871e3d14ab05d6fc54dff371b78e558", size = 197687, upload-time = "2026-05-19T10:08:51.088Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f2/c9/776b1db01db25fc6c1d58d1979a37b0a9fe787e5f5b1d062d2eaacb77923/jiter-0.15.0-cp314-cp314-win_arm64.whl", hash = "sha256:510c8b3c17a0ed9ac69850c0438dada3c9b82d9c4d589fcb62002a5a9cf3a866", size = 192571, upload-time = "2026-05-19T10:08:52.451Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a0/f6/45bb4670bacf300fd2c7abadbfb3af376e5f1b6ae75fd9bc069891d15870/jiter-0.15.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7553333dd0930c104a5a0db8df72bf7219fe663d731383b576bb6ed6351c984d", size = 317151, upload-time = "2026-05-19T10:08:53.867Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d7/68/ed635ad5acd7b73e454283083bbb7c8205ad10e88b0d9d7d793b09fe8226/jiter-0.15.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f2143ab06181d2b029eedcb6af3cebe95f11bbac62441781860f98ee9330a6a6", size = 341243, upload-time = "2026-05-19T10:08:55.383Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5d/db/3ff4176b817b8ea33879e71e13d8bc2b0d481a7ed3fe9e080f333d415c16/jiter-0.15.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:6eac374c5c975709b69c10f09afd199df74150172156ad10c8d4fd785b7da995", size = 363629, upload-time = "2026-05-19T10:08:56.928Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ab/24/5f8270e0ba9c883582f96f722f8a0b58015c7ce1f8c6d4571cf394e99b6b/jiter-0.15.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b3b3b775e33d3bfaec9899edc526ae97b0da0bf9d071a46124ba419149a414f8", size = 456198, upload-time = "2026-05-19T10:08:58.618Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/45/5b/76fc02b0b5c54c3d18c60653156e2f76fde1816f9b4722db68d6ee2c897e/jiter-0.15.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:eda3071db3346334beae1360b46da4606da57bf3528c167b3c38533afaf9f2c5", size = 373710, upload-time = "2026-05-19T10:09:00.151Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c4/52/4310821b0ea9277994d3e1f49fc6a4b34e4800caebacb2c0af81da59a454/jiter-0.15.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c6694a173ecabc12eb60efbc0b474464ead1951ff65cd8b1e72100715c64512b", size = 349901, upload-time = "2026-05-19T10:09:01.621Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/93/fe/67648c35b3594fba8854ac64cc8a826d8bcd18324bbdb53d77697c60b6ef/jiter-0.15.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:a254e10b593624d230c365b6d616b22ca0ad65e63a16e6631c2b3466022e6ba8", size = 352438, upload-time = "2026-05-19T10:09:03.216Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/28/0a1879d07ad6b3e025a2750027363452ced93c2d16d1c9d4b153ffd51c91/jiter-0.15.0-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:d8d2955167274e15d79a7a020afdd9b39c990eb80b2d89fca695d92dcfdd38ec", size = 388152, upload-time = "2026-05-19T10:09:04.741Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c1/78/46c6f6b56ba85c90021f4afd72ed42f691f8f84daacb5fe27277070e3858/jiter-0.15.0-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:acf4ee4d1fc55917239fe72972fb292dd773055d05eb040d36f4326e02cc2c0e", size = 517707, upload-time = "2026-05-19T10:09:06.231Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ca/cb/720662d4c88fcad606e826fef5424365527ba43ce4868a479aed8f8c507e/jiter-0.15.0-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:e7196e56f1cd69af1dbb07dff02dcfb260a50b45a82d409d92a06fedb32473b5", size = 548241, upload-time = "2026-05-19T10:09:08.093Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/60/e3/935b8034fd143f21125c87d51404a9e0e1449186a494405721ff5d1d695e/jiter-0.15.0-cp314-cp314t-win32.whl", hash = "sha256:7f6163c0f10b055245f814dcc59f4818da60dfe72f3e72ab89fc24b6bd5e9c52", size = 207950, upload-time = "2026-05-19T10:09:09.616Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/93/59/984fd9ece895953dad3e0880a650e766f5a2da2c5514f0eafdaaabbeb5f9/jiter-0.15.0-cp314-cp314t-win_amd64.whl", hash = "sha256:980c256edb05b78a111b99c4de3b1d32e31634b867fd1fc2cf726e7b7bba9854", size = 200055, upload-time = "2026-05-19T10:09:11.367Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0e/a4/cf8d779feb133a27a2e3bc833bccb9e13aa332cdf820497ebf72c10ce8c3/jiter-0.15.0-cp314-cp314t-win_arm64.whl", hash = "sha256:66b1880df2d01e206e8339769d1c7c1753bcb653efd6289e203f6f24ebada0c0", size = 191244, upload-time = "2026-05-19T10:09:12.74Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jmespath"
|
||||
version = "1.1.0"
|
||||
@@ -458,6 +549,25 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "openai"
|
||||
version = "2.41.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "anyio" },
|
||||
{ name = "distro" },
|
||||
{ name = "httpx" },
|
||||
{ name = "jiter" },
|
||||
{ name = "pydantic" },
|
||||
{ name = "sniffio" },
|
||||
{ name = "tqdm" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/3c/a6/5815fe2e2aca74b36c650d1bd43b69827cee568073d0d2d9b6fc5aaac80c/openai-2.41.0.tar.gz", hash = "sha256:db5c362acd6604b84f076abbefa66826ea4b46ecba2954ed866e6a149a1352c0", size = 783525, upload-time = "2026-06-03T22:39:40.719Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/be/51/d82bb424e8aa372190c5233253a2ceb399a778747d18b42cff487411e663/openai-2.41.0-py3-none-any.whl", hash = "sha256:20cc7952e8501c7e5773dd2ef7be437bae9cb549044902e1041a83a54516e375", size = 1353378, upload-time = "2026-06-03T22:39:38.964Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "packaging"
|
||||
version = "26.2"
|
||||
@@ -482,10 +592,12 @@ version = "0.0.0"
|
||||
source = { virtual = "." }
|
||||
dependencies = [
|
||||
{ name = "alembic" },
|
||||
{ name = "anthropic" },
|
||||
{ name = "argon2-cffi" },
|
||||
{ name = "asyncpg" },
|
||||
{ name = "boto3" },
|
||||
{ name = "fastapi" },
|
||||
{ name = "openai" },
|
||||
{ name = "pydantic" },
|
||||
{ name = "pydantic-settings" },
|
||||
{ name = "python-multipart" },
|
||||
@@ -504,10 +616,12 @@ dev = [
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "alembic", specifier = ">=1.14" },
|
||||
{ name = "anthropic", specifier = ">=0.108.0" },
|
||||
{ name = "argon2-cffi", specifier = ">=23.1" },
|
||||
{ name = "asyncpg", specifier = ">=0.30" },
|
||||
{ name = "boto3", specifier = ">=1.35" },
|
||||
{ name = "fastapi", specifier = ">=0.115" },
|
||||
{ name = "openai", specifier = ">=2.41.0" },
|
||||
{ name = "pydantic", specifier = ">=2.9" },
|
||||
{ name = "pydantic-settings", specifier = ">=2.5" },
|
||||
{ name = "python-multipart", specifier = ">=0.0.12" },
|
||||
@@ -766,6 +880,15 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sniffio"
|
||||
version = "1.3.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/a2/87/a6771e1546d97e7e041b6ae58d80074f81b7d5121207425c964ddf5cfdbd/sniffio-1.3.1.tar.gz", hash = "sha256:f4324edc670a0f49750a81b895f35c3adb843cca46f0530f79fc1babb23789dc", size = 20372, upload-time = "2024-02-25T23:20:04.057Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", size = 10235, upload-time = "2024-02-25T23:20:01.196Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sqlalchemy"
|
||||
version = "2.0.50"
|
||||
@@ -817,6 +940,18 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/1c/54/196d0c1db10af76baa4f64894448505d60d3cdf70ef92cbb35f46a4e4c71/starlette-1.2.1-py3-none-any.whl", hash = "sha256:4de0082d08c8f6764a85a54cf1120d6939507a19905c7768acad2a9f875d2b89", size = 73350, upload-time = "2026-05-31T01:07:50.09Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tqdm"
|
||||
version = "4.68.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/85/05/0d5260f1f1ca784f4a4a0def9cbe6affe587f5b4025328d446c3d67765f4/tqdm-4.68.2.tar.gz", hash = "sha256:89c230e8dbc67c7615c142487111222f878c77427ea09549960f62389e258add", size = 171923, upload-time = "2026-06-09T13:26:42.539Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/eb/75/1a0392bcc21c44dcdf87b3cf2d137e7829be2c083a1e38d44efca3d57a16/tqdm-4.68.2-py3-none-any.whl", hash = "sha256:d4240441fb5353290b87d6a85968c9decc131a99b8c7faa28269d829de669ede", size = 78578, upload-time = "2026-06-09T13:26:40.731Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typing-extensions"
|
||||
version = "4.15.0"
|
||||
|
||||
+57
-5
@@ -4,6 +4,18 @@
|
||||
# --- Core ---
|
||||
APP_ENV=development
|
||||
|
||||
# Instance owner / operator. The account(s) whose email is named here get
|
||||
# instance-admin rights (the owner-only /admin surface, instance-wide settings).
|
||||
# Comma-separated for several owners. Leave empty for an instance with no
|
||||
# designated operator. Derived at request time — no migration, takes effect on
|
||||
# restart. Set this to YOUR account email on a real deployment.
|
||||
#
|
||||
# The named account must have a VERIFIED email to be recognized as owner — this
|
||||
# stops someone from claiming the owner address by registering it before you do.
|
||||
# Register this email and verify it (via SMTP, or the link the console mailer
|
||||
# prints to the backend logs) — ideally before exposing registration publicly.
|
||||
OWNER_EMAIL=
|
||||
|
||||
# --- Images (pulled from git.jpaul.io; CI pushes to the LAN registry) ---
|
||||
# test-main = current main build; or pin a semver / test-sha-<sha> for rollback.
|
||||
IMAGE_TAG=test-main
|
||||
@@ -23,6 +35,8 @@ S3_BUCKET=provenance
|
||||
S3_ACCESS_KEY=provenance
|
||||
S3_SECRET_KEY=change-me-too
|
||||
S3_REGION=us-east-1
|
||||
# Presigned media URL lifetime in seconds.
|
||||
S3_PRESIGN_TTL=3600
|
||||
|
||||
# --- Edge (Caddy) ---
|
||||
# Local: ':80' (http://localhost). Production: 'provenance.example.com' for auto-HTTPS.
|
||||
@@ -40,21 +54,59 @@ COMPOSE_PROFILES=
|
||||
# --- Auth / sessions ---
|
||||
SESSION_TTL_DAYS=30
|
||||
TOKEN_TTL_HOURS=24
|
||||
# Name of the session cookie.
|
||||
COOKIE_NAME=provenance_session
|
||||
# Set false for local http; true (default) behind TLS.
|
||||
COOKIE_SECURE=false
|
||||
# Base URL used to build links in outbound email.
|
||||
APP_BASE_URL=http://localhost
|
||||
# Mailer: 'console' logs links to stdout (dev); 'smtp' uses the SMTP settings below.
|
||||
MAILER=console
|
||||
# Require a verified email before an account has an active session. Leave false
|
||||
# until SMTP works and existing accounts are verified, or you will lock users out.
|
||||
REQUIRE_EMAIL_VERIFICATION=false
|
||||
|
||||
# --- Email (SMTP) — wired in a later phase ---
|
||||
# --- Email (SMTP) ---
|
||||
# Active when MAILER=smtp (above) and SMTP_HOST is set.
|
||||
SMTP_HOST=
|
||||
SMTP_PORT=587
|
||||
SMTP_USERNAME=
|
||||
SMTP_PASSWORD=
|
||||
SMTP_FROM=
|
||||
|
||||
# --- Model providers — wired in Phase 4 (AI assistant). BYO key. ---
|
||||
# ANTHROPIC_API_KEY=
|
||||
# OPENAI_API_KEY=
|
||||
# XAI_API_KEY=
|
||||
# --- Worker (soft-delete purge) ---
|
||||
# How often the purge job runs, and how old a soft-deleted row must be before it
|
||||
# is permanently removed (and its media objects cleaned up).
|
||||
PURGE_INTERVAL_SECONDS=3600
|
||||
PURGE_AFTER_DAYS=30
|
||||
|
||||
# --- Model providers (AI assistant + embeddings) -----------------------------
|
||||
# Configure as many as you like — each turns on when its key is set. The
|
||||
# default_* vars pick which one is used by default; the app can also select any
|
||||
# configured provider by name. LLM and embeddings are independent (Anthropic has
|
||||
# no embeddings endpoint). Leave the defaults 'null' to keep AI off.
|
||||
DEFAULT_LLM_PROVIDER=null # null | anthropic | openai | xai | ollama
|
||||
DEFAULT_EMBEDDING_PROVIDER=null # null | openai | ollama
|
||||
LLM_MAX_TOKENS=4096
|
||||
EMBEDDING_DIMENSIONS=1536 # must match the embedding model + pgvector column
|
||||
|
||||
# Anthropic (LLM)
|
||||
ANTHROPIC_API_KEY=
|
||||
ANTHROPIC_MODEL=claude-opus-4-8
|
||||
|
||||
# OpenAI (LLM + embeddings)
|
||||
OPENAI_API_KEY=
|
||||
OPENAI_BASE_URL=https://api.openai.com/v1
|
||||
OPENAI_MODEL=gpt-4o
|
||||
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
|
||||
|
||||
# xAI / Grok — OpenAI-compatible (LLM)
|
||||
XAI_API_KEY=
|
||||
XAI_BASE_URL=https://api.x.ai/v1
|
||||
XAI_MODEL=grok-2-latest # set to your account's current Grok model
|
||||
|
||||
# Ollama — local, OpenAI-compatible, no key (LLM + embeddings)
|
||||
OLLAMA_ENABLED=false
|
||||
OLLAMA_BASE_URL=http://localhost:11434/v1
|
||||
OLLAMA_MODEL=llama3.1
|
||||
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
# Backup & restore
|
||||
|
||||
`backup.sh` produces a single bundle containing the Postgres database and the
|
||||
MinIO object store. Run it from this `deploy/` directory on the host that runs
|
||||
the stack.
|
||||
|
||||
## Back up
|
||||
|
||||
```bash
|
||||
./backup.sh
|
||||
# → backups/provenance-backup-20260609T140000Z.tar
|
||||
```
|
||||
|
||||
The bundle contains:
|
||||
|
||||
- `db.sql.gz` — `pg_dump --clean --if-exists` of the database, gzipped.
|
||||
- `minio-data.tar.gz` — the MinIO `/data` directory (objects + bucket metadata).
|
||||
- `MANIFEST.txt` — what's inside and when it was made.
|
||||
|
||||
Optional retention: `BACKUP_RETAIN_DAYS=30 ./backup.sh` also deletes bundles
|
||||
older than 30 days. Schedule it from cron for off-box copies, e.g.:
|
||||
|
||||
```cron
|
||||
15 3 * * * cd /path/to/provenance/deploy && BACKUP_RETAIN_DAYS=30 ./backup.sh
|
||||
```
|
||||
|
||||
(Copy the resulting bundle off the host — a backup on the same disk isn't one.)
|
||||
|
||||
## Restore
|
||||
|
||||
Restoring overwrites live data — stop the app first.
|
||||
|
||||
```bash
|
||||
ts=20260609T140000Z # the bundle you're restoring
|
||||
mkdir -p /tmp/restore && tar xf backups/provenance-backup-$ts.tar -C /tmp/restore
|
||||
|
||||
# 1. Database — the dump is --clean, so it drops & recreates objects.
|
||||
docker compose stop backend worker
|
||||
gunzip -c /tmp/restore/db.sql.gz \
|
||||
| docker compose exec -T postgres psql -U "${POSTGRES_USER:-provenance}" -d "${POSTGRES_DB:-provenance}"
|
||||
|
||||
# 2. Objects — replace the MinIO data directory.
|
||||
docker compose stop minio
|
||||
docker compose run --rm --no-deps -T -v provenance_miniodata:/data minio \
|
||||
sh -c 'rm -rf /data/* && tar xzf - -C /data' < /tmp/restore/minio-data.tar.gz
|
||||
docker compose up -d
|
||||
|
||||
rm -rf /tmp/restore
|
||||
```
|
||||
|
||||
Notes:
|
||||
- The MinIO `/data` archive is filesystem-level; restore into the **same** MinIO
|
||||
major version it was taken from.
|
||||
- Verify the volume name (`docker volume ls | grep miniodata`) — compose prefixes
|
||||
it with the project name; adjust the `-v` mount accordingly.
|
||||
Executable
+57
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# One-command backup of a Provenance deployment: the Postgres database and the
|
||||
# MinIO object store, into a single timestamped bundle under ./backups/.
|
||||
#
|
||||
# ./backup.sh # write backups/provenance-backup-<UTC>.tar
|
||||
# BACKUP_RETAIN_DAYS=30 ./backup.sh # also prune bundles older than 30 days
|
||||
#
|
||||
# Run it from the host where `docker compose` manages the stack (i.e. this
|
||||
# deploy/ directory). Restore steps are in BACKUP.md.
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "$0")" # the deploy/ directory (where docker-compose.yml lives)
|
||||
|
||||
# Config comes from the compose .env (same file the stack uses); fall back to
|
||||
# the compose defaults so a vanilla stack still backs up.
|
||||
if [ -f .env ]; then set -a; . ./.env; set +a; fi
|
||||
PGUSER="${POSTGRES_USER:-provenance}"
|
||||
PGDB="${POSTGRES_DB:-provenance}"
|
||||
|
||||
dc() { docker compose "$@"; }
|
||||
|
||||
ts="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
work="backups/.work-$ts"
|
||||
mkdir -p "$work" backups
|
||||
|
||||
cleanup() { rm -rf "$work"; }
|
||||
trap cleanup EXIT
|
||||
|
||||
echo "→ Dumping Postgres database '$PGDB'…"
|
||||
dc exec -T postgres pg_dump -U "$PGUSER" -d "$PGDB" --no-owner --clean --if-exists \
|
||||
| gzip > "$work/db.sql.gz"
|
||||
|
||||
echo "→ Archiving MinIO object store…"
|
||||
# Tar MinIO's data directory straight from the container (objects + bucket
|
||||
# metadata). Restored by extracting back into the miniodata volume.
|
||||
dc exec -T minio tar czf - -C /data . > "$work/minio-data.tar.gz"
|
||||
|
||||
cat > "$work/MANIFEST.txt" <<EOF
|
||||
Provenance backup
|
||||
created: $ts
|
||||
database: $PGDB (pg_dump --clean --if-exists, gzip)
|
||||
objects: MinIO /data (tar.gz)
|
||||
restore: see deploy/BACKUP.md
|
||||
EOF
|
||||
|
||||
bundle="backups/provenance-backup-$ts.tar"
|
||||
# Contents are already gzipped, so the outer archive is a plain tar.
|
||||
tar cf "$bundle" -C "$work" db.sql.gz minio-data.tar.gz MANIFEST.txt
|
||||
|
||||
echo "✓ Backup written: $bundle ($(du -h "$bundle" | cut -f1))"
|
||||
|
||||
if [ -n "${BACKUP_RETAIN_DAYS:-}" ]; then
|
||||
echo "→ Pruning bundles older than ${BACKUP_RETAIN_DAYS} days…"
|
||||
find backups -maxdepth 1 -name 'provenance-backup-*.tar' -type f \
|
||||
-mtime "+${BACKUP_RETAIN_DAYS}" -print -delete
|
||||
fi
|
||||
+50
-12
@@ -40,23 +40,59 @@ services:
|
||||
retries: 10
|
||||
restart: unless-stopped
|
||||
|
||||
# One-shot schema migration: runs `alembic upgrade head` and exits. Backend
|
||||
# and worker wait for it to finish, so on `docker compose up` the schema is
|
||||
# always current before the app serves traffic — no manual migrate step.
|
||||
# NOTE: a pure Watchtower image-swap recreates only the long-running
|
||||
# containers, not this one-shot job, so Watchtower deploys should be paired
|
||||
# with a `compose up` (see deploy docs) to re-run migrations.
|
||||
migrate:
|
||||
image: git.jpaul.io/justin/provenance-backend:${IMAGE_TAG:-test-main}
|
||||
command: ["uv", "run", "--no-dev", "alembic", "upgrade", "head"]
|
||||
labels:
|
||||
com.centurylinklabs.watchtower.enable: "true"
|
||||
# All app config comes from .env (twelve-factor) — no per-setting allow-list
|
||||
# to maintain. The `environment:` block below only pins values that must NOT
|
||||
# come from .env. See the backend service for the full rationale.
|
||||
env_file:
|
||||
- path: .env
|
||||
required: false
|
||||
environment:
|
||||
DATABASE_URL: ${DATABASE_URL:-postgresql+asyncpg://provenance:provenance@postgres:5432/provenance}
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
restart: "no"
|
||||
|
||||
backend:
|
||||
image: git.jpaul.io/justin/provenance-backend:${IMAGE_TAG:-test-main}
|
||||
labels:
|
||||
com.centurylinklabs.watchtower.enable: "true"
|
||||
# Twelve-factor: ALL application settings come straight from .env — owner,
|
||||
# AI providers, mailer/SMTP, S3, sessions, everything in app/core/config.py.
|
||||
# No per-setting allow-list to maintain, so a new setting in .env (and
|
||||
# .env.example) reaches the app with no compose edit. The `environment:`
|
||||
# block below is only for values that must NOT come from .env:
|
||||
# - RUN_MIGRATIONS: backend-only flag, not an app setting.
|
||||
# - DATABASE_URL: pinned to the compose-internal host as a safety net —
|
||||
# the code default points at localhost, which is wrong inside the
|
||||
# network. (.env normally sets it; this guards against it being absent.)
|
||||
# `environment:` wins over `env_file`, so these always take effect.
|
||||
# Trade-off (accepted): env_file also exposes infra secrets (POSTGRES_*,
|
||||
# MINIO_*, CLOUDFLARE_TUNNEL_TOKEN) to the app process; the app ignores them.
|
||||
env_file:
|
||||
- path: .env
|
||||
required: false
|
||||
environment:
|
||||
APP_ENV: ${APP_ENV:-development}
|
||||
RUN_MIGRATIONS: "1"
|
||||
DATABASE_URL: ${DATABASE_URL:-postgresql+asyncpg://provenance:provenance@postgres:5432/provenance}
|
||||
S3_ENDPOINT_URL: ${S3_ENDPOINT_URL:-http://minio:9000}
|
||||
S3_BUCKET: ${S3_BUCKET:-provenance}
|
||||
S3_ACCESS_KEY: ${S3_ACCESS_KEY:-provenance}
|
||||
S3_SECRET_KEY: ${S3_SECRET_KEY:-change-me-too}
|
||||
S3_REGION: ${S3_REGION:-us-east-1}
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
minio:
|
||||
condition: service_healthy
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
healthcheck:
|
||||
test:
|
||||
- CMD-SHELL
|
||||
@@ -76,19 +112,21 @@ services:
|
||||
command: ["uv", "run", "--no-dev", "python", "-m", "app.worker"]
|
||||
labels:
|
||||
com.centurylinklabs.watchtower.enable: "true"
|
||||
# Same .env-driven config as the backend (see its comment). The worker reads
|
||||
# the model-provider settings too, so the upcoming embedding/matching jobs
|
||||
# are configured the moment they land — no compose change needed.
|
||||
env_file:
|
||||
- path: .env
|
||||
required: false
|
||||
environment:
|
||||
APP_ENV: ${APP_ENV:-development}
|
||||
DATABASE_URL: ${DATABASE_URL:-postgresql+asyncpg://provenance:provenance@postgres:5432/provenance}
|
||||
S3_ENDPOINT_URL: ${S3_ENDPOINT_URL:-http://minio:9000}
|
||||
S3_BUCKET: ${S3_BUCKET:-provenance}
|
||||
S3_ACCESS_KEY: ${S3_ACCESS_KEY:-provenance}
|
||||
S3_SECRET_KEY: ${S3_SECRET_KEY:-change-me-too}
|
||||
S3_REGION: ${S3_REGION:-us-east-1}
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
minio:
|
||||
condition: service_healthy
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
restart: unless-stopped
|
||||
|
||||
frontend:
|
||||
|
||||
+17
-14
@@ -69,7 +69,7 @@ Layered, dependency pointing inward:
|
||||
- **Service layer** — all domain logic and the only place writes happen. Enforces invariants (e.g., "a write must carry an actor for the audit log"). The privacy engine is invoked here on every read.
|
||||
- **Repository layer** — data access over SQLAlchemy; no business rules.
|
||||
- **Domain models** — the entities in §5.
|
||||
- **Integrations** — adapters behind interfaces: `AuthProvider`, `ObjectStore`, `Mailer`, `ModelProvider`, `SourceConnector`, `Queue`. Swapping an implementation is a config change, not a code change.
|
||||
- **Integrations** — adapters behind interfaces: `AuthProvider`, `ObjectStore`, `Mailer`, `LLMProvider` / `EmbeddingProvider` (two separate model abstractions), `SourceConnector`, `Queue`. Swapping an implementation is a config change, not a code change.
|
||||
|
||||
Async throughout (FastAPI + async SQLAlchemy). Anything that can be slow or can fail externally (model calls, scraping, large imports) goes to the worker, never inline in a request.
|
||||
|
||||
@@ -87,8 +87,9 @@ Core entities and the important relationships. (Illustrative, not final DDL.)
|
||||
|
||||
### Tenancy & identity
|
||||
- **User** — a person with login. Auth method(s) are attached but identity is internal, so one user can link multiple providers.
|
||||
- **Tree** — the top-level tenant boundary for genealogical data. Owned by a User; may have additional members.
|
||||
- **TreeMembership** — (User, Tree, role) where role ∈ {owner, editor, viewer}. The basis for authorization.
|
||||
- **Tree** — the top-level tenant boundary for genealogical data. Owned by a User; may have additional members. Carries a per-tree **AI model policy** (owner-configured): `ai_member_provider` and `ai_recommender_provider` name configured providers from the model-provider registry (null = no model for that role); the owner may use any configured provider, while these cap what members and the recommender may use. Set via the owner-only `GET`/`PATCH /trees/{id}/ai`.
|
||||
- **TreeMembership** — (User, Tree, role) where role ∈ {owner, editor, viewer}. The basis for authorization *within a tree*.
|
||||
- **Instance owner / operator** — orthogonal to tree roles. The account(s) whose email is named in the `OWNER_EMAIL` env var **and whose email is verified** are the instance's operator(s), with access to the owner-only `/api/v1/admin` surface (operational status, instance-wide config). Derived from the env at request time — no DB column, no migration, can't drift, survives DB resets. The verified-email requirement is deliberate: registration is open, so without it whoever registers the owner address first would seize the role — verification ties ownership to proven control of the inbox. Crucially this is **not** a privacy bypass: an instance owner gets operational/config rights, **not** read access to other users' private trees or living-person PII — those still resolve only through the privacy engine. (`is_instance_owner` in `api/deps.py`.)
|
||||
|
||||
### Genealogical core
|
||||
- **Person** — belongs to a Tree. Has many **Name** records (with parts: given, surname, prefix/suffix, and a type such as birth/married/alias) to support variants and changes over time. Carries living/deceased status.
|
||||
@@ -108,7 +109,7 @@ Core entities and the important relationships. (Illustrative, not final DDL.)
|
||||
### Cross-cutting
|
||||
- **AuditEntry** — append-only: actor (User *or* the assistant principal acting for a User), action, entity, before/after snapshot, timestamp. Immutable.
|
||||
- **SoftDelete** — entities carry `deleted_at`; a scheduled worker purges rows older than 30 days. Recovery = clearing `deleted_at` within the window.
|
||||
- **ChangeProposal** — a pending set of writes generated by the assistant (or potentially a collaborator suggestion later): a structured diff the user approves, edits, or rejects. Approved proposals are applied through the normal service layer (so they hit the privacy engine and the audit log like any other write).
|
||||
- **ChangeProposal** — a pending set of writes: records an `origin` (`assistant` | `contributor` — collaborator suggestions are encoded today, not just a future idea), a `status` (pending/applied/rejected), a structured `operations` diff (JSONB list of `{op, entity_type, entity_id?, payload}`), a summary/rationale, and review/apply-error metadata. The user approves, edits, or rejects; approved proposals are applied through the normal service layer (so they hit the privacy engine and audit log like any other write). *Note: v1 apply is not cross-op transactional — see `docs/design/change-proposal.md`.*
|
||||
|
||||
## 6. Privacy engine
|
||||
|
||||
@@ -118,11 +119,12 @@ A single function conceptually:
|
||||
visible(viewer, entity) -> { full | redacted | hidden }
|
||||
```
|
||||
|
||||
Inputs: viewer's role on the entity's Tree (including "anonymous"), the Tree's visibility (public/unlisted/private), per-Person privacy override, and living-person status.
|
||||
Inputs: viewer's role on the entity's Tree (including "anonymous"), the Tree's visibility (public / site_members / unlisted / private), per-Person privacy override, and living-person status.
|
||||
|
||||
Rules:
|
||||
- **Tree private** → only members see anything.
|
||||
- **Tree public/unlisted** → non-members get a read view, *but* every Person is run through the living-person check and per-person override first.
|
||||
- **Tree site_members** → any authenticated account on this instance gets a read view (anonymous viewers get nothing), still per-person living/override filtered.
|
||||
- **Tree unlisted / public** → non-members *including anonymous viewers* get a read view, *but* every Person is run through the living-person check and per-person override first. Unlisted is gated only by knowing the link (never listed or search-indexed); public is listed in `/explore` and indexable.
|
||||
- **Living-person rule** — a Person with no death fact, whose birth is within a configurable recency window (default ~100 years; unknown birth treated as possibly-living), is redacted (name minimized, vitals/events/media hidden) for non-owners. Owners may override per Person.
|
||||
- The engine is invoked in the **service layer**, so it covers API, server-rendered public pages, search results, and any data the assistant can read. There is intentionally no path that returns rows without passing through it.
|
||||
|
||||
@@ -130,7 +132,7 @@ Rules:
|
||||
|
||||
Three parts, deliberately separated:
|
||||
|
||||
1. **Model provider abstraction** (`ModelProvider`) — one interface over hosted models (Anthropic, OpenAI, xAI) and self-hosted/local models via an OpenAI-compatible endpoint or Ollama. Configurable per deployment; keys supplied by the operator (this deployment) or by the user (BYO-key deployments).
|
||||
1. **Model provider abstraction** — two separate interfaces, `LLMProvider` and `EmbeddingProvider` (configured independently — e.g. Anthropic has no embeddings endpoint), over hosted models (Anthropic, OpenAI, xAI) and self-hosted/local models via an OpenAI-compatible endpoint or Ollama. An operator can configure **several providers at once** through a registry (`build_llm_providers()`/`configured_llm_providers()`), each selectable by name — the basis for the per-tree AI policy and the `default_llm_provider`/`default_embedding_provider` settings. Keys supplied by the operator (this deployment) or by the user (BYO-key deployments).
|
||||
2. **Scoped tool surface** — the assistant can only act through a constrained set of tools that map to service-layer operations, **scoped to the user it is helping.** It is its own principal: it cannot exceed that user's rights, and every action is attributed to "assistant (on behalf of User X)" in the audit log. This is the MCP-style boundary referenced in the PRD — the assistant gets capabilities, not raw database access.
|
||||
3. **Source connectors** (`SourceConnector`) — a plugin framework for *reading* external data: FamilySearch API, Find A Grave, WikiTree, BLM/GLO land patents, USGS maps, public-domain newspapers, public county records. Only legally permissible sources ship with the project; operators can add their own. Connectors are read-only and rate-limited, and run in the worker.
|
||||
|
||||
@@ -148,7 +150,8 @@ Three parts, deliberately separated:
|
||||
- `AuthProvider` interface with implementations for **local** (password + email verification/reset), **OIDC** (validated against Authentik; expected to work with Keycloak, Auth0, etc.), and **social** (Google, Apple, Facebook).
|
||||
- Operators enable any subset via config. This deployment will use Authentik (`auth.jpaul.io`) plus selected social providers; a bare self-hoster can run local-only.
|
||||
- Sessions are backend-issued; the assistant principal is minted per-session and scoped to the acting user.
|
||||
- *Status:* **local auth has landed** — Argon2id password hashing, opaque backend-issued sessions (only the token hash is stored; presented as a Bearer token or HttpOnly cookie), and email verification + password reset via the `Mailer` interface (console in dev, SMTP for operators). OIDC and social providers are Phase 5. Every write records an attributable actor in the audit log.
|
||||
- *Status:* **local auth has landed** — Argon2id password hashing, opaque backend-issued sessions (only the token hash is stored; presented as a Bearer token or HttpOnly cookie), and email verification + password reset via the `Mailer` interface (console in dev, SMTP for operators). An opt-in gate (`REQUIRE_EMAIL_VERIFICATION`, default off so SMTP-less self-hosts and pre-existing accounts aren't locked out) refuses sessions for accounts without a verified email — login is denied and existing sessions stop resolving until the address is verified. OIDC and social providers are Phase 5. Every write records an attributable actor in the audit log.
|
||||
- **Instance owner / operator** (orthogonal to the per-tree roles): the account(s) whose email is in `OWNER_EMAIL` *and* is verified are the instance operator(s), with the owner-only `/api/v1/admin` surface (operational status, instance-wide config). Derived from the env at request time — no DB column. It is an operator/config role, **not** a privacy bypass: it grants no read access to other users' private trees or living-person PII. (`is_instance_owner` in `api/deps.py`.)
|
||||
|
||||
## 10. Search
|
||||
|
||||
@@ -175,20 +178,20 @@ Jobs are idempotent and retryable; an external failure degrades gracefully rathe
|
||||
- Tag scheme: `test-main` (current main), `test-sha-<long>` (rollback pins), the component version, and `latest` on `v*` tags.
|
||||
- Servers **pull** new images to deploy — no build on the host. The deploy compose references `git.jpaul.io/justin/provenance-{backend,frontend}:${IMAGE_TAG:-test-main}`; `docker-compose.dev.yml` is a local-build override.
|
||||
- **Caddy** terminates TLS and reverse-proxies frontend + backend. **Cloudflare Tunnel** is the preferred ingress (no open inbound ports) but is never required; a plain Caddy-on-a-public-host deployment is equally supported.
|
||||
- **Configuration** is entirely environment-driven (twelve-factor). One `.env` plus the compose file is enough to stand up a deployment.
|
||||
- **Migrations** run on backend start (or via an explicit job) so an image pull + restart is a complete upgrade.
|
||||
- **Backups:** documented procedure for Postgres dump + object-store sync; restore is the inverse.
|
||||
- **Configuration** is entirely environment-driven (twelve-factor). One `.env` plus the compose file is enough to stand up a deployment; the backend/worker/migrate services read it via `env_file`, so every setting in `app/core/config.py` is configurable without a compose edit.
|
||||
- **Migrations** run on backend start (`RUN_MIGRATIONS=1`) and via a one-shot `migrate` compose service, so an image pull + restart is a complete upgrade. A **schema-drift guard** (defense in depth) makes a half-applied deploy loud rather than a silent storm of 500s: `/health/ready` returns 503 and startup logs a CRITICAL `SCHEMA DRIFT` line when the DB's `alembic_version` is behind the heads baked into the image (`app/core/schema_version.py`).
|
||||
- **Backups:** a one-command operator script (`deploy/backup.sh` — `pg_dump` + MinIO object sync, see `deploy/BACKUP.md`) plus a per-account ZIP export; restore is the inverse.
|
||||
|
||||
**Repository layout (as scaffolded):**
|
||||
|
||||
```
|
||||
/backend # FastAPI, uv-managed. app/{api/v1, services (+privacy), repositories, models, schemas, integrations (auth/mailer), core}; migrations/ = Alembic
|
||||
/deploy # docker-compose.yml, Caddyfile, .env.example
|
||||
/backend # FastAPI, uv-managed. app/{api/v1, services (+privacy), repositories, models, schemas, integrations (auth, mailer, objectstore, models = LLM/embedding providers), core}; migrations/ = Alembic
|
||||
/deploy # docker-compose.yml (+ docker-compose.dev.yml), Caddyfile, .env.example, backup.sh + BACKUP.md
|
||||
/.gitea/workflows # Gitea Actions: build images → Gitea registry
|
||||
/frontend # Next.js (App Router, TS, Tailwind). app/ pages, lib/api (openapi-typescript client), components/ui, Dockerfile (standalone)
|
||||
```
|
||||
|
||||
The compose stack runs `postgres` (pgvector image — includes `pgvector`; `pg_trgm` ships in contrib), `minio`, `backend`, and `caddy`. The **worker** container (same image as backend, worker mode) joins once queue-driven jobs exist. Phase 0 ships a minimal backend with `/health` (liveness) and `/health/ready` (Postgres reachability) to validate the deploy wiring before the data model lands.
|
||||
The compose stack runs `postgres` (pgvector image — includes `pgvector`; `pg_trgm` ships in contrib), `minio`, a one-shot `migrate` job, `backend`, the **worker** (same image as backend, worker mode — runs the scheduled soft-delete purge), `caddy`, and an optional `cloudflared` tunnel. The backend exposes `/health` (liveness) and `/health/ready` (Postgres reachability + schema-drift check).
|
||||
|
||||
## 13. Observability
|
||||
|
||||
|
||||
+430
@@ -0,0 +1,430 @@
|
||||
<!-- Generated by the genealogy-feature-gap-backlog workflow on 2026-06-09. -->
|
||||
<!-- Gap analysis vs commercial (Ancestry/MyHeritage/FamilySearch) and OSS
|
||||
(GRAMPS/Gramps Web/webtrees) genealogy software, verified against the
|
||||
codebase. Statuses reflect the repo at workflow launch (before the
|
||||
tree-visibility phases 1-3 landed; some items are now closed). -->
|
||||
|
||||
# Provenance — Product Backlog
|
||||
|
||||
> Status legend: **Have** (shipped) · **Partial** (substrate exists, surface incomplete) · **Planned** (on roadmap, no code) · **Missing** (no code, off roadmap).
|
||||
> Importance: Critical / High / Medium / Low. Effort: S / M / L / XL.
|
||||
> Phase references map onto the existing 0–9 roadmap. "NN#" = non-negotiable invariant.
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
**Where Provenance is strong today.** The foundation is genuinely solid and, in several places, ahead of the OSS field:
|
||||
|
||||
- **Sources-first spine is real.** A reusable `Source` + per-fact `Citation` two-tier model with a `exactly_one_target` CHECK constraint, confidence enum, and full backend CRUD. This is the architectural thing webtrees/Gramps get right and most commercial tools bury.
|
||||
- **Privacy architecture is the right shape — and coverage is now broad.** A single `privacy.py` engine, `TenantScoped` mixin on every row, living-person heuristic (`is_possibly_living`, unknown-birth-treated-as-living), and media served **through the backend rather than via raw S3 URLs**. Non-member reads of persons, events, media, names, and relationships all route through `person_visibility` (#46). The remaining gap is the `citation`/`source` list endpoints, which still gate only on `can_view_tree` — see §2.10.
|
||||
- **Non-destructive by design.** Soft-delete with timed purge worker, immutable `AuditEntry` (before/after JSONB, `actor_type` ready for the assistant), GEDCOM merge that copies rather than overwrites, full account export/import.
|
||||
- **Modeling maturity.** Typed parent/child qualifiers (biological/adoptive/step/foster/donor/guardian), typed alternate names with one-primary invariant, dual verbatim+normalized dates, duplicate-relationship guards, UUID surrogate keys.
|
||||
- **Standards core.** GEDCOM 5.5.1 import/export is **functional** (with preview/merge-vs-create resolution UI), pg_trgm fuzzy name search, multi-tenant tree hosting with visibility tiers. Round-trip *fidelity* has three tracked gaps (custom tags, PLAC coords/hierarchy, non-UTF-8 encoding) — see §2.11.
|
||||
|
||||
**Documentation-vs-code gaps to correct now (per "docs travel with code").** Two repo claims are not yet true and should be edited in the same spirit they were written:
|
||||
|
||||
- **pgvector is claimed as used; it is not.** Only `pg_trgm` is created. ARCHITECTURE references pgvector for match ranking.
|
||||
- **i18n "from day one" is documented but unmet.** PRD §6 promises externalized strings; every label is a hardcoded literal.
|
||||
|
||||
These two doc edits are themselves trivial quick wins (see §3).
|
||||
|
||||
**The biggest gaps vs commercial (Ancestry / MyHeritage / FamilySearch).** Provenance is not trying to be a record provider, and correctly so — but it is missing several things mainstream users treat as table stakes:
|
||||
|
||||
- **No record hints, no "save to tree," no connector framework.** The entire SourceConnector layer (FamilySearch/Find A Grave/WikiTree) is unbuilt — this gates AI search, hints, and auto-citation.
|
||||
- **No person merge outside GEDCOM import.** Merging duplicate people is fundamental hygiene and is currently impossible in-tree — the single highest-value near-term matching gap.
|
||||
- **No maps at all.** No place autocomplete, no geocoding, no interactive/migration/birthplace maps — a glaring hole for an app whose thesis is *family **and** land*.
|
||||
- **No report/print/PDF output.** Charts render on-screen only; there is no Ahnentafel, family group sheet, narrative report, or any PDF/SVG/HTML export. The whole "Charts, reports & printing" category is on-screen-viewing only.
|
||||
- **DNA absent** (deliberately parked — treat as open question, not a gap).
|
||||
|
||||
**The biggest gaps vs OSS (GRAMPS / Gramps Web / webtrees).** These are where a privacy-first self-host product is expected to compete and currently trails:
|
||||
|
||||
- **Collaboration management is now reachable, but minimal.** `TreeMembership` roles are enforced on every read/write, and a list/add/change-role/remove API + UI now ship (§2.9), satisfying the full-CRUD invariant (NN#8). The remaining gap is the richer **email invite/grant flow** (pending-invite state, resend/expire), still scheduled for Phase 6.
|
||||
- **Living-person redaction is now near-uniform.** Non-member reads of persons, events, media, names, and relationships all redact possibly-living people (#46); the `citation`/`source` list endpoints are the remaining hold-outs (they gate only on `can_view_tree`) — a narrowed PII gap on public/unlisted trees (NN#3, NN#2).
|
||||
- **No place as a usable first-class entity** (model exists, created by GEDCOM, but no read/edit/delete — a create-only entity, which is a bug per NN#8).
|
||||
- **No research log, to-do/task planner, kinship calculator, data-quality checker, or i18n/string externalization** (the last is a documented day-one commitment that is currently unmet).
|
||||
|
||||
**Security-priority correctness fixes (do these first, regardless of phase).** The redaction defects all shipped — child resources (#46) and now citations/sources too — leaving one config switch:
|
||||
|
||||
1. **Self-registration approval-mode switch (§2.10)** — the read-side enforcement now exists: `REQUIRE_EMAIL_VERIFICATION` gates login/session on `email_verified_at` (#53). The remaining gap is the env switch to choose open vs admin-approval vs closed self-registration. *(The citation/source living-person leak is now closed — citation/source list endpoints apply `person_visibility` for non-members via `public_view_service`.)*
|
||||
|
||||
**Strategic posture.** The differentiators worth pressing — property chain-of-title, the ChangeProposal AI model, the anonymous mutual-consent hint system, and true self-host data ownership — are mostly still ahead on the roadmap. The near-term job is (a) close the **privacy/auth correctness** and **collaboration** gaps that the architecture already implies, (b) ship the **maps + reports + merge** table stakes, and (c) finish the back-half spine — the **connector framework** plus wiring the now-landed **ChangeProposal/ModelProvider** into the assistant — that unlocks the entire back half of the roadmap.
|
||||
|
||||
---
|
||||
|
||||
## 2. Backlog by category
|
||||
|
||||
### 2.1 Tree & data model
|
||||
|
||||
Core CRUD, typed relationships, dates, soft-delete, and naming are **have**. Remaining work is about reusable sub-entities, shared/event-centric modeling, and research-grade conveniences.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Repository as first-class entity | Promote `Source.repository` string to a reusable `Repository` (name/address/call-numbers) with dedup. | Partial | Med | M | 1–2 | If promoted, full CRUD in API+UI (NN#8) — don't half-build. |
|
||||
| Note as first-class entity (SNOTE) | Promote inline `notes` text fields to reusable shared `Note`/SNOTE records. | Missing | Low | M | 2 | Full CRUD; GEDCOM 7 round-trip parity. |
|
||||
| Shared/event-centric model + witnesses | Remove the `subject_person_xor_relationship` XOR; add participant/role join so one event has many people (FAN/cluster research). | Missing | Med | M | later | Unlocks FAN club + richer sourcing; participants must redact via privacy engine. |
|
||||
| Non-family associations (FAN) | Add associate/neighbor relationship types; best delivered with shared-event participants. | Missing | Low | M | later | — |
|
||||
| Relationship-status enum | Add married/divorced/annulled status on partnership rather than inferring from events. | Partial | High | M | 1–2 | — |
|
||||
| Family/couple unit (GEDCOM FAM) | Persist a true FAM entity (own ID/sources, childless couples) instead of rebuilding on export. | Partial | High | L | 2 | Improves GEDCOM fidelity. |
|
||||
| Kinship / relationship calculator | "How is A related to B" path + cousinship. Graph edges already exist. | Missing | High | M | 1–2 | Self-contained; reads via privacy engine. |
|
||||
| **Read-only audit-log viewer / activity feed** | Surface `AuditEntry` as a per-tree/per-person change feed. Smaller and higher-leverage than value-level undo; partially satisfies NN#8's "read" for AuditEntry and is the substrate for watch/follow + webhooks. | Missing | High | M | 2 | Privacy-filtered projections only — never raw before/after JSON to non-members (NN#2/#3). |
|
||||
| Per-field revision history + restore-prior-value | Value-level history view + undo, built atop the audit feed above. | Partial | High | L | 6 | Audit-log *UI* is the feed item; this is the larger value-level-undo work (NN#8 correction ethos). |
|
||||
| Color-coded tags & custom labels | Tag people for lineages/research-status/grouping. | Missing | Med | M | 2 | Full CRUD; tenant-scoped. |
|
||||
| Person timeline / LifeStory | Sort the merged event list; add place/age enrichment + narrative presentation. | Partial | Med | M | 2 | Sort is trivial (`localeCompare` on `date_start`); narrative is the larger piece. |
|
||||
| Multi-calendar normalization | Store + parse Julian/Hebrew/French Republican (only `calendar` tag stored today, only Gregorian normalized). | Partial | Low | M | 2 | See also Localization §2.17. |
|
||||
| Evidence/persona vs conclusion model | GEDCOM-X persona layer separate from conclusion person. | Missing | Med | XL | later | Large modeling change; strengthens sourcing + hint matching. |
|
||||
| Negative assertions | Boolean "event did not happen" on Event. | Missing | Low | S | 2 | Cheap interop nicety. |
|
||||
| Custom groups / networks | Named manual or rules-based groupings. | Missing | Low | M | later | Lower priority than tags. |
|
||||
| Raw GEDCOM record editor / configurable fact tabs | webtrees-style raw editor + fact-type registry. | Partial | Med | L | later | Open vocabularies give de-facto custom facts today. |
|
||||
| Health/medical, historical-facts index, LDS ordinances | Niche entities. | Missing | Low | M–L | later | LDS BAPL/ENDL/SLGS should map to distinct types if ever pursued; medical is special-category PII. |
|
||||
|
||||
---
|
||||
|
||||
### 2.2 Sources & citations
|
||||
|
||||
The two-tier model is **have** and production-grade on the backend. The gaps are almost all UI/CRUD-completeness and the connector-dependent "save to tree" flows.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Citation confidence selector in UI | Confidence enum is modeled + API-writable but the `citeControl` form never sets it — every UI citation is NULL confidence. | Partial | High | S | 1 | **Quick win.** Full CRUD in UI (NN#8); reinforces evidence-quality thesis. |
|
||||
| Source edit UI + all 8 fields | Source UI is add/list/delete only and create exposes ~3 of 8 fields (no author/source_type/publication_info/quality_note/citation_text). | Partial | High | S | 1 | Update API exists but no edit form — violates NN#8. |
|
||||
| `GET /{tree}/citations/{id}` | Citation API has list but no single-read endpoint. | Partial | Med | S | 1 | API symmetry (NN#8). |
|
||||
| Transcription / abstract / extract fields | Add `transcription_text` + `abstract_text` to Source; don't conflate with `citation_text` (GEDCOM SOUR.TEXT currently dumped into citation_text). | Missing | Med | S | 1–2 | **Quick win.** Central to evidence analysis; full CRUD (NN#8). |
|
||||
| Evidence-Explained guided citation builder | Structured fields → formatted citation (Chicago/MLA/APA) instead of hand-typed `citation_text`. | Missing | High | L | 2 | Signature provenance feature; citation_text should be generated, not typed. |
|
||||
| Citations on OwnershipEvents | Add `ownership_event_id` to Citation + extend CHECK to 5 targets when property lands. | Partial | Critical | S | 3 | **Quick win once Property exists** — single FK + constraint edit (NN#5). |
|
||||
| Record-to-source attachment ("save to tree") | Search a connector record and attach its facts. | Missing | High | XL | 4 | Gated on connector framework; assistant attach must emit ChangeProposal (NN#1); legal sources only (NN#6). |
|
||||
| Source Linker (one record → many persons) | Bulk-attach a record's facts across people. | Missing | Med | L | 4 | Downstream of connectors; reads/writes via service layer. |
|
||||
| Auto-citation on save/match | Generate citation when a hint/record is confirmed. | Missing | Med | L | 4/7 | Blocked on connectors + hints; ChangeProposal if assistant-driven. |
|
||||
| Memories-as-sources (cite a photo directly) | Allow media to be a citation target, not only attachable to a Source. | Partial | Low | M | 2 | Reads stay on privacy-checked media endpoint (NN#2). |
|
||||
| GPS / Proof-Standard reasoning artifact | Container linking sources/citations into a proof narrative reconciling conflicts. | Missing | Med | L | later | Serious-researcher differentiator; full CRUD (NN#8). |
|
||||
| Proprietary record collections | 1921 census, UK sets, etc. | Missing | Low | XL | — | **Out of scope** — conflicts with NN#6 / self-host. Do not pursue. |
|
||||
|
||||
---
|
||||
|
||||
### 2.3 Search & matching
|
||||
|
||||
Fuzzy trigram name search is **have**; everything that depends on connectors, embeddings, or multiple populated trees is planned/missing. The standout near-term gap is **in-tree person merge**.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Standalone duplicate detection | Lift the GEDCOM `_best_match` logic into a "find duplicates in my tree" scan. | Partial | High | M | 2 | Logic already written; results via privacy engine (NN#2). |
|
||||
| Interactive two-person merge (side-by-side, field-select, undo) | General merge of duplicate persons with citation re-pointing — impossible outside import today. | Partial | High | L | 2 | **Highest-value matching gap.** Preserve + re-point Citations (NN#5); write-once is a bug (NN#8). |
|
||||
| Advanced search (wildcards, boolean, date/place facets, sort) | Search exposes only `?q`. | Partial | High | M | 2 | Keep per-person privacy filter in the search loop (NN#2). |
|
||||
| Phonetic matching (Soundex/Metaphone/DM) | Enable `fuzzystrmatch`; trigram is char-similarity, not phonetic. | Partial | High | M | 2 | Pure utility. |
|
||||
| Semantic / vector search (pgvector) | **Docs claim pgvector is used; it is not** — only pg_trgm extension is created. Add `CREATE EXTENSION vector` + embedding columns (and correct the docs). | Missing | Med | L | 7 | Embedding provider is an open question (PRD §11) — don't pick silently; candidates via privacy engine. |
|
||||
| Tree-to-tree matching (Smart Matches) | Cross-tree candidate generation + ranking. | Planned | High | XL | 7 | Anonymous until mutual consent (NN#4); living-person protection (NN#3). |
|
||||
| Mutual-consent match notification | Anonymous notification, reveal only after both opt in. | Planned | High | L | 7 | **Mandated invariant**, not a toggle (NN#4, NN#3); rides the notification substrate (§2.9). |
|
||||
| Match confirm/reject + "not a match" memory | Persistent rejected-match store (today scoring lives only inside import). | Partial | High | M | 7 | Prevents re-notifying once hints land. |
|
||||
| External search deep-links | Pre-fill FamilySearch/Find A Grave/BLM-GLO search URLs from a person's name/dates/place. | Missing | Med | M | 2–4 | **High value, low risk** before full connectors; legal targets only (NN#6). |
|
||||
| **Automated record hints** | Proactive per-person record suggestions from connectors — a marquee mainstream feature. | Missing | High | XL | 7 | Connector-gated (NN#6); surfaced anonymously where cross-tree (NN#4); attach via ChangeProposal (NN#1). |
|
||||
| Jurisdiction-aware record-search hints | Map place/jurisdiction → relevant collections. Place hierarchy is a ready foundation. | Missing | Med | L | 8 | Suggested collections must be legal (NN#6). |
|
||||
| Cross-language / transliteration matching | Cyrillic/Hebrew/CJK ↔ Latin. | Missing | Med | XL | later | See Localization. |
|
||||
| Record Detective, newspaper matches, collection catalog, GQL query builder, OCR full-text search | Connector/record-layer dependent. | Missing/Planned | Low–Med | L–XL | 4/7/8 | All gated on the connector framework; any query path runs through privacy engine (NN#2). |
|
||||
|
||||
---
|
||||
|
||||
### 2.4 Media & documents
|
||||
|
||||
Universal media attachment is **have**; the earlier privacy leak is now **closed** (#46), and the remaining gaps are the asset-processing pipeline (EXIF strip, thumbnails).
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **Media privacy gating on serve paths** | `list_media`/`get_media`/`media_content` now apply `person_visibility` for non-members (#46): media is exposed only when linked to a FULL-visibility person (`list_public_media`/`can_view_media`), so living-person photos no longer leak on public/unlisted trees. | Have | **Critical** | M | 1 | **Resolved (NN#3/NN#2).** Serve paths check attached `person_id` visibility and 404 otherwise. |
|
||||
| EXIF / GPS stripping on upload | Raw bytes stored verbatim; family photos leak GPS/home addresses/timestamps. | Planned | High | M | 1 | **Security-priority**, not cosmetic. Parse EXIF on ingest, strip/quarantine by default, allow override. |
|
||||
| Thumbnail / preview generation | No image pipeline (no Pillow). Async, idempotent worker job. | Planned | High | L | 1 | Derived thumbnail must inherit parent privacy — no bypass path. |
|
||||
| Image reference regions | Mark the rectangle of a census image that supports a Citation. | Missing | Med | M | later | Tenant-scoped, full CRUD; region→Citation preferred over region→Person. |
|
||||
| Photo/face tagging (manual) | Multi-person tagging via single FK today. | Missing | Med | XL(ML)/M(manual) | later | Owner-only, in-deployment; face tags inherit redaction (NN#3); full CRUD. |
|
||||
| Mobile photo scanning + auto-split | Shoebox digitization. | Missing | Med | L | later | Reuse privacy-gated upload + EXIF strip. |
|
||||
| AI photo dating / colorize / restore / animate / narrate | Model-driven media features. | Missing | Low | L–XL | 4+ | Must route through ModelProvider (NN#7), require approval (NN#1), preserve original; animating living faces raises consent issues. |
|
||||
| British Library / paywalled archives, pay-per-view credits | Licensed content + metering. | Missing | Low | XL | — | **Out of scope** — conflicts with NN#6 and the self-host model. |
|
||||
|
||||
---
|
||||
|
||||
### 2.5 DNA & genetic genealogy
|
||||
|
||||
DNA is an **explicit PRD non-goal / open question** — treat as parked, not a backlog to grind through. Across every DNA row the rule is uniform: **a user uploading their own export is permissible; vendor connectors/scrapers (23andMe / Ancestry / MyHeritage / GEDmatch) are barred (NN#6).** Kits and matches are living-tester PII and route through the privacy engine.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| DNA-confirmed relationship flag | Model DNA confirmation as a Source/Citation backing a Relationship (not free text). | Missing | Med | M | parked | Best sources-first fit (NN#5); full CRUD (NN#8). |
|
||||
| Raw DNA upload (own file) | User uploads own export; no vendor scraping. | Missing | Med | L | parked | User's own file is fine; vendor connectors barred (NN#6); special-category PII via privacy engine. |
|
||||
| Kit/Match entities linked to persons | Kit (tester) + Match tied to Person, tenant-scoped/audited. | Missing | Med | M | parked | Kits = living-tester PII (NN#2/#3); full CRUD (NN#8). |
|
||||
| Autosomal match list, segments, chromosome browser, triangulation, ThruLines/AutoTree, ethnicity/admixture, haplogroups, GEDmatch, NPE detection | Full genetic-genealogy suite. | Missing | Low–Med | L–XL | parked | DNA scope is an unresolved open question — **surface the dependency, don't build speculatively.** Own-data only (NN#6); cross-user surfacing obeys NN#4. |
|
||||
|
||||
---
|
||||
|
||||
### 2.6 Maps, places & gazetteers
|
||||
|
||||
This category is almost entirely **missing** despite being half the product thesis. The Place model has the right bones (parent_id, lat/long, PlaceName with date ranges) but no API/UI and no maps.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **Place as usable first-class entity** | Place rows are created by GEDCOM but have **no read/edit/delete** API or UI — a create-only entity. | Partial | High | M | 2–3 | **Violates NN#8** (create-but-not-edit = bug). Make Place citable too (NN#5). |
|
||||
| Place autocomplete + picker in event editor | No `/places` router; the event form has no place input, so users can't attach a place at all. | Missing | High | M | 2 | Table stakes; lookup is low-risk. |
|
||||
| Geocoding (manual coords + forward) | lat/long columns exist; no UI, no geocoder. | Partial | High | M | 3 | Provider via env (NN#7), ToS-compliant (NN#6). |
|
||||
| Pluggable geocoding provider | Nominatim/GeoNames/Bing/Google swappable. | Missing | Med | L | 3 | Provider+keys via env (NN#7); legal providers only (NN#6). |
|
||||
| Bulk/batch geocoding (worker) | Geocode hundreds of GEDCOM-imported places. | Missing | Med | M | 3 | Idempotent, rate-limited worker job; provider via env. |
|
||||
| Place merge/split (dedup) | GEDCOM imports produce near-duplicate place strings. | Missing | High | M | 2–3 | Needs Place update/delete (NN#8); audited merges. |
|
||||
| Place-name cleanup tools | Extend the existing preview→apply cleanup UX to places. | Missing | Med | M | 2 | Preview-first + audited like existing cleanup. |
|
||||
| Standardized-name vs original text | Mirror the verbatim+normalized date pattern for places. | Missing | Med | M | 2–3 | GEDCOM fidelity. |
|
||||
| Alternate/historical place names with date ranges | `PlaceName` model exists with valid_from/to but no CRUD and never populated. | Partial | Med | M | 2–3 | Stored entity with no CRUD surface (NN#8). |
|
||||
| Interactive map of events & places | No map library in frontend. Core to family+land positioning. | Missing | High | L | 3 | Plot via `person_visibility` so non-owners never see living locations (NN#2/#3). |
|
||||
| Migration trail / pedigree-birthplace maps | Per-person life path; ancestor birthplace map. | Missing | Med | L | 3 | Redact living subjects for non-owners (NN#3). |
|
||||
| Bundled world gazetteer | Offline GeoNames-style authority. | Missing | Med | XL | later | GeoNames (CC-BY) verify AGPL-compat; env-configurable. |
|
||||
| Historical boundary overlays, time slider, heatmaps, radius/nearby, tile-provider switch | Advanced geo. | Missing | Low–Med | S–XL | later | PostGIS is an open question (ARCH §14) — **surface dependency**, don't adopt silently; tiles legal (NN#6). |
|
||||
|
||||
---
|
||||
|
||||
### 2.7 Charts, reports & printing
|
||||
|
||||
On-screen pedigree/descendant/fan/hourglass charts are **have**. The entire **output/print/report** half is missing — this is the linchpin gap of the category.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **Multi-format export (PDF / SVG / image / HTML)** | No export/print path, no `@media print`, no `window.print()`. Charts and reports can't leave the screen. | Missing | High | L | 2/6 | **Linchpin.** Generate from privacy-filtered data so living people redacted in shared output (NN#3). |
|
||||
| Ahnentafel report | Numbered-ancestor report; all data exists. | Missing | High | M | 6 | — |
|
||||
| Family group sheet / individual summary | Printable summary; data available, needs print layout. | Missing | High | M | 6 | — |
|
||||
| Narrative descendant/ancestor reports | Multi-standard prose with inline sources. | Missing | High | L | 6 | Cite Sources inline (NN#5); redact living (NN#3). |
|
||||
| Sentence-template narrative engine | Deterministic fact→prose underpinning reports. | Missing | Med | L | 6 | Keep template-based; report text never mutates tree (NN#1). |
|
||||
| Photo boxes in charts | Pass privacy-checked media URLs to `setCardDisplay`; CSS already present. | Missing | High | M | 2 | Stream via privacy-checked /media (NN#2/#3). |
|
||||
| Drag-to-edit / interactive chart canvas | Tree canvas renders but interactive node editing (drag to re-parent, inline edit on the chart) is only partly present. | Partial | Med | M | 2 | Edits go through service layer + audit (NN#1); honor redaction. |
|
||||
| Statistics dashboard | Surname/place/date distributions + tree-health. | Missing | Med | M | 6 | Reads via privacy engine (NN#2). |
|
||||
| Kinship/relationship diagram report | Needs path-finding (see §2.3 calculator) + renderer. | Missing | Med | M | 6 | — |
|
||||
| List reports (sources/places/repos/media) | Printable indexes (current screens are management, not reports). | Missing | Med | M | 6 | — |
|
||||
| Color-by-lineage, fan overlays, lifespan/timeline charts | Sex-coloring exists; lineage/overlay/timeline don't. | Partial/Missing | Med | M | later | Overlays respect privacy engine. |
|
||||
| Book/multi-report compiler, wall-chart tiling, page-setup, customizable charts | Print-shop-grade output. | Missing | Low–Med | L–XL | later | Saved "book" entity = full CRUD (NN#8); honor living-person privacy. |
|
||||
| Bowtie/couple-rooted/circular-sun/3D/network/calendar | Niche chart variants. | Missing/Partial | Low–Med | M–L | later | — |
|
||||
| Print-shop products, XML template engine, blank forms | Commercial/template extras. | Missing | Low | S–XL | later | Weak fit for self-host. |
|
||||
|
||||
---
|
||||
|
||||
### 2.8 Research workflow & automation
|
||||
|
||||
The preview→approve **bulk cleanup** tool is a genuine **have** and a differentiator. The missing pieces are the serious-researcher workflow entities.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Data-quality / consistency checker | Extend cleanup beyond name issues: child-before-parent, death-before-birth, implausible ages, orphans, dups; severity tiers. | Partial | High | L | 2 | New auto-fixes keep preview→apply (NN#1). |
|
||||
| Research log | Searches, repositories visited, negative results, findings — distinct from the system audit log. | Missing | High | M | 6 | Reference reusable Sources (NN#5); tenant-scoped full CRUD (NN#8). |
|
||||
| To-do / research task planner | Tasks on Person/Tree with status/priority/due/assignment. | Missing | High | M | 6 | Full CRUD in API+UI (NN#8). |
|
||||
| Source-driven data entry | Start from a Source document and transcribe facts into the tree. | Missing | High | M | 2 | Natural sources-first differentiator (NN#5). |
|
||||
| Task↔log linkage | FK + joined view once both entities exist. | Missing | Med | S | 6 | Cheap once predecessors land. |
|
||||
| Family chronology / timeline | Sort merged events; family-wide chronology (parents' marriage, siblings' births). | Partial | Med | M | 2 | Sort is trivial; presentation over privacy-filtered data. |
|
||||
| Navigation: active person / history / bookmarks | Large trees rely on browser back only. | Missing | Med | M | 2 | Per-user, tenant-scoped, full CRUD; don't expose redacted persons (NN#2/#3). |
|
||||
| Saved-record shoebox / review queue | Stage candidate records before committing. | Missing | Med | M | 4/7 | Auto-attach via ChangeProposal (NN#1); legal sources (NN#6). |
|
||||
| Guided research suggestions | Proactive "research next" engine (today only flags problems). | Partial | High | L | 4 | Advisory; writes via ChangeProposal (NN#1); cross-tree via privacy engine (NN#2). |
|
||||
| Persona-adaptive onboarding | Family Keeper / Serious Researcher / Property Researcher selector (PRD US-002, documented but unbuilt). | Missing | Low–Med | L | 2 | Pure presentation. |
|
||||
| Dashboard widgets, scratchpad, research-link sidebar, blog/narrative authoring, research wiki, crowd indexing | Conveniences. | Missing | Low | S–XL | later | Widgets/published narratives read via privacy engine (NN#2/#3). |
|
||||
|
||||
---
|
||||
|
||||
### 2.9 Collaboration & sharing
|
||||
|
||||
Authorization is enforced everywhere, and a **minimal management surface now ships** — list/add/change-role/remove via `api/v1/members.py` plus a members page (#233). The remaining gap is the richer email invite/grant flow. The minimal slice landed at Phase 2 as planned; the invite/email UX stays at Phase 6.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **Membership PATCH/DELETE + role change (minimal slice)** | Add/adjust/revoke a collaborator and change `role` — GET/PATCH/DELETE on `/trees/{id}/members` (`api/v1/members.py`) plus a frontend members page now ship (#233). Resolves the create-only NN#8 break without the full invite flow. | Have | **Critical** | S–M | 2 | Resolves the create-only NN#8 break. Revocation routes through the single privacy point. |
|
||||
| Full invite/grant flow (email + UI) | Email-based invitations, pending-invite state, role-grant UI, resend/expire. Builds on the minimal slice. | Partial | High | L | 6 | Invitation email via configured SMTP (NN#7); membership changes through the one enforcement point. |
|
||||
| **Read-only public tree share** | Anonymous read surface shipped: optional-auth `CurrentUserOrNone` dep, `api/v1/public.py` + `public_view_service.py`, and server-rendered pages at `/p/[treeId]` (+ `/persons/[personId]`) and `/explore`. Living-safe by construction via `person_visibility`. | Have | High | M | 2 | Highest-leverage near-term sharing feature; living-safe by construction via `person_visibility` (NN#2/#3). |
|
||||
| SEO public profile pages (server-rendered) | Server-rendered public pages (`/p/[treeId]`, `/explore`) and `robots.ts` now ship. Deferred follow-ups: a public-only `sitemap.ts` and per-tree `noindex,nofollow` meta for `unlisted`/`site_members` pages. | Partial | Med | L | 2 | NN#2 explicitly names server-rendered public pages — must go through privacy engine, no direct row queries. |
|
||||
| **Notification / event-dispatch substrate** | Shared enabler seeded from `AuditEntry`: subscription + dispatch layer emitting privacy-filtered projections. Underpins watch/follow, mutual-consent match notices, comments, moderation, and in-app messaging. | Missing | High | L | 6 | **Privacy-filtered projections only — never raw before/after JSON** (NN#2/#3). |
|
||||
| Comments / discussion threads | Per-profile discussion (target = person/event/source), threaded. | Missing | High | M | 6 | Comments on living persons redacted for non-members (NN#2/#3); rides the dispatch substrate. |
|
||||
| In-app messaging (contact details hidden) | SMTP exists; no Message/Thread model. | Planned | High | L | 6 | Hide contact details; opens after mutual consent (NN#4); redact living-person content; rides dispatch substrate. |
|
||||
| Watch/follow + change notifications | `AuditEntry` is the natural event source; needs subscription entity + dispatch (substrate above). | Planned | Med | M | 6 | Notification builder reads via privacy engine, not raw rows. |
|
||||
| **Optimistic concurrency / lost-update protection** | No version/etag/`updated_at` precondition checks; concurrent multi-user edits can silently clobber. | Missing | High | M | 6 | Full-CRUD + multi-user without this risks lost updates; concurrent paths still route through privacy engine. |
|
||||
| Pending-changes moderation (human edits) | Queue contributor edits for owner approval — shares infra with the AI ChangeProposal queue. | Missing | Med | L | 6 | **Design together with ChangeProposal** (NN#1). |
|
||||
| Field-by-field profile merge & approval | WikiTree-style merge center + unmerge with per-field provenance. | Missing | Med | XL | later | Conflicting facts each retain Source/Citation (NN#5). |
|
||||
| Ownership transfer | `owner_id` is effectively write-once; needed for self-host longevity. A minimal reassignment endpoint is the NN#8 fix. | Missing | Med | M | 6 | **Violates write-once invariant** (NN#8) — importance/phase tension noted; ship the minimal slice when membership lands. |
|
||||
| Narrative website / HTML export | Static narrated site (reuse public-page renderer). | Missing | Med | L | later | Redact living persons at build time (static bypasses runtime engine) (NN#3). |
|
||||
| Two-way desktop↔online sync | Bidirectional sync with change journals. Audit log could seed a change feed. | Missing | Med | XL | later | No Ancestry TreeShare / paywalled sync (NN#6). |
|
||||
| Curator roles, trusted-list ACLs, field locking, projects/workspaces, forum, honor code, free-space wiki, portal homepage | Community-platform features. | Missing | Low | S–XL | later | New roles/ACLs/locks integrate with the **single** enforcement point, not parallel checks. |
|
||||
| Real-time co-editing | Out of scope; only optimistic concurrency planned. | Planned | Med | XL | later | Concurrent paths must route through privacy engine. |
|
||||
|
||||
---
|
||||
|
||||
### 2.10 Privacy & access control
|
||||
|
||||
The architecture is correct (single engine, tenant mixin, audit, soft-delete + purge are **have**), but enforcement coverage and configurability have real holes — two of which are security-priority.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **Uniform living-person redaction across child resources** | `person_visibility` now runs for non-members on the event, media, name, relationship endpoints (#46) and the citation/source list endpoints, all delegating to `public_view_service`: citations resolve to FULL-visibility person(s); sources show only when they back a visible citation. | Have | High | S | 1–2 | **Resolved (NN#3/NN#2).** No child-resource path leaks a redacted living person's facts. |
|
||||
| **Email-verification enforcement gate** | Read-side check now ships (#53): `REQUIRE_EMAIL_VERIFICATION` gates login/session on `email_verified_at` (`auth_service.py`). Opt-in (default off) so SMTP-less self-hosts still work. | Have | **High** | S | 1–2 | Read-side trust path now enforced (NN#7); the registration-mode switch below is the separate larger piece. |
|
||||
| Self-registration mode gating (approve / open / closed) | No env switch to choose open vs admin-approval vs closed registration. | Partial | High | M | 2/5 | Twelve-factor registration control (NN#7); pairs with the verification gate above. |
|
||||
| Instance owner / operator role | `OWNER_EMAIL`-declared operator (#240): `is_instance_owner` on `/users/me`, owner-only `GET /api/v1/admin/instance`, `/admin` UI. | Have | Med | S | 2/5 | Owner-only operational surface, twelve-factor via env (NN#7); reads stay through the service layer. |
|
||||
| **Fix `site_members` visibility tier** | `can_view_tree` now handles `site_members` (`privacy.py:56`): any authenticated account gets a read view, anonymous is refused. | Have | Critical | S | 1 | Honors the tier the UI offers; reads still route through `person_visibility`. |
|
||||
| Make `LIVING_RECENCY_YEARS` configurable | Hardcoded 100 at `privacy.py:23`. | Partial | High | S | 2 | **Quick win.** Twelve-factor (NN#7). |
|
||||
| Privacy-stripped export (redact living) | GEDCOM + account export emit full tree; no "strip living" mode. | Missing | High | M | 2 | Reuse `person_visibility`/`_redact` (NN#3). Owner self-export is safe today; shareable variant is the gap. |
|
||||
| Per-fact / per-field privacy + record flags | tentative/rejected/preferred/private flags on facts. | Missing | Med | L | later | If added, route through the single engine (NN#2). |
|
||||
| Granular rules by record type & viewer relationship | webtrees-style "hide marriages from non-descendants". | Missing | Med | L | later | Single enforcement point. |
|
||||
| OIDC / external IdP login | `AuthProvider` interface ready; only Local implemented. Authentik is the intended real auth. | Planned | High | L | 5 | Additive by design. |
|
||||
| Two-factor auth (TOTP) | Bearer/cookie session auth is solid; no MFA. | Partial | High | L | 5 | — |
|
||||
| DB-level audit immutability | Audit is insert-only by convention; no trigger/constraint. Verified as "adequate for self-host," so importance downgraded to match. | Have(soft) | Med | S | 9 | Adequate for self-host; upgrade to trigger only if true immutability is required. |
|
||||
| Block/hide users, family-group private space, DNA opt-in controls | Depend on messaging/DNA. | Missing | Low–Med | M–XL | 6/parked | DNA parked (NN#6). |
|
||||
|
||||
---
|
||||
|
||||
### 2.11 Import/export & standards
|
||||
|
||||
GEDCOM 5.5.1 import/export and full data-portability export are **have**; the remaining fidelity gaps (custom tags, PLAC coords/hierarchy, non-UTF-8 encoding) still undercut the provenance thesis.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **Citation links on GEDCOM export** | Export now selects Citations and emits `SOUR`/`PAGE` per fact (#232), so fact→source links survive a Provenance→Provenance round-trip. (Citation detail/confidence beyond page still to round-trip.) | Have | **Critical** | M | 2 | Closes the silent data-loss / destructive round-trip on the product's signature data (NN#5); satisfies PRD US-013. |
|
||||
| GEDCOM 7.0 import/export | Version hardcoded `5.5.1`; no v7 semantics, SCHMA, SUBM, or UID handling. | Partial | High | L | 2 | Stated differentiator (FamilySearch interop). |
|
||||
| Custom/underscore tag preservation | `_MARNM` becomes `TYPE married`, other custom tags dropped — violates ≥99% round-trip goal. | Missing | High | L | 2 | Tension with provenance thesis (faithful record). |
|
||||
| PLAC FORM hierarchy + MAP coordinate round-trip | Import reads only PLAC text; export emits flat PLAC. lat/long + hierarchy lost on round-trip. | Missing | High | M | 2–3 | Round-trip fidelity for the land/maps pillar. |
|
||||
| Encoding detection (ANSEL/UTF-16) | UTF-8 round-trips; non-UTF-8 files silently mangled via `errors='replace'`; CHAR tag ignored. | Partial | High | S | 2 | **Near quick win.** Detect/honor CHAR; reject or transcode rather than corrupt. |
|
||||
| HEAD completeness | HEAD at `gedcom.py:740` emits only `SOUR/GEDC/VERS/CHAR` — missing required `2 FORM LINEAGE-LINKED` (under GEDC) and `1 SUBM`. | Partial | Med | S | 2 | **Quick win.** Pure conformance. |
|
||||
| GEDCOM media (OBJE) round-trip | OBJE in skip-tags; media ignored on import, never emitted on export. | Partial | Med | M | 2 | Any media bundle keeps privacy gating. |
|
||||
| GEDZIP (.gdz) bundle | Bundled-media packaging. | Missing | Med | M | 2 | Natural once v7 + OBJE land. |
|
||||
| Selective / filtered export | Clippings-cart / branch subset. | Missing | Med | M | later | Maintain single-enforcement-point on export (NN#2). |
|
||||
| Import conformance validation | Preview is a mapping report, not structural/cardinality validation; bad lines silently skipped. | Partial | Med | M | 2 | — |
|
||||
| GEDCOM-X, Gramps XML, multi-format import, FHISO/ELF, PRF upload, KML export | Interop extras. | Missing | Low | L | later | PRF needs FamilySearch API (permitted, NN#6). |
|
||||
|
||||
---
|
||||
|
||||
### 2.12 Mobile & offline
|
||||
|
||||
Responsive web is **partial**; PWA and offline-first are absent. Native apps are an explicit deferral.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| PWA (manifest + icons + viewport + service worker) | No manifest, no SW, no `next-pwa`; responsive coverage exists but unaudited on heavy views (tree canvas fixed 74vh). | Partial | High | M | 2 | If SW caches API responses, never retain non-owner PII; cache only what the session is authorized to see (NN#3). |
|
||||
| Responsive parity audit | 17 breakpoint usages; small-screen parity on tree/person views unverified. | Partial | High | M | 1–2 | Feature parity is an ARCH requirement. |
|
||||
| Offline-first editing + reconnect sync | No SW, no local store, no mutation queue. Valuable for archive/courthouse field research. | Missing | High | XL | later | Replayed edits go through service layer + audit (NN#1); cached data respects living-person rule (NN#3). |
|
||||
| Native mobile apps | Explicitly deferred (responsive web only). | Missing | Med | XL | later | If built, reads through one backend privacy engine (NN#2/#3/#4). |
|
||||
| Companion app w/ cross-device sync | Largely redundant with server-backed web. | Missing | Low | XL | later | Sync boundary enforces privacy (NN#2); full CRUD parity (NN#8). |
|
||||
| Relatives Around Me | Nearby-relatives discovery. | Missing | Low | L | later | Explicit opt-in; anonymous until mutual consent (NN#4). |
|
||||
|
||||
---
|
||||
|
||||
### 2.13 API & extensibility
|
||||
|
||||
Internal REST + OpenAPI + generated TS client are **have**. The externalized developer story and the connector/plugin spine are not built.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Public read-only API + scoped tokens (OAuth) | The unauthenticated public read surface (`public.py`) now ships (#41–#51), but for a *developer* API the bearer token is still opaque session only and `TokenPurpose` lacks scopes — no scoped/OAuth token path. | Partial | High | L | 5–6 | Any scoped-token path routes through `person_visibility` + living-person redaction (NN#2/#3). |
|
||||
| SourceConnector framework | Only AuthProvider/ObjectStore/Mailer base classes exist; no connector base/loader/registry. Gates AI, hints, property connectors. | Planned | Med | L | 4 | Read-only, rate-limited; findings via ChangeProposal (NN#1); legal sources only (NN#6). |
|
||||
| Webhooks / change feeds | `AuditEntry` is the natural substrate (shares the notification dispatch layer, §2.9); no feed/webhook layer. | Missing | Med | L | 6 | Emit privacy-filtered, tenant-scoped projections — never raw before/after JSON (NN#2/#3). |
|
||||
| CLI / scripting surface | No `[project.scripts]`, no Typer/Click; worker is a purge loop only. Self-hosters want bulk admin. | Missing | Med | M | 9 | Funnel reads through privacy.py, writes through audit; admin-scoped, no assistant-write path. |
|
||||
| Plugin/addon architecture | Connector framework only; no general UI/report/theme plugin system planned. | Planned | Med | L | later | Sandbox via service layer; no privacy/audit bypass, no writes outside ChangeProposal. |
|
||||
| In-app query tooling (SuperTool) | Power-user expression engine. | Missing | Low | L | later | Execute through privacy engine — no row enumeration bypass (NN#2). |
|
||||
| Certified partner program | Organizational, not software. | Missing | Low | XL | — | Out of scope until a hosted offering exists. |
|
||||
|
||||
---
|
||||
|
||||
### 2.14 Performance & scale
|
||||
|
||||
Postgres + S3, multi-tenant isolation are **have**. Queue, observability, backups, pagination, and scale validation are the gaps that gate Phases 4/7 — several are current functional limitations, not late-phase validation tasks.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Real job queue (Postgres/Redis-backed) | Worker is a fixed-interval purge loop; GEDCOM import and account export run **inline in the request**. | Partial | High | L | 4 (pre-req) | Blocks NN#1 (assistant in worker) and NN#4 (hint matching in worker). Queue backend is an open question (PRD §11). |
|
||||
| **Pagination on list endpoints + server-side tree loading** | List endpoints (`persons.py:37`, events, relationships) take no `limit/offset/skip`; the tree view loads the whole graph client-side. A *current* limitation against the 50k-person target. | Planned | High | M | 1–2 | **Split out from scale validation** — this is a correctness/functional gap now, not a Phase 9 task. |
|
||||
| Scale validation (50k+ trees, P95<2s, load test) | No benchmark or load test exists. | Planned | High | L | 9 | Inline heavy ops risk partial writes — moving to the queue is what makes "failures never corrupt state" true. |
|
||||
| **Operator backup: one-command `pg_dump` + MinIO sync** | `deploy/backup.sh` + `deploy/BACKUP.md` now provide a scripted DB+object dump (#234). Remaining: scheduled/off-host/verified-restore tooling (row below). | Have | Critical | M | 1–2 | Restore must re-apply privacy state faithfully (NN#3); safety net for NN#8. |
|
||||
| Scheduled / cloud automated backup + restore tooling | Cron-driven, off-host, verified-restore workflow. | Partial | High | L | 9 | Builds on the one-command slice above. |
|
||||
| ARM64 build matrix | CI builds `linux/amd64` only; many self-hosters run ARM SBCs. | Partial | High | S | 1 | **Quick win.** Add arm64 + QEMU to buildx (NN#7 container-native). |
|
||||
| Structured JSON logs + Prometheus metrics | Plain-text stdlib logging; no `/metrics`. | Partial | Med | M | 9 | Logs/metrics reference UUIDs, never names/PII (NN#3/#4). |
|
||||
| pgvector enablement | Image has pgvector; app never creates the extension or adds embedding columns (docs claim otherwise). | Partial | Med | M | 7 | See §2.3 — embedding provider open question; candidates via privacy engine. |
|
||||
| Database check-and-repair | No orphan/dangling-edge/cycle scanner (recent "harden tree render" commit shows bad graphs occur). | Missing | Med | M | 9 | Tenant-scoped + audited; auto-fix via ChangeProposal (NN#1). |
|
||||
| Pluggable DB backend, billions-scale shared tree, weekly record releases | Different product models. | Missing | Low | XL | — | **Out of scope** — Postgres-only is consistent with the invariants; global shared tree conflicts with NN#2/#3/#4. |
|
||||
|
||||
---
|
||||
|
||||
### 2.15 Property / land chain-of-title — *headline differentiator*
|
||||
|
||||
The entire "land" half is **planned/missing** but fully specified. This is where Provenance has no real competitor.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| Property/parcel first-class entity | No model/endpoint/service/migration. Foundation for the whole category. | Planned | High | L | 3 | Full CRUD in API+UI (NN#8); reads added carefully to the **single** privacy engine (NN#2). |
|
||||
| Typed OwnershipEvents | grant/patent, purchase, sale, inheritance, gift, tax sale, foreclosure, eminent domain — with grantor/grantee Persons + Citation. | Planned | High | L | 3 | Each event carries a Citation (NN#5); grantor/grantee living-person links redacted (NN#3). |
|
||||
| Chain-of-title timeline + gap flagging | Ordered OwnershipEvents first-grant→present, breaks flagged. | Planned | High | M | 3 | The genuinely differentiating analytical piece (PRD US-032). |
|
||||
| Bidirectional owner↔person, parcel↔place | "Every property a person held" / "every parcel at a place." | Planned | High | M | 3 | Reverse traversals filtered through privacy engine (NN#2). |
|
||||
| Citations on OwnershipEvents | Add `ownership_event_id` to Citation (5th target). | Partial | Critical | S | 3 | **Quick win once Property lands** — single FK + CHECK edit (NN#5). |
|
||||
| Legal description verbatim storage | metes-and-bounds / PLSS township-range-section as-written. | Planned | Med | L | 3 | Part of the Property model; preserves the record faithfully. |
|
||||
| Parcel/plat boundary geometry | Optional geometry; plain coords first. | Planned | Med | L | 3+ | PostGIS is an open question (ARCH §14) — surface dependency. |
|
||||
| PLSS / metes-and-bounds parsing → geometry | Automated survey parsing. | Planned | Med | XL | later | Hard; gated on PostGIS. |
|
||||
| BLM/GLO federal land-patent connector | Marquee US land source. | Planned | High | L | 8 | Permitted source (NN#6); patents surface as ChangeProposals (NN#1); read-only + rate-limited. |
|
||||
| USGS map + public county-deed connectors | Per-jurisdiction grantor/grantee indexes. | Planned | Med | L | 8 | Each connector verifies a legally open source (NN#6). |
|
||||
| Co-ownership roles / tenure types | joint tenants, TIC, life estate, heirs. | Planned | Low | M | later | Multiple parties likely free with OwnershipEvent; role typing is a refinement. |
|
||||
| Tax/assessment rolls, UK Tithe, Lloyd George Domesday | Valuation + non-US collections. | Missing | Low | M–L | — | US-focused v1; international formats out of scope (model is country-agnostic). |
|
||||
|
||||
---
|
||||
|
||||
### 2.16 AI assistant — *defining differentiator*
|
||||
|
||||
The spine has now **landed**: the `ChangeProposal` model/schema/service, its migration, the GET/POST API, and a review UI all ship, and the `LLMProvider`/`EmbeddingProvider` abstraction with null/Anthropic/OpenAI-compat (OpenAI/xAI/Ollama) providers + registry is in place. The audit substrate (`actor_type=assistant`, before/after JSONB) is the right foundation; the remaining work is wiring the assistant's tools to emit proposals and building the chatbot/RAG surface on top.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **ChangeProposal (propose-then-confirm)** | The defining invariant. Model/schema/service (`models/change_proposal.py`, `services/change_proposal_service.py`), migration `a1b2c3d4e5f6`, GET/POST `api/v1/proposals.py`, and a `/trees/[id]/proposals` review UI all ship. Remaining: wire assistant tools to emit proposals. | Have | **Critical** | L | 4 | **IS NN#1.** Enforce structurally: assistant tools return proposals; only user action applies one; application flows through the normal service layer (privacy + audit). ChangeProposal itself needs full CRUD (NN#8). |
|
||||
| Pluggable LLM + embedding provider | `LLMProvider`/`EmbeddingProvider` ABCs (`integrations/models/base.py`) with null, Anthropic, and OpenAI-compat (OpenAI/xAI/Ollama) implementations + registry. | Have | Critical | M | 4 | **Twelve-factor, no hard-coded keys/endpoints** (NN#7); the Ollama/self-hosted path is what makes the privacy-first promise real. |
|
||||
| Per-tree AI model policy | Owner-only per-tree model selection (`Tree.ai_member_provider`/`ai_recommender_provider`, GET/PATCH `/trees/{id}/ai`, `/trees/[id]/ai` UI) (#238). | Have | Med | S | 4 | Owner-only; selects which configured provider a tree uses — keys stay in env, twelve-factor (NN#7). |
|
||||
| AI research-assistant chatbot (RAG over tree) | Marquee feature; needs ModelProvider + connector + retrieval through privacy engine. | Planned | High | XL | 4 | NN#1 propose-only, NN#2 privacy retrieval, NN#3 redaction. |
|
||||
| Conversational / connector record search | Search legal sources via the assistant. | Planned | High | L | 4 | Legal sources (NN#6); findings = Source + Citation (NN#5). |
|
||||
| Fact extraction from documents | Extracted facts map cleanly to ChangeProposal review. | Missing | Med | M | 4 | Canonical NN#1 use case; each fact carries a Citation (NN#5). |
|
||||
| OCR/HTR transcription + document translation | Worker job via ModelProvider. | Missing | Med | L | 4+ | Output → Source/Citation (NN#5); via ModelProvider (NN#7); auto-extraction emits ChangeProposal (NN#1). |
|
||||
| Next-step research guidance | Gap analysis → suggested next record. | Planned | Med | M | 4 | Reads via privacy engine; advisory unless it queues fetches. |
|
||||
| AI biography / audio narration | Read-only generation grounded in tree data. | Missing | Low | M–L | later | Must not leak living-person PII (NN#3); via ModelProvider (NN#7); stored biographies = full CRUD (NN#8). |
|
||||
|
||||
---
|
||||
|
||||
### 2.17 Localization & accessibility
|
||||
|
||||
A documented **day-one commitment** ("UI strings externalized from day one") that is currently **unmet** — every label is a hardcoded literal. Correct the PRD claim or close the gap.
|
||||
|
||||
| Item | Description | Status | Imp | Eff | Phase | Non-negotiable |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **UI string externalization** | No i18n lib, no message catalogs; all copy hardcoded in TSX. Gating prerequisite; cheapest to do now while the surface is small. | Missing | High | L | 1–2 | PRD §6 promises this "from day one" — **docs-vs-code gap; edit the doc now.** |
|
||||
| Multi-language UI (40–60+ langs) | Translation pipeline after externalization (frontend + backend-generated messages). | Missing | High | XL | later | Table stakes across all competitors. |
|
||||
| Accessibility / WCAG 2.2 AA | Some ARIA/focus styling; no CI a11y audit, no skip-links, SVG tree viz not keyboard/screen-reader navigable. | Partial | High | L | 2/9 | Stated PRD §6 target; add axe/pa11y in CI; accessible alternate to the chart. |
|
||||
| Unicode-correct non-Latin names | Stores fine (UTF-8); no NFC normalization on write, no locale-aware collation, no romanized search. | Partial | High | M | 2 | Apply `unicodedata.normalize('NFC')` on input; add COLLATE; supports faithful-record goal. |
|
||||
| Structured/compound surname components | Surname is a single field; no support for Spanish/Portuguese paternal+maternal, Arabic nasab, particles/prefixes. | Missing | Med | M | 2 | New Name sub-fields ship with full CRUD (NN#8); preserves the name as recorded. |
|
||||
| Non-Gregorian calendar dates | `calendar` column is a placeholder; GEDCOM calendar escapes never parsed/populated. | Partial | Med | L | 2 | Preserve original calendar as recorded (sources-first). |
|
||||
| Language tags / romanized variants per name | No language_tag/script/romanized fields; GEDCOM ROMN/LANG unhandled. | Missing | Med | M | 2 | New Name sub-fields ship with full CRUD (NN#8). |
|
||||
| RTL support | `lang="en"` hardcoded, no `dir`, physical CSS properties throughout. | Missing | Med | M | later | Convert to logical CSS properties; cheaper once i18n exists. |
|
||||
| Selectable themes | Light/dark/system works; brand palette intentionally single. | Partial | Med | M | later | Confirm whether additional themes are a deliberate non-goal (brand guide constrains palette). |
|
||||
| Multi-language report/diagram output | Depends on i18n + reports, neither shipped. | Missing | Low | L | later | — |
|
||||
|
||||
---
|
||||
|
||||
## 3. Quick wins (high importance / low effort)
|
||||
|
||||
Ordered by leverage. All are S-effort or a thin slice of a larger item, and most close a stated invariant gap.
|
||||
|
||||
1. **Fix `site_members` visibility tier** (Privacy, Critical/S) — **done:** `can_view_tree` now handles `site_members` (`privacy.py:56`), giving any authenticated account a read view while refusing anonymous.
|
||||
2. **Email-verification enforcement gate** (Privacy/Auth, High/S) — **done (#53):** the read-side `email_verified_at` check now ships behind `REQUIRE_EMAIL_VERIFICATION`, so a freshly registered, unverified user doesn't get a live authenticated session. The registration-mode env switch (open/approve/closed) is the larger follow-on (§2.10, M-effort — not a quick win).
|
||||
3. **Citation confidence selector in the cite form** (Sources, High/S) — confidence is modeled and API-writable but unreachable in the UI; every UI citation is currently NULL. Honors NN#8 and the evidence-quality thesis.
|
||||
4. **Source edit UI + expose all 8 fields** (Sources, High/S) — update API exists but there is no edit form and create exposes ~3 fields; a create-but-not-edit entity violates NN#8.
|
||||
5. **Make `LIVING_RECENCY_YEARS` env-configurable** (Privacy, High/S) — hardcoded 100 at `privacy.py:23`; twelve-factor (NN#7).
|
||||
6. **Add `ownership_event_id` to Citation** (Property/Sources, Critical/S) — single FK + CHECK-constraint edit the moment Property lands; the spine is already built (NN#5).
|
||||
7. **GEDCOM encoding detection** (Standards, High/S) — detect/honor the CHAR tag; reject or transcode ANSEL/UTF-16 rather than silently mangling with `errors='replace'`.
|
||||
8. **GEDCOM HEAD completeness** (Standards, Med/S) — emit the required `2 FORM LINEAGE-LINKED` (under GEDC) and `1 SUBM` at `gedcom.py:740`. Pure conformance.
|
||||
9. **ARM64 CI build matrix** (Perf/Scale, High/S) — add `linux/arm64` + QEMU to buildx for both images; many self-hosters run ARM SBCs.
|
||||
10. **`GET /{tree}/citations/{id}` endpoint** (Sources, Med/S) — API symmetry (NN#8).
|
||||
11. **Transcription/abstract fields on Source** (Sources, Med/S) — add `transcription_text` + `abstract_text`, distinct from `citation_text`; core to evidence analysis.
|
||||
12. **Sort the merged person timeline** (Research workflow, Med/S) — `shownEvents.sort()` on `date_start`; currently appended unsorted.
|
||||
13. **Doc corrections (docs-vs-code)** (Meta, trivial/S) — edit CLAUDE.md / ARCHITECTURE so the pgvector "used" claim and the i18n "from day one" claim match reality. The repo convention requires docs to travel with code.
|
||||
|
||||
> **Shipped this cycle:** the **media privacy leak** (§2.4) and the **child-resource redaction gap** (§2.10) are fully closed — person/event/media/name/relationship (#46) and citation/source endpoints all apply `person_visibility` for non-members. No residual living-person leak on the read surface.
|
||||
|
||||
---
|
||||
|
||||
## 4. Strategic differentiators
|
||||
|
||||
Where to invest to make Provenance distinct rather than a webtrees clone. Each leans on a non-negotiable as a *feature*, not a constraint.
|
||||
|
||||
**1. Property chain-of-title (the "land" half).** No surveyed competitor models ownership as a typed, cited event chain tying parties across time, with gap-flagging and bidirectional owner↔person / parcel↔place traversal, fed by **legal** public sources (BLM/GLO patents, USGS, public county deeds). This is the single clearest "no one else does this" capability. Sequence: Property + OwnershipEvent + Citation-target (Phase 3) → chain-of-title view → BLM/GLO connector (Phase 8). The Citation extension is a quick win; the entity is the prerequisite for everything else in the category.
|
||||
|
||||
**2. The ChangeProposal AI model.** "The assistant never writes autonomously" is a *trust* differentiator in a market where users fear AI corrupting their research. The structural spine has **landed** — the `ChangeProposal` model/API/review UI and the pluggable `LLMProvider`/`EmbeddingProvider` abstraction both ship — so the remaining work is wiring the assistant's tools to emit proposals (never mutating directly). Assistant tools return proposals; only an explicit human action applies one; application flows through the normal service layer so it always hits the privacy engine and audit log. The same approval queue moderates untrusted human-contributor edits (Collaboration §2.9), so design them together.
|
||||
|
||||
**3. Anonymous, mutual-consent cross-tree hints.** The privacy model already redacts living people for anonymous viewers, so a hint system that reveals *nothing identifying* until both sides opt in is achievable by construction — and is a categorically more trustworthy version of MyHeritage Smart Matches / Ancestry hints. Requires the matching engine (pgvector enablement + candidate generation, Phase 7), the notification/event-dispatch substrate (§2.9), and the messaging channel that opens only post-consent.
|
||||
|
||||
**4. True self-hosting + data ownership.** Full account export/import, soft-delete recovery (with owner-confirmed on-demand purge to delete a trashed tree immediately rather than waiting out the 30-day window), GEDCOM round-trip, env-driven everything, a one-command operator backup, and (to-build) scheduled off-host backup + ARM support make Provenance the genealogy app you actually own. The two correctness items that gated the promise have **landed**: GEDCOM export now preserves citations (the Provenance→Provenance round-trip keeps the sources graph), and operator backup moved from "documented procedure" to a one-command dump (`deploy/backup.sh`). What remains is scheduled/verified-restore tooling and ARM builds. The Ollama/self-hosted ModelProvider path means even the AI assistant runs without tree data leaving the deployment — a promise no commercial competitor can make.
|
||||
|
||||
**5. Sources-first as a felt experience.** The two-tier model is built, and citations now **survive GEDCOM export** (#232); the remaining differentiator is making sourcing *visible and low-friction*: a guided Evidence-Explained citation builder, transcription/abstract fields, source-driven data entry (transcribe a document into the tree), and per-fact confidence surfaced in the UI. These turn "every fact links to where it came from" from an architecture note into the product's personality.
|
||||
+15
-9
@@ -1,8 +1,8 @@
|
||||
# Provenance — Product Requirements Document
|
||||
|
||||
**Status:** Draft v0.1
|
||||
**Status:** Draft v0.1 — now describes a partially-implemented system: Phase 0 complete, Phase 1 done, with early slices of later phases shipped.
|
||||
**Owner:** Justin Paul
|
||||
**Last updated:** 2026-06-06
|
||||
**Last updated:** 2026-06-10
|
||||
|
||||
---
|
||||
|
||||
@@ -94,7 +94,7 @@ Acceptance criteria (AC) are written to be testable.
|
||||
- **US-033** I view every property a person held, and every parcel ever recorded at a place. *AC:* both reverse lookups return correct sets.
|
||||
|
||||
### Privacy & sharing
|
||||
- **US-040** I set a tree to public, unlisted, or private. *AC:* visibility enforced for anonymous and non-owner users.
|
||||
- **US-040** I set a tree to one of four visibility levels — private, unlisted, site_members, or public. *AC:* visibility enforced for anonymous and non-owner users; at the **site_members** level the tree is visible to any authenticated instance user (signed in but not a member of the tree) and hidden from anonymous visitors.
|
||||
- **US-041** I mark any individual private even within a public tree. *AC:* that person's details hidden from non-owners regardless of tree setting.
|
||||
- **US-042** Living people are hidden from non-owners by default. *AC:* a person with no death fact and a plausibly-living birth date shows only minimal/no PII to non-owners; owner can override per person.
|
||||
- **US-043** I add a co-owner to a tree. *AC:* co-owner can edit per role; action attributed to them in the audit log.
|
||||
@@ -132,6 +132,7 @@ Acceptance criteria (AC) are written to be testable.
|
||||
### 5.1 Identity & access
|
||||
- Pluggable authentication: local password (with email verification and reset), social sign-in (Google, Apple, Facebook), and generic **OIDC** (validated against Authentik; should work with Keycloak, Authentik, Auth0, etc.). Operators enable any subset.
|
||||
- Roles per tree: **owner**, **co-owner/editor**, **viewer**. Public/unlisted trees also have an implicit anonymous viewer.
|
||||
- **Instance owner/operator:** an env-declared operator role (via `OWNER_EMAIL`, requiring a verified email), distinct from the per-tree roles. It is an operations/config role only and is **not** a privacy bypass — it grants no access to others' tree data or PII.
|
||||
- The AI assistant acts as a distinct, scoped principal bound to the user it is helping — it can never exceed that user's rights, and its actions are separately attributable.
|
||||
|
||||
### 5.2 Data model (core entities)
|
||||
@@ -155,6 +156,7 @@ Acceptance criteria (AC) are written to be testable.
|
||||
|
||||
### 5.5 Privacy engine
|
||||
- Effective visibility = function(tree visibility, person override, living status, viewer role).
|
||||
- Tree visibility has four levels: **private** (members only; default), **unlisted** (anyone with the link, not listed/indexed), **site_members** (any authenticated instance user), and **public** (anonymous + listed/indexable).
|
||||
- Living-person rule: absent a death fact and within a configurable recency window (default ~100 years from birth, or unknown birth treated as possibly-living), non-owners see minimal or no PII.
|
||||
- Public/link views must render through the same privacy engine — no bypass path.
|
||||
|
||||
@@ -168,6 +170,7 @@ Acceptance criteria (AC) are written to be testable.
|
||||
|
||||
### 5.8 AI research assistant
|
||||
- Provider-agnostic abstraction over hosted models (Anthropic, OpenAI, xAI) and self-hosted/local models (e.g., an OpenAI-compatible endpoint or Ollama).
|
||||
- Operators register one or more model providers (env / registry); a tree owner then selects the active provider(s) for that tree via an owner-only AI settings surface.
|
||||
- Tool-mediated access to the same CRUD operations a user has, scoped to that user, via a server with explicitly scoped capabilities (an MCP-style tool boundary).
|
||||
- **Propose-then-confirm is mandatory.** The assistant drafts changes as diffs; nothing persists without explicit user approval.
|
||||
- Source connectors are a **plugin framework**; the project ships only legal sources (e.g., FamilySearch API, Find A Grave, WikiTree, BLM/GLO land patents, USGS maps, public-domain newspapers, public county records). Operator-supplied scrapers can be added later.
|
||||
@@ -181,6 +184,7 @@ Acceptance criteria (AC) are written to be testable.
|
||||
### 5.11 Administration & operations
|
||||
- All integration points (auth, SMTP, object storage, database, model providers, scrapers) are environment/config-driven.
|
||||
- Health endpoints; structured logs; a documented backup/restore procedure; safe upgrade via image pull + migration.
|
||||
- Owner-only operator surface: instance status and configuration (`GET /api/v1/admin/instance` and the `/admin` UI), scoped to the instance owner and exposing no tree contents or PII.
|
||||
|
||||
## 6. Non-functional requirements
|
||||
|
||||
@@ -206,17 +210,19 @@ Acceptance criteria (AC) are written to be testable.
|
||||
|
||||
Provenance ships continuously and is stood up in a live lab as it goes; there is no hard MVP/v2 line, but features land in dependency order so each tranche is usable.
|
||||
|
||||
- **Phase 0 — Foundation:** backend + DB schema; local auth + email verify; frontend scaffold; container images; CI/CD (Gitea Actions → Gitea registry → server pull); one-command compose deploy.
|
||||
- **Phase 1 — Core tree:** people, relationships, events; sources & citations; media uploads; soft delete + recovery; tree-level privacy.
|
||||
- **Phase 2 — Standards & polish:** GEDCOM 7 import/export; search with fuzzy names; living-person protection; person-level privacy override; onboarding + persona selector.
|
||||
- **Phase 0 — Foundation:** *(shipped)* backend + DB schema; local auth + email verify; frontend scaffold; container images; CI/CD (Gitea Actions → Gitea registry → server pull); one-command compose deploy.
|
||||
- **Phase 1 — Core tree:** *(shipped)* people, relationships, events; sources & citations; media uploads; soft delete + recovery; tree-level privacy (now four levels: private/unlisted/site_members/public).
|
||||
- **Phase 2 — Standards & polish:** *(partly shipped — GEDCOM 7 import/export #232; fuzzy/trigram search)* GEDCOM 7 import/export; search with fuzzy names; living-person protection; person-level privacy override; onboarding + persona selector.
|
||||
- **Phase 3 — Property:** property entity; ownership events; chain-of-title view; property-aware sources.
|
||||
- **Phase 4 — AI assistant:** provider abstraction (hosted + local); scraper plugin framework; first connectors (FamilySearch, Find A Grave); propose-diff approval flow; assistant actions in audit log.
|
||||
- **Phase 5 — Federated auth:** OIDC (Authentik), then Google/Apple/Facebook sign-in.
|
||||
- **Phase 6 — Collaboration:** tree co-owners; audit-log UI; direct messaging; notifications.
|
||||
- **Phase 4 — AI assistant:** *(partly shipped early — provider abstraction + multi-provider registry #235/#237; ChangeProposal propose-then-confirm #236)* provider abstraction (hosted + local); scraper plugin framework; first connectors (FamilySearch, Find A Grave); propose-diff approval flow; assistant actions in audit log.
|
||||
- **Phase 5 — Federated auth:** *(not shipped — only the `AuthProvider` ABC exists)* OIDC (Authentik), then Google/Apple/Facebook sign-in.
|
||||
- **Phase 6 — Collaboration:** *(tree membership #233 landed early)* tree co-owners; audit-log UI; direct messaging; notifications.
|
||||
- **Phase 7 — Cross-tree hints:** async matching engine (embeddings-assisted); anonymous match notifications; mutual-consent reveal.
|
||||
- **Phase 8 — Land sources:** BLM/GLO patents; USGS map integration; additional county-deed connectors (merge existing scrapers).
|
||||
- **Phase 9 — Hardening & dogfooding** toward a possible hosted offering.
|
||||
|
||||
**Shipped ahead of sequence (operations & platform):** instance-owner/operator role (#240); operator backup tooling (#234); a schema-drift guard (#239). These landed early because the live lab deployment needed them. Note that despite their later issue numbers, **Phase 5 federated auth/OIDC is not yet shipped** — only the `AuthProvider` ABC is in place.
|
||||
|
||||
Rationale: enabling work (schema, auth, deploy, sources) precedes everything; GEDCOM lands before the assistant so AI writes target a stable model; property follows a well-tested people graph; hints come late because they require multiple populated trees.
|
||||
|
||||
## 9. Technical direction (summary)
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
# Design note: ChangeProposal (propose-then-confirm)
|
||||
|
||||
Status: **Shipped (#214/#236)** — model, service, API, and review UI landed; the
|
||||
assistant producer and cross-op transactional apply remain as follow-ups (see
|
||||
Out of scope). Implements non-negotiable #1 (CLAUDE.md): *the AI
|
||||
assistant never writes autonomously.* Every assistant "write" emits a
|
||||
**ChangeProposal** — a structured diff a human approves, edits, or rejects.
|
||||
|
||||
## The invariant, structurally
|
||||
|
||||
There must be **no code path where a model response mutates tree data**. We get
|
||||
this by construction, not convention:
|
||||
|
||||
- Model providers (`app/integrations/models/*`) are read-only text/vector
|
||||
producers — they never import a repository or session-mutating service.
|
||||
- The assistant's tools, when they land, will call `change_proposal_service.propose(...)`,
|
||||
which only **inserts a pending ChangeProposal**. It performs no domain mutation.
|
||||
- A ChangeProposal's operations are executed **only** by
|
||||
`change_proposal_service.apply(...)`, which:
|
||||
1. requires the actor be an **editor/owner** of the tree (`privacy.can_edit_tree`),
|
||||
2. dispatches each operation through the **normal editing services**
|
||||
(`person_service`, `event_service`, …) — so every change passes the privacy
|
||||
engine and writes an `AuditEntry` with the **human** as `actor`,
|
||||
3. flips the proposal to `applied`.
|
||||
|
||||
So an assistant can *suggest* anything, but a change reaches the database only
|
||||
when a human with edit rights approves it, and only via the same services a human
|
||||
edit uses.
|
||||
|
||||
## Data model
|
||||
|
||||
`ChangeProposal` (`TenantScoped` tree_id, `Timestamps`, `SoftDelete`):
|
||||
|
||||
| field | notes |
|
||||
|---|---|
|
||||
| `tree_id` | tenant boundary |
|
||||
| `status` | `pending` \| `applied` \| `rejected` |
|
||||
| `origin` | `assistant` \| `contributor` — who proposed it (the contributor case also moderates untrusted human edits) |
|
||||
| `created_by_user_id` | the user on whose behalf the assistant acted, or the contributor |
|
||||
| `summary` | one-line human description ("Add birth 1850 to John Smith") |
|
||||
| `rationale` | the assistant's reasoning / sources (text) |
|
||||
| `operations` | JSONB list of ops (the structured diff) |
|
||||
| `reviewed_by_user_id`, `reviewed_at`, `review_note` | set on approve/reject |
|
||||
| `apply_error` | populated if application failed (proposal stays `pending`) |
|
||||
|
||||
An **operation** is `{op, entity_type, entity_id?, payload}`:
|
||||
- `op` ∈ `create` | `update` | `delete`
|
||||
- `entity_type` ∈ `person` | `name` | `event` | `relationship` | `source` | `citation`
|
||||
- `entity_id` — null for `create`; the target id for `update`/`delete`
|
||||
- `payload` — proposed field values (`create`/`update`); ignored for `delete`
|
||||
|
||||
A proposal may carry several operations (e.g. "add a person and link them as a
|
||||
child" = create person + create relationship), applied **in order**. The editing
|
||||
services each commit, so v1 application is **not transactional across ops** — if
|
||||
op N fails, ops 1..N-1 are already applied and the proposal stays `pending` with
|
||||
`apply_error` set so the reviewer can fix and re-apply the remainder. Single-op
|
||||
proposals (the common near-term case) are effectively atomic. Cross-op atomicity
|
||||
is a follow-up (it needs the services to accept a no-commit mode).
|
||||
|
||||
## Service surface
|
||||
|
||||
- `propose(session, *, tree, origin, created_by, summary, rationale, operations) -> ChangeProposal`
|
||||
— inserts a `pending` proposal. The **only** thing the assistant can call.
|
||||
- `list_proposals` / `get_proposal` — visible to tree members.
|
||||
- `apply(session, *, actor, tree, proposal_id, edited_operations=None) -> ChangeProposal`
|
||||
— editor-only. Optional `edited_operations` lets the reviewer tweak the diff
|
||||
before applying ("edit" in approve/edit/reject). Dispatches each op through the
|
||||
editing services; on failure it records `apply_error` and leaves the proposal
|
||||
pending — it does **not** roll back ops already committed by earlier dispatches
|
||||
(v1 is not cross-op transactional; see Data model).
|
||||
- `reject(session, *, actor, tree, proposal_id, note=None)` — editor-only.
|
||||
|
||||
## API
|
||||
|
||||
`/trees/{id}/proposals`: `GET` (list, `?status=`), `POST` (create — used by tests
|
||||
and the future contributor flow), `GET /{pid}`, `POST /{pid}/apply`,
|
||||
`POST /{pid}/reject`, `DELETE /{pid}`.
|
||||
|
||||
## Out of scope (follow-ups)
|
||||
|
||||
- The assistant itself (it will be the primary producer; #-future).
|
||||
- A rich diff/edit UI — v1 ships a review list with approve/reject; "edit before
|
||||
apply" is supported in the API and can get UI later.
|
||||
- Dispatch for media/place/tree-settings ops (added when a producer needs them).
|
||||
@@ -0,0 +1,168 @@
|
||||
# Design note: tree visibility & the public viewing surface
|
||||
|
||||
Status: **Shipped (#41-#51)**. Owner: Justin. Created 2026-06-09.
|
||||
|
||||
This is a privacy-critical change (it created the first anonymous read surface in
|
||||
Provenance). Per CLAUDE.md, it was designed before code and shipped in small,
|
||||
individually-reviewable PRs, with tests on the privacy engine and the public read
|
||||
path landing before any anonymous endpoint was exposed.
|
||||
|
||||
## 1. The model
|
||||
|
||||
Visibility flattens **two axes** — *who may read* and *how discoverable* — into one
|
||||
ordered enum for the UI:
|
||||
|
||||
| Level | Anonymous (no login) | Any logged-in user | Tree members | In-app directory | Search-indexed |
|
||||
|---|---|---|---|---|---|
|
||||
| `public` — anyone on the web | ✅ view¹ | ✅ view¹ | ✅ full | ✅ listed to everyone | ✅ sitemap + indexable |
|
||||
| `site_members` — Public, Site Members | ❌ | ✅ view¹ | ✅ full | ✅ listed to logged-in users | ❌ (`noindex`) |
|
||||
| `unlisted` — anyone with the link | ✅ via direct link¹ | ✅ via link¹ | ✅ full | ❌ never listed | ❌ (`noindex`) |
|
||||
| `private` | ❌ | ❌ | ✅ full | ❌ | ❌ |
|
||||
|
||||
¹ **Every non-member view passes through the privacy engine.** Living people are
|
||||
redacted, and per-person `private` hides / `public` reveals, exactly as
|
||||
`person_visibility()` already does (`backend/app/services/privacy.py:100-110`).
|
||||
This is the single enforcement point — no public code path may issue a raw query.
|
||||
|
||||
Decisions captured (2026-06-09):
|
||||
- **Unlisted** = anyone with the link, no account required. The link must be
|
||||
**unguessable** (the tree UUID is already non-enumerable; do not add a public
|
||||
integer id). Unlisted trees are excluded from the directory and sitemap and
|
||||
served `noindex`.
|
||||
- **Public** discovery for v1 includes **an in-app public browse/search**, not
|
||||
just search-engine indexing.
|
||||
- **Public – Site Members** = *any* registered account on this instance (not an
|
||||
invite list — that is already tree membership / `private`).
|
||||
|
||||
## 2. Data model
|
||||
|
||||
`TreeVisibility` enum (`backend/app/models/enums.py`) gains a value:
|
||||
|
||||
```
|
||||
public # anyone on the web
|
||||
site_members # any authenticated user of this instance <-- NEW
|
||||
unlisted # anyone with the link
|
||||
private # members only (default)
|
||||
```
|
||||
|
||||
- Alembic migration to `ALTER TYPE tree_visibility ADD VALUE 'site_members'`
|
||||
(Postgres enum add-value cannot run inside a transaction with other DDL — use
|
||||
`op.execute` with autocommit, separate migration).
|
||||
- Default stays `private`. Existing rows unchanged.
|
||||
- `TreeRead`/`TreeUpdate`/`TreeCreate` schemas already carry the enum; they pick
|
||||
up the new value automatically. The OpenAPI client regen (`gen:api`) exposes it
|
||||
to the frontend.
|
||||
|
||||
## 3. Privacy engine
|
||||
|
||||
`can_view_tree()` today treats `public` and `unlisted` identically and ignores
|
||||
whether the viewer is anonymous vs authenticated (`privacy.py:44-49`). Replace the
|
||||
final line with explicit branching on viewer auth state:
|
||||
|
||||
```
|
||||
if membership: return True # members always
|
||||
match tree.visibility:
|
||||
public, unlisted: return True # anonymous OK (unlisted gated only by knowing the link)
|
||||
site_members: return user_id is not None # any logged-in account
|
||||
private: return False
|
||||
```
|
||||
|
||||
`person_visibility()` is unchanged — it already redacts living/private people for
|
||||
non-members. Add focused unit tests: anonymous + each visibility; living person
|
||||
redacted on public/unlisted; `site_members` denies anonymous but allows a
|
||||
logged-in non-member; `private` denies both.
|
||||
|
||||
## 4. The anonymous read path (the careful part)
|
||||
|
||||
**Shipped: a dedicated read-only public API namespace**, not optional-auth on the
|
||||
existing endpoints. Rationale: it is far easier to audit a small, purpose-built
|
||||
surface that *always* funnels through `person_visibility` than to weaken the
|
||||
membership checks on the authenticated endpoints and hope every branch is covered.
|
||||
|
||||
- Router `app/api/v1/public.py`, mounted at `/api/v1/public`, with an
|
||||
**optional-auth** dependency `CurrentUserOrNone` (returns `User | None`; never
|
||||
401s). Contrast with `CurrentUser` (`deps.py:30-36`) which hard-401s.
|
||||
- Endpoints (read-only; no create/update/delete):
|
||||
- `GET /public/trees` — directory: lists `public` to everyone; additionally
|
||||
lists `site_members` when the caller is authenticated. Paginated, search via
|
||||
existing `pg_trgm`. Never lists `unlisted`/`private`.
|
||||
- `GET /public/trees/{id}` — tree metadata if `can_view_tree(user_or_none)`.
|
||||
- `GET /public/trees/{id}/persons`, `/persons/{pid}`, `/persons/{pid}/names`,
|
||||
`/relationships`, `/events` — each filtered through `person_visibility`.
|
||||
(Media is not exposed on the public surface yet — deferred.)
|
||||
- **Redaction happens in the service, before serialization** — this is the safety
|
||||
guarantee. It did **not** ship as a separate `PublicPersonRead` schema (that
|
||||
recommendation was not adopted): the public router **reuses the member read
|
||||
schemas** (`PersonRead`, `RelationshipRead`, `EventRead`, `NameRead`), and only
|
||||
the tree projection (`PublicTreeRead`) is distinct. Safety comes from
|
||||
`public_view_service` resolving `person_visibility` and then **dropping hidden
|
||||
rows and redacting possibly-living people** (`person_service._redact` rewrites
|
||||
the name to "Living person", etc.) *before* a row is ever validated into a
|
||||
schema. No route hands a raw row to the serializer.
|
||||
- **Rate limiting** on the public namespace (per-IP) is **deferred** — it is not
|
||||
implemented in the app and may be handled at the Caddy edge if needed.
|
||||
- **Audit**: count public reads; do not log PII.
|
||||
|
||||
## 5. Frontend public pages
|
||||
|
||||
- New **server-rendered** routes outside the authed app shell, e.g.
|
||||
`/p/[treeId]` (tree), `/p/[treeId]/[personId]` (person), `/explore` (directory).
|
||||
Server components fetch the `/api/v1/public/*` endpoints; no login redirect.
|
||||
- `robots`: ships a coarse `allow: ["/", "/p/"]` rule (`frontend/app/robots.ts`)
|
||||
that keeps the authed app out of the index. Per-tree `noindex, nofollow` meta
|
||||
for `unlisted`/`site_members` and a `public`-only **sitemap** did **not** ship —
|
||||
both are **deferred** follow-ups (per-tree noindex needs server rendering;
|
||||
meanwhile `unlisted`/`site_members` trees aren't linked or listed, so they
|
||||
aren't crawl-discoverable).
|
||||
- The directory `/explore` is anonymous for `public`; shows `site_members` trees
|
||||
only to logged-in users.
|
||||
- Reuse the tree/person view components where possible, fed by the redacted
|
||||
schema.
|
||||
|
||||
## 6. UI control
|
||||
|
||||
Update the visibility dropdown (`frontend/app/trees/page.tsx`, shipped in PR #41)
|
||||
from 3 to 4 options with helper text:
|
||||
|
||||
```
|
||||
Private — only you and people you invite
|
||||
Public – Members — any signed-in user on this site
|
||||
Unlisted — anyone with the link (not listed or indexed)
|
||||
Public — anyone on the web; listed and search-indexable
|
||||
```
|
||||
|
||||
A short confirmation when switching *to* `public` ("This makes <tree> visible to
|
||||
anyone on the web. Living people stay hidden.") is worthwhile given the stakes.
|
||||
|
||||
## 7. Guardrails / invariants
|
||||
|
||||
- One enforcement point: every public response is built from `person_visibility`
|
||||
output. No raw repository reads in the public router.
|
||||
- Living-person protection holds regardless of tree visibility.
|
||||
- Unlisted relies on UUID unguessability; never expose a sequential public id.
|
||||
- Per-tree `noindex` (everything except `public`) and a `public`-only sitemap are
|
||||
**deferred** (see §5); today `robots.ts` keeps the authed app out of the index
|
||||
and `unlisted`/`site_members` trees aren't linked or listed.
|
||||
- Tests gate the merge: privacy-engine matrix + an integration test that hits the
|
||||
public endpoints anonymously and asserts no living-person PII leaks.
|
||||
|
||||
## 8. Suggested phasing (small PRs)
|
||||
|
||||
1. Enum value + migration + regen client (+ dropdown → 4 options). No behavior
|
||||
change yet for non-members.
|
||||
2. Privacy-engine branching + unit tests.
|
||||
3. Public read API namespace (optional-auth, redacted schema, rate limit) + tests.
|
||||
4. Public frontend pages (`/p/...`) + robots/sitemap.
|
||||
5. In-app `/explore` directory + search.
|
||||
|
||||
Steps 2–3 are the privacy-critical core and should be reviewed hardest.
|
||||
|
||||
## 9. Open questions
|
||||
|
||||
- Caching: public pages are cacheable for SEO, but cache keys must not blur the
|
||||
redacted-vs-member rendering. Likely: cache only the anonymous projection at the
|
||||
edge; never cache member responses.
|
||||
- Do `site_members` trees appear in the sitemap for logged-in crawling? (Default:
|
||||
no — `noindex`.)
|
||||
- Per-tree opt-out of the directory even when `public`? (Probably unnecessary;
|
||||
`unlisted` already covers "reachable but not listed.")
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user