Fix #215: pluggable LLM + embedding provider abstraction

Adds the vendor-agnostic seam the AI assistant + match-ranking plug into:
- LLMProvider / EmbeddingProvider ABCs (base.py). LLM and embeddings are
  SEPARATE abstractions — Anthropic has no embeddings endpoint, so each is
  configured independently and either can be off.
- NullLLMProvider / NullEmbeddingProvider — the default; fail loud with a clear
  "not configured" error so AI-off deployments don't silently no-op.
- AnthropicLLMProvider — first concrete LLM impl, via the official anthropic SDK
  (default model claude-opus-4-8). A local provider (e.g. Ollama) would be
  another subclass of the same interface.
- Factory in deps.py (get_llm_provider / get_embedding_provider) selects by
  env (MODEL_PROVIDER / EMBEDDING_PROVIDER); documented in .env.example.

Providers are read-only text/vector producers — they never touch the DB, so the
"AI never writes autonomously" invariant (CLAUDE.md #1) holds; writes will go
through ChangeProposal (#214).

Tests: provider selection (null default, anthropic when keyed, fallback without
key) + null providers raise. 81 passed.

Closes #215

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Justin Paul <justin@jpaul.me>
This commit is contained in:
2026-06-09 12:51:01 -04:00
parent d540dc3f32
commit 330543f9ce
10 changed files with 281 additions and 0 deletions
+27
View File
@@ -10,6 +10,8 @@ from app.core.db import get_session
from app.integrations.mailer.base import Mailer
from app.integrations.mailer.console import ConsoleMailer
from app.integrations.mailer.smtp import SMTPMailer
from app.integrations.models.base import EmbeddingProvider, LLMProvider
from app.integrations.models.null import NullEmbeddingProvider, NullLLMProvider
from app.integrations.objectstore.base import ObjectStore
from app.integrations.objectstore.s3 import S3ObjectStore
from app.models.user import User
@@ -67,3 +69,28 @@ def get_objectstore() -> ObjectStore:
ObjectStoreDep = Annotated[ObjectStore, Depends(get_objectstore)]
def get_llm_provider() -> LLMProvider:
settings = get_settings()
if settings.model_provider == "anthropic" and settings.anthropic_api_key:
from app.integrations.models.anthropic_provider import AnthropicLLMProvider
return AnthropicLLMProvider(
api_key=settings.anthropic_api_key,
model=settings.llm_model,
max_tokens=settings.llm_max_tokens,
)
return NullLLMProvider()
LLMProviderDep = Annotated[LLMProvider, Depends(get_llm_provider)]
def get_embedding_provider() -> EmbeddingProvider:
# Only the null provider exists today; concrete embedders (Ollama/Voyage)
# implement the same interface and are selected here by settings.embedding_provider.
return NullEmbeddingProvider()
EmbeddingProviderDep = Annotated[EmbeddingProvider, Depends(get_embedding_provider)]