Fix #215: pluggable LLM + embedding provider abstraction
Adds the vendor-agnostic seam the AI assistant + match-ranking plug into: - LLMProvider / EmbeddingProvider ABCs (base.py). LLM and embeddings are SEPARATE abstractions — Anthropic has no embeddings endpoint, so each is configured independently and either can be off. - NullLLMProvider / NullEmbeddingProvider — the default; fail loud with a clear "not configured" error so AI-off deployments don't silently no-op. - AnthropicLLMProvider — first concrete LLM impl, via the official anthropic SDK (default model claude-opus-4-8). A local provider (e.g. Ollama) would be another subclass of the same interface. - Factory in deps.py (get_llm_provider / get_embedding_provider) selects by env (MODEL_PROVIDER / EMBEDDING_PROVIDER); documented in .env.example. Providers are read-only text/vector producers — they never touch the DB, so the "AI never writes autonomously" invariant (CLAUDE.md #1) holds; writes will go through ChangeProposal (#214). Tests: provider selection (null default, anthropic when keyed, fallback without key) + null providers raise. 81 passed. Closes #215 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Justin Paul <justin@jpaul.me>
This commit is contained in:
@@ -10,6 +10,8 @@ from app.core.db import get_session
|
||||
from app.integrations.mailer.base import Mailer
|
||||
from app.integrations.mailer.console import ConsoleMailer
|
||||
from app.integrations.mailer.smtp import SMTPMailer
|
||||
from app.integrations.models.base import EmbeddingProvider, LLMProvider
|
||||
from app.integrations.models.null import NullEmbeddingProvider, NullLLMProvider
|
||||
from app.integrations.objectstore.base import ObjectStore
|
||||
from app.integrations.objectstore.s3 import S3ObjectStore
|
||||
from app.models.user import User
|
||||
@@ -67,3 +69,28 @@ def get_objectstore() -> ObjectStore:
|
||||
|
||||
|
||||
ObjectStoreDep = Annotated[ObjectStore, Depends(get_objectstore)]
|
||||
|
||||
|
||||
def get_llm_provider() -> LLMProvider:
|
||||
settings = get_settings()
|
||||
if settings.model_provider == "anthropic" and settings.anthropic_api_key:
|
||||
from app.integrations.models.anthropic_provider import AnthropicLLMProvider
|
||||
|
||||
return AnthropicLLMProvider(
|
||||
api_key=settings.anthropic_api_key,
|
||||
model=settings.llm_model,
|
||||
max_tokens=settings.llm_max_tokens,
|
||||
)
|
||||
return NullLLMProvider()
|
||||
|
||||
|
||||
LLMProviderDep = Annotated[LLMProvider, Depends(get_llm_provider)]
|
||||
|
||||
|
||||
def get_embedding_provider() -> EmbeddingProvider:
|
||||
# Only the null provider exists today; concrete embedders (Ollama/Voyage)
|
||||
# implement the same interface and are selected here by settings.embedding_provider.
|
||||
return NullEmbeddingProvider()
|
||||
|
||||
|
||||
EmbeddingProviderDep = Annotated[EmbeddingProvider, Depends(get_embedding_provider)]
|
||||
|
||||
Reference in New Issue
Block a user