Model providers: OpenAI/xAI/Ollama + run several at once (registry)
Extends the #215 abstraction: - OpenAICompatibleLLMProvider / OpenAICompatibleEmbeddingProvider — one impl (via the official openai SDK) covers OpenAI, xAI (api.x.ai/v1), Ollama (…:11434/v1), OpenRouter, etc.; they differ only by base_url, key, and model. - Registry factory: build_llm_providers() / build_embedding_providers() return every provider whose credentials are configured, so you can run several concurrently. get_llm_provider(name)/get_embedding_provider(name) select by name, falling back to default_*_provider, then Null. - Per-provider env config (ANTHROPIC_*, OPENAI_*, XAI_*, OLLAMA_*) + DEFAULT_LLM_PROVIDER / DEFAULT_EMBEDDING_PROVIDER; documented in .env.example. Defaults keep AI off (empty registry). Embeddings now have real backends (OpenAI/Ollama), still separate from the LLM since Anthropic offers no embeddings endpoint. Tests cover multi-provider selection, default resolution, disabled-without-credentials, and null fail-loud. Full suite 87 passed. Relates to #215. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Justin Paul <justin@jpaul.me>
This commit is contained in:
+28
-10
@@ -57,16 +57,34 @@ SMTP_USERNAME=
|
||||
SMTP_PASSWORD=
|
||||
SMTP_FROM=
|
||||
|
||||
# --- Model providers (AI assistant + embeddings; both optional, default off) ---
|
||||
# LLM: 'null' disables AI features; 'anthropic' uses the Claude API.
|
||||
MODEL_PROVIDER=null
|
||||
ANTHROPIC_API_KEY=
|
||||
LLM_MODEL=claude-opus-4-8
|
||||
# --- Model providers (AI assistant + embeddings) -----------------------------
|
||||
# Configure as many as you like — each turns on when its key is set. The
|
||||
# default_* vars pick which one is used by default; the app can also select any
|
||||
# configured provider by name. LLM and embeddings are independent (Anthropic has
|
||||
# no embeddings endpoint). Leave the defaults 'null' to keep AI off.
|
||||
DEFAULT_LLM_PROVIDER=null # null | anthropic | openai | xai | ollama
|
||||
DEFAULT_EMBEDDING_PROVIDER=null # null | openai | ollama
|
||||
LLM_MAX_TOKENS=4096
|
||||
# Embeddings are separate (Anthropic has no embeddings endpoint). 'null' for now.
|
||||
EMBEDDING_PROVIDER=null
|
||||
EMBEDDING_DIMENSIONS=1536 # must match the embedding model + pgvector column
|
||||
|
||||
# --- Model providers — wired in Phase 4 (AI assistant). BYO key. ---
|
||||
# ANTHROPIC_API_KEY=
|
||||
# OPENAI_API_KEY=
|
||||
# Anthropic (LLM)
|
||||
ANTHROPIC_API_KEY=
|
||||
ANTHROPIC_MODEL=claude-opus-4-8
|
||||
|
||||
# OpenAI (LLM + embeddings)
|
||||
OPENAI_API_KEY=
|
||||
OPENAI_BASE_URL=https://api.openai.com/v1
|
||||
OPENAI_MODEL=gpt-4o
|
||||
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
|
||||
|
||||
# xAI / Grok — OpenAI-compatible (LLM)
|
||||
XAI_API_KEY=
|
||||
XAI_BASE_URL=https://api.x.ai/v1
|
||||
XAI_MODEL=grok-2-latest # set to your account's current Grok model
|
||||
|
||||
# Ollama — local, OpenAI-compatible, no key (LLM + embeddings)
|
||||
OLLAMA_ENABLED=false
|
||||
OLLAMA_BASE_URL=http://localhost:11434/v1
|
||||
OLLAMA_MODEL=llama3.1
|
||||
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
|
||||
# XAI_API_KEY=
|
||||
|
||||
Reference in New Issue
Block a user