On this page
Providers
Providers & API keys
Providers & API keys
okfsmith's LLM mode (ingest extraction, chat generative answers) works with any OpenAI-compatible endpoint. Out of the box, the default is local and free: Ollama. When you're ready for hosted models, pick one of the 15 presets and add an API key.
Quick setup: a hosted provider in 60 seconds
export OKFSMITH_PROVIDER=openrouter
export OKFSMITH_API_KEY=your-key-here
export OKFSMITH_MODEL=openai/gpt-4o-mini
okfsmith chat ./kb
That's it. Verify your key is detected (without ever seeing the key) with:
okfsmith doctor
The 15 provider presets
--provider accepts any of these. Each maps to the provider's OpenAI-compatible base URL:
| Preset | What it's for |
|---|---|
ollama | Local models via Ollama — the default, free, private |
lmstudio | Local models via LM Studio |
openai | OpenAI's API (GPT models) |
groq | Groq's fast inference API |
mistral | Mistral AI's API |
deepseek | DeepSeek's API |
openrouter | OpenRouter — one key, hundreds of models |
together | Together AI inference |
fireworks | Fireworks AI inference |
deepinfra | DeepInfra inference |
anyscale | Anyscale endpoints |
perplexity | Perplexity's API |
xai | xAI's API (Grok) |
gemini | Google's Gemini via its OpenAI-compatible endpoint |
agentrouter | Agent Router — gateway to many providers |
# Same command, any provider — change only the preset
okfsmith ingest ./kb report.pdf --provider groq --model llama-3.3-70b-versatile
Environment variables
| Variable | Purpose |
|---|---|
OKFSMITH_API_KEY | API key for the hosted endpoint (preferred) |
OKFSMITH_PROVIDER | Provider preset name (e.g. openrouter) |
OKFSMITH_API_BASE | Custom OpenAI-compatible base URL (see below) |
OKFSMITH_MODEL | Default model name (overridden by --model) |
AGENTROUTER_API_KEY | Honored when the provider resolves to agentrouter |
OPENAI_API_KEY | Legacy fallback for hosted OpenAI-compatible endpoints |
Example: Agent Router
export OKFSMITH_PROVIDER=agentrouter
export AGENTROUTER_API_KEY=your-key-here
okfsmith chat ./kb --model anthropic/claude-sonnet-4
Flags
| Flag | What it does |
|---|---|
--provider <name> | Choose a preset (e.g. openrouter) |
--model <id> | Model id for extraction/chat answers |
--api-base <url> | Any other OpenAI-compatible endpoint — overrides --provider |
--api-key <key> | Pass a key directly ⚠️ warns: this lands in your shell history |
Example: anything else via --api-base
Azure OpenAI, self-hosted vLLM or llama.cpp, any compat proxy — all work:
# keep the key in an env var — never as a flag (flags land in shell history)
export OKFSMITH_API_KEY=your-key-here
okfsmith ingest ./kb docs/ --api-base https://my-proxy.example.com/v1 \
--model my-model
Precedence order
When several sources are set, okfsmith resolves them in this order:
- Flags —
--provider,--model,--api-base,--api-key - Environment variables —
OKFSMITH_PROVIDER,OKFSMITH_MODEL,OKFSMITH_API_BASE,OKFSMITH_API_KEY - Defaults — Ollama at
http://localhost:11434/v1, modelqwen3:8b
So a flag always wins over an env var, which wins over the built-in default.
Ollama: the local default
With no keys and no flags, okfsmith expects Ollama on your machine:
# 1. Install Ollama from https://ollama.com, then pull a model
ollama pull qwen3:8b
# 2. Use okfsmith as-is — no keys, no env vars
okfsmith ingest ./kb report.pdf
doctor reports the Ollama reachability, provider, base URL, and model it's using — handy when something feels off (cli.html#okfsmith-doctor).
Anthropic's native API
Anthropic's own API is not OpenAI-compatible, so there is no --provider anthropic. To use Claude models with okfsmith, go through a compatible proxy or gateway:
# Option 1: OpenRouter
export OKFSMITH_PROVIDER=openrouter
export OKFSMITH_API_KEY=your-key-here
okfsmith chat ./kb --model anthropic/claude-sonnet-4
# Option 2: Agent Router
export OKFSMITH_PROVIDER=agentrouter
export AGENTROUTER_API_KEY=your-key-here