BETAv0.24.11
GitHub Install
Providers and models
Docs/Use/Providers and models

Providers and models

Edit on GitHub
docs-site/use/providers.md

A provider is where the model runs: a hosted API (Anthropic, OpenAI, Gemini), a local server (Ollama, vLLM) or any OpenAI-compatible endpoint. Each provider is an extension, and your build lists the ones it has:

orrery providers
PROVIDER       KIND           KEY          SET UP  MODELS  WHAT IT IS
gemini         gemini         missing                      Google Gemini
ollama         ollama         none needed  in use  1       Ollama, on its own machine
openai-compat  openai-compat  none needed                  Any OpenAI-compatible endpoint

KEY says where the key comes from (in config.toml, $VAR, from GEMINI_API_KEY, signed in, missing, none needed), never the key itself.

Set one up

Three routes, one result. Whichever you take, what is saved is keys in ~/.orrery/config.toml.

In the TUI: type /providers, pick one, answer its steps.

From a script:

orrery providers add gemini --api-key '$GEMINI_API_KEY' --model gemini-2.5-flash --use
orrery providers add openai-compat --name groq --base-url https://api.groq.com/openai/v1 --api-key '$GROQ_API_KEY' --model llama-3.3-70b
orrery providers add ollama --set contextLength=16384
echo "$KEY" | orrery providers add anthropic --api-key-stdin --model claude-sonnet-4-5
FlagMeaning
--name <name>A name for a second setup of the same kind. Defaults to the kind.
--base-url <url>The endpoint, when not the provider's own.
--api-key <KEY|$VAR>The key, or $VAR to keep only the variable's name in the file.
--api-key-stdinRead the key from stdin, so it never shows in a process list.
--model <model>The model to use with it.
--useAlso make it the one in use.
--set key=valueA setting the provider declares. Repeatable.
--no-verifySave without asking the provider first.

By hand:

[providers]
default = "groq"                   # the setup in use

[providers.gemini]
apiKey = "$GEMINI_API_KEY"         # a variable's name; the file never holds the key
model = "gemini-2.5-flash"

[providers.groq]
kind = "openai-compat"             # a second setup of a provider
baseUrl = "https://api.groq.com/openai/v1"
apiKey = "$GROQ_API_KEY"
model = "llama-3.3-70b"

[providers.ollama]
model = "qwen3:4b"
models = ["qwen3:8b"]              # extra models /model offers

A [providers.<name>] table is one setup. Its kind defaults to the table name. A literal key (apiKey = "sk-...") is accepted only in the user layer, never in a workspace file.

Where a key comes from

First match wins:

  1. the table's apiKey;
  2. a sign-in stored in ~/.orrery/credentials (OAuth);
  3. the variables the provider declares, such as GEMINI_API_KEY (only for the provider's own table, not a named setup).
orrery providers status          # exits 0 when every setup is ready, 5 when one needs a key
orrery providers remove groq     # the table, and any stored sign-in

Pick a model

orrery models                            # every model across your setups, in one list
orrery models add ollama qwen3:8b --use  # add a model id, save it, use it

In the TUI, /model shows the same list. Pick one and the same conversation carries on with it.

For one run only, without saving: --provider <id>:<model>[@<url>].

orrery --provider ollama:qwen3:4b run -p "hello"
orrery --provider openai-compat:my-model@http://localhost:8080/v1 run -p "hello"

--provider ? lists the ids this build has.

Precedence. Provider: --provider, then [providers] default. Model: the --provider spec, then a top-level model =, then the setup's own model.

How hard it thinks

One key on any provider:

[providers.anthropic]
reasoning = "medium"   # off | low | medium | high

Each provider maps it to its own parameter. Providers without one (openai-compat, vllm) accept the key and ignore it.

Rate limits

A 429 or quota error is retried with back-off, 3 attempts by default ([retry] maxAttempts). Then the run stops and says when to try again.