Providers API

Providers are the backend adapters that agents and knowledge features use. There are four provider families: LLM providers (for chat/completion), embedding providers (for vector search), cross-encoder providers (for re-ranking), and semantic-search providers (pgvector, pgvectorscale, Lance).

How providers relate to tools and toolsets.

Configure an agent to use a provider.

Provider families and base paths

LLM providers

Endpoints

LLM provider object

{
  "id": "anthropic-prod",
  "provider": "anthropic",
  "models": [
    {"name": "claude-sonnet-4-6", "context_length": 200000}
  ],
  "config": {
    "api_key": "***"
  },
  "limits": {
    "max_concurrency": 4
  }
}

Note on secrets: config.api_key (and equivalent fields for other providers) is never returned in plaintext. The GET and LIST responses replace the value with a masked string. The field is always present (not omitted) so you can confirm a key was stored.

LLM stream timeout

limits.request_timeout_seconds is a per-event inactivity timeout, not a total-generation cap. A long but progressing response is not killed; only a complete stall (no new event arriving) triggers it. The default 300 s covers most real-world runs. For local servers such as LM Studio on slower hardware you can lower this (e.g. 60 s) for faster failure detection; raise or disable (null) for very large models that have long gaps between tokens.

Create an LLM provider

POST /v1/llm_providers - returns 201 Created.

curl -X POST https://your-host/v1/llm_providers \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "anthropic-prod",
    "provider": "anthropic",
    "models": [{"name": "claude-sonnet-4-6", "context_length": 200000}],
    "config": {"api_key": "sk-ant-..."},
    "limits": {"max_concurrency": 4}
  }'

import httpx
r = httpx.post(
    "https://your-host/v1/llm_providers",
    headers={"Authorization": f"Bearer {token}"},
    json={
        "id": "anthropic-prod",
        "provider": "anthropic",
        "models": [{"name": "claude-sonnet-4-6", "context_length": 200000}],
        "config": {"api_key": "sk-ant-..."},
        "limits": {"max_concurrency": 4},
    },
)
assert r.status_code == 201

const r = await fetch("/v1/llm_providers", {
  method: "POST",
  headers: {"Authorization": `Bearer ${token}`, "Content-Type": "application/json"},
  body: JSON.stringify({
    id: "anthropic-prod",
    provider: "anthropic",
    models: [{name: "claude-sonnet-4-6", context_length: 200000}],
    config: {api_key: "sk-ant-..."},
    limits: {max_concurrency: 4}
  })
})

Errors: 422 if models is empty or config shape does not match provider. 409 on duplicate id.

Get configured model names

GET /v1/llm_providers/{id}/models returns the model names stored in the provider row. This endpoint is row-cached - it never calls the upstream LLM. It reflects the models list you configured, not a live discovery call.

curl https://your-host/v1/llm_providers/anthropic-prod/models \
  -H "Authorization: Bearer $TOKEN"

import httpx
r = httpx.get("https://your-host/v1/llm_providers/anthropic-prod/models",
              headers={"Authorization": f"Bearer {token}"})
print(r.json())  # {"models": ["claude-sonnet-4-6"]}

const r = await fetch("/v1/llm_providers/anthropic-prod/models", {
  headers: {"Authorization": `Bearer ${token}`}
})
const {models} = await r.json()

Response 200 OK:

{"models": ["claude-sonnet-4-6"]}

Errors: 404 if the provider id does not exist.

Invalidate cached adapter

POST /v1/llm_providers/{id}/invalidate drops the in-memory adapter for the given provider. Call this after a PUT to ensure the next request picks up the updated config. Returns 204 No Content - including when the id does not exist (no-op).

curl -X POST https://your-host/v1/llm_providers/anthropic-prod/invalidate \
  -H "Authorization: Bearer $TOKEN"

import httpx
r = httpx.post("https://your-host/v1/llm_providers/anthropic-prod/invalidate",
               headers={"Authorization": f"Bearer {token}"})
assert r.status_code == 204

await fetch("/v1/llm_providers/anthropic-prod/invalidate", {
  method: "POST",
  headers: {"Authorization": `Bearer ${token}`}
})

Embedding providers

Endpoints

Embedding provider object

{
  "id": "hf-embedder",
  "provider": "huggingface",
  "models": [
    {"name": "sentence-transformers/all-MiniLM-L6-v2"},
    {"name": "sentence-transformers/all-mpnet-base-v2"}
  ],
  "config": {"token": "***"},
  "limits": {"max_concurrency": 2}
}

Create an embedding provider

curl -X POST https://your-host/v1/embedding_providers \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "hf-embedder",
    "provider": "huggingface",
    "models": [{"name": "sentence-transformers/all-MiniLM-L6-v2"}],
    "config": {"token": "hf-..."},
    "limits": {"max_concurrency": 2}
  }'

import httpx
r = httpx.post(
    "https://your-host/v1/embedding_providers",
    headers={"Authorization": f"Bearer {token}"},
    json={
        "id": "hf-embedder",
        "provider": "huggingface",
        "models": [{"name": "sentence-transformers/all-MiniLM-L6-v2"}],
        "config": {"token": "hf-..."},
        "limits": {"max_concurrency": 2},
    },
)

await fetch("/v1/embedding_providers", {
  method: "POST",
  headers: {"Authorization": `Bearer ${token}`, "Content-Type": "application/json"},
  body: JSON.stringify({
    id: "hf-embedder",
    provider: "huggingface",
    models: [{name: "sentence-transformers/all-MiniLM-L6-v2"}],
    config: {token: "hf-..."},
    limits: {max_concurrency: 2}
  })
})

The /models and /invalidate sub-resources work identically to the LLM provider equivalents. After a PUT, call /invalidate to force the runtime to reload the updated config.

Cross-encoder providers

Endpoints

Only the huggingface backend is supported. Object shape follows the same pattern as embedding providers (id, provider, models, config, limits). The id is optional on create: omit it and the server assigns a type-prefixed id (e.g. cross-encoder-provider-3f9a1c8d); it is immutable after creation.

curl -X POST https://your-host/v1/cross_encoder_providers \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "hf-reranker",
    "provider": "huggingface",
    "models": [{"name": "cross-encoder/ms-marco-MiniLM-L-6-v2"}],
    "config": {"token": "hf-..."},
    "limits": {"max_concurrency": 1}
  }'

import httpx
r = httpx.post(
    "https://your-host/v1/cross_encoder_providers",
    headers={"Authorization": f"Bearer {token}"},
    json={
        "id": "hf-reranker",
        "provider": "huggingface",
        "models": [{"name": "cross-encoder/ms-marco-MiniLM-L-6-v2"}],
        "config": {"token": "hf-..."},
        "limits": {"max_concurrency": 1},
    },
)

await fetch("/v1/cross_encoder_providers", {
  method: "POST",
  headers: {"Authorization": `Bearer ${token}`, "Content-Type": "application/json"},
  body: JSON.stringify({
    id: "hf-reranker",
    provider: "huggingface",
    models: [{name: "cross-encoder/ms-marco-MiniLM-L-6-v2"}],
    config: {token: "hf-..."},
    limits: {max_concurrency: 1}
  })
})

Semantic-search providers

Endpoints

Supported provider values: pgvector, pgvectorscale, lance. The id is optional on create: omit it and the server assigns a type-prefixed id (e.g. semantic-search-provider-3f9a1c8d); it is immutable after creation.

curl -X POST https://your-host/v1/ssp \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "pgvec-main",
    "provider": "pgvector",
    "config": {"url": "postgresql://user:pass@db:5432/primer"}
  }'

import httpx
r = httpx.post(
    "https://your-host/v1/ssp",
    headers={"Authorization": f"Bearer {token}"},
    json={
        "id": "pgvec-main",
        "provider": "pgvector",
        "config": {"url": "postgresql://user:pass@db:5432/primer"},
    },
)

await fetch("/v1/ssp", {
  method: "POST",
  headers: {"Authorization": `Bearer ${token}`, "Content-Type": "application/json"},
  body: JSON.stringify({
    id: "pgvec-main",
    provider: "pgvector",
    config: {url: "postgresql://user:pass@db:5432/primer"}
  })
})

Secret handling

API keys and tokens submitted in config are stored encrypted. GET and LIST responses always include the api_key (or equivalent) field but never return the plaintext value - the field contains a masked string. This applies to all provider families.

Errors note

All error responses use the RFC 7807 ProblemDetails envelope. Common status codes:

• 422 - validation failed (e.g. models list is empty, or config shape does not match provider)
• 409 - duplicate id
• 404 - provider not found (also returned by GET /models; note that POST /invalidate returns 204 even for missing ids)
• 204 - successful delete or invalidate (no body)