Resources / InferenceProvider
InferenceProvider
Where and how a model is called: one account or endpoint of a provider, with its credential. A project may have many, of the same type under different accounts. The platform checks the credential with a free call, lists the models it can use and keeps its health. An InferenceEndpoint names the providers it routes to.
Overview
| API | intelligence.etorm.com/v1alpha1 |
| Resource | inferenceproviders (cluster-scoped inside its cluster) |
| Short names | infprov |
| Available in | project clusters only |
| Status | written by a controller through /status |
Minimal example
apiVersion: intelligence.etorm.com/v1alpha1
kind: InferenceProvider
metadata:
name: example
spec:
type: anthropic
credentialRef:
name: "<name>"
Spec
| Field | Type | Required | Default | Allowed | Description |
|---|---|---|---|---|---|
type | string (one of) | yes | anthropic, openai, openai-compatible, openrouter, deepseek, gemini | anthropic (Messages API), openai (the OpenAI API with an API key), openai-compatible (any server with the OpenAI API at baseUrl: vLLM, Ollama, LM Studio and the like), openrouter, deepseek, gemini (Google Gemini API). A ChatGPT account is not a type: it has no programmatic access. | |
baseUrl | string | up to 512 characters | Overrides the type's address (https only). Required for openai-compatible. | ||
credentialRef | object | yes | The Secret holding the provider's API key. Setting it needs the use right on the secret. | ||
credentialRef.name | string | yes | length 1 to 63 | Name of the Secret in this project. | |
credentialRef.key | string | "api-key" | up to 63 characters | The key inside the secret that holds the API key. | |
organization | string | up to 128 characters | openai only: the organization id. | ||
projectId | string | up to 128 characters | openai only: the project id. | ||
api | string (one of) | "ChatCompletions" | ChatCompletions, Responses | openai only: which of OpenAI's chat APIs to call. | |
region | string | up to 64 characters | A note on where the provider processes data (for example eu, us); shown to people, not enforced. | ||
defaultModel | object | The model used when an endpoint names none for this provider. | |||
defaultModel.provider | string | up to 63 characters | Name of the ModelProvider. | ||
defaultModel.id | string | up to 128 characters | The model id in that catalog. | ||
allowedModels | list of string | up to 200 items | If set, only these models (the provider's own ids) may be called through this provider. | ||
modelMap | list of object | up to 200 items | Which catalog model a provider id is, for ids no vendor prefix explains (a model you serve yourself under your own name). | ||
modelMap[].id | string | yes | length 1 to 128 | The provider's own id. | |
modelMap[].modelProvider | string | yes | length 1 to 63 | Name of the ModelProvider. | |
modelMap[].model | string | yes | length 1 to 128 | The model's id in that catalog. | |
modelMap[].kind | string (one of) | chat, embedding | What it does, when the provider does not say. | ||
limits | object | {} | Limits on this account. | ||
limits.requestsPerMinute | integer | 1 to 100000 | Requests per minute through this provider, all endpoints together. | ||
limits.spendCapMicros | integer | at least 0 | Spend per UTC month beyond which the provider is not used, in millionths of a USD (0: no cap). Counted from published prices; the cap set at the provider itself is the hard stop. |
Status
| Field | Type | Required | Default | Allowed | Description |
|---|---|---|---|---|---|
phase | string | Pending, Ready, Degraded or Failed. | |||
modelCount | integer | Models the provider lists. | |||
models | list of object | What the provider says it can serve, matched to vendor catalogs where possible. | |||
models[].id | string | The provider's own id of the model. | |||
models[].modelProvider | string | The ModelProvider it was matched to. | |||
models[].model | string | The catalog model id it was matched to. | |||
models[].kind | string | chat or embedding, when known. | |||
models[].contextWindow | integer | Input tokens, when the provider says. | |||
models[].inputMicrosPerMillion | integer | Published price per million input tokens (millionths of a USD), when the provider says. | |||
models[].outputMicrosPerMillion | integer | Published price per million output tokens. | |||
health | object | The last check. | |||
health.checkedAt | string | When (RFC 3339). | |||
health.checkedGeneration | integer | The generation of the spec that was checked. | |||
health.latencyMs | integer | How long the list-models call took. | |||
health.consecutiveFailures | integer | Failed checks in a row. | |||
health.lastError | string | The last failure, without anything secret. | |||
usage | object | What was called through this provider. | |||
usage.month | string | The UTC month of the figures below (YYYY-MM). | |||
usage.requests | integer | Requests this month. | |||
usage.inputTokens | integer | Input tokens this month. | |||
usage.outputTokens | integer | Output tokens this month. | |||
usage.spendMicros | integer | Spend this month from published prices, millionths of a USD. | |||
conditions | list of object | Detailed state; Ready says whether the provider can be used. | |||
conditions[].type | string | Aspect. | |||
conditions[].status | string | True or False. | |||
conditions[].reason | string | Machine-readable reason. | |||
conditions[].message | string | Human-readable detail. |
Columns in listings
| Column | From |
|---|---|
| Type | .spec.type |
| Models | .status.modelCount |
| Latency | .status.health.latencyMs |
| Phase | .status.phase |
| Detail | .status.conditions[0].message |