Resources / InferenceProvider

InferenceProvider

Where and how a model is called: one account or endpoint of a provider, with its credential. A project may have many, of the same type under different accounts. The platform checks the credential with a free call, lists the models it can use and keeps its health. An InferenceEndpoint names the providers it routes to.

Overview

APIintelligence.etorm.com/v1alpha1
Resourceinferenceproviders (cluster-scoped inside its cluster)
Short namesinfprov
Available inproject clusters only
Statuswritten by a controller through /status

Minimal example

apiVersion: intelligence.etorm.com/v1alpha1
kind: InferenceProvider
metadata:
  name: example
spec:
  type: anthropic
  credentialRef:
    name: "<name>"

Spec

FieldTypeRequiredDefaultAllowedDescription
typestring (one of)yesanthropic, openai, openai-compatible, openrouter, deepseek, geminianthropic (Messages API), openai (the OpenAI API with an API key), openai-compatible (any server with the OpenAI API at baseUrl: vLLM, Ollama, LM Studio and the like), openrouter, deepseek, gemini (Google Gemini API). A ChatGPT account is not a type: it has no programmatic access.
baseUrlstringup to 512 charactersOverrides the type's address (https only). Required for openai-compatible.
credentialRefobjectyesThe Secret holding the provider's API key. Setting it needs the use right on the secret.
credentialRef.namestringyeslength 1 to 63Name of the Secret in this project.
credentialRef.keystring"api-key"up to 63 charactersThe key inside the secret that holds the API key.
organizationstringup to 128 charactersopenai only: the organization id.
projectIdstringup to 128 charactersopenai only: the project id.
apistring (one of)"ChatCompletions"ChatCompletions, Responsesopenai only: which of OpenAI's chat APIs to call.
regionstringup to 64 charactersA note on where the provider processes data (for example eu, us); shown to people, not enforced.
defaultModelobjectThe model used when an endpoint names none for this provider.
defaultModel.providerstringup to 63 charactersName of the ModelProvider.
defaultModel.idstringup to 128 charactersThe model id in that catalog.
allowedModelslist of stringup to 200 itemsIf set, only these models (the provider's own ids) may be called through this provider.
modelMaplist of objectup to 200 itemsWhich catalog model a provider id is, for ids no vendor prefix explains (a model you serve yourself under your own name).
modelMap[].idstringyeslength 1 to 128The provider's own id.
modelMap[].modelProviderstringyeslength 1 to 63Name of the ModelProvider.
modelMap[].modelstringyeslength 1 to 128The model's id in that catalog.
modelMap[].kindstring (one of)chat, embeddingWhat it does, when the provider does not say.
limitsobject{}Limits on this account.
limits.requestsPerMinuteinteger1 to 100000Requests per minute through this provider, all endpoints together.
limits.spendCapMicrosintegerat least 0Spend per UTC month beyond which the provider is not used, in millionths of a USD (0: no cap). Counted from published prices; the cap set at the provider itself is the hard stop.

Status

FieldTypeRequiredDefaultAllowedDescription
phasestringPending, Ready, Degraded or Failed.
modelCountintegerModels the provider lists.
modelslist of objectWhat the provider says it can serve, matched to vendor catalogs where possible.
models[].idstringThe provider's own id of the model.
models[].modelProviderstringThe ModelProvider it was matched to.
models[].modelstringThe catalog model id it was matched to.
models[].kindstringchat or embedding, when known.
models[].contextWindowintegerInput tokens, when the provider says.
models[].inputMicrosPerMillionintegerPublished price per million input tokens (millionths of a USD), when the provider says.
models[].outputMicrosPerMillionintegerPublished price per million output tokens.
healthobjectThe last check.
health.checkedAtstringWhen (RFC 3339).
health.checkedGenerationintegerThe generation of the spec that was checked.
health.latencyMsintegerHow long the list-models call took.
health.consecutiveFailuresintegerFailed checks in a row.
health.lastErrorstringThe last failure, without anything secret.
usageobjectWhat was called through this provider.
usage.monthstringThe UTC month of the figures below (YYYY-MM).
usage.requestsintegerRequests this month.
usage.inputTokensintegerInput tokens this month.
usage.outputTokensintegerOutput tokens this month.
usage.spendMicrosintegerSpend this month from published prices, millionths of a USD.
conditionslist of objectDetailed state; Ready says whether the provider can be used.
conditions[].typestringAspect.
conditions[].statusstringTrue or False.
conditions[].reasonstringMachine-readable reason.
conditions[].messagestringHuman-readable detail.

Columns in listings

ColumnFrom
Type.spec.type
Models.status.modelCount
Latency.status.health.latencyMs
Phase.status.phase
Detail.status.conditions[0].message

All resources