AI Gateway API

The AI Gateway API provides model discovery, inference, embeddings, and image generation. Clients choose a request format and model while project policy constrains eligible deployments.

Models and deployments

A model describes capabilities and accepted inputs. A deployment identifies where that model is served, including its residency, precision, retention, and pricing metadata.

The public catalog exposes model and deployment metadata. Authenticated discovery can reflect a project's inference policy. A listed model may not support every deployment or protocol for your request.

Request formats

Inference endpoints support provider-compatible formats, including OpenAI and Anthropic formats. The provider gateway forwards requests using a provider-specific path. The endpoint reference defines supported paths, parameters, and response handling.

Image generation produces project uploads. The Projects API manages those stored assets. Embedding generation produces vectors. The Knowledge API stores and searches indexed vectors.

An application can request a model response directly or use inference within an agent run. The Execution API records the agent run and its progress.

A model deployment identifies where inference is served. It is distinct from an application deployment, which assigns a project release to an environment.

Policies and spending

An account's model policy applies to all its projects. A project policy can only narrow that selection. Serving restrictions also depend on the deployment configuration and stored project policy. Deployments that retain the EU-only default apply it alongside model rules. Changing a model default or account model rules does not remove a stored project tier restriction.

Choose the models a project uses describes model rules. Use models served outside the EU explains serving restrictions.

Budgets and usage belong to the Billing and Usage API. Authentication and API key ownership belong to the Identity and Access API.

Get started

Generate a model response: discover a model and send a chat-completion request.

API references