OpenAI-compatible API
Understand the de facto standard inference API and what compatibility does and does not guarantee.
Draft
Model interaction
Draft page
This page is an outline. It describes what will be covered and is not yet complete technical documentation.
A large share of inference servers and client libraries speak an OpenAI-compatible HTTP API. Treating it as a portability layer is usually right, but compatibility is a spectrum rather than a guarantee.
This page describes the surface conceptually. For the endpoints exposed by this team’s service, see the LLM Inference module User Guide.
What you will learn
- Which endpoints make up the commonly implemented core of the API.
- How chat completions differ from the older text completion shape.
- How streaming responses are delivered and consumed.
- Where compatible implementations tend to diverge in practice.
Recommended outline
This page is an outline. The subsections below are the planned structure; they are filled in as the handbook is written.