OpenAI

A POST to /v1/embeddings sends a model plus input (a string or an array of strings) and returns a data array of vectors, each carrying its embedding and the index of the input it came from, plus a usage object counting input tokens.

OpenAI bills per token, quoted per million tokens, and output tokens cost several times more than input tokens. The API is stateless, so every turn resends the whole conversation as input — the input bill grows with each turn.

previous_response_id points at a stored earlier response, so the server replays that whole conversation as the prefix of the new request. You send only the new turn, but you are still billed for every replayed input token.

Upload the files, add them to a vector store, and wait for indexing to finish, then reference that store's id from the file_search tool on your request. OpenAI handles chunking, embedding, and ranking; you pay for storage per day.

The endpoint stores nothing between calls: there is no conversation id and no server-side memory. Your application keeps the transcript and re-sends the whole messages array every turn, appending each new user message and each returned assistant message.

Nothing errors — both are applied to the same sampling step, and OpenAI's own guidance is to change one or the other, not both, because their combined effect is hard to predict. Defaults are temperature 1 and top_p 1.

Passing dimensions asks a text-embedding-3-small or text-embedding-3-large call to return a shorter vector — for example 256 instead of 3072. The models are trained so the leading coordinates carry the most information, so quality degrades gradually rather than collapsing.

Pick text-embedding-3-large when retrieval quality is the bottleneck and the corpus is small enough that its higher per-token price and 3072-dimensional vectors are affordable. text-embedding-3-small is the sane default: far cheaper, 1536 dimensions, and close on benchmark quality.

Retry it with exponential backoff plus random jitter and a capped attempt count, using the x-ratelimit-reset headers to time the wait. First check the error code: rate_limit_exceeded is transient and worth retrying, while insufficient_quota is a billing problem no retry will clear.

OpenAI enforces several limits at once — requests per minute, tokens per minute, and daily caps — per model and per project. Exceeding any single one returns 429, so a handful of very large prompts can exhaust the token budget while the request count stays trivial.

json_object only promises the reply parses as JSON — which keys appear and what types they hold is left to the prompt. json_schema with strict true constrains decoding to a schema you supply, so the returned object provably matches that shape.

Strict mode accepts a restricted JSON Schema subset: every object must set additionalProperties to false and list every one of its properties in required. Optionality is expressed by making the type a union with null, never by leaving the key out of required.

You send a tools array whose entries describe a function with a JSON Schema. The model replies with message.tool_calls instead of prose; your code runs each call, then appends a role 'tool' message carrying the matching tool_call_id.

tool_choice auto lets the model decide and is the default when tools are present; required forces at least one tool call this turn; none forbids calling while still sending the definitions; and a named function object pins the call to that one function.

parse() accepts a Pydantic model as response_format, converts it into a strict JSON Schema for you, and hands back the reply already deserialised into an instance of that model on message.parsed — plus the separate refusal field to check first.

A run is created as queued, moves to in_progress, and normally ends completed. It can pause at requires_action while it waits for tool outputs, and can instead end as failed, cancelled, incomplete, or expired.

Set stream to true and iterate the Server-Sent Events response. Each event is a chat.completion.chunk whose choices[0].delta holds the newest fragment; concatenate those fragments yourself, and stop when the stream sends its final done marker.

OpenAI embeddings are returned normalised to unit length, so cosine similarity reduces to a plain dot product and gives exactly the same ranking as Euclidean distance. Cosine is recommended because it is the cheapest and least error-prone of the equivalent choices.

Get the encoding for your model with tiktoken.encoding_for_model, then take len(encoding.encode(text)) for each message. Add the per-message framing overhead, because raw text encoding alone undercounts a chat request and ignores tool schemas and images entirely.

One assistant message can carry several entries in tool_calls; run them concurrently, then append one tool message per tool_call_id before the next request. Set parallel_tool_calls to false when calls must be ordered or have side effects.

Stream when a human is waiting on the text, because polling only reveals the answer after the run finishes. Streaming does not remove the tool-output round trip, the one-active-run-per-thread rule, or the need to re-read the run after a dropped connection.

max_completion_tokens is the current parameter and caps everything the model generates, including the invisible reasoning tokens on reasoning models. max_tokens is the older, deprecated name for the output cap and is not accepted by the newer reasoning models.

Chunk documents under the model's 8192-token input limit, pack many chunks per request as an array (up to 2048 entries), reassemble results by the index field, and for very large corpora submit the work through the asynchronous Batch API instead of live calls.

Stop relying on retries and add admission control: a shared token-bucket limiter sized to the tightest limit, so work waits in your queue instead of bouncing off the API. Then reduce demand — realistic completion caps, smaller models for cheap traffic, bulk work moved to the Batch API.

The guarantee covers completed generations only. A refusal returns a refusal string instead of content, a generation cut off by the token limit returns a valid prefix that is not valid JSON, and content filtering or transport errors return no usable body at all.

Loop until a response has no tool_calls and finish_reason is stop, and bound it: a maximum step count, a token or cost budget, per-tool timeouts, and duplicate-call detection. Force a prose answer with tool_choice none on the final step.

With stream enabled, tool calls arrive as fragments in delta.tool_calls. Each fragment has an index; the first for that index carries the id and function name, later ones carry pieces of the arguments string you concatenate before parsing.

Hosted state is wrong when storage is contractually forbidden, when transcripts must be your own for audit, evaluation or portability, or when you need explicit control over what the model sees each turn. It saves client complexity, never token cost.

There is no default — the API errors rather than trimming for you, so the policy is yours. Choose between a sliding window, rolling summarization, and retrieving only relevant past turns, budgeting tokens against the window and leaving room for the answer.