How do you embed a large document corpus with Cohere's Embed API without hitting limits?

## The hard limits Two caps shape any ingest job: 1. **Texts per call.** A single embed request takes at most **96** texts. Sending more is a 400, not an auto-split. Your batcher must chunk the work; the client will not do it for you. 2. **Tokens per text.** Each input is bounded by the model's context length. Embed v3 models cap around 512 tokens per input — short enough that ordinary paragraphs sometimes exceed it. Later Embed generations accept far longer inputs, which changes chunking strategy substantially, so pin the model and check its documented limit rather than assuming. ## truncate: choose it deliberately `truncate` takes `NONE`, `START` or `END`. `END` (the common default behaviour) keeps the beginning of an over-long text and discards the tail; `START` does the reverse; `NONE` returns an error instead of embedding a partial text. The senior move is to run **ingest with `NONE`**. Silent truncation is exactly the kind of defect that survives to production: your chunker emits a 900-token chunk, the API keeps the first 512, the tail is never searchable, and nothing anywhere reports it. Failing the call surfaces the bug at ingest time, where you can fix the chunker. Once the pipeline is proven, a permissive setting is a reasonable safety valve for pathological inputs — but only as a conscious choice with a metric counting how often it fires. ## Throughput Within the caps this is standard concurrent-client work: - **Fixed batches.** Group chunks into batches at or below the cap and keep the batch list around — response order is the only way to rejoin vectors to chunks, so the exact list you sent must survive until the response is consumed. - **Bounded concurrency.** A worker pool of a handful of in-flight requests, not an unbounded fan-out of every batch at once. Unbounded fan-out is how you convert a rate limit into a thundering herd of retries. - **429 handling.** Rate limits differ sharply between trial and production keys, so do not hardcode a rate — treat 429 as the signal, back off exponentially with jitter, and reduce concurrency adaptively. Retry 429 and 5xx; do not retry 400, which means your request is wrong and will stay wrong. - **Timeouts and idempotency.** A timed-out call may have been billed. Checkpoint by batch: record which chunk ranges are durably stored, and on restart resume from the checkpoint rather than from zero. Re-embedding a 10M-chunk corpus because a job died at 90% is a real and avoidable cost. ## Cost tracking `meta.billed_units.input_tokens` on every response gives exact billing. Sum it as you go and log it per batch; you get a live cost meter for the run and a sanity check on your token estimates before committing to the full corpus. Always pilot on a 1% sample first and extrapolate — both cost and wall-clock. ## Consistency across the run Every vector in one index must be produced with the same `model`, the same `input_type` (`search_document` for corpus text) and the same `embedding_types`. A run that changes any of them halfway — because someone bumped a model id mid-job, or a retry path used different defaults — leaves an index with two incompatible vector populations and no way to tell them apart afterwards. Pin these in one config object read once at job start, and record them as index metadata. Since a single call can return several `embedding_types` at one token cost, decide up front whether to capture a compressed form alongside float. Capturing it during ingest is nearly free; adding it later is a full re-embed. ## When to stop writing the loop For genuinely large corpora, Cohere offers an **asynchronous Embed Jobs** path: you upload the data as a dataset and submit a job that embeds it, rather than driving batches from your own process. That trades interactive control for not owning the retry, concurrency and checkpoint logic, and it is the right default when the corpus is a bulk backfill rather than a streaming ingest. Incremental, near-real-time ingest stays on the synchronous endpoint. ## What a strong answer sounds like Name the 96-text cap and the per-input token cap; make a deliberate `truncate` choice and justify it; describe backoff, bounded concurrency and checkpointing; mention consistency of model/input_type/embedding_types as an index invariant; and finish by noting the async job path exists for bulk work.