How much memory do 50 million 1536-dimension float32 embeddings require?

## The arithmetic An embedding is a fixed-length array of numbers. Its storage size is entirely mechanical: ``` bytes = number_of_vectors x dimensions x bytes_per_value ``` With 50 million documents, 1536 dimensions, and 32-bit floats (4 bytes each), that is 50,000,000 x 1536 x 4 = 307,200,000,000 bytes — about 307 GB in decimal units, or roughly 286 GiB. A single 1536-dimension float32 vector is 6,144 bytes; a 768-dimension one is 3,072 bytes. There is nothing subtle here, and interviewers ask it precisely because a candidate who cannot do this arithmetic will size a system wrong by an order of magnitude. ## Why the raw number is only the floor Three things sit on top of the raw vectors: **Index overhead.** A brute-force scan over 307 GB is not going to answer in tens of milliseconds, so you build an approximate-nearest-neighbour structure over the vectors. Every such structure adds its own bytes per vector — graph links, partition assignments, or centroid tables depending on the family. Treat this as a real multiplier, sized from the specific index you pick. **Payload and metadata.** Document ids, tenant ids, timestamps, and any attributes you filter on all live alongside the vectors, and filterable metadata often has to be resident too. **Replication and headroom.** One replica for availability doubles the figure. Rebuilds, compactions, and background merges want spare room on top. So a 307 GB raw corpus is realistically a 600 GB to 1 TB memory-tier commitment. At typical cloud memory prices that is a five-figure monthly line item, which is why dimension shows up in budget conversations and not just in model-selection ones. ## Where else the cost lands Engineers tend to think of dimension as a storage question. It touches four budgets at once: 1. **RAM.** Latency-sensitive vector search wants the searchable structure resident. This is usually the binding constraint, and RAM is the expensive tier. 2. **CPU per comparison.** Computing a dot product or Euclidean distance is one pass over d values. Doubling d roughly doubles the work of every candidate comparison the search performs. With an approximate index you do far fewer comparisons than a full scan, but each one still scales linearly in d. 3. **Index build and rebuild time.** Every distance computation during construction carries the same d factor, so a re-embedding or re-index job takes proportionally longer. 4. **Network and serialization.** Embeddings crossing a service boundary as JSON are dramatically worse than the binary figure — a float serialized as text can run 10 or more bytes per value. Batch ingestion pipelines and cross-region replication feel this immediately. ## Dimension as a dial Because all four scale together, dimension is the cheapest single lever you have on infrastructure cost, and unlike most levers it is continuous. Going from 1536 to 768 dimensions cuts the raw footprint of that 50M corpus from ~307 GB to ~154 GB and roughly halves comparison work. The question is always what quality you give up, and the honest answer is: measure it. Retrieval quality against dimension is usually **flat then cliff** — quality barely moves as you shrink, then falls off sharply past some corpus-specific point. Nobody can tell you where your cliff is from first principles; you sweep dimensions and plot recall@10 on your own labeled queries. ## Practical routes to a smaller footprint - **Pick a lower-dimension model.** Often a smaller modern encoder matches a larger older one at a fraction of the width. - **Shorten the vectors you already produce.** Models trained with nested representations let you keep a prefix of the dimensions; a general encoder can be projected down with a fitted linear reduction. Both need validation on your data. - **Two-pass retrieval.** Shortlist on short vectors, rescore the top candidates on full-width vectors. You pay full width only for a few hundred documents per query. - **Store fewer vectors.** Chunking strategy multiplies vector count as surely as dimension multiplies vector size; 50M vectors from 5M documents is a chunking decision, and it is often the larger lever. There are also compression schemes applied at the index layer that reduce bytes per value rather than the number of values — a separate axis from the dimension question, and one you evaluate on its own terms. ## What a good answer sounds like Do the multiplication out loud, state the ~300 GB figure, then immediately say it is the floor and name index overhead and replicas. Finish by pointing at RAM as the binding constraint. That progression — arithmetic, overhead, tier — is what the question is testing.