Why does a SavedModel exported without an input_signature reject a different batch size?

## Where the pinning comes from A SavedModel signature is not a loosely typed function. It is a `SignatureDef` holding, for each input, a name, a dtype, and a shape — and those values are copied from the concrete function you registered. A concrete function is a single specialization: one dtype and one shape per argument. So the question of what your served model accepts is decided entirely at export time, by whatever specs the concrete function carries. If you produced that concrete function by calling your `tf.function` on a real sample — `fn(sample_batch)` — the specs come from that sample, batch dimension included. ## What the server sees TF Serving loads the model, publishes the signature, and validates every incoming request against those specs. A request whose batch dimension differs produces an error response mentioning the expected and actual shape. Two things make this painful in practice: - **It loads cleanly.** Nothing warns you at deploy time; the failure is per-request. - **It often passes your smoke test.** If your smoke test sends the same batch size you happened to trace with, it succeeds, and the first real traffic with a different batch fails. A fixed batch size also destroys server-side batching: a batching-enabled server assembles requests into batches of varying size, and a model that only accepts one exact size cannot participate. ## Fixing it with TensorSpec The explicit spelling is the fix: `@tf.function(input_signature=[tf.TensorSpec(shape=[None, 10], dtype=tf.float32, name="features")])` Now the `tf.function` has exactly one form, it can be handed to `tf.saved_model.save(..., signatures={"serving_default": fn})` directly, and the exported spec has an unknown leading dimension. Equivalent: `fn.get_concrete_function(tf.TensorSpec([None, 10], tf.float32))`. Use `None` for every dimension that genuinely varies at inference — batch always, and sequence length or image height/width when your graph supports them. Do **not** blanket-`None` everything: a fully unknown shape prevents shape inference inside the graph, can force ops to be less efficient, and hides real bugs where a dimension should have been fixed. The `name=` on the spec is worth setting; it becomes the input key clients use in a columnar request, and an unnamed spec gets a generated name. ## dtype is the other half The spec pins dtype as well. Tracing on a NumPy array of Python floats gives `float64`, while most clients and most Keras models send `float32`; the resulting signature then rejects perfectly reasonable input. JSON requests to the REST endpoint are parsed to the signature's dtype, so a signature expecting `int64` will reject `1.0` and one expecting `float32` will accept `1`. Being deliberate about dtype in the `TensorSpec` avoids a whole class of confusing client-side errors. ## Keras models A Keras 3 model exported with `model.export(path)` derives its serving signature from the model's own input spec, which normally already has an unknown batch dimension — Keras builds layers with `batch_size=None` unless you told it otherwise. You get the fixed-shape problem back when you explicitly built the model with a fixed batch, or when you export a custom `tf.function` yourself with `keras.export.ExportArchive` and pass a concrete sample instead of a spec to `add_endpoint(..., input_signature=[...])`. ## The pre-ship check One command answers the question definitively: `saved_model_cli show --dir m/1 --all` Read the `inputs:` block. A free batch dimension prints as `(-1, 10)`; a pinned one prints as `(32, 10)`. Make this check part of the export job rather than a thing someone remembers to do, and assert on it: if the first dimension is not `-1`, fail the build. It costs nothing and catches the defect before an endpoint ever sees traffic. A second, stronger check is `saved_model_cli run` with an input of a *different* batch size than the one used in training — if the artifact is right, it succeeds locally, and you have proven the contract without deploying anything.