In tf.data, how do Dataset.from_tensors and Dataset.from_tensor_slices differ?
answer
- one element versus many
- the leading axis is the difference
- cardinality 1 versus N
- tuples slice in lockstep
- element_spec tells you which you built
basics
~10 sDataset.from_tensors makes a one-element dataset holding the whole tensor as a single item. Dataset.from_tensor_slices splits along the first axis, yielding one element per row. Training on N examples almost always wants from_tensor_slices.
solid answer
~40 sBoth build an in-memory `tf.data.Dataset`, but they disagree about what one element is. `Dataset.from_tensors(x)` treats the entire tensor as a single element: given an array of shape `(100, 4)` you get a dataset of cardinality 1 whose `element_spec` is shape `(100, 4)`. `Dataset.from_tensor_slices(x)` slices along axis 0: the same array gives 100 elements of shape `(4,)`. Passing a tuple works component-wise — `from_tensor_slices((features, labels))` yields `(x, y)` pairs and requires every component to have the same size on axis 0. In practice you use `from_tensor_slices` for a training set and `from_tensors` only when you deliberately want one fixed blob, for example `Dataset.from_tensors(batch).repeat()` to feed the same batch over and over in a benchmark or an overfit-one-batch sanity check.
code
python · 15 linesimport numpy as np
import tensorflow as tf
x = np.zeros((100, 4), dtype=np.float32)
y = np.zeros((100,), dtype=np.int32)
whole = tf.data.Dataset.from_tensors(x)
print(whole.element_spec, whole.cardinality().numpy()) # (100, 4) 1
rows = tf.data.Dataset.from_tensor_slices(x)
print(rows.element_spec, rows.cardinality().numpy()) # (4,) 100
pairs = tf.data.Dataset.from_tensor_slices((x, y))
for xb, yb in pairs.take(1):
print(xb.shape, yb.shape) # (4,) ()go deeper
Be able to say plainly that from_tensor_slices gives one element per row while from_tensors gives one element total, and that supervised training data normally uses from_tensor_slices((x, y)).
Explain the lockstep slicing rule for tuples and the matching-first-axis requirement, and show how element_spec and cardinality() let you verify which one you built before training.
Point out that neither constructor streams — the array lives in memory and can be captured into the graph — so at real dataset sizes you move to TFRecord or a file-listing source instead.
Frame it as an input-contract choice: where the example boundary is defined determines what shuffling, sharding across workers and caching can do later, so it is decided when the data format is designed, not at the call site.
## What a Dataset element is Everything in `tf.data` is defined in terms of *elements*. A `tf.data.Dataset` is a sequence of elements, and every transformation (`map`, `filter`, `batch`, `shuffle`) operates on one element at a time. So the only question these two constructors answer is: when I hand you this NumPy array or tensor, what counts as one element? ## from_tensors: the whole thing is one element `tf.data.Dataset.from_tensors(x)` produces a dataset with exactly one element, and that element is `x` itself, shape unchanged. ``` import numpy as np, tensorflow as tf ds = tf.data.Dataset.from_tensors(np.zeros((100, 4), dtype=np.float32)) ds.element_spec # TensorSpec(shape=(100, 4), dtype=float32) ds.cardinality() # 1 ``` If you pass a tuple or dict, the *structure* is preserved and each component stays whole. Nothing is sliced, nothing is checked for matching leading dimensions — the components can have completely different shapes. ## from_tensor_slices: slice along axis 0 `tf.data.Dataset.from_tensor_slices(x)` removes the leading axis and produces one element per index along it. ``` ds = tf.data.Dataset.from_tensor_slices(np.zeros((100, 4), dtype=np.float32)) ds.element_spec # TensorSpec(shape=(4,), dtype=float32) ds.cardinality() # 100 ``` With a tuple, all components are sliced in lockstep, which is exactly what a supervised pipeline needs: ``` ds = tf.data.Dataset.from_tensor_slices((features, labels)) # each element is (feature_row, label_scalar) ``` The constraint that follows from lockstep slicing is that every component must have the same size along axis 0. A `(100, 4)` feature array with 99 labels raises immediately at construction time, which is a useful early error. ## Why the confusion costs you The classic bug is calling `from_tensors` on a training array and then `batch(32)`. You get a dataset of one element, batching it produces one batch of shape `(1, 100, 4)`, and training runs one step per epoch on a rank-3 input. Nothing raises — the model may even accept the shape — you just train on garbage. The other direction is subtler: `from_tensor_slices` on a Python list of lists of differing lengths fails, because the argument must first convert to a dense tensor. Ragged data needs `tf.ragged.constant`, a generator source, or per-record files. ## Memory: both keep the data in the graph Neither constructor streams. Both embed the tensor into the dataset, which for large arrays means the data is held in memory and, under graph execution, can be captured as a constant. That is why the guidance for anything past a few hundred megabytes is to move to a file-based source — `tf.data.TFRecordDataset`, `Dataset.list_files` plus a read step, or `Dataset.from_generator` — rather than to materialize the entire array first. ## Checking what you built Two cheap habits catch nearly every mistake in this area. `ds.element_spec` prints the shape and dtype of one element, and `ds.cardinality()` prints the number of elements (or `tf.data.UNKNOWN_CARDINALITY` / `tf.data.INFINITE_CARDINALITY` for sources and pipelines where it cannot be known statically). Iterating a couple of elements with `for x in ds.take(2): print(x.shape)` confirms it directly. ## When from_tensors is the right answer It is not a trap constructor, it just has narrow uses: - `tf.data.Dataset.from_tensors(one_batch).repeat()` gives an infinite stream of one fixed batch — the standard overfit-a-single-batch debugging trick, and a clean input for measuring pure step time with the input pipeline removed from the picture. - Building a dataset of *whole* objects where the leading axis is meaningful data, not an example index — for instance one element that is a full sequence you will later `unbatch` or `window` yourself. - Composing with `Dataset.concatenate` or `Dataset.zip` when one side is a single constant. A useful identity to remember: `from_tensors(x)` is equivalent to `from_tensor_slices(x).batch(len(x))` in what it yields, and `from_tensor_slices(x)` is equivalent to `from_tensors(x).unbatch()`.
- What happens if you pass features with 100 rows and labels with 99 to from_tensor_slices?It raises at construction. `from_tensor_slices` slices every component of the tuple along axis 0 in lockstep, so all components must agree on that dimension. This is a helpful early failure — the alternative, silently truncating to the shorter one, would misalign every label in training.
- How would you build a dataset from data too large to hold in memory as one array?Move to a file-based or callable source. Write the data as sharded TFRecord files and read them with `tf.data.TFRecordDataset`, or list files with `Dataset.list_files` and read each one in a `map`/`interleave`. `Dataset.from_generator` also works when a Python generator already streams the data, at the cost of running Python per element.
- Is there a way to convert between the two forms after the fact?Yes. `Dataset.from_tensors(x).unbatch()` gives the same elements as `from_tensor_slices(x)`, and `from_tensor_slices(x).batch(n)` where n is the full length reassembles the single blob. `unbatch` and `batch` are the general operations for adding or removing a leading axis anywhere in a pipeline.
saying these in an interview costs you the question
- Saying the two constructors are interchangeable aliases
- Calling from_tensors on a training array, then batching it
- Assuming from_tensors slices when given a tuple
- Thinking either constructor streams data from disk
- Believing from_tensor_slices slices along the last axis