skip to content

What does itertools.islice consume, and why does it reject negative indices?

level: middleimportance: should knowfreq 40%

answer

  1. Sequences can subscript; iterators cannot
  2. Forward-only, so nothing can be counted backwards
  3. Skipped elements are pulled and dropped
  4. The source stays past the last item
  5. Use a bounded deque for the final N

basics

~20 s

itertools.islice pulls items from the underlying iterator and cannot rewind, so it consumes and discards everything it skips and leaves the source positioned after the last item it yielded. Negative start, stop or step would require knowing the length, so they raise ValueError.

solid answer

~40 s

`itertools.islice(iterable, stop)` or `islice(iterable, start, stop, step)` is the way to slice something that has no `__getitem__` - a one-shot iterator or a generator object. It is lazy: it pulls one item at a time and yields an iterator, not a list. Because it can only move forward, the elements before `start` and between steps are still *pulled from the source and thrown away*, and the source is left sitting just after the last item islice yielded - so a second `islice` over the same iterator continues where the first stopped rather than restarting. Negative values for `start`, `stop` or `step` all raise `ValueError`, because getting the last N items or walking backwards would require knowing the length or rewinding, and an iterator offers neither. `stop=None` means run to exhaustion.

code

pycon · 10 lines
pycon
>>> from itertools import islice
>>> it = iter(range(10))
>>> list(islice(it, 3))
[0, 1, 2]
>>> next(it)
3
>>> list(islice(it, 2, 5))
[6, 7, 8]
>>> list(it)
[9]

go deeper

for a junior

Know that islice is how you slice something you cannot subscript, such as a generator object or a file object, and that you must wrap the result in list() to see the values.

for a middle

Explain the consumption model out loud: forward-only, skipped items are pulled and discarded, the source is left just past the last yielded item. Say why negative start, stop and step are ValueError rather than silently buffered.

for a senior

Demonstrate the practical consequences: reusing one iterator across two consumers splits the data instead of duplicating it, and the deque(maxlen=n) rolling window is the correct answer for a tail. Know when materialising is simply cheaper than being clever.

for a principal

Frame the tradeoff for an API you own: accepting an iterator gives callers streaming and constant memory but makes single-use semantics part of your contract, and it surprises people. Decide when a function should accept a sequence and say so, rather than accepting any iterable.

### The problem islice exists to solve Sequences support subscripting because they implement `__getitem__` and know their own length. An iterator implements only `__iter__` and `__next__`: it has no length, no random access, and no way to go back. So the subscript form is simply unavailable on a generator object or on the result of `map`, `filter`, `open` or `itertools.count`. `itertools.islice` fills that gap by doing the only thing an iterator can do - stepping forward and choosing what to hand on. The signature mirrors the sequence form deliberately: `islice(iterable, stop)` or `islice(iterable, start, stop, step)`. There is no one-argument-means-start form, and there are no `None` placeholders you can skip - to say "from the tenth onwards" you write `islice(it, 10, None)`, where `stop=None` means run until the source is exhausted. ### What it consumes This is the part interviews probe. `islice` returns an iterator; nothing is read until you iterate it. But when you do, everything it passes over is genuinely pulled out of the source. To yield the item at index 10 it must call `next()` eleven times, and the first ten values are consumed and discarded - they are gone from the source forever. Likewise `step=2` pulls the skipped items and drops them; there is no seeking. The consequence people get wrong is *where the source ends up*. In CPython, after `list(islice(it, 3))` over `it = iter(range(10))` the source is positioned exactly at index 3, so `next(it)` returns `3` - islice does not read ahead past what it yielded. And because the source has advanced, applying `islice` again continues from there. That composability is a feature: a `while` loop of successive `islice` calls is the classic pre-3.12 chunking idiom. It is also a trap if you assumed each `islice` starts from the beginning. Memory is O(1) in the number of items skipped or yielded. `islice` holds no buffer, which is exactly why it is the right tool for a large or endless source and why calling it on an already-materialised sequence rarely buys anything. ### Why negative values are refused `islice(it, -3)` raises `ValueError` with a message stating that the stop argument must be `None` or an integer in `0 <= x <= sys.maxsize`. A negative `start` or `stop` means "counted from the end", and an iterator does not know where its end is until it gets there. A negative `step` means walking backwards, which an iterator cannot do at all. CPython refuses rather than silently buffering the whole stream, because silently buffering an endless source would be a hang or an out-of-memory kill. `step=0` is refused too - it would never advance. So how *do* you take the last N items of an iterator? With a bounded rolling window: `collections.deque(it, maxlen=n)` consumes the whole source once and keeps only the final `n` items, in O(n) memory. That is the honest cost of the operation, and making it explicit is the point of islice's refusal. ### Reading the arguments correctly A few details worth having ready. `islice(it, 0)` yields nothing and consumes nothing. `islice` never raises `IndexError` or `StopIteration` for a short source - if the source runs out before `stop`, you just get fewer items, exactly like a sequence slice that overshoots. The result is an iterator, so `islice(rows, 5)` is not a list; wrap it in `list()` when you need one, and do not print it expecting values. And a subtle one: because the arguments are evaluated when `islice` is called but the source is only touched during iteration, an invalid argument fails immediately while an empty result is only observable later. ### Where it earns its keep Three everyday uses. First, taking a bounded prefix of an unbounded source, which is what makes `itertools.count`, `cycle` and `repeat` usable at all. Second, sampling or skipping in a stream you must not materialise - dropping a header line from a file object, or taking every tenth record from a long log. Third, chunking, by calling it repeatedly against the same iterator until it yields nothing. The one-sentence summary an interviewer is listening for: islice is forward-only, it consumes what it skips, it leaves the source positioned after what it yielded, and it refuses anything that would require knowing the length.

  • After list(itertools.islice(it, 3)) on an iterator, where is it positioned?
    Exactly at the fourth element. islice pulls only as many items as it yields and does not read ahead, so `next(it)` returns the item at index 3. That is why successive islice calls over the same iterator compose into chunks instead of repeating the prefix - and why passing the same iterator to two consumers gives each of them a different slice of the data.
  • How do you get the last N items of an iterator, given that islice refuses negative indices?
    Use `collections.deque(iterable, maxlen=n)`. It consumes the source once and keeps a rolling window of the final n items, so memory is O(n) rather than O(total). This is the honest cost: you cannot know which items are last until the source ends, so something has to buffer n of them, and the deque makes that bound explicit.
  • What does itertools.islice(rows, 10, None) mean?
    Skip the first ten items - pulling and discarding them - then yield everything remaining until the source is exhausted. `stop=None` is the explicit 'no end' marker, the equivalent of an omitted stop in a sequence slice. It is the standard way to drop a fixed-size header from a stream you do not want to materialise.

saying these in an interview costs you the question

  • Expects islice(it, -3) to return the last three items
  • Thinks islice copies the iterator instead of consuming it
  • Assumes a second islice call restarts from the beginning
  • Believes a negative step reverses the iterator
  • Says islice returns a list rather than an iterator
  • Claims islice buffers everything before the start index

context