skip to content

Which strings does CPython intern automatically, and what does `sys.intern` do?

level: middleimportance: should knowfreq 48%

answer

  1. One canonical copy per distinct value
  2. Compile time versus run time construction
  3. The compiler is picky about which literals
  4. Identifier-shaped text gets the treatment
  5. The call returns it, does not mutate it

basics

~20 s

CPython keeps one canonical copy of certain strings in an intern table: identifier-like string constants in compiled code, plus names, attribute names, the empty string and single characters. sys.intern forces a string into that table and returns the canonical object.

solid answer

~40 s

Interning means storing one canonical object per distinct string value so equal strings share it. CPython interns automatically where it pays off: string constants in compiled code that look like identifiers (ASCII letters, digits and underscores), every variable, function, attribute and module name, single-character latin-1 strings and the empty string. Strings produced at run time — by concatenation, slicing, `str.join`, decoding or reading a file — are **not** interned, so two equal ones are usually distinct objects. `sys.intern(s)` puts the value in the table and returns the canonical object; you must keep the returned reference, since the argument is not modified. It buys memory savings on heavily duplicated strings and faster dictionary hits, because a lookup can succeed on a pointer comparison before comparing characters. It never changes what `==` reports.

code

python · 7 lines
python
import sys

a = "clinical lab"
b = " ".join(["clinical", "lab"])
print(a is b)                          # False
print(sys.intern(a) is sys.intern(b))  # True
print(a == b)                          # True either way

go deeper

for a junior

Know the vocabulary: interning means one shared object per distinct string value, and Python does it for some strings automatically. Remember that strings built while the program runs are usually not shared, so compare text with ==.

for a middle

Explain the split precisely: identifier-like string constants and namespace or attribute names are interned by CPython, run-time constructions are not, and sys.intern returns the canonical object rather than modifying what you passed in.

for a senior

Demonstrate judgement about when to intern deliberately — low-cardinality fields repeated across a large parse, for memory and dictionary-lookup wins — and about the cost, since interned strings are immortal from 3.12 and high-cardinality interning grows without bound.

for a principal

Frame it as a measured optimisation with a stated cardinality budget rather than folklore: decide which fields are interned at ingest, document why, and keep correctness independent of identity so a version change in what gets auto-interned cannot alter behaviour.

### What interning is Interning is the technique of keeping exactly one canonical object per distinct value, in a table the runtime consults whenever it needs that value. Because Python strings are immutable, sharing is always safe: nobody can mutate the shared copy out from under another holder. CPython uses interning for two concrete wins — memory, when the same text appears thousands of times, and speed, because comparing two interned strings can stop at a pointer comparison. ### What CPython interns without being asked The compiler interns string constants in the code it produces when the text looks like an identifier: only ASCII letters, digits and underscores. `"total"` is interned; `"clinical lab"`, with its space, is not. Beyond constants, the runtime interns the strings it uses structurally: variable, function, class, module and attribute names, and the keys those names become in module and class dictionaries. Single-character latin-1 strings and the empty string are also cached objects. That set is chosen because attribute lookup and namespace lookup are the hottest string-keyed operations in the interpreter, and they benefit most from pointer-equality fast paths. Everything built at run time is outside that set. `" ".join(parts)`, `text[4:9]`, `bytes.decode`, `str.format`, an f-string, a line read from a file — each produces a fresh object even when an equal string already exists. This is exactly why identity comparisons on parsed data behave differently from identity comparisons on literals. ### `sys.intern` `sys.intern(s)` takes a `str`, ensures the intern table holds a canonical object for that value, and **returns** it. Two properties trip people up. First, it does not modify the argument: `sys.intern(a)` leaves `a` alone, so you must bind the result. Second, it is only useful if you route *every* copy through it — interning one of two equal strings does nothing for identity between them. ```python import sys a = "clinical lab" b = " ".join(["clinical", "lab"]) print(a is b) # False print(sys.intern(a) is sys.intern(b)) # True print(a == b) # True regardless ``` ### When deliberate interning pays The classic case is bulk parsing where a small vocabulary repeats enormously: status codes, column names, units, category labels, panel names. A loader that reads millions of records will otherwise hold millions of separate objects for a few dozen distinct strings. Interning those fields as they are parsed collapses them to one object each, cutting resident memory and making the dictionaries keyed by them faster, because a hit can be decided by comparing pointers instead of walking characters. The opposite case is a caution: interning high-cardinality values — patient identifiers, request ids, free text — puts unbounded distinct strings into a table that CPython treats as long-lived. Since 3.12, PEP 683 made interned strings immortal objects, so you should not assume their memory is promptly reclaimed. Intern low-cardinality, high-repetition fields only. ### What interning does not do It does not change equality. `==` compares values and is unaffected. It does not make `is` a valid way to compare strings: the moment one side comes from a source you do not control, or from a code path that builds it at run time, identity fails while equality still holds. Interning is a performance tool, and any correctness that appears to follow from it is coincidence. CPython warns you about the most visible form of that mistake — writing `is` against a string literal produces a `SyntaxWarning` at compile time naming the literal type and suggesting `==`. ### Interviewer's angle What is being tested is whether you know *which* strings share an object and why, rather than whether you can recite that interning exists. A strong answer names the identifier-like rule for compile-time constants, points out that run-time construction escapes it, describes `sys.intern` as returning the canonical object rather than mutating anything, gives the duplicate-heavy parsing use case, and closes by refusing to treat any of it as a correctness guarantee.

  • Why does CPython intern identifier-like constants specifically, rather than every string constant?
    Because those are the strings that become namespace and attribute keys, where the interpreter does its hottest dictionary lookups and can shortcut on a pointer comparison. Interning arbitrary text would grow a permanent table for values that are never used as keys, paying memory for no lookup benefit.
  • Does interning make `==` on two long equal strings faster?
    Only when both operands are interned: the comparison can then answer True on the pointer check without scanning characters, and inequality of interned strings can be decided by pointer too. If either side is a fresh run-time string the full character comparison happens as normal, so interning speeds up the case where you control both sides.
  • What is the risk of calling `sys.intern` on every string a parser produces?
    High-cardinality values fill the intern table with entries that are effectively permanent — since 3.12 interned strings are immortal — so a long-running load turns a memory optimisation into steady growth. Intern only low-cardinality fields that repeat heavily, such as status codes or column names.

A print shop keeps one master plate per commonly-requested word and reuses it; anything you dictate over the phone gets set fresh in type, even if the same word is already on a plate.

saying these in an interview costs you the question

  • Says all equal strings share one object in Python
  • Believes `sys.intern` mutates its argument in place
  • Thinks interning changes what `==` returns
  • Claims run-time concatenation results are interned
  • Recommends interning every parsed string for speed
  • Uses `is` on strings because interning usually works

context