When would you bind an existing shared library rather than write your own Python extension?

middleshould knowfreq 34%

answer

Ask who owns the algorithm
Reuse tested compiled code where it exists
The wrapper's job is small but real
Error codes to exceptions, handles to close
The stdlib is full of this pattern

basics

~20 s

Bind when a mature, tuned implementation already exists, so the only code you own is a thin wrapper. Write your own when the hot code is your domain logic, or when a large native dependency buys very little.

solid answer

~50 s

The question is who owns the algorithm. If a well-tested compiled library already implements the thing you need — a codec, a parser, a numerical kernel — binding it means the only code you maintain is the wrapper: converting arguments, translating error codes into exceptions, and managing the lifetime of whatever handles the library hands back. That is a much smaller surface than writing and tuning the algorithm yourself. Write your own extension when the hot loop is domain logic no library implements, when converting your data into the library's layout would cost more than the computation, or when a large native dependency buys fifty lines of work. Either way keep the boundary coarse: one call over a whole batch, not one call per element. The Python standard library itself is full of the binding pattern — zlib, lzma, sqlite3 and, since 3.14, compression.zstd are wrappers over C libraries.

The question is who owns the algorithm. If a well-tested compiled library already implements the thing you need — a codec, a parser, a numerical kernel — binding it means the only code you maintain is the wrapper: converting arguments, translating error codes into exceptions, and managing the lifetime of whatever handles the library hands back. That is a much smaller surface than writing and tuning the algorithm yourself. Write your own extension when the hot loop is domain logic no library implements, when converting your data into the library's layout would cost more than the computation, or when a large native dependency buys fifty lines of work. Either way keep the boundary **coarse**: one call over a whole batch, not one call per element. The Python standard library itself is full of the binding pattern — `zlib`, `lzma`, `sqlite3` and, since 3.14, `compression.zstd` are wrappers over C libraries.

code

python · 5 lines

python

import zlib

data = b"metric.samples scrape_duration_seconds " * 5000
blob = zlib.compress(data, level=6)
print(len(data), len(blob), zlib.decompress(blob) == data)

go deeper

for a junior

Know that plenty of fast code you already use — compression, database drivers, hashing — is Python wrapping an existing compiled library, and that reusing such a library beats writing your own version of the algorithm.

for a middle

Explain the wrapper's actual duties: converting arguments without needless copies, turning error codes into exceptions, and releasing handles the library allocated. Be able to say when writing your own extension is the better trade instead.

for a senior

Show judgement about the boundary: batch size per crossing, avoiding conversion on every call, and containing the native dependency behind one module with a pure-Python fallback so tests and unusual platforms still work.

for a principal

Own the dependency decision — supply chain, platform matrix, who on the team can debug a crash inside the library — and set the rule for when a team may take on native code at all.

Two different jobs

"Leaving pure Python" is not one decision, it is two. Either you are writing new compiled code because the work you need does not exist yet, or you are calling compiled code that already exists. The cost profile of those two is completely different, and interviewers ask this to see whether a candidate reaches for the second before the first.

Why binding is usually the cheaper trade

A mature compiled library brings years of correctness work, edge-case handling and tuning that you would otherwise repeat badly. When you bind it, the only code you own is the wrapper, and a wrapper has a small, reviewable job:

Marshal arguments. Turn Python values into whatever the library expects, and ideally hand it a pointer to a buffer you already have rather than copying.
Translate errors. Native libraries usually signal failure with a return code or an error struct. Nothing turns that into a Python exception unless your wrapper does; a wrapper that ignores return codes is a source of silent corruption.
Own lifetimes. A library that hands back a handle, a context or an allocated block expects an explicit release. Python's reference counting knows nothing about that memory: if the wrapper never calls the release function, the process leaks, and the leak is invisible to object-graph tools because no Python object is growing. Wrapping the handle in a context manager, or giving the owning object a deterministic close, is the fix.
Respect the library's threading rules. Reentrancy and thread-safety are properties of the native library, not of Python.

Notice that none of these require you to be an expert on the algorithm itself. That asymmetry is the whole argument.

When writing your own is right

The hot code is your domain logic. Nobody has published a compiled implementation of your matching rules or your parsing state machine, and shoehorning them into a general library costs more than writing the loop.
The conversion would dominate. If using the library means rebuilding your data into its layout on every call, you may spend more time converting than computing. Owning the code lets you consume your layout directly.
The dependency is disproportionate. Taking on a large native dependency — with its own build, versioning, platform matrix and security surface — for fifty lines of arithmetic is a bad trade. Distribution and build concerns get real here, and they belong in the decision.
You need a narrow, stable surface. A small extension you control is easier to keep working across interpreter versions and build variants than a third-party binding you do not.

The rule that applies to both: keep crossings coarse

Whichever route you pick, the boundary has a fixed per-call cost — argument conversion, validation, and the call itself. Calling a compiled routine once per element in a Python loop reliably gives back most of the speedup you went to get, and sometimes all of it. The design goal is one crossing per batch: pass a buffer, get a result, do the looping on the far side. This is the single most common reason a native rewrite disappoints in practice.

What you inherit either way

Going native moves you off the safety net. A bug that would have raised IndexError in Python can now be an out-of-bounds write that corrupts memory or kills the process, and a stack trace stops at the boundary. Debugging needs native tooling; profiling tools that only understand Python frames show one opaque call. Your build and distribution story grows a platform matrix. None of that argues against the move — it argues for keeping the native surface small, wrapping it behind a plain Python interface, and keeping a pure-Python reference implementation you can test against.

What the interviewer is listening for

They want the reflex of asking "does this already exist, compiled and tested?" before "how do I write this in C?", plus awareness that a wrapper carries real obligations — error translation and resource release above all — and that the call granularity, not the language of the inner loop, is what usually decides whether the change was worth it.

## Two different jobs "Leaving pure Python" is not one decision, it is two. Either you are **writing new compiled code** because the work you need does not exist yet, or you are **calling compiled code that already exists**. The cost profile of those two is completely different, and interviewers ask this to see whether a candidate reaches for the second before the first. ## Why binding is usually the cheaper trade A mature compiled library brings years of correctness work, edge-case handling and tuning that you would otherwise repeat badly. When you bind it, the only code you own is the wrapper, and a wrapper has a small, reviewable job: * **Marshal arguments.** Turn Python values into whatever the library expects, and ideally hand it a pointer to a buffer you already have rather than copying. * **Translate errors.** Native libraries usually signal failure with a return code or an error struct. Nothing turns that into a Python exception unless your wrapper does; a wrapper that ignores return codes is a source of silent corruption. * **Own lifetimes.** A library that hands back a handle, a context or an allocated block expects an explicit release. Python's reference counting knows nothing about that memory: if the wrapper never calls the release function, the process leaks, and the leak is invisible to object-graph tools because no Python object is growing. Wrapping the handle in a context manager, or giving the owning object a deterministic close, is the fix. * **Respect the library's threading rules.** Reentrancy and thread-safety are properties of the native library, not of Python. Notice that none of these require you to be an expert on the algorithm itself. That asymmetry is the whole argument. ## When writing your own is right * **The hot code is your domain logic.** Nobody has published a compiled implementation of your matching rules or your parsing state machine, and shoehorning them into a general library costs more than writing the loop. * **The conversion would dominate.** If using the library means rebuilding your data into its layout on every call, you may spend more time converting than computing. Owning the code lets you consume your layout directly. * **The dependency is disproportionate.** Taking on a large native dependency — with its own build, versioning, platform matrix and security surface — for fifty lines of arithmetic is a bad trade. Distribution and build concerns get real here, and they belong in the decision. * **You need a narrow, stable surface.** A small extension you control is easier to keep working across interpreter versions and build variants than a third-party binding you do not. ## The rule that applies to both: keep crossings coarse Whichever route you pick, the boundary has a fixed per-call cost — argument conversion, validation, and the call itself. Calling a compiled routine once per element in a Python loop reliably gives back most of the speedup you went to get, and sometimes all of it. The design goal is one crossing per **batch**: pass a buffer, get a result, do the looping on the far side. This is the single most common reason a native rewrite disappoints in practice. ## What you inherit either way Going native moves you off the safety net. A bug that would have raised `IndexError` in Python can now be an out-of-bounds write that corrupts memory or kills the process, and a stack trace stops at the boundary. Debugging needs native tooling; profiling tools that only understand Python frames show one opaque call. Your build and distribution story grows a platform matrix. None of that argues against the move — it argues for keeping the native surface small, wrapping it behind a plain Python interface, and keeping a pure-Python reference implementation you can test against. ## What the interviewer is listening for They want the reflex of asking "does this already exist, compiled and tested?" before "how do I write this in C?", plus awareness that a wrapper carries real obligations — error translation and resource release above all — and that the call granularity, not the language of the inner loop, is what usually decides whether the change was worth it.

What is the most common bug you see in hand-written wrappers around a native library?
Resource lifetime. The library allocates a context or handle and expects an explicit release; reference counting does not know about it, so a wrapper that forgets to release leaks memory the interpreter cannot see. The second most common is ignoring return codes, so a failed native call quietly returns garbage instead of raising.
Why does calling a compiled routine once per element often produce no speedup?
Because each crossing has a fixed cost — converting arguments, validating them, making the call — and in a per-element loop you pay it as many times as the interpreted loop paid its own overhead. You have replaced one constant with another. The design that pays off hands the whole batch across once and lets the loop run on the far side.
How would you keep a native dependency from spreading through the codebase?
Put it behind one small pure-Python module that exposes an ordinary interface, and keep a pure-Python reference implementation of the same interface for tests and for platforms where the native build is unavailable. Then the fast path stays replaceable and the rest of the code never learns that a native library exists.

saying these in an interview costs you the question

Reaches for writing C before checking whether a library already exists
Treats a wrapper as free — no error translation, no resource release
Assumes reference counting frees memory the native library allocated
Calls the native routine once per element and expects a speedup
Ignores that a native dependency brings a build and platform matrix

When would you bind an existing shared library rather than write your own Python extension?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

Two different jobs

Why binding is usually the cheaper trade

When writing your own is right

The rule that applies to both: keep crossings coarse

What you inherit either way

What the interviewer is listening for

▾common follow-ups3 questions

▾red flags5 flags

§ context

▾related questions5 questions

▾related topics4 topics

Two different jobs

Why binding is usually the cheaper trade

When writing your own is right

The rule that applies to both: keep crossings coarse

What you inherit either way

What the interviewer is listening for

answer

go deeper

context