skip to content

Greedy Algorithms

Greedy as a paradigm with a correctness story: making the locally best choice and knowing why it stays globally optimal. Interviewers probe whether you can justify a greedy solution — or spot when it silently fails — not just code one.

part ofData structures & algorithmsoverview, primer and where to startread it →
on this pageshow

questions

28

Why must a Huffman code be prefix-free, and what breaks in decoding without it?

level: juniorimportance: must knowfreq 70%

answer

  1. Where does one codeword stop?
  2. Symbols live only at leaves
  3. Decoder walks down, emits, restarts
  4. No codeword starts another codeword
  5. Codes 0, 01, 1 decode two ways

basics

~20 s

Prefix-free means no codeword is the beginning of another, so a decoder reading bits one at a time always knows exactly where a symbol ends. Without that property the same bit stream decodes several different ways.

solid answer

~50 s

Huffman builds a binary tree in which symbols sit **only at leaves**, and a symbol's codeword is the root-to-leaf path, `0` for one child and `1` for the other. Because no leaf lies on the path to another leaf, no codeword can be a prefix of another. That is what makes the stream self-delimiting: the decoder walks down from the root bit by bit, emits a symbol the instant it lands on a leaf, and jumps back to the root. Take away the property and ambiguity appears immediately — with codewords `0`, `01` and `1`, the bits `01` are either the second symbol alone or the first followed by the third, and no amount of lookahead settles it. Prefix-freeness costs zero extra bits, which is why it beats spending bits on separators or on a length field per symbol.

go deeper

for a junior

Recall the definition and be able to show the failure in one line: with codewords 0, 01 and 1, the bits 01 have two readings. Then say where symbols sit in the tree — leaves only.

for a middle

Explain why leaf placement makes the property structural rather than something you verify afterwards, and walk a short bit stream down the tree out loud, emitting and restarting at the root.

for a senior

Show you know the boundary between decodability and optimality, and raise the end-of-stream problem unprompted: padding bits decode into phantom symbols unless a count or a terminator symbol is carried.

for a principal

Frame the choice as a format decision: prefix-free coding buys instantaneous decoding at zero bit cost, and the alternatives — length fields, separators, fixed width — all trade bits or decoder complexity for the same guarantee.

## Why boundaries are the real problem A compression scheme that gives frequent symbols short codewords and rare symbols long ones must solve a problem that fixed-width codes never have: **where does one codeword end and the next begin?** With a fixed 2-bit code over a four-symbol alphabet the answer is trivial — cut every two bits. Once lengths vary, the decoder is handed an undifferentiated run of bits and has to find the cut points itself. There are only three ways to give it those cut points: 1. **Fixed width** — no savings, which defeats the purpose. 2. **Explicit separators or per-symbol length fields** — you spend bits describing the encoding instead of the data, and on a skewed stream that overhead can swamp the savings. 3. **A prefix-free (also called prefix, or instantaneous) code** — the codeword set itself is arranged so the cut points are unambiguous. This is free. Huffman coding takes the third route. ## The definition, precisely A code is **prefix-free** when no codeword is a proper prefix of any other codeword. `{0, 10, 110, 111}` is prefix-free. `{0, 01, 1}` is not: `0` is a prefix of `01`. And that failure is not cosmetic — the bits `01` decode as the single symbol coded `01`, or as the symbol coded `0` followed by the symbol coded `1`. Two legal readings of the same bits means the encoding has lost information. Note what prefix-free does **not** mean. It does not mean all codewords are the same length — that is the thing we are trying to escape. It does not mean codewords are unique — uniqueness is necessary but nowhere near sufficient, as `{0, 01, 1}` shows. And it is not the only way to be uniquely decodable: a code can be decodable only after reading ahead an unbounded distance. Prefix-free codes are the ones decodable **instantaneously**, symbol by symbol, with no lookahead and no backtracking. ## The tree makes it automatic The elegance of the tree formulation is that prefix-freeness is not something you check afterwards; it is a structural consequence. Build a binary tree, label the edge to one child `0` and to the other `1`, and place each symbol at a distinct **leaf**. A codeword is the sequence of edge labels from root to that leaf. Codeword X is a prefix of codeword Y exactly when X's node lies on the path to Y's node — and a leaf, by definition, has nothing below it. So leaf placement alone guarantees the property. Consider a four-symbol stream in which one symbol dominates: frequencies 0.90, 0.06, 0.03, 0.01. Huffman produces a tree giving codewords `1`, `01`, `001`, `000`. Feed the decoder `1 1 001 1 01`, i.e. the bits `110011 01`: it reads `1`, lands on a leaf, emits the dominant symbol, restarts; reads `1`, emits again; reads `0`, `0`, `1`, lands on the depth-3 leaf, emits the rare symbol; and so on. At no point does it need to know how long the next codeword will be. ## The end of the stream is a separate problem Prefix-freeness fixes boundaries **between** symbols; it says nothing about where the stream stops. Compressed output usually ends mid-byte, and the padding bits at the tail are a perfectly valid path down the tree, so a naive decoder emits one or more phantom symbols. Real formats solve this by storing an explicit symbol count alongside the data, or by reserving one extra alphabet member as an end-of-stream marker and coding it like any other symbol. Expect this as a follow-up. ## Prefix-free is a constraint, not the optimization A final distinction worth having straight: prefix-freeness makes a code **decodable**, not **good**. Infinitely many prefix-free codes exist over any alphabet, most of them terrible — the fixed-width code is itself prefix-free. What Huffman's greedy construction adds is that among all prefix-free codes for a given frequency table, the tree it builds minimizes the total weighted codeword length. The two properties are independent, and interviewers probe whether a candidate has conflated them. One more fact for the curious: the code lengths achievable by *some* prefix-free binary code are exactly those satisfying the Kraft inequality, the sum of 2 raised to the power of minus each length being at most 1. Lengths 1, 2, 2, 2 fail it (0.5 + 0.25 + 0.25 + 0.25 = 1.25), which is why no prefix-free code assigns them — a fast sanity check when someone proposes a code-length table.

  • Does being prefix-free make a code optimal?
    No. Prefix-freeness is a decodability constraint, and infinitely many prefix-free codes exist for any alphabet — the fixed-width code is one of them. Huffman's contribution is choosing, among all prefix-free codes for a given frequency table, the one that minimizes total weighted codeword length. A candidate who treats the two properties as the same thing has missed what the greedy construction is actually optimizing.
  • How does the decoder know when the stream is finished?
    Prefix-freeness resolves boundaries between symbols but says nothing about the end. The tail padding bits form a valid path down the tree, so a naive decoder emits phantom symbols. Fix it either by storing an explicit symbol count in the header or by adding a dedicated end-of-stream member to the alphabet, giving it a frequency of one and coding it like any other symbol.
  • Could a symbol ever be stored at an internal node?
    Not in a prefix-free code. An internal node lies on the path to every leaf beneath it, so its codeword would be a prefix of all of theirs, and the decoder could never tell whether to stop there or keep descending. Any construction that places symbols at internal nodes has abandoned instantaneous decoding and needs separators to work at all.

It is the reason no country's phone numbers start with the emergency number: once you have dialled that prefix, the exchange must be able to act without waiting to see if more digits are coming.

saying these in an interview costs you the question

  • Suggests separating codewords with a delimiter bit pattern
  • Thinks prefix-free means all codewords share one length
  • Places symbols at internal nodes of the code tree
  • Claims the decoder needs each codeword's length up front
  • Assumes prefix-free automatically means the code is optimal

context

open as a page

Why does earliest-finish-time beat shortest-duration when picking the most non-overlapping talks?

level: juniorimportance: must knowfreq 78%

basics

~20 s

Choosing the talk that finishes earliest leaves the largest possible remaining window for everything after it, and that rule is provably optimal. Shortest-duration is not: one short talk can straddle the seam between two longer talks and cost you a slot.

open as a page

Why does one furthest-reach counter decide whether you can cross a river of stones, each stone i allowing a leap of at most a[i] forward?

level: juniorimportance: must knowfreq 72%

basics

~20 s

Reachability is monotone: standing on stone i makes every stone up to i + a[i] reachable too. So one pass keeping the maximum of i + a[i], abandoning the crossing as soon as the index passes that maximum, settles it in O(n).

open as a page

In an exchange argument for a greedy algorithm, what do you swap and what must the swap show?

level: juniorimportance: must knowfreq 55%

basics

~20 s

An exchange argument starts from any optimal solution, finds the first point where it disagrees with greedy, and swaps greedy's choice in. The swap must stay feasible and no worse, so some optimal solution agrees with greedy.

open as a page

Why can nearest-stop greedy routing, taking the closest undelivered address each time, produce a poor total route?

level: juniorimportance: must knowfreq 68%

basics

~20 s

Each nearest-stop pick is cheapest right now but reshapes what remains. Cheap early hops strand far-apart addresses for the end, so a chain of locally best moves need not add up to the best route.

open as a page

A fare machine stocks denominations 1, 3 and 4 - why can largest-first change-making use more coins than necessary?

level: juniorimportance: must knowfreq 62%

basics

~20 s

For an amount of 6, largest-first takes 4, then 1 and 1 - three coins - while 3 + 3 uses only two. Taking the biggest coin left a remainder of 2 that only unit coins could pay.

open as a page

Why is the optimal Huffman tree for an alphabet where one symbol is 90% of the stream unbalanced?

level: middleimportance: must knowfreq 62%

basics

~20 s

Huffman minimizes average codeword length weighted by frequency, not tree height. A symbol carrying 90% of the stream gets pulled to depth 1 and the rare symbols pushed deep, because balance would spend bits on symbols almost nobody sends.

open as a page

Why does cancelling the fewest conflicting ad slots reduce to keeping the earliest-ending ones?

level: middleimportance: must knowfreq 60%

basics

~20 s

Every slot is either kept or cancelled, so minimizing cancellations means maximizing the kept conflict-free set: cancellations equal total minus kept. That maximum comes from earliest-finish selection, so from any conflicting pair you drop the one ending later.

open as a page

In Dijkstra's algorithm, why can the closest unsettled vertex be finalized immediately?

level: middleimportance: must knowfreq 65%

basics

~20 s

Any rival route must first leave the settled set through some unsettled vertex whose tentative cost is already at least as large, and with non-negative edge weights the rest of that route can only add cost.

open as a page

Why is optimal substructure alone not enough to justify a greedy algorithm, and what else is required?

level: middleimportance: must knowfreq 62%

basics

~20 s

Optimal substructure only says an optimal solution contains optimal subproblem solutions, and dynamic-programming problems have it too. Greedy also needs the greedy-choice property: some optimal solution starts with the locally best choice, so committing is never undone.

open as a page

Sorting cargo by value per tonne is optimal for divisible grain but wrong for indivisible crates - why?

level: middleimportance: must knowfreq 66%

basics

~20 s

Divisible cargo lets you fill the last sliver of capacity with the best remaining density, so no swap can improve the load and greedy is provably optimal. Indivisible crates can strand capacity, so a lower-density crate may pay more.

open as a page

Dijkstra's algorithm runs on a fee graph with one negative-cost leg but no negative cycle — can you trust its output?

level: seniorimportance: must knowfreq 60%

basics

~20 s

No — one negative edge is enough. The algorithm finalizes a vertex as soon as it holds the smallest tentative cost, and a negative leg discovered later can undercut that finalized value. Negative cycles are a separate problem.

open as a page

Can Prim's and Kruskal's algorithms produce spanning trees of different total weight on the same graph?

level: juniorimportance: should knowfreq 50%

basics

~20 s

No. Both are proven to produce a minimum spanning tree, and every minimum spanning tree has the same total weight. When edge weights tie the two may choose different edges; when all weights are distinct the tree is unique.

open as a page

Merging sorted log segments costs the sum of the two sizes — which merge order minimizes total cost?

level: middleimportance: should knowfreq 48%

basics

~10 s

Always combine the two smallest remaining segments: pull the two minima from a min-heap, add their sum to the running total, and push the result back. This is Huffman's construction wearing different clothes.

open as a page

How do you prove earliest-finish-time selection is optimal for activity selection?

level: middleimportance: should knowfreq 45%

basics

~20 s

Take any optimal schedule and swap its first activity for the greedy's first pick. The greedy's pick finishes no later, so every remaining activity still fits and the schedule stays the same size. Repeat down the list: greedy can never be smaller.

open as a page

When counting minimum relay hops across towers with forward ranges r[i], what does the correct greedy maximize at each step?

level: middleimportance: should knowfreq 55%

basics

~20 s

It maximizes the reach achievable from anywhere inside the current hop's window, not the length of the individual hop. Sweep the window, track the best i + r[i] seen inside it, and when the sweep exhausts the window, count one hop and extend the window to that best reach.

open as a page

A reachability scan takes the maximum of i + a[i] over every stone but never compares i to that maximum — what breaks?

level: middleimportance: should knowfreq 42%

basics

~20 s

It credits stones you can never stand on, so it reports crossings that are impossible. With leaps [1, 0, 0, 5] you stall on the second stone, but the unguarded scan folds in the fourth stone's range and answers yes. The fix is to stop as soon as the index passes the recorded reach.

open as a page

Why is the cheapest edge crossing any cut of a weighted graph safe to put in a minimum spanning tree?

level: middleimportance: should knowfreq 42%

basics

~20 s

An exchange argument proves it: adding that edge to any minimum spanning tree creates exactly one cycle, and that cycle must contain another edge crossing the same cut. Swapping them keeps the tree spanning and never increases total weight.

open as a page

In a greedy-stays-ahead proof for covering houses on a line with fewest towers, what is the induction hypothesis?

level: middleimportance: should knowfreq 42%

basics

~20 s

The hypothesis is that after k towers, the greedy has covered at least as long a prefix of the houses as any other solution's leftmost k towers. Never being behind means greedy never needs more towers than an optimal solution.

open as a page

Why does sorting the weights first make heaviest-with-lightest pairing a safe greedy move under a per-load cap?

level: middleimportance: should knowfreq 45%

basics

~20 s

Sorting identifies the extremes the rule needs. The heaviest passenger occupies a boat regardless, and the lightest is the best companion who could fit beside them, so pairing those two never costs a boat. Unsorted, neither end is known.

open as a page

A diff flips an interval-selection guard from start >= lastEnd to start > lastEnd — what breaks?

level: seniorimportance: should knowfreq 40%

basics

~20 s

If back-to-back items are legal, the strict comparison rejects an item starting exactly when the previous one ends, so the algorithm silently returns too few selections. Nothing crashes and most tests still pass, because only inputs with touching endpoints expose it.

open as a page

On a circular route with gain[i] at each stop and cost[i] to the next, why is the single-pass restart greedy correct?

level: seniorimportance: should knowfreq 40%

basics

~20 s

Two facts carry it. A feasible start exists exactly when total gain over the loop is at least total cost. And if a run starting at s dies before reaching stop j, every stop between them dies no later, so the scan can skip them all and restart after j — one lap, O(n) time, O(1) space.

open as a page

A teammate balances two shards by sorting record sizes descending and alternating — how do you disprove it?

level: seniorimportance: should knowfreq 38%

basics

~20 s

Produce one concrete counterexample. Sizes 3, 2, 2 alternate into shards of 5 and 2, while 3 against 2 plus 2 gives a larger shard of only 4. A single instance settles it; a passing test suite never could.

open as a page

Sorting digit segments so their concatenation is largest: why is the pairwise rule a+b vs b+a correct?

level: seniorimportance: should knowfreq 45%

basics

~20 s

An adjacent-swap exchange argument proves it: swapping a neighbouring pair that breaks the rule never shrinks the result, since the surrounding text and the pair's span are unchanged. Sorting by the rule is valid only because it is transitive.

open as a page

Why does serving the shortest support tickets first minimize total waiting time, when a colleague says order barely matters?

level: seniorimportance: should knowfreq 42%

basics

~20 s

Each ticket's handling time is re-paid by everyone still queued behind it, so serving a long one early multiplies its cost. Shortest-first keeps the big multipliers on the small durations, and the remaining queue is the same problem again.

open as a page

Before you commit to a greedy solution in an interview, how do you convince yourself the greedy choice is safe?

level: seniorimportance: should knowfreq 50%

basics

~10 s

Two moves: try to prove the greedy choice belongs to some optimal solution with an exchange argument, and simultaneously hunt for a small counterexample against exhaustive search. Passing the provided examples proves nothing.

open as a page

Denominations are fixed but payout amounts reach 10^9 - do you ship largest-first or the exact method?

level: principalimportance: should knowfreq 36%

basics

~10 s

The exact method's cost scales with the amount's numeric value, so it is unusable at 10^9. Prove the fixed denomination set greedy-optimal offline, ship largest-first, and guard that proof with an automated check.

open as a page

Two Huffman encoders build different codebooks from the same frequency table — is either one wrong?

level: seniorimportance: nice to knowfreq 22%

basics

~20 s

Not necessarily. Ties between equal weights during merging, and the free choice of which child gets bit 0, yield several distinct trees that all hit the same minimum weighted total, so the codebook has to travel with the compressed data.

open as a page