In Rust, a hashmap! macro sizes its map through a recursive internal @count rule; how does that work, and why does it fail for around 128 entries?
answer
- an arm that starts with @
- consume one token tree, recurse
- every step is a nested expansion
- default recursion limit of 128
basics
~20 sThe @count arms peel off one token tree and call the macro again on the rest, adding 1 each time. Each call nests one expansion deeper, so a literal with about 128 entries exceeds rustc's default recursion limit of 128.
solid answer
~40 sAn **internal rule** is an arm that starts with a literal token, conventionally `@count`, that no public input begins with. `(@count $head:tt $($tail:tt)*) => { 1usize + hashmap!(@count $($tail)*) }` is a **tt muncher**: it consumes one token tree and recurses on the tail until `(@count) => { 0usize }` ends it. The public arm calls `HashMap::with_capacity(hashmap!(@count $($key)*))`. Each recursive call is a nested expansion, and rustc caps nesting at the crate's `recursion_limit`, 128 by default, so a literal with 130 entries fails with "recursion limit reached while expanding `hashmap!`". Fix it with a non-recursive count, `<[()]>::len(&[$(hashmap!(@unit $key)),*])`, or raise `#![recursion_limit = "256"]`. The `@` rules are a convention: any caller can invoke them.
code
rust · 17 linesuse std::collections::HashMap;
macro_rules! hashmap {
(@count) => { 0usize };
(@count $head:tt $($tail:tt)*) => { 1usize + hashmap!(@count $($tail)*) };
($($key:expr => $value:expr),* $(,)?) => {{
let mut map = HashMap::with_capacity(hashmap!(@count $($key)*));
$( map.insert($key, $value); )*
map
}};
}
fn main() {
let ports = hashmap! { "http" => 80, "https" => 443, "ssh" => 22 };
println!("len={} cap_ok={}", ports.len(), ports.capacity() >= 3);
println!("{}", hashmap!(@count a b c d)); // internal rules are callable
}go deeper
Know that a macro_rules! macro can call itself and that arms are tried top to bottom, so the first matching arm wins.
Explain the @-prefixed internal rule convention and read a simple recursive counting rule, including where its recursion stops.
Diagnose recursion-limit failures in downstream crates, replace a recursive count with a flat one, and recognise the opaque-fragment trap when an expr is forwarded into a literal-matching arm.
Set expectations for shared macros: internal rules are public in practice, recursion depth must not scale with user input, and helpers belong in a hidden, documented macro.
## Internal rules A `macro_rules!` macro has one name and one list of arms, so helper logic lives in extra arms. By convention these **internal rules** start with a literal marker token, usually `@` plus a word: `(@count ...)`, `(@unit ...)`. `@` is chosen because it cannot begin an expression, a type or most other fragments, so public input never accidentally lands in a helper arm and helper calls do not collide with public arms. - Arms are tried in order and the first match wins, so internal rules are commonly listed **first**. That matters when a public arm is a catch-all such as `($($t:tt)*)`, which would otherwise swallow `@count a b`. - Internal rules are **not private**. Any caller can write `hashmap!(@count a b c)` and get `3`; the `@` prefix is a naming convention. Moving helpers into a separate macro marked `#[doc(hidden)]` keeps them out of the docs, though it stays callable. ## The tt muncher ```rust macro_rules! hashmap { (@count) => { 0usize }; (@count $head:tt $($tail:tt)*) => { 1usize + hashmap!(@count $($tail)*) }; ($($key:expr => $value:expr),* $(,)?) => {{ let mut map = ::std::collections::HashMap::with_capacity(hashmap!(@count $($key)*)); $( map.insert($key, $value); )* map }}; } ``` A **tt muncher** consumes its input a token tree at a time: 1. `hashmap!(@count k1 k2 k3)` matches the second arm with `$head = k1` and `$tail = k2 k3`. 2. It expands to `1usize + hashmap!(@count k2 k3)`, which rustc expands again, one level deeper. 3. The recursion ends at `(@count)`, giving `1usize + 1usize + 1usize + 0usize`, a constant. The public arm forwards each captured `$key` into the muncher. A captured `expr` fragment travels as a single, opaque token tree, so `$head:tt` matches one key at a time even when the key is `a + b`. ## Why it breaks near 128 entries Each recursive call is a macro expansion nested inside the previous one. rustc limits that nesting with the crate-level `recursion_limit` attribute; the Reference notes that the default in rustc is **128**. Compiled with rustc 1.99.1, a literal with 120 entries builds, and one with 130 fails: ```plaintext error: recursion limit reached while expanding `hashmap!` = help: consider increasing the recursion limit by adding a `#![recursion_limit = "256"]` attribute to your crate ``` The error names the crate that **uses** the macro, which can be a different crate from the one that defines it, so a library's macro can break a downstream build that simply has a long literal. | Approach | Depth used | Notes | |---|---|---| | recursive `@count` muncher | one level per entry | fails past the limit; the remaining tokens are matched again at every step | | `#![recursion_limit = "256"]` | unchanged | moves the ceiling; every user crate must add it | | `<[()]>::len(&[$(hashmap!(@unit $key)),*])` | constant | one helper call per key, side by side, no recursion | ## The non-recursive count The last row maps each key to `()` with `(@unit $_key:tt) => { () }` and measures the resulting array of unit values. All helper calls sit side by side in one array, so the depth stays constant: rustc 1.99.1 builds a 1,000-entry literal this way. The array holds zero-sized values and its length is a constant the optimiser folds. ## A trap when forwarding fragments The Reference states that a fragment forwarded to another macro is seen as an opaque piece of the fragment's type: the receiving macro cannot match it with literal tokens, only with a fragment specifier of the same kind. Fragments captured as `ident`, `lifetime` or `tt` are the exception and stay matchable by literal tokens. rustc also lets a `tt` matcher take an opaque fragment as one token tree, which the muncher above relies on. That means: - forwarding `$key:expr` into an arm `("id") => ...` never matches the literal arm, even when the caller wrote `"id"`; - if a later `$other:tt` arm exists, rustc silently takes it, so the macro picks the wrong branch without an error; - forwarding with `tt` keeps the tokens matchable, which is why munchers capture `tt` rather than `expr` when a later step needs to inspect the input. ## When to reach for a muncher A tt muncher is the only way a declarative macro can parse input that has no fixed shape, such as mixed options or a small embedded grammar. Keep its recursion proportional to something small, prefer flat repetitions when the input is a uniform list, and document the internal rules so callers do not depend on them.
- Why does the recursion-limit error appear in a crate that never wrote a macro?The limit belongs to the crate where the expansion happens, which is the crate that invokes the macro. A library's recursive macro works for short inputs in its own tests and then fails in a downstream crate with a long literal, and only that downstream crate can add `#![recursion_limit]`. A non-recursive design avoids pushing that fix onto users.
- Why do munchers usually capture tt instead of expr?A captured `expr` becomes opaque when forwarded: the next macro can only match it as a whole `expr` or `tt`, never by its tokens. Capturing `tt` keeps each token inspectable, so later steps can still match keywords, literals or punctuation. The cost is weaker validation, so check the shape at the end of munching.
- Arms that start with @ are private to the macro
- rustc has no limit on recursive macro expansion
- A forwarded expr fragment still matches literal-token arms
- Raising the recursion limit in the macro's crate fixes every user
- Arm order never matters because rustc picks the best match