skip to content

In Rust, why does rustc reject a JSON-style hashmap macro_rules! matcher $($key:expr : $value:expr),*, and how do you fix it?

level: seniorimportance: should knowfreq 30%

answer

  1. rejected at the definition
  2. future-proofing the grammar
  3. only three tokens after expr
  4. literal and tt are unrestricted

basics

~20 s

An expr fragment may only be followed by =>, a comma or a semicolon, so rustc rejects the colon when it reads the macro definition. Use => as the separator, or capture keys as literal or tt, which any token may follow.

solid answer

~40 s

macro_rules! has **follow-set restrictions**: a fragment must be followed by a token the language promises will never extend that kind of syntax. For `expr` and `stmt` the set is `=>`, `,` and `;`, so rustc 1.99.1 rejects the definition itself with "`$key:expr` is followed by `:`, which is not allowed for `expr` fragments", even if the macro is never called. The point is future-proofing: if `:` were allowed and later became part of expression syntax, existing macros would misparse. Fixes: keep `expr` and use `=>`; switch to `$key:literal :` for literal keys like `"http"` or `-1`; or `$key:tt :` for one-token keys such as a variable name, at the cost of rejecting `prefix.clone(): 22` with "no rules expected `.`". `ty` and `path` have a wider set that includes `:`.

code

rust · 25 lines
rust
use std::collections::HashMap;

macro_rules! hashmap_lit {
    ($($key:literal : $value:expr),* $(,)?) => {{
        let mut map = HashMap::new();
        $( map.insert($key, $value); )*
        map
    }};
}

macro_rules! hashmap_tt {
    ($($key:tt : $value:expr),* $(,)?) => {{
        let mut map = HashMap::new();
        $( map.insert($key, $value); )*
        map
    }};
}

fn main() {
    let ports = hashmap_lit! { "http": 80, "https": 443 };
    let signs = hashmap_lit! { -1: "minus one" };
    let k = "ssh";
    let more = hashmap_tt! { k: 22, "ftp": 21 };
    println!("{} {} {}", ports.len(), signs[&-1], more.len());
}

go deeper

for a junior

Know that a macro_rules! matcher cannot put any token after an expr fragment, and that => and commas are the safe separators.

for a middle

Recite the follow set for expr and explain why a JSON-style colon is rejected while => works, and fix it with literal or tt keys.

for a senior

Diagnose definition-time follow-set errors quickly, explain the future-proofing reason, and weigh what literal, tt and expr keys each accept before choosing the macro's syntax.

for a principal

Prefer macro syntax that stays inside the follow sets, such as => separators, over syntax that forces tt and weaker validation, and make that a review norm for shared macros.

## The symptom A team wants JSON-like syntax, `hashmap!{ "http": 80, "https": 443 }`, and writes: ```rust macro_rules! hashmap { ($($key:expr : $value:expr),* $(,)?) => {{ /* ... */ }}; } ``` rustc 1.99.1 refuses it before any call is compiled: ```plaintext error: `$key:expr` is followed by `:`, which is not allowed for `expr` fragments = note: allowed there are: `=>`, `,` or `;` ``` The error is about the **definition**. A macro that is never invoked still fails, which surprises people who expect macros to be checked only when used. ## Why the rule exists: future-proofing The Reference explains the **follow-set ambiguity restrictions**: the macro parser is limited "to prevent ambiguity in current or future versions of the language." When rustc parses an `expr` fragment it consumes tokens for as long as they can continue an expression. If a macro were allowed to put `:` right after an `expr`, and a later Rust release gave `:` a meaning inside expressions, the same invocation would suddenly parse differently and break working code. So each fragment kind gets a fixed set of tokens that may follow it, chosen because they will never continue that kind of syntax. The Reference's own example: `$i:expr [ , ]` is rejected even though `[,]` is not valid expression syntax today, because `[` can start a trailing index expression. ## The follow sets | Fragment | May be followed by | |---|---| | `expr`, `stmt` | `=>`, `,`, `;` | | `pat` | `=>`, `,`, `=`, `if`, `in` | | `pat_param` | `=>`, `,`, `=`, `\|`, `if`, `in` | | `path`, `ty` | `=>`, `,`, `=`, `\|`, `;`, `:`, `>`, `>>`, `[`, `{`, `as`, `where`, or a `block` metavariable | | `vis` | `,`, an identifier other than a non-raw `priv`, a token that can begin a type, or an `ident`, `ty` or `path` metavariable | | `ident`, `literal`, `tt`, `block`, `item`, `lifetime`, `meta` | anything | The end of the matcher, or of a delimited group such as `( ... )`, is always fine, which is why `($e:expr)` works. The same rule bites elsewhere: `($cond:expr { $($body:tt)* })` for an `if`-like macro is rejected because `{` may not follow `expr`; `($cond:expr => { $($body:tt)* })` is accepted. ## Four ways out 1. **Keep `expr` and change the separator.** `$($key:expr => $value:expr),*` accepts any key expression; this is why std-style map macros use `=>`. 2. **Capture keys as `literal`.** `literal` matches a literal with an optional leading `-` and has no follow restriction, so `$($key:literal : $value:expr),*` accepts `"http": 80` and `-1: "minus one"`, but not a variable. 3. **Capture keys as `tt`.** A single token tree has no follow restriction either. It accepts a variable name `k: 22`, a literal, or anything wrapped in parentheses, `(a.b()): 1`. A multi-token expression such as `prefix.clone(): 22` fails with "no rules expected `.`", because `tt` took only `prefix` and the matcher then wanted `:`. 4. **Use `ident`** when keys are always bare names, for example field names in a struct-like syntax. What each option accepts: | Key matcher | `"http": 80` | `k: 22` | `-1: 0` | `prefix.clone(): 22` | |---|---|---|---|---| | `$key:literal :` | yes | no | yes | no | | `$key:tt :` | yes | yes | no, `-1` is two tokens | no | | `$key:expr =>` (with `=>`) | yes | yes | yes | yes | ## How to diagnose it quickly - The error appears at the macro definition, with a caret on the forbidden token and a note listing the allowed tokens; it never mentions the call sites. - It is a hard error, not a lint, so `#[allow]` cannot silence it. - If you need the forbidden token, change the fragment, not the token: wrap the fragment in a group, `($key:expr) : $value:expr`, so callers write `(p.clone()): 1`, or switch to `tt` or `literal`. The practical rule for reviews: anything after an `expr` other than `=>`, `,` or `;` is a definition-time error, and the fix is to pick the syntax around the fragment, not to fight the parser.

  • Why does the error appear even though the macro is never called?
    rustc checks follow sets when it reads a `macro_rules!` definition, because the restriction is about the matcher's grammar, not about any particular input. A macro whose matcher could become ambiguous in a future release is rejected up front, so no call site can ever depend on it.
  • Does the rule only concern the token right after the fragment?
    It also applies through repetitions: a repetition's separator must be allowed after its contents, and whatever follows the repetition must be allowed after them too. `$($e:expr),*` is fine because `,` may follow `expr`; a `:` separator would be rejected for the same reason as in the JSON-style matcher.

  • rustc only checks macro matchers when the macro is invoked
  • Any token that cannot start an expression may follow an expr
  • A #[allow] attribute can silence the follow-set error
  • tt keys accept any expression, method calls included
  • ty fragments have the same follow set as expr