skip to content

Attribute and Function-Like Macros

Attribute macros rewrite the item they decorate and function-like macros parse any token input, which powers embedded DSLs. Interviewers ask about compile-time cost and when not to use one.

on this pageshow

questions

5

In Rust, how does a function-like procedural macro such as `sql!("SELECT id FROM users")` check a query at compile time, and what limits its input?

level: middleimportance: should knowfreq 40%

answer

  1. one stream, inside the delimiters
  2. invocation replaced by the output
  3. input must lex as Rust
  4. why a string literal
  5. errors as compile_error!

basics

~20 s

A #[proc_macro] function gets the tokens inside sql!( ... ), runs Rust over them during compilation, and returns tokens that replace the call, or a compile_error!. The input must lex as Rust tokens, so SQL travels as a string literal.

solid answer

~40 s

A function-like procedural macro is a `pub fn(TokenStream) -> TokenStream` marked `#[proc_macro]`. Its input is everything inside the invocation's delimiters, and its output replaces the whole `sql!(...)` call, in expression, statement, pattern, type or item position. To check a query, the macro parses the input (here as a `syn::LitStr`), runs any Rust it likes over the text, and either expands to the literal or returns a `compile_error!` spanned on the literal, so the error underlines the query. A panic also becomes a compile error, `proc macro panicked`. The limit is the lexer: rustc tokenizes the input before the macro runs, so delimiters must balance and `'bob smith'` fails as `E0762`, an unterminated character literal. The macro sees only tokens: no types, and no schema unless it reads files or the network during the build.

code

rust · 14 lines
rust
use proc_macro::TokenStream;
use quote::quote;
use syn::{LitStr, parse_macro_input};

#[proc_macro]
pub fn sql(input: TokenStream) -> TokenStream {
    let query = parse_macro_input!(input as LitStr);
    if !query.value().trim_start().to_ascii_uppercase().starts_with("SELECT ") {
        return syn::Error::new(query.span(), "sql!: only SELECT queries are allowed")
            .to_compile_error()
            .into();
    }
    quote! { #query }.into()
}

go deeper

for a junior

Recall that sql!(...) can be a compiled Rust function that receives the tokens between the parentheses and returns the code that replaces the call.

for a middle

Explain why the input must lex as Rust, why query macros take a string literal, and how a spanned compile_error! beats a panic for reporting a bad query.

for a senior

Show where compile-time checking stops: syntax and schema-file checks in the macro, type checks pushed into generated code, and builds kept reproducible when the macro reads outside its input.

for a principal

Judge whether compile-time validation is worth a proc-macro dependency and a build that may touch files or services, against a runtime check or a plain typed API.

## What a function-like procedural macro is Rust has two ways to write a macro that is called like `name!(...)`: a **declarative** `macro_rules!` macro, which matches the input against patterns, and a **function-like procedural macro**, which is a compiled Rust function. The procedural kind is: - a `pub fn(TokenStream) -> TokenStream` at the root of a `proc-macro` crate; - marked `#[proc_macro]`, which defines a macro with the function's name; - given the tokens **inside** the invocation's delimiters (`(...)`, `[...]` or `{...}`), not the name and not the delimiters. The returned stream **replaces the entire invocation**. Because the Reference allows these macros in any macro invocation position (statements, expressions, patterns, types, items, and items inside `extern` blocks, `impl` blocks and trait definitions), one macro can expand to a value, a type or a whole set of items. ## Checking a query at compile time The `sql!` macro in the example parses its input as a string literal with `syn::parse_macro_input!(input as LitStr)`, reads the text with `value()` and checks it. Here the check is only "starts with SELECT", but it could run a full SQL parser, because the macro body is ordinary Rust running on the build machine. On success it expands to the literal, so `let q: &str = sql!("SELECT id, name FROM users");` compiles to a plain `&str`. On failure it returns `syn::Error::new(span, msg).to_compile_error()`, which expands to a `compile_error!` invocation carrying the literal's span. The error then underlines the query in the user's code, as the second example shows. Choosing those spans well belongs with custom derive macros. A function-like macro uses the same tools. The Reference allows two ways to report an error: 1. **Emit `compile_error!`**, which gives a precise span and can report several errors. 2. **Panic.** The compiler catches it and prints `error: proc macro panicked`, pointing at the whole invocation, with the panic message as help. That works, but it underlines the entire call instead of the bad token. A macro that loops forever is not caught. It hangs the compiler. ## What limits the input | Limit | Why | Consequence | |---|---|---| | Input must lex as Rust tokens | rustc tokenizes the call before the macro runs | `'bob smith'` fails with `E0762`; unbalanced `(` is rejected | | Only tokens, no meaning | expansion runs before type checking | the macro cannot ask the type of `user_id`; schema knowledge must come from outside the tokens | | No view outside the call | the macro gets only its own input | it cannot read the surrounding function or other invocations | The first limit is why query macros take a **string literal**. Bare tokens such as `sql!(SELECT * FROM posts WHERE id = 1)` work while the SQL happens to be valid Rust tokens. A single-quoted SQL string is not, and the build stops in the lexer with `unterminated character literal`, before the macro has run. A string literal can hold any SQL (a raw string such as `r#"..."#` avoids escaping double quotes), and `LitStr::value()` gives back the unescaped text. The second limit decides what "checked at compile time" can mean. A macro can check **syntax**, count placeholders against arguments, or compare the text with a schema file. Anything that needs Rust types has to be done by the code it generates. For example, it can expand to a call whose generic bounds make rustc check the argument types. ## Reaching outside the tokens A procedural macro runs with the compiler's own resources. It can read files, environment variables and the network, and the Reference warns that this raises the **same security concerns as build scripts**. A `sql!` that connects to a live database at build time makes every build depend on that database being reachable and unchanged. Checking against a **committed schema file** keeps builds reproducible and offline. ## Function-like or declarative? - Choose `macro_rules!` when the input fits its fragment specifiers and the output is a fixed template. You need no extra crate, no syn, and no build cost beyond the expansion itself. - Choose `#[proc_macro]` when the macro must **compute**: parse a foreign language, validate it, or generate identifiers and tables from the input. - Either kind expands before type checking, so neither can make decisions based on Rust types. Both can only emit code that rustc then checks. The cost of the procedural route is a proc-macro crate and, usually, syn on the build's critical path, so the extra power should be worth that.

  • What does the user see if `sql!` checks the query with `assert!` and the assertion fails?
    The compiler catches the panic and reports `error: proc macro panicked` at the invocation, with the panic message as a help note. The build fails as it should, but the error underlines the whole `sql!(...)` call, not the bad part of the query, and there is room for only one message. Returning a spanned `compile_error!` points at the exact token and can report several problems.
  • Why can't `sql!` check that a bound parameter's Rust type matches the column's type?
    Macros expand before type checking, so the macro sees the token `user_id` but not its type. It can generate code that makes rustc do the check, for instance a call to a generic function whose trait bound encodes the column's type. The type error then appears during normal compilation, after expansion.

  • A function-like proc macro can accept any text, even unbalanced quotes
  • The macro can look up the types of the variables passed to it
  • A panic inside the macro is a runtime panic when that line executes
  • Function-like procedural macros can only be used in expression position
  • A compile-time query check cannot reach outside the source, so it is always reproducible
open as a page

In Rust, what does a `#[proc_macro_attribute]` function receive for `#[route(GET, "/users")] fn list_users()`, and what replaces that function?

level: middleimportance: should knowfreq 48%

basics

~10 s

It receives two TokenStreams: the attribute's arguments (GET, "/users") and the whole decorated item with its later attributes. Whatever it returns replaces the item, so the macro must re-emit the function to keep it.

open as a page

After a Rust workspace adopts a `#[route]` attribute macro built on syn with its `full` feature, builds slow down — where does the time go, and how do you cut it?

level: seniorimportance: should knowfreq 35%

basics

~20 s

The time goes to building the macro crate and syn's full syntax tree before any user crate starts, re-running the macro on every compile, and compiling the code it emits. Cut it by parsing less, enabling fewer features and generating less code.

open as a page

How do you test a Rust attribute macro such as `#[route]`, including its compile errors, when `proc_macro` APIs panic outside a macro?

level: seniorimportance: should knowfreq 30%

basics

~20 s

Write the transformation over proc_macro2 types and unit-test it with syn::parse2, check behaviour and error messages with trybuild's pass and compile_fail cases against .stderr snapshots, and inspect the generated code with cargo expand or nightly -Zunpretty=expanded.

open as a page

Your Rust team is offered a `#[route]` attribute macro and an `sql!` checker; when do you accept a procedural macro over a generic API or `macro_rules!`?

level: principalimportance: should knowfreq 25%

basics

~20 s

Accept one only when a function, a generic API or macro_rules! cannot do the job: reading an item, generating new identifiers, or validating a language at compile time. Keep it thin over a normal API and budget its build and tooling cost.

open as a page