skip to content

Custom Derive Macros

A derive macro is a compiled function from TokenStream to TokenStream, usually written with syn and quote. Interviewers ask how to write one and report errors at the right span.

on this pageshow

questions

5

In Rust, what does a custom derive such as #[derive(Builder)] do at compile time, and can it change the struct it annotates?

level: juniorimportance: should knowfreq 45%

answer

  1. a function the compiler runs
  2. tokens in, tokens out
  3. new items after the item
  4. redefining the input: E0428

basics

~20 s

A custom derive is a function in a proc-macro crate that rustc runs while compiling: it receives the struct's tokens and returns new items, which rustc appends after the struct. It cannot edit or remove the struct itself.

solid answer

~40 s

`#[derive(Builder)]` calls a function marked `#[proc_macro_derive(Builder)]` in a separate crate built as a procedural-macro library. rustc hands that function the struct, enum or union definition as a `proc_macro::TokenStream`, and the function returns another `TokenStream` that must be zero or more items: usually an `impl` block, sometimes a companion type such as `ServerConfigBuilder`. rustc appends those items after the original item in the same module or block, and the original definition stays exactly as written. So a derive adds code but never rewrites the struct: if it returns its input, the type is defined twice and rustc reports E0428. A panic inside the derive becomes a compile error, not a run-time failure.

code

rust · 7 lines
rust
// builder_derive/src/lib.rs, in a crate built as a proc-macro library
use proc_macro::TokenStream;

#[proc_macro_derive(Reemit)]
pub fn derive_reemit(input: TokenStream) -> TokenStream {
    input // a derive's output is appended after the item, so this defines it twice
}

go deeper

for a junior

Know that a derive is code that runs inside the compiler and writes extra items, typically an impl block, next to your type. Be able to say it never changes the type itself.

for a middle

Explain the TokenStream-in, items-out contract and that output is appended after the item. Mention the separate proc-macro crate and that a panic becomes a compile error.

for a senior

Show you know the consequences: E0428 if the derive re-emits its input, why rewriting needs an attribute macro instead, and why derives report errors with spans instead of panicking.

for a principal

Point out that derives run arbitrary code inside every build, with the compiler's file and environment access, so a team reviews a derive dependency the way it reviews a build script.

## What a custom derive is A **custom derive** (also called a **derive macro**) is one of Rust's three kinds of **procedural macro**. You use it exactly like a standard derive: `#[derive(Builder)]` on a struct. Behind that name sits an ordinary Rust function, compiled ahead of time into a library that the compiler loads and **calls while it compiles your crate**. Nothing happens at run time: by the time your binary exists, the derive has already done its work and left behind plain Rust code. The function is declared like this: - it is `pub`, lives at the root of its crate, and has the signature `fn(TokenStream) -> TokenStream`, where `TokenStream` is `proc_macro::TokenStream` from the compiler-provided `proc_macro` crate; - it carries `#[proc_macro_derive(Builder)]`, which makes `Builder` usable inside `#[derive(...)]`; - its crate is built as a procedural-macro library (in Cargo, `proc-macro = true` under `[lib]`). ## The contract: tokens in, items out When rustc meets `#[derive(Builder)]` on `struct ServerConfig { ... }`, it does the following: 1. It turns the item definition, the struct with its attributes, fields and generics, into a **token stream**: identifiers, punctuation, literals and delimited groups, each carrying a **span** that records where in your source it came from. 2. It calls the derive function with that stream. 3. It parses whatever the function returns as **items** (an `impl` block, a struct, a function, a constant) and **appends them after the original item**, in the same module or block. 4. It compiles the result as if you had typed those items yourself. The Reference states it plainly: the output "must be a (possibly empty) set of items", and those items "are appended following the input item". ## Why a derive cannot change the struct Because the output is **added**, not **substituted**, a derive has no way to rename a field, add a field, change a type or delete the struct. The original definition is compiled no matter what the derive returns. A derive that naively returns its own input demonstrates this: the struct now appears twice and rustc rejects the crate with **E0428**, "the name `Point` is defined multiple times". This is the line between a derive and the other procedural macros. An **attribute macro** receives an item and its output **replaces** that item, so it can rewrite it; a **function-like macro** replaces its own invocation. Those two kinds are covered with attribute and function-like macros; the derive's rule is "append only". | Macro kind | Invoked as | Effect on the annotated code | |---|---|---| | Derive | `#[derive(Builder)]` on a struct, enum or union | output appended after it; item untouched | | Attribute | `#[my_attr]` on an item | output replaces the item | | Function-like | `my_macro!(...)` | output replaces the invocation | ## Where the function lives A procedural macro must be compiled before the crate that uses it, so it lives in its **own crate**. rustc enforces two consequences: - you cannot use a procedural macro in the crate that defines it ("can't use a procedural macro from the same crate that defines it"); - a procedural-macro crate may export only its macro functions, so a trait the derive implements (say `Describe`) lives in an ordinary library crate, often re-exported beside the macro. ## Errors and side effects A derive reports failure in one of two ways: it **panics**, which rustc catches and turns into a compile error ("proc-macro derive panicked") pointing at the derive name, or it returns tokens containing a `compile_error!` invocation, which lets it point at the exact field at fault. An infinite loop is not caught and hangs the build. Because the function runs inside the compiler with the compiler's access to files, environment and standard streams, the Reference notes that procedural macros carry the same security concerns as build scripts. ## Misconceptions to avoid - A derive is **not reflection**: there is no run-time type inspection, only code written at compile time. - A derive is **not limited to standard-library traits**: any crate can define one, and its output can be any items. - The derive sees **tokens, not types**: it knows a field is written `Vec<String>`, not what traits that type implements.

  • What does the crate using a derive see when the derive function panics?
    rustc catches the panic and reports a compile error, "proc-macro derive panicked", underlining the derive name with the panic message as help. The build fails cleanly, but the error points at `#[derive(...)]` rather than at the field that caused it, so well-written derives return a `compile_error!` with a precise span instead of panicking.
  • Why can't the trait a derive implements live in the same crate as the derive?
    A procedural-macro crate is compiled into a library the compiler loads, and rustc lets it export only functions tagged `#[proc_macro]`, `#[proc_macro_derive]` or `#[proc_macro_attribute]`. A public trait in it is rejected, so the trait goes into an ordinary library crate, which often depends on the macro crate and re-exports the derive next to the trait.

A derive is like a clerk who reads a filed form and staples new pages behind it: the clerk can add anything after the form, but cannot cross out or rewrite what is already on it.

  • A derive macro inspects types at run time, like reflection.
  • A derive can add or rename fields of the struct it annotates.
  • The derive's output replaces the struct, so it must re-emit the struct.
  • Custom derives can only implement traits from the standard library.
  • A derive macro can be defined and used in the same crate.
open as a page

In a Rust derive macro, how do you accept a #[builder(default)] field attribute, and why does rustc reject it as unknown without that?

level: middleimportance: should knowfreq 30%

basics

~20 s

Declare it in the derive as #[proc_macro_derive(Builder, attributes(builder))]. That registers builder as an inert helper attribute on the item and its fields; without it rustc treats #[builder] as an unknown attribute. The derive then finds and parses it in each Field's attrs.

open as a page

How do you write a Rust #[proc_macro_derive] function with syn and quote that generates a builder for a named-field struct?

level: middleimportance: should knowfreq 40%

basics

~20 s

Parse the input with syn's parse_macro_input!(input as DeriveInput), take the struct's ident and named fields from its data, build the output with quote! using #name interpolation and #( ... )* repetitions, then return it with .into().

open as a page

A Rust derive's error about one field points at #[derive(Defaults)] instead of the field; how do syn::Error spans and quote_spanned! fix that?

level: seniorimportance: should knowfreq 25%

basics

~20 s

Tokens written inside quote! get Span::call_site(), the derive invocation, so rustc blames #[derive(...)]. Report your own checks with syn::Error::new_spanned(node, msg).to_compile_error(), and wrap per-field code in quote_spanned!{field.ty.span()=> ...} so rustc's own errors land on the field.

open as a page

A Rust derive emits `impl #name { ... }` and fails with E0107 on `struct Labeled<T>`; why, and how does syn's split_for_impl fix it?

level: seniorimportance: should knowfreq 30%

basics

~20 s

The generated header names Labeled without its parameters, so rustc sees impl Labeled and reports missing generics. Generics::split_for_impl returns the declared parameters with bounds, the bare argument list, and the where clause, to place as impl #impl_generics #name #ty_generics #where_clause.

open as a page