skip to content

What problem does json.RawMessage solve when a JSON envelope's payload shape depends on a type field?

level: middleimportance: should knowfreq 46%

answer

  1. you cannot pick a type before reading one
  2. hold the bytes, decode them later
  3. a []byte that is its own marshaler
  4. the same field also splices bytes back out

basics

~20 s

json.RawMessage is a []byte that implements both JSON interfaces, so a field of that type is filled with the payload's raw bytes instead of being decoded. You read the discriminator first, then unmarshal those bytes into the concrete type.

solid answer

~50 s

A polymorphic envelope has a chicken-and-egg problem: you cannot pick the payload's Go type until you have read the `type` field, but `json.Unmarshal` decodes the whole document in one pass. `json.RawMessage` breaks the tie. It is defined as `[]byte` with a `MarshalJSON` that returns its own bytes and an `UnmarshalJSON` that copies the raw bytes in, so a `Payload json.RawMessage` field captures that part of the input verbatim — syntax-checked, but not parsed into any structure. You then switch on the discriminator and call `json.Unmarshal(env.Payload, &concrete)` with the right type. It works in the other direction too: assigning already-encoded JSON to a `RawMessage` field splices those bytes into the output rather than re-encoding them, which is what lets a transform step rewrite one field and pass everything else through untouched, and what lets a router forward a payload it does not understand.

code

go · 19 lines
go
type Envelope struct {
	Type string `json:"type"`
	Payload json.RawMessage `json:"payload"`
}

var env Envelope
if err := json.Unmarshal(data, &env); err != nil {
	return err
}
switch env.Type {
case "user.created":
	var u UserCreated
	if err := json.Unmarshal(env.Payload, &u); err != nil {
		return err
	}
	return handleUserCreated(u)
default:
	return fmt.Errorf("unknown event type %q", env.Type)
}

go deeper

for a junior

Know that json.RawMessage is a []byte you can put in a struct to hold a chunk of JSON undecoded, and that you decode it later with a second json.Unmarshal call once you know what it is.

for a middle

Explain the two-stage decode of a discriminated envelope, and that RawMessage works in both directions because its own MarshalJSON returns its bytes unchanged, which is how pre-encoded JSON is spliced into output.

for a senior

Weigh what deferring the decode costs: unvalidated bytes travelling further into the system, lost field order when a map is re-encoded, and a null where a caller expected an absent member. Say where you would validate instead of forward.

for a principal

Decide what a shared transform is allowed to do with payloads it does not own. Pass-through bytes keep the platform decoupled from every producer's schema, but they also mean the platform can no longer promise anything about what it forwards.

## The shape of the problem An event bus carries messages that all look like this: ```json {"type":"user.created","id":"9f2","payload":{"email":"[email protected]"}} ``` The outer fields are fixed; `payload` is different for every value of `type`. Go wants a concrete type per field, and `json.Unmarshal` decodes top to bottom in one pass, so a `Payload UserCreated` field is a decision you have to make *before* you have read `type`. The usual first attempt is `Payload map[string]any`. It decodes, but you have thrown the structure away: every number is now a `float64`, every nested object another `map[string]any`, and re-encoding it produces a document that is only accidentally the same as the input. Worse, for a transform that must forward the message onward, you have retyped data you never intended to interpret. ## What json.RawMessage is ```go type RawMessage []byte ``` with two methods that make it interesting to `encoding/json`: - `func (m RawMessage) MarshalJSON() ([]byte, error)` returns `m` itself — or `null` when it is nil. - `func (m *RawMessage) UnmarshalJSON(data []byte) error` copies `data` into `*m`. It is a marshaler and an unmarshaler that do as little as possible. As a struct field it means "remember the bytes of this value; do not interpret them". The decoder still validates: the input as a whole must be well-formed JSON, and the bytes you get are exactly the bytes of that one value from the input. They are copied, so the `RawMessage` outlives the input buffer safely. ## Two-stage decoding ```go type Envelope struct { Type string `json:"type"` Payload json.RawMessage `json:"payload"` } var env Envelope if err := json.Unmarshal(data, &env); err != nil { return err } switch env.Type { case "user.created": var u UserCreated if err := json.Unmarshal(env.Payload, &u); err != nil { return err } ... } ``` Stage one reads the envelope and the discriminator; stage two decodes the payload with full type safety into the struct the discriminator selected. Errors stay precise: a malformed payload fails with a message about the payload's own fields, not about the envelope. Note that `UnmarshalJSON` here is on `*RawMessage`, and it is found because `json.Unmarshal` is given `&env` — struct fields reached through a pointer are addressable, so the decoder can take the address of the field. ## Encoding: bytes straight through Because `RawMessage.MarshalJSON` returns its own bytes, assigning pre-encoded JSON to such a field splices it in as a JSON value: ```go env.Payload = json.RawMessage(`{"email":"[email protected]"}`) ``` This is the half people forget, and it is what makes `RawMessage` a *pass-through* type rather than just a lazy-decode trick. A transform step in a pipeline can decode into `map[string]json.RawMessage`, rewrite the one member it owns, and re-encode: every other member is emitted from the original bytes, so unknown fields survive, numbers keep their exact literal form, and a payload the step does not understand is forwarded unharmed. (Two details to expect: re-encoding a map sorts the member names, so field order changes; and the encoder compacts what your `RawMessage` holds, so interior whitespace from the input is dropped.) The same property makes `RawMessage` useful for a cached or precomputed fragment: encode an expensive sub-document once, keep the bytes, and slot them into many responses without re-encoding. ## Where it is the wrong answer - If you actually need to read the payload's fields in this function, decode it into a struct. `RawMessage` defers work; it does not remove it. - If the goal is "keep whatever the client sent so I can echo it back", think about whether you want to forward bytes you never validated. `RawMessage` guarantees the value is syntactically JSON and nothing more — no schema, no size bound, no check that it is an object. - A `RawMessage` in a struct that crosses a package boundary exports an obligation: every caller must know what to do with those bytes. Sometimes the honest API is an interface plus a decode function that returns a concrete type. ## Gotchas worth naming A nil `RawMessage` encodes as `null`, not as an omitted member or an empty object; if the field must disappear, use `*json.RawMessage` with `omitempty` or drop the field from the wire struct. An empty, non-nil `RawMessage` — zero bytes — is not a valid JSON value and makes the encode fail, which is the usual cause of a `json: error calling MarshalJSON for type json.RawMessage`. And a field declared `json.RawMessage` but never assigned before encoding hits exactly that case, so initialise it or make it a pointer.

  • How is a json.RawMessage field different from a map[string]any field here?
    `map[string]any` decodes the payload immediately and loses its typing — every number becomes a `float64`, every object another map — so re-encoding produces a document only accidentally like the input. `RawMessage` keeps the original bytes, so the payload can later be decoded into the right struct or forwarded unchanged.
  • What does an unset json.RawMessage field encode as?
    A nil `RawMessage` encodes as `null`, since its `MarshalJSON` returns `null` for nil. An empty but non-nil one holds zero bytes, which is not a valid JSON value, so the encode fails with an error naming `json.RawMessage`. Use `*json.RawMessage` with `omitempty` if the member should vanish.
  • Does keeping a payload as raw bytes mean it was never checked?
    It was syntax-checked — the whole document had to parse for the decode to succeed, so the bytes are a well-formed JSON value. Nothing else was checked: no schema, no size limit, no guarantee it is an object. Forwarding it onward is forwarding unvalidated input.

saying these in an interview costs you the question

  • Thinks json.RawMessage skips JSON syntax validation entirely
  • Expects an unset RawMessage field to be omitted rather than null
  • Says the bytes alias the input buffer instead of being copied
  • Assumes re-encoding a map of RawMessage preserves field order
  • Uses map[string]any for a payload that is only being forwarded