skip to content

In Ruby 3.4 and 4.0, what changed when Prism became CRuby's default parser, and when would you run ruby --parser=parse.y?

level: middleimportance: nice to knowfreq 20%

answer

  1. 3.3 opt-in, 3.4 default
  2. +PRISM in ruby -v
  3. --parser=parse.y or RUBYOPT
  4. AbstractSyntaxTree.of raises under Prism
  5. Prism.parse as the portable API

basics

~20 s

Since Ruby 3.4, CRuby parses code with Prism instead of parse.y; programs should behave the same. Run ruby --parser=parse.y to rule out a Prism bug or to keep MRI-only AST tooling such as RubyVM::AbstractSyntaxTree.of working.

solid answer

~40 s

Ruby 3.3 shipped Prism as a default gem with `--parser=prism` for debugging only; **Ruby 3.4 made Prism CRuby's default parser**, and 4.0 keeps it. The language did not change: Prism builds the tree that CRuby compiles to YARV bytecode, and `ruby -v` shows `+PRISM` in its description. You pass `--parser=parse.y` (or put it in `RUBYOPT`) in two cases: to check whether a parse failure or odd behaviour is a **Prism bug**, and to keep tools working that read MRI's own AST, because `RubyVM::AbstractSyntaxTree.of` raises `RuntimeError` ("cannot get AST for ISEQ compiled by prism") on Prism-compiled code. The long-term fix for such tools is the `Prism` API itself, `Prism.parse(source)`, which JRuby and TruffleRuby also use.

code

bash · 3 lines
bash
ruby -v                           # description includes +PRISM
ruby --parser=parse.y -v          # same Ruby, no +PRISM
RUBYOPT="--parser=parse.y" bundle exec rake test

go deeper

for a junior

Recall that CRuby 3.4 and 4.0 parse code with Prism by default and that ruby -v shows +PRISM.

for a middle

Explain where the parser sits: it builds the tree that CRuby compiles to YARV bytecode. Show --parser=parse.y and RUBYOPT for switching.

for a senior

Use the parser switch as a diagnostic during upgrades, recognise the AbstractSyntaxTree.of RuntimeError in older tooling, and plan the move to the Prism API.

for a principal

Treat the shared Prism parser as ecosystem infrastructure: tooling built on it is portable across engines, so steer internal linters and code generators toward it.

## What a parser swap means Every Ruby engine first **parses** source text into a syntax tree and then turns that tree into something executable. For decades CRuby's parser was `parse.y`, a grammar file that a parser generator (Bison, replaced by Lrama in 3.3) turned into C. **Prism** is a new, hand-written recursive-descent parser for Ruby, designed to be portable, error tolerant and usable as a library. The timeline: 1. **Ruby 3.3** - Prism ships as a default gem; `ruby --parser=prism` exists but is marked "for debugging only". 2. **Ruby 3.4** - **Prism becomes CRuby's default parser**. `--parser=parse.y` selects the conventional parser. 3. **Ruby 4.0** - Prism stays the default; Ruby 4.0.7 ships prism 1.8.1 as a default gem. The switch is meant to be invisible: the same program means the same thing, and CRuby compiles Prism's tree to the same kind of YARV instruction sequences. ## How to tell which parser ran - `ruby -v` and the `RUBY_DESCRIPTION` constant append **`+PRISM`** when Prism is the active parser. Run `ruby --parser=parse.y -v` and it disappears. - `--parser` accepts exactly `prism` or `parse.y`; any other value stops the interpreter with `RuntimeError` "unknown parser". - `--parser` is also accepted inside `RUBYOPT`, which is how you flip a whole test suite or a process manager's children without editing commands. - `RubyVM::InstructionSequence.compile` uses the current parser, and `compile_prism` / `compile_parsey` force one, which helps when bisecting a difference. ## When you would reach for --parser=parse.y - **Suspected parser bug.** A file that fails to parse, or behaves differently, only on 3.4+ can be tested under both parsers in seconds. If `--parser=parse.y` fixes it, you have a minimal reproduction for an upstream report and a temporary workaround. - **Tools that read MRI's own AST.** `RubyVM::AbstractSyntaxTree` exposes parse.y's node tree and is documented as MRI-specific and experimental. Its `.of(method_or_proc)` locates the source of already-compiled code, and when that code was compiled by Prism it raises `RuntimeError`: "cannot get AST for ISEQ compiled by prism". Older debugging, coverage or source-extraction tools that call it break on a default 3.4+ process. - **Comparing behaviour during an upgrade.** Running the suite once with each parser separates "the parser changed" from "the upgrade changed something else". A practical routine during an upgrade is to run the suite twice in CI, once with the default and once with `RUBYOPT="--parser=parse.y"`, only until the first run is green; after that the second job has done its work and should be removed. Treat `--parser=parse.y` as a diagnostic or a short bridge, not a permanent setting. The long-term direction is Prism everywhere. ## The Prism API for tooling The `prism` gem is Ruby's official API for parsing Ruby code, and `RubyVM::AbstractSyntaxTree`'s own docs point users to it for anything that must be stable or work across engines: - `Prism.parse(source)` returns a `Prism::ParseResult`; `#value` is the program node, `#errors` lists problems, `#success?` says whether it parsed cleanly. - `Prism.parse_success?(source)` and `Prism.parse_comments(source)` cover common checks. - `Prism::Translation` offers adapters, including a Ripper-compatible one, for tools written against older parser APIs. Prism is not CRuby-only. JRuby and TruffleRuby also build on Prism's tree, and the gem loads as a C extension on CRuby and through FFI elsewhere (`Prism::BACKEND` says which). That shared parser is one reason syntax support is converging across engines. | | parse.y | Prism | |---|---|---| | Default in CRuby | up to 3.3 | 3.4 and 4.0 | | Selected with | `--parser=parse.y` | default, or `--parser=prism` | | Library API | Ripper, `RubyVM::AbstractSyntaxTree` (MRI-only) | `Prism.parse` and friends | | Used by other engines | no | JRuby, TruffleRuby | ## Error tolerance Prism also changes what a syntax error feels like. It is **error tolerant**: it keeps parsing after the first problem, so editors and linters built on it can report several mistakes in one pass and still produce a partial tree. That is a tooling benefit, not a change to which programs are valid. ## What interviewers listen for That the parser changed and the language did not; that `+PRISM` and `--parser=parse.y` are the tools for checking; and that AST-dependent tooling is where the switch really bites.

  • Did Prism change what any Ruby program means?
    It is not supposed to. Prism implements the same grammar and CRuby compiles its tree to YARV bytecode as before; the switch is an implementation change. Where the two parsers differ on an edge case, that is a bug, and running with `--parser=parse.y` is how you confirm and report it.
  • How would you migrate a tool that relies on RubyVM::AbstractSyntaxTree?
    Move it to the `prism` gem: parse source with `Prism.parse` or `Prism.parse_file` and walk the resulting nodes. That API is documented as the official, stable way to parse Ruby and works on JRuby and TruffleRuby too. Until then, run the tool's process with `--parser=parse.y`.

saying these in an interview costs you the question

  • Prism became the default parser in Ruby 3.3
  • Switching to Prism changed Ruby's syntax and semantics
  • --parser=parse.y makes Ruby 4.0 behave like Ruby 3.3
  • RubyVM::AbstractSyntaxTree is the portable way to parse Ruby code
  • Prism replaces YARV, so Ruby no longer compiles to bytecode