skip to content

What are masks in the Vector API, and how do they let you handle loop tails and conditional (branchless) computation?

level: seniorimportance: should knowfreq 25%

answer

  1. mask = one boolean per lane = enable switch
  2. indexInRange(i, n) → safe partial-block mask
  3. masked load/store never touch out-of-range lanes
  4. compare → mask → blend = branchless per-lane select
  5. AVX-512 mask registers k0..k7

basics

~20 s

A mask is a per-lane on/off switch. It tells an operation which lanes to actually apply to. You use it to safely process a partial final chunk of an array and to do if-style logic without branches by selecting values per lane.

solid answer

~50 s

A VectorMask<E> is a boolean for each lane that gates a vector operation: only lanes where the mask is true take effect, the rest keep their prior (or default) value. Two big uses. First, loop tails: instead of a separate scalar remainder loop, you build a mask from the species (S.indexInRange(i, n)) that marks only the valid lanes in the final partial block, then do one masked load/op/store so the out-of-range lanes are inert and never read or write past the array. Second, branchless conditionals: a lane-wise comparison like va.compare(GT, vb) produces a mask, which you feed to blend() to select between two vectors per lane, or to a masked arithmetic op. This replaces data-dependent branches that would defeat vectorization, keeping the whole computation in vector registers. Masks are the mechanism that makes the Vector API both safe at array edges and expressive for conditional numeric code.

code

java · 12 lines
java
static final VectorSpecies<Float> S = FloatVector.SPECIES_PREFERRED;

// ReLU with a masked tail: out[i] = max(in[i], 0), single loop, no scalar remainder
static void relu(float[] in, float[] out) {
    for (int i = 0; i < in.length; i += S.length()) {
        VectorMask<Float> m = S.indexInRange(i, in.length);
        FloatVector v = FloatVector.fromArray(S, in, i, m);   // masked load (false lanes = 0)
        VectorMask<Float> pos = v.compare(VectorOperators.GT, 0.0f);
        v.blend(0.0f, pos)                                     // keep positives, else 0
         .intoArray(out, i, m);                                // masked store
    }
}

go deeper

for a junior

Knows a mask turns lanes on or off and helps avoid going past the end of an array.

for a middle

Can use indexInRange to mask the tail and understands masked load/store stay in bounds.

for a senior

Uses compare/blend for branchless conditionals, knows masks map to AVX-512 mask registers, and judges masked-tail vs scalar-tail trade-offs.

for a principal

Reasons about predication cost across architectures, composes masks for complex conditions, and decides masking strategy as part of a tuned numeric kernel.

## What a mask is A **`VectorMask<E>`** is a vector of **booleans**, one per lane, that acts as an **enable switch**. When you pass a mask to an operation, the operation is applied **only to the lanes whose mask bit is true**; lanes whose bit is false are left untouched (they keep a previous value or a supplied default). Think of it as a per-lane `if` that runs all branches' arithmetic but only *commits* results where the condition holds. Masks map directly to hardware: AVX-512 has dedicated mask registers (`k0`–`k7`), and other architectures emulate masking with blends. So masked code is not a software hack — it lowers to real predicated instructions where available. ## Use 1: safe loop tails The remainder problem: arrays rarely divide evenly by the lane count, so the last block is partial. A naive masked tail: ```java for (int i = 0; i < n; i += S.length()) { VectorMask<Float> m = S.indexInRange(i, n); // true for lanes i..n-1, false beyond var va = FloatVector.fromArray(S, a, i, m); // masked load: out-of-range lanes read 0, not the array va.add(1.0f).intoArray(c, i, m); // masked store: only valid lanes written } ``` Here `indexInRange(i, n)` builds a mask that is `true` exactly for lanes still inside `[0, n)`. A **masked load** does not touch memory for false lanes (so no out-of-bounds read), and a **masked store** writes only the true lanes (so no out-of-bounds write). This lets a *single* loop body handle both full and partial blocks, eliminating the separate scalar tail — though a scalar tail is often just as fast and simpler. ## Use 2: branchless conditional computation Data-dependent branches (`if (a[i] > b[i]) ... else ...`) are poison for SIMD because different lanes might take different branches. The Vector API solves this with **compare → mask → select**: ```java VectorMask<Float> gt = va.compare(VectorOperators.GT, vb); // lane-wise a>b ? FloatVector result = va.blend(vb, gt); // pick va where gt true, else vb ``` - A **comparison** (`compare`) returns a mask: one bit per lane saying whether the condition held in that lane. - **`blend(other, mask)`** produces a new vector taking each lane from `this` where the mask is true and from `other` where false — a per-lane ternary, with no actual branch. - You can also pass the mask to arithmetic (`va.add(vb, mask)`) to apply the op only to selected lanes. This keeps the computation fully vectorized: all lanes execute uniformly, and the mask decides which results survive. The classic example is `max(a, b)` per element, ReLU in ML (`max(x, 0)`), or clamping. ## Why masks matter - **Correctness at edges**: masked loads/stores never read or write outside the array. - **Branchless logic**: replaces unpredictable, vectorization-killing branches with predicated, uniform execution. - **Composability**: masks combine (`and`, `or`, `not`), letting you express complex conditions across lanes. ## Caveats - Masked operations can be **slower than unmasked** on hardware without first-class mask support, so for hot inner loops a scalar tail is sometimes preferred over a masked tail. - Masked **loads** still must not be used to imply you can index arbitrarily — the index base must be valid for the true lanes.

  • How would you compute an element-wise maximum without a branch?
    Build a mask m = a.compare(GT, b), then a.blend(b, m) — or simply use a.max(b) if provided. The mask/blend form generalizes to any condition.
  • When is a scalar tail preferable to a masked tail?
    On hardware without efficient mask registers, masked vector ops can be slower than a few scalar iterations. For very hot kernels, profiling often favors a plain scalar remainder loop for its simplicity and speed.

saying these in an interview costs you the question

  • Thinking a mask changes the vector's size — it only gates which lanes act
  • Believing masked tails are always faster than a scalar tail (often not, on weak-masking hardware)
  • Using data-dependent if/else inside the lane loop instead of compare+blend, defeating vectorization
  • Assuming masked loads read garbage from out-of-range lanes — they read a defined default (zero)

context

resolving links…