skip to content

In JMeter's Response Assertion, what do the Not and Or checkboxes change about a pattern list?

level: middleimportance: should knowfreq 54%

answer

  1. Two checkboxes, two different levels
  2. One inverts, the other combines
  3. Per pattern versus per list
  4. Both ticked is surprisingly weak
  5. Watch which pattern stops evaluation

basics

~20 s

Not inverts each individual pattern's outcome, so a pattern passes when it is not found. Or changes how the list combines: any one pattern passing is enough, instead of all of them having to pass.

solid answer

~50 s

They are two independent modifiers sitting beside the four matching rules, and they act at different levels. **`Not`** flips the result of **each pattern**: with `Not` ticked, a pattern passes precisely when it is *not* found in the field. **`Or`** changes how the **list combines**: by default the patterns are ANDed and every one must pass, and with `Or` ticked the assertion succeeds as soon as one pattern passes. They also change the short-circuit: in AND mode evaluation stops at the first pattern that fails, so later patterns are never tested and never appear in the failure message; in OR mode it stops at the first pattern that passes, and if none does, the failure message concatenates the text for every pattern tried. Because they compose, `Not` plus `Or` reads as "at least one of these strings is absent".

code

xml · 5 lines
xml
<!-- Substring (16) with Not (4) ticked -->
<intProp name="Assertion.test_type">20</intProp>

<!-- Contains (2) with Or (32) ticked -->
<intProp name="Assertion.test_type">34</intProp>

go deeper

for a junior

Know that the two checkboxes exist next to the matching rules and that one means "must not be found" while the other relaxes the list from all-must-pass to any-may-pass.

for a middle

Explain the levels they act at: Not inverts each pattern's own result, Or replaces the AND combination of the list. Be able to give the passing condition for all four combinations.

for a senior

Point out the practical traps: Not plus Or is a near-always-true condition, AND mode hides later broken patterns by short-circuiting, and an empty field passes a Not assertion instead of failing it.

for a principal

Decide how much boolean logic a plan may carry inside one element before it should be split into separate assertions or moved into script, and make that reviewable across the plans your teams maintain.

## Two modifiers, two different levels The **Pattern Matching Rules** panel holds a radio group of four rules and two checkboxes beside it, labelled `Not` and `Or`. They are stored as separate bits on the same underlying property, so they are independent of the rule and of each other. What matters when you read a plan is that they act at different levels: - **`Not` acts on each pattern.** For every entry in *Patterns to Test*, the assertion computes whether the pattern was found and then inverts that answer. - **`Or` acts on the list as a whole.** It replaces the default AND combination with an OR combination. ## What Not actually inverts `Not` is not a switch that flips the assertion's final verdict at the end. It flips each pattern's own outcome before the combination happens. With three patterns, `Not` ticked and `Or` clear, the assertion passes only when **none** of the three is found — every inverted pattern has to pass. That is usually what people want when they write "the body must not contain `"error"`, `"exception"` or `"stack trace"`". One special case falls out of this. When the selected field is empty, a normal assertion fails with `Response was null`, but a `Not` assertion **succeeds** — nothing can be found in an empty field, so every inverted pattern passes. ## What Or actually combines With `Or` clear, the patterns are ANDed: all of them must pass. With `Or` ticked, the assertion is satisfied by the first pattern that passes. The evaluation order is visible in the failure messages, and this is what people usually notice first: 1. **AND mode** stops at the **first pattern that fails**. Its failure text becomes the assertion's message; the remaining patterns are never evaluated, so a second broken pattern is invisible until the first one is fixed. 2. **OR mode** stops at the **first pattern that passes**. If none passes, the failure message concatenates the text collected for every pattern that was tried, so you see the full set. ## The four combinations | Not | Or | The assertion passes when | |---|---|---| | clear | clear | every pattern is found | | clear | ticked | at least one pattern is found | | ticked | clear | no pattern is found | | ticked | ticked | at least one pattern is absent | The bottom-right cell is the one worth pausing over. `Not` plus `Or` is a very weak condition — it passes as long as any single string is missing — and it is almost never what an author intended when they ticked both boxes reaching for "none of these". "None of these" is `Not` alone. ## Where the modifiers do and do not help Things the modifiers make easy: - **A denylist of error markers.** `Not` with `Substring` and several patterns, one per marker. - **Accepting either of two valid shapes.** `Or` with `Contains` and one expression per shape. Things they cannot express: - **Different rules per pattern.** The radio group is a property of the element, so one assertion cannot apply `Substring` to one pattern and `Matches` to another. Use two assertions. - **Different fields per pattern.** *Field to Test* is likewise per element. - **Nested boolean logic.** There is no grouping, no precedence and no parentheses; the list is flat. When the logic outgrows a flat list, the answer is more than one assertion under the sampler, or a JSR223 Assertion where the condition can be written as ordinary code. ## Reading it back out of the .jmx The rule and both modifiers are stored as one integer, `Assertion.test_type`, built by OR-ing bit values: `Matches` is 1, `Contains` 2, `Not` 4, `Equals` 8, `Substring` 16 and `Or` 32. So `20` is `Substring` with `Not` ticked, and `34` is `Contains` with `Or` ticked. That is worth knowing for a review of a plan diff, where the checkbox states are otherwise invisible. ## The worked case The checkout body must carry an `orderId` and must not carry a rejection marker. That is two conditions with opposite polarity, so it is **two assertions**, not one: a `Substring` assertion with pattern `"orderId"`, and a second `Substring` assertion with `Not` ticked and the rejection markers listed. Ticking `Not` on a single assertion that also contains the `"orderId"` pattern would invert that pattern too, and the plan would start demanding that the order id be absent.

  • A colleague ticks both Not and Or to mean "none of these three markers appears". What did they actually configure?
    "At least one of the three markers is absent", which passes on almost every response. `Not` inverts each pattern and `Or` then accepts a single inverted pattern passing. "None of these appears" is `Not` ticked with `Or` clear, so that every inverted pattern has to pass.
  • Why does a Response Assertion with two broken patterns report only one failure message in AND mode?
    Because the element stops at the first pattern that fails. It records that pattern's failure text and returns without evaluating the rest, so the second broken pattern surfaces only after the first is fixed. In OR mode the message instead concatenates the text for every pattern tried.

saying these in an interview costs you the question

  • Saying Not inverts the assertion's final verdict
  • Saying Or lets each pattern test a different field
  • Ticking both boxes to mean none of these
  • Expecting every pattern to be evaluated on failure
  • Assuming patterns can each use a different rule