skip to content

A C# log parser calls IndexOfAny with a char[] of six separators on every line; why does .NET 8's SearchValues<char> help, and how do you use it?

level: seniorimportance: should knowfreq 28%

answer

  1. work done per call
  2. analyse the set once
  3. a static readonly field
  4. more than five values
  5. CA1870 suggests it

basics

~20 s

Beyond five values, IndexOfAny prepares its search from the value list on each call. SearchValues.Create (.NET 8) does that analysis once: cache the instance in a static readonly field and pass it to IndexOfAny, ContainsAny or SplitAny.

solid answer

~40 s

`MemoryExtensions.IndexOfAny(ReadOnlySpan<char>, ReadOnlySpan<char>)` has special paths for one to five values; beyond that, unless the input is short enough for a simple nested loop, each call tries to build a vectorized ASCII lookup from the value list and falls back to a probabilistic map, all before scanning the line. Done per line, that set-up is repeated work. `SearchValues.Create(" |;,\t=")` (.NET 8, `System.Buffers`) analyses the set once and returns an immutable `SearchValues<char>` backed by an implementation chosen for that set. Store it in a `static readonly` field and call `line.IndexOfAny(Separators)`; the same object works with `ContainsAny`, `IndexOfAnyExcept` and, on .NET 10, `SplitAny`. CA1870 flags the pattern for more than five constant values. Since .NET 9, `SearchValues<string>` does the same for substrings with Ordinal or OrdinalIgnoreCase.

code

csharp · 25 lines
csharp
using System.Buffers;

Console.WriteLine(LogScan.FirstSeparator("ts=10:15:03|GET;/api"));   // 2
Console.WriteLine(LogScan.FirstSeparator("nofieldshere"));          // -1
Console.WriteLine(LogScan.CountFields("a b|c;d,e\tf"));             // 6
Console.WriteLine(LogScan.HasSecret("GET /login?Password=x"));      // True

static class LogScan
{
    // Built once: the set is analysed at Create time, then reused on every line.
    private static readonly SearchValues<char> Separators = SearchValues.Create(" |;,\t=");
    private static readonly SearchValues<string> Secrets =
        SearchValues.Create(["password", "token"], StringComparison.OrdinalIgnoreCase);

    public static int FirstSeparator(ReadOnlySpan<char> line) => line.IndexOfAny(Separators);

    public static int CountFields(ReadOnlySpan<char> line)
    {
        int n = 0;
        foreach (Range r in line.SplitAny(Separators)) n++;
        return n;
    }

    public static bool HasSecret(ReadOnlySpan<char> line) => line.ContainsAny(Secrets);
}

go deeper

for a junior

Recall that SearchValues is a precomputed set of characters for IndexOfAny-style searches, created once and reused.

for a middle

Explain that IndexOfAny with more than five values redoes its set-up on each call, and that SearchValues.Create moves that work out of the loop.

for a senior

Spot the pattern with CA1870, cache the instance in a static readonly field, avoid creating it per call, and confirm the gain with a benchmark.

for a principal

Decide where precomputed search sets belong in shared parsing code, and accept CA1870's suggestion level or raise it for hot-path libraries.

## The per-call cost of IndexOfAny `MemoryExtensions.IndexOfAny<T>(this ReadOnlySpan<T> span, ReadOnlySpan<T> values)` finds the first element of `span` that equals any element of `values`. In `MemoryExtensions.cs` the method switches on `values.Length`: for **one to five** values it calls dedicated routines that compare several values at once. For **more than five** `char` values it goes to `ProbabilisticMap.IndexOfAny`, which on each call: 1. uses a simple nested loop when the input is short compared with the value list, 2. otherwise tries to build a vectorized ASCII lookup from the values, 3. if that cannot be used, falls back to a probabilistic map built from them, 4. only then scans the input. The work in steps 2 and 3 depends only on the values, not on the line. A log parser that searches for the same six separators in every line repeats that set-up a million times for a million lines. ## What SearchValues does `SearchValues<T>` (namespace `System.Buffers`, added in .NET 8) is described in its source as "an immutable, read-only set of values optimized for efficient searching", meant for "situations where the same set of values is frequently used for searching at runtime". The factory does the analysis up front: - `SearchValues.Create(ReadOnlySpan<char> values)` and `SearchValues.Create(ReadOnlySpan<byte> values)` (.NET 8), - `SearchValues.Create(ReadOnlySpan<string> values, StringComparison comparisonType)` (.NET 9), for substrings, accepting only `Ordinal` or `OrdinalIgnoreCase`. `Create` inspects the set and returns an internal subclass suited to it, for example a vectorized ASCII lookup for an all-ASCII set or a range check for contiguous values. After that, each search goes straight to the scan. ## How to use it 1. Create the instance **once**: `private static readonly SearchValues<char> Separators = SearchValues.Create(" |;,\t=");` 2. Pass it wherever a span method accepts `SearchValues<T>`: - `IndexOfAny`, `LastIndexOfAny`, `IndexOfAnyExcept`, `ContainsAny`, `ContainsAnyExcept`, - on .NET 10, `SplitAny(SearchValues<T>)`, which enumerates the fields between separators as `Range` values. 3. Share it freely: the instance is immutable, so many threads can search with it at once. ## The rule that points you there Code analysis rule **CA1870**, "Use a cached 'SearchValues' instance", flags `IndexOfAny` or `ContainsAny` calls with many constant values. Its page says it ignores calls with up to five values, "as those already use an optimal implementation", that it is enabled by default in .NET 10 as a suggestion, and that a code fix moves the values into a `static readonly SearchValues<T>` field. | Call | Set-up work | When it fits | |---|---|---| | `IndexOfAny('a', 'b')`, up to five values | None worth caching | Small fixed sets | | `IndexOfAny(charArray)`, more than five | Repeated on every call | One-off searches | | `IndexOfAny(searchValues)` | Once, in `Create` | The same set on a hot path | | `SearchValues.Create(...)` inside the loop | Every iteration, plus an allocation | Never; this is the main mistake | ## Mistakes to avoid - **Creating it per call.** `line.IndexOfAny(SearchValues.Create(" |;"))` does the analysis and allocates every time, which is worse than the `char[]` it replaced. - **Expecting culture-aware string search.** `SearchValues<string>` takes only `Ordinal` or `OrdinalIgnoreCase`; anything else throws `ArgumentException`. - **Claiming a speed-up without measuring.** The source shows what work moves out of the loop; how much it saves depends on the set and the input, so confirm with a benchmark before quoting numbers. Benchmarking itself belongs with the .NET performance tools. ## Where it sits among span searches SearchValues is one tool among several, and the choice follows the shape of the search: - **One value**: `IndexOf(',')` is already vectorized; nothing to cache. - **Two to five values**: `IndexOfAny(' ', '|', ';')` uses the dedicated paths; a cached set adds little. - **A larger fixed set, many searches**: `SearchValues<char>` or `SearchValues<byte>` in a `static readonly` field. - **Everything except a set**: `IndexOfAnyExcept(searchValues)` finds the first character outside an allowed set, such as the first character that is not a valid token character, without a hand-written loop. - **Several substrings**: `SearchValues<string>` (.NET 9) instead of one `IndexOf` per marker. The same instance serves every one of those methods, so a parser usually declares a handful of named sets (separators, token characters, secret markers) next to each other and reuses them throughout. ## Interview angle The answer that lands names the per-call set-up in `IndexOfAny`, moves it into `SearchValues.Create`, caches the result in a `static readonly` field and mentions CA1870 as the rule that finds the pattern. Adding the .NET 9 string form and the caveat about measuring shows judgment rather than a memorised tip.

  • Your separator set has only three characters; should you still switch to SearchValues?
    Usually not for speed alone. `IndexOfAny` has dedicated paths for one to five values, and CA1870 deliberately ignores such calls because they already use an optimal implementation. A cached `SearchValues<char>` can still read well when the same set is used in several places, but it is not the fix it is for larger sets.
  • How would you scan log lines for any of several secret markers like password or token, ignoring case?
    Use .NET 9's `SearchValues.Create(["password", "token"], StringComparison.OrdinalIgnoreCase)` once in a static readonly field, then `line.ContainsAny(Secrets)` or `IndexOfAny(Secrets)` per line. It searches for all substrings in one pass instead of one `IndexOf` per marker, and it accepts only Ordinal or OrdinalIgnoreCase comparisons.

  • Calls SearchValues.Create inside the per-line method.
  • Believes IndexOfAny with a large char[] does no set-up work per call.
  • Expects SearchValues<string> to accept CurrentCultureIgnoreCase.
  • Quotes a speed-up from one Stopwatch run as proof.
  • Thinks a SearchValues instance must be locked before sharing across threads.

Assumes .NET 10; SearchValues<char> and SearchValues<byte> arrived in .NET 8, SearchValues<string> in .NET 9, and the snippet's SplitAny(SearchValues<T>) is used on .NET 10.