skip to content

On Microsoft.Testing.Platform, when do you configure an MSTest 4 run with a .runsettings file and when with testconfig.json?

level: seniorimportance: nice to knowfreq 18%

answer

  1. copied as AssemblyName.testconfig.json
  2. mstest section: parallelism, timeout, execution
  3. --settings for runsettings on MTP
  4. many RunConfiguration nodes ignored
  5. both MSTest sections: run refused

basics

~20 s

On Microsoft.Testing.Platform, testconfig.json is the native file: copied next to the executable, it holds platform options, an mstest section and, from MTP 2.3, command-line options. A .runsettings file still works through --settings for MSTest's own settings, but many VSTest-only nodes are ignored.

solid answer

~40 s

**`testconfig.json`** is MTP's configuration file. Put it beside the test project and the build copies it to the output as `<AssemblyName>.testconfig.json`, where the executable loads it; `--config-file` points at a shared one instead. It carries `platformOptions` (such as `resultDirectory`), an `mstest` section — `parallelism`, `timeout`, `execution`, `output` — and, since MTP 2.3, `commandLineOptions` for options like `report-trx`. Precedence is command line, then environment variables, then the file, then defaults. **`.runsettings`** remains VSTest's format; on MTP, MSTest still reads it through `--settings`, including its `MSTest` element and `TestRunParameters`, but `RunConfiguration` nodes such as `MaxCpuCount`, `ResultsDirectory` and `TargetPlatform` and all data collectors and loggers have no effect. Use `testconfig.json` for new MTP projects and keep `.runsettings` only while a project may still run on VSTest. MSTest refuses a run where both files carry MSTest settings.

code

json · 13 lines
json
{
  "platformOptions": {
    "resultDirectory": "./TestResults"
  },
  "mstest": {
    "parallelism": { "enabled": true, "scope": "class", "workers": 0 },
    "timeout": { "test": 30000 },
    "execution": { "mapInconclusiveToFailed": true }
  },
  "commandLineOptions": {
    "report-trx": true
  }
}

go deeper

for a junior

Recall that testconfig.json is the Microsoft.Testing.Platform configuration file and .runsettings is VSTest's, still readable by MSTest through --settings.

for a middle

Explain how testconfig.json is copied as AssemblyName.testconfig.json, its mstest and platformOptions sections, and the command line over environment over file precedence.

for a senior

Migrate configuration deliberately: move the MSTest element, replace unsupported RunConfiguration nodes, loggers and collectors with options, and never ship both files with MSTest sections.

for a principal

Set one configuration convention for the portfolio: a shared testconfig.json through --config-file, with runsettings kept only for projects that genuinely still run on VSTest.

## Two configuration files, two platforms A `.runsettings` file is an XML file defined by **VSTest**; for years it was where MSTest's adapter settings, run configuration, data collectors and loggers lived. **Microsoft.Testing.Platform (MTP)** has its own JSON file, `testconfig.json`. When MSTest projects move to MSTest.Sdk and MTP — this leaf's reserved scenario, done in a CI pipeline — the question is which file to keep. ## How `testconfig.json` is found 1. Create `testconfig.json` in the test project folder. 2. The build copies it to the output folder as **`<AssemblyName>.testconfig.json`**, beside the test executable, overwriting any earlier copy. 3. The test application loads it automatically at start-up. 4. Alternatively, `--config-file <path>` points at a file elsewhere — for example one shared by all test projects, passed through the `TestingPlatformCommandLineArguments` MSBuild property in `Directory.Build.props`. When a setting is given more than once, MTP takes the first match in this order: **command-line argument, environment variable, `testconfig.json`, built-in default**. ## What goes into it - **`platformOptions`** — platform behaviour, such as `resultDirectory` for where reports land. - **`mstest`** — MSTest's own settings, grouped by area: `parallelism` (`enabled`, `scope`, `workers`), `timeout` (test and fixture timeouts in milliseconds, plus cooperative cancellation), `execution` (for example `mapInconclusiveToFailed`, `disableAppDomain`), `output` (trace capture). - **`commandLineOptions`** (MTP 2.3+) — any option a referenced extension accepts, written without the leading dashes, for example `"report-trx": true`. The extension must still be referenced; configuration never installs one. - **`environmentVariables`** (MTP 2.3+) — variables set for the test process before it starts. ## What `.runsettings` still does on MTP MSTest on MTP accepts a runsettings file through **`--settings <file>`** and reads its `MSTest` element and `TestRunParameters`. The rest of the file is VSTest vocabulary, and much of it has no meaning when the test project is its own executable: | `.runsettings` content | On MTP | |---|---| | `MSTest` element (parallelism, timeouts, execution flags) | read by MSTest; `testconfig.json`'s `mstest` section is the native equivalent | | `TestRunParameters` | read; on the command line, `--test-parameter key=value` | | `RunConfiguration` `ResultsDirectory` | not supported — use `--results-directory` or `platformOptions.resultDirectory` | | `RunConfiguration` `MaxCpuCount` | not supported — process-level parallelism belongs to the build and test tooling | | `RunConfiguration` `TargetPlatform`, `TargetFrameworkVersion` | ignored — the project's runtime identifier and target framework decide | | `RunConfiguration` `TestAdaptersPaths` | not supported — frameworks are compiled in | | `DataCollectionRunSettings`, `LoggerRunSettings` | not used — replaced by extensions and their options | A runsettings file carried over unchanged therefore half-works: MSTest's parallelism and timeouts still apply, while the results path, coverage collector and TRX logger silently stop mattering. ## Do not keep both MSTest refuses a run where the runsettings file has an `MSTest` section and `testconfig.json` has an `mstest` section, failing with "Both '.runsettings' and '.testconfig.json' files have been detected. Please select only one of these test configuration files." That rule forces a decision instead of letting two files disagree about, say, parallelism. ## A worked move A solution's test projects carried one `CI.runsettings` with three things in it: MSTest parallelism, a `ResultsDirectory`, and the code coverage data collector. After the move to MSTest.Sdk and MTP: 1. the `Parallelize` element becomes `"mstest": { "parallelism": { "enabled": true, "scope": "class" } }` in `testconfig.json`; 2. `ResultsDirectory` becomes `platformOptions.resultDirectory`, or `--results-directory` in the pipeline; 3. the coverage collector becomes the `--coverage` option of the coverage extension, which the `Default` profile already references; 4. the runsettings file and the `--settings` argument are deleted, which also avoids the both-files error. ## Choosing - **New MTP projects:** use `testconfig.json` only. - **Projects moving from VSTest:** move the `MSTest` element into the `mstest` section, the results directory into `platformOptions`, loggers and collectors into extension options, and delete the runsettings file. - **Projects that must still run on VSTest** (`UseVSTest` set to `true`, or tooling that only understands VSTest): keep `.runsettings`, because VSTest never reads `testconfig.json`.

  • How do several MSTest projects on MTP share one testconfig.json?
    Keep one file at the repository root and pass `--config-file <path>` to every test application, typically by appending it to the `TestingPlatformCommandLineArguments` MSBuild property in `Directory.Build.props`. A file passed with `--config-file` takes precedence over a per-project `<AssemblyName>.testconfig.json`.
  • Where do runsettings TestRunParameters go after moving to MTP?
    MSTest still reads them from a runsettings file passed with `--settings`, and MTP also accepts them on the command line as `--test-parameter key=value`, repeated per parameter. Tests read them through `TestContext.Properties` either way.

  • MTP ignores .runsettings files entirely, so MSTest settings in them are lost.
  • RunConfiguration ResultsDirectory in runsettings still sets the MTP results path.
  • testconfig.json must be named testconfig.json in the output folder to load.
  • testconfig.json settings override the same option given on the command line.
  • Keeping both files is fine; MSTest merges their MSTest sections.
  • Listing report-trx in testconfig.json installs the TRX extension.

Assumes MSTest 4.4.1 on Microsoft.Testing.Platform 2.4.1; commandLineOptions and environmentVariables in testconfig.json need MTP 2.3 or later.