What file formats can conftest test, and how does a CI job learn it failed?

## The idea Most policy tools are welded to one input. A Kubernetes admission controller only ever sees Kubernetes objects; a Terraform-specific scanner only understands Terraform. conftest takes the opposite approach: it is one binary that knows how to **parse many configuration formats into a single generic document model**, and then evaluates the same kind of policy over whatever came out. That is why a platform team can run one engine across dozens of repositories that hold a mix of manifests, pipeline definitions, Dockerfiles and infrastructure code. ## Parsing conftest ships parsers for, among others, YAML (including multi-document files), JSON, HCL and HCL2, Dockerfile, TOML, INI, CUE, Jsonnet, XML and properties files. The parser is selected from the file extension. Two consequences follow immediately: - A file with an unusual extension will not be parsed the way you expect. `--parser yaml` (or `dockerfile`, `hcl2`, and so on) forces the choice, which is also how you evaluate content piped in on stdin. - Parsing is **structural, not textual**. Policies never see the original text, indentation or comments; they see the data structure the parser produced. A rule that tries to reason about the raw file contents is written against the wrong thing. A multi-document YAML file is handled as multiple documents rather than one, so a file holding a Deployment and a Service is not a two-element object you have to index into. ## Where policies live Policies are Rego files in a policy directory, `./policy` by default, overridable with `-p/--policy` (repeatable). Each file declares a package, and conftest calls that package the rule's **namespace**. By default it evaluates only the `main` namespace; `-n/--namespace` selects others and `--all-namespaces` evaluates everything it loaded. That selection is how one shared rule library can hold Kubernetes rules, Dockerfile rules and pipeline rules side by side while a given repository only runs the subset that applies to it. ## The run loop Without extra flags, conftest evaluates **each input file independently**: file in, document out, policy evaluated, results collected, next file. So results are naturally per-file and a failure message can be attributed to the file that caused it. (The `--combine` flag changes this to a single evaluation over all files at once, which is a different mode with different rule semantics.) ## Results and the exit code The run prints each finding and then a summary line counting tests, passes, warnings and failures. What matters for automation is the process exit status: - **Failures present -> non-zero exit.** The CI step fails and, if that step is a required check, the change is blocked. - **Only warnings -> exit zero**, unless you pass `--fail-on-warn`. Warnings exist precisely so a rule can be visible before it is enforced. - **An error** - a policy that will not compile, a file conftest cannot parse, a policy directory that contains no policies at all - is also a non-zero exit, and it is a different situation from a policy failure even though CI sees the same red. `--output` controls the human/machine formatting (table, JSON, TAP, JUnit and similar) for dashboards and test reports. It does not change the verdict; the pipeline's decision still rests on the exit code, so never wrap the invocation in something that swallows it. ## What a green run does and does not mean This is where beginners get burned. Exit zero means *no rule in the selected namespaces produced a failure for the files you passed*. It does not mean the file is safe, and it does not even mean any rule ran. If the glob missed the file, if the extension has no parser attached, if the rules live in a namespace you did not select, or if the policy package was renamed, the run is green and empty. Treating a green conftest step as evidence that a control was enforced is exactly the claim an auditor will ask you to substantiate, and "the job passed" is not enough on its own. ## Where it sits Because it is a static binary reading files off disk, conftest runs anywhere the files are: a pre-commit hook, a pull-request check, a build step before an image is pushed, or a pre-deploy step over rendered manifests. Running the same binary and the same rules in more than one of those places is normal, and it means a developer sees the same message locally that the pipeline will produce.