You apply your first Istio AuthorizationPolicy with action ALLOW to one workload, and calls that previously worked start returning 403. Explain how Istio evaluates AuthorizationPolicy, including how the CUSTOM, DENY and ALLOW actions relate.

## The rule that surprises everyone Istio's authorization model has no global switch between "allow by default" and "deny by default". The default is **per workload and implicit**: a workload that no `ALLOW` policy selects accepts everything; a workload that at least one `ALLOW` policy selects accepts only what those policies' rules match. Writing your first policy therefore does two things at once — it grants the access it describes, and it revokes everything it does not describe. This is what produces the classic incident: a team writes a policy allowing the frontend to call the API, ships it, and discovers that the batch job, the admin tool and the health scraper are now getting 403s. Nothing about those callers changed; the workload's default did. ## Anatomy of a policy ```yaml apiVersion: security.istio.io/v1 kind: AuthorizationPolicy metadata: name: api-callers namespace: payments spec: selector: matchLabels: app: api action: ALLOW rules: - from: - source: principals: ["cluster.local/ns/web/sa/frontend"] to: - operation: methods: ["GET", "POST"] paths: ["/v1/*"] when: - key: request.headers[x-tenant] values: ["acme"] ``` The boolean structure matters and is easy to get backwards: - Separate entries in `rules` are **ORed** — any rule matching is enough. - Within one rule, `from`, `to` and `when` are **ANDed** — all present blocks must match. - Within a list (several `principals`, several `methods`), entries are **ORed**. So one rule with two sources and three methods means "either of those callers, using any of those methods"; two separate rules mean two independent grants. Two degenerate forms are worth memorising because they are the idiomatic deny-all and allow-all: ```yaml spec: {} # action defaults to ALLOW, rules absent -> matches nothing -> deny all spec: rules: - {} # one rule matching everything -> allow all ``` ## The three actions, in order **CUSTOM** is evaluated first. It delegates the decision to an external authorization service declared as an extension provider in mesh configuration — the escape hatch for a policy engine Istio's own rule language cannot express. If the external service denies, the request stops there. **DENY** is evaluated next, and it is absolute. If any DENY policy matches, the request is rejected regardless of what any ALLOW policy says. There is no priority field and no way for a narrower ALLOW to carve an exception out of a broader DENY. This makes DENY a blunt instrument: correct for "nobody may ever reach `/internal/*` from outside", dangerous as a general-purpose tool, because a mesh-wide DENY written by a platform team cannot be overridden by any team beneath it. **ALLOW** is evaluated last, and only for requests that survived the previous stages. If no ALLOW policy selects the workload, the request is allowed; otherwise it must match one. There is also **AUDIT**, which records a match without affecting the decision — useful for observing what a proposed rule *would* have caught. ## Scoping A policy applies to the workloads its `selector` matches within its own namespace. With no selector it covers the whole namespace. Placed in the mesh's root namespace (`istio-system` by default) with no selector, it covers the entire mesh. Unlike peer authentication, authorization policies **accumulate rather than override**: every policy that selects the workload participates in the evaluation above. A namespace-wide deny-all and a workload-specific allow therefore coexist, and the workload ends up reachable only for what the allow describes. ## Identifying the caller `source.principals` matches the peer identity from the mesh certificate, written without the URI scheme — `cluster.local/ns/web/sa/frontend`. `source.namespaces` matches the namespace part of that same identity. `source.requestPrincipals` is a different axis entirely: it matches the end-user identity extracted from a validated token, not the calling workload. `source.ipBlocks` matches the remote address and is the weakest of the three, because an IP is not an authenticated claim. A critical dependency follows: **principal-based rules only mean something when plaintext cannot arrive**. If the workload is in permissive peer-authentication mode, a plaintext caller has no principal at all, so a principals rule cannot match it — but any rule you wrote using only paths and methods will happily admit that unauthenticated caller. ## Debugging the 403 The denial is produced by the sidecar, not the application, so application logs show nothing. The response body is the literal string `RBAC: access denied`, which is the fastest way to confirm what rejected the request. From there, list every AuthorizationPolicy that selects the workload — including namespace-wide and root-namespace ones — and walk the CUSTOM/DENY/ALLOW order by hand against the request's principal, path and method.