skip to content

In TLS 1.3, why might a server with a valid, correctly chained certificate still be unable to authenticate to a client?

level: seniorimportance: should knowfreq 44%

answer

  1. valid is not the same as usable
  2. the client says what it can verify
  3. two lists, one for each signature
  4. no certificate reaches the wire
  5. suite no longer names the authentication

basics

~20 s

Because authentication needs a signature scheme both sides accept. The client's signature_algorithms(13) list, and signature_algorithms_cert(50) where present, constrain which certificate the server may select; a server whose only key or chain falls outside those lists cannot sign at all.

solid answer

~40 s

Validity and chaining are not the only constraints on certificate selection. A TLS 1.3 client advertises `signature_algorithms(13)` — the schemes it will accept in `CertificateVerify` — and may separately advertise `signature_algorithms_cert(50)` for the signatures found *inside* the certificates it is offered. The server must pick a certificate whose key can produce one of the offered schemes, and whose chain is signed with schemes the client accepts. An endpoint holding only an elliptic-curve key, facing an old client that offers only RSA schemes, has nothing it can sign with; the handshake ends in `handshake_failure(40)` without a single certificate being sent. The certificate is faultless. The negotiation is what failed, and the fix is either a second certificate or a client that offers more.

code

pseudocode · 16 lines
pseudocode
# client offers
signature_algorithms(13)      = [ rsa_pkcs1_sha256(0x0401) ]
signature_algorithms_cert(50) = absent   # so the list above governs both
server_name(0).host_name      = "declarations.example"

# server holds one certificate: elliptic-curve key, EC-signed chain
function select_certificate(offered_schemes, host):
    candidates = certificates naming host
    for each candidate in candidates:
        if key type of candidate cannot produce any scheme in offered_schemes:
            drop candidate
        if any signature in candidate chain is outside offered_schemes:
            drop candidate
    if candidates is empty:
        return handshake_failure(40)    # no Certificate(11) is ever sent
    return best candidate

go deeper

for a junior

Take away one fact: the client tells the server which signature schemes it can verify, and a certificate the client cannot verify is no use however correct it looks.

for a middle

Explain the selection procedure in order — name, then key type against the offered schemes, then the chain's own signatures — and what each of the two extensions scopes.

for a senior

Recognise the failure by its shape: no certificate on the wire means selection failed, not validation. Test key-type and algorithm changes against the oldest client population you still serve.

for a principal

This is where a dual-certificate deployment earns its cost. Decide deliberately whether to carry a second key type for an old population or to set a floor and cut it off, and say what evidence would settle it.

## Authentication is a negotiated capability, not a property of the certificate A certificate is a static object: a name, a public key, a validity window and an issuing chain. Whether the server can *use* it on a given connection depends on what the client said it would accept. TLS 1.3 makes that explicit — a client performing certificate-based authentication is required to send `signature_algorithms(13)`, and the values in it are what the server has to work within. So a certificate can be simultaneously: - unexpired, - correctly chained with every intermediate present, - naming exactly the host the client asked for, and still unusable, because the client cannot verify a signature made with its key. ## The two extensions and what each one scopes | Extension | Constrains | If absent | |---|---|---| | `signature_algorithms(13)` | the signature the server makes in `CertificateVerify` — and, unless the other extension is present, the signatures inside the certificates too | required for certificate-based authentication in TLS 1.3 | | `signature_algorithms_cert(50)` | only the signatures **within** the offered certificates | `signature_algorithms(13)` governs both | The split exists because the two questions genuinely differ. A client may be able to verify a modern scheme in a live handshake while its certificate-path code accepts a wider or narrower set, or the reverse. Values from the registry look like `rsa_pkcs1_sha256(0x0401)`, `ecdsa_secp256r1_sha256(0x0403)`, `rsa_pss_rsae_sha256(0x0804)` and `ed25519(0x0807)`. ## How the server actually chooses 1. Read the `host_name` from `server_name(0)`, if any, and reduce the configured certificates to those that name that host. 2. Discard any whose key type cannot produce a scheme listed in `signature_algorithms(13)`. 3. If `signature_algorithms_cert(50)` is present, discard any whose chain contains a signature outside that list. 4. If `certificate_authorities(47)` was sent, prefer a chain that ends at one of the named authorities. 5. If nothing survives, the handshake cannot be authenticated and ends with `handshake_failure(40)` — before `Certificate` is ever sent. Step 5 is the shape of the failure that confuses people: there is no certificate on the wire to inspect, because the server never got far enough to choose one. ## Where this bites in production - **A key-type migration.** An endpoint moves to a single elliptic-curve certificate. Modern callers are unaffected; an integration partner running an old client that offers only RSA schemes stops connecting overnight, with nothing in the certificate to blame. - **A digest a strict client refuses.** The chain is signed with an algorithm a hardened client has removed from its list. The end-entity certificate is fine; an intermediate's signature is what falls outside `signature_algorithms_cert(50)`. - **Dual-certificate deployments.** Servers holding both an RSA and an elliptic-curve certificate select per connection precisely to cover both populations — which is a deliberate answer to this constraint, not an accident. - **A client that advertises generously but verifies narrowly.** Offering a scheme it cannot actually check produces a failure one message later, at signature verification, rather than at selection. ## Hints that narrow selection further `certificate_authorities(47)` lets a peer list the authorities whose chains it is prepared to accept, so a server holding several chains can send the one that terminates at an anchor the other side actually has. It is a **hint**: a server is not obliged to satisfy it, and a chain outside the list may still be accepted. A related hint exists that constrains selection by certificate contents, but it travels only in a request for a *client* certificate and so belongs to client authentication rather than here. ## What this is not - It is **not** the cipher suite. In TLS 1.3 a suite names only the authenticated-encryption algorithm and the hash; the authentication algorithm was removed from the suite string precisely so it could be negotiated separately here. - It is **not** about key strength or preference ordering as a security policy — the mechanism is capability matching, and a client offering only one scheme is expressing what it can verify, not what it considers strong. - It is **not** a path-building problem. Nothing has been validated yet; the server is choosing what to send. The operational lesson is to test certificate changes from the **oldest client population you actually serve**, because the certificate itself will look correct from every modern one.

  • What does signature_algorithms_cert(50) add that signature_algorithms(13) does not?
    It separates two questions that are usually conflated: which schemes the peer accepts for the live `CertificateVerify` signature, and which it accepts for the signatures baked into the certificates themselves. When `signature_algorithms_cert(50)` is present it governs the second question alone, and `signature_algorithms(13)` governs the first. When it is absent, `signature_algorithms(13)` governs both.
  • What does certificate_authorities(47) let a peer express, and how binding is it?
    It lists the authorities whose chains the sender is prepared to accept, so a server holding several chains can choose one that ends at an anchor the other side actually holds. It is a hint, not a constraint: the server is free to send a chain outside the list, and the other side may still accept it. Treating it as a hard filter overstates what the specification requires.
  • The handshake failed and no certificate appeared on the wire at all. What does that narrow the cause to?
    To something decided before selection completed: no certificate naming the requested host, or none whose key and chain fit the offered signature schemes. Anything wrong *with* a certificate — expiry, a missing intermediate, a name mismatch — can only be discovered after the `Certificate` message is sent, so its absence rules that whole class out.

saying these in an interview costs you the question

  • Assumes a valid unexpired certificate is always usable
  • Thinks the cipher suite in TLS 1.3 names the authentication algorithm
  • Treats certificate_authorities(47) as a hard requirement on the server
  • Believes the server picks its signature scheme unilaterally
  • Looks for the certificate on the wire when none was ever sent