skip to content

Why does a JMeter distributed setup that ran last week suddenly fail its RMI connection?

level: seniorimportance: nice to knowfreq 20%

answer

  1. Nothing on the hosts changed at all
  2. Every generator failed on the same day
  3. The convenience script chose a short life
  4. Look at the keytool line JMeter ships
  5. Seven is the number in that line

basics

~10 s

The stock keystore expires. JMeter's bin/create-rmi-keystore.sh and .bat call keytool with -validity 7, so the rmi_keystore.jks they produce is good for seven days. Nothing on either node changed; the key pair simply aged out.

solid answer

~40 s

Both `bin/create-rmi-keystore.sh` and `bin/create-rmi-keystore.bat` run one `keytool -genkey` line that includes `-validity 7`, and the JMeter manual states the same thing: the generated key pair is valid for seven days. Because `server.rmi.ssl.disable` defaults to `false`, that short-lived keystore is on the critical path of every distributed run, so a setup proven on Monday stops connecting the following week with a security-layer failure rather than a network one. Both scripts pass their extra arguments straight through to `keytool`, but the durable fix for a fleet is to generate a longer-lived keystore deliberately and point `server.rmi.ssl.keystore.file` at it on every node, then treat regeneration and redistribution as a scheduled task rather than an incident.

go deeper

for a junior

Know that the keystore JMeter's helper script produces is short-lived, so a distributed setup can stop working without anyone changing anything.

for a middle

Point at the -validity 7 in bin/create-rmi-keystore.sh and explain that the file it produces is what server.rmi.ssl.keystore.file names by default.

for a senior

Separate this from a network fault by its shape: it fails everywhere at once, on the same day, regardless of where the controller runs, and always before a test starts.

for a principal

Treat the RMI keystore as a managed credential with an owner, a lifetime and a rotation job, not as a file someone once copied around during a first setup.

This is the JMeter distributed-testing failure that arrives with no change to blame. Nobody edited the plan, nobody touched the firewall, and the same command that worked before now fails before the first request. ## What actually expired The convenience script that the manual points you at is a single `keytool` invocation, identical in the shell and batch versions: ``` keytool -genkey -keyalg RSA -alias rmi -keystore rmi_keystore.jks \ -storepass changeit -validity 7 -keysize 2048 "$@" ``` `-validity 7`. The manual is explicit about it too: the script "will generate a key-pair, that is valid for seven days". Since `server.rmi.ssl.disable` defaults to `false`, that key pair is load-bearing for every remote run, so the useful life of a keystore created by following the quick-start instructions is one week. ## Why it presents as a mystery Three things make this hard to spot the first time: - **Nothing changed.** The property files, the ports and the plan are all identical to the run that worked. - **It fails everywhere at once.** Every generator stops connecting on the same day, which looks far more like a network or DNS event than a per-host problem. - **It fails early.** The failure lands while the transport is being established, so there is no test, no samples and no results file to inspect — only a connection that will not come up. The cheap discriminator is location: a filtered port behaves differently depending on which machine you run the controller from, while an expired keystore fails identically from a generator sitting on the same switch. ## Fixing it once 1. Generate a keystore with a validity that matches how long the environment is meant to live, rather than re-running the seven-day script. 2. Put it where the properties expect it, or set `server.rmi.ssl.keystore.file` to an absolute path so it does not depend on which directory the process was started from. 3. Keep the alias and password aligned with `server.rmi.ssl.keystore.alias` and `server.rmi.ssl.keystore.password`, or set those properties to match what you chose. 4. Redistribute to **every** node, controller included, and record the expiry date somewhere a human will see before the next campaign. ## Where the boundary is How validity periods are represented and checked, and what a trust chain does with an expired certificate, is TLS material and is covered by the certificate topics rather than here. What belongs to JMeter is narrow and worth memorising: the script it ships, the `-validity 7` in it, the `rmi_keystore.jks` name that the `server.rmi.ssl.*` defaults expect, and the fact that this file is a scheduled maintenance item on any long-lived generator fleet.

  • How would you tell an expired JMeter RMI keystore apart from a newly blocked port?
    Change where you run the controller from. A packet filter is positional: two hosts on the same segment usually still connect. An expired keystore is not positional, so a controller sitting beside the generator fails exactly the same way as one across the network.
  • What else about the shipped create-rmi-keystore script is worth knowing before you rely on it?
    It hardcodes `-alias rmi` and `-storepass changeit`, which is why an untouched `jmeter.properties` accepts its output — the defaults for `server.rmi.ssl.keystore.alias` and `server.rmi.ssl.keystore.password` are the same two values. Both variants also forward their extra arguments to `keytool`.

saying these in an interview costs you the question

  • Assumes a working keystore never needs revisiting
  • Blames the network because every host failed together
  • Sets server.rmi.ssl.disable=true to unblock the run
  • Regenerates on one node and not the rest
  • Never records when the keystore expires