skip to content

A JMeter remote run starts behind a corporate firewall but no samples arrive. Which RMI leg is blocked?

level: seniorimportance: should knowfreq 44%

answer

  1. Where it stops names the leg
  2. One connection is opened by the generator
  3. Read the generator's log, not the controller's
  4. The missing rule points inbound
  5. Pin the base before requesting the rule

basics

~20 s

The return leg. The registry and engine ports are open, since the plan is running, but the engine cannot connect back to the controller's callback ports. Pin them with client.rmi.localport and allow them inbound on the controller.

solid answer

~40 s

Where a distributed JMeter run stops tells you which port rule is missing. If it dies at the registry lookup, the `server.rmi.port` leg (1099) is shut. If the controller prints `Using remote object: ...` and then stalls, the lookup worked but calls to the exported engine on `server.rmi.localport` cannot get through. If instead the generators start the plan and the controller's sample count stays at zero, both outbound legs are fine and the missing rule is the **inbound** one on the controller: the engine calls back into `RemoteThreadsListenerImpl` and `RemoteSampleListenerImpl`, exported on `client.rmi.localport` + 1 and + 2 and anonymous while that property is `0`. Set it on the controller, open that range inbound, and check `jmeter-server.log` for the connect failures that prove the direction.

code

properties · 6 lines
properties
# controller user.properties
client.rmi.localport=4100

# firewall request that goes with it:
#   inbound  controller:4100-4102  from each generator
#   outbound controller -> generator:1099 and generator:4000

go deeper

for a junior

Know that a remote JMeter run needs the generator to connect back to the controller. If the run starts and no samples appear, the return path is the first thing to suspect.

for a middle

Explain which object lives on which port, and why the controller printing its remote-object line already proves that the registry leg and the lookup succeeded.

for a senior

Diagnose from logs alone: map the stopping point to a leg, confirm it in the generator's log, then write the exact inbound rule and property change that closes it.

for a principal

Turn one painful first run into a standard: a documented port block, a pre-flight connectivity check the pipeline runs before the plan, and a firewall request template the network team already recognises.

"It worked on my laptop with two local engines" and "it hangs in the data centre" are the same test plan meeting a different packet filter. Because a JMeter distributed run uses three separate TCP legs in two directions, the *point* at which it stops identifies the leg for you, without any packet capture. ## Signature 1 - the registry leg is shut The controller fails almost immediately, while looking the engine up. `ClientJMeterEngine` calls `LocateRegistry.getRegistry(host, port, ...)` and then `lookup("JMeterEngine")`, and that lookup is the first packet of the whole run. Nothing is printed about a remote object because none was ever obtained. This is the leg on `server.rmi.port` (default `1099`), controller to generator. ## Signature 2 - the engine object's own port is shut The controller prints `Using remote object: ` with an RMI reference, then goes quiet. That line proves the registry answered, because it is printed straight after a successful lookup. What follows is the first call to the exported engine itself, which lives on `server.rmi.localport` — a random high port unless you pinned it. The generator's `jmeter-server.log` shows the engine bound to the registry and then nothing further; no test is ever configured on it. ## Signature 3 - the return leg is shut This is the one that looks like a hang rather than an error, and it is the classic first-run-behind-a-firewall symptom. The generators accept the plan and start running it — `jmeter-server.log` shows the engine creating its JMeter engine and starting the test — while the controller's summariser sits at zero samples and the results file stays empty apart from its header. The direction is the giveaway: results are **pushed by the generator into the controller**, not pulled. The controller exports `RemoteThreadsListenerImpl` and `RemoteSampleListenerImpl` and ships their stubs inside the test tree; the generator opens a fresh TCP connection back to those ports. With the default sender — `mode=StrippedBatch`, a `DataStrippingSampleSender` wrapping `BatchSampleSender` — the `RemoteException` that blocked connection raises is only logged, so what you grep the generator for is `ERROR ... BatchSampleSender: sampleOccurred` with a nested `java.net.ConnectException`; only the `Standard`, `Asynch` and `DiskStore` senders escalate such a cause into `JMeterError: Could not return sample`. Either way the generator's log fills with connect failures while the controller's console shows nothing at all. Read the *generator's* log, not the controller's, whenever the controller looks idle. ## Reading the three signatures | Stops at | Leg | Direction | Property to pin | |---|---|---|---| | Registry lookup | registry | controller to generator | `server.rmi.port` | | First engine call | engine object | controller to generator | `server.rmi.localport` | | Zero samples, plan running | result callbacks | generator to controller | `client.rmi.localport` | ## The fix for signature 3 1. Set `client.rmi.localport` on the **controller** to a base you can request from the network team, for example `4100`. 2. Remember the offset: `RemoteThreadsListenerImpl` binds base + 1 and `RemoteSampleListenerImpl` binds base + 2, so `4100` means 4101 and 4102 are the live ports. The manual advises allowing up to three ports starting at the base, which is the safe rule to write down. 3. Ask for an **inbound** rule on the controller host, source = each generator. This is the request people forget, because every other rule in the run points the other way. 4. Restart the controller; the property is read when the listener classes load. ## Rule out the look-alikes first - **Address, not port.** If the generator advertises an address the controller cannot route to, no port rule helps. `RmiUtils.getRmiHost()` uses `java.rmi.server.hostname` when set, and the `jmeter-server` script carries a commented `RMI_HOST_DEF` line for exactly that; the script's own comment names a connection-refused-to-127.0.0.1 failure as the symptom. - **SSL, not routing.** If `server.rmi.ssl.disable` is at its default and the keystore is missing or mismatched on one node, the link fails during the handshake and looks a lot like a filtered port. Check that every node has the same `rmi_keystore.jks` before blaming the network. - **How the filter refuses matters.** Whether a block surfaces as a quick error or a long stall is a property of the network rule, not of JMeter, so do not read the delay itself as JMeter behaviour. How many samples the engines buffer before delivery, and which delivery mode they use, is a separate concern owned by JMeter's result-collection settings; the point here is only that the delivery socket is opened by the generator, towards the controller.

  • The controller shows zero samples. Which log would you open first, and what are you looking for?
    The generator's `jmeter-server.log`. If it shows the engine configured and the test running, the outbound legs are fine and the problem is the return connection; the connect failures raised when it tries to deliver a sample appear there, not on the controller, because the controller is simply never called.
  • How would you tell a blocked RMI port apart from a missing rmi_keystore.jks?
    By where it fails and what it says. A missing or mismatched keystore fails in the SSL handshake with a security-layer error, and it fails identically from every host, including one on the same subnet. A filtered port stalls or refuses at the TCP level and usually works fine between two machines the filter does not sit between.
  • Why does pinning server.rmi.localport not fix a run that starts but returns nothing?
    That property controls the generator's own exported engine object, which is on the leg the controller calls out to. It has already succeeded if the plan is running. The unpinned ports in that state are the controller's callback ports, governed by `client.rmi.localport`.

The controller phones the generator to place the order, but the generator phones back on a different, unlisted number to deliver each result. The switchboard passes outgoing calls and blocks incoming ones from unknown numbers, so the order is placed, the work is done, and nothing is ever delivered.

saying these in an interview costs you the question

  • Assumes results are pulled by the controller
  • Only ever opens ports towards the generators
  • Blames the test plan when the plan is running fine
  • Never opens the generator's jmeter-server.log
  • Disables SSL hoping it fixes a routing problem
  • Believes one firewall rule covers all three legs