Your graph network must use evidence six hops away, but accuracy drops past three layers — how do you design for it?

### Read the situation first Two facts are in play. The label needs evidence six hops away, and a plain stack past three layers loses accuracy. The naive reading is "we need six layers, so fix depth". The correct reading is that you have **two** problems and they need different treatments. The accuracy drop at depth is over-smoothing: repeated neighbourhood averaging is a low-pass filter, so node representations converge toward each other and the head can no longer separate classes. The six-hop requirement raises the second problem: even if depth were free, the number of nodes within six hops grows roughly exponentially, and all of it has to be compressed into fixed-width vectors as it travels — over-squashing. Fixing only the first leaves you with a six-layer model that trains stably and still cannot use the distant evidence. ### Step 1 — establish what the ceiling really is Before designing anything, measure. Train the plain stack at 2, 4, 8 and 12 layers, and at each depth log both validation accuracy and mean pairwise cosine similarity between node embeddings. Separately, stratify the evaluation set by the graph distance from each target node to the decisive evidence. Those two curves tell you the split between the failures: similarity climbing toward 1 is smoothing; error concentrated on far-away-evidence cases with near cases fine is squashing. Also check whether six hops is genuinely required — often a cheap feature at the target node correlates with the distant cause well enough that no long-range propagation is needed at all, and that is the outcome worth finding early. ### Step 2 — make depth survivable These are the changes that let a deeper stack keep working, none of which enlarge what the model can reach: - **Jumping-knowledge readout.** Instead of classifying from the last layer only, keep the output of every layer 1..k for each node and combine them at the readout by concatenation or element-wise max. Each node then effectively selects its own neighbourhood radius: a hub sitting in a dense region, whose representation saturates after one round, can lean on its layer-1 view, while a node out on the periphery uses its layer-4 view. That per-node adaptivity is the point — a fixed depth forces one radius on a graph whose nodes have wildly different local structure. - **Initial residual connections.** At every layer, mix a fixed fraction of the layer-0 representation back in, so the node's own input features never wash out no matter how deep the stack goes. Paired with an identity-biased weight matrix, this is what makes very deep graph convolution stacks trainable at all. - **Decoupling propagation from parameters.** Compute a prediction from node features with a small feed-forward model, then propagate that prediction over the graph with a personalized-PageRank-style operator that teleports back to the original prediction with some probability at each step. You get many rounds of propagation with only a couple of learned layers, and the teleport term is precisely what stops the fixed point from being the fully smoothed one. ### Step 3 — make the distance shorter Depth-survivability alone does not defeat the bottleneck, so change the graph the model sees: - **Add a virtual global node** connected to all nodes. Any pair is then two hops apart. This is cheap and effective when the long-range signal is diffuse, but it is a firehose: it mixes everything with everything and accelerates smoothing, so it pairs badly with a deep stack and well with a shallow one. - **Targeted rewiring.** Add a small number of edges where the bottleneck diagnosis says the graph is pinched, rather than densifying everywhere. Fewer added edges means less extra smoothing. - **Coarsen and go hierarchical.** Cluster the graph, message-pass within clusters, message-pass over the cluster graph, then broadcast back down. Six hops in the original graph may be one or two in the coarse graph. - **Reframe as a subgraph task.** Extract the k-hop subgraph around each target and run a model over that, so the long-range structure becomes local structure inside a small input. - **Precompute multi-hop features.** Powers of the propagation operator applied to input features can be computed once, offline, and fed as extra inputs to a shallow model. ### Step 4 — pick, and justify the cost The judgment an interviewer is testing is that you *do not* pick all of these. Order by cost: start with the jumping-knowledge readout and initial residuals, since they change the model and not the data pipeline. If distance-stratified error is still bad on far cases, add targeted rewiring or a virtual node, and re-check the similarity curve because you have just made smoothing worse. Reach for hierarchical coarsening or subgraph extraction only when the graph is large enough that inference cost forces it anyway. State the risk too. Rewiring changes the data, so anything measured on the rewired graph is not comparable to the original baseline, and an added global node is a real inference-cost change on large graphs. And keep the honest fallback on the table: if the long-range signal turns out to be weak, a two-layer model with good hand-built features is a better system than a fragile deep one.