In Ragas, how do you control the single-hop vs multi-hop query mix?

## What a query synthesizer is After the knowledge graph is built and enriched, ragas needs to decide what kind of question to write from it. A query synthesizer is the component that does that: it selects node(s) from the graph, applies a prompt strategy, and emits a sample containing the question, the contexts it drew on, and a reference answer. Different synthesizer classes select differently and prompt differently, and that is the whole source of difficulty variation in a generated testset. ## The three synthesizers **`SingleHopSpecificQuerySynthesizer`** picks a single node and writes a question whose answer sits in that one node — typically anchored on a concrete extracted item such as a keyphrase or an entity. These are the easy questions. A retriever only has to find one chunk. **`MultiHopSpecificQuerySynthesizer`** picks two nodes connected by an entity-overlap relationship and writes a question that requires both. Think "what does document A say about X that document B contradicts" — concrete, but the answer is not in either chunk alone. **`MultiHopAbstractQuerySynthesizer`** also uses two connected nodes, but works at the summary/theme level rather than the entity level, producing broader, more conceptual questions that generalise across the pair. ## The distribution `query_distribution` is a list of `(synthesizer_instance, weight)` tuples. The weights are proportions and must sum to 1. Each synthesizer is instantiated with the LLM it should use, so you can in principle even give different synthesizers different models. `default_query_distribution(llm)` builds the standard three-way mix; calling it and then editing the weights is usually easier than assembling the list from scratch, because it also gets the constructor arguments right for you. You pass it either to `generate_with_langchain_docs(docs, testset_size=N, query_distribution=dist)` or to `generate(testset_size=N, query_distribution=dist)`. ## Why this is the tunable that matters A testset made entirely of single-hop specific questions is a weak instrument. Almost any embedding-based retriever finds the one chunk that contains the answer, so every candidate pipeline scores near the top and the test tells you nothing about which is better. Difficulty is what creates discrimination. Shifting weight toward the multi-hop synthesizers gives you questions where retrieval has to assemble evidence, which is exactly the failure mode real users hit and naive retrieval misses. The opposite error is also real. A testset that is overwhelmingly multi-hop abstract can be *unfairly* hard: it may demand a synthesis that your product was never designed to do, so scores are uniformly low and again fail to discriminate. A mix — enough easy questions that a regression in the basics is visible, enough hard ones that improvements register — is the practical target. ## The silent failure Weights are a request, not a guarantee. Multi-hop synthesizers need pairs of *connected* nodes; if the transform pipeline that ran over your graph did not include relationship builders, or its similarity threshold was strict enough that few edges were drawn, the multi-hop synthesizers will find nothing eligible and contribute few or no samples. You asked for 50% hard questions and got 5%, with no error raised. The diagnostic is cheap and you should make it routine: after generation, group the `to_pandas()` output by `synthesizer_name` and compare the realised proportions against the weights you passed. If they do not match, inspect the graph's relationship count before touching the weights again — reweighting a distribution whose synthesizers have no input does nothing. ## Cost implications Multi-hop synthesis is not free relative to single-hop: it draws more context into the prompt and asks the model to do more. Increasing the multi-hop share increases the per-sample generation cost. This is usually the smaller half of the bill compared with graph transforms over a large corpus, but it scales with `testset_size` rather than corpus size, so for a small corpus and a large testset it can dominate. ## Version note These three class names belong to the knowledge-graph generation line. Very early ragas releases had a different set of synthesizer names and a ratio-based API; code written against those does not port, and the mix concept is the only thing that carries over.