How does node2vec turn a graph with no node features into one vector per node?
answer
- a graph has no sentences to read
- sample sequences, then reuse a text objective
- sliding window over each walk
- the model is one row per node id
basics
~20 snode2vec samples many random walks starting from every node, treats each walk as a sentence of node ids, and trains a skip-gram objective so that nodes co-occurring inside a sliding window over those walks get similar vectors.
solid answer
~50 sIt converts structure into sequences and then reuses a text objective. From each node you sample a fixed number of random walks of fixed length — commonly reported settings are ten walks per node of length 80 — and each walk is treated as a sentence whose words are node ids. A sliding window (often 10 positions) over each walk defines which node pairs count as context, and a skip-gram objective is trained so a node's vector predicts the nodes that appear near it in walks. The only learned parameters are the vectors themselves: the model is a lookup table with one row per node id and nothing else. DeepWalk does this with uniform random walks; node2vec adds a second-order bias, controlled by a return parameter `p` and an in-out parameter `q`, that decides whether walks hug the source or push outward. No node attributes are used — only who is connected to whom.
go deeper
Be ready to say, in one breath, that node2vec samples random walks, treats each walk as a sentence of node ids, and learns a vector per node from co-occurrence — no node attributes involved.
Explain the mechanics: how many walks and how long, what the sliding window defines, that skip-gram is trained by stochastic gradient descent, and that the learned model is a table with one row per node id.
Show you know what the parameterisation costs you in production: memory that scales with the node count, tables that are not comparable between runs, and a refit cadence you have to schedule and budget for.
Own the call of when a shallow embedding is the right investment at all — a static graph with weak attributes and no labels — versus when the money should go into feature engineering or a model that computes embeddings instead of storing them.
## The problem Many graphs arrive with nothing but structure: a set of nodes, a set of edges, and no useful attributes per node. A hyperlink graph of web pages, a follow graph of accounts, a co-purchase graph of products. You want a dense vector per node that you can hand to any downstream model — a classifier, a clustering routine, a nearest-neighbour lookup. Shallow graph embeddings are the oldest practical answer, and node2vec (with its predecessor DeepWalk) is the canonical method. ## Step 1 — sample walks A random walk starts at a node and repeatedly steps to a neighbour. Run `r` walks from every node, each of length `l`. Commonly reported settings are `r = 10` walks per node and `l = 80` steps. If the graph has one million nodes, that gives ten million sequences of 80 ids each — a corpus. DeepWalk samples each step uniformly among the current node's neighbours. node2vec biases the step using where the walk just came from, which is why it is called a second-order walk. Standing at node `v` having arrived from `t`, the unnormalised weight of stepping to a candidate `x` is `1/p` if `x` is `t` itself (walking back), `1` if `x` is also a neighbour of `t`, and `1/q` if `x` is two hops from `t`. The return parameter `p` therefore controls backtracking, and the in-out parameter `q` controls whether the walk stays in the source's neighbourhood or pushes away from it. ## Step 2 — treat walks as sentences The key move is an analogy, not a theorem: a walk is a sentence, a node id is a word. Text embedding methods learn word vectors from co-occurrence in a sliding context window; here the same machinery learns node vectors from co-occurrence in a sliding window over walks. With a window size of 10, two nodes are a training pair when they sit within ten positions of each other in the same walk. Note what this is *not*: it is not the same as being within ten hops in the graph — the walk may loop back on itself, and it may cross a bridge and never return. ## Step 3 — train skip-gram The skip-gram objective maximises the probability of the observed context nodes given the centre node, using its vector. Optimisation is ordinary stochastic gradient descent over the sampled pairs. Nodes that keep showing up in each other's windows end up with high inner product; nodes that never co-occur drift apart. ## What is actually learned This is the point most candidates miss. The model *is* the embedding table. Its parameter count is roughly the number of nodes times the embedding dimension (128 is a common choice), and there is no function that maps a node's attributes or edges to a vector — only an index lookup. Three consequences follow immediately: - **Attributes are ignored.** A user's signup date, a page's text, an item's price contribute nothing. If you have such features, you concatenate them to the embedding afterwards. - **Two training runs are not comparable.** Random initialisation and random walk sampling leave each run in its own arbitrarily oriented space. Vectors from run A cannot be compared with vectors from run B, and a classifier trained on one run's table is invalid on another's. - **Unseen nodes have no vector at all**, because they have no row. This is the transductive ceiling of the whole family. ## The knobs that matter - **Walks per node `r`** — how many samples each node contributes; raises training cost roughly linearly. - **Walk length `l`** — how far a single walk can drift and how many windows it yields. - **Window size `k`** — the actual definition of context; only pairs within `k` positions ever become training examples, so a long walk with a small window still only teaches local co-occurrence, just from more starting points along the walk. - **Dimension `d`** — capacity of the table. - **`p` and `q`** — the sampler's bias, which decides *what kind* of similarity the vectors end up encoding. ## When it is the right tool Shallow embeddings are cheap, need no labels, and often make a strong unsupervised feature set for a static graph whose node set barely changes between refits: community detection, similar-item recommendation, a feature block for a downstream classifier. They stop being the right tool the moment nodes arrive continuously or the useful signal lives in node attributes rather than in topology.
- With walk length 80 and window size 10, which node pairs actually become training pairs?Only pairs sitting within ten positions of each other in the same walk. The length-80 walk matters because it produces many overlapping windows and lets the walk drift far from its start, but two nodes 40 steps apart in the walk never form a pair. It is also not the same as being within ten hops in the graph — walks revisit nodes, so a close pair may co-occur many times and a two-hop pair may never co-occur.
- Roughly how many parameters does the model have, and why does that number surprise people?About the number of nodes times the embedding dimension — ten million nodes at dimension 128 is well over a billion numbers. It surprises people because there are no layers and no weights that generalise: every parameter belongs to exactly one node. Capacity scales with the graph, not with the complexity of the task, and memory becomes the binding constraint long before compute does.
- If your nodes have rich attributes already, is node2vec still worth running?Sometimes, as a complementary block. It sees only topology, so it adds signal exactly where attributes are weak — a page whose text looks innocuous but whose link neighbourhood does not. Concatenate the embedding with the attribute columns and let a downstream model weigh them. If the attributes already dominate and the graph is sparse or nearly random, the embedding is expensive noise.
Reading a graph the way you would read a book: you cannot read a map left to right, so you send a wanderer through it, write down the route, and treat the route as a sentence.
saying these in an interview costs you the question
- Says node2vec uses node attributes when it only uses topology
- Thinks the walks are shortest paths between node pairs
- Confuses window size with walk length
- Claims vectors from two separate runs are directly comparable
- Says the model generalises to any node once trained