skip to content

Derivatives & Gradients

Derivatives as instantaneous rates of change, partial derivatives, and the gradient vector that points uphill on a surface. Interviewers start here because every 'the model learns' story rests on it.

on this pageshow

explore

questions

11

Using the limit definition of the derivative, what is the derivative of f(x) = x^2?

level: juniorimportance: must knowfreq 72%

answer

  1. slope of a shrinking secant line
  2. expand (x + h)^2 first
  3. cancel h before letting h go to 0
  4. what is left over is 2x + h
  5. result agrees with n times x^(n-1)

basics

~10 s

The derivative of x^2 is 2x. The difference quotient ((x+h)^2 - x^2)/h expands to (2xh + h^2)/h, which simplifies to 2x + h, and letting h shrink to 0 leaves 2x.

solid answer

~40 s

The derivative is defined as `f'(x) = lim(h -> 0) (f(x+h) - f(x))/h` — the slope of the secant line through the points at `x` and `x+h`, in the limit as the second point slides into the first. For `f(x) = x^2`, the numerator is `(x+h)^2 - x^2 = x^2 + 2xh + h^2 - x^2 = 2xh + h^2`. Dividing by `h` is legal because the limit only considers `h` near 0 but never equal to 0, giving `2x + h`. As `h` goes to 0 that expression goes to `2x`, so `f'(x) = 2x`. This is the power rule `d/dx x^n = n*x^(n-1)` for `n = 2`, and the same argument run on `f(x) = 1/x` gives `-1/x^2`.

go deeper

for a junior

Be ready to write the difference quotient from memory and grind through the algebra for x^2 without hesitating. Knowing that the derivative is a slope, not a value, is the point being checked.

for a middle

Explain why cancelling h is not division by zero, and connect the worked result back to the power rule so the shortcut stops looking like magic.

for a senior

Show you know where the definition bites in practice: wherever that limit fails to exist the derivative does not exist either, however tidy the formula looks.

for a principal

Frame why the first-principles derivation is worth teaching at all. Teams that only memorise rules cannot reason about non-smooth objectives, and that gap surfaces later as silent modelling mistakes.

### What the derivative measures The derivative of a function `f` at a point `x` is the instantaneous rate of change of `f` there, equivalently the slope of the tangent line to the graph at that point. The word *instantaneous* is the whole difficulty: slope is normally defined between two distinct points, and at a single point there is only one point. The limit definition solves this by computing the slope between `x` and a nearby point `x + h`, then asking what that slope approaches as the nearby point slides in. ### The definition itself ``` f'(x) = lim(h -> 0) [ f(x + h) - f(x) ] / h ``` The fraction inside is the **difference quotient**. Geometrically it is the slope of the *secant* line cutting the graph at the two points `(x, f(x))` and `(x+h, f(x+h))`. The derivative is the number that secant slope converges to. If no single number is approached — for instance if approaching from the right and from the left give different answers — the derivative does not exist at that point. ### Working f(x) = x^2 from the definition Substitute and expand: 1. `f(x + h) = (x + h)^2 = x^2 + 2xh + h^2` 2. `f(x + h) - f(x) = x^2 + 2xh + h^2 - x^2 = 2xh + h^2` 3. Divide by `h`: `(2xh + h^2)/h = 2x + h` 4. Take the limit: as `h -> 0`, `2x + h -> 2x` So `f'(x) = 2x`. Sanity check at `x = 3`: the slope is 6, and indeed the function is rising steeply there; at `x = 0` the slope is 0, matching the flat bottom of the parabola; at `x = -3` the slope is -6, matching the descent on the left branch. ### Why cancelling the h is not division by zero A frequent objection is that step 3 divides by something that is heading to zero. The limit operation never evaluates the expression *at* `h = 0`; it asks only about values of `h` arbitrarily close to 0 and different from 0. For every such `h` the cancellation is ordinary arithmetic. Only after simplifying to `2x + h`, which is a perfectly well-behaved expression at `h = 0`, do we take the limit. Substituting `h = 0` *before* simplifying produces the meaningless `0/0`, which is why the algebra must come first. ### The same machine on f(x) = 1/x For `x` not equal to 0: 1. `f(x + h) - f(x) = 1/(x+h) - 1/x = [x - (x+h)] / [x(x+h)] = -h / [x(x+h)]` 2. Divide by `h`: `-1 / [x(x+h)]` 3. Let `h -> 0`: `-1 / x^2` So `d/dx (1/x) = -1/x^2`, which is the power rule again with `n = -1`, since `1/x = x^(-1)` and `-1 * x^(-2) = -1/x^2`. The derivative is negative everywhere, matching the fact that `1/x` is decreasing on each side of the origin. ### The rules this definition justifies Nobody re-derives every derivative from the limit in practice; the definition is used once to prove the shortcut rules, which are then applied mechanically: - **Constant**: `d/dx c = 0` - **Power rule**: `d/dx x^n = n*x^(n-1)`, valid for any real exponent `n` on the domain where it makes sense - **Sum and constant multiple**: `d/dx [a*f + b*g] = a*f' + b*g'` - **Product rule**: `(uv)' = u'v + uv'` — note it is *not* `u'v'` - **Quotient rule**: `(u/v)' = (u'v - uv')/v^2` — the minus sign and the squared denominator both matter - **Exponential and logarithm**: `d/dx e^x = e^x`, `d/dx ln x = 1/x` for `x > 0`, and `d/dx a^x = a^x * ln a` for a constant `a > 0` ### Why interviewers ask for the derivation Quoting `2x` takes two seconds and proves nothing. Producing the difference quotient, expanding it, cancelling correctly and only then taking the limit demonstrates that you know what a derivative *is* rather than which symbol to write down. It also sets up every later idea: a derivative is a limit, so wherever that limit fails to exist — a corner, a jump, a vertical tangent, wild oscillation — the derivative simply does not exist, no matter how smooth the formula looks.

  • Why is it legitimate to cancel the h in (2xh + h^2)/h before taking the limit?
    Because the limit only inspects values of `h` arbitrarily close to 0 but never equal to 0. For every such `h` the division is ordinary arithmetic, so the quotient genuinely equals `2x + h`. Only after that simplification, which is well defined at `h = 0`, do we take the limit. Substituting `h = 0` first would produce the meaningless form `0/0`.
  • What does the limit definition give for f(x) = 1/x?
    `-1/x^2` for any `x` other than 0. The numerator `1/(x+h) - 1/x` combines to `-h/[x(x+h)]`; dividing by `h` leaves `-1/[x(x+h)]`, and letting `h` go to 0 gives `-1/x^2`. It matches the power rule with exponent `-1`.
  • How does the limit definition explain why a constant function has derivative 0?
    For `f(x) = c` the numerator is `c - c = 0` for every `h`, so the difference quotient is `0/h = 0` for all non-zero `h`, and the limit of the constant 0 is 0. Geometrically the graph is a horizontal line: every secant is flat, so the tangent is flat.

A car's average speed over the last ten seconds is a secant slope. Shrinking that window toward zero seconds is exactly what the limit does, and what it converges to is the number on the speedometer: the instantaneous rate.

saying these in an interview costs you the question

  • Calls the derivative an average rate of change rather than an instantaneous one
  • Substitutes h = 0 into the quotient first, producing 0/0
  • Reports 2x + h as the derivative, forgetting to take the limit
  • Confuses the derivative 2x with the function value x^2
  • Quotes the power rule when explicitly asked to work from the definition

context

open as a page

For f(x, y) = x^2 * y, what are the two partial derivatives and the gradient at (2, 3)?

level: juniorimportance: must knowfreq 82%

basics

~20 s

Differentiate one variable at a time, holding the other fixed: df/dx = 2xy and df/dy = x^2. At (2, 3) those evaluate to 12 and 4, so the gradient there is the vector (12, 4).

open as a page

Why is the absolute value function f(x) = |x| not differentiable at x = 0?

level: middleimportance: must knowfreq 66%

basics

~10 s

At 0 the difference quotient |h|/h equals +1 for positive h and -1 for negative h, so the two one-sided limits disagree and no single tangent slope exists. The function is still continuous there.

open as a page

Why does the gradient of a differentiable function point in the direction of steepest ascent?

level: middleimportance: must knowfreq 74%

basics

~20 s

The rate of change in a unit direction equals the gradient's length times the cosine of the angle between them. Cosine peaks at 1 when the direction aligns with the gradient, so that bearing rises fastest.

open as a page

Does continuity of a function at a point imply that it is differentiable there?

level: middleimportance: should knowfreq 48%

basics

~10 s

No. The implication runs one way only: differentiability forces continuity, never the reverse. The function x times sin(1/x), with value 0 at the origin, is continuous at 0 yet its difference quotient oscillates forever.

open as a page

Why does the logistic function s(x) = 1/(1 + e^-x) have derivative s(x)(1 - s(x))?

level: middleimportance: should knowfreq 55%

basics

~20 s

The quotient rule turns 1/(1 + e^-x) into e^-x/(1 + e^-x)^2. That expression splits into 1/(1 + e^-x) times e^-x/(1 + e^-x), and the second factor is exactly 1 minus the first, giving s times (1 - s).

open as a page

For f(x, y) = x^2 + 3xy at (1, 2), what is the directional derivative along (1, 1)/sqrt(2)?

level: middleimportance: should knowfreq 55%

basics

~20 s

The gradient of x^2 + 3xy is (2x + 3y, 3x), which is (8, 3) at (1, 2). Combining it with the unit direction gives (8 + 3)/sqrt(2) = 11/sqrt(2), about 7.78 per unit distance.

open as a page

Why is a nonzero gradient always perpendicular to the level curves of a function f(x, y)?

level: middleimportance: should knowfreq 48%

basics

~10 s

Along a level curve the function is constant, so the rate of change in the tangent direction is zero. With a nonzero gradient, that combination vanishes only if the two are at right angles.

open as a page

What is the subgradient set of the convex function f(x) = |x| at the point x = 0?

level: seniorimportance: should knowfreq 36%

basics

~20 s

The subdifferential of |x| at 0 is the closed interval from -1 to 1: every slope of magnitude at most 1 gives a line supporting the V from below. Zero lies inside, which certifies 0 as the minimiser.

open as a page

How does logarithmic differentiation give the derivative of f(x) = x^x?

level: middleimportance: nice to knowfreq 24%

basics

~20 s

Take logs first: ln y = x ln x for y = x^x, which differentiates to y'/y = ln x + 1, giving y' = x^x times (ln x + 1). Base and exponent both vary, so neither standard rule applies.

open as a page

What are the gradients of w^T x and of x^T A x with respect to the vector x?

level: seniorimportance: nice to knowfreq 38%

basics

~20 s

The gradient of the linear form w^T x with respect to x is w. The gradient of the quadratic form x^T A x is (A + A^T) x, which becomes 2Ax when A is symmetric.

open as a page