skip to content

Derivative Rules

The limit definition of a derivative, the product, quotient and power rules, and what a non-differentiable kink like |x| means. Interviewers check you can differentiate a simple loss by hand.

on this pageshow

questions

6

Using the limit definition of the derivative, what is the derivative of f(x) = x^2?

level: juniorimportance: must knowfreq 72%

answer

  1. slope of a shrinking secant line
  2. expand (x + h)^2 first
  3. cancel h before letting h go to 0
  4. what is left over is 2x + h
  5. result agrees with n times x^(n-1)

basics

~10 s

The derivative of x^2 is 2x. The difference quotient ((x+h)^2 - x^2)/h expands to (2xh + h^2)/h, which simplifies to 2x + h, and letting h shrink to 0 leaves 2x.

solid answer

~40 s

The derivative is defined as `f'(x) = lim(h -> 0) (f(x+h) - f(x))/h` — the slope of the secant line through the points at `x` and `x+h`, in the limit as the second point slides into the first. For `f(x) = x^2`, the numerator is `(x+h)^2 - x^2 = x^2 + 2xh + h^2 - x^2 = 2xh + h^2`. Dividing by `h` is legal because the limit only considers `h` near 0 but never equal to 0, giving `2x + h`. As `h` goes to 0 that expression goes to `2x`, so `f'(x) = 2x`. This is the power rule `d/dx x^n = n*x^(n-1)` for `n = 2`, and the same argument run on `f(x) = 1/x` gives `-1/x^2`.

go deeper

for a junior

Be ready to write the difference quotient from memory and grind through the algebra for x^2 without hesitating. Knowing that the derivative is a slope, not a value, is the point being checked.

for a middle

Explain why cancelling h is not division by zero, and connect the worked result back to the power rule so the shortcut stops looking like magic.

for a senior

Show you know where the definition bites in practice: wherever that limit fails to exist the derivative does not exist either, however tidy the formula looks.

for a principal

Frame why the first-principles derivation is worth teaching at all. Teams that only memorise rules cannot reason about non-smooth objectives, and that gap surfaces later as silent modelling mistakes.

### What the derivative measures The derivative of a function `f` at a point `x` is the instantaneous rate of change of `f` there, equivalently the slope of the tangent line to the graph at that point. The word *instantaneous* is the whole difficulty: slope is normally defined between two distinct points, and at a single point there is only one point. The limit definition solves this by computing the slope between `x` and a nearby point `x + h`, then asking what that slope approaches as the nearby point slides in. ### The definition itself ``` f'(x) = lim(h -> 0) [ f(x + h) - f(x) ] / h ``` The fraction inside is the **difference quotient**. Geometrically it is the slope of the *secant* line cutting the graph at the two points `(x, f(x))` and `(x+h, f(x+h))`. The derivative is the number that secant slope converges to. If no single number is approached — for instance if approaching from the right and from the left give different answers — the derivative does not exist at that point. ### Working f(x) = x^2 from the definition Substitute and expand: 1. `f(x + h) = (x + h)^2 = x^2 + 2xh + h^2` 2. `f(x + h) - f(x) = x^2 + 2xh + h^2 - x^2 = 2xh + h^2` 3. Divide by `h`: `(2xh + h^2)/h = 2x + h` 4. Take the limit: as `h -> 0`, `2x + h -> 2x` So `f'(x) = 2x`. Sanity check at `x = 3`: the slope is 6, and indeed the function is rising steeply there; at `x = 0` the slope is 0, matching the flat bottom of the parabola; at `x = -3` the slope is -6, matching the descent on the left branch. ### Why cancelling the h is not division by zero A frequent objection is that step 3 divides by something that is heading to zero. The limit operation never evaluates the expression *at* `h = 0`; it asks only about values of `h` arbitrarily close to 0 and different from 0. For every such `h` the cancellation is ordinary arithmetic. Only after simplifying to `2x + h`, which is a perfectly well-behaved expression at `h = 0`, do we take the limit. Substituting `h = 0` *before* simplifying produces the meaningless `0/0`, which is why the algebra must come first. ### The same machine on f(x) = 1/x For `x` not equal to 0: 1. `f(x + h) - f(x) = 1/(x+h) - 1/x = [x - (x+h)] / [x(x+h)] = -h / [x(x+h)]` 2. Divide by `h`: `-1 / [x(x+h)]` 3. Let `h -> 0`: `-1 / x^2` So `d/dx (1/x) = -1/x^2`, which is the power rule again with `n = -1`, since `1/x = x^(-1)` and `-1 * x^(-2) = -1/x^2`. The derivative is negative everywhere, matching the fact that `1/x` is decreasing on each side of the origin. ### The rules this definition justifies Nobody re-derives every derivative from the limit in practice; the definition is used once to prove the shortcut rules, which are then applied mechanically: - **Constant**: `d/dx c = 0` - **Power rule**: `d/dx x^n = n*x^(n-1)`, valid for any real exponent `n` on the domain where it makes sense - **Sum and constant multiple**: `d/dx [a*f + b*g] = a*f' + b*g'` - **Product rule**: `(uv)' = u'v + uv'` — note it is *not* `u'v'` - **Quotient rule**: `(u/v)' = (u'v - uv')/v^2` — the minus sign and the squared denominator both matter - **Exponential and logarithm**: `d/dx e^x = e^x`, `d/dx ln x = 1/x` for `x > 0`, and `d/dx a^x = a^x * ln a` for a constant `a > 0` ### Why interviewers ask for the derivation Quoting `2x` takes two seconds and proves nothing. Producing the difference quotient, expanding it, cancelling correctly and only then taking the limit demonstrates that you know what a derivative *is* rather than which symbol to write down. It also sets up every later idea: a derivative is a limit, so wherever that limit fails to exist — a corner, a jump, a vertical tangent, wild oscillation — the derivative simply does not exist, no matter how smooth the formula looks.

  • Why is it legitimate to cancel the h in (2xh + h^2)/h before taking the limit?
    Because the limit only inspects values of `h` arbitrarily close to 0 but never equal to 0. For every such `h` the division is ordinary arithmetic, so the quotient genuinely equals `2x + h`. Only after that simplification, which is well defined at `h = 0`, do we take the limit. Substituting `h = 0` first would produce the meaningless form `0/0`.
  • What does the limit definition give for f(x) = 1/x?
    `-1/x^2` for any `x` other than 0. The numerator `1/(x+h) - 1/x` combines to `-h/[x(x+h)]`; dividing by `h` leaves `-1/[x(x+h)]`, and letting `h` go to 0 gives `-1/x^2`. It matches the power rule with exponent `-1`.
  • How does the limit definition explain why a constant function has derivative 0?
    For `f(x) = c` the numerator is `c - c = 0` for every `h`, so the difference quotient is `0/h = 0` for all non-zero `h`, and the limit of the constant 0 is 0. Geometrically the graph is a horizontal line: every secant is flat, so the tangent is flat.

A car's average speed over the last ten seconds is a secant slope. Shrinking that window toward zero seconds is exactly what the limit does, and what it converges to is the number on the speedometer: the instantaneous rate.

saying these in an interview costs you the question

  • Calls the derivative an average rate of change rather than an instantaneous one
  • Substitutes h = 0 into the quotient first, producing 0/0
  • Reports 2x + h as the derivative, forgetting to take the limit
  • Confuses the derivative 2x with the function value x^2
  • Quotes the power rule when explicitly asked to work from the definition

context

open as a page

Why is the absolute value function f(x) = |x| not differentiable at x = 0?

level: middleimportance: must knowfreq 66%

basics

~10 s

At 0 the difference quotient |h|/h equals +1 for positive h and -1 for negative h, so the two one-sided limits disagree and no single tangent slope exists. The function is still continuous there.

open as a page

Does continuity of a function at a point imply that it is differentiable there?

level: middleimportance: should knowfreq 48%

basics

~10 s

No. The implication runs one way only: differentiability forces continuity, never the reverse. The function x times sin(1/x), with value 0 at the origin, is continuous at 0 yet its difference quotient oscillates forever.

open as a page

Why does the logistic function s(x) = 1/(1 + e^-x) have derivative s(x)(1 - s(x))?

level: middleimportance: should knowfreq 55%

basics

~20 s

The quotient rule turns 1/(1 + e^-x) into e^-x/(1 + e^-x)^2. That expression splits into 1/(1 + e^-x) times e^-x/(1 + e^-x), and the second factor is exactly 1 minus the first, giving s times (1 - s).

open as a page

What is the subgradient set of the convex function f(x) = |x| at the point x = 0?

level: seniorimportance: should knowfreq 36%

basics

~20 s

The subdifferential of |x| at 0 is the closed interval from -1 to 1: every slope of magnitude at most 1 gives a line supporting the V from below. Zero lies inside, which certifies 0 as the minimiser.

open as a page

How does logarithmic differentiation give the derivative of f(x) = x^x?

level: middleimportance: nice to knowfreq 24%

basics

~20 s

Take logs first: ln y = x ln x for y = x^x, which differentiates to y'/y = ln x + 1, giving y' = x^x times (ln x + 1). Base and exponent both vary, so neither standard rule applies.

open as a page