10.207 min read

Differentiability vs Continuity

We have seen that differentiability is a stronger requirement than continuity: differentiable $\implies$ continuous, but continuous $\not\implies$ differentiable. The function $|x|$ is continuous but not differentiable at $0$ .

Where exactly does a function fail to be differentiable? Three places: at kinks (corners, where left and right derivatives differ), at cusps (vertical tangents, where the difference quotient diverges to $\pm\infty$ ), and at oscillatory discontinuities of the derivative (where the derivative exists but oscillates wildly). In each case, the limit defining the derivative fails to exist as a finite number.

This logical hierarchy — continuity at minimum, differentiability a step above — will expand in the next section to include $D^1$ (differentiable) vs $C^1$ (continuously differentiable).

Formal View

Theorem 10.3 — Differentiability Implies Continuity

f

is differentiable at

x_0

, then

f

is continuous at

x_0

Proof: $f(x) - f(x_0) = \frac{f(x)-f(x_0)}{x-x_0} \cdot (x-x_0) \to f'(x_0) \cdot 0 = 0$ as $x \to x_0$ . The converse fails: $|x|$ is continuous but not differentiable at $0$ .

Why This Matters

Understanding the hierarchy of regularity is essential for knowing when optimization algorithms have convergence guarantees.

Subgradient methods extend gradient descent to non-differentiable (but continuous) functions like $|x|$ .
Smooth optimization ( $C^1$ functions) has better convergence rates than non-smooth optimization.
Regularization: L2 regularization ( $\|w\|^2$ , smooth) vs L1 ( $\|w\|_1$ , non-smooth) leads to different algorithmic requirements.

Learning Resources

Differentiability implies continuity

Khan Academy

Proof that differentiability implies continuity, with counterexample for converse.

8 min

Non-smooth optimization

MIT OpenCourseWare

Overview of challenges in optimizing non-differentiable functions.

17 min

Quiz

Question 1

Which implication is TRUE?

Question 2

A function with a kink (corner) at $x_0$ can still be continuous at $x_0$ .

Common Mistakes

Believing continuity implies differentiability — $|x|$ is the classic counterexample.
Thinking a kink makes the function discontinuous — it does not; continuity and differentiability are different conditions.