Linear Algebra
10.76 min read

Limits at Infinity

A limit at infinity lim⁡x→∞f(x)=L\lim_{x \to \infty} f(x) = L means f(x)f(x) approaches LL as xx grows without bound. Similarly, lim⁡x→∞f(x)=∞\lim_{x \to \infty} f(x) = \infty means ff grows without bound.

Common examples: lim⁡x→∞1x=0\lim_{x \to \infty} \frac{1}{x} = 0 (decays to zero), lim⁡x→∞ex=∞\lim_{x \to \infty} e^x = \infty (grows exponentially), lim⁡x→∞x2+1x2+x=1\lim_{x \to \infty} \frac{x^2 + 1}{x^2 + x} = 1 (ratio approaches 1 for large xx).

Limits at infinity describe asymptotic behavior — what a function "looks like" for very large inputs. In machine learning, this describes how a model performs as training data grows. In algorithms, this is the basis of Big-O notation.

Formal View

Definition 10.6 — Limit at Infinity
We write lim⁡x→∞f(x)=L\lim_{x \to \infty} f(x) = L if for every ε>0\varepsilon > 0 there exists M>0M > 0 such that x>M  ⟹  ∣f(x)−L∣<εx > M \implies |f(x) - L| < \varepsilon. We write lim⁡x→−∞f(x)\lim_{x \to -\infty} f(x) similarly for x→−∞x \to -\infty.

Why This Matters

Asymptotic analysis — understanding function behavior for large inputs — is fundamental to algorithm analysis and convergence proofs.

  • Convergence of iterative algorithms: does ∥xk−x∗∥→0\|\mathbf{x}_k - \mathbf{x}^*\| \to 0 as k→∞k \to \infty?
  • Statistical consistency: does an estimator converge to the truth as n→∞n \to \infty?
  • Activation functions: lim⁡x→∞σ(x)=1\lim_{x \to \infty} \sigma(x) = 1 for the sigmoid.

Quiz

Question 1

What is lim⁡x→∞1x2\lim_{x \to \infty} \frac{1}{x^2}?

Question 2

lim⁡x→∞sin⁡(x)\lim_{x \to \infty} \sin(x) does not exist.

Common Mistakes

  • Treating ∞\infty as a number — lim⁡x→∞f(x)=∞\lim_{x\to\infty} f(x) = \infty means ff grows without bound, not that it equals a number called ∞\infty.
  • Thinking lim⁡x→∞f(x)=0\lim_{x \to \infty} f(x) = 0 means ff reaches zero — it only means ff gets arbitrarily close to zero.