Linear Algebra
8.18 min read

Functions on Rⁿ

A function on Rn\mathbb{R}^n takes a vector x=(x1,x2,…,xn)\mathbf{x} = (x_1, x_2, \ldots, x_n) as input and returns a single real number. You can picture it as a machine: feed in a point in nn-dimensional space, get out a height. These scalar-valued functions are the core objects of optimization — you want to find the input vector that makes the output as small (or as large) as possible.

Functions can have different levels of "complexity." The simplest are constant functions (same output no matter what x\mathbf{x} you put in), followed by linear functions (output scales proportionally with the input), and then quadratic functions (output involves products of pairs of variables). This chapter focuses on all three types, because understanding them is essential for optimization in later chapters.

Formal View

Definition 8.1 — Scalar-Valued Function on Rⁿ
A scalar-valued function (or real-valued function) on Rn\mathbb{R}^n is a map f:Rn→Rf: \mathbb{R}^n \to \mathbb{R}. It assigns to each vector x∈Rn\mathbf{x} \in \mathbb{R}^n a single real number f(x)∈Rf(\mathbf{x}) \in \mathbb{R}.

In optimization we seek x∗=arg⁡min⁡x∈Rnf(x)\mathbf{x}^* = \arg\min_{\mathbf{x} \in \mathbb{R}^n} f(\mathbf{x}), the input that minimizes the function's output.

Why This Matters

Nearly every real-world optimization problem — from training a neural network to engineering a bridge — reduces to minimizing a scalar function on a vector space.

  • Loss functions in machine learning: f(w)=1N∑i=1N(yi−w⊤xi)2f(\mathbf{w}) = \frac{1}{N}\sum_{i=1}^N (y_i - \mathbf{w}^\top \mathbf{x}_i)^2 measures how well parameters w\mathbf{w} fit data.
  • Potential energy in physics: f(q)f(\mathbf{q}) gives the energy stored in a mechanical system at configuration q\mathbf{q}.
  • Portfolio risk in finance: f(w)=w⊤Σwf(\mathbf{w}) = \mathbf{w}^\top \Sigma \mathbf{w} measures the variance of a portfolio with weights w\mathbf{w}.

Quiz

Question 1

Which of the following is an example of a scalar-valued function on R2\mathbb{R}^2?

Question 2

A scalar-valued function on Rn\mathbb{R}^n can return a vector as its output.

Common Mistakes

  • Confusing scalar-valued functions f:Rn→Rf: \mathbb{R}^n \to \mathbb{R} with vector-valued functions f:Rn→Rm\mathbf{f}: \mathbb{R}^n \to \mathbb{R}^m — the chapter focuses exclusively on scalar outputs.
  • Thinking that "function on Rn\mathbb{R}^n" implies anything about the complexity of ff — it can be any rule that assigns a number to each input vector.