2.912 min read

Linear Dependence

A set of vectors is linearly dependent if one of them can be written as a linear combination of the others — it's "redundant." More precisely: the equation $c_1\mathbf{a}_1 + \cdots + c_k\mathbf{a}_k = \mathbf{0}$ has a nontrivial solution (not all $c_i = 0$ ).

A set is linearly independent if the only solution to $c_1\mathbf{a}_1 + \cdots + c_k\mathbf{a}_k = \mathbf{0}$ is the trivial one: $c_1 = \cdots = c_k = 0$ . No vector in the set is redundant.

Geometrically: two vectors are dependent iff they are parallel (point in the same or opposite direction). Three vectors in $\mathbb{R}^3$ are dependent iff they are coplanar. Dependence = geometric redundancy.

Formal View

Definition 2.9 — Linear Independence

Vectors

\mathbf{a}_1, \ldots, \mathbf{a}_k

are linearly independent if

c_1\mathbf{a}_1 + \cdots + c_k\mathbf{a}_k = \mathbf{0} \implies c_1 = c_2 = \cdots = c_k = 0

They are linearly dependent if a nontrivial solution exists (some

c_i \neq 0

Interactive Visualization

Linear Dependence Explorer

Why This Matters

Linear independence is the mathematical formalization of "no redundancy" — every vector adds new information.

Independent sensors: each sensor measurement adds new information about the state
Data science: linearly dependent features are redundant and should be removed
A basis must be linearly independent — any redundancy wastes parameters

Learning Resources

Linear independence

Khan Academy

Testing for linear dependence and independence.

13 min

Quiz

Question 1

The vectors $(1, 2)$ and $(2, 4)$ are linearly independent.

Question 2

A set of vectors that includes the zero vector is:

Common Mistakes

Testing dependence only by checking if one vector is a multiple of another — three or more vectors can be dependent without any being a multiple of another.
Confusing "zero vector" with "zero coefficients" — a set containing the zero vector is always dependent.