Eigenvalues and Eigenvectors

The Special Directions of an Operator

A matrix \(A\) acting on a vector usually does two things at once: it stretches the vector and it rotates it into a new direction. For most inputs both effects are tangled together. But for a handful of special directions the rotation disappears entirely — the operator only stretches or compresses the vector along the line it already occupies. Those privileged directions, and the amounts by which the operator scales them, are the eigenvectors and eigenvalues. They are the capstone of linear algebra because they expose the simplest possible description of what a matrix does ¹.

A nonzero vector \(\mathbf v\) is an eigenvector of a square matrix \(A\), with eigenvalue \(\lambda\), when

Read this literally: feeding \(\mathbf v\) through the operator gives back the same vector, merely rescaled by the number \(\lambda\). If \(\lambda>1\) the vector is stretched; if \(0<\lambda<1\) it shrinks; if \(\lambda<0\) it flips and rescales; and \(\lambda=0\) signals a direction the operator collapses to zero. We insist \(\mathbf v\neq\mathbf 0\) because the zero vector trivially satisfies the equation for any \(\lambda\) and tells us nothing.

Finding Them: The Characteristic Equation

Rewrite the defining equation as \(A\mathbf v-\lambda\mathbf v=\mathbf 0\), or \((A-\lambda I)\mathbf v=\mathbf 0\), where \(I\) is the identity matrix. We are asking for a nonzero vector in the null space of \(A-\lambda I\). A square matrix has a nonzero null space exactly when it is singular, i.e. when its determinant vanishes. This gives the characteristic equation ¹:

For an \(n\times n\) matrix this is a degree-\(n\) polynomial in \(\lambda\); its roots are the eigenvalues. Once an eigenvalue is known, substitute it back and solve \((A-\lambda I)\mathbf v=\mathbf 0\) for the eigenvectors.

A Worked 2×2 Example

Take

Form \(A-\lambda I\) and take its determinant:

Expanding, \((2-\lambda)^2=1\), so \(2-\lambda=\pm 1\), giving \(\lambda_1=3\) and \(\lambda_2=1\). (As a check, the eigenvalues should sum to the trace \(2+2=4\) and multiply to the determinant \(4-1=3\); indeed \(3+1=4\) and \(3\cdot 1=3\).)

For \(\lambda_1=3\), solve \((A-3I)\mathbf v=\mathbf 0\):

Both rows give \(-x+y=0\), so \(y=x\) and an eigenvector is \(\mathbf v_1=\begin{bmatrix} 1 \\ 1 \end{bmatrix}\). Verifying: \(A\mathbf v_1=\begin{bmatrix} 3 \\ 3 \end{bmatrix}=3\mathbf v_1\). For \(\lambda_2=1\) the same procedure yields \(y=-x\), so \(\mathbf v_2=\begin{bmatrix} 1 \\ -1 \end{bmatrix}\), and \(A\mathbf v_2=\begin{bmatrix} 1 \\ -1 \end{bmatrix}=1\cdot\mathbf v_2\).

Eigenspaces

Any nonzero scalar multiple of an eigenvector is again an eigenvector for the same eigenvalue, since \(A(c\mathbf v)=c A\mathbf v=c\lambda\mathbf v=\lambda(c\mathbf v)\). The full set of vectors satisfying \((A-\lambda I)\mathbf v=\mathbf 0\) — that is, the null space of \(A-\lambda I\) together with the zero vector — forms a subspace called the eigenspace of \(\lambda\) ¹. Its dimension counts how many independent directions the operator scales by that single factor. In the example each eigenspace is a one-dimensional line.

Diagonalization

Suppose an \(n\times n\) matrix has \(n\) independent eigenvectors. Stack them as the columns of a matrix \(P\) and place the eigenvalues on the diagonal of \(D\). Because \(AP\) applies \(A\) to each column, \(AP=PD\), and since \(P\) is invertible,

This diagonalization is the operator viewed in its own natural coordinate system, where it acts as pure scaling. Its great payoff is computing powers. Since \(A^k=PD^kP^{-1}\) and \(D^k\) merely raises each diagonal entry to the \(k\)-th power, an otherwise expensive repeated multiplication collapses to scaling each eigenvalue ¹. This is the engine behind solving difference and differential equations driven by a fixed matrix.

Why Engineers Care

Eigenvalues are not an algebraic curiosity; they describe how physical systems behave.

• Natural frequencies and modes of vibration. The free vibration of a structure reduces to an eigenvalue problem. Eigenvectors give the mode shapes — the patterns in which a bridge, wing, or molecule oscillates — and the eigenvalues set their natural frequencies. Designers avoid driving a structure near these to prevent resonance.
• Stability of dynamical systems. For a linear system \(\dot{\mathbf x}=A\mathbf x\), solutions evolve like \(e^{\lambda t}\) along each eigenvector. If every eigenvalue has negative real part, disturbances decay and the system is stable; a positive real part means growth and instability, while imaginary parts produce oscillation ¹.
• Principal axes. For symmetric matrices — like inertia tensors or stress tensors — the eigenvectors are mutually perpendicular and define the principal axes along which the physics decouples, turning a coupled problem into independent one-dimensional ones.
• Data and PCA. The eigenvectors of a data set’s covariance matrix point along directions of greatest variance, and the eigenvalues measure that variance. Principal component analysis keeps the largest few, compressing data while preserving its dominant structure — eigen-decomposition applied to information rather than motion.

In every case the same idea recurs: find the directions the operator leaves intact, and a complicated system reveals a set of simple, independent behaviors.