Vectors and Matrices

Calculus gives you tools for change in one variable at a time. Engineering problems, by contrast, almost always involve many quantities that move together: forces on a structure, currents in a circuit, pixels in an image. Linear algebra is the mathematics of these coupled systems, and its two basic objects are the vector and the matrix ¹.

Vectors

A vector is an ordered list of numbers, called its components. We write a vector in \(n\)-dimensional space as a column of \(n\) numbers, for example

There are two complementary pictures. Algebraically, \(\mathbf a\) is just the ordered tuple \((2, -1, 3)\). Geometrically, it is an arrow from the origin to the point with those coordinates. Both pictures matter: the tuple lets you compute, the arrow lets you reason about direction and length.

Two operations make vectors useful. Addition is component-by-component:

which geometrically places the arrows tip-to-tail. Scalar multiplication stretches a vector: \(c\mathbf a\) multiplies every component by \(c\), so \(2\mathbf a\) is twice as long in the same direction and \(-\mathbf a\) points the opposite way. Combining these two operations produces a linear combination such as \(c_1\mathbf a + c_2\mathbf b\), the single most important construction in the subject ¹.

The dot product, norm, and angle

The dot product of two vectors of the same length is a single number formed by multiplying matching components and summing:

It also has a geometric meaning that ties the algebra to the picture:

where \(\theta\) is the angle between the two arrows ¹. Three consequences follow immediately. First, taking the dot product of a vector with itself gives the squared norm (length): \(\mathbf a \cdot \mathbf a = \|\mathbf a\|^2\), so \(\|\mathbf a\| = \sqrt{a_1^2 + \cdots + a_n^2}\). Second, since \(\cos\theta\) appears in the formula, the dot product measures how much two vectors point the same way. Third, when \(\cos\theta = 0\) the vectors are perpendicular, so \(\mathbf a \cdot \mathbf b = 0\) is exactly the test for orthogonality.

In three dimensions there is a second product, the cross product \(\mathbf a \times \mathbf b\), which returns a vector rather than a number. It is perpendicular to both inputs, and its length \(\|\mathbf a\|\|\mathbf b\|\sin\theta\) equals the area of the parallelogram they span. It is indispensable in mechanics and electromagnetism, but unlike the dot product it exists only in 3D and is not central to what follows ¹.

Matrices

A matrix is a rectangular array of numbers with \(m\) rows and \(n\) columns. Like vectors, matrices add component-by-component (only between matrices of the same shape) and scale by multiplying every entry by a scalar. The deeper view, and the one engineers rely on, is that a matrix is a linear operator: it is a machine that takes an input vector and returns an output vector, transforming the whole space in a way that respects addition and scaling.

The most important operation is matrix multiplication, defined so that the product represents applying one transformation after another. The entry in row \(i\), column \(j\) of \(AB\) is the dot product of row \(i\) of \(A\) with column \(j\) of \(B\) — “row by column.” For \(2\times 2\) matrices:

For the product \(AB\) to exist, the number of columns of \(A\) must equal the number of rows of \(B\). Note that matrix multiplication is generally not commutative: \(AB\) and \(BA\) need not be equal, which reflects that doing transformations in a different order gives a different result ¹.

Matrix–vector products and the central problem

A vector is just a matrix with one column, so the same rule defines the matrix–vector product \(A\mathbf x\). The clearest way to read it is as a linear combination of the columns of \(A\), with the entries of \(\mathbf x\) as the weights:

This viewpoint makes the geometry obvious: the outputs \(A\mathbf x\) are exactly the vectors you can build from the columns of \(A\) ¹.

Two special matrices round out the basics. The identity matrix \(I\) has ones on the diagonal and zeros elsewhere; it is the transformation that changes nothing, so \(I\mathbf x = \mathbf x\) for every \(\mathbf x\). The transpose \(A^{\mathsf T}\) flips a matrix across its diagonal, turning rows into columns; with it, the dot product can be written compactly as \(\mathbf a \cdot \mathbf b = \mathbf a^{\mathsf T}\mathbf b\).

With these pieces in place, nearly every applied problem reduces to one question: given a matrix \(A\) and a target vector \(\mathbf b\), find the input \(\mathbf x\) for which

Solving a circuit, fitting a model, or balancing a structure all take this form. The equation \(A\mathbf x = \mathbf b\) — when it has a solution, when it has many, and how to find them — is the central object of the lessons that follow ¹.