The
Math
Book

light mode dark mode pink mode

Linear Algebra

menu

Basis and Dimension

Dimensions comes naturally when speaking of “vectors" as arrows that live in our geometric space. The zero dimension vector will just be thought of as a point with no direction, one dimension vector an arrow on a line, two dimension vector on a plane, three dimension an arrow in the space that we live in.

To use the notion of dimension for our generalized vector space, we want to observe how vectors behave in the space. If you just have two vectors sitting on the one-dimensional line, no matter how you add them or scale them, they stay on that line. This is the notion that they are linearly dependent. But once the second vector deviates away from the line, in other words, once it has some component perpendicular to the line, it “opens up" another dimension and span into the 2 dimensional space. Then you get two linearly independent vectors. Now we generalize this concept in vector space:

Let V be a vector space over F. A subset S of V consisting of distinct vectors α1, α2, …, αn is said to be linearly dependent (or simply, dependent) if there exist scalars c1, c2, …, cn in F, not all of which are 0 , such that

c1α1 + c2α2 + ⋯ + cnαn = 0.

A set which is not linearly dependent is called linearly independent. If the set S contains only finitely many vectors α1, α2, …, αn, we sometimes say that α1, α2, …, αn are linearly dependent (or independent).

The following are easy consequences of the definition.

  1. Any subset of a linearly independent set is linearly independent.

  2. Any set which contains the 0 vector is linearly dependent; for 1 ⋅ 0 = 0.

  3. A set S of vectors is linearly independent if and only if each finite subset of S is linearly independent, i.e., if and only if for any distinct vectors α1, …, αn of S, c1α1 + ⋯ + cnαn = 0 implies each ci = 0. Another way to think about it is that each vector is linearly independent with each other.

We can see how this definition fits our vector as arrows. If we have one vector in one dimensional line, it will have to be scaled by scalar 0 to reach become the zero vector. So, the vector is by itself, linearly independent. Once we add another vector, we can always scale them to equal lengths and opposite direction so they add up to the zero vector. They are therefore linearly dependent.

Let V be a vector space. A basis for V is a set of linearly independent vectors in V that spans V. i.e. all vectors in V can be written as a linear combination of the elements of the basis. The space V is finite dimensional if it has a finite basis.

Let F be a subfield of the complex numbers. In F3 the vectors

$$\begin{aligned} \alpha_{1} & =(1,0,2) \\ \alpha_{2} & =(1,1,-1) \\ \alpha_{3} & =(2,-2,0) \\ \alpha_{4} & =(1,3,3) \end{aligned}$$

are linearly dependent, since

2α1 + α2 − α3 − α4 = 0.

Think back to arrows in three dimension, if we choose a coordinate system. We will need three coordinates to specify the direction of the arrow. This nicely corresponds to three 3-tuples in F3. Now when we try to add another 3-tuple, it will have to fall into the span of the three 3-tuples we already have. It is therefore linearly dependent with the original three 3-tuples. To make it linearly independent, we will have to make all the n-tuples 4-tuples, and the added 4-tuple needs to have a component perpendicular to the three dimensional space we have. We now have an intuition from geometry that if we have more element in the basis then the dimension of space, we necessarily have a linearly dependent basis.

Now obviously: $$\begin{aligned} & \epsilon_{1}=(1,0,0) \\ & \epsilon_{2}=(0,1,0) \\ & \epsilon_{3}=(0,0,1) \end{aligned}$$

are linearly independent: to linearly combine them into 0, they each have to be scaled by 0. In fact, {ϵ1, ϵ2} is linearly independent by consequence [consequence:subset_of_linearly_independent_set] above.

Let P be an invertible n × n matrix with entries in the field F. Let P1, …, Pn be the columns of P, $$\begin{bmatrix} \begin{bmatrix}\\\\P_1\\\\\\ \end{bmatrix} \begin{bmatrix}\\\\P_2\\\\\\ \end{bmatrix} \cdots \begin{bmatrix}\\\\P_n\\\\\\ \end{bmatrix} \end{bmatrix}$$ They form a basis for the space of column matrices, Fn × 1. $$\begin{bmatrix} x_{1} \\ x_{2} \\ \vdots \\ x_{n} \\ \end{bmatrix}$$To see that they indeed form a basis, we need them to be linearly independent and span the space Fn × 1. We use the properties of an inverse matrix here, first we write linear combination of column vectors Pk as the matrix P multiply by a column matrix X is a column matrix,

x1P1 + ⋯ + xnPn = PX.

But P is an invertible matrix, so PX = 0 has only the trivial solution X = 0, and {P1,…,Pn} is a linearly independent set. To see that it spans Fn × 1, if let Y be any column matrix, we will want to write it as a linear combination of the set {P1,…,Pn}. But P has a inverse, so we can always get another column matrix X such that X = P−1Y. Then Y = PX, that is,

Y = x1P1 + ⋯ + xnPn.

So {P1,…,Pn} is a basis for Fn × 1. This gives us the [cols_rows_independent] and [cols_rows_span] of our equivalence theorem.

Basis for solution space of a system of linear equations Back to solving m linear equations with n unknowns, we hope to find a basis for the solution space so we can express the solution as a linear combination of the bases elements. Let A be an m × n matrix and let S be the solution space for the homogeneous system AX = 0 (Example 7). Let R be a rowreduced echelon matrix which is row-equivalent to A. Then S is also the solution space for the system RX = 0. If R has r non-zero rows, then the system of equations RX = 0 simply expresses r of the unknowns x1, …, xn in terms of the remaining (nr) unknowns xj. Suppose that the leading non-zero entries of the non-zero rows occur in columns k1, …, kr. Let J be the set consisting of the n − r indices different from k1, …, kr :

J = {1, …, n} − {k1,…,kr}.

The system RX = 0 has the form

$$\begin{aligned} & x_{k 1}+\sum_{J} c_{1 j} x_{j}=0 \\ & \vdots \begin{array}{cc}\vdots & \vdots \\x_{k_{r}}+\sum_{J} c_{r j} x_{j}=0\end{array} \end{aligned}$$

where the cij are certain scalars. All solutions are obtained by assigning (arbitrary) values to those xj ’s with j in J and computing the corresponding values of xk1, …, xkr. For each j in J, let Ej be the solution obtained by setting xj = 1 and xi = 0 for all other i in J. We assert that the (nr) vectors Ej, j in J, form a basis for the solution space.

Since the column matrix Ej has a 1 in row j and zeros in the rows indexed by other elements of J, the reasoning of Example 13 shows us that the set of these vectors is linearly independent. That set spans the solution space, for this reason. If the column matrix T, with entries t1, …, tn, is in the solution space, the matrix

N = ∑JtjEj

is also in the solution space and is a solution such that xj = tj for each j in J. The solution with that property is unique; hence, N = T and T is in the span of the vectors Ej.