Linear independence, dimension, and basis of a linear space

The possibility of expressing a vector as a linear combination of other vectors, or lack thereof, plays a role in many settings. In order to do so, we must ensure that the set of vectors that we want to use as a building blocks is “rich enough.” If we are given a set of vectors , and we take linear combinations of them, we generate a linear subspace of vectors. We speak of a subspace, because it is in general a subset of ; it is linear in the sense that any linear combination of vectors included in the subspace belongs to the subspace as well. We say that the set of vectors is a spanning set for the subspace. However, we would like to have a spanning set that is “minimal,” in the sense that all of the vectors are really needed and there is no redundant vector. It should be clear that if one of the vectors in the spanning set is a linear combination of the others, we can get rid of it without changing the spanning set.

Example 3.9 Consider the unit vectors in :

It is easy to see that we may span the whole space  by taking linear combinations of e₁ and e₂. If we also consider vector e₃ = [1, 1]^T, we get another spanning set, but we do not change the spanned set, as the new vector is redundant:

This can be rewritten as follows:

The previous example motivates the following definition.

DEFINITION 3.3 (Linear dependence) Vectors  are linearly dependent if and only if there exist scalars α₁, α₂, …, α_k, not all zero, such that

Indeed, vectors e₁, e₂, and e₃ in the example above are linearly dependent. However, vectors e₁ and e₂ are clearly not redundant, as there is no way to express one of them as a linear combination of the other one.

DEFINITION 3.4 (Linear independence) Vectors  are linearly independent if and only if

implies α₁ = α₂ = ··· = α_k = 0.

Consider now the space . How many vectors should we include in a spanning set for ? Intuition suggests that

• The spanning set should have at least n vectors.
• If we include more than n vectors in the spanning set, some of them will be redundant.

This motivates the following definition.

DEFINITION 3.5 (Basis of a linear space) Let v₁, v₂, …, v_k be a collection of vectors in a linear space V. These vectors form a basis for V if

1. v₁, v₂, …, v_k span V.
2. v₁, v₂, … v_k are linearly independent.

Example 3.10 Referring again to Example 3.9, the set {e₁, e₂, e₃} is a spanning set for , but it is not a basis. To get a basis, we should get rid of one of those vectors. Note that the basis is not unique, as any of these sets does span . These sets have all dimension 2.

We cannot span  with a smaller set, consisting of just one vector. However, let us consider vectors on the line y = x. This line consists of the set of vectors w of the form

This set W is a linear subspace of  in the sense that

Vector e₃ is one possible basis for subspace W.

The last example suggests that there are many possible bases for a linear space, but they have something in common: They consist of the same number of vectors. In fact, this number is the dimension of the linear space. We also see that a space like  may include a smaller linear subspace, with a basis smaller than n.

As a further example, we may consider the plane xy as a subspace of . This plane consists of vectors of the form

and the most natural basis for it consists of vectors

Such unit vectors yield natural bases for many subspaces, but they need not be the only (or the best) choice.

We close this section by observing that if we have a basis, and a vector is represented using that basis, the representation is unique.