Chapter -1.2

Span

The span of a set of vectors is the set of all linear combinations.

$\text{Span}({\text{v}}_1,{\text{v}}_2,...,{\text{v}}_n):=\{{\sum }_{j=1}^n{\lambda }_j{\text{v}}_j:{\lambda }_j\in \mathbb{R}\text{for all}j=[n]\}$ in ${\mathbb{R}}^3$ there are three possible cases:

• a line through the origin
• a plane
• the entire space

$\text{0}\in \text{Span}$ for any vectors, even if it’s the empty set ($\text{Span}()=\{\text{0}\}$).

LEMMA

Adding or removing an element from the set doesn’t change the span if the element is a linear combination of the other vectors.

LEMMA

The span of $m$ linearly independent vectors is ${\mathbb{R}}^m$.

Matrices

The addition of two matrices happens element-wise. The scalar multiplication of a matrix is also element-wise. The zero matrix is the matrix in which every element is 0. If m (rows) = n (columns) the matrix is square.

Kronecker delta

The Kronecker delta is a function taking in $i,j$ and outputting $1$ if $i=j$ else $0$. It’s used to define the Identity matrix.

Identity Matrix

The identity Matrix $\text{I}$ is defined as $\text{I}:=[{\delta }_{ij}{]}_{i=1,j=1}^{mm}$

Matrix-Vector multiplication

$A\text{x}$ is the multiplication of Matrix $A\in {\mathbb{R}}^{m\times n}$ with the vector $\text{v}\in {\mathbb{R}}^n$.

• The result is a vector with $m$ rows: each row is the scalar product of row $a_i$ with $x$.
• The resulting vector is the sum of all columns $a_j\times x_i$ → this is the definition of the linear combination.

The matrix-scalar multiplication is thus basically a linear combination.

The Identity matrix times a vector gives back that exact vector.

Column space

The column space of a matrix $\text{C}(A)$ of $A$ is the span (set of all linear combinations) of the columns (interpreted as vectors).

In the same way as $\text{0}$ is always in the span of any set of vectors, $0$ is always part of the column space.

Rank of a Matrix

The rank of a matrix $\text{rank}(A)$ $A\in {\mathbb{R}}^{m\times n}$ is a number between $0$ and $n$ which counts the number of independent columns.

A column is independent if it is not the linear combination of the previous ones (or the next ones, if you do it the other way round).

The rank is $0$ exactly if $A$ is the zero matrix.

We can reorder the columns of a matrix and thus get different independent columns, but the rank will stay the same.

Column space and Rank

The independent columns of $A$ span the column space $\text{C}(A)$ of $A$.

This can be proven by the lemma that says that adding elements that are a linear combination of the other ones, doesn’t change the span (Lemma 1.26 in the script).