7. The Determinant

The determinant of a matrix $A$ can be understood as the volume of the unit-cube after transformation. It can be stretched, squished or be reduced to a point (non-invertible matrix).

(Informally) The signed volume of this parallelepiped gives us the determinant.

7.1 Definition from Properties

Properties of the determinant

1. $\det (A)=\det (A^T)$
2. $\det (I)=1$
3. $\det (A)=0$ if linearly dependent columns.
4. Exchanging two rows flips the sign of the determinant.
5. Subtracting two rows does not change the $\det$. (we can use G-J (only row subs/adds) to simplify calculations…)

Multilinearity

The determinant preserves “multilinearity”. This means that changing only a single row will preserve the rest of the determinant (it’s linear for each row): $\left|\begin{array}{cc}ta& tb\\ c& d\end{array}\right|=t\cdot \left|\begin{array}{cc}a& b\\ c& d\end{array}\right|$ $\left|\begin{array}{cc}a+a^{\prime }& b+b^{\prime }\\ c& d\end{array}\right|=\left|\begin{array}{cc}a& b\\ c& d\end{array}\right|+\left|\begin{array}{cc}{a}^{\prime }& b^{\prime }\\ c& d\end{array}\right|$

Example: For a $2\times 2$ matrix, fixing the second column: $\left|\begin{array}{cc}\alpha a+\beta c& b\\ \alpha d+\beta f& e\end{array}\right|=\alpha \left|\begin{array}{cc}a& b\\ d& e\end{array}\right|+\beta \left|\begin{array}{cc}c& b\\ f& e\end{array}\right|$ Which gives us $(\alpha a+\beta c)e-b(\alpha d+\beta f)=\alpha (ae-bd)+\beta (ce-bf)$.

Important: Multilinearity does not mean $\det (\alpha A)=\alpha \det (A)$. Instead: $\det (\alpha A)={\alpha }^n\det (A)$ because scaling affects all $n$ columns simultaneously.

Determinant of Upper-Triangular

The $\det$ of $U$ upper (or lower) triangular is $\det U=(u_{11})(u_{22})\dots (u_{nn})$.

Using Gauss-Jordan to simply calculations

Because the determinant of an Upper- (or Lower)-Triangular matrix is the product of the pivots, we can often use G-J-Elimination to make computation easier.

We cannot exchange rows (without keeping track of the sign), nor can we multiply rows (without multiplying the determinant by that same factor) in the end.

7.2 General Case (def. using Permutations)

7.2.1 Sign of a Permutation

Given a permutation $\sigma :\{1,\dots ,n\}\to \{1,\dots ,n\}$ of $n$ elements, it’s sign $\text{sgn}(\sigma )$ can be -1 or 1. The sign counts the parity of the number of elements that are out of order (inversions) after applying the permutation.

The permutation $\pi$, defined as $\pi (1)=1$, $\pi (2)=3$ , $\pi (3)=2$ , $\pi (4)=4$, we have the pairs $(1,2),(1,3),(1,4),(2,3),(2,4),(3,4)$ for $i<j$. For all these pairs $(i,j)$, $\pi (i)<\pi (j)$ except for $(2,3)$ which gives $(\pi (2),\pi (3))=(3,2)$. Thus $\text{sgn}(\pi )=-1$.

This can also be counted as the parity of the number of row swaps necessary to get back to the identity.

Properties of the sign

1. The sign of a permutation is multiplicative: $\text{sgn}(\sigma \circ \lambda )=\text{sgn}(\sigma )\cdot \text{sgn}(\lambda )$.
2. For all $n\ge 2$, exactly half of the permutations have sign $1$ and the rest have sign $-1$.

7.2.3 Determinant formula

Given a square matrix $A\in {\mathbb{R}}^{n\times n}$, the determinant $\det (A)$ is defined as: $\det (A)={\sum }_{\sigma \in {\Pi }_n}\text{sgn}(\sigma ){\prod }_{i=1}^n{A}_{i,\sigma (j)}$ where ${\Pi }_n$ is the set of all permutations of $n$ elements (of which there are $n!$).

Visualised:

Examples In a $2\times 2$ matrix, there are $2!=1\cdot 2$ permutations, the identity and the inversion. From this we can get the formula for $2\times 2$ matrices.

Note that this is the multilinear decomposition These are the terms we get in the sum for $\det A$ according to the above formula. Each of them is one of the $3!=6$ permutations applied to the matrix. We then want to find the sign of the permutation: number of row exchanges to get back to a diagonal.

7.2.4 Properties of the Determinant

1. Given a permutation matrix $P\in {\mathbb{R}}^{n\times n}$ corresponding to a permutation $\sigma$, then $\det (P)=\text{sgn}(\sigma )$ (this is as $P$ is also an orthogonal matrix, see 3.). We sometimes write $\text{sgn}(P)$.
2. Given a triangular (either upper or lower) matrix $T\in {\mathbb{R}}^{n\times n}$, we have $\det (T)={\prod }_{k=1}^n{T}_{kk}$ in particular, $\det (I)=1$.
3. If $Q\in {\mathbb{R}}^{n\times n}$ is an orthogonal matrix then $\det (Q)=1\text{ or }\det (Q)=-1$ This makes sense as there is no scaling in an orthogonal matrix.

Intuition:

1. For the permutation matrix, each row contains only one entry: a $1$. Thus the only permutation $\sigma$ in the product that doesn’t have a $0$ factor is the permutation corresponding to the matrix $P$ itself. The product is $1\cdot 1\cdots \cdot 1$ thus we get $\text{sgn}(\sigma )=\text{sgn}(P)$.
2. For a triangular matrix, if we choose an element off the diagonal, we are then forced to choose one in the $0$s thus making that factor $0$. The only valid permutation is thus the $\text{id}$, which means we just multiply the diagonals.
3. As $Q$ is orthogonal, we don’t scale (preserves $\top$) thus the unit cube is just turned, not scaled.

Note that for a triangular matrix, if one of the diagonal entries is zero, the determinant is also 0 (as it’s in the product). This matches what we observed before.

7.2.5 Determinant of the Transpose

Given a matrix $A\in {\mathbb{R}}^{n\times n}$, then $\det (A)=\det (A^{\top })$

Proof Idea: This follows from the fact that the inverse of a permutation has the same sign, and transposing is the same as doing the inverse permutation. This also means that we can choose a row or a column when doing cofactor calculations.

7.2.6 Determinant Properties 2

1. A matrix $A\in {\mathbb{R}}^{n\times n}$ is invertible if and only if $\det (A)\ne 0$
2. Given matrices $A,B\in {\mathbb{R}}^{n\times n}$, we have $\det (AB)=\det (A)\cdot \det (B)$
3. Given a matrix $A$ such that $\det (A)\ne 0$ then $A$ is invertible and $\det (A^{-1})=\frac{1}{\det (A)}$

Intuitively, these properties make sense if you think of the determinant as being the volume of the unit cube.

1. If the unit cube collapses to have 0 volume (i.e. $\det (A)=0$) then we lost a dimension and $A$ cannot be invertible.
2. If we multiply first by $A$ then $B$ the unit cube will be stretched the same way as if we did both at once.
3. If $A$ shrinks the unit cube and $A^{-1}$ inflates it back to the unit dimensions then the ratio of the changes is $1$.

Property from the sheet 10

For $A\in {\mathbb{R}}^{n\times n}$ and $\lambda \in \mathbb{R}$ we have $\det (\lambda B)={\lambda }^n\det (B)$.

7.3 Cofactors, Cramer’s Rule and Beyond

7.3.1 Cofactors

Given $A\in {\mathbb{R}}^{n\times n}$ for each $1\le i,j\le n$ let ${\mathcal{A}}_{ij}$ denote the $(n-1)\times (n-1)$ matrix obtained by removing row $i$ and column $j$ from $A$. Then we define the co-factors of $A$ as $C_{ij}=(-1{)}^{i+j}\det ({\mathcal{A}}_{ij})$

Note that the sign basically draws a + - + - + - grid on the matrix: $\left[\begin{array}{ccccccc}+& -& +& -& +& \dots & \\ -& +& -& +& -& \dots & \\ +& -& +& -& +& \dots & \\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \end{array}\right]$ We can then express the determinant as a sum of these co-factors: $\det (A)={\sum }_{j=1}^n{A}_{ij}{C}_{ij}$ in which we multiply the cofactor of every element by the element itself, as is clear in the example above for a 3x3.

We can also define the inverse in terms of co-factors now: $A^{-1}=\frac{1}{\det (A)}{C}^{\top }$ or rewritten as $A{C}^{\top }=\det (A)I$. Note that this is not an efficient way to find the inverse.

7.3.6 $\det (PA)$

If $A$ is a matrix and $P$ is a permutation that swaps two elements (i.e. $\text{sgn}(P)=-1$) then $PA$ corresponds to swapping two rows of $A$ then $\det (PA)=-\det (A)$.

7.3.7 Determinant is Linear

The determinant is linear in each row (or each column). In other words for any $a_0,a_1,a_2,\dots ,a_n\in {\mathbb{R}}^n$ and ${\alpha }_0,{\alpha }_1\in \mathbb{R}$ we have

Tricks

Constraints on Permutations

During the determinant calculation we have to look at all possible permutations and sum those. If $\sigma (i)\ne i$, i.e. we venture off the diagonal (identity permutation), we’re going to have to venture off the diagonal for another $j$ as well: $\sigma (j)\ne j$.

If we have a matrix $A=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$, the only permutation that doesn’t produce a $0$ product is the $\text{id}$ permutation.