Linear Algebra: IX Four Fundamental Subspaces

Four Fundamental Subspaces


Consider a linear transformation $T: \mathbb{R}^n \rightarrow \mathbb{R}^m$ defined by $T(\vec{x}) = A\vec{x}$ where $A$ is an $m$ by $n$ matrix.


Row Echelon Form (REF)

An $m$ by $n$ matrix $A$ is in row echelon form (REF) if:


  • Each row has a non-zero leading entry called its pivot.

  • The next pivot is to the right of the previous pivot above it.

  • Every entry below each pivot is zero.

  • All zero rows are at the bottom.

Reduced Row Echelon Form (RREF)

An $m$ by $n$ matrix $A$ is in reduced row echelon form (RREF) if also:


  • Each pivot is normalized to one.

  • Every entry above each pivot is zero.

Every $m$ by $n$ matrix $A$ has a unique RREF.


Column Space

The column space of an $m$ by $n$ matrix $A$, denoted $\mathrm{Col}(A)$, is the set of all linear combinations of the columns $\vec{c}_1 \cdots \vec{c}_n \in \mathbb{R}^m$ of $A$. It describes the set of all output vectors in $\mathbb{R}^m$ of the linear transformation $T$:

$$ \mathrm{Col}(A) = \mathrm{span}\{\vec{c}_1, \dots, \vec{c}_n\} = \{ T(\vec{x}) \in \mathbb{R}^m \mid \vec{x} \in \mathbb{R}^n \} \subseteq \mathbb{R}^m $$

Row Space

The row space of an $m$ by $n$ matrix $A$, denoted $\mathrm{Row}(A)$, is the set of all linear combinations of the rows $\vec{r}_1^\top \cdots \vec{r}_m^\top \in \mathbb{R}^n$ of $A$. It is thus the column space of the transpose of $A$. It describes the set of all vectors in $\mathbb{R}^n$ whose directions determine the output vectors of the linear transformation $T$:

$$ \mathrm{Row}(A) = \mathrm{span}\{\vec{r}_1^\top, \dots, \vec{r}_m^\top\} = \mathrm{Col}(A^\top) = \{ A^\top \vec{y} \in \mathbb{R}^n \mid \vec{y} \in \mathbb{R}^m \} \subseteq \mathbb{R}^n $$

Comments

Post a Comment

Popular posts from this blog

Linear Algebra: III Square and Inverse Matrices

Square and Inverse Matrices An $n$ by $n$ square matrix has the same number of rows and columns: $$ A = \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & & \vdots \\ a_{n1} & \cdots & a_{nn} \end{bmatrix} $$ If $A$ is an $n$ by $n$ square matrix and $\vec{x} \in \mathbb{R}^n$, then the product $A\vec{x}$ is also a vector in $\mathbb{R}^n$, defining a linear transformation $T: \mathbb{R}^n \to \mathbb{R}^n$. The Identity Matrix The identity matrix $I$ is an $n$ by $n$ square matrix with 1s on the main diagonal and 0s elsewhere: $$ I = \begin{bmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & & 0 \\ \vdots & & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{bmatrix} $$ It satisfies: $$ AI = IA = A $$ Inverse Matrices The inverse of a square matrix $A$, denoted $A^{-1}$, is a matrix such that: $$ A A^{-1} = A^{-1} A = I $$ Which may not exist. If $A^{-1}$ exists, then $A$ is ...
Read more

Linear Algebra: VII Orthonormal Matrices

Orthonormal Matrices A square $n$ by $n$ matrix $Q$ is called orthonormal if: $$ QQ^\top = Q^\top Q = I $$ $$ Q^{-1} = Q^\top $$ The columns $\vec{q}_1, \ldots, \vec{q}_n$ of $Q$ (and the rows) form an orthonormal basis of $\mathbb{R}^n$: $$ \vec{q}_i^\top \vec{q}_j = 0 \text{ if } i \neq j, \quad \|\vec{q}_i\| = 1 $$ An orthonormal matrix preserves inner products / dot products: $$ (Q\vec{x})^\top(Q\vec{y}) = \vec{x}^\top Q^\top Q \vec{y} = \vec{x}^\top I \vec{y} = \vec{x}^\top \vec{y} $$ An orthonormal matrix also preserves magnitude: $$ \|Q\vec{x}\|^2 = (Q\vec{x})^\top Q\vec{x} = \vec{x}^\top Q^\top Q \vec{x} = \vec{x}^\top I \vec{x} = \vec{x}^\top \vec{x} = \|\vec{x}\|^2 $$ So an orthonormal matrix preserves angles between vectors: $$ \cos{\theta} = \frac{(Q\vec{x})^\top(Q\vec{y})}{\|Q\vec{x}\|\|Q\vec{y}\|} = \frac{\vec{x}^\top \vec{y}}{\|\vec{x}\|\|\vec{y}\|} $$ There are two special types of orthogonal matrices. First, a...
Read more

Linear Algebra: VI Triangular Matrices and LU Decomposition

Triangular Matrices and LU Decomposition A square matrix $A$ is called upper triangular if: $$ a_{ij} = 0 \quad \text{for all } i > j $$ A square matrix $A$ is called lower triangular if: $$ a_{ij} = 0 \quad \text{for all } i Upper and lower triangular $n \times n$ square matrices both contain at least $$ \frac{n(n-1)}{2} $$ Zeros and $$ \frac{n(n+1)}{2} $$ Other entries. The identity matrix $I$ is both upper triangular and lower triangular. Scaling elementary matrices and their products are also both upper and lower triangular. Shearing elementary matrices are either upper triangular or lower triangular but never both. Swapping elementary matrices are neither upper nor lower triangular. Linear Properties The sum of upper triangular matrices is upper triangular, and the sum of lower triangular matrices is lower triangular. The scalar multiple of an upper triangular matrix is upper triangular, and the scalar multiple of an lowe...
Read more