Linear Algebra: II Matrices

Matrices


An $m$ by $n$ matrix is an ordered collection of $n$ $m$th dimension column vectors as well as $m$ $n$ dimensional row vectors written as:


$$ A= \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & & \vdots \\ a_{m1} & \cdots & a_{mn} \end{bmatrix} = \begin{bmatrix} \vec{c}_1 & \cdots & \vec{c}_n \end{bmatrix} \quad \text{where} \quad \vec{c}_j = \begin{bmatrix} a_{1j} \\ \vdots \\ a_{mj} \end{bmatrix} $$
$$ A= \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & & \vdots \\ a_{m1} & \cdots & a_{mn} \end{bmatrix} = \begin{bmatrix} \vec{r}_1^\top \\ \vdots \\ \vec{r}_m^\top \end{bmatrix} \quad \text{where} \quad \vec{r}_i^\top = \begin{bmatrix} a_{i1} & \cdots & a_{in} \end{bmatrix} $$

A column vector is a $n$ by $1$ matrix, a row vector is a $1$ by $n$ matrix, and a scalar is a $1$ by $1$ matrix. If $A$ is a matrix with $m$ rows and $n$ columns, then its transpose $A^\top$ is a matrix with $n$ rows and $m$ columns, switching the columns and rows so that:


$$ a^\top_{ij} = a_{ji} $$

Note that $(A^\top)^\top = A$.


Matrix addition is defined using column vectors as:


$$ A+B = \begin{bmatrix} \vec{a}_1 +\vec{b}_1 & \ldots & \vec{a}_n +\vec{b}_n \end{bmatrix} $$

Which is associative and commutative


$$ (A+B)+C=A+(B+C) $$

And commutative.


$$ A + B = B + A $$

Scalar multiplication for matrices is defined using column vectors as:


$$ cA = \begin{bmatrix} c\vec{a}_1 & \ldots & c\vec{a}_n \end{bmatrix} $$

Which is associative,


$$ c_1(c_2A) = (c_1c_2)A $$

Commutative,


$$ c_1c_2A = c_2c_1A $$

And distributive.


$$ c(A+B)=cA+cB $$

Let $A$ be an $m$ by $n$ matrix and $\vec{x}$ be an $n$ dimensional column vector or $n$ by $1$ matrix. Then matrix-vector multiplication is defined as the inner product / dot product of the rows of $A$ and $\vec{x}$:


$$ A\vec{x} = \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & & \vdots \\ a_{m1} & \cdots & a_{mn} \end{bmatrix} \begin{bmatrix} x_1 \\ \vdots \\ x_n \end{bmatrix} = \begin{bmatrix} \sum_{i=1}^n a_{1i} x_i \\ \vdots \\ \sum_{i=1}^n a_{mi} x_i \end{bmatrix} $$

This defines a linear transformation:


$$ T : \mathbb{R}^n \to \mathbb{R}^m, \quad T(\vec{x}) = A\vec{x} $$

Where for any linear combination of vectors $\vec{v}_1,\ldots,\vec{v}_k \in \mathbb{R}^n$:


$$ T(a_1 \vec{v}_1 + \ldots + a_k \vec{v}_k) = a_1 T(\vec{v}_1) + \ldots + a_k T(\vec{v}_k) \in \mathbb{R}^m $$

Matrix multiplication is defined as:


$$ AB = \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & & \vdots \\ a_{m1} & \cdots & a_{mn} \end{bmatrix} \begin{bmatrix} b_{11} & \cdots & b_{1p} \\ \vdots & & \vdots \\ b_{n1} & \cdots & b_{np} \end{bmatrix} = \begin{bmatrix} \sum_{i=1}^n a_{1i} b_{i1} & \cdots & \sum_{i=1}^n a_{1i} b_{ip} \\ \vdots & & \vdots \\ \sum_{i=1}^n a_{mi} b_{i1} & \cdots & \sum_{i=1}^n a_{mi} b_{ip} \end{bmatrix} $$

Which is associative,


$$(AB)C=A(BC)$$

Distributive,


$$A(B+C) = AB + AC$$

But not commutative.


$$AB \neq BA$$

Transpose of a matrix sum:


$$ (A + B)^\top = A^\top + B^\top $$

Transpose of a matrix product:


$$ (AB)^\top = B^\top A^\top $$

Comments

Post a Comment

Popular posts from this blog

Linear Algebra: III Square and Inverse Matrices

Square and Inverse Matrices An $n$ by $n$ square matrix has the same number of rows and columns: $$ A = \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & & \vdots \\ a_{n1} & \cdots & a_{nn} \end{bmatrix} $$ If $A$ is an $n$ by $n$ square matrix and $\vec{x} \in \mathbb{R}^n$, then the product $A\vec{x}$ is also a vector in $\mathbb{R}^n$, defining a linear transformation $T: \mathbb{R}^n \to \mathbb{R}^n$. The identity matrix $I$ is an $n$ by $n$ square matrix with 1s on the main diagonal and 0s elsewhere: $$ I = \begin{bmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & & 0 \\ \vdots & & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{bmatrix} $$ It satisfies: $$ AI = IA = A $$ The inverse of a square matrix $A$, denoted $A^{-1}$, is a matrix such that: $$ A A^{-1} = A^{-1} A = I $$ Which may not exist. If $A^{-1}$ exists, then $A$ is called invertible . If $A$ and $B$ are inve...
Read more

Linear Algebra: VII Orthonormal Matrices

Orthonormal Matrices A square $n$ by $n$ matrix $Q$ is called orthonormal if: $$ QQ^\top = Q^\top Q = I $$ $$ Q^{-1} = Q^\top $$ The columns $\vec{q}_1, \ldots, \vec{q}_n$ of $Q$ (and the rows) form an orthonormal basis of $\mathbb{R}^n$: $$ \vec{q}_i^\top \vec{q}_j = 0 \text{ if } i \neq j, \quad \|\vec{q}_i\| = 1 $$ An orthonormal matrix preserves inner products / dot products: $$ (Q\vec{x})^\top(Q\vec{y}) = \vec{x}^\top Q^\top Q \vec{y} = \vec{x}^\top I \vec{y} = \vec{x}^\top \vec{y} $$ An orthonormal matrix also preserves magnitude: $$ \|Q\vec{x}\|^2 = (Q\vec{x})^\top Q\vec{x} = \vec{x}^\top Q^\top Q \vec{x} = \vec{x}^\top I \vec{x} = \vec{x}^\top \vec{x} = \|\vec{x}\|^2 $$ So an orthonormal matrix preserves angles between vectors: $$ \cos{\theta} = \frac{(Q\vec{x})^\top(Q\vec{y})}{\|Q\vec{x}\|\|Q\vec{y}\|} = \frac{\vec{x}^\top \vec{y}}{\|\vec{x}\|\|\vec{y}\|} $$ There are two special types of orthogonal matrices. First, a...
Read more

Linear Algebra: VI Triangular Matrices and LU Decomposition

Triangular Matrices and LU Decomposition A square matrix $A$ is called upper triangular if: $$ a_{ij} = 0 \quad \text{for all } i > j $$ A square matrix $A$ is called lower triangular if: $$ a_{ij} = 0 \quad \text{for all } i Upper and lower triangular $n \times n$ square matrices both contain at least $$ \frac{n(n-1)}{2} $$ Zeros and $$ \frac{n(n+1)}{2} $$ Other entries. The identity matrix $I$ is both upper triangular and lower triangular. Scaling elementary matrices and their products are also both upper and lower triangular. Shearing elementary matrices are either upper triangular or lower triangular but never both. Swapping elementary matrices are neither upper nor lower triangular. If $A$ and $B$ are upper triangular $n \times n$ square matrices, then $AB$ is also upper triangular. Indeed, for $i > j$: $$ (ab)_{ij} = \sum_{k=1}^n a_{ik} b_{kj} $$ If $i > k$, then $a_{ik} = 0$. If $k \ge i$, then $k > j...
Read more