Math is Hard

Monoid

monoid=magma+associativity+identity

Magma

A magma is a set M matched with an operation • that sends any two elements a, b ∈ M to another element, a • b ∈ M, i.e. $(M,\cdot)$ is a magma, if

$$\forall a,b \in M, a \cdot b \in M$$

Determinant of 3x3 matrix and its derivative

$$A_{3\times3}(t)=[v_1(t)\ v_2(t)\ v_3(t)]\\ \mathrm{det}(A(t))=(v_1(t)\times v_2(t))\cdot v_3(t)\\$$

By applying Leibniz rule for vector and dot product,

$$\frac{\mathrm{d}}{\mathrm{d}t}\mathrm{det}A(t)=\mathrm{det}[\dot{v}_1(t)\ v_2(t)\ v_3(t)]+\mathrm{det}[v_1(t)\ \dot{v}_2(t)\ v_3(t)]+\mathrm{det}[v_1(t)\ v_2(t)\ \dot{v}_3(t)]$$

Hessian Matrix

Wiki

$$H_{i,j}=\frac{\partial ^2 f}{\partial x_i \partial x_j}$$ $$H=\nabla ^2 f$$

Determinant

Properties:

Property 1

Let $A\in R^{n\times n}$

$$det(A)=det(A^T) \\ det(A)=(-1)^n det(-A)$$

$-1$ can be replaced by any number $c$

Property 2

Multiply one column (or one row) by $k$,

$$\left| \begin{matrix}{l} a_{11}& a_{12}& \cdots& a_{1n}\\ \vdots& \vdots& \ddots& \vdots\\ ka_{i1}& ka_{i2}& \cdots& ka_{in}\\ \vdots& \vdots& \ddots& \vdots\\ a_{n1}& a_{n2}& \cdots& a_{nn}\\ \end{matrix} \right|=k\left| \begin{matrix}{l} a_{11}& a_{12}& \cdots& a_{1n}\\ \vdots& \vdots& \ddots& \vdots\\ a_{i1}& a_{i2}& \cdots& a_{in}\\ \vdots& \vdots& \ddots& \vdots\\ a_{n1}& a_{n2}& \cdots& a_{nn}\\ \end{matrix} \right|$$

The determinant of the new matrix is k times the determinant of the original matrix.

Properties about Cross Product

$$a\cdot \left( b\times c \right) =b\cdot \left( c\times a \right) =c\cdot \left( a\times b \right) \\ a\times a=0 \\ (a\times b) \cdot a =0 \\ a\times \left( b\times c \right) +b\times \left( c\times a \right) +c\times \left( a\times b \right) =0 \\ \left( a\times b \right) \times \left( a\times c \right) =\left( a\cdot \left( b\times c \right) \right) a$$

For $a,b,c,d\in \mathbb{R}^3$:

$$a\times \left( b\times c \right) =\left( a\cdot c \right) b-\left( a\cdot c \right) c \\ (a\times b)\times (a\times c)=(a\cdot (b\times c))a=det(a,b,c)a$$

which is also know as the vector triple product.

$$(a\times b)\cdot(c\times d)=(a\cdot c)(b\cdot d)-(a\cdot d)(b\cdot c)$$

whose proof is given here:

Property of Orthonormal Matrix

$$Q^TQ=I\\ Q^T=Q^{-1}$$

Linear Transformation In Transformed Basis

$$v'=P^{-1}APv$$

where $P$ is the basis of the transformed basis (new basis) in the standard basis (our basis) and $A$ is the transformation in the standard basis.

Rotation Matrix

$$R=\left[ \begin{matrix} \cos \theta& -\sin \theta& 0\\ \sin \theta& \cos \theta& 0\\ 0& 0& 1\\ \end{matrix} \right]$$

rotates points in the $xy$ plane counterclockwise through an angle $\theta$.

Judge whether a matrix is a rotation matrix

A matrix is a rotation matrix iff $R^T=R^{-1}$ i.e. $R^TR=I$ and $detR=1$

Calculate the axis of the rotation matrix

The axis of the rotation matrix stays unchanged after rotation, i.e. $Rx=x$, $(R-I)x=0$

Solving the equation and we get the axis of the rotation.

Calculate the rotation angle of the rotation matrix

Choose a random vector, usually $v=(1,0,0,…,0)$. Therefor, $w=Rv$, i.e. $v$ after rotation, is the actually the first column of $R$ (or you can just do the calculate).

Let $a$ be the vector representing the axis of the rotation.

We can use $v’=v-\frac{aa^T}{a^Ta}v$ to get the vector perpendicular to the axis while pointing from the axis to the end of $v$. Similarly, use $w’=w-\frac{aa^T}{a^Ta}w$ to get the vector perpendicular to the axis while pointing from the axis t othe end of $w$.

It’s easy to find that $Span(v’,w’)$ is perpendicular to the axis $a$. Therefor, the angle between $v’$ and $w’$ is the angle of the rotation, i.e. $\arccos(\frac{}{||v’||||w’||})$

QR Factorization

Reference: this video and this video(better, starts at 25:30)

$$q_3=\frac{C}{||C||} \\ C=c-\frac{A^T c}{A^T A}A-\frac{B^T c}{B^T B}B$$

Nullity of a Matrix

Basically, the dimension of the null space.

Null Space / Kernel of a Matrix

Reference: this video and this video(talks about how to calc)

The null space of $A$ is all solution $x$ (a vector), s.t. $Ax=0$. Often denoted as $N(A)$ or $ker(A)$

Rank of a Matrix

Basically, the dimension of the column space.

Reference: this page

It tell you how to calc the rank of a matrix.

Projection Matrix

Reference: this page

Use Ctrl+F to search for the Projection Matrix part, in which you can find how to solve projection matrix onto certain space.

A projection matrix onto a col space of $Q$ is:

$$P=Q(Q^TQ)^{-1}Q^T$$

Specifically, if $Q$ is a square orthonormal matrix, then $P=I$