5. Matrix Operations#
A short summary of the properties of some matrix operations.
5.1. Transpose#
Suppose \(\mathbf{A}\) and \(\mathbf{B}\) are matrices and \(\mathbf{x}\) is a vector.
\((\mathbf{A} + \mathbf{B})^T = \mathbf{A}^T + \mathbf{B}^T\)
\((\mathbf{A}^T)^T = \mathbf{A}\)
\((\mathbf{A x})^T = \mathbf{x}^T \mathbf{A}^T\)
\((\mathbf{x}_1^T \mathbf{A} \mathbf{x}_2)^T\) = \(\mathbf{x}_1^T \mathbf{A}^T \mathbf{x}_2\)
5.2. Transpose Proofs#
\((\mathbf{A} + \mathbf{B})^T = \mathbf{A}^T + \mathbf{B}^T\)
Let \(\mathbf{A}_i\) and \(\mathbf{B}_i\) be the \(i\)-th column of \(\mathbf{A}\) and \(\mathbf{B}\) respectively.
\[\begin{split}\newcommand{\vertbar}{\rule[-1ex]{1pt}{2.5ex}}
\newcommand{\horzbar}{\rule[.5ex]{2.5ex}{1pt}}
(\mathbf{A} + \mathbf{B})^T
= &
\left(
\begin{bmatrix}
\vertbar & \vertbar & \dots & \vertbar \\
\mathbf{A}_1 & \mathbf{A}_2 & \dots & \mathbf{A}_n \\
\vertbar & \vertbar & \dots & \vertbar \\
\end{bmatrix}
+
\begin{bmatrix}
\vertbar & \vertbar &\dots & \vertbar \\
\mathbf{B}_1 & \mathbf{B}_2 & \dots & \mathbf{B}_n \\
\vertbar & \vertbar & \dots & \vertbar \\
\end{bmatrix}
\right)^T \\
= &
\begin{bmatrix}
\vertbar & \vertbar & \dots & \vertbar \\
\mathbf{A}_1 + \mathbf{B}_1 & \mathbf{A}_2 + \mathbf{B}_2 & \dots & \mathbf{A}_n + \mathbf{B}_n \\
\vertbar & \vertbar & \dots & \vertbar \\
\end{bmatrix}^T \\
= &
\begin{bmatrix}
\horzbar & \mathbf{A}_1 + \mathbf{B}_1 & \horzbar \\
\horzbar & \mathbf{A}_2 + \mathbf{B}_2 & \horzbar \\
\vdots & \vdots & \vdots \\
\horzbar & \mathbf{A}_n + \mathbf{B}_n & \horzbar \\
\end{bmatrix}^T.\end{split}\]
And,
\[\begin{split}\mathbf{A}^T + \mathbf{B}^T
= &
\begin{bmatrix}
\horzbar & \mathbf{A}_1 & \horzbar \\
\horzbar & \mathbf{A}_2 & \horzbar \\
\vdots & \vdots & \vdots \\
\horzbar & \mathbf{A}_n & \horzbar \\
\end{bmatrix}^T.
+
\begin{bmatrix}
\horzbar & \mathbf{B}_1 & \horzbar \\
\horzbar & \mathbf{B}_2 & \horzbar \\
\vdots & \vdots & \vdots \\
\horzbar & \mathbf{B}_n & \horzbar \\
\end{bmatrix}^T.
=
\begin{bmatrix}
\horzbar & \mathbf{A}_1 + \mathbf{B}_1 & \horzbar \\
\horzbar & \mathbf{A}_2 + \mathbf{B}_2 & \horzbar \\
\vdots & \vdots & \vdots \\
\horzbar & \mathbf{A}_n + \mathbf{B}_n & \horzbar \\
\end{bmatrix}^T.\end{split}\]
End of proof.
\((\mathbf{A}^T)^T = \mathbf{A}\)
\[\begin{split}(A^T)^T =
\begin{bmatrix}
\horzbar & \mathbf{A}_1 & \horzbar \\
\horzbar & \mathbf{A}_2 & \horzbar \\
\vdots & \vdots & \vdots \\
\horzbar & \mathbf{A}_n & \horzbar \\
\end{bmatrix}^T
=
\begin{bmatrix}
\vertbar & \vertbar & \dots & \vertbar \\
\mathbf{A}_1 & \mathbf{A}_2 & \dots & \mathbf{A}_n \\
\vertbar & \vertbar & \dots & \vertbar \\
\end{bmatrix}
= \mathbf{A}\end{split}\]
End of proof.
\((\mathbf{A} \mathbf{x})^T = \mathbf{x}^T \mathbf{A}^T\)
Let \(\mathbf{A}_i\) be the \(i\)-th row.
\[\begin{split}(\mathbf{A} \mathbf{x})^T
=
\begin{bmatrix}
\mathbf{A}_1 \mathbf{x} \\
\mathbf{A}_2 \mathbf{x} \\
\vdots \\
\mathbf{A}_n \mathbf{x} \\
\end{bmatrix}^T
=
\begin{bmatrix}
\mathbf{A}_1 \mathbf{x} &
\mathbf{A}_2 \mathbf{x} &
\dots &
\mathbf{A}_n \mathbf{x}
\end{bmatrix}\end{split}\]
And,
\[\begin{split}\mathbf{x}^T \mathbf{A}^T
=
\mathbf{x}^T
\begin{bmatrix}
\vertbar & \vertbar & \dots & \vertbar \\
\mathbf{A}_1 & \mathbf{A}_2 & \dots & \mathbf{A}_n \\
\vertbar & \vertbar & \dots & \vertbar \\
\end{bmatrix}
=
\begin{bmatrix}
\mathbf{A}_1 \mathbf{x} &
\mathbf{A}_2 \mathbf{x} &
\dots &
\mathbf{A}_n \mathbf{x}
\end{bmatrix}\end{split}\]
End of proof.
\(\mathbf{x}_1^T \mathbf{A} \mathbf{x}_2\) = \(\mathbf{x}_2^T \mathbf{A} \mathbf{x}_1\)
Let \(\mathbf{A}_i\) be the \(i\)-th row of \(\mathbf{A}\).
\[\begin{split}\begin{bmatrix}
\horzbar & \mathbf{x}_1 & \horzbar
\end{bmatrix}
\begin{bmatrix}
\horzbar & \mathbf{A}_1 & \horzbar \\
\horzbar & \mathbf{A}_2 & \horzbar \\
\vdots & \vdots & \vdots \\
\horzbar & \mathbf{A}_n & \horzbar \\
\end{bmatrix}
\begin{bmatrix}
\vertbar \\ \mathbf{x}_2 \\ \vertbar
\end{bmatrix}
& =
\begin{bmatrix}
\horzbar & \mathbf{x}_1 & \horzbar
\end{bmatrix}
\begin{bmatrix}
\mathbf{A}_1 \mathbf{x_2} \\
\mathbf{A}_2 \mathbf{x_2} \\
\vdots \\
\mathbf{A}_n \mathbf{x_2} \\
\end{bmatrix} \\
& =
\begin{bmatrix}
\mathbf{A}_1 \mathbf{x_2} &
\mathbf{A}_2 \mathbf{x_2} &
\dots &
\mathbf{A}_n \mathbf{x_2} &
\end{bmatrix}
\begin{bmatrix}
\vertbar \\ \mathbf{x}_1 \\ \vertbar
\end{bmatrix} \\
& =
\begin{bmatrix}
\horzbar & \mathbf{x}_2 & \horzbar
\end{bmatrix}
\begin{bmatrix}
\vertbar & \vertbar & \dots & \vertbar \\
\mathbf{A}_1 & \mathbf{A}_2 & \dots & \mathbf{A}_n \\
\vertbar & \vertbar & \dots & \vertbar \\
\end{bmatrix}
\begin{bmatrix}
\vertbar \\ \mathbf{x}_1 \\ \vertbar
\end{bmatrix} \\
& =
\mathbf{x}_2^T \mathbf{A} \mathbf{x}_1\end{split}\]