Understanding Matrix Eigenvalues and Eigenvectors

Understanding Matrix Eigenvalues and Eigenvectors

The fundamental property of a matrix is the transformation space, so eigenvalues describe the scaling strength in a certain direction. Eigenvectors describe that direction.

image of matrix-eigen

Pre-requisite: Understanding Matrix Determinant

Meaning #

An eigenvector is a non-zero vector \(v\) whose direction remains unchanged after being transformed by matrix \(A\), only elongated or shortened, and the scaling factor \(\lambda\) is the eigenvalue.

$$ A \cdot v = \lambda \cdot v $$

Square Matrix! #

The matrix must be a square matrix, i.e. \(n \times n\), and the eigenvector is also an \(n\)-dimensional vector. Because only when the input and output dimensions are the same can we discuss the problem of a vector not changing direction after transformation.

Eigenvalues #

According to the definition, the following derivation is obtained:

$$ A \cdot v - \lambda \cdot v = (A - \lambda \cdot I) \cdot v = 0 $$

Obtain a new homogeneous matrix \(M = A - \lambda \cdot I\) multiplied by the vector \(v\) to obtain the zero vector.

$$ M \cdot v = 0 $$

Because \(v\) is a non-zero vector, the matrix \(M\) must be a singular matrix, because only singular matrices can transform a non-zero vector into a zero vector. Then the determinant of \(M\) is zero, so the characteristic equation is obtained:

$$ \det(A - \lambda \cdot I) = 0 $$

Conversely, a non-singular matrix in \(n\) dimensions can always transform any non-zero vector in an \(n\)-dimensional space into another non-zero vector.

\(A - \lambda \cdot I\) represents subtracting the eigenvalue \(\lambda\) from the diagonal elements of matrix \(A\). For example, for a \(3 \times 3\) matrix:

$$ \lambda \cdot I = \begin{bmatrix} \lambda & 0 & 0 \\ 0 & \lambda & 0 \\ 0 & 0 & \lambda \end{bmatrix} $$

For example, for a two-dimensional matrix \(\begin{bmatrix} 4 & 1 \\ 2 & 3 \end{bmatrix}\), if the determinant is zero, then:

$$ \begin{align} \det(\begin{bmatrix} 4 - \lambda & 1 \\ 2 & 3 - \lambda \end{bmatrix}) = (4 - \lambda)(3 - \lambda) - 2 \\ = \lambda^2 - 7\lambda + 10 = 0 \end{align} $$

Thus, two eigenvalues are obtained: \(\lambda_1 = 2, \lambda_2 = 5\).

Eigenvectors #

After obtaining the eigenvalues, substitute them into the characteristic equation to obtain the eigenvectors.

$$ (A - \lambda \cdot I) \cdot v = 0 $$

First, for \(\lambda_1 = 2\), substitute to get:

$$ \begin{align} A - 2 \cdot I = \begin{bmatrix} 4 - 2 & 1 \\ 2 & 3 - 2 \end{bmatrix} = \begin{bmatrix} 2 & 1 \\ 2 & 1 \end{bmatrix} \end{align} $$

Then convert it to an augmented matrix:

$$ \begin{bmatrix} 2 & 1 \\ 2 & 1 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} $$

Which means:

$$ \begin{align} 2x + y = 0 \\ 2x + y = 0 \end{align} $$

In this direction, there are countless vectors to choose from, but generally, we choose the unit vector (i.e., \(x = 1\)), which gives \(y = -2\). So the eigenvector \(v_1\) is:

$$ \begin{bmatrix} 1 \\ -2 \end{bmatrix} $$

Similarly, substitute \(\lambda_2 = 5\) to get:

$$ \begin{align} A - 5 \cdot I = \begin{bmatrix} 4 - 5 & 1 \\ 2 & 3 - 5 \end{bmatrix} = \begin{bmatrix} -1 & 1 \\ 2 & -2 \end{bmatrix} \end{align} $$

Finally, the eigenvector \(v_2\) is obtained as: \(\begin{bmatrix} 1 \\ 1 \end{bmatrix}\)

The result shows that the “transformation ability” of matrix \(A\) is: stretching \(2\) times in the direction of \(v_1 = \begin{bmatrix} 1 \\ -2 \end{bmatrix}\) and stretching \(5\) times in the direction of \(v_2 = \begin{bmatrix} 1 \\ 1 \end{bmatrix}\).

Why are there multiple values? #

The reason why there are multiple eigenvalues and eigenvectors is that the matrix \(A\) is a two-dimensional matrix, which must have two directions of stretching. For an \(n\)-dimensional matrix, there are \(n\) eigenvalues and eigenvectors.