Lie algebra
6. Lie groups and Lie algebra
We have already seen - but not formally acknowledged - that the set of all rotations in a n-dimensional space forms a group. Simply said, the product of any two rotation matrices is also a rotation matrix, matrix multiplication (and hence composition of rotations) is associative, for every rotation matrix there is an inverse rotation matrix, and the identity matrix is a rotation matrix. This group is called the special orthogonal group and is denoted by \( SO(n) \).
Another interesting property about this set of rotations is that one can find two elements that are arbitrarily close to each other. You can imagine the elements of this set as points in a space. The points are so close to each other that they form a surface (or a manifold). Now we will move on this smooth manifold, which is also a group, and try to gain some insights from it. A group which is also a smooth manifold (which admittedly was not rigorously defined) is called a Lie group. Thus the set of all rotations in a n-dimensional space is a Lie group.
6.1 First approach - taking a small step on this manifold
Let's say you are at the identity matrix in this manifold. You take a small step in the direction of a matrix \( R \). You reach a point \( I + \epsilon R \) where \( \epsilon \) is a small number. Since \( I + \epsilon R \) is a rotation matrix, we can see that:
\( (I + \epsilon R)^T(I + \epsilon R) = I \)
\( \Rightarrow I + \epsilon (R + R^T) + \epsilon ^{2} R^TR = I \)
\( \Rightarrow R + R^T = 0 \) since \( \epsilon ^{2} ≈ 0 \)
Thus, any small step \( R \) on this manifold should satisfy \( R^T = -R \). In other words, \( R \) is a skew-symmetric matrix.
6.2 Lie algebra for 2D Rotations
In 2D, any skew-symmatric matrix can be written as \( \begin{bmatrix} 0 & -\theta \\ \theta & 0 \end{bmatrix} \) which can be re-written as \( \theta \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} = \theta S \) where \( S = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} \). Now, a small step in the direction of this matrix can be represented as \( I + \frac{1}{N} \theta \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} = I + \frac{\theta S}{N} \) where \( N \) is a large number. If we take a large number ( = \( N \)) of such steps, the resulting point (or rotation matrix) will be \( (I + \frac{\theta S}{N})^N \). As \( N \) approaches infinity, this will be equal to \( e^{R_{\theta}} = e^{\theta \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}} \). Now note that:Thus we can write the exponential of the skew-symmetric matrix as:
\( S^2 = -I \text{ and } S^3 = -S \)
\( e^{\theta S} = I + \theta S + \frac{\theta^2 S^2}{2!} + \frac{\theta^3 S^3}{3!} + ... = (cos \theta) I + (sin \theta) S = \begin{bmatrix} cos \theta & -sin \theta \\ sin \theta & cos \theta \end{bmatrix} \)
This is the matrix representation of a 2D rotation by an angle \( θ \). The key insight here is how we were able to interpret any element of 2D rotation as an exponential of a skew-symmetric matrix, which was an outcome of taking small steps on the Lie group of 2D rotations.import numpy as np
from scipy.linalg import expm, logm
angle = np.pi/4
base_skew = np.array([[0, -1], [1, 0]])
skew = base_skew * angle
# same as np.array([[np.cos(angle), -np.sin(angle)], [np.sin(angle), np.cos(angle)]])
print(expm(skew))
'''
Output:
array([[ 0.70710678, -0.70710678],
[ 0.70710678, 0.70710678]])
'''
# it can also be written as cos(angle) * I + sin(angle) * skew_symmetric_matrix
R = np.cos(angle) * np.eye(2) + np.sin(angle) * base_skew
print(R)
'''
Output:
array([[ 0.70710678, -0.70710678],
[ 0.70710678, 0.70710678]])
'''
# logm takes a matrix and returns the skew-symmetric matrix
print(logm(R))
'''
Output: (equal to angle * base_skew up to floating point errors)
array([[-2.84558859e-16, -7.85398163e-01],
[ 7.85398163e-01, -4.85080479e-17]])
'''
6.3 Second approach - differentiating the constraint \( R^TR = I \)
Imagine moving on the Lie group of rotations. At time \( t = 0 \), you are at the identity matrix. At any given time, you are at a point given by the rotation matrix \( R(t) \). The constraint that you are always on the Lie group of rotations is given by \( R(t)^TR(t) = I \). For simplicity, we write \( R(t) = R \). Since \( R^TR = I \), we can differentiate this equation with respect to \( t \):Thus we have \( R^T \dot{R} = S \Rightarrow \dot{R} = RS \Rightarrow \frac{dR}{dt} = RS \). This is a differential equation with the initial condition of \( R(0) = I \). The solution to this equation is: \( R = e^S \).
\( \frac{d}{dt} (R^TR) = 0 \)
\( \Rightarrow \frac{dR^T}{dt} R + R^T \frac{dR}{dt} = 0 \)
\( \Rightarrow \dot{R}^T R + R^T \dot{R} = 0 \) where \( \dot{R} = \frac{dR}{dt} \)
\( \Rightarrow (R^T \dot{R})^T + R^T \dot{R} = 0 \)
\( \Rightarrow R^T \dot{R} = - (R^T \dot{R})^T \)
\( \Rightarrow R^T \dot{R} \) is a skew-symmetric matrix \( S \)
We arrived at the same conclusion as before - that any element of the Lie group of rotations can be represented as an exponential of a skew-symmetric matrix.
6.4 Lie algebra for 3D Rotations
Now we look at how to create general skew-symmatric matrices for 3D rotations. We can write any skew-symmetric matrix as:Here \( S_x, S_y, S_z \) can be considered the basis vectors for the Lie algebra of 3D rotations. It can be shown that it is isomorphic to the \( R^3 \) vector space. In other words, an element of the 3D vector space can be converted into a skew-symmetric matrix which can be exponentiated to get a rotation matrix.
\(
S = \begin{bmatrix} 0 & -z & y \\ z & 0 & -x \\ -y & x & 0 \end{bmatrix}
= x \begin{bmatrix} 0 & 0 & 0 \\ 0 & 0 & -1 \\ 0 & 1 & 0 \end{bmatrix} +
y \begin{bmatrix} 0 & 0 & 1 \\ 0 & 0 & 0 \\ -1 & 0 & 0 \end{bmatrix} +
z \begin{bmatrix} 0 & -1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}
= xS_x + yS_y + zS_z
\)
\(
\begin{bmatrix}
x \\ y \\ z
\end{bmatrix}
\)\( \xrightarrow{\text{hat map}} \)\(
\begin{bmatrix}
0 & -z & y \\ z & 0 & -x \\ -y & x & 0
\end{bmatrix}
\)\( \xrightarrow{\text{exponential map}} \)\( \text{Rotation matrix} \)
Vector space → Lie algebra → Lie manifold
def hat_map(x:float, y:float, z:float) -> np.ndarray:
return np.array([[0, -z, y], [z, 0, -x], [-y, x, 0]])
def exponential_map(skew_symmatric_matrix:np.ndarray) -> np.ndarray:
return expm(skew_symmatric_matrix)
# a very small rotation around the x-axis
print(exponential_map(hat_map(0.01, 0, 0)))
'''
Output:
[[ 1. 0. 0. ]
[ 0. 0.99995 -0.00999983]
[ 0. 0.00999983 0.99995 ]]
'''
There is a lot more to be said about Lie algebras but it may require another series of articles.