Matrices gone Brrr.
Matrix Multiplication
Matrix multiplication, in essense is kind of powering the entire AI landscape in modern era, the tokens, images and siri voice are fundamentally these huge matrix multiplications operations happening at scale as a part of various mathematical circuitary. Not to discount other components but matrix multiplication is key in training most models. 3D graphics rendering engines in essence employ matrix multiplication to power cool realistic 3D effects leveraging high performance GPUs.
I’ve been wanting to dive into 3D graphics and writing my own shaders for a while. TBH my unfinished blender projects from last month are embarassing me at the moment. So I guess brushing up on plain old matrix multiplication lessons wouldn’t harm . In no way, brushing up on linear algebra will spark any creativity for other projects but always good to keep up with it. So my expectation is basically rewiring my brain to keep matrices and operations in mind while working on AI training, graphics or who knows GPU kernels in future.
So what is it?
Matrices are mostly these random numbers spread across big square brackets looking like a mesh of numbers.
\[Matrix_a = \begin{bmatrix} 1 & 2 & 3\\ 4 & 5 & 6\\ 7 & 8 & 9 \\ \end{bmatrix}\]is a $3 \times 3$ matrix . Where we use the matrix to denote the variable coefficients while solving system of linear equations.
Ok, but why do we need to multiply these monstrocities with anything? What does it even mean to multiply two matrices or with a vector?
Before diving into why to multiply, a better question would be, what does a matrix denote? Is it a number? set of numbers? or a just a low pixel ghibili art? It turns out, that matrices are mere representation in grid form. This rectangular mesh of numbers can represent anything ranging from pixel values to representing a graph state.
And now mathematically speaking it’s not the numbers but what they do in general.
Started with manim. This sucks .
