A matrix transpose is just swapping rows and 
columns, right? You take a matrix A, turn its columns into rows, then the new matrix is called 
A transpose, but there is way more to it than just moving things around. In fact, we can define the 
transpose of a linear map without ever referring to the matrix at all. A deeper understanding of 
how transpose came about will help us visualise it, and along the way, we will come across 
deeper topics like covectors, duality, and so on. We will first go through a very general 
big picture of what transpose really is, then try to visualise something called covectors, 
then transpose, giving some examples; and finally, a very subtle but also important point 
that links to special relativity. [CHAPTER 1: THE BIG PICTURE] The starting point is the concept 
of a covector, or dual, or 1-form,

because in different contexts, different names 
are used, but I’m going with “covectors” here. Covectors are basically machines that will eat a 
vector, and output a number. I like to call this machine a measuring device - it “measures the 
vector”. These are special devices in that they are “linear”. That means if you have two vectors 
v_1 and v_2, and they are measured by this machine to be a_1 and a_2 respectively, then this machine 
will measure v_1 + v_2 to be a_1 + a_2; and if you instead scale the vector v_1 by a number lambda, 
then it will be measured to be lambda a_1. Now let’s say we have a linear transformation,
alpha, transforming a vector v to another vector w. To emphasise that v and w can be very 
different, let’s say v is a 2-dimensional vector,

and w could be a 3-dimensional one. And 
for some reason, you want to measure w. A bit of a silly analogy would be: 
you want to measure your own weight, and the vector is you here. But you don’t 
have a bathroom scale, and you have to use your neighbour's. So one way to do it would be 
to go to your neighbour’s house, then take the measurement there directly. But you could also 
just call the neighbour to lend you the scale, so that you can take the measurements at home, and 
get the same answer. The scale being moved from the neighbour’s house to yours is the transpose 
of alpha. So alpha is you going to the neighbour’s house, and alpha transpose is moving the 
measuring device from the neighbour’s to yours. But how is this related to the matrix transpose 
that you know? A matrix is, at its core, only a summary of a linear map. For a 2-dimensional 
linear map, for instance, the first column tells

you where the vector (1,0) goes; and the second column tells you where the vector (0,1) goes. If we then want to know where an arbitrary vector (x,y) goes, just multiply the first column by x, and add to y times the second column! Just knowing these four numbers is enough to determine the entire linear map, so the matrix is essentially the summary of the linear map. So you have the linear transformations alpha and its transpose, and also the two matrices that are summaries of the transformations. Because the matrix on the right summarises the transpose of alpha, it makes sense to call it A^T. It just turns out that the summary matrices of alpha and alpha transpose can be obtained by swapping rows and columns of each other. The most important point here, though, is the big picture: a transpose is just transforming some measurement device in other spaces back to that in the original.

In what remains, we will, first of all, try to visualise those covectors, and then the transpose. [CHAPTER 2: VISUALISING COVECTORS] Covectors are linear, so just like our linear transformations, in the 2-dimensional case, it suffices to know what (1,0) and (0,1) are measured. If the measurements are a and b respectively, for our purpose, let’s say this is an (a, b) covector. Now if you take any vector (x, y), which is just x times (1,0) plus y times (0,1), just by linearity, it will be measured as x * a + y * b. So in the 2-dimensional case, an (a, b) covector measures the vector (x, y) to be xa + yb. This is just a function of two variables.

A natural way to visualize this would be to plot a 3-dimensional graph, with the z-coordinate being the function value at (x, y). In this case, it is just a giant plane passing through the origin. However, because we might want to consider a 3-dimensional covector later on, we use another slightly roundabout way of visualizing such a function. Instead, we consider its level sets. That means you plot all the points (x, y), such that it gets measured as a number c. In this specific case, we plot the line graph ax + by = c, where a is 2, and b is 3, so this represents a (2,3)-covector. By varying c, we get a glimpse of how this function behaves. In this case, it just moves up and down. We label this line to be 1, because that’s all the points that will be measured to be 1. And

we only draw a line graph for, say, integer 
values of c, so we have parallel lines at regular intervals. Of course, this is just 
one covector. Different covectors will have different sets of parallel lines with different 
directions and densities. But now the question is, given a particular set of parallel lines, can we 
quickly spot which covector this is? In this case, it is the (-1, -0.5)-covector, so let’s compare 
with the (-1, -0.5) vector and see what happens. The corresponding vector is perpendicular to all 
the lines. So ok, given these parallel lines, all we know is that the corresponding vector 
lies on this perpendicular line, but why is this exact vector, then? If we scale this vector, we 
notice that the longer the vector, the denser the parallel lines are. So we expect the length to be 
proportional to the density of the parallel lines.

More quantitatively, if we have the gap size between the lines, then the density of the lines should be 1/gap size - how many gaps could you fit in 1 unit. And indeed, the length of the vector is exactly 1/gap size. Ok, so you know the length, but these two vectors have the same length, how do you determine which one? It’s actually impossible to tell unless you label back those lines. This line, for example, was labelled 2, because they are all the points where the covector would measure them to be 2. With these labels, we can tell these two vectors apart. The corresponding vector points towards positive labels. So here is a summary of how you know which covector you have just by looking at these lines. First off, you know the corresponding vector is perpendicular to the lines, so the vector must be on this white line. You take a look at the gap size

between the lines, then you know it’s one of 
these two vectors whose length is 1/gap size. Then you take a look at the hidden labels of 
the lines, and you choose the one pointing towards the positive labels. Finally, you read 
off the coordinates of the corresponding vector. So we have visualised 2-dimensional covectors 
using parallel lines. 3-dimensional covectors are similar, except we have parallel planes 
instead. The corresponding vector would still be perpendicular to the planes, the length 
of the vector would still be the density, or 1/gap size of those parallel planes, and the 
vector still points towards the positive labels. Now that we have visualised covectors, 
we are ready for the transpose. [CHAPTER 3: VISUALISING TRANSPOSE] As we have seen in the big picture of transpose, 
alpha transpose transforms a measuring device, a covector, in the other space back to the 
original space. So we transform a covector,

to another covector with the transpose. 
The trouble is that 1 covector already occupies quite a lot of space. Two covectors 
appearing together is already the maximum. So we can only visualise the transformation of two 
covectors at once. Which two covectors should we choose? Just like normal linear transformations, 
the two representatives are the (1,0)-covector, as well as the (0,1)-covector.
Like a normal vector transformation, we observe where the (1,0)-covector goes. 
Whichever covector it becomes would be our first column of the transpose matrix. Similarly, we 
observe the transformation of the (0,1)-covector. So for a covector transformation, we can similarly 
write down a matrix representation. The first column tells you where the (1,0)-covector 
goes, and the second column tells you where the (0,1)-covector goes. In what follows, we 
will use the right shear as an example. What

should the transpose of a right shear look like? 
We will need to go back to this big picture. In this specific case, alpha is a right shear, 
and we now just have a 2D to 2D transformation. Now, this (1,0)-covector measures the sheared 
vector. And the job of the transpose is to find out which covector could we use to directly 
measure the original vector, such that we get the same measurement at the end. Now go back 
to the visuals: a right shear would transform the basis vectors as such. Next, we see how the 
(1,0)-covector measures our sheared vectors. In this case, the measurements are both 1. But rather 
than thinking about their exact measurements, just notice the tips all lie on this line. 
Remember that each line represents the tips of those vectors whose measurements are the same 
fixed constant c, so in this visualisation,

which line the tip of the vectors are on, tells you the measurement of the vectors. This is why the transpose does this: first off, the basis vectors have gone back to their original place. And the tips of both vectors still lie on the bold line, which means that you get the same measurements as before. This is exactly what a transpose should do! Anyway, let’s see what the transpose does to the (0,1)-covector instead. Again, we do the right shear to the basis vectors, then use the (0,1)-covector to measure them. This time, their measurements are different. I use the bold pink line to indicate the measurement of the pink vector, and the bold green line to indicate the measurement of the green vector. For the transpose, we want to bring back the basis vectors, and retain the measurements. This time, we don’t need to do anything to the covector, because the tip of the pink vector still lies

on the pink line, and the tip of the green 
vector still lies on the green line. Their measurements are not changed when the 
vectors go back to their original place. So altogether, this is the situation after you 
have done the right shear to your basis vectors. The transpose wants the basis vectors to go 
back to the original place, while retaining the measurements, and so it essentially does a left 
shear to the whole picture. Wait… originally, we have a right shear, and somehow the 
transpose is a left shear? While it *looks* like a left shear, the gridlines are actually 
made up of two separate, different covectors. The original right shear has the matrix (1,1; 
0,1), because the (1,0) vector stays put, and it is still (1,0) after the shear, but the 
(0,1) vector has sheared to the (1,1) vector.

Compare this with the transpose. This is the (1,0)-covector. The so-called “left shear” transforms such a covector to this one. Which covector is this? We know the corresponding vector lies on this perpendicular line, then we need to figure out the gap size. With the help of this vector telling us how long 1 unit is, the gap size can be calculated to be 1/sqrt (2). So because the length of the corresponding vector is 1/gap size, the length is now sqrt(2). This leaves us with two choices: either a (1,1) or a (-1,-1) covector. Because originally, this bold line is measured positively by this (1,0)-covector, after the transformation, this bold line indicates the positive direction, and so it must be the (1,1)-covector. On the other hand, the (0,1)-covector is unaffected by the left shear.

so it stays a (0,1)-covector. So this so-called “left shear” should be represented by this matrix, because the first column tells us that the (1,0)-covector has transformed to this (1,1)-covector, and the (0,1)-covector has just stayed put, still being the (0,1)-covector. Now compare this matrix of this so-called “left shear” with the original right shear. You can see that the matrices are indeed just swapping rows and columns of each other! We will have more examples later on, but this hopefully convinces you that transpose *looks like* inverse, but you have to treat those gridlines as covectors instead. The first of the two other examples I would give would be a diagonal matrix, such as the one with entries 3, 2 on the diagonal. The first column just tells us to enlarge the pink vector by a

factor of 3, and the second column tells us to enlarge the green vector by a factor of 2. As before, we bring on the covectors, and do the inverse transformation to get the basis vectors to where it started, but when viewed as covectors, this is actually a (3,0)-covector, because the gap size has now reduced to 1/3 the original size. The length of the corresponding vector is actually 3. Exactly the same story for the other covector, except the gap size is just halved, so the length of the corresponding vector is 2, so we end up with a (0,2)-covector. This is why this transformation is still represented by the same matrix, because we view those gridlines as covectors, and indeed, if you do the swapping rows and columns thing, you still get the same matrix! Another example would be a rotation. Again, we start off with these basis vectors, and let’s

say we rotate anticlockwise by 60 degrees.
This is our linear transformation. As usual, we bring on the covector grid, and do the inverse transformation to get the basis vectors to where it started. This time, we rotate clockwise by 60 degrees. What is this covector then? Well, it is from rotating this original covector clockwise by 60 degrees, and this is the original corresponding vector. During the rotation, the corresponding vector rotates the same way, clockwise 60 degrees. This is because the gap size hasn’t changed just by rotation, so we don’t need to change the length of the corresponding vector. The same story for the other covector. So in this case, the transpose doesn’t just “look like” inverse, it *is* the inverse when acting on the corresponding vector. That means, if I write down the matrix corresponding to anticlockwise 60 degree rotation,

and the corresponding transpose, in this case, 
are inverses of each other. In other words, for rotation, A^T = A^(-1). In general, 
matrices satisfying this are called orthogonal, so rotation matrices are orthogonal. By now, you 
might be thinking, this visualisation depends on having an inverse, and yes, indeed, there is a bit 
of limitation when it comes to singular matrices, and in general, in technical terms, the rank is 
smaller than the dimension of your original space, but those are all the cases 
where this visualisation fails. And even in those cases, you can still sort 
of displace your linear maps a tiny little bit to do the visualisation, so this really 
is a powerful visualisation for transposes. [CHAPTER 4: SUBTLETIES] I want to emphasise a very subtle and technical 
but important point. The orthogonality condition

A^T = A^(-1) should not have made any 
sense to you. On the left hand side, the transpose is supposed to transform 
covectors to covectors, but on the right, A^(-1) transforms vectors to vectors. 
They aren’t even the same type of object, how can they be equal? Well, the trick 
is to pair up vectors and covectors, treating them sort of the same, which 
is what we have done in this video. For this (a,b)-covector, where we measure (1,0) 
as a, (0,1) as b, we have associated with it the vector (a,b), or in the visualisation of covectors 
as a bunch of parallel lines, we have associated it with the vector perpendicular to the lines, and 
length being the density of the lines. Of course, we can do the other way around as well - given 
the vector, we can draw those parallel lines,

and given the vector (a, b), we can 
also easily construct that covector. That’s how we have achieved the 1-1 
correspondence between vectors and covectors. A technical remark is that this has to be a 
linear correspondence, which our pairing is, but we don’t have to go into the details 
of what linear means in this context. Let’s list a bunch of vectors and covectors 
that we have. If a transformation transforms a covector epsilon, to another covector eta; 
and epsilon is paired up with the vector v, eta with the vector w, then it looks 
*as if* v is being transformed to w. This covector-to-covector transformation 
does not even know the existence of v, but because of this pairing, we can treat this as 
a vector-to-vector transformation. In a similar way, if a vector-to-vector transformation 
transforms v to w, we can think of it as

transforming from epsilon to eta. It is in this 
sense that we say something like A^T = A^(-1). There are two reasons why I want to 
emphasise the process of pairing up. First, while this way of identifying a vector 
with a covector is very standard and very natural, this is not the only way of doing it. You 
could have also paired up with the vector (-a, b) instead, or (a, -b). These will still yield a 
1-1 correspondence. There are many other options, but I’m choosing these two simply because either 
of these two will give you the mathematics of special relativity in 1+1 dimensions, and any 
matrix that fulfils this orthogonality condition, under this pairing, are the Lorentz 
transformations. Don’t worry if any 
of these doesn’t make sense - I’ll probably make 
videos on special and general relativity later on.

If you know something about relativity and are 
wondering why this has anything to do with it, note that the act of going from vectors to 
covectors is the job of the metric tensor, or metric for short; and going from covectors to 
vectors is the job of the inverse metric. Anyway, the point is, it is very important 
to specify which 1-1 correspondence, or the metric you are using, because 
different metrics yield different mathematics. The other thing is you might not even be able 
to pair those up. In infinite-dimensional vector spaces, there are WAY more covectors than vectors, 
so you can never achieve a 1-1 correspondence. If you still insist on trying to define 
orthogonality, then you need to restrict the kind of covectors you consider, and hope that those 
special covectors have a 1-1 correspondence with the vectors. This will be better explained with 
the concept of an adjoint, which I have put some

resources in the description. This adjoint concept 
is the bread and butter of quantum mechanics. So in this video, we have the big picture of the 
transpose, and used it to visualise transpose with a few examples like shear, as you see here, 
as well as diagonal matrices and rotations, deducing that transposes do look like inverses, 
but we have to treat those gridlines as covectors. By the way, I’m still collecting questions 
for the 100k subs Q and A, so if you want to ask any questions, this is the last chance 
to ask, because if everything goes well, next video will be it. But please do not ask any 
animation related questions anymore, because there are already many questions like that, and I will 
definitely answer that - don’t worry. In any case, thanks for the patrons, subscribe, hit the 
bell, like, comment, share. See you next time!

The deeper meaning of matrix transpose

Hi, I am Trevor. This channel discusses maths (or physics) concepts that I am passionate about, and tackles them in a novel perspective. For example, most people introduced Green's functions with lots of DE concepts, but my video on the topic did so from electrostatics, providing a vivid image of point charges. The new perspectives are not meant to be "easier" to understand, and I have never intended the videos to be pedagogical; the videos are just some interesting concepts and arguments you have not heard before on YouTube (maybe anywhere). 

For those who are wondering, I don't use coding to produce my videos, and while stylistically similar to 3Blue1Brown, I don't use Manim (Grant's mathematical animation Python library) to produce my videos. I actually use PowerPoint, GeoGebra and (sometimes) Mathematica to produce my videos.
