WEBVTT
1
00:00:00.000 --> 00:00:16.880
Okay.
2
00:00:16.880 --> 00:00:17.960
Cameras are rolling.
3
00:00:17.960 --> 00:00:24.160
This is lecture 14, starting a new chapter.
4
00:00:24.160 --> 00:00:31.160
What it means for vectors to be orthogonal.
5
00:00:31.160 --> 00:00:35.160
What it means for subspaces to be orthogonal.
6
00:00:35.160 --> 00:00:39.160
What it means for bases to be orthogonal.
7
00:00:39.160 --> 00:00:46.160
So, 90 degree, this is a 90 degree chapter.
8
00:00:46.160 --> 00:00:49.160
So, what does it mean?
9
00:00:49.160 --> 00:00:52.160
Let me jump to subspaces.
10
00:00:52.160 --> 00:00:56.160
Because I've drawn here the big picture.
11
00:00:56.160 --> 00:01:01.160
This is the 1806 picture here.
12
00:01:01.160 --> 00:01:06.160
And, hold it down, guys.
13
00:01:06.160 --> 00:01:12.160
So, this is the picture and we know a lot about that picture already.
14
00:01:12.160 --> 00:01:16.160
We know the dimension of every subspace.
15
00:01:16.160 --> 00:01:24.160
We know that these dimensions are R and N minus R.
16
00:01:24.160 --> 00:01:30.160
We know that these dimensions are R and N minus R.
17
00:01:30.160 --> 00:01:36.160
What I want to show now is what this figure is saying.
18
00:01:36.160 --> 00:01:45.160
That the angle, but the figure is like just my attempt to draw what I'm now going to say.
19
00:01:45.160 --> 00:01:52.160
That the angle between these subspaces is 90 degrees.
20
00:01:52.160 --> 00:01:56.160
And, the angle between these subspaces is 90 degrees.
21
00:01:56.160 --> 00:01:58.160
Now, I have to say what does that mean?
22
00:01:58.160 --> 00:02:03.160
What does it mean for subspaces to be orthogonal?
23
00:02:03.160 --> 00:02:09.160
But, I hope you appreciate the beauty of this picture.
24
00:02:09.160 --> 00:02:16.160
That those subspaces are going to come out to be orthogonal.
25
00:02:16.160 --> 00:02:20.160
Those two and also those two.
26
00:02:20.160 --> 00:02:27.160
So, that's like one point, one important point,
27
00:02:27.160 --> 00:02:31.160
to step forward in understanding those subspaces.
28
00:02:31.160 --> 00:02:33.160
We knew what each subspace was like.
29
00:02:33.160 --> 00:02:35.160
We could compute bases for them.
30
00:02:35.160 --> 00:02:40.160
Now, we know more, or we will in a few minutes.
31
00:02:40.160 --> 00:02:46.160
Okay, I have to say, first of all, what does it mean for two vectors to be orthogonal?
32
00:02:46.160 --> 00:02:51.160
So, let me start with that.
33
00:02:51.160 --> 00:02:59.160
Or, orthogonal vectors.
34
00:02:59.160 --> 00:03:04.160
The word orthogonal is just another word for perpendicular.
35
00:03:04.160 --> 00:03:12.160
It means that in an indimensional space, the angle between those vectors is 90 degrees.
36
00:03:12.160 --> 00:03:15.160
It means that they form a right triangle.
37
00:03:15.160 --> 00:03:21.160
It even means that the going way back to the Greeks,
38
00:03:21.160 --> 00:03:31.160
that this triangle, a vector x, a vector y,
39
00:03:31.160 --> 00:03:39.160
and the vector x plus y, of course, that will be the hypotenuse.
40
00:03:39.160 --> 00:03:45.160
So, what was it's Greeks figured out and it's neat?
41
00:03:45.160 --> 00:03:50.160
It's the fact that the, so these are orthogonal.
42
00:03:50.160 --> 00:03:59.160
This is the right angle, if, so let me put the great name down, put Sagarris.
43
00:03:59.160 --> 00:04:02.160
I'm looking for, what am I looking for?
44
00:04:02.160 --> 00:04:10.160
I'm looking for the condition, if you give me two vectors, how do I know if they're orthogonal?
45
00:04:10.160 --> 00:04:13.160
How can I tell two perpendicular vectors?
46
00:04:13.160 --> 00:04:16.160
And actually, you probably know the answer.
47
00:04:16.160 --> 00:04:18.160
Let me write the answer down.
48
00:04:18.160 --> 00:04:22.160
Or, orthogonal vectors, what's the test for orthogonality?
49
00:04:22.160 --> 00:04:32.160
I take the dot product, which I tend to write as x transpose y,
50
00:04:32.160 --> 00:04:35.160
because that's a row times a column,
51
00:04:35.160 --> 00:04:40.160
and that matrix multiplication just gives me the right thing,
52
00:04:40.160 --> 00:04:45.160
that x1, y1 plus x2, y2, and so on.
53
00:04:45.160 --> 00:04:47.160
So, these vectors are orthogonal.
54
00:04:47.160 --> 00:04:56.160
If this result x transpose y is 0, that's the test.
55
00:04:56.160 --> 00:04:57.160
Okay.
56
00:04:57.160 --> 00:05:03.160
Can I connect that to other things?
57
00:05:03.160 --> 00:05:05.160
I mean, it's like it's amazing.
58
00:05:05.160 --> 00:05:10.160
It's just beautiful that here we have, we're in end dimensions,
59
00:05:10.160 --> 00:05:14.160
we've got a couple of vectors, we want to know the angle between them,
60
00:05:14.160 --> 00:05:21.160
and the right thing to look at is the simplest thing that you can imagine the dot product.
61
00:05:21.160 --> 00:05:22.160
Okay.
62
00:05:22.160 --> 00:05:23.160
Now, why?
63
00:05:23.160 --> 00:05:25.160
So, I'm answering the question now.
64
00:05:25.160 --> 00:05:26.160
Why?
65
00:05:26.160 --> 00:05:35.160
Let's add some justification to this fact that that's the test.
66
00:05:35.160 --> 00:05:37.160
Okay.
67
00:05:37.160 --> 00:05:40.160
So, Pythagoras would say, we've got a right triangle.
68
00:05:40.160 --> 00:05:45.160
If that length squared plus that length squared equals that length squared.
69
00:05:45.160 --> 00:05:46.160
Okay.
70
00:05:46.160 --> 00:05:57.160
Can I write it as x squared plus y squared equals x plus y squared?
71
00:05:57.160 --> 00:06:01.160
Now, don't, please don't think that this is always true.
72
00:06:01.160 --> 00:06:05.160
This is only going to be true in this,
73
00:06:05.160 --> 00:06:10.160
it's going to be equivalent to orthogonality.
74
00:06:10.160 --> 00:06:13.160
For other triangles, of course, it's not true.
75
00:06:13.160 --> 00:06:15.160
For other triangles, it's not.
76
00:06:15.160 --> 00:06:21.160
But for a right triangle, somehow that fact should connect to that fact.
77
00:06:21.160 --> 00:06:26.160
Can we just make that connection?
78
00:06:26.160 --> 00:06:33.160
What's the connection between this test for orthogonality and this statement of orthogonality?
79
00:06:33.160 --> 00:06:38.160
Well, I guess I have to say, what is the length squared?
80
00:06:38.160 --> 00:06:45.160
So, let's continue on the board underneath with that equation.
81
00:06:45.160 --> 00:06:50.160
Give me another way to express the length squared of a vector.
82
00:06:50.160 --> 00:06:52.160
Let me just give you a vector.
83
00:06:52.160 --> 00:06:56.160
The vector 1, 2, 3.
84
00:06:56.160 --> 00:06:58.160
That's in three dimensions.
85
00:06:58.160 --> 00:07:06.160
What is the length squared of the vector x equals 1, 2, 3?
86
00:07:06.160 --> 00:07:09.160
So, how do you find the length squared?
87
00:07:09.160 --> 00:07:17.160
Well, really, you're just, you want the length of that vector that goes along one up to an out three,
88
00:07:17.160 --> 00:07:22.160
and we'll come back to that right triangle stuff.
89
00:07:22.160 --> 00:07:31.160
The length squared is, this is exactly x transpose x.
90
00:07:31.160 --> 00:07:37.160
Whenever I see x transpose x, I know I've got a number that's positive.
91
00:07:37.160 --> 00:07:47.160
It's a length squared, unless x happens to be the zero vector, that's the one case where the length is zero.
92
00:07:47.160 --> 00:07:54.160
So, right, this is just x1 squared plus x2 squared plus so on plus xn squared.
93
00:07:54.160 --> 00:08:02.160
So, in the example I gave you, what was the length squared of that vector 1, 2, 3?
94
00:08:02.160 --> 00:08:07.160
So, you square those, you get 1, 4, and 9, you add, you get 14.
95
00:08:07.160 --> 00:08:11.160
So, the vector 1, 2, 3 has length 14.
96
00:08:11.160 --> 00:08:15.160
So, let me just put down a vector here.
97
00:08:15.160 --> 00:08:22.160
Let x be the vector 1, 2, 3.
98
00:08:22.160 --> 00:08:28.160
Let me cook up a vector that's orthogonal to it.
99
00:08:28.160 --> 00:08:31.160
So, what's the vector that's orthogonal?
100
00:08:31.160 --> 00:08:38.160
So, right down here, x squared is 1 plus 4 plus 9, 14.
101
00:08:38.160 --> 00:08:48.160
Let me cook up a vector that's orthogonal to it, we'll get.
102
00:08:48.160 --> 00:08:51.160
Right, that's those two vectors that are orthogonal.
103
00:08:51.160 --> 00:08:59.160
The length of y squared is 5.
104
00:08:59.160 --> 00:09:09.160
And x plus y is 1 and 2 making 3, 2 and minus 1 making 1, 3 and 0 making 3.
105
00:09:09.160 --> 00:09:18.160
And the length of this squared is 9 plus 1 plus 9, 19.
106
00:09:18.160 --> 00:09:24.160
And sure enough, I haven't proved anything.
107
00:09:24.160 --> 00:09:33.160
So, let me just like check to see that my x transpose y equals 0,
108
00:09:33.160 --> 00:09:35.160
which is true, right?
109
00:09:35.160 --> 00:09:38.160
Everybody sees that x transpose y is 0 here?
110
00:09:38.160 --> 00:09:45.160
That's maybe the main point that you should get really quick at doing x transpose y.
111
00:09:45.160 --> 00:09:49.160
So, it's just this plus this plus this and that's 0.
112
00:09:49.160 --> 00:09:56.160
And sure enough, that clicks with 14 plus 5 agreeing with 19.
113
00:09:56.160 --> 00:10:00.160
Now, let me just do that in letters.
114
00:10:00.160 --> 00:10:04.160
So, that's y transpose y.
115
00:10:04.160 --> 00:10:11.160
And this is x plus y transpose x plus y.
116
00:10:11.160 --> 00:10:12.160
Okay.
117
00:10:12.160 --> 00:10:17.160
So, I'm looking again, this isn't always true, I repeat.
118
00:10:17.160 --> 00:10:21.160
This is going to be true when we have a right angle.
119
00:10:21.160 --> 00:10:26.160
And let's just, well, of course, I'm just going to simplify this up here.
120
00:10:26.160 --> 00:10:29.160
There's an x transpose x there.
121
00:10:29.160 --> 00:10:33.160
And there's a y transpose y there.
122
00:10:33.160 --> 00:10:37.160
And there's an x transpose y.
123
00:10:37.160 --> 00:10:45.160
And there's a y transpose x.
124
00:10:45.160 --> 00:10:51.160
I knew I could do that simplification because I'm just doing matrix multiplication
125
00:10:51.160 --> 00:10:53.160
and I've just followed the rules.
126
00:10:53.160 --> 00:10:54.160
Okay.
127
00:10:54.160 --> 00:10:59.160
So, x transpose x is cancel, y transpose y is cancel.
128
00:10:59.160 --> 00:11:01.160
And what about these guys?
129
00:11:01.160 --> 00:11:09.160
What can you tell me about the inner product of x with y and the inner product of y with x?
130
00:11:09.160 --> 00:11:12.160
Is there a difference?
131
00:11:12.160 --> 00:11:20.160
I think if we, what we're doing real vectors, which is all we're doing now,
132
00:11:20.160 --> 00:11:22.160
there isn't a difference.
133
00:11:22.160 --> 00:11:23.160
There's no difference.
134
00:11:23.160 --> 00:11:27.160
If I take x transpose y, that'll give me zero.
135
00:11:27.160 --> 00:11:35.160
If I took y transpose x, I would have the same x1, y1, and x2, y2, and x3, y3.
136
00:11:35.160 --> 00:11:37.160
It would be the same.
137
00:11:37.160 --> 00:11:40.160
So, this is the same as that.
138
00:11:40.160 --> 00:11:47.160
This is really, I'll knock that guy out and say, two of these.
139
00:11:47.160 --> 00:11:55.160
So, actually, this equation boiled down to this thing being zero.
140
00:11:55.160 --> 00:11:56.160
Right?
141
00:11:56.160 --> 00:11:58.160
Everything else canceled.
142
00:11:58.160 --> 00:12:00.160
And this equation boiled down to that one.
143
00:12:00.160 --> 00:12:03.160
So, that's really all I wanted.
144
00:12:03.160 --> 00:12:14.160
I just wanted to check that, Pythagoras, for a right triangle, led me to this,
145
00:12:14.160 --> 00:12:22.160
of course, I cancel the two now, no problem, to x transpose y is zero as the 10.
146
00:12:22.160 --> 00:12:23.160
Fair enough.
147
00:12:23.160 --> 00:12:24.160
Okay.
148
00:12:24.160 --> 00:12:26.160
You knew it was coming.
149
00:12:26.160 --> 00:12:30.160
The dot product of orthogonal vectors is zero.
150
00:12:30.160 --> 00:12:33.160
It's just, I just want to say, that's really neat.
151
00:12:33.160 --> 00:12:36.160
It comes, that it comes out so well.
152
00:12:36.160 --> 00:12:38.160
All right.
153
00:12:38.160 --> 00:12:44.160
Now, what about, so now I know if two vectors, what it means when two vectors are orthogonal.
154
00:12:44.160 --> 00:12:49.160
By the way, what about if one of these guys is the zero vector?
155
00:12:49.160 --> 00:12:55.160
Suppose x is the zero vector, and y is whatever.
156
00:12:55.160 --> 00:12:58.160
Are they orthogonal?
157
00:12:58.160 --> 00:13:01.160
Sure.
158
00:13:01.160 --> 00:13:05.160
In math, the one thing about math is you're supposed to follow the rules.
159
00:13:05.160 --> 00:13:13.160
So, you're supposed to, if x is the zero vector, you're supposed to take the zero vector dotted with y.
160
00:13:13.160 --> 00:13:16.160
And of course, you get always get zero.
161
00:13:16.160 --> 00:13:24.160
So, just so we're all sure, the zero vector is orthogonal to everybody.
162
00:13:24.160 --> 00:13:36.160
But, what I want to, what I now want to speak about is subspace.
163
00:13:36.160 --> 00:13:44.160
What does it mean for me to say that some subspace is orthogonal to some other subspace?
164
00:13:44.160 --> 00:13:45.160
So, okay.
165
00:13:45.160 --> 00:13:47.160
Now, I've got to write this down.
166
00:13:47.160 --> 00:14:06.160
We're defining definition of subspace f is orthogonal, so, to subspace, let's say, t.
167
00:14:06.160 --> 00:14:11.160
So, I've got a couple of subspaces.
168
00:14:11.160 --> 00:14:15.160
And what should it mean for those guys to be orthogonal?
169
00:14:15.160 --> 00:14:23.160
It's just a sort of, what's the natural extension from orthogonal vectors to orthogonal subspaces?
170
00:14:23.160 --> 00:14:37.160
Well, and in particular, let's think of some orthogonal subspaces like this wall.
171
00:14:37.160 --> 00:14:40.160
Let's say in three dimensions.
172
00:14:40.160 --> 00:14:45.160
So, the blackboard extended to infinity, right?
173
00:14:45.160 --> 00:14:51.160
Is a subspace, a plane, a two-dimensional subspace?
174
00:14:51.160 --> 00:14:58.160
It's a little bumpy, but anyway, let's think of it as a subspace.
175
00:14:58.160 --> 00:15:05.160
Let me take the floor as another subspace.
176
00:15:05.160 --> 00:15:18.160
Again, it's not a great subspace, and I see you only built it like so-so.
177
00:15:18.160 --> 00:15:22.160
And I'll put the origin right here.
178
00:15:22.160 --> 00:15:27.160
So, the origin of the world is right there.
179
00:15:27.160 --> 00:15:30.160
Okay.
180
00:15:30.160 --> 00:15:35.160
Thereby giving linear algebra its proper importance in this.
181
00:15:35.160 --> 00:15:36.160
Okay.
182
00:15:36.160 --> 00:15:46.160
So, there's one subspace, there's another one the floor, and are they orthogonal?
183
00:15:46.160 --> 00:15:49.160
What does it mean for two subspaces to be orthogonal?
184
00:15:49.160 --> 00:15:52.160
And in that special case, are they orthogonal?
185
00:15:52.160 --> 00:15:53.160
All right.
186
00:15:53.160 --> 00:15:55.160
Let's finish this sentence.
187
00:15:55.160 --> 00:15:56.160
What does it mean?
188
00:15:56.160 --> 00:16:04.160
We have to know what we're talking about here.
189
00:16:04.160 --> 00:16:09.160
So, what would be a reasonable idea of orthogonal?
190
00:16:09.160 --> 00:16:12.160
Well, let me put the right thing up.
191
00:16:12.160 --> 00:16:25.160
It means that every vector in F, every vector in F is orthogonal.
192
00:16:25.160 --> 00:16:30.160
Two.
193
00:16:30.160 --> 00:16:31.160
Two.
194
00:16:31.160 --> 00:16:34.160
What am I going to say?
195
00:16:34.160 --> 00:16:42.160
Every vector in T.
196
00:16:42.160 --> 00:16:51.160
That's a reasonable, and it's a good, and it's the right definition for two subspaces to be orthogonal.
197
00:16:51.160 --> 00:16:54.160
But I just want you to say, hey, what does that mean?
198
00:16:54.160 --> 00:17:01.160
So, answer the question about the blackboard and the floor.
199
00:17:01.160 --> 00:17:05.160
Are those two subspaces, they're two-dimensional, right?
200
00:17:05.160 --> 00:17:18.160
And we're in R3, it's like the FB plane or something, and the XY plane.
201
00:17:18.160 --> 00:17:21.160
Are they orthogonal?
202
00:17:21.160 --> 00:17:26.160
Is that every vector in the blackboard or orthogonal to every vector in the floor?
203
00:17:26.160 --> 00:17:31.160
Starting from the origin right there.
204
00:17:31.160 --> 00:17:33.160
Yes or no?
205
00:17:33.160 --> 00:17:36.160
I could take a vote.
206
00:17:36.160 --> 00:17:39.160
I'll begin with some notes.
207
00:17:39.160 --> 00:17:40.160
No is the answer.
208
00:17:40.160 --> 00:17:42.160
They're not.
209
00:17:42.160 --> 00:17:55.160
You can tell me a vector in the blackboard and a vector in the floor that are not orthogonal.
210
00:17:55.160 --> 00:17:59.160
Well, you can tell me quite a few, I guess.
211
00:17:59.160 --> 00:18:11.160
Maybe, like I could take some 45-degree guy in the blackboard and some thing in the floor.
212
00:18:11.160 --> 00:18:13.160
But there aren't as 90 degrees, right?
213
00:18:13.160 --> 00:18:22.160
In fact, even more, you could tell me a vector that in both the blackboard plane and the floor plane,
214
00:18:22.160 --> 00:18:25.160
so it's certainly not orthogonal to itself.
215
00:18:25.160 --> 00:18:30.160
So, for sure, those two planes aren't orthogonal.
216
00:18:30.160 --> 00:18:31.160
What would that mean?
217
00:18:31.160 --> 00:18:37.160
There was a vector that's in the blackboard in both of those planes.
218
00:18:37.160 --> 00:18:42.160
This guy running along the track there, where in the intersection?
219
00:18:42.160 --> 00:18:44.160
The intersection.
220
00:18:44.160 --> 00:18:47.160
A vector, look, it's two sub-spaces.
221
00:18:47.160 --> 00:18:49.160
We need to sum vector.
222
00:18:49.160 --> 00:18:59.160
Well, then, for sure, they're not orthogonal because that vector is in one and it's in the other and it's not orthogonal to itself.
223
00:18:59.160 --> 00:19:00.160
Unless it's bigger.
224
00:19:00.160 --> 00:19:07.160
So, the only, let me tell you what's orthogonal.
225
00:19:07.160 --> 00:19:09.160
For me, it's a two sub-spaces orthogonal.
226
00:19:09.160 --> 00:19:19.160
First of all, I'm certainly saying that they don't intersect in any non-zero vector.
227
00:19:19.160 --> 00:19:24.160
But also, I mean, more than that, not the intersect because it's good enough.
228
00:19:24.160 --> 00:19:34.160
For example, oh, let's say in the plane, oh, well, when do we have orthogonal sub-spaces in the plane?
229
00:19:34.160 --> 00:19:36.160
Yeah, tell me in the plane.
230
00:19:36.160 --> 00:19:39.160
So, we don't there aren't that any different sub-spaces in the plane.
231
00:19:39.160 --> 00:19:45.160
What do we have in the plane as possible, some spaces?
232
00:19:45.160 --> 00:19:53.160
There's zero vector here, real small, aligned through the origin, or the whole plane.
233
00:19:53.160 --> 00:19:55.160
Okay.
234
00:19:55.160 --> 00:20:04.160
Now, so when is aligned through the origin orthogonal to the whole plane?
235
00:20:04.160 --> 00:20:05.160
Nope.
236
00:20:05.160 --> 00:20:06.160
Right?
237
00:20:06.160 --> 00:20:07.160
Never.
238
00:20:07.160 --> 00:20:13.160
When is aligned through the origin orthogonal to the zero sub-spaces?
239
00:20:13.160 --> 00:20:15.160
Oh, right.
240
00:20:15.160 --> 00:20:20.160
When is aligned through the origin orthogonal to a different line through the origin?
241
00:20:20.160 --> 00:20:24.160
Well, that's the case that we all have a clear picture of.
242
00:20:24.160 --> 00:20:27.160
The two lines have to meet at 90 degrees.
243
00:20:27.160 --> 00:20:29.160
They have only the...
244
00:20:29.160 --> 00:20:33.160
So, that's like this simple case I'm talking about.
245
00:20:33.160 --> 00:20:36.160
There's one sub-space, there's the other sub-space.
246
00:20:36.160 --> 00:20:39.160
They only meet at zero, and they're orthogonal.
247
00:20:39.160 --> 00:20:40.160
Okay.
248
00:20:40.160 --> 00:20:47.160
Now, so we now know what it means for two sub-spaces to be orthogonal.
249
00:20:47.160 --> 00:20:53.160
And now I want to say that this is true for the row space and the null space.
250
00:20:53.160 --> 00:20:54.160
Okay.
251
00:20:54.160 --> 00:20:58.160
So that's the neat fact.
252
00:20:58.160 --> 00:21:10.160
So, row space is orthogonal to the null space.
253
00:21:10.160 --> 00:21:17.160
Now, how did I come up with that?
254
00:21:17.160 --> 00:21:19.160
But you see the light is great.
255
00:21:19.160 --> 00:21:24.160
And that means that these sub-spaces are just the right things.
256
00:21:24.160 --> 00:21:32.160
They're just cutting the whole space up into two perpendicular sub-spaces.
257
00:21:32.160 --> 00:21:33.160
Okay.
258
00:21:33.160 --> 00:21:38.160
So what?
259
00:21:38.160 --> 00:21:42.160
Well, what have I got to work with?
260
00:21:42.160 --> 00:21:45.160
All I know is the null space.
261
00:21:45.160 --> 00:21:51.160
The null space has vector that's all AX equals zero.
262
00:21:51.160 --> 00:22:01.160
So this is a guy X, X is in the null space, then AX is zero.
263
00:22:01.160 --> 00:22:08.160
So why is it orthogonal to the rows of A?
264
00:22:08.160 --> 00:22:13.160
If I write down AX equals zero, which is all I know about the null space,
265
00:22:13.160 --> 00:22:18.160
then I guess I want you to see that that's telling me,
266
00:22:18.160 --> 00:22:23.160
just that equation right there is telling me that the rows of A,
267
00:22:23.160 --> 00:22:24.160
let me write it out.
268
00:22:24.160 --> 00:22:34.160
There's row one of A, row two, row M.
269
00:22:34.160 --> 00:22:35.160
Okay.
270
00:22:35.160 --> 00:22:37.160
That's A.
271
00:22:37.160 --> 00:22:45.160
And it's multiplying X and it's producing zero.
272
00:22:45.160 --> 00:22:46.160
Okay.
273
00:22:46.160 --> 00:22:52.160
Written out that way, you'll see it.
274
00:22:52.160 --> 00:22:59.160
So I'm saying that a vector in the row space is perpendicular to this guy X in the null space.
275
00:22:59.160 --> 00:23:02.160
And you see what?
276
00:23:02.160 --> 00:23:11.160
Because this equation is telling you that row one of A, multiplying, that's a dot product, right?
277
00:23:11.160 --> 00:23:16.160
Row one of A dot product with this X is producing this zero.
278
00:23:16.160 --> 00:23:21.160
So X is orthogonal to the first row.
279
00:23:21.160 --> 00:23:25.160
And to the second row, row two of A, X is giving that zero.
280
00:23:25.160 --> 00:23:28.160
Row M of A, X is giving that zero.
281
00:23:28.160 --> 00:23:30.160
So X is zero.
282
00:23:30.160 --> 00:23:36.160
The equation is telling me that X is orthogonal to all the rows.
283
00:23:36.160 --> 00:23:37.160
Right?
284
00:23:37.160 --> 00:23:39.160
It's just sitting there.
285
00:23:39.160 --> 00:23:44.160
It had to be sitting there because we didn't know anything more about the null space than this.
286
00:23:44.160 --> 00:23:57.160
And now I guess to be totally complete here, I've now checked that X is orthogonal to each separate row.
287
00:23:57.160 --> 00:24:04.160
But what else, strictly speaking, do I have to do?
288
00:24:04.160 --> 00:24:13.160
To show that some space is orthogonal, I have to take this X in the null space and show that it's orthogonal to every.
289
00:24:13.160 --> 00:24:17.160
Vector in the row space, every vector in the row space.
290
00:24:17.160 --> 00:24:20.160
So what else is in the row space?
291
00:24:20.160 --> 00:24:24.160
This row is in the row space, that row is in the row space.
292
00:24:24.160 --> 00:24:28.160
They're all there, but it's not the whole row space, right?
293
00:24:28.160 --> 00:24:30.160
We just have to like remember.
294
00:24:30.160 --> 00:24:31.160
What does it mean?
295
00:24:31.160 --> 00:24:34.160
What does that word space tell?
296
00:24:34.160 --> 00:24:39.160
And what else is in the row space?
297
00:24:39.160 --> 00:24:46.160
Besides the rows, all there, and the common.
298
00:24:46.160 --> 00:24:54.160
So I really have to check that sure enough, if X is perpendicular to the row one, row two, all the different separate rows,
299
00:24:54.160 --> 00:24:59.160
then also X is perpendicular to what combination of the rows.
300
00:24:59.160 --> 00:25:02.160
And that's just matrix multiplication again.
301
00:25:02.160 --> 00:25:09.160
You know, I have row one, transpose X is zero.
302
00:25:09.160 --> 00:25:16.160
So on row two, transpose X is zero.
303
00:25:16.160 --> 00:25:23.160
So I'm entitled to multiply that by some C1, this by some C2, I still have zero.
304
00:25:23.160 --> 00:25:25.160
I'm entitled to add.
305
00:25:25.160 --> 00:25:39.160
So I have C1, row one, so all this, when I put that together, that big parentheses, C1, row one, plus C2, row two, and so on.
306
00:25:39.160 --> 00:25:43.160
Transpose X is zero.
307
00:25:43.160 --> 00:25:51.160
Right? I just added the zero, and got zero, and I just added these following the rules.
308
00:25:51.160 --> 00:25:57.160
No, no big deal, the whole point was right sitting in that.
309
00:25:57.160 --> 00:26:00.160
Okay.
310
00:26:00.160 --> 00:26:16.160
So if I come back to this figure now, I'm like a happier person, because I have this, I now see how those sub-faces are oriented.
311
00:26:16.160 --> 00:26:25.160
And these sub-faces are also oriented. Well, actually, why is that orthogonality?
312
00:26:25.160 --> 00:26:31.160
Well, it's the same statement for A transpose, that that one was for A.
313
00:26:31.160 --> 00:26:45.160
So I won't take time to prove it again, because we've checked it for every matrix, and A transpose is just as good a matrix as A, so we're orthogonal over there.
314
00:26:45.160 --> 00:27:07.160
So we really have carved out this, this, this, this was like carving up and dimensional space in the two sub-faces, and this one was carving up and dimensional space in the two sub-spaces.
315
00:27:07.160 --> 00:27:16.160
And, well, one more thing here. One more important thing.
316
00:27:16.160 --> 00:27:21.160
Let me move in the three dimensions.
317
00:27:21.160 --> 00:27:36.160
Tell me a couple of orthogonal sub-faces in three dimensions that somehow don't carve up the whole space or stuff like that.
318
00:27:36.160 --> 00:27:40.160
I'm thinking of a couple of orthogonal lines.
319
00:27:40.160 --> 00:27:51.160
If I, suppose I'm in three dimensions, R3, and I have one line, one, one dimensional sub-space, and a perpendicular one.
320
00:27:51.160 --> 00:27:59.160
Could those be the row space and the null space? Could those be the row space and the null space?
321
00:27:59.160 --> 00:28:11.160
Could I, could I be in three dimensions and have a row space that's aligned and a null space that's aligned?
322
00:28:11.160 --> 00:28:19.160
No. Why not? Because the dimensions aren't right, right? The dimensions are no good.
323
00:28:19.160 --> 00:28:40.160
The dimensions here are an n minus r. They add up to three. They add up to n. If I take, just, if I just follow that example, if, if, if the row space is one dimensional, suppose a is, what's a good?
324
00:28:40.160 --> 00:28:49.160
I'm in r3. I want a one dimensional row space. Let me take one, two, five, two, four, ten.
325
00:28:49.160 --> 00:28:54.160
What's the dimension of that row space?
326
00:28:54.160 --> 00:29:01.160
One. What's the dimension of the null space?
327
00:29:01.160 --> 00:29:12.160
What's the null space look like in that case? The row space is aligned, right? One dimensional. It's aligned through one, two, five.
328
00:29:12.160 --> 00:29:21.160
See, it's basically what's the row space look like? It's a, what's the dimension?
329
00:29:21.160 --> 00:29:42.160
So, so here, r here, n is three. The rank is one. So, the dimension of the null space, I'm looking at this x, x1, x2, x3, together with zero.
330
00:29:42.160 --> 00:29:58.160
So, the dimension of the null space is, we all know it, two. Right, it's a plane. And now, actually, we know we see better what plane is it. What plane is it?
331
00:29:58.160 --> 00:30:10.160
It's the plane that's perpendicular to one, two, five. Right? We now see, in fact, the two, four, ten didn't actually have any effect at all.
332
00:30:10.160 --> 00:30:23.160
I could just ignore that. That didn't change the row space or the null space. I'll just make that one equation. Yeah, okay, sure.
333
00:30:23.160 --> 00:30:43.160
That's the easiest to deal with. One equation, three unknown. And I want to add what does the equation give me? It gives me the null space.
334
00:30:43.160 --> 00:31:03.160
And you would have said, actually, in September, you would have said, it gives you a plane. And we're completely right. And the plane that gives you, the normal vector, remembering calculus, don't normal vector, oh, n. Well, there it is, one, two, five.
335
00:31:03.160 --> 00:31:23.160
Okay. So, what is the point I want to make here? I want to emphasize that not only are the, let me write it in words.
336
00:31:23.160 --> 00:31:49.160
So, I want to write the null space. And the row space are, or does not. That's the neat back, which we just checked from AX equals zero.
337
00:31:49.160 --> 00:32:03.160
But now, I want to say, I want to say more because in a little more of its true. There are dimensions, add to the whole space. So, that's like a little extra information.
338
00:32:03.160 --> 00:32:21.160
It's not like I couldn't have a line and a line in three dimensions. Those don't add up one and one don't add to three. So, I use the word orthogonal complement.
339
00:32:21.160 --> 00:32:41.160
And the idea of this word complement is that the orthogonal complement of a row space contains not the sum vectors that are orthogonal, so it's like all.
340
00:32:41.160 --> 00:33:01.160
So, what does that mean? That means that the null space contains all, not just the sum, but all vectors that are perpendicular to the row space.
341
00:33:01.160 --> 00:33:21.160
Really, what I've done in this pack of the lecture is just notice some of the nice geometry that we didn't pick up before because we didn't discuss perpendicular vectors before.
342
00:33:21.160 --> 00:33:36.160
But it was all sitting there and now we picked it up that these vectors are orthogonal complement. And I guess I wouldn't call this part one of the fundamental theorem of linear algebra.
343
00:33:36.160 --> 00:33:51.160
The fundamental theorem of linear algebra is about these four sub spaces. So part one is about their dimension. Maybe I should call it part two now. They're dimensions we got.
344
00:33:51.160 --> 00:34:07.160
Now we're getting their orthogonality at part two and part three will be about basic orthogonal basis. So that coming up.
345
00:34:07.160 --> 00:34:26.160
So I'm happy with that geometry right now. Okay. Okay. Now what's my next goal in this chapter? Here's the main problem of the chapter.
346
00:34:26.160 --> 00:34:50.160
The main problem of the chapter is, so this is coming, coming at fraction. This is the very last chapter that's about A, A, B.
347
00:34:50.160 --> 00:35:07.160
I would like to solve that system of equation when there is no solution. You may say what a ridiculous thing to do, but I have to say it's done all the time.
348
00:35:07.160 --> 00:35:31.160
In fact, it has to be done. You get, so the problem is solve as the best possible. So I'll put a quote. Hey, I think it will be when there is no solution.
349
00:35:31.160 --> 00:35:55.160
And of course, what does that mean? B is in the, in the total. And it's quite typical. If this matrix A is rectangular, if I maybe I have M equation and that bigger than the number of unknown, then for sure the rank is not M.
350
00:35:55.160 --> 00:36:13.160
The rank couldn't be M now. So there will be a lot of right hand sides with no solution. But it's here that you can. Some satellite is buzzing along. You measure it position.
351
00:36:13.160 --> 00:36:33.160
You make a thousand. So that gives you a thousand equations for the, for the parameters that, that just gives a position. But there are a thousand parameters. There's just maybe six or you're measuring.
352
00:36:33.160 --> 00:36:51.160
You're doing question error. You're measuring. You're taking a poll. You're measuring somebody's pulse rate. Okay, there's just one unknown, the post rate.
353
00:36:51.160 --> 00:37:09.160
So you measure it once. Okay, fine. But if you really want to know it, you measure it multiple times. But then the measurements have no reason. So there's the problem is that in many, many problems, we got too many equations.
354
00:37:09.160 --> 00:37:28.160
And I got noise in the right hand side. So A, they will be. I can't expect to solve exactly right. Because I don't even know what the error is. There's error. There's the measurement mistake in B. But there's information too.
355
00:37:28.160 --> 00:37:48.160
There's a lot of information about that. And what I want to do is like separate the noise that's jumped from the information. And so this is a straightforward linear out of a problem. How do I solve? What's the best solution?
356
00:37:48.160 --> 00:38:15.160
Okay, now let me. I want to say so that's like describes the problem in algebraic way. I've got some equations. I'm looking for the best solution. Well, one way to find it is one, one way to find eight solution is to roll away.
357
00:38:15.160 --> 00:38:32.160
Equations until you've got a nice square invertible system and solve that. That's not satisfactory. There's no reason in these measurements to say these measurements are perfect and these measurements are used.
358
00:38:32.160 --> 00:38:41.160
We want to use all the measurements to get the best information to get the maximum information. But that's okay.
359
00:38:41.160 --> 00:39:04.160
Let me anticipate a matrix that's going to show up. This A is typically rectangle. But a matrix that shows up whenever you have and we've chapter three was all about rectangle and we know when this is all for you to do elimination on it.
360
00:39:04.160 --> 00:39:17.160
But I'm thinking, hey, you do elimination and you get equation zero equals other non zero. I'm thinking we really elimination is something.
361
00:39:17.160 --> 00:39:29.160
So that's our question. Elimination will get us down to. We'll tell it if there is a solution or not, but I'm not thinking not.
362
00:39:29.160 --> 00:39:40.160
Okay, so what are we going to do? All right. I want to I want to tell you to jump ahead to the matrix that will play a key role.
363
00:39:40.160 --> 00:39:55.160
So this is the matrix that you want to understand for this chapter four. And it's the matrix A transpose A.
364
00:39:55.160 --> 00:40:12.160
Tell me something about that. So A is this and by and matrix rectangular. But now I'm saying that's a good matrix that shows up in the end is a transpose A.
365
00:40:12.160 --> 00:40:29.160
So tell me something about that. Is it? Yeah, tell me what's the first thing you know about a transpose is square right? Square because this is and by and and this is and by and.
366
00:40:29.160 --> 00:40:45.160
So this is the result is and by and good square. What else? It's symmetric. Good. It's symmetric.
367
00:40:45.160 --> 00:41:05.160
But you remember how to do that if we transpose that matrix. Let's transpose it. A transpose A. If I transpose it. Then that comes first transpose. This comes second transpose.
368
00:41:05.160 --> 00:41:21.160
And then transposing twice is leaves it brings it back to the same. So it's the matrix. Good. Now we now know how to ask more about a matrix.
369
00:41:21.160 --> 00:41:38.160
I'm interested in. Is it invertible? If not what's it's no space? So I want to know about because you're going to see well let me let me even.
370
00:41:38.160 --> 00:41:57.160
Well I shouldn't do this but I will let me tell you what equation to solve. When you can solve that. The good equation comes from multiplying both sides by a transpose.
371
00:41:57.160 --> 00:42:16.160
So the good equation that you get to is this one a transpose a x equals a transpose b. That will be the central equation in the chapter.
372
00:42:16.160 --> 00:42:41.160
So I think why not let me admit it right away. I have to I should really give X. I want to sort of indicate that this is. I mean this was the solution to that equation if it existed but probably didn't.
373
00:42:41.160 --> 00:42:57.160
Now let me give this a different name X half. Because I'm hoping this one will have a solution. And I'm saying that it's my best solution.
374
00:42:57.160 --> 00:43:16.160
I have to say what is best me. But that's going to be my my plan. I'm going to say that's the best solution. So this equation so you see right away why I'm so interested in this matrix a transpose a and in it invertibility.
375
00:43:16.160 --> 00:43:30.160
OK now when is it invertible. OK let me take a take let me just do an example and then.
376
00:43:30.160 --> 00:43:56.160
I'll just pick a matrix here. Just so we see what a transpose a looks like. So let me take a matrix a 111 125 just invent a matrix. So there's a matrix a notice that it has m equals three rows and n equals two columns.
377
00:43:56.160 --> 00:44:02.160
It's ranked is. The rank of that matrix is.
378
00:44:02.160 --> 00:44:14.160
Two right. Yeah the columns are independent does a X equal B. If I look at a X equal B so X is just X 1 X to.
379
00:44:14.160 --> 00:44:33.160
And B is B 1 B 2 B 3. Do I expect to solve a X equal B what what's no way right I mean linear algebra is great but solving three equations with only two unknowns usually we can't do it.
380
00:44:33.160 --> 00:44:54.160
We can only solve it if this vector B is what I can solve that equation if that vector B 1 B 2 B 3 is in the column space it is a combination of those columns and fun but usually it won't be.
381
00:44:54.160 --> 00:45:15.160
So the combination is just fill up a plane and most vectors aren't on that plane. So what I'm saying is that I'm going to work with the matrix a transpose a and I just want to figure out in this example what a transpose a here.
382
00:45:15.160 --> 00:45:25.160
So it's two by two the first entry is a three the next entry is an eight this entry is a.
383
00:45:25.160 --> 00:45:36.160
What's that entry. Eight for sure we do it had to be a miss entry is.
384
00:45:36.160 --> 00:45:48.160
Now 30 is that right. And is that matrix invertible there's an eight transpose a.
385
00:45:48.160 --> 00:45:59.160
And it is invertible right three eight is not a multiple of eight thirty and and it's invertible and that's the normal that's what I expect.
386
00:45:59.160 --> 00:46:17.160
So this is what I want to I want to show it so here's the final here's the key point the no space of a transpose a it's not going to be always invertible.
387
00:46:17.160 --> 00:46:37.160
Tell me of the matrix. Yeah now I have to say that I can't say a transpose a always invertible because that has been too much I mean what could the matrix a be for example so that a transpose a was not invertible well even could be the zero.
388
00:46:37.160 --> 00:46:54.160
I mean that's like stream case the pose I make it rank. Suppose I change to that a.
389
00:46:54.160 --> 00:47:02.160
Now I figure out a transpose a again and I get what do I get.
390
00:47:02.160 --> 00:47:15.160
I get nine I get nine of course and here I get. What's that entry 20 set.
391
00:47:15.160 --> 00:47:29.160
And is that matrix invertible no. And why do I I know it wouldn't be a vertical anyway because this matrix only has range.
392
00:47:29.160 --> 00:47:39.160
What if I have a product of matrices of rank one that product is not going to have a rank figure than one.
393
00:47:39.160 --> 00:47:45.160
So I'm not surprised that the answer only has one and that's what I.
394
00:47:45.160 --> 00:48:10.160
So all that always happens that the rank of a transpose a come up to equal the rank of a. So yeah so the no space of a transpose a equals the no space of a the rank of a transpose a equals the rank of a.
395
00:48:10.160 --> 00:48:39.160
So let's I'll tell you in the as soon as I can why that's true but let's draw from that what the fact that I want this tells me that this square symmetric matrix is invertible if so here's my conclusion a transpose a is invertible.
396
00:48:39.160 --> 00:49:08.160
If exactly when exactly this no space is only going to zero which means the columns of a are independent so a transpose a is invertible exactly a has independent.
397
00:49:08.160 --> 00:49:37.160
Color that's the that's the fact that I need about a transpose a and then you'll see next time how a transpose a enters everything next next lecture is actually a crucial one here I'm preparing for it by getting us thinking about a transpose a.
398
00:49:37.160 --> 00:49:42.160
And it's ranked is the same as a rank of a and we can decide when it's in.
399
00:49:42.160 --> 00:50:10.160
Okay so see you for a.