WEBVTT
1
00:00:00.000 --> 00:00:19.680
Okay guys, we're almost ready to make this lecture immortal.
2
00:00:19.680 --> 00:00:20.680
Okay.
3
00:00:20.680 --> 00:00:21.680
Are we on?
4
00:00:21.680 --> 00:00:22.680
All right.
5
00:00:22.680 --> 00:00:23.680
This is an important lecture.
6
00:00:23.680 --> 00:00:31.520
It's about projection.
7
00:00:31.520 --> 00:00:41.040
And let me start by just projecting a vector B down on a vector A. So you see what the geometry
8
00:00:41.040 --> 00:00:45.600
looks like in the two dimensions.
9
00:00:45.600 --> 00:00:51.640
I'd like to find the point along with line.
10
00:00:51.640 --> 00:00:55.240
So that line through A is a one dimensional subspace.
11
00:00:55.240 --> 00:00:57.560
So I'm starting with one dimension.
12
00:00:57.560 --> 00:01:04.200
I'd like to find the point on that line closest to B. And I just take that problem first
13
00:01:04.200 --> 00:01:11.040
and then I'll explain why I want to do it and why I want to project on other subspaces.
14
00:01:11.040 --> 00:01:16.400
So where is the point closest to B that's on that line?
15
00:01:16.400 --> 00:01:19.080
It's somewhere there.
16
00:01:19.080 --> 00:01:24.880
And let me connect that.
17
00:01:24.880 --> 00:01:27.640
And what's the whole point of my picture now?
18
00:01:27.640 --> 00:01:31.240
What's the where does orthogonality come into this picture?
19
00:01:31.240 --> 00:01:40.320
The whole point is that this best point, that's the projection piece, of B onto the line
20
00:01:40.320 --> 00:01:43.880
where's orthogonality?
21
00:01:43.880 --> 00:01:48.200
It's the fact that that's the right angle.
22
00:01:48.200 --> 00:01:53.600
That this, the error, this is like how much I'm wrong about.
23
00:01:53.600 --> 00:02:01.760
This is the difference between B and P. The whole point is that that's perpendicular to
24
00:02:01.760 --> 00:02:05.400
A. That's going to give us the equation.
25
00:02:05.400 --> 00:02:11.040
That's got to tell us that's the one fact we know that's got to tell us where that projection
26
00:02:11.040 --> 00:02:12.040
is.
27
00:02:12.040 --> 00:02:18.560
Let me also say, look, I've drawn a triangle there.
28
00:02:18.560 --> 00:02:27.040
So if we were doing trigonometry, we would do like we would have angle theta and distances
29
00:02:27.040 --> 00:02:30.480
that would involve sine theta and cos theta.
30
00:02:30.480 --> 00:02:34.960
That leads to lousy formulas compared to linear algebra.
31
00:02:34.960 --> 00:02:40.160
The formula that we want comes out nicely.
32
00:02:40.160 --> 00:02:43.960
And what do we know?
33
00:02:43.960 --> 00:02:50.800
We know that T, this projection, is some multiple of A, right?
34
00:02:50.800 --> 00:02:52.400
It's on that line.
35
00:02:52.400 --> 00:02:55.960
So we know it's in that one dimensional sub space.
36
00:02:55.960 --> 00:03:01.080
It's some multiple, let me call that multiple X of A.
37
00:03:01.080 --> 00:03:05.240
So really it's that number X I'd like to find.
38
00:03:05.240 --> 00:03:09.360
So this is going to be simple in 1D.
39
00:03:09.360 --> 00:03:15.560
So let's just carry it through and then see how it goes in high dimensions.
40
00:03:15.560 --> 00:03:16.560
Okay.
41
00:03:16.560 --> 00:03:22.760
The key fact is, so the key, the key to everything is that perpendicular.
42
00:03:22.760 --> 00:03:37.000
The fact that A is perpendicular to E, which is B minus A XA.
43
00:03:37.000 --> 00:03:40.200
I don't care if I say XA.
44
00:03:40.200 --> 00:03:43.960
So that equals 0.
45
00:03:43.960 --> 00:03:51.320
You see that as the central equation, that's saying that this A is perpendicular to this
46
00:03:51.320 --> 00:03:52.320
correction.
47
00:03:52.320 --> 00:03:55.560
And that's going to tell us what X is.
48
00:03:55.560 --> 00:04:01.560
Let me just raise the board and simplify that and out will come X.
49
00:04:01.560 --> 00:04:02.560
Okay.
50
00:04:02.560 --> 00:04:05.760
So if I simplify that, let's see.
51
00:04:05.760 --> 00:04:09.800
I'll move one term to one side.
52
00:04:09.800 --> 00:04:11.400
The other term will be on the other side.
53
00:04:11.400 --> 00:04:24.360
It looks to be like X times A transpose A is equal to A transpose B.
54
00:04:24.360 --> 00:04:28.640
I have A transpose B is one term A transpose A is the other.
55
00:04:28.640 --> 00:04:35.200
So right away, here's my A transpose A, but it's just a number now.
56
00:04:35.200 --> 00:04:37.600
And I divide by it.
57
00:04:37.600 --> 00:04:38.880
And I get the answer.
58
00:04:38.880 --> 00:04:47.320
X is A transpose B over A transpose X.
59
00:04:47.320 --> 00:04:52.680
And C, the projection I wanted, that's the right multiple.
60
00:04:52.680 --> 00:04:58.280
That's got a cosine theta built in, but we don't need to look at angles.
61
00:04:58.280 --> 00:05:00.160
We've just got vectors here.
62
00:05:00.160 --> 00:05:10.600
The projection is T is A times that X, for X times that A.
63
00:05:10.600 --> 00:05:15.480
But I'm really going to eventually, I'm going to want that X coming on the right hand side.
64
00:05:15.480 --> 00:05:27.080
So if you see that I've got two of the three formulas already, right here, I've got the
65
00:05:27.080 --> 00:05:31.640
equation that leads me to the answer.
66
00:05:31.640 --> 00:05:36.160
Here's the answer for X, and here's the projection.
67
00:05:36.160 --> 00:05:37.160
OK.
68
00:05:37.160 --> 00:05:42.040
Can I do add just one more thing to this, one dimensional problem.
69
00:05:42.040 --> 00:05:50.640
One more like lift it up into linear algebra, into matrices.
70
00:05:50.640 --> 00:05:52.400
Here's the last thing I want to do.
71
00:05:52.400 --> 00:05:54.560
But don't forget those four minutes.
72
00:05:54.560 --> 00:05:58.520
A transpose B over A transpose A.
73
00:05:58.520 --> 00:06:02.600
Actually, let's look at that for a moment first.
74
00:06:02.600 --> 00:06:11.480
Suppose, and A, well, let me take this next step.
75
00:06:11.480 --> 00:06:15.240
So T is A times X. So can I write that in?
76
00:06:15.240 --> 00:06:19.200
T is A times this neat number.
77
00:06:19.200 --> 00:06:28.840
A transpose B over A transpose A. That's our projection.
78
00:06:28.840 --> 00:06:34.640
Can I ask a couple of questions about it, just while we look, get that digest that formula?
79
00:06:34.640 --> 00:06:39.160
Suppose B is double.
80
00:06:39.160 --> 00:06:43.840
Suppose I change B to 2B.
81
00:06:43.840 --> 00:06:48.400
What happens to the projection?
82
00:06:48.400 --> 00:06:52.560
So suppose I instead of that vector B that I grew on the board, make it 2B.
83
00:06:52.560 --> 00:06:54.080
Flies this law.
84
00:06:54.080 --> 00:06:57.760
What's the projection now?
85
00:06:57.760 --> 00:06:59.720
It's double 2, right?
86
00:06:59.720 --> 00:07:01.240
It's going to be twice as far.
87
00:07:01.240 --> 00:07:03.480
B goes twice as far.
88
00:07:03.480 --> 00:07:05.160
The projection will go twice as far.
89
00:07:05.160 --> 00:07:06.400
And you see it there.
90
00:07:06.400 --> 00:07:12.160
If I put it an extra factor 2, then P's got that factor 2.
91
00:07:12.160 --> 00:07:16.280
Now, what about if I double A?
92
00:07:16.280 --> 00:07:22.320
What if I double the vector A that I'm projecting onto?
93
00:07:22.320 --> 00:07:25.120
What changes?
94
00:07:25.120 --> 00:07:28.040
No, the projection doesn't change at all, right?
95
00:07:28.040 --> 00:07:31.240
Because I'm just, the line didn't change.
96
00:07:31.240 --> 00:07:35.760
If I double A or I take minus A, it's still the same line.
97
00:07:35.760 --> 00:07:37.680
The projection's still in the same place.
98
00:07:37.680 --> 00:07:42.880
And of course, if I double A, I get a 4 up above.
99
00:07:42.880 --> 00:07:44.960
And I get a 4, an extra 4 below.
100
00:07:44.960 --> 00:07:45.960
So I can't fill out.
101
00:07:45.960 --> 00:07:47.960
And the projection is the same.
102
00:07:47.960 --> 00:07:48.960
OK.
103
00:07:48.960 --> 00:07:58.120
So really, I want to look at this as the projection.
104
00:07:58.120 --> 00:08:00.000
There's a matrix here.
105
00:08:00.000 --> 00:08:04.320
The projection is carried out by some matrix
106
00:08:04.320 --> 00:08:07.480
that I'm going to call the projection matrix.
107
00:08:07.480 --> 00:08:12.400
And in other words, the projection is some matrix that
108
00:08:12.400 --> 00:08:16.480
acts on this guy B and produces the projection.
109
00:08:16.480 --> 00:08:23.640
The projection P is the projection matrix acting
110
00:08:23.640 --> 00:08:26.640
on whatever the input is.
111
00:08:26.640 --> 00:08:30.640
The input is B. The projection matrix is P. OK.
112
00:08:30.640 --> 00:08:34.120
Actually, you can tell me right away what this projection
113
00:08:34.120 --> 00:08:34.880
matrix is.
114
00:08:34.880 --> 00:08:39.960
So this is a pretty interesting matrix.
115
00:08:39.960 --> 00:08:43.080
What matrix is multiplying B?
116
00:08:43.080 --> 00:08:46.920
I'm just from my formula, and I see what P is.
117
00:08:46.920 --> 00:08:54.800
P, this projection matrix, is what is it?
118
00:08:54.800 --> 00:08:59.120
I see A, A transpose above.
119
00:08:59.120 --> 00:09:02.080
And I see A transpose A below.
120
00:09:05.680 --> 00:09:07.040
And those don't cancel.
121
00:09:07.040 --> 00:09:09.760
That's not one.
122
00:09:09.760 --> 00:09:10.760
That's a matrix.
123
00:09:10.760 --> 00:09:15.080
Because down here, the A transpose A, that's just the number.
124
00:09:15.080 --> 00:09:17.880
A transpose A, that's the length of A squared.
125
00:09:17.880 --> 00:09:21.880
And up above is a column times a row.
126
00:09:21.880 --> 00:09:24.640
Column times a row is a matrix.
127
00:09:24.640 --> 00:09:28.560
So this is a full scale N by N matrix,
128
00:09:28.560 --> 00:09:31.480
if I can find an N dimension.
129
00:09:31.480 --> 00:09:34.880
And it's kind of an interesting one.
130
00:09:34.880 --> 00:09:38.000
And it's the one, which if I multiply by B,
131
00:09:38.000 --> 00:09:39.120
then I get this.
132
00:09:39.120 --> 00:09:42.360
And you see, once again, I'm putting parentheses
133
00:09:42.360 --> 00:09:43.760
in different places.
134
00:09:43.760 --> 00:09:49.080
I'm putting the parentheses like there.
135
00:09:49.080 --> 00:09:53.400
I'm saying, OK, that's really the matrix that produces
136
00:09:53.400 --> 00:09:58.640
this projection.
137
00:09:58.640 --> 00:09:59.600
OK.
138
00:09:59.600 --> 00:10:02.920
Now, tell me.
139
00:10:02.920 --> 00:10:03.840
All right.
140
00:10:03.840 --> 00:10:08.080
What are the properties of that matrix?
141
00:10:08.080 --> 00:10:10.160
I'm just using letters here, A and B.
142
00:10:10.160 --> 00:10:11.760
I could put in numbers.
143
00:10:11.760 --> 00:10:14.760
But I think it's for once it's clearer with letters.
144
00:10:14.760 --> 00:10:17.520
Because all formulas are simple.
145
00:10:17.520 --> 00:10:20.120
A transpose, the over A transpose A.
146
00:10:20.120 --> 00:10:23.920
That's the number that multiplies the A.
147
00:10:23.920 --> 00:10:28.040
And then I see, wait a minute, there's a matrix here.
148
00:10:28.040 --> 00:10:31.960
And what's the rank of that matrix far away?
149
00:10:31.960 --> 00:10:33.480
What's the rank of that matrix?
150
00:10:33.480 --> 00:10:33.680
Yeah.
151
00:10:33.680 --> 00:10:36.560
Let me just ask you about that matrix.
152
00:10:36.560 --> 00:10:40.960
Which was a little strange, A, A transpose over this number.
153
00:10:40.960 --> 00:10:48.880
But I like to ask you with column space.
154
00:10:48.880 --> 00:10:49.280
Yeah.
155
00:10:49.280 --> 00:10:51.640
Let me ask you a column space.
156
00:10:51.640 --> 00:10:54.920
So what's the column space of a matrix?
157
00:10:54.920 --> 00:10:59.200
It's if you multiply that matrix by anything,
158
00:10:59.200 --> 00:11:02.560
you always get in the column space, right?
159
00:11:02.560 --> 00:11:07.640
The column space of a matrix is when you multiply any vector
160
00:11:07.640 --> 00:11:12.960
by that matrix, any vector B, by that matrix,
161
00:11:12.960 --> 00:11:16.080
you always land in the column space.
162
00:11:16.080 --> 00:11:19.080
That's what column spaces work.
163
00:11:19.080 --> 00:11:22.840
Now, what space do we always land in?
164
00:11:22.840 --> 00:11:24.920
What's the column space?
165
00:11:24.920 --> 00:11:29.240
What's the result when I multiply this any vector B
166
00:11:29.240 --> 00:11:32.320
by my matrix?
167
00:11:32.320 --> 00:11:37.560
So a T times B, where M?
168
00:11:37.560 --> 00:11:39.000
I'm on that one, right?
169
00:11:39.000 --> 00:11:40.920
The column space.
170
00:11:40.920 --> 00:11:43.480
So here are facts about this matrix.
171
00:11:43.480 --> 00:11:48.520
The column space of B of this projection
172
00:11:48.520 --> 00:11:52.160
matrix is the line through it.
173
00:11:52.160 --> 00:12:05.480
And the rank of this matrix is, you can all say that one.
174
00:12:05.480 --> 00:12:07.480
One, right?
175
00:12:07.480 --> 00:12:09.480
The rank is one.
176
00:12:09.480 --> 00:12:11.680
This is a rank one matrix, actually,
177
00:12:11.680 --> 00:12:15.080
which is exactly the form that we're familiar with with rank
178
00:12:15.080 --> 00:12:20.720
one matrix, with a rank one matrix, a column times a row.
179
00:12:20.720 --> 00:12:22.680
That's a rank one matrix.
180
00:12:22.680 --> 00:12:30.280
That column is the basis for the column space.
181
00:12:30.280 --> 00:12:31.680
Just one dimension.
182
00:12:31.680 --> 00:12:32.720
OK.
183
00:12:32.720 --> 00:12:35.520
So I know that much about the matrix.
184
00:12:35.520 --> 00:12:38.440
But now there are two more facts about the matrix
185
00:12:38.440 --> 00:12:42.840
that I want to know.
186
00:12:42.840 --> 00:12:45.960
Is, first of all, is the matrix symmetric?
187
00:12:45.960 --> 00:12:48.520
That's a natural question for matrices.
188
00:12:48.520 --> 00:12:51.280
And the answer is, yes.
189
00:12:51.280 --> 00:12:55.720
If I take the transpose of this, there's a number down there.
190
00:12:55.720 --> 00:13:00.200
The transpose of A, A transpose is A, A transpose.
191
00:13:00.200 --> 00:13:02.160
So P is symmetric.
192
00:13:02.160 --> 00:13:07.440
P transpose equals P. So that's the key property.
193
00:13:07.440 --> 00:13:10.720
That's the projection matrix is symmetric.
194
00:13:10.720 --> 00:13:15.400
One more property now, and this is the real one.
195
00:13:15.400 --> 00:13:21.520
What happens if I do the projection twice?
196
00:13:21.520 --> 00:13:27.200
So I'm looking for some information about P squared.
197
00:13:27.200 --> 00:13:32.800
But just to be in terms of that picture, in terms of my picture,
198
00:13:32.800 --> 00:13:38.920
I take any vector B and multiply it by my projection matrix.
199
00:13:38.920 --> 00:13:40.360
And that's what's to be there.
200
00:13:40.360 --> 00:13:44.040
So this is P B.
201
00:13:44.040 --> 00:13:48.280
And now I've projected it.
202
00:13:48.280 --> 00:13:49.880
What happens now?
203
00:13:49.880 --> 00:13:55.080
What happens when I apply the projection matrix a second time?
204
00:13:55.080 --> 00:13:57.720
So I'm applying it.
205
00:13:57.720 --> 00:14:02.800
Once it brings me here, and the second time brings me,
206
00:14:02.800 --> 00:14:05.280
I say, pull.
207
00:14:05.280 --> 00:14:07.760
The projection, for a point on this line,
208
00:14:07.760 --> 00:14:11.640
the projection is right where it is.
209
00:14:11.640 --> 00:14:13.760
The projection is the same point.
210
00:14:13.760 --> 00:14:20.440
So that means that if I project twice,
211
00:14:20.440 --> 00:14:24.560
I get the same answer as I did in the first projection.
212
00:14:24.560 --> 00:14:30.440
So those are the two properties that tell me
213
00:14:30.440 --> 00:14:33.840
I'm looking at a projection matrix.
214
00:14:33.840 --> 00:14:38.280
It's symmetric, and it's square in itself,
215
00:14:38.280 --> 00:14:40.720
because if I project the second time,
216
00:14:40.720 --> 00:14:44.080
it's the same result as the first projection.
217
00:14:44.080 --> 00:14:45.200
OK.
218
00:14:45.200 --> 00:14:51.280
So that's, and then here the exact formula,
219
00:14:51.280 --> 00:14:54.240
when I know what I'm projecting on through that line
220
00:14:54.240 --> 00:14:57.680
through A, then I know what P is.
221
00:14:57.680 --> 00:15:00.560
So you see that I have all the pieces here
222
00:15:00.560 --> 00:15:03.240
for projection on a line.
223
00:15:03.240 --> 00:15:08.160
Now, and those, please remember them.
224
00:15:08.160 --> 00:15:10.640
So there are three formulas to remember.
225
00:15:10.640 --> 00:15:15.160
The formula for x, the formula for p, which is just a,
226
00:15:15.160 --> 00:15:20.960
and then the formula for capital P, which is the matrix.
227
00:15:20.960 --> 00:15:21.720
Good.
228
00:15:21.720 --> 00:15:22.440
Good.
229
00:15:22.440 --> 00:15:23.680
OK.
230
00:15:23.680 --> 00:15:30.080
Now, I want to move to more dimensions.
231
00:15:30.080 --> 00:15:32.680
So we're going to have three formulas again,
232
00:15:32.680 --> 00:15:37.120
but you'll have to be a little different,
233
00:15:37.120 --> 00:15:40.280
because we won't have a single line.
234
00:15:40.280 --> 00:15:46.600
But playing or three dimensional or end dimensional,
235
00:15:46.600 --> 00:15:47.920
something.
236
00:15:47.920 --> 00:15:48.880
OK.
237
00:15:48.880 --> 00:15:52.200
So now I'm moving to the next question.
238
00:15:56.160 --> 00:16:06.200
Maybe I'll say first, let me say first why I want this
239
00:16:06.200 --> 00:16:09.640
projection, and then we'll figure out what it is.
240
00:16:09.640 --> 00:16:12.800
We'll go completely parallel there.
241
00:16:12.800 --> 00:16:15.160
And then we'll use it.
242
00:16:15.160 --> 00:16:15.680
OK.
243
00:16:15.680 --> 00:16:17.880
Why do I want this projection?
244
00:16:17.880 --> 00:16:21.040
Well, so why project?
245
00:16:29.040 --> 00:16:33.800
It's because, as I mentioned last time,
246
00:16:33.800 --> 00:16:46.400
this new chapter deals with equations A x equal b
247
00:16:46.400 --> 00:16:51.440
may have no solution.
248
00:16:58.160 --> 00:17:00.320
So that's really my problem.
249
00:17:00.320 --> 00:17:05.960
I'm given a bunch of equations, probably too many equations,
250
00:17:05.960 --> 00:17:08.400
more equations than I know.
251
00:17:08.400 --> 00:17:11.600
And I can't solve it.
252
00:17:11.600 --> 00:17:12.640
OK.
253
00:17:12.640 --> 00:17:14.640
So what do I do?
254
00:17:14.640 --> 00:17:19.000
I solve the closest problem that I can solve.
255
00:17:19.000 --> 00:17:21.360
And what's the closest one?
256
00:17:21.360 --> 00:17:26.600
Well, A x will always be in the column space of A. That's my problem.
257
00:17:26.600 --> 00:17:31.200
My problem is, A x has to be in the column space.
258
00:17:31.200 --> 00:17:34.840
And B is probably not in the column space.
259
00:17:34.840 --> 00:17:40.320
So I change B to what?
260
00:17:40.320 --> 00:17:45.160
I choose the closest vector in the column space.
261
00:17:45.160 --> 00:17:54.080
So I'll solve A x equal p instead.
262
00:17:54.080 --> 00:17:56.360
Now, that one I can do.
263
00:17:56.360 --> 00:18:08.200
Where p is, this is the projection of B onto the column space.
264
00:18:08.200 --> 00:18:10.840
That's why I want to be able to do this,
265
00:18:10.840 --> 00:18:14.240
because I have to find a solution here.
266
00:18:14.240 --> 00:18:17.200
And I'm going to put a little hat there
267
00:18:17.200 --> 00:18:20.960
to indicate that it's not the x.
268
00:18:20.960 --> 00:18:25.120
It's not the x that doesn't exist.
269
00:18:25.120 --> 00:18:30.800
It's the x hat that's best possible.
270
00:18:30.800 --> 00:18:38.360
So I must be able to figure out what the good projection
271
00:18:38.360 --> 00:18:41.720
there, what's the good right hand side that is in the column
272
00:18:41.720 --> 00:18:45.200
space that's as close as possible to B.
273
00:18:45.200 --> 00:18:47.640
And then I know what to do.
274
00:18:47.640 --> 00:18:48.440
OK.
275
00:18:48.440 --> 00:18:51.320
So now I've got that problem.
276
00:18:51.320 --> 00:18:54.480
So that's why I have the problem again.
277
00:18:54.480 --> 00:18:58.280
But now let me say I'm in three dimensions.
278
00:18:58.280 --> 00:19:02.920
So I have a plane, maybe, for example.
279
00:19:02.920 --> 00:19:06.320
And I have a vector B that's not in the plane.
280
00:19:10.240 --> 00:19:14.600
And I want to project B down into the plane.
281
00:19:14.600 --> 00:19:16.400
OK.
282
00:19:16.400 --> 00:19:19.440
So there's my question.
283
00:19:19.440 --> 00:19:20.560
How do I project a vector?
284
00:19:20.560 --> 00:19:24.960
And I'm really what I'm looking for is a nice formula.
285
00:19:24.960 --> 00:19:28.120
And I'm counting on linear algebra.
286
00:19:28.120 --> 00:19:30.440
So just come out right.
287
00:19:30.440 --> 00:19:36.720
A nice formula for the projection of B into the plane,
288
00:19:36.720 --> 00:19:38.320
the nearest formula.
289
00:19:38.320 --> 00:19:43.040
So again, a right angle is going to be crucial.
290
00:19:43.040 --> 00:19:45.840
OK.
291
00:19:45.840 --> 00:19:51.920
Now, so what's, first of all, I better say, what is that plane?
292
00:19:51.920 --> 00:19:55.360
So again, the formula I have to tell you what the plane is.
293
00:19:55.360 --> 00:19:57.880
How am I going to tell you a plane?
294
00:19:57.880 --> 00:20:00.480
I'll tell you a basis for the plane.
295
00:20:00.480 --> 00:20:07.440
I'll tell you two vectors A1 and A2 that give you a basis
296
00:20:07.440 --> 00:20:07.960
for the plane.
297
00:20:07.960 --> 00:20:11.240
So let me say there's an A1.
298
00:20:11.240 --> 00:20:16.480
And here's a vector A2.
299
00:20:16.480 --> 00:20:19.680
They don't have to be perpendicular,
300
00:20:19.680 --> 00:20:21.680
but they better be independent.
301
00:20:21.680 --> 00:20:23.400
Because then that tells me the plane.
302
00:20:23.400 --> 00:20:30.720
The plane is the plane of A1 and A2.
303
00:20:35.720 --> 00:20:40.960
And actually, going back to this connection,
304
00:20:40.960 --> 00:20:54.480
this plane is a column space of what matrix?
305
00:20:54.480 --> 00:20:56.400
What matrix?
306
00:20:56.400 --> 00:21:00.640
So how do I connect the two questions?
307
00:21:00.640 --> 00:21:06.040
I'm thinking, how do I protect onto a plane?
308
00:21:06.040 --> 00:21:10.080
And I want to get a matrix in here.
309
00:21:10.080 --> 00:21:13.840
Everything's cleaner if I write it in terms of a matrix.
310
00:21:13.840 --> 00:21:18.840
So what matrix has that column space?
311
00:21:18.840 --> 00:21:21.760
Well, of course, it does the matrix that
312
00:21:21.760 --> 00:21:28.840
has A1 in the first column and A2 in the second column.
313
00:21:28.840 --> 00:21:32.000
Let's be sure we've got the question
314
00:21:32.000 --> 00:21:34.200
before we get to the end.
315
00:21:34.200 --> 00:21:39.680
So I'm looking for, again, I'm given a matrix A
316
00:21:39.680 --> 00:21:42.360
with two columns.
317
00:21:42.360 --> 00:21:48.280
And really, once I get to 2, I'm ready for N.
318
00:21:48.280 --> 00:21:49.760
So it could be two columns.
319
00:21:49.760 --> 00:21:51.800
It could be N columns.
320
00:21:51.800 --> 00:21:55.080
I'll write the answer in terms of the matrix A.
321
00:21:55.080 --> 00:22:00.840
And the point will be those two columns describe the plane.
322
00:22:00.840 --> 00:22:03.120
They describe the column space.
323
00:22:03.120 --> 00:22:06.480
And I want to protect.
324
00:22:06.480 --> 00:22:11.080
And I'm given a vector B that's probably not in the column space.
325
00:22:11.080 --> 00:22:13.680
Of course, if B is in the column space,
326
00:22:13.680 --> 00:22:15.720
my prediction is simple.
327
00:22:15.720 --> 00:22:22.200
It's just B. But most likely, I have an error E
328
00:22:22.200 --> 00:22:28.800
with B minus P part, which is probably not zero.
329
00:22:28.800 --> 00:22:29.880
OK.
330
00:22:29.880 --> 00:22:37.320
But the beauty is that I know from geometry,
331
00:22:37.320 --> 00:22:39.400
or I could get it from calculus,
332
00:22:39.400 --> 00:22:43.080
or I could get it from linear algebra,
333
00:22:43.080 --> 00:22:50.160
that this vector, this is the part of B that's
334
00:22:50.160 --> 00:22:52.960
perpendicular to the plane.
335
00:22:52.960 --> 00:23:02.360
That E is perpendicular to the plane.
336
00:23:02.360 --> 00:23:07.880
Your intuition is saying that that's the crucial fact.
337
00:23:07.880 --> 00:23:11.280
That's going to give us the answer.
338
00:23:11.280 --> 00:23:11.880
OK.
339
00:23:11.880 --> 00:23:16.160
So let me, that's the problem now for the answer.
340
00:23:16.160 --> 00:23:23.360
So this is an lecture that's really like moving along.
341
00:23:23.360 --> 00:23:28.080
Because I'm just popping that problem up there
342
00:23:28.080 --> 00:23:32.160
and asking you what combination, now, yeah.
343
00:23:32.160 --> 00:23:34.360
So what is P?
344
00:23:34.360 --> 00:23:36.520
What is this projection P?
345
00:23:36.520 --> 00:23:46.720
This is the projection P. It's some multiple of the basis
346
00:23:46.720 --> 00:23:47.480
guys, right?
347
00:23:47.480 --> 00:23:51.240
Some multiple of the columns.
348
00:23:51.240 --> 00:23:56.200
But I don't like writing out x1a1, x2a2.
349
00:23:56.200 --> 00:24:01.280
I would rather write that as a, x.
350
00:24:01.280 --> 00:24:05.360
Well, actually, I should put, if I'm really doing everything
351
00:24:05.360 --> 00:24:10.040
right, I should put a little hat on to remember that this x,
352
00:24:10.040 --> 00:24:13.320
that those are the numbers, and I could have put a hat
353
00:24:13.320 --> 00:24:18.680
way back there, is, right?
354
00:24:18.680 --> 00:24:24.960
So this is the projection P. P is a x bar.
355
00:24:24.960 --> 00:24:27.400
And I'm looking for x bar.
356
00:24:27.400 --> 00:24:28.920
So that's what I want in equation.
357
00:24:28.920 --> 00:24:36.320
So now I've got hold of the problem.
358
00:24:36.320 --> 00:24:41.360
The problem is find the right combination of the column
359
00:24:41.360 --> 00:24:48.680
so that the error vector is perpendicular to the plane.
360
00:24:48.680 --> 00:24:53.280
Now, let me turn that into an equation.
361
00:24:53.280 --> 00:24:57.720
So I'll raise the board and just turn that,
362
00:24:57.720 --> 00:24:59.640
turn what was just done into an equation.
363
00:25:03.120 --> 00:25:06.680
So let me all write again the main point.
364
00:25:06.680 --> 00:25:11.280
The projection is a x bar, x, x, x.
365
00:25:11.280 --> 00:25:13.360
And our problem is find x, x.
366
00:25:17.320 --> 00:25:26.080
And the key here is that b minus a, x, x, x,
367
00:25:26.080 --> 00:25:27.760
that's the error.
368
00:25:27.760 --> 00:25:35.080
This is the e is perpendicular to the plane.
369
00:25:39.800 --> 00:25:43.400
That's got the give me, well, what am I looking for?
370
00:25:43.400 --> 00:25:45.720
I'm looking for two equations now, because I've
371
00:25:45.720 --> 00:25:50.400
got an x, 1, and an x, and 2.
372
00:25:50.400 --> 00:25:54.680
And I'll get two equations because, so this thing, e,
373
00:25:54.680 --> 00:25:56.600
is perpendicular to the plane.
374
00:26:00.200 --> 00:26:01.400
So what does that mean?
375
00:26:01.400 --> 00:26:07.600
I guess it means it's perpendicular to a1 and also to a2.
376
00:26:07.600 --> 00:26:10.160
Those are two vectors in the plane.
377
00:26:10.160 --> 00:26:13.160
And the things that are perpendicular to the plane
378
00:26:13.160 --> 00:26:15.600
are perpendicular to a1 and a2.
379
00:26:15.600 --> 00:26:20.240
Let me just, this guy then is perpendicular to the plane.
380
00:26:20.240 --> 00:26:23.240
So it's perpendicular to that vector and that vector.
381
00:26:23.240 --> 00:26:26.840
Not it's perpendicular to that, of course.
382
00:26:26.840 --> 00:26:30.800
But it's perpendicular to everything in the plane.
383
00:26:30.800 --> 00:26:36.320
And the plane is really told me by a1 and a2.
384
00:26:36.320 --> 00:26:44.520
So really, I have the equation a1 transpose b minus a, x,
385
00:26:44.520 --> 00:26:54.160
b0, and also a2 transpose b minus a, x, b.
386
00:26:57.440 --> 00:26:58.560
Those are my two equations.
387
00:27:01.840 --> 00:27:07.320
But I want those in matrix form.
388
00:27:07.320 --> 00:27:09.320
I want to put those two equations together
389
00:27:09.320 --> 00:27:10.920
as a matrix equation.
390
00:27:10.920 --> 00:27:13.560
And it just comes out right.
391
00:27:13.560 --> 00:27:17.040
Look at the matrix a transpose.
392
00:27:17.040 --> 00:27:21.240
Put a1, a1 transpose is its first row.
393
00:27:21.240 --> 00:27:24.160
A2 transpose is its second row.
394
00:27:24.160 --> 00:27:32.720
That multiplies this b minus a, x, and gives me the 0
395
00:27:32.720 --> 00:27:33.320
and the 0.
396
00:27:33.320 --> 00:27:46.320
And you see that I'm just a, this is one way to come up
397
00:27:46.320 --> 00:27:48.040
with this equation.
398
00:27:48.040 --> 00:27:54.800
So the equation I'm coming up with is a transpose b minus a,
399
00:27:54.800 --> 00:27:57.600
x, x, and x.
400
00:27:57.600 --> 00:28:00.360
OK.
401
00:28:00.360 --> 00:28:02.600
OK.
402
00:28:02.600 --> 00:28:04.960
That's my equation.
403
00:28:04.960 --> 00:28:05.960
All right.
404
00:28:05.960 --> 00:28:09.000
I want to stop for a moment before I solve it
405
00:28:09.000 --> 00:28:11.360
and just think about it.
406
00:28:11.360 --> 00:28:18.280
First of all, do you see that that equation, back in the very
407
00:28:18.280 --> 00:28:26.120
first problem I solved on a line, what was on a line,
408
00:28:26.120 --> 00:28:28.880
that the matrix a only had one problem.
409
00:28:28.880 --> 00:28:32.320
It was just little a.
410
00:28:32.320 --> 00:28:37.760
So in the first problem I solved, projecting on a line,
411
00:28:37.760 --> 00:28:40.680
for capital a, you just change that to little a
412
00:28:40.680 --> 00:28:44.360
and you have the same equation that we solved before.
413
00:28:44.360 --> 00:28:47.840
A transpose t equals 0.
414
00:28:47.840 --> 00:28:48.640
OK.
415
00:28:48.640 --> 00:28:53.680
Now, a second problem.
416
00:28:53.680 --> 00:29:00.120
I would like to, since I know about these four sub spaces,
417
00:29:00.120 --> 00:29:03.000
I would like to get them into this picture.
418
00:29:06.040 --> 00:29:08.600
So let me ask a question.
419
00:29:08.600 --> 00:29:12.520
What sub space is this thing in?
420
00:29:12.520 --> 00:29:16.560
Which is a four sub spaces is that error by per e.
421
00:29:16.560 --> 00:29:23.640
If this is nothing but e, if this guy coming down the
422
00:29:23.640 --> 00:29:30.640
perpendicular to the plane, what sub space is e?
423
00:29:30.640 --> 00:29:34.480
From this equation.
424
00:29:34.480 --> 00:29:39.760
What the equation is saying, a transpose e is 0.
425
00:29:39.760 --> 00:29:45.320
So I'm learning here that e is in the null space of a
426
00:29:45.320 --> 00:29:53.920
transpose. That's my equation and now I just wanted the
427
00:29:53.920 --> 00:29:56.600
pay, of course, that was right.
428
00:29:56.600 --> 00:30:03.360
Because things that are in the null space of a transpose,
429
00:30:03.360 --> 00:30:08.480
what do we know about the null space of a transpose?
430
00:30:08.480 --> 00:30:14.520
So the last lecture gave us the geometry of these sub spaces
431
00:30:14.520 --> 00:30:16.560
and the orthogonality of them.
432
00:30:16.560 --> 00:30:21.360
And do you remember what was on the right side of our big
433
00:30:21.360 --> 00:30:22.440
figure?
434
00:30:22.440 --> 00:30:29.360
We always have the null space of a transpose and column space
435
00:30:29.360 --> 00:30:32.600
of a and their orthogon.
436
00:30:32.600 --> 00:30:39.840
So e in the null space of a transpose is saying e is
437
00:30:39.840 --> 00:30:44.320
perpendicular to the column space of a.
438
00:30:44.320 --> 00:30:50.320
Yes.
439
00:30:50.320 --> 00:30:52.320
I just feel OK.
440
00:30:52.320 --> 00:30:55.920
Again, they came out right.
441
00:30:55.920 --> 00:31:03.560
The equation for the equation that I struggled to find for e
442
00:31:03.560 --> 00:31:07.400
really said what I wanted.
443
00:31:07.400 --> 00:31:12.520
That the error e is perpendicular to the column space of it.
444
00:31:12.520 --> 00:31:17.480
Just from our core fundamental sub spaces, we knew that
445
00:31:17.480 --> 00:31:20.200
that is the same as that.
446
00:31:20.200 --> 00:31:23.440
To say e is in the null space of a transpose as these
447
00:31:23.440 --> 00:31:25.040
perpendicular to the column space.
448
00:31:25.040 --> 00:31:26.040
OK.
449
00:31:26.040 --> 00:31:28.720
So we got this equation now.
450
00:31:28.720 --> 00:31:29.320
All right.
451
00:31:29.320 --> 00:31:37.680
Let me just rewrite it as a transpose a x cap equals a
452
00:31:37.680 --> 00:31:40.240
transpose p.
453
00:31:40.240 --> 00:31:42.240
That's our equation.
454
00:31:42.240 --> 00:31:43.240
That gives us x.
455
00:31:48.560 --> 00:31:55.000
And oh, allow me to keep remembering the one-dimensional case.
456
00:31:55.000 --> 00:31:59.800
The one-dimensional case, this was little x.
457
00:31:59.800 --> 00:32:01.160
So this was just a number.
458
00:32:01.160 --> 00:32:06.120
Little x transpose a transpose a was just a vector row
459
00:32:06.120 --> 00:32:08.440
times a column of number.
460
00:32:08.440 --> 00:32:10.120
And this was a number.
461
00:32:10.120 --> 00:32:13.080
And x was the ratio of those numbers.
462
00:32:13.080 --> 00:32:15.840
But now we've got matrices.
463
00:32:15.840 --> 00:32:17.960
This one is n by s.
464
00:32:17.960 --> 00:32:20.840
A transpose a is n by n matrix.
465
00:32:20.840 --> 00:32:21.320
OK.
466
00:32:21.320 --> 00:32:26.080
So can I move to the next word for the solution?
467
00:32:31.920 --> 00:32:33.080
OK.
468
00:32:33.080 --> 00:32:36.520
There's the key equation.
469
00:32:36.520 --> 00:32:40.800
Now I'm ready for the formulas that we have to remember.
470
00:32:40.800 --> 00:32:43.640
What's x cap?
471
00:32:43.640 --> 00:32:45.880
What's the projection?
472
00:32:45.880 --> 00:32:47.360
What's the projection matrix?
473
00:32:47.360 --> 00:32:49.280
Those are my three questions.
474
00:32:49.280 --> 00:32:51.400
That we answered in the one-b case
475
00:32:51.400 --> 00:32:55.400
and now we're ready for n-dimensional case.
476
00:32:55.400 --> 00:32:56.600
So what is x cap?
477
00:32:56.600 --> 00:32:58.720
Well, what can I say?
478
00:32:58.720 --> 00:33:09.280
But a transpose a inverse a transpose b.
479
00:33:09.280 --> 00:33:14.600
That's the solution to our question.
480
00:33:14.600 --> 00:33:15.680
What's the projection?
481
00:33:15.680 --> 00:33:17.160
That's more interesting.
482
00:33:17.160 --> 00:33:19.280
What's the projection?
483
00:33:19.280 --> 00:33:24.280
The projection is a x cap.
484
00:33:24.280 --> 00:33:27.480
That's how x cap got into the picture in the first place.
485
00:33:27.480 --> 00:33:33.800
x cap was the combination of columns.
486
00:33:33.800 --> 00:33:35.520
And then I had to look for those numbers.
487
00:33:35.520 --> 00:33:37.240
And now I found them.
488
00:33:37.240 --> 00:33:40.840
Was the combination of the columns of a that gave me the
489
00:33:40.840 --> 00:33:42.160
projection?
490
00:33:42.160 --> 00:33:42.800
OK.
491
00:33:42.800 --> 00:33:45.760
So now I know what this guy is.
492
00:33:45.760 --> 00:33:48.320
So it's just I multiply by a.
493
00:33:48.320 --> 00:33:58.480
A transpose a inverse a transpose b.
494
00:33:58.480 --> 00:34:06.120
That's looking a little messy, but not bad.
495
00:34:06.120 --> 00:34:13.480
That combination is our magic combination.
496
00:34:13.480 --> 00:34:20.400
This is the thing which is what which is life.
497
00:34:20.400 --> 00:34:21.160
What's it like?
498
00:34:21.160 --> 00:34:25.040
What was it in the dimension?
499
00:34:25.040 --> 00:34:25.640
What was it?
500
00:34:25.640 --> 00:34:28.960
We must have had this thing lay back at the beginning
501
00:34:28.960 --> 00:34:31.040
of the lecture.
502
00:34:31.040 --> 00:34:34.240
What did we, oh, then a was just a column.
503
00:34:34.240 --> 00:34:42.520
So it was little a, little a transpose over a transpose a.
504
00:34:42.520 --> 00:34:44.160
That's what it was in 1d.
505
00:34:49.160 --> 00:34:53.240
You see what's happened in more dimensions.
506
00:34:53.240 --> 00:34:57.080
I'm not allowed to just divide because I don't have a
507
00:34:57.080 --> 00:34:57.480
number.
508
00:34:57.480 --> 00:35:01.280
I have to put inverse because I have an n by n matrix.
509
00:35:01.280 --> 00:35:05.080
But say it for me.
510
00:35:05.080 --> 00:35:09.280
And now tell me what the projection matrix.
511
00:35:09.280 --> 00:35:18.240
What matrix is multiplying b to give the projection?
512
00:35:18.240 --> 00:35:20.000
Right there.
513
00:35:20.000 --> 00:35:23.360
There are not even already underlined it by action.
514
00:35:23.360 --> 00:35:30.280
The projection matrix which I use capital P is this.
515
00:35:30.280 --> 00:35:31.280
It's that sense.
516
00:35:31.280 --> 00:35:38.480
So I write it again, a times a transpose a inverse times a
517
00:35:38.480 --> 00:35:39.000
transpose.
518
00:35:47.720 --> 00:35:56.200
Now if you bear with me, I'll think about what I've done.
519
00:35:56.200 --> 00:35:59.160
I've got this form.
520
00:35:59.160 --> 00:36:06.240
Now the first thing that occurs to me is something bad.
521
00:36:06.240 --> 00:36:11.480
Look, why don't I just, you know, here's a product of two
522
00:36:11.480 --> 00:36:14.760
matrices and I want it to inverse.
523
00:36:14.760 --> 00:36:18.120
Why don't I just use the formula I know for the inverse of
524
00:36:18.120 --> 00:36:23.480
a product and say, OK, that a inverse times a transpose
525
00:36:23.480 --> 00:36:24.000
inverse.
526
00:36:24.000 --> 00:36:31.000
What will happen if I do that?
527
00:36:31.000 --> 00:36:37.000
What will happen if I say, hey, this is a inverse times a
528
00:36:37.000 --> 00:36:38.600
transpose inverse.
529
00:36:38.600 --> 00:36:42.240
Then shall I do it?
530
00:36:42.240 --> 00:36:45.640
It's going to go on videotape if I do it.
531
00:36:45.640 --> 00:36:48.640
And I don't, all right, I'll put it there.
532
00:36:48.640 --> 00:36:54.600
Don't take the video tape like so carefully.
533
00:36:54.600 --> 00:36:58.840
OK, so if I put that thing, it would be a a inverse, a
534
00:36:58.840 --> 00:37:05.840
transpose inverse, a transpose, and what's that?
535
00:37:05.840 --> 00:37:06.840
That's the idea.
536
00:37:09.920 --> 00:37:12.920
But what's going on?
537
00:37:12.920 --> 00:37:18.440
So, right, you see, my question is somehow,
538
00:37:18.440 --> 00:37:20.840
I did something wrong.
539
00:37:20.840 --> 00:37:21.920
That wasn't the last.
540
00:37:21.920 --> 00:37:26.280
And why is that?
541
00:37:26.280 --> 00:37:29.840
Because a is not a square matrix.
542
00:37:29.840 --> 00:37:31.400
A is not a square matrix.
543
00:37:31.400 --> 00:37:34.000
It's not a matrix.
544
00:37:34.000 --> 00:37:38.200
So I have to leave it that way.
545
00:37:38.200 --> 00:37:39.680
This is not OK.
546
00:37:39.680 --> 00:37:43.760
If an a was a square inverse of a matrix, then I couldn't
547
00:37:43.760 --> 00:37:48.400
leave them plain.
548
00:37:48.400 --> 00:37:51.360
Let me think about that case.
549
00:37:51.360 --> 00:37:54.400
But my main case, the whole reason I'm doing all this,
550
00:37:54.400 --> 00:38:01.200
is that a is this matrix that has too many rows.
551
00:38:01.200 --> 00:38:05.520
It's just got a couple of columns, like a 1 and a 2.
552
00:38:05.520 --> 00:38:09.160
But lots of rows, not square.
553
00:38:09.160 --> 00:38:14.280
And if it's not square, this a transpose a is square, but I
554
00:38:14.280 --> 00:38:16.920
can't pull it apart, like this.
555
00:38:16.920 --> 00:38:22.040
So I'm not allowed to do this pull apart, except if a was
556
00:38:22.040 --> 00:38:23.120
square.
557
00:38:23.120 --> 00:38:26.680
Now, if a is square, what's going on if a is a square
558
00:38:26.680 --> 00:38:32.760
matrix, a nice square inverse of a matrix?
559
00:38:32.760 --> 00:38:33.960
What's up with that?
560
00:38:33.960 --> 00:38:36.240
What's with that case?
561
00:38:36.240 --> 00:38:40.840
So this is all equal to the forward law of work thing 2.
562
00:38:40.840 --> 00:38:44.600
If a is a nice square and vertical matrix, what's
563
00:38:44.600 --> 00:38:46.240
its column state?
564
00:38:46.240 --> 00:38:49.000
So it's a nice n by n is vertical, everything
565
00:38:49.000 --> 00:38:50.640
great matrix.
566
00:38:50.640 --> 00:38:53.880
What's its column state?
567
00:38:53.880 --> 00:38:57.240
The whole of rm.
568
00:38:57.240 --> 00:39:01.040
So what's the projection matrix, if I'm projecting,
569
00:39:01.040 --> 00:39:03.480
under the whole state?
570
00:39:03.480 --> 00:39:07.080
It's the identity.
571
00:39:07.080 --> 00:39:12.040
If I'm projecting, B, under the whole state, not just
572
00:39:12.040 --> 00:39:16.840
under a plane, but onto all of 3D, then B is already
573
00:39:16.840 --> 00:39:18.400
in the column state.
574
00:39:18.400 --> 00:39:20.080
The projection is the identity.
575
00:39:20.080 --> 00:39:23.560
And this gives me the correct formula.
576
00:39:23.560 --> 00:39:25.280
P is up.
577
00:39:25.280 --> 00:39:29.240
But if I'm projecting under a sub space,
578
00:39:29.240 --> 00:39:31.240
then I can't split those apart,
579
00:39:31.240 --> 00:39:33.920
and I have to say with that formula.
580
00:39:33.920 --> 00:39:36.640
OK.
581
00:39:36.640 --> 00:39:40.720
And what can I say?
582
00:39:40.720 --> 00:39:43.840
So I remember this formula for 1D,
583
00:39:43.840 --> 00:39:47.400
and that's what it looks like in endometrics.
584
00:39:47.400 --> 00:39:50.960
And what are the properties that I expected for any projection
585
00:39:50.960 --> 00:39:54.040
matrix, and I still expect for this one?
586
00:39:54.040 --> 00:39:59.640
That matrix should be symmetric, and it is P transpose P,
587
00:39:59.640 --> 00:40:04.160
because if I transpose this, this guy is symmetric,
588
00:40:04.160 --> 00:40:06.120
and it is inverse of symmetric.
589
00:40:06.120 --> 00:40:10.280
And if I transpose this one, when I transpose it,
590
00:40:10.280 --> 00:40:14.320
will pop up there, and become A. The A transpose will
591
00:40:14.320 --> 00:40:18.160
pop up here, and I'm back to P again.
592
00:40:18.160 --> 00:40:21.760
And do we dare try the other property, which
593
00:40:21.760 --> 00:40:31.320
is P squared equal P?
594
00:40:31.320 --> 00:40:32.440
It's got to be right.
595
00:40:35.480 --> 00:40:39.920
Because we know geometrically that the first projection
596
00:40:39.920 --> 00:40:42.960
pops us on in the column space, and the second one
597
00:40:42.960 --> 00:40:44.560
leaves us where we are.
598
00:40:44.560 --> 00:40:51.280
So I expect that if I multiply by, if I multiply by another
599
00:40:51.280 --> 00:40:55.560
P, well, there's another A, another A,
600
00:40:55.560 --> 00:41:02.960
transpose A, inverse A, transpose.
601
00:41:02.960 --> 00:41:05.160
Can you?
602
00:41:05.160 --> 00:41:12.120
God, A, A's in a row is like a C. But do you see that it works?
603
00:41:15.000 --> 00:41:16.400
So I'm squaring back.
604
00:41:16.400 --> 00:41:17.560
So what do I do?
605
00:41:17.560 --> 00:41:19.440
How do I see that multiplication?
606
00:41:19.440 --> 00:41:24.400
Well, yeah, I just want to put parentheses in good places.
607
00:41:24.400 --> 00:41:26.000
So I see what's happening.
608
00:41:26.000 --> 00:41:29.920
Yeah, here's an A transpose A sitting together.
609
00:41:29.920 --> 00:41:33.120
So when that A transpose A multiplies this inverse,
610
00:41:33.120 --> 00:41:36.720
all that stuff goes.
611
00:41:36.720 --> 00:41:39.520
And we just the A transpose at the end, which
612
00:41:39.520 --> 00:41:42.320
is just what we want.
613
00:41:42.320 --> 00:41:46.880
So P squared equals P. So sure enough, those two properties
614
00:41:46.880 --> 00:41:49.640
hold.
615
00:41:49.640 --> 00:41:51.640
That's it.
616
00:41:51.640 --> 00:41:55.200
OK, we really have got now all the points.
617
00:41:55.200 --> 00:42:00.960
X hat, the projection P, and a projection matrix capital P.
618
00:42:00.960 --> 00:42:05.720
And now my job is to Newton.
619
00:42:05.720 --> 00:42:13.720
OK, so when would I have a bunch of equations?
620
00:42:13.720 --> 00:42:18.280
Too many equations, and yet I want the best answer.
621
00:42:18.280 --> 00:42:24.560
And the most important example, the most common example,
622
00:42:24.560 --> 00:42:39.840
is if I have a point of here is the application, these squares,
623
00:42:39.840 --> 00:42:43.960
which are being made by a lot.
624
00:42:49.200 --> 00:42:51.960
So I'll start this application today,
625
00:42:51.960 --> 00:42:57.320
and there's more in it than I can do in this same lecture.
626
00:42:57.320 --> 00:43:01.200
So that'll give me a chance to recap the formulas.
627
00:43:01.200 --> 00:43:04.040
And there they are.
628
00:43:04.040 --> 00:43:07.000
And recap the idea.
629
00:43:07.000 --> 00:43:11.560
So let me start the problem today.
630
00:43:11.560 --> 00:43:17.640
I'm given a bunch of data points.
631
00:43:17.640 --> 00:43:21.560
And they lie close to a line, but not on a one.
632
00:43:21.560 --> 00:43:22.760
Let me take that.
633
00:43:22.760 --> 00:43:27.520
Say a T for the 1, 2, and 3.
634
00:43:27.520 --> 00:43:32.880
I have 1, and 2, and 2 again.
635
00:43:32.880 --> 00:43:36.960
So my data points are, this is the time
636
00:43:36.960 --> 00:43:39.000
direction.
637
00:43:39.000 --> 00:43:44.360
And this is like, well, the name, call that B or Y or something.
638
00:43:44.360 --> 00:43:47.000
I'm given these three points.
639
00:43:47.000 --> 00:43:53.280
And I want to fit the by a lot by the best straight line.
640
00:43:53.280 --> 00:44:02.880
So the problem is fit the point 1, 1 is the first point.
641
00:44:02.880 --> 00:44:09.960
The second point is T equals 2, V equals 1.
642
00:44:09.960 --> 00:44:16.760
And the third point is T for 3, V equals 2.
643
00:44:16.760 --> 00:44:19.000
So those are my three points, T for 2.
644
00:44:19.000 --> 00:44:20.000
That's 2.
645
00:44:24.640 --> 00:44:26.640
So this is the point 1, 1.
646
00:44:26.640 --> 00:44:28.000
This is the point 2, 2.
647
00:44:28.000 --> 00:44:30.160
And that's the point 3, 2.
648
00:44:30.160 --> 00:44:34.720
And of course, there isn't a line that goes through.
649
00:44:34.720 --> 00:44:36.400
So I'm looking for the best line.
650
00:44:36.400 --> 00:44:41.320
I'm looking for a line that probably goes somewhere.
651
00:44:41.320 --> 00:44:44.480
Do you think it goes somewhere like that?
652
00:44:44.480 --> 00:44:45.480
I'm looking at it.
653
00:44:45.480 --> 00:44:48.800
I didn't mean to make it go through that point at 1.
654
00:44:48.800 --> 00:44:51.840
It'll kind of go between.
655
00:44:51.840 --> 00:44:56.560
So the error there and the error there and the error there
656
00:44:56.560 --> 00:45:02.120
are a small as I can get there.
657
00:45:02.120 --> 00:45:02.960
OK.
658
00:45:02.960 --> 00:45:07.560
What I'd like to do is find the matrix A.
659
00:45:07.560 --> 00:45:12.360
Because once I found the matrix A, the formula is take over.
660
00:45:12.360 --> 00:45:20.560
So what I'm looking for there's a line, V is C plus VT.
661
00:45:20.560 --> 00:45:24.360
So this is in the homework that I turned out today.
662
00:45:24.360 --> 00:45:25.680
Find the best line.
663
00:45:25.680 --> 00:45:31.240
So I'm looking for the numbers, C and V.
664
00:45:31.240 --> 00:45:33.680
That's tell me the line.
665
00:45:33.680 --> 00:45:37.600
And I want them to be as close to going through those three
666
00:45:37.600 --> 00:45:39.680
points as I can get.
667
00:45:39.680 --> 00:45:41.240
I can't get exactly.
668
00:45:41.240 --> 00:45:43.640
So there are three equations to go through the three
669
00:45:43.640 --> 00:45:45.240
points.
670
00:45:45.240 --> 00:45:50.320
It will go exactly through that point if the C.
671
00:45:50.320 --> 00:45:52.480
That first point has T equal to 1.
672
00:45:52.480 --> 00:45:56.600
So that would say C plus D equal to 1.
673
00:45:56.600 --> 00:45:59.400
This is the 1, 1.
674
00:45:59.400 --> 00:46:02.720
The second point, T is 2.
675
00:46:02.720 --> 00:46:09.240
So C plus 2D should come out to equal to 2.
676
00:46:09.240 --> 00:46:11.920
But I also want to get the third equation.
677
00:46:11.920 --> 00:46:14.200
And at that third equation, T is 3.
678
00:46:14.200 --> 00:46:25.440
So C plus 3D equals only 2.
679
00:46:25.440 --> 00:46:26.680
That's the key.
680
00:46:26.680 --> 00:46:28.200
It's the right down.
681
00:46:28.200 --> 00:46:32.560
What equation we would like to solve but can't?
682
00:46:32.560 --> 00:46:34.720
Reason, if we could solve them, that
683
00:46:34.720 --> 00:46:37.920
would mean that we could put a line through all three points.
684
00:46:37.920 --> 00:46:44.080
And of course, if these numbers 1, 2, 2 were different,
685
00:46:44.080 --> 00:46:46.680
we could do it.
686
00:46:46.680 --> 00:46:49.680
But with those numbers, 1, 2, 2, we can't.
687
00:46:49.680 --> 00:46:54.160
So what is our equation A, X, equal to A, X, equal to B,
688
00:46:54.160 --> 00:46:56.920
that we can't solve?
689
00:46:56.920 --> 00:47:00.760
I just want to say what's the matrix here, what's
690
00:47:00.760 --> 00:47:03.680
the unknown X, and what's the right hand side?
691
00:47:03.680 --> 00:47:11.600
So this is the matrix is 1, 1, 1, 1, 1, 2, 3.
692
00:47:11.600 --> 00:47:18.040
The unknown is C and D. And the right hand side is 1, 2, 2.
693
00:47:24.640 --> 00:47:25.000
Right?
694
00:47:25.000 --> 00:47:30.560
I just taken my equations and I said, what is A, X, and what is B?
695
00:47:36.520 --> 00:47:39.720
Then there's no solution.
696
00:47:39.720 --> 00:47:42.480
This is the typical case, where I have three equations
697
00:47:42.480 --> 00:47:44.840
to unknown no solution.
698
00:47:44.840 --> 00:47:48.440
But I'm still looking for the best solution.
699
00:47:48.440 --> 00:47:55.680
And the best solution is to solve, not
700
00:47:55.680 --> 00:48:01.360
this equation A, X, equal B, which has no solution.
701
00:48:01.360 --> 00:48:05.960
But the equation that does have a solution, which
702
00:48:05.960 --> 00:48:09.120
was this one.
703
00:48:09.120 --> 00:48:11.560
So that's the equation that's all.
704
00:48:11.560 --> 00:48:13.960
That's the central equation of the subject.
705
00:48:13.960 --> 00:48:16.680
I can't solve A, X, equal B.
706
00:48:16.680 --> 00:48:22.240
But magically, when I multiply both sides by A transpose,
707
00:48:22.240 --> 00:48:25.720
I get an equation that I can solve.
708
00:48:25.720 --> 00:48:30.800
And if solution gives me X, the best X, the best projection,
709
00:48:30.800 --> 00:48:35.160
and I discover what's the matrix that's behind it.
710
00:48:35.160 --> 00:48:35.960
OK.
711
00:48:35.960 --> 00:48:41.080
So next time I'll complete this example in America's
712
00:48:41.080 --> 00:48:44.520
today was all whether it's a number in the next time.
713
00:48:44.520 --> 00:49:14.320
OK.