Minimizing the Error

By geometry: Find the nearest point

By algerbra: Split $b$ into two parts.Column space is $p$.Perpendicular part is $e$.

By caculus :$E=||Ax-b||^2$

The Big Picture for Least Squares

Fitting a Straight Line

The line $C+Dt$ minimizes $e^2_1+...+e^2_m=||Ax-b||^2$ ,when $A^TA\hat x=A^Tb$

Fitting by a Parabola

Problem : Fit heights $b_1, ... , b_m$ at times $t_1, ... , t_m$ by a parabola $C + Dt + Et^2$

Solution :With $m>3$ points,the $m$ equations for an exact fit are generally unsolvable