Normal Equations

Let with and . The Least-Squares Approximation of the system is the solution to the system:

Remarks

  1. need not be invertible, but there should be a solution to , which is guaranteed by the existence of projection.
  2. Normal equations are NOT a good way of solving least squares approximation in a real problems since:
    • It is not stable numerically
    • QR does not have this issue