Norms, conditioning, and numerical stability
From measuring error to condition numbers, understand how sensitive a computation is to perturbations.
Learning goal
Interpret norms for error measurement, explain condition numbers, and distinguish residual, error, and sensitivity.
Prerequisites
- Basic linear algebra
- Vectors and matrices
- Familiarity with residuals
What does a norm measure?
To compare errors, we need a way to measure their size. The 2-norm gives the Euclidean length of a vector.
Absolute and relative error
Relative error puts the error on the scale of the quantity being estimated.
Condition number
A well-conditioned problem does not amplify small input perturbations into very large output changes. For many linear problems, the condition number measures this sensitivity.
A large κ indicates sensitivity to perturbations. This is different from numerical error introduced by the algorithm.
Residual is not the same as error
A small residual means the approximate solution is consistent with the equation, but solution error also depends on conditioning.
Connection to a numerical solver
In the Convex QP Solver, diagnostics go beyond the objective value. Stationarity, primal feasibility, complementarity, and duality gap provide several views of solution quality.
Exercises
- Compute the 2-norms of (3,4) and (1,1).
- Explain absolute versus relative error with an example.
- Why does a small residual not always imply a small solution error?
Related project
Convex Quadratic Programming Solver
Residual diagnostics and numerical validation are used to inspect solver output.
Open sourceHow trustworthy is a numerical result?
Error, sensitivity, and validation criteria can be defined for a concrete problem.