AMSC/CMSC 660: Scientific Computing I
The curriculum was updated in the Fall of 2023
Computer numbers and sources of errors
- Sources of errors in numerical calculations: roundoff, truncation, termination of iterations, statistical error in Monte Carlo
- Computer numbers: integers and floating point numbers. Overflow and underflow.
- Stability and conditioning.
Lecture notes: ErrorsNumbers.pdf
Numerical Linear Algebra
- Vector spaces, norms, inner products
- Spectral decomposition, Schur form, QR, SVD, least squares problem
- Condition numbers for for matrix-vector products, solving of linear systems, and eigenvalue problems.
- Basic algorinthms: PLU and Cholesky. Counting flops. Properties of symmetric positive definite matrices.
- Ky-Fan norms and Eckart-Young-Mirsky theorem
- Nonnegative matrix factorization. Gradient descent and projected gradient descent, Lee and Seung, coordinate descent.
- Collaborative filtering and matrix completion
- Conjugate graduent algorithm
- Direct methods for solving linear systems with sparse and structured matrices. Nested Dissection.
- Finite difference methods in 2D: different types of boundary conditions, convergence.
Optimization
- Line-search methods. The Wolfe conditions. Convergence.
- Trust-region methods. Methods for solving the trust-region subproblem.
- Choosing search direction or model: Newton's, BFGS, L-BFGS.
- Nonlinear least-squares problem. Gauss-Netwon. Levenberg-Marquardt.
- Stochastic graduent descent. Convergence theory.
- Nesterov's accelegated descent. Adam.
- Constrained optimization. Karush-Kuhn-Tucker optimality conditions. Active-set method.