Optimization by Vector Space Methods

 
ECE 580 / Math 587 , Spring 2008 M&W @ 11, 106B6 Engg. Hall

This is an introductory course in functional analysis and infinite dimensional optimization, with applications in least-squares estimation, nonlinear programming in Banach spaces, optimal and robust control of lumped and distributed parameter systems, and differential games.

Prerequisites

MATH 415 and MATH 547, or consent of instructor.
Exposure to optimization at the level of ECE 490 or MATH 484 recommended.

Resources

Course text: Optimization by Vector Space Methods, D. G. Luenberger, Wiley, 1997.

A pdf version of this course information is available here.

The following texts are on reserve:

  • 512.523L96o Luenberger, David G.; Optimization by Vector Space Methods
  • 515.7t21i Taylor, Angus; Introduction to Functional Analysis
  • 510m435v.132 Curtain/ Pritchard; Functional Analysis in Modern Appled Mathematics
  • 515.7b18a Balakrishnan, A.V.; Applied Functional Analysis
  • 515.7246d91l v.1 c.4 Dunford, Nelson; Linear Operators
  • 515.7l74e:e c.3 Liusternik/ Sobolev; Elements of Functional Analysis

Homework, etc.

Homework problems will be assigned on a weekly basis, available at this link,
to be handed in at the beginning of class on the date due. They will be graded and returned the following week. Late homework cannot be accepted.

There will be one midterm, a final project, and a final exam. The midterm will be held Tuesday eve., March 4, 6:00-7:30, in 141 CSL. Further details can be found here.

Outline

  1. An introduction to functional analytic approach to optimization; Finite- versus infinite-dimensional spaces; Application examples (1 hr)
  2. Normed linear spaces (3 hrs)
  3. Optimization of functionals -- General results on existence and uniqueness of an optimum (1 hr)
  4. Fixed points of transformations on Banach Spaces -- Applications to solutions of differential (ordinary and partial) and integral equations; Minimax and Nash equilibrium theorems of game theory (5 hrs)
  5. Hilbert Spaces -- The Projection Theorem; Minimum distance to a convex set (2 hrs)
  6. Examples of complete orthonormal sequences; Wavelets (2 hrs)
  7. Hilbert Spaces of random variables and stochastic processes; Least-squares estimation (3 hrs)
  8. Dual Spaces. The Hahn-Banach Theorem, with applications to minimum norm problems (5 hrs)
  9. Linear operators and adjoints (4 hrs)
  10. Calculus in Banach Spaces; Gateaux and Frechet derivatives. Local theory of unconstrained optimization; Euler-Lagrange equations (3 hrs)
  11. Global theory of unconstrained optimization; Fenchel duality theory (2 hrs)
  12. Constrained optimization of functionals; Local and global theory. Nonlinear programming and the Kuhn-Tucker Theorem in infinite dimensions (4 hrs)
  13. Optimal control and Pontryagin's Minimum Principle (3 hrs)
  14. Differential Games (2 hrs)
  15. Numerical Methods (1 hr)
  16. Other related topics of interest from computer science, control, or statistics as time and interest permits.
    Site Meter