Stochastic Methods II

Stochastic Methods II

An introduction to stochastic process theory with emphasis on applications to communications, control, signal processing and machine learning.   The course covers basic models, including Markov processes, and how they lead to algorithms for classification prediction, inference and model selection.

The course mainly follow’s Hajek’s excellent textbook.  Markov chains and Monte-Carlo methods are based on chapters 7 and 8 of the new monograph

  1. Review of stochastic processes
  2. Convergence theory & asymptotic statistics
  3. Random processes & applications
  4. Inference for stochastic processes
  5. Monte-Carlo methods (MCMC, stochastic approximation, particle filter, … as time permits)

Most of the topics in this course are basic to any machine learning / statistical learning course.  Applications to stochastic optimization and reinforcement learning are touched on throughout;  topics such as relative-entropy, mutual information, and hidden Markov models are relevant in every corner of communications, control and signal processing. Wikipedia will list applications ranging from  nance to natural language processing.

Essential background: Probability theory and appreciation for mathematical foundations.  Experience with Matlab or Python is also essential.

Notes for 2023:

Course information from Spring 2022

More Resources