An introduction to stochastic process theory with emphasis on applications to communications, control, signal processing and machine learning.   The course covers basic models, including Markov processes, and how they lead to algorithms for classification prediction, inference and model selection.

The course mainly follow’s Hajek’s excellent textbook.  Markov chains and Monte-Carlo methods are based on chapters 7 and 8 of the new monograph

Outline Review of stochastic processes Convergence theory & asymptotic statistics Random processes & applications Inference for stochastic processes Monte-Carlo methods (MCMC, stochastic approximation, particle filter, … as time permits)

Most of the topics in this course are basic to any machine learning / statistical learning course.  Applications to stochastic optimization and reinforcement learning are touched on throughout;  topics such as relative-entropy, mutual information, and hidden Markov models are relevant in every corner of communications, control and signal processing. Wikipedia will list applications ranging from  nance to natural language processing.

Essential background: Probability theory and appreciation for mathematical foundations.  Experience with Matlab or Python is also essential.

Course information from Spring 2022

More Resources

• Ramon van Handel’s lecture notes on HMMs