An introduction to stochastic process theory with emphasis on applications to communications, control, signal processing and machine learning. The course covers basic models, including Markov processes, and how they lead to algorithms for classification prediction, inference and model selection.
The course mainly follow’s Hajek’s excellent textbook. Markov chains and Monte-Carlo methods are based on chapters 7 and 8 of the new monograph
Most of the topics in this course are basic to any machine learning / statistical learning course. Applications to stochastic optimization and reinforcement learning are touched on throughout; topics such as relative-entropy, mutual information, and hidden Markov models are relevant in every corner of communications, control and signal processing. Wikipedia will list applications ranging from nance to natural language processing.
Essential background: Probability theory and appreciation for mathematical foundations. Experience with Matlab or Python is also essential.
- Ramon van Handel’s lecture notes on HMMs