Markov Chains & Stochastic Stability | Control Techniques for Complex Networks |

## Selected references

### Power Systems

- Electricity Markets – Archival

See also the 2013 INI tutorial,*Control of the Grid in 2020, and How Economics Can Help Us**,*and the 2015 National Academies tutorial

- Ancillary Service to the Grid from Deferrable Loads
- Demand Dispatch with Heterogeneous Intelligent Loads
- Smart Fridge / Dumb Grid?
- State Estimation for the Individual and the Population in Mean Field Control with Application to Demand Dispatch
- Individual Risk in Mean Field models
- Buildings as Batteries
- Convex Hull Pricing in Energy and Reserves Markets
- Where is the Missing Money? The Impact of Ramping Costs in Dynamic Electricity Markets

### Popular articles

- Regaining Control in our Electric Grid
- Could one million smart pool pumps ‘store’ renewable energy better than giant batteries?
- Energy storage: Power revolution –
*Peter Fairley*. Nature, 2015 - Mathematical Sciences Research Challenges for the Next-Generation Electric Grid (see also
*video from 2015 NAS meeting*, which includes survey of UF/Inria research). - Math Model Explains High Prices in Electricity Markets –
*Sara Robinson*(survey of Cho & Meyn)

### Detection and Estimation

- Feedback Particle Filter
- Multivariable Feedback Particle Filter (much more at Mehta Inc.)
- Generalized Error Exponents For Small Sample Universal Hypothesis Testing
- Workload and Optimal Control for Dynamic Bipartite Matching Models

### Stochastic Control and Applications

- Control Techniques for Complex Networks
- Differential TD learning for value function approximation
- ODE Methods For MDPs and Application to Kullback-Leibler Control Cost
- Feature Selection for Neuro-Dynamic Programming
- Workload and Optimal Control for Dynamic Bipartite Matching Models
*Q*-learning and Pontryagin’s Minimum Principle- Approximate Dynamic Programming using Fluid and Diffusion Approximations with Applications to Power Management

### Mathematical foundations

- Markov Chains and Stochastic Stability
- Ergodic Theory for Controlled Markov Chains with Stationary Inputs
- On the
*f*-Norm Ergodicity of Markov Processes in Continuous Time - On the Efficiency of Equilibria in Mean-field Oscillator Games
- Large deviation asymptotics for busy periods (sequel to
*Most likely paths to error when estimating the mean of a reflected random walk*) - Approximating a diffusion by a Hidden Markov Model