ME 254: DYNAMIC PROGRAMMING

University of California, Santa Barbara

This graduate course provides a comprehensive treatment of dynamic programm ing (DP) as the fundamental framework for staged optimization problems unde r uncertainty. Core topics include the principle of optimality, Bellman equ ations, value and policy iteration, deterministic and stochastic dynamic pr ogramming, and Markov decision processes. Applications span linear-quadrati c optimal control, inventory control, asset management, hypothesis testing, the Viterbi algorithm, among others. The course concludes with modern rein forcement learning topics including Q-learning, temporal difference learnin g, and value function approximation, demonstrating how classical DP princip les form the theoretical foundation for contemporary AI algorithms.