Markov decision process

Markov Decision Processes
Tutorial Slides by Andrew Moore
How do you plan efficiently if the results of your actions are uncertain? There is some remarkably good news, and some some significant computational hardship.

Markov decision processes (MDPs), named after Andrey Markov, provide a mathematical framework for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision maker.

24.6 Markov Decision Process Architecture
24.7 Advanced: Plan-Based Dialogue Agents
24.7.1 Plan-Inferential Interpretation and Production ...

AI has been able to use tools drawn from economics, such as decision theory and decision analysis,[42] Bayesian decision networks,[98] information value theory,[43] Markov decision processes,[103] dynamic decision networks, ...

The environment is typically formulated as a finite-state Markov decision process (MDP), and reinforcement learning algorithms for this context are highly related to dynamic programming techniques.

"The focus of this research project is on planning under uncertainty using Markov decision processes. The main application areas is the design of automated planning and control systems for stochastic domains including mobile robotics.

Hidden Markov model
Examples of Markov chains
Markov decision process
Mark V Shaney ...

See Sections 1 through 5 from ``Decision-Theoretic Planning and Markov Decision Processes'' by Tom Dean for an introduction to representing dynamical systems as Bayesian networks, ...

See also: Reinforcement learning, Machine learning, Knowledge, Artificial intelligence, Dynamic programming