Summary of Week 12 -- Capsule 3 -- MDP Objective

00:00:00 - 00:10:00

In this video, the presenter explains the concept of a Markov decision process and how to calculate the expected utility of a policy in a stochastic environment. He introduces the idea of discounted rewards and discusses how they can be used to evaluate different policies. Finally, he explains how the optimal policy is the one with the highest expected utility.

• 00:00:00 In this video, the objectives of the markov decision process (mdp) are introduced. Different quality of policies are evaluated by how much their rewards sum to, using a discount factor.
• 00:05:00 In this video, the author explains the concept of discounted rewards and how they can be used to evaluate policies. He also discusses the concept of expected utility, which is a measure of the goodness of a policy. Finally, he explains how to calculate expected utility in a stochastic environment.
• 00:10:00 In this video, the presenter covers the basics of the mathematical problem of solving a problem called an "mdp." An "mdp" is a problem that asks for the optimal action to be taken in a given state, given the possible future states. A policy, or set of actions, is then evaluated based on its expected utility, which takes into account the probability of transitioning into each possible future state. The optimal policy is the one with the highest expected utility, which in this case is the policy represented by pi star.