Consider the gridworld where Left and Right actions are always successful. Specifically, the available actions in each
state are to move to the neighboring grid squares. From state a, there is also an exit action available, which results
in going to the terminal state and collecting a reward of 10. Similarly, in state e, the reward for the exit action is 1.
Exit actions are always successful. The discount factor γ = 1. Now, consider the policy π shown below, and evaluate the following quantity for this policy.
V π (a) =