Scribd
Upload
11 views3 pages

Numerical Example Explanation Q Learning

This document explains a numerical example of Q-learning, detailing the steps involved in calculating the Q-value for moving from state B to goal state F. It outlines the initialization of the Q-matrix, the selection of actions based on a reward matrix, and the application of the Q-learning update formula. The final Q-value for the action B to F is determined to be 100, indicating a high reward for reaching the goal state.

Uploaded by

pardeep9262865788
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
11 views3 pages

Numerical Example Explanation Q Learning

This document explains a numerical example of Q-learning, detailing the steps involved in calculating the Q-value for moving from state B to goal state F. It outlines the initialization of the Q-matrix, the selection of actions based on a reward matrix, and the application of the Q-learning update formula. The final Q-value for the action B to F is determined to be 100, indicating a high reward for reaching the goal state.

Uploaded by

pardeep9262865788
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Numerical Example Explanation (Q-Learning)

Step 1: Given Information


 Learning parameter (discount factor)
γ =0.8

 Initial state: Room B


 Reward matrix R represents the environment.
 States: A, B, C, D, E, F
 Reward 100 indicates reaching the goal state (F).
 Reward 0 indicates allowed movement.
 Reward – means movement not allowed.

Step 2: Initialize Q-Matrix


 The Q-matrix stores learned values.
 Initially, all Q-values are set to 0:
Q ( s , a )=0 for all states and actions

Step 3: Possible Actions from Initial State B


From row B of reward matrix R:
 Possible actions:
o Go from B → D
o Go from B → F
 Assume random selection, choose action B → F

Step 4: Move to Next State


 After taking action B → F, the agent reaches state F
 Now look at row F of reward matrix:
o From F, agent can go to:
 B
 E
 F (self-loop, goal state)

Step 5: Apply Q-Learning Update Formula


General formula:
' '
Q(s , a)=R (s , a)+ γ max ⁡Q (s , a )

For this case:


 Current state = B
 Action = F
 Next state = F
Q(B , F )=R(B , F)+0.8 ×max ⁡[Q(F , B), Q( F , E) , Q(F , F)]

Step 6: Substitute Values


 From reward matrix:
R(B , F)=100

 Since Q-matrix is initially zero:


Q(F , B)=Q(F , E)=Q(F , F)=0

Q(B , F )=100+0.8 × 0
Step 7: Final Calculation
Q(B , F )=100

Final Interpretation
 Since F is the goal state, it gives a high immediate reward.
 No future reward is added because Q-values are initially zero.
 The agent learns that going from B to F is a very good action.
 This value will later help propagate rewards backward to other states.

One-Line Exam Conclusion


The Q-value for state B and action F becomes 100, indicating that moving from
B directly to the goal state F yields maximum reward.

You might also like