Scribd
Upload
6 views1 page

Program 10

The document outlines a Q-learning algorithm implemented in Python using NumPy and Matplotlib. It defines parameters for a grid-based environment, executes the learning process over multiple epochs, and updates a Q-table based on exploration and exploitation strategies. Finally, it visualizes the learned Q-values in a grid format and prints the Q-table to the console.

Uploaded by

rakshaakash2111
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
6 views1 page

Program 10

The document outlines a Q-learning algorithm implemented in Python using NumPy and Matplotlib. It defines parameters for a grid-based environment, executes the learning process over multiple epochs, and updates a Q-table based on exploration and exploitation strategies. Finally, it visualizes the learned Q-values in a grid format and prints the Q-table to the console.

Uploaded by

rakshaakash2111
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

1 import numpy as np

2 import [Link] as plt


3
4 # Parameters
5 n_states = 16
6 n_actions = 4
7 goal_state = 15
8
9 Q_table = [Link]((n_states, n_actions))
10
11 learning_rate = 0.8
12 discount_factor = 0.95
13 exploration_prob = 0.2
14 epochs = 1000
15
16 # Q-learning process
17 for epoch in range(epochs):
18 current_state = [Link](0, n_states)
19
20 while current_state != goal_state:
21
22 # Exploration vs. Exploitation (ϵ-greedy policy)
23 if [Link]() < exploration_prob:
24 action = [Link](0, n_actions)
25 else:
26 action = [Link](Q_table[current_state])
27
28 # Transition to the next state (circular movement for simplicity)
29 next_state = (current_state + 1) % n_states
30
31 # Reward function (1 if goal_state reached, 0 otherwise)
32 reward = 1 if next_state == goal_state else 0
33
34 # Q-value update rule (TD update)
35 Q_table[current_state, action] += learning_rate * \
36 (reward + discount_factor * [Link](Q_table[next_state]) - Q_table[current_state,
action])
37
38 current_state = next_state # Update current state
39
40 # Visualization of the Q-table in a grid format
41 q_values_grid = [Link](Q_table, axis=1).reshape((4, 4))
42
43 # Plot the grid of Q-values
44 [Link](figsize=(6, 6))
45 [Link](q_values_grid, cmap='coolwarm', interpolation='nearest')
46 [Link](label='Q-value')
47 [Link]('Learned Q-values for each state')
48 [Link]([Link](4), ['0', '1', '2', '3'])
49 [Link]([Link](4), ['0', '1', '2', '3'])
50 [Link]().invert_yaxis() # To match grid layout
51 [Link](True)
52
53 # Annotating the Q-values on the grid
54 for i in range(4):
55 for j in range(4):
56 [Link](j, i, f'{q_values_grid[i, j]:.2f}', ha='center', va='center', color='black')
57
58 [Link]()
59
60 # Print learned Q-table
61 print("Learned Q-table:")
62 print(Q_table)

You might also like