Scribd
Upload
2 views3 pages

Reinforcement Learning Environment Wrapper

This document outlines the design and implementation of a custom Reinforcement Learning environment using OpenAI Gym, featuring a GridWorld and a Deep Q-Network (DQN) agent. It details the environment's state and action spaces, dynamics, and reward structure, as well as the DQN agent's architecture and training procedure. Results indicate that the agent improves performance over time, consistently reaching the goal with minimal steps.

Uploaded by

kowshiksreebahadursha2005
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
2 views3 pages

Reinforcement Learning Environment Wrapper

This document outlines the design and implementation of a custom Reinforcement Learning environment using OpenAI Gym, featuring a GridWorld and a Deep Q-Network (DQN) agent. It details the environment's state and action spaces, dynamics, and reward structure, as well as the DQN agent's architecture and training procedure. Results indicate that the agent improves performance over time, consistently reaching the goal with minimal steps.

Uploaded by

kowshiksreebahadursha2005
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Reinforcement Learning Environment

Wrapper (GridWorld + DQN)

1. Introduction

This project demonstrates the design and implementation of a custom Reinforcement Learning (RL)
environment using OpenAI Gym and the application of a Deep Q-Network (DQN) agent to solve the
environment. The objective is to build an end-to-end RL pipeline that includes environment
modeling, agent training, and performance evaluation.

Reinforcement learning is particularly suited for sequential decision-making problems where an


agent learns optimal behavior through interaction with an environment. This project focuses on
clarity, correctness, and stability rather than environment complexity.

2. Environment Design

The environment is a GridWorld, represented as a two-dimensional grid of fixed size.

2.1 State Space

 The observation space consists of the agent’s current position in the grid.

 State is represented as a tuple (x, y) corresponding to the agent’s coordinates.

2.2 Action Space

The action space is discrete with four possible actions:

1. Move Up

2. Move Down

3. Move Left

4. Move Right

Boundary conditions ensure the agent remains within the grid.

2.3 Environment Dynamics

 The grid contains fixed obstacles that penalize the agent if encountered.

 A terminal goal state ends the episode when reached.

2.4 Reward Function

 +10 for reaching the goal

 −0.05 for each step to encourage shorter paths

 −1 for colliding with an obstacle

This reward structure balances exploration and efficiency.


3. DQN Agent Implementation

The agent is implemented using PyTorch and follows the standard DQN framework.

3.1 Neural Network Architecture

 Fully connected feedforward network

 Two hidden layers with ReLU activation

 Output layer predicts Q-values for each action

3.2 Key Components

 Experience Replay Buffer: Stores past transitions and samples random batches to reduce
correlation.

 Target Network: A periodically updated copy of the policy network to stabilize training.

 ε-greedy Policy: Controls exploration and exploitation with gradual epsilon decay.

4. Training Procedure

The agent is trained over multiple episodes. In each episode:

1. The environment is reset.

2. The agent selects actions based on the ε-greedy policy.

3. Transitions are stored in replay memory.

4. The DQN is updated using mini-batch gradient descent.

5. The target network is updated at fixed intervals.

Training performance is tracked using the total reward per episode.

5. Results and Evaluation

 Initial episodes show high variance due to exploration.

 As training progresses, rewards increase steadily.

 The smoothed reward curve stabilizes near the maximum achievable reward, indicating
convergence.

 The trained agent consistently reaches the goal with minimal steps.

6. Technologies Used

 Python 3

 OpenAI Gym

 PyTorch

 NumPy

 Matplotlib
7. Results

6. Technologies Used

 Python 3

 OpenAI Gym

 PyTorch

 NumPy

 Matplotlib

You might also like