Introduction

Developing reinforcement learning (RL) agents often feels like fighting the environment as much as the algorithm. For years, researchers struggled with fragmented APIs and inconsistent environment implementations that made comparing results nearly impossible. Gymnasium, a community-maintained successor to the legendary OpenAI Gym, solves this by providing a standardized interface for single-agent RL environments. With its widespread adoption across the AI research community and its role as the foundation for libraries like Stable Baselines3 and RLlib, Gymnasium has become the de-facto standard for building and testing RL agents.

What Is Gymnasium?

Gymnasium is an open-source Python library that provides a standard API for single-agent reinforcement learning environments, including a diverse set of reference environments and related utilities. Maintained by the Farama Foundation, it is a fork of the original OpenAI Gym, designed to ensure the long-term stability and community-driven development of the RL ecosystem. Licensed under the MIT License, it allows developers to create, interact with, and benchmark RL agents in a consistent way across different tasks.

At its core, Gymnasium abstracts the interaction between an agent and its environment into a simple loop: the agent observes the state, takes an action, and the environment returns a new state, a reward, and a termination signal. This abstraction allows researchers to swap environments without rewriting their agent’s logic, facilitating rapid prototyping and rigorous benchmarking.

Why Gymnasium Matters

Before Gymnasium, the RL community faced a “standardization gap.” Many researchers implemented their environments in custom ways, which meant that an algorithm that performed well in one environment could not be easily tested on another. This lack of consistency hindered the reproducibility of research and slowed the progress of the field. Gymnasium fills this gap by enforcing a strict API contract that all environments must follow.

The transition from OpenAI Gym to Gymnasium was critical because OpenAI stopped prioritizing the maintenance of the original Gym library. By moving the project to the Farama Foundation, the community gained a maintained, stable, and updated version of the API that supports modern Python versions and updated dependencies like NumPy 2.0. This ensures that the RL infrastructure remains production-stable and compatible with the latest hardware and software stacks.

For developers, investing time in Gymnasium is essential because it is the primary target for almost every major RL library. If you are using Stable Baselines3, Ray RLlib, or CleanRL, you are likely already using Gymnasium under the hood. Learning the Gymnasium API is the single most important step for anyone looking to implement RL algorithms from scratch or use existing high-level frameworks.

Key Features

• Standardized API: Provides a consistent interface (make(), reset(), step()) that allows agents to interact with any Gymnasium-compatible environment without changing the agent’s code.
• Diverse Reference Environments: Includes a wide array of built-in tasks, from simple “Classic Control” problems like CartPole and MountainCar to complex physics simulations in MuJoCo and Atari 2600 games.
• Vectorized Environments: Supports running multiple copies of the same environment in parallel via make_vec(), significantly increasing the throughput of experience collection for on-policy algorithms.
• Robust Observation and Action Spaces: Uses a formal system of Space objects (e.g., Box, Discrete, Tuple) to define exactly what an agent can see and do, ensuring type safety and easy validation.
• Environment Wrappers: Provides a powerful system of wrappers that allow developers to modify environment behavior (e.g., changing rewards, capping observations) without altering the original environment’s source code.
• Reproducibility Tools: Implements strict seeding mechanisms in reset(seed=...) to ensure that experiments are reproducible across different runs and platforms.
• MIT License: Distributed under a permissive license that is suitable for both academic research and commercial product development.

How Gymnasium Compares

The primary differentiator between Gymnasium and its predecessor, OpenAI Gym, is the API refinement. In legacy Gym, the step() function returned a single done flag. Gymnasium replaces this with two distinct signals: terminated (when the agent reaches a goal or fails) and truncated (when the episode is cut short by a time limit). This distinction is mathematically critical for correctly implementing value-function based RL algorithms, as it prevents the agent from confusing a terminal state with a time-out.

Compared to Unity ML-Agents, Gymnasium is far more lightweight and focused on the API standard rather than the simulation engine. While Unity provides a high-fidelity 3D environment, Gymnasium provides the interface that allows you to connect any simulation (including Unity) to an RL agent. Gymnasium is the better choice for researchers who need a standardized, lightweight Python-first approach to benchmarking and algorithm development.

Getting Started: Installation

Gymnasium can be installed via pip. Depending on your needs, you can install the core library or include specific environment families (extras) to avoid downloading unnecessary dependencies.

Core Installation

To install the base Gymnasium library with no reference environments:

pip install gymnasium

Installing Environment Families

To install specific sets of environments, use the following commands:

• Classic Control: pip install "gymnasium[classic-control]"
• Box2D: pip install "gymnasium[box2d]"
• Atari: pip install "gymnasium[atari]"
• MuJoCo: pip install "gymnasium[mujoco]"

Full Installation

To install everything, including all reference environments and their dependencies:

pip install "gymnasium[all]"

Prerequisites: Gymnasium supports Python 3.10, 3.11, 3.12, and 3.13 on Linux and macOS. While Windows is supported via community PRs, it is not officially supported by the Farama Foundation.

How to Use Gymnasium

Interacting with a Gymnasium environment follows a consistent loop. First, you create an environment using gym.make(), then you reset it to start a new episode, and finally, you step through the environment using env.step() until the episode ends.

The basic workflow is as follows:

1. Initialize: Create the environment instance with a specific ID (e.g., 'CartPole-v1').
2. Reset: Call env.reset() to get the initial observation and an info dictionary.
3. Act: The agent chooses an action based on the current observation.
4. Step: Call env.step(action) to apply the action and receive the next observation, reward, termination signals, and info.
5. Repeat: Continue this loop until terminated or truncated is True.

If you want to visualize the agent’s behavior, you can set the render_mode argument in gym.make() to 'human' or 'rgb_array'.

Code Examples

Below are examples of how to use Gymnasium, ranging from a simple random agent to a vectorized environment setup.

Basic Random Agent Loop

This example shows the most basic interaction loop with the CartPole environment.

import gymnasium as gym

env = gym.make("CartPole-v1", render_mode="human")
observation, info = env.reset(seed=42)

for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        observation, info = env.reset()

env.close()

Creating a Custom Environment

Gymnasium allows you to define your own environments by inheriting from gym.Env. You must define the observation_space, action_space, reset(), and step() methods.

import gymnasium as gym
from gymnasium import spaces
import numpy as np

class MyEnv(gym.Env):
    def __init__(self):
        super().__init__()
        # Define action space: 0 = left, 1 = right
        self.action_space = spaces.Discrete(2)
        # Define observation space: a single float between 0 and 1
        self.observation_space = spaces.Box(low=0, high=1, shape=(1,), dtype=np.float32)

    def reset(self, seed=None, options=None):
        super().reset(seed=seed)
        # Reset the environment to an initial state
        self.state = np.array([0.5], dtype=np.float32)
        return self.state, {}

    def step(self, action):
        # Apply the action and update state
        self.state += 0.1 if action == 1 else -0.1
        # Calculate reward
        reward = 1.0 if 0.4 < self.state[0] < 0.6 else 0.0
        # Check for termination
        terminated = True if self.state[0] < 0 or self.state[0] > 1 else False
        truncated = False
        
        return self.state, reward, terminated, truncated, {}

env = gym.make("MyEnv-v0")
# Note: You would need to register the environment using gym.register()

Real-World Use Cases

Gymnasium is used across various domains where decision-making agents are need to be trained in simulation before deployment to real-world hardware.

• Robotics Control: Researchers use MuJoCo environments in Gymnasium to train controllers for humanoid robots or robotic arms. By training in simulation first, they can avoid damaging expensive hardware during the early stages of learning.
• Autonomous Driving: Developers use Gymnasium-compatible environments (like HighwayEnv) to train agents to change lanes or merge into traffic, providing a safe, reproducible way to test driving policies.
• Financial Trading: Quantitative analysts use Gymnasium to build custom environments that simulate stock market data, allowing them to train RL agents to optimize trading strategies and manage risk.
• AI Benchmarking: AI labs use the Atari 2600 suite in Gymnasium to test new RL algorithms against a set of standard baselines, ensuring that their new method is truly an improvement over existing state-of-the-art agents.

Contributing to Gymnasium

Gymnasium is a community-driven project. If you find a bug or want to improve the documentation, the Farama Foundation encourages contributions through GitHub.

To contribute, you can report bugs via the GitHub Issues tab or submit a pull request for bug fixes and documentation improvements. While the project maintainers generally do not accept new reference environments into the core library to prevent bloat, they encourage developers to publish their environments as separate packages that adhere to the Gymnasium API standard.

All contributors are expected to follow the project’s Code of Conduct to maintain a professional and community-focused environment.

Community and Support

Gymnasium is supported by the Farama Foundation, which is a non-profit organization dedicated to maintaining RL libraries. Because it is part of a larger ecosystem, support is primarily handled through GitHub Discussions and the Farama Foundation’s official channels.

The project provides comprehensive documentation at gymnasium.farama.org, which includes a la a basic usage guide, API reference, and migration guides for those moving from OpenAI Gym.

The community is vast, as Gymnasium is the foundation for most modern RL research. You can find extensive support through the RL community on X (Twitter), research forums, and the associated libraries like Stable Baselines3.

Conclusion

Gymnasium is more than just a library; it is the essential infrastructure for modern reinforcement learning. By providing a standardized API, it has transformed the way RL agents are developed, tested, and benchmarked. Whether you are a student learning the basics of RL or a researcher developing the next generation of AI, Gymnasium is the right choice for your environment interface.

If you are still using OpenAI Gym, the recommendation is clear: migrate to Gymnasium immediately. The API changes are minor but mathematically significant, and the community support and maintenance are now centered around this project. Star the repo, try the quickstart, and join the community of RL developers.

Resources

• Gymnasium GitHub Repository
• Official Gymnasium Gymnasium Documentation
• Gymnasium White Paper (arXiv)