A framework for converting dynamical systems to RL environments.

DEgym addresses the challenge of creating RL environments for process optimization in systems which are described by dynamical systems, in particular differential algebraic equations (DAEs) or ordinary differential equations (ODEs). Traditional RL frameworks require significant domain expertise to bridge the gap between domain specific knowledge (e.g. chemical engineering) and RL implementation. DEgym provides:

• Unified Architecture: Separates RL-specific logic (common across domains) from use-case-specific logic
• DAE/ODE Integration: Native support for differential equations that govern the dynamics of the environment
• Modular Design: Extensible framework with well-defined interfaces for easy customization
• AI-Agent Ready: Rich context and clear interfaces designed for Software 3.0 development
• Production Ready: Gymnasium-compatible environments for seamless integration with RL libraries

DEgym implements a clean separation of concerns:

• RL-Related Logic (implemented once): Data flow, interfaces, step function structure, observation/reward handling
• Use-Case Logic (user implements): Specific reactor dynamics, actions, states, and domain constraints

This separation enables both human developers and AI agents to create new environments systematically by focusing only on the domain-specific aspects.

Perfect for optimizing:

• Chemical Reactors: CSTR, batch reactors, flow reactors.
• Biological Systems: Fermentation processes, metabolic networks, population dynamics model.
• Environmental and Earth Sciences: Energy balance climate models, water reservoir systems, groundwater level dynamics, carbon cycle models.
• Sociology and Psychology: Opinion dynamics models (e.g., DeGroot model), adoption of innovation (Bass model), population-level behavior change models.
• Economics and Financial Systems: Solow-Swan model, dynamics of financial market (e.g., simplified Black-Scholes-Merton model).

Traditional approaches require implementing RL environments from scratch for each reactor type. DEgym provides:

• Faster Development: Focus on dynamics of the system, not RL boilerplate
• Consistent Quality: Proven RL patterns and best practices built-in
• Easy Maintenance: Modular architecture scales across different use-cases
• Research Acceleration: Rapid prototyping for optimization research

DEgym uses uv for package management. Install uv following the uv documentation, then:

# Clone the repository
git clone [email protected]:instadeepai/degym.git
cd degym

# Basic installation (recommended for most users)
uv sync

# Development installation (includes testing and linting tools)
uv sync --group test

To create a new environment with DEgym:

1. Define your environment's DAE/ODE system (mass/energy balances)
2. Subclass the Environment class and implement abstract components
3. Get a gymnasium-compatible environment ready for RL optimization

# Example: CSTR environment usage
from degym_tutorials.cstr_tutorial.make_env import make_cstr_environment

env = make_cstr_environment(config)
obs, info = env.reset()

done = False
while not done:
    action = agent.get_action(obs)  # Your RL agent
    obs, reward, terminated, truncated, info = env.step(action)
    done = terminated or truncated

• Documentation home page: Full documentation website.
• Installation Guide: Comprehensive setup instructions for all use cases
• DEgym Essentials: Core architecture and design principles
• Tutorial: Step-by-step guide to creating custom environments

If you wish to contribute to DEGym, please refer to the contribution guidelines and follow the development installation for setting up the development environment with testing and linting tools.

This is the list of maintainers, who are in charge of triaging issues, reviewing and approving PRs from contributors.

If you use DEgym in your work, please cite:

@misc{degym,
    title={DEgym: A framework for developing reinforcement learning environments for dynamical systems},
    author={Nima H. Siboni, Marco Carobene, Frédéric Renard, Alind Gupta, Miguel Arbesú},
    year={2025},
    url={https://github.com/instadeepai/degym/},
}