AMD Schola is a library for developing reinforcement learning (RL) agents in Unreal® Engine, and training with your favorite python-based RL frameworks: Gym, RLlib and Stable Baselines 3.
Download the latest version - v2.1.0
What’s new in AMD Schola
AMD Schola v2.1.0 is a major feature and tooling release since v2.0.1, with new simulation/training capabilities, expanded CLI workflows, and broad improvements to testing and docs.
Highlights
- Added StateTree integration for training and inference.
- Added ExternalSimulator support and Kubernetes distributed training assets.
- Expanded imitation learning support and Minari-related workflows.
- Added eval commands for both RLlib and SB3 CLIs.
- Added YAML config support for training CLIs.
- Improved multi-agent RLlib support with agent type mappings to control how agents are mapped to policies.
- Added frame stacking utilities (e.g.,BoxStacker, DictStacker).
- Added project simulator to build and train with a single command.
- Added support spawning multiple processes when training from the cli.
- Updated ONNX export flow and fixed export issues.
Python SDK & CLI
- Refactored protocol and simulator layers (including async protocol support for sb3).
- Reorganized and expanded command modules for:
- RLlib train/eval
- SB3 train/eval
- Minari data collection
- Modernized Python packaging (pyproject-based setup).
- Dependency updates:
- grpc -> 1.80.0
- protoc -> 31.1
- ray[rllib] constraint updated to <2.55, >=2.49
Unreal plugin / C++
- Broad updates across core Source/Schola* modules.
- New and improved actuator/sensor behavior and tests.
- Logging and profiling improvements across runtime/editor modules.
- Expanded C++ test coverage and infrastructure.
Testing and docs
- Significant test expansion across core, gym, RLlib, SB3, Minari, and scripts.
- New docs and guides for distributed training, CLI conventions, and pytest workflows.
Breaking/migration notes
- Python module and script path refactors may require updating imports in custom tooling.
- CLI command layout evolved (including new eval commands and YAML configuration support).
- Dependency bumps (gRPC/Protobuf/RLlib) may require environment re-pinning.
Prerequisites
- Unreal® Engine 5.5+ (tested with 5.5 - 5.7)
- Python 3.10-3.12
- Microsoft Visual Studio® 2022 with MSVC v143 build tools (Windows®)
Compatibility
| 2.1.x | 5.5-5.7 | 3.10-3.12 | ✅ Current |
| 2.0.x | 5.5-5.6 | 3.10-3.12 | legacy |
| 1.3 | 5.5-5.6 | 3.9-3.11 | Legacy |
| 1.2 | 5.5 | 3.9-3.11 | Legacy |
Features
Inference in C++
Schola provides tools for connecting and controlling agents with ONNX models inside Unreal Engine, allowing for inference with or without Python.
Simple Unreal interfaces
Schola exposes simple interfaces in Unreal Engine for the user to implement, allowing you to quickly build and develop reinforcement learning environments.
Modular components
Environments in Schola are modular so you can quickly design new agents from existing components, such as sensors and actuators.
Multi-agent training
Train multiple agents to compete against each other at the same time using RLlib and multi-agent environments built with Schola.
Vectorized training
Run multiple copies of your environment within the same Unreal Engine process to accelerate training.
Headless training
Run training without rendering to significantly improve training throughput.
AMD Schola v1.3 sample environments
Basic
The Basic environment features an agent that can move in the X-dimension and receives a small reward for going five steps in one direction and a bigger reward for going in the opposite direction.
MazeSolver: Using raycasts
The MazeSolver environment features a static maze that the agent learns to solve as fast as possible. The agent observers the environment using raycasts, moves by teleporting in 2 dimensions and is given a reward for getting closer to the goal.
3DBall: Physics based environments
The 3DBall environment features an agent that is trying to balance a ball on-top of itself. The agent can rotate itself and receives a reward every step until the ball falls.
BallShooter: Building your own actuator
The BallShooter environment features a rotating turret that learns to aim and shoot at randomly moving targets. The agent can rotate in either direction, and detects the targets by using a cone shaped ray-cast.
Pong: Collaborative training
The Pong environment features two agents playing a collaborative game of pong. The agents receive a reward every step as long as the ball has not hit the wall behind either agent. The game ends when the ball hits the wall behind either agent.
Tag: Competitive multi-agent training
The Tag environment features a 3v1 game of tag, where one agent(the runner) has to run away from the other agents which are trying to collide with it. The agents move using forward, left and right movement input, and observe the environment with a combination of ray-casts and global position data.
RaceTrack: Controlling chaos vehicles with Schola
The RaceTrack environment features a car implemented with Chaos Vehicles, that learns to follow a race track. The agent controls the throttle, break and steering of the car, and can see it’s velocity and position relative to the center of the track.
Additional resources
Endnotes
Unreal® is a trademark or registered trademark of Epic Games, Inc. in the United States of America and elsewhere.
“Python” is a trademark or registered trademark of the Python Software Foundation.