| Environments | Description | Algorithms |
| 3DBall: 3D Balance Ball | A balance-ball task, where the agent balances the ball on it’s head. | PPO & SAC |
| GridWorld | A multi-goal version of the grid-world task. Scene contains agent, goal, and obstacles. The agent must navigate the grid to the appropriate goal while avoiding the obstacles. | PPO & SAC |
| PushBlock | A platforming environment where the agent can push a block around. | Imitation & PPO & SAC |
| Wall Jump | A platforming environment where the agent can jump over a wall. | PPO & SAC |
| Crawler | A creature with 4 arms and 4 forearms. | Imitation & PPO & SAC |
| Worm | A worm with a head and 3 body segments. | PPO & SAC |
| Food Collector | A multi-agent environment where agents compete to collect food. | PPO & SAC |
| Hallway | Environment where the agent needs to find information in a room, remember it, and use it to move to the correct goal. | Imitation & PPO & SAC |
| Soccer Twos | Environment where four agents compete in a 2 vs. 2 toy soccer game. | MA-POCA |
| Strikers Vs. Goalie | Environment where two agents compete in a 2 vs. 1 soccer variant. | PPO & SAC |
| Walker | Physics-based Humanoid agents with 26 degrees of freedom. The agents must move its body toward the goal direction without falling. | PPO & SAC |
| Pyramids | Environment where the agent needs to press a button to spawn a pyramid, then navigate to the pyramid, knock it over, and move to the gold brick at the top. | Imitation & PPO & SAC |
| Match 3 | Simple match-3 game. Matched pieces are removed, and remaining pieces drop down. New pieces are spawned randomly at the top, with a chance of being “special”. | PPO |
| Sorter | The Agent is in a circular room with numbered tiles placed randomly. The agent visits all the tiles in ascending order. | PPO |
| Cooperative Push Block | Similar to Push Block, the agents are in an area with blocks that need to be pushed into a goal. | MA-POCA |
| Dungeon Escape | Agents are trapped in a dungeon with a dragon, and must work together to escape. The goal is to unlock the dungeon door and leave. | MA-POCA |