The Next Monty 3D Environment Simulator

[tslominski]

This is an overview of the state of Monty’s 3D environment simulators as of December, 2024. It discusses existing constraints and highlights where contributions would be particularly helpful.

At a high level, the Monty framework consists of an experiment which coordinates the flow of data between Monty and a simulated environment.

As a sensorimotor learning system, Monty continuously generates motor output during training and inference. The simulated environment processes this motor output and provides sensory input back to Monty.

The following Wardley Map depicts Monty’s general dependency on simulators. If Wardley Maps are new, just ignore the special symbols and the Evolution axis.

Specifically, as of December, 2024, Monty depends on the Habitat Sim simulator.

The Habitat Sim version that Monty depends on is version 0.2.2. The latest available version is 0.3.2.

We’ll return to Habitat Sim shortly, but first, a quick digression on the state of Python version support as of December, 2024.

Depicted are the Python versions from 3.8 through 3.14. The version 3.8 is now obsolete and, as a result, shown in the Custom stage of Evolution. Versions 3.9 through 3.13 are actively supported and shown in the Commodity stage of Evolution. The 3.14 version is in development and will be released in October, 2025, when it will enter the Commodity stage of Evolution. Lastly, version 3.9 is shown becoming obsolete and entering the Custom stage in October, 2025.^[1]

Back to Habitat Sim, it is noteworthy to highlight a dependency constraint between the Habitat Sim versions and the Python versions. The Habitat Sim version 0.2.2 is coupled to the Python 3.8 Application Binary Interface (ABI). The latest Habitat Sim version 0.3.2, is coupled to the Python 3.9 ABI. These constraints are depicted in the figure below.

In summary, Monty depends on Habitat Sim for a simulated environment. The version of Habitat Sim used by Monty depends on an obsolete Python version 3.8. Upgrading to the latest Habitat Sim version will couple Monty to Python 3.9, which will be obsolete in October, 2025.

There is interest from the Thousand Brains Project community to run Monty on Windows OS. This is challenging because Habitat Sim does not run on Windows OS. Additionally, Monty code has unintentional coupling to Habitat Sim that effectively limits viable operating systems on which Monty runs to Linux and Mac OS.

While the work to remove the unintentional coupling to Habitat Sim is ongoing, even with the coupling removed, there is no example of support for a simulator that runs on Windows OS.

I believe it is time to consider a simulator that runs on Windows OS and does not limit Monty to the oldest Python versions.

Some available simulators are listed in the figure below: PyBullet, MuJoCo, Webots, Unreal Engine, NVIDIA Isaac Sim, Unity.

Of additional note is the Robot Operating System (ROS) which can serve as a common interface to some of the simulators listed.

Among these options, I think MuJoCo is the most promising to evaluate. As I don’t have direct experience with it, any knowledgeable advice or guidance will be helpful. If you want to take the lead, I can contribute to a Request For Comments (RFC) where we’d get into evaluation and integration details. A prototype implementation would also be helpful.

If you think Monty should support a different simulator, that will be helpful as well. Supporting more than one simulator will serve as a forcing function to ensure no unintentional coupling.

---

1. Status of Python versions. Python Developer’s Guide. Retrieved December 8, 2024, from Status of Python versions ↩︎

[DLed]

great to see Wardley Maps as a supporting artefact to the discussion! To my brain the maps tend to be a bit overcrowded, although, the multi-aspect information density is quite high. It definitely needs some time getting used to.

As for the reasoning - it sounds quite solid. If coupling is too tight for the wanted progress, a de-coupling though an abstraction might be an option, probably, speeding up the shift to commodity, as a world simulator is likely not Monty’s core domain. Depending on the bandwidth of communication with Monty, decoupling of the run-times through distribution might be an option.

MuJoCo sounds reasonable, providing explicit bindings that could be patched to the desired Python version - potentially easier than safely porting (or waiting for a port of) a large code-base between python versions.

[Sieme]

While I’m biased towards Gazebo, because I would like to see training in simulation transferred to the real world, on real hardware, i.m.o. Monty should not be tied to any specific 3D sim environment.

What about having wrappers towards multiple simulation environments?

[DLed]

another aspect of the options: the constraint “Habitat Sim doesn’t run on Windows”, as discussed in the Github issue, may be overcome by using e.g. WSL2. Another layer on top could be the use of Dev Containers which would potentially simplify the onboarding experience by taking care of the environment in a unified way. Both PyCharm and VSCode seem to support them.

Previously one would use Vagrant for this, but nowadays, Dev Containers are probably a sensible lightweight choice.

[rolly]

Habitat Sim also doesn’t run on the Raspberry Pi OS.

[DLed]

Considering, RPis are not machines designed for simulation tasks, perhaps, not running the world simulator there is not the biggest loss. If, instead, it’d be collocated with a number of sensors, in a robotics application, perhaps, habitat sim wouldn’t even be needed. Reasonable?

[rolly]

All well & good. Adding a new sensor app with a part that runs where Monty has its world simulator, and part in a Raspberry Pi (or Windows machine). The same could be done for a Monty motor app. This solves the problem of getting a simulator that runs on all the useful platforms.
This local/remote sensor & motor apps is something I want to have a try at. I’ve been rummaging thru the code to look for cleaving points. @Doonhammer’s suggestion for a Distributed CMP (https://thousandbrains.discourse.group/t/possibility-of-sharing-monty-learnings/206) might be a good vehicle for transporting info to and fro

[tslominski]

I had Gazebo on a list but omitted it due to a lack of Windows support. However, having more than one simulator is still beneficial to designing the wrappers for the simulation environments. So, if you’re interested in implementing Gazebo support, that would be helpful.

Two interface points—one from Monty’s Motor System to the simulator and one from the simulator back to Monty—provide context for implementing another simulator.

Regarding the Motor System to the simulator code. The HabitatActuator is an example of an Actuator that interfaces between Actions generated by the Motor System and Habitat-specific action implementation.

Regarding how the data is read from the simulator, see the current implementation of EnvironmentDataset.__getitem__:

    def __getitem__(self, action: Action):
        observation = self.env.step(action)
        state = self.env.get_state()
        if self.transform is not None:
            observation = self.apply_transform(self.transform, observation, state)

        return observation, state

Additional points in the code would need updates, but hopefully, these two provide some orientation for getting started.

[tslominski]

Thanks, @DLed. These are all good ideas.

One challenge that I am uncertain about is the simulator’s ability to interface with the graphics card hardware. HabitatSim, for example, has a graphics hardware expectation/dependency. When I attempted bypassing the hardware dependency using software emulation in our GitHub Actions runners, it became challenging to find dependency configurations that work. Specifically, my initial attempts to upgrade the HabitatSim or Ubuntu versions have resulted in non-working configurations.

If dev containers and VMs can help us bypass this challenge, they would greatly help.

[Sieme]

I spent several hours on WSL2 and at least on my machine graphics was a pain (opengl, nvidia drivers, etc). You also have to run a windows X server (like VcXsrv) and get all the configuration right.
I just decided to make my laptop dual boot with ubuntu.

[Sieme]

Thank you Tristan for the thorough reply.
But i’m not the one for that at the moment, didn’t want to imply.

[tslominski]

Oh, no worries @Sieme . Maybe it speaks to someone else who comes across it.

[tslominski]

As @elpizo pointed out today, Genesis is a noteworthy entrant into the simulator space. I added it to the TBP Monty Simulators Wardley Map.