Linux-aarch64 support for Monty (habitat-sim 0.3.3, Python 3.12)

Monty Code
1

Hi folks,

I’ve been working on porting Monty to run natively on linux-aarch64 (ARM64 Linux). No emulation, no Rosetta. This required upgrading from habitat-sim 0.2.2 / Python 3.8 to habitat-sim 0.3.3 / Python 3.12, plus updating the full dependency stack. I wanted to share what I have and see if there’s interest before opening a formal PR.

What exists today: two branches, across two repos:

1. habitat-sim: GitHub - cherron/habitat-sim at monty-compat-fixes · GitHub , based on v0.3.3. Adds linux-aarch64 build support and two bug fixes Monty needs:

- EGL context creation fix for non-CUDA systems (Mesa/llvmpipe)

- sim.reset() fix where API-added rigid objects became invisible after reset

2. tbp.monty — GitHub - cherron/tbp.monty at linux-aarch64 · GitHub , based on v0.27.0. Two commits covering:

- Dependency upgrades: Python ≥3.12, PyTorch ≥2.0, NumPy 2.x, scipy ≥1.14, scikit-learn ≥1.4, PyG 2.7

- habitat-sim 0.2.2 → 0.3.3 API changes (subtree_sensors, contact_test removal, headless rendering)

- NumPy 2.x compat (0-d array truth values, complex eigenvalue handling in curvature computation, repr changes)

- PyTorch 2.x compat (torch.load weights_only)

- PyG 2.7 compat (Data.keys became a method)

- Multiprocessing fix (fork → spawn for OpenGL context safety)

- Float tolerance fixes for aarch64 (Bullet physics scalar vs SSE precision)

Test results with these changes (linux-aarch64, llvmpipe software rendering):

All 583 tests pass (391 unit, 60 config, 132 integration, 0 failures). The randrot_noise_10distinctobj_surf_agent benchmark completes with 100/100 episodes correct (90 exact match, 10 correct most-likely-hypothesis).

Scope of changes:

22 files changed, ~150 lines of insertions against ~80 lines of deletions (plus 5,700 lines from removing the stale uv.lock). The changes are mechanical — no architectural changes, no new features, no behavior changes introduced on x86_64.

Dependencies / Blockers:

The Monty port depends on the habitat-sim fixes. I have a PR: Add linux-aarch64 build support by cherron · Pull Request #2626 · facebookresearch/habitat-sim · GitHub submitted for the build support, and the two Monty-specific bug fixes are on my fork branch. Until those land upstream, the Monty port would need to pin habitat-sim to my fork (cherron/habitat-sim@monty-compat-fixes).

Disclosure: This work was done with heavy assistance from Claude Code.

Questions for the community

  • Is there interest in linux-aarch64 support?

  • Any concerns about the dependency version jumps (especially Python 3.8 → 3.12, habitat-sim 0.2.2 → 0.3.3)?

  • Preferred approach for the habitat-sim dependency — wait for upstream, or accept a fork pin for now? Its unclear if the PR will be accepted - there is no movement on the receiving end.

  • Is there any policy or preference on AI-assisted contributions that I should be aware of?

I’m happy to answer questions, adjust the approach, or walk through any of the changes in detail.

Thanks,

Chris

4 Likes
2

That’s some fine work. You even corrected the gpu_device_id problem I circled around as part of my WSL adventures.

In the issue below, @tslominski talked about concerns regarding building Habitat-Sim from source, and mentioned the goal of moving toward MuJoCo instead, with an eventual migration to Python 3.10 (not sure why 3.10 specifically tho, end-of-life is in 6 months):

In particular, he also mentioned “Habitat version that’s distributed for Python 3.9 breaks a bunch of stuff”. Your unit tests seem successful, so it’s possible those issues might be resolved by your changes. I guess the team would have to test your fork on macOS to confirm.

It should also be noted that Habitat-Sim maintainers have recently upgraded to Python 3.10-3.12, which we can assume will eventually be part of a future 0.3.4 release. However, their maintainers do not seem responsive to PRs, so I wouldn’t hold my breath over your Habitat PR if I were you.

In any case, I think your Monty-sided changes would make sense once Habitat-Sim does get that 0.3.4 release. In which case, Monty’s Python could technically be upgraded (if nobody expects the Spanish Inquisition :wink:) and linux-aarch64 users could be instructed to build your Habitat fork as needed.

1 Like
3

@AgentRev covered a lot of @tslominski and my thinking on this, so I don’t have much to add there.

not sure why 3.10 specifically tho, end-of-life is in 6 months

Ideally, we want to get to the latest version of Python, but we’re going to take it step by step to make sure nothing breaks, since different library versions required different Python versions. Currently we’ve been targeting 3.9, which is already EOL, with an intent to jump to 3.10 or later immediately after that.

We made a lot of progress (more than I was expecting) on the MuJoCo simulator at our retreat in London a couple of weeks ago, getting so far as implementing a prototype of the distant and surface agents, and getting Monty to recognize objects with a high degree of accuracy in the experiments we were testing with, using training data from Habitat, which is really exciting. There’s more work to be done since it’s a prototype, cleaning up the code and updating tests and benchmarks, but I think now we have a very clear path forward.

2 Likes
4

I see, that makes a lot of sense. Good job with MuJoCo, I noticed those draft PRs. MuJoCo supports Linux-aarch64, so that would solve the problem.

Would Habitat also be upgraded to 0.3.3? Or is the Habitat code getting externalized in a separate repo like @nunoo’s work? Or are you thinking about abandoning Habitat altogether?

5

@tslominski can correct me if I’m wrong, but the plan as I understand it is to replace Habitat with MuJoCo. Habitat seems less supported and they don’t seem to be publishing packages to PyPI. We’re wanting to move away from using Conda and Habitat is one of the things keeping us on it.

2 Likes
The Next Monty 3D Environment Simulator
6

I have no notes corrections to what @jshoemaker stated :slight_smile:.

For additional details of why we’re chosing to remove Habitat and adopt MuJoCo, see The Next Monty 3D Environment Simulator. While it’s an old post, the reasons are largely unchanged.

2 Likes
7

Thanks everybody. I was motivated to get a running setup on macOS or containerized ARM64 linux, and wasn’t aware of the intended direction towards MuJoCo. I shall watch for MuJoCo to make its way onto main.

2 Likes
8

No worries, the discussion was scattered in multiple threads, and the removal of Habitat wasn’t fully confirmed before Tristan’s last reply. Your other changes could prove useful nonetheless. :slightly_smiling_face:

1 Like