Brainstorming API ideas for Monty

[AdamLD]

Hi,

I’ve read somewhere in the threads that a refactoring of the codebase is being considered.

As a new person getting into the project, I’d like to offer my perspective and ideas.

I know it will take months if the refactoring is even done. So my hope is just to start a discussion, and brainstorm some ideas beforehand.

So as someone new getting started with Monty, here is how I would have liked the API to be:

# Create a new untrained AI model
ai = AI(num_learning_modules=10) 

# Or load one that is saved on file
ai = AI().load('/pretrained/my_pretrained_ai.monty')

# Then we can add agents to the AI (and we can add different agents to the same trained model, depending on what we want to do)
eye = EyeAgent()
eye.add_vision_sensor(num_patches=10, patch_resolution=0.5) 

finger = FingerAgent()
finger.add_touch_sensor(num_patches=20, patch_area=0.05, pressure_threshold=0.1) 
finger.add_actuator(MotorAgent()) 

ear = EarAgent()
ear.add_sound_sensor(num_patches=15, frequency_range=(20, 20000), sensitivity=0.8) 

# Add the agents to the AI
ai.add_agents(eye, finger, ear)

# Set up the environment (in meters)
env = Environment(width=10, height=10, depth=10) 
env.add_AI(ai, xyz = [5, 5, 0]) 
env.add_object("cup", xyz = [6, 5, 0]) 
env.add_object("chair", xyz = [3, 4, 0]) 

# This can show the actual 3D habitat on our screen
insights = Insights()
insights.show_environment(env)
insights.show_benchmarks(ai)

# Continuous interaction loop (or with a limited number of steps). 
while True:
    # Runs all the new input from the sensor modules throught he learning modules, and generates the motor output instructions.
    ai.observe_environment() 

    # Actually have the motor agents follow the motor instructions, and interact with the environment.
    ai.act() 

# Once done, save the new learning modules data to file.
ai.save('/pretrained/my_pretrained_ai_v2.monty')

---

And the users can build their custom version of these classes, extending the base classes, and specifying all their configuration in the custom classes. So the main code would be more like this:

ai = MyAI() # Loads model, and agents
env = MyEnvironment(ai) 

while True:
    # Observes and acts at the same time,
    # so it's one "tick" in the system.
    ai.tick() 

ai.save() # Save the fined tuned model

---

So the principles I hope will be considered are:

1. The separation of business logic with the implementation details. (I believe having the code show both business logic and implementation details in the same methods is a distraction and wastes mental energy.)

2. The modules are separated from a conceptual point of view. The AI class doesn’t know about the Environment class. The Insights class or Benchmarks (plotting the environment on our screens) should be separated from the SDK as well.

3. Don’t have the system run in “epochs” at a higher level. Why? Because ideally I would have Monty run all the time, non stop (once we figure out the kinks).

   Use case: I would want to just start Monty and let it run on its own. In that time, I can go for a walk, go shopping, wash the dishes, etc. (lol) Then when I’m back I can press a key to stop Monty, and save the new data in the learning modules to a file.

I think this is in line with how Monty will actually be used in the future in real production application. For example, I imagine a house robot running Monty. And the robot will run on a loop. It’s not like the robot will be doing a task, like sweeping the floor and then just “stop” with the broom in hand - because the epochs loop has finished. lol

---

Anyway, I don’t have a strong preference for any of this. I only want to start a discussion and maybe plant the seeds for things to consider in the future.

[brainwaves]

Thank you for your thoughts @AdamLD, this is a great conversation to have.

[tslominski]

Hi @AdamLD, you raise good points.

API

My level of detail on code API planning so far is (I hope to elaborate on this in the near future):

Stable external API

• External means
  • Between Monty and the environment (which contains simulators, robots, or virtual things)
  • Between Monty and experimental setup
  • Between Monty and task/job setup

As you can see, there’s a lot of detail to fill in so your proposal helps .

The imperative API you proposed looks reasonable. Other designs adjacent to an imperative API are the fluent API and the builder pattern.

Configuration

One reason I haven’t discussed the code API in detail is that we are still trying to figure out our configuration story.

The way I’m framing it, is that we have an undesirable effect of configuration being complex. We have multiple factors that contribute some magnitude to the overall complexity. For one, our configurations are coupled in unexpected ways, not all combinations work. The configuration is deeply nested. Also, we have few defaults, so (151 All Platform components must be configured), which leads to (148 Platform configuration surface being large). Lastly, using code for configuration adds some complexity since it can be arbitrarily clever, so we also want to support launching Monty via configuration without writing code. But to support that, we’ll need to improve our current configuration story.

Runtime

We also want to reach a point where Monty can run without the experimental assumptions.

I am looking at Reinforcement Learning (RL) for inspiration for the Monty main loop. The current main loop is rather complex: monty_main_loop_20250418.md · GitHub. So, the idea is to make the main loop more RL-like in its API design. Here’s our working notes so far

The platform main loop

• Should be simplified to resemble reinforcement learning loop
  • Currently, more complex than it needs to be
• Monty should be able to execute its own loop and allow the loop to be externally driven
  • Currently, the Experiment controls the Monty loop
  • I would want to be able to hand Monty a job and have it execute its loop until job completion, or if I’m running an experiment, then I want to control the Monty loop from the experiment myself
  • Able to execute own loop ties into how Monty outputs results; in that a task/job framing would expect a “Result” action to come out of Monty that’s running by itself (and not inspecting internal LM states).

That last bullet point highlights an interesting consideration for Monty’s main loop regarding how to read the results. As @vclay often points out, our brains only output action. So, while current experiments perform the equivalent of inserting a probe into a learning module to read out a result, we would want Monty to generate Actions that would communicate results.

[AdamLD]

Laravel uses a combination of the fulent and builder patterns. I love those.

Configuration is very important. I agree.
Yeah, it makes sense to start with that in mind.

I have some ideas to suggest for consideration on this topic as well. It’s a 3 point layered approach:

1. Have an .env file (that is in .gitignore as a default)

Why? It can specify the current env type, and many configs dependant on the current system. For example, the main config in this file could be MONTY_ENV which can take one of multiple values:
MONTY_ENV=production
MONTY_ENV=dev
MONTY_ENV=testing

Because this can provide us with different configs for different environments. Say for example a person, or company is using Monty and running it in 4 locations:
A. On their local PC.
B. In a robot they are building.
C. In the cloud with a headless habitat to train the model.
D. On a production server exposing their trained Monty model as a demo.

Each of those environments will require different base configuration, like should logging be turned on or off, the logging folder location, etc.

This method can provide people a way to just use git clone and have automated deployments and continuous integration with a single git repositry. (git status in production whould not show any changed files.)

That’s because the .env file is not commited to git, but it will still exist in all the environments. So in the code people can do different things based on the environment, like this:

e = os.getenv("MONTY_ENV")

if e == 'PRODUCTION':
    # logging off
    # run custom code, ex: use a custom Learning Module
else:
    # logging on
    # use a dev Learning Module

I also think we could also have a way to read & set these configs during runtime in the code with a helper function like:

e = config('monty_env') # Reads the value
config('monty_env', 'MY_CUSTOM_ENV') # Sets the value

---

2. Have a /config folder with the default configs as arrays. Like this:

/config/app.py 
/config/habitat.py
/config/logging.py
/config/benchmarks.py
etc.

Even though we could use environment variables like:
export MONTY_LOGS=/path/to/log/folder, I don’t think most people prefer to use these. I think if given the choice, people will want to set these in config files.

Also, this can simplify the deployment process. For example say you boot up a new server in the cloud to expand Monty’s capacity. It would be an indentical new server, so you could just use git clone, and git pull. There would be no need to run different scrips to set environment variables.

Here’s a concrete example for the config folder of a Laravel app:

laravel/config at 12.x · laravel/laravel

And for example, taking the logging config file:

config/logging.php

You can see it defines a default value, that can be overwritten by the value in the .env file:
'default' => env('LOG_CHANNEL', 'stack')

So if LOG_CHANNEL is not set in the .env file, then this config will have the value of stack.

---

3. Service Providers & Service Container

I think we should add wrappers to different modules. And only use the wrappers in the Monty codebase. This way if someone wants to use a different library or module, they only need to change one file in the codebase. They would not have to search all the codebase to see where the module they don’t want was used with import.

That’s what Service Providers can do. They can also set the config for a module at runtime, based on the .env file. Here’s an example:

Service Providers - Laravel 12.x - The PHP Framework For Web Artisans

Let’s pick the Dataloader. We could have a Dataloader Service, that registers the Dataloader module and specifies the config for it.

class DataLoaderProvider(ServiceProvider):
    def register(self) -> None:
        # Register as a singleton
        self.app.singleton(DataLoader, lambda app: MontyDataLoader())

    def boot(self) -> None:
        # Optional: anything you want to run after all providers are registered

Then people, based on their needs can define their own services, and only update the providers. For example, say a company using Monty wants to use their data from the AWS cloud. They can just make a new DataLoader custom to their needs, and register it in the provider. Example:

import AWSDataLoader

class DataLoaderProvider(ServiceProvider):
    def register(self) -> None:
        if env(MONTY_ENV) == 'production':
            self.app.singleton(DataLoader, lambda app: AWSDataLoader())
        else 
            self.app.singleton(DataLoader, lambda app: MontyDataLoader())

Yeah, I think something like this could really be used to manage Monty’s 151 (151+ assuming that number will grow in the future haha) dependencies and configs. What do you think?