Very excited to join the community! I’ve been following this research from the distance, since I read On Intelligence in 2007. I enjoyed very much the videos from HTM school, some years ago. I read the papers on SDR… and even I pre-ordered and read the Thousand Brains book. But I need to catch up with Monty—the progress I saw in yesterday’s meet up was amazing!
I am researcher and my background is in optimization, reinforcement and imitation learning, and control theory, both single agent and multiagent systems. On the multiagent side, I have worked on multiple topics, from distributed stochastic approximations (the consensus voting I saw yesterday reminded me of those) to Markov game theory. I also have good experience with deep learning, from MLP to diffusion models, mainly for policies but also world models, etc.
I would like to work on adding goal oriented behaviour to Monty and I’d love to collaborate on the area. I have an idea about where to start. I hope to be able to work on this project as part of my regular job, so I will start with the starting guide and I will really appreciate support if there are bumps in the road.
Most of my team is located around the globe, especially in UK, but I am based in Spain.
Hey Sergio, it’s great to hear from you and welcome to the forums!
That sounds exciting, we’re definitely interested in collaborating.
Given your background, I wonder whether one item you might find interesting is the improvement of policies during learning, see: Model-Based Exploration Policy
For example, many probabilistic formulations of RL and active inference have a notion of exploration and intrinsic reward related to curiosity. It would be interesting if we could formulate something similar, but as a concrete implementation in Monty that benefits from its structured models. The link I shared above gives a few examples of the kinds of exploration we would hope to see.
We already have the JumpToGoalState sub-routine/policy, so the key would just be equipping LMs with the ability to propose an interesting location and view to move to in object-centric coordinates. The rest of Monty should be able to handle the rest.
This has the advantage of being something that could be worked on today, in contrast to a lot of complex goal and action planning. I’d also be interesting to hear more about what you already had in mind though when you’ve got a chance!
Also welcome from me @sergioval (sorry for the late reply, the holidays have been busy).
I’m excited to hear about your interest in the project and in contributing! As Niels and Will already mentioned, we are happy to help find a good starting point and clarify and questions. Just one more resource to highlight is our new future work table: Project Roadmap You can filter it by your interests, such as reinforcement-learning or topic (like goal-policy) or generally look in the motor-system subsection.
Best wishes and I am looking forward to hearing more from you,
Hi Viviane! thank you very much! I am also landing back from holidays. I have caught up with most videos and read the latest papers. I am very excited and I have lots of questions! My next step is to have a working setup where I can reproduce some the demos. Next, I will need to setup an environment where I can test some ideas on goal oriented behaviour, ideally Mujoco—I remember there was someone already working on cleaning the environment-related classes, so I will need to catch up on this effort too. Step by step
