After the explainer video, I wanted to understand the Cortical Messaging Protocol more deeply—it’s the architectural backbone Hawkins emphasized as critical for tying Monty’s components together, but I was fuzzy on the details.
So, I created this 15-minute dialogue exploring CMP through the Core videos.
Same methodology as before (ChatGPT for extraction → NotebookLM for synthesis). What I find valuable about the dialogue format is how it makes complex material more accessible—hearing two voices work through concepts conversationally builds intuition in a way that linear explanations sometimes don’t.
This is likely my last synthesis for a while—I want to spend time actually participating in discussions rather than just processing videos. But if this format is useful, happy to hear which concepts would benefit from similar treatment.
3 Likes
Hi @Bryce_Bate I watched the video and thought parts of it were strong, especially the final section on sim-to-real. That said, there are a few points where I think the framing does not quite line up with how we usually describe the theory or the current state of the work.
A few specific things that stood out:
- The video frames continual learning as the central problem, whereas we think of the problem we’re tackling as 1. understanding how human intelligence works and 2. using the derived core principles to develop machine intelligence.
- The description of “knowledge transfer” via voting doesn’t match how we think about it - the votes are hypotheses used for coalescing on an understanding, rather than transferring knowledge.
- It refers to “vector cells.” When we think about the way biology models the position of a sensor in space we frame it in terms of grid, place, and head-direction cells. Which are found in other parts of the brain that we deduce must exist in some form in each column in the neocortex.
- The reference frame discussion seems to imply the coordinate system is centered on the object, whereas objects are anchored arbitrarily in their reference frame.
- The hierarchy description sounded closer to a traditional feed-forward view (edges low, objects high), which is in contrast with our view that even bottom-level cortical columns deal with complete objects.
I appreciate your corrections @brainwaves and do see how this form of summarization can miss important details or get things wrong. What I’m learning in this early journey into TBP is that they’re aren’t too many shortcuts to understanding the theory. I’ve read the TBT book a few times, watched a lot of videos, and have recently successfully run one of the experiments (randrot_noise_10distinctobj_surf_agent). That was amazing to watch!
But it’s a steep climb. I want to climb faster than I think will be prudent, if I don’t want to risk slipping.
This is what I have been thinking a lot about…Benjamin Bergen’s “embodied simulations” and Monty’s traversal of reference frames. I’ll probably not say this quite right but I’ve been thinking about how we use language to, essentially, share our thoughts, or as Bergen might put it, to trigger the similar embodied simulations in each other. That’s how we come to share meanings and understandings.
Okay, so what if we did something like that between a human operator and Monty using a language module? To start, I want to see if I can use a natural language request like ‘Pick up the red box’ to act as a top-down signal. Instead of Monty just wandering until it finds something, the language would ‘prime’ the system to look for a specific reference frame and help the GSG prioritize certain sensory features (like the color red) to get the job done faster.
Think of it this way…the words would be just ‘sensory tokens’ that provide a massive amount of predictive evidence for a specific reference frame BEFORE the physical sensors even make contact. Monty is already (in my way of thinking, following Bergen and what I take away from Redish’s study with rats) simulating-moving-simulating before deciding to act on a hypothesis. This would be using natural language to help shape the simulation from the outset, much as my words are trying to do when I tell you “Think of it this way…”
Anyway, that’s what I’m aiming for and want to contribute if I can. I’ll admit it’s like taking swimming lessons while trying to swim the English Channel. Ah, see? Right there. I made you have a measurable simulation. Ha! I want to find a way to impart those in Monty. Any guidance would be appreciated.
4 Likes
I 100% agree that there really are no shortcuts with this topic (I mean, aside from completely ignoring all the very complicated neuorscience that the researchers have to understand). There are a number of conceptual shifts that need to happen to be able to think productively about biological, plausible, human intelligence.
We think about language in much the same way: that words are used to invoke models in your targets brain that you’re reasonably sure are similar to your models. Sometimes you come to realize that the target brain does not have this concept or model and you have to describe the new concept with other models that they do have.
In order to train a monty instance to understand that words represent objects (and later abstract concepts and compositional objects and behaviors), we would need a learning module that has a model of words, and a way to link the word with the models in other learning modules. Those facilities don’t yet exist in Monty but we’re working on many of them this year. Have a look at the high level goals page in our docs for more info here: Capabilities of the System
3 Likes
