Thoughts on CMP Routing, Voting, and Attention

Research and Theory
1

Apparently, there are some CMP-related open questions regarding (at least!) Routing and Voting:

Routing

  • What is the routing mechanism?
  • How does attention come into play here?

Voting

  • Can we assume that models of the same object in different LMs were learned using the same reference frame/in the same orientation?

It strikes me that the introduction of “Relay Modules” might help to address these sorts of questions (e.g., by taking on some of the functions of the Hippocampus).

Assumptions

Let’s assume, for purposes of discussion, that:

  • Our Monty instance has a large number of Sensor and Learning Modules, supported by some sort of Motor System(s).
  • These components (mostly) communicate via the CMP.
  • Message traffic is handled by one or more virtual “crossbar switches” (e.g., as supplied by the BEAM).
  • These switches are largely location-agnostic. That is, they have no concept of distance, at least within a given BEAM node).

Notions

  • Create a “deep learning” network of Relay Modules.
  • Send (some) CMP traffic to this network.
  • Have this network:
    • monitor and augment message traffic
    • detect and analyze coincidences, recognizing:
      • neighboring sensor patches
      • related modules in the heterarchy
      • …
    • suggest additional (or modified) CMP routing

So, for example, when two tactile sensors are nearby (e.g., on a fingertip), this could let their first-level learning modules “know about” each other. As more sensors come into play, the network could “learn” about their relative locations, etc.

Alternatively, learning modules that seem to be related to a given object (at whatever level) could be “introduced”, letting them set up long-distance connections.

Finally, feedback and deep learning could be used to “tune” the routing network(s) to optimize connectivity.

Does any of this seem even remotely plausible? (ducks)

-r

P.S. As a useful (or at least amusing) mental image, think of Monty’s modules as converging on a muddy “hollow” (i.e., a “Hippo Campus”) on the banks of the “cool Shalimar”:

… Now more hippopotami began to convene
On the banks of that river so wide
I wonder now what-am-I to say of the scene
That ensued by the Shalimar side? …

– Hippopotamus song by Flanders and Swann

2

I think these are really good questions and points you make, honestly.

With respect to your notion of "deep-leaning networks’ via Relay Module, this actually isn’t too disimilar from something in the brain: thalamic-pulvinar interactions.

Back in 2021, researchers were able to create a virtual reconstruction of these systems, granting them the ability to gate off non-relavant stimuli, analysize coincedences and make system-wide (global) predictions. They called it Deep Predictive Learning. If you’re interested the paper is linked here, though admittedly, it is a bit dense. This would be a good thing to throw at ChatGPT for simplification.

But I think the question here is, if you’d wanted to implement some sort of global attention system, which approach do you think would work best:

(1) One which works within the current architectual constraints of Monty (perhaps some sort of distrubuted policy?)

(2) One which works outside of Montys core architecture (ala a plugin/addon which can interface with LMs via CMP).

1 Like
3

I mostly leave reading dense papers to folks less dense than myself (though your ChatGPT suggestion is worth a try). That said, it’s nice to hear that I may be close to something plausible.

I’m not sure what you have in mind in (1). However, anything that makes the LMs and SMs more complex seems like a Bad Idea to me. The less that they “know” about global mechanisms (e.g., attention, policy, routing), the easier it will be to change things around.

So, I’d like to take advantage of Monty’s proposed (CMP- and message-based, distributed) architecture. In theory (:-}, this sort of network could be added, changed, or removed without requiring any real changes to the LMs, SMs, etc. (They might just see it as changes to their “address lists”.)

More generally, I suspect that a lot of Monty’s capabilities will emerge out of the dynamic interaction and routing of large numbers of modules. So, I’d like to keep the global architecture very “pluggable”, in tune with the vision laid out by @vclay.

2 Likes