I’d like to explore the notion of creating Elixir-based subsystem(s) for Monty. Gentle discussion and feedback are solicited…
Rationale
The Elixir programming language, which runs on the Erlang VM, encourages the use of lightweight processes and message passing (i.e., the Actor model). It excels at massive concurrency, distribution, failsoft operation, metaprogramming, etc. And, with the addition of Nx (a numerical computing library), it can efficiently handle math-heavy tasks.
It strikes me that these characteristics might be useful to Monty, allowing compute-heavy tasks (e.g., Learning Modules) to be offloaded to a platform other than Python. Monty’s Cortical Messaging Protocol appears to be a fairly simple data structure, so it should be possible to map it in and out of Elixir message formats, making the interaction relatively seamless.
Questions
Q: Are there any obvious reasons why this notion doesn’t make sense?
Q: Are Learning Modules the most appropriate parts to offload?
Q: Are there specific types of Learning Modules that stand out?
Q: What coordination issues suggest themselves for this approach?
You can think of all the modules as candidates for offload: Learning Modules, Sensor Modules, and Motor Modules. Motor modules haven’t been mentioned so far, but that’s because the framework has been treating the Motor System as singular. This was done for convenience and is not a hard constraint.
EvidenceGraphLM would be the latest Learning Module to consider. MontyForEvidenceGraphMatching would be the latest Monty class to review to learn how the modules interact.
MontyForEvidenceGraphMatching (and the classes it inherits from) would be the place to look at to identify the coordination going on.
I’d like to explore this idea in a bit more detail. However, to avoid making design decisions prematurely, let’s try to avoid Elixir-specific approaches. That way, if some other language seems interesting, the transition won’t be needlessly difficult.
What data format(s) would make sense for interaction with a computation subsystem? For generality, I’d probably suggest JSON (but please let me know if Monty “prefers” something else).
I assume that Monty’s Python code should be in charge, making calls and casts to subsystem(s) as needed. This would facilitate adding and/or substituting resources. However, subsystem code might also need to send messages to Monty asynchronously. So, what architecture would make the most sense?
Finally, can Monty use any industry-standard protocols for its messaging? Are there any peculiar gotchas involved?
I’m not sure, I’d try to decide so many things in advance. A good start would be to find a minimal example that would demonstrate the value of using anything than the currently active implementation. I made the mistake once by starting a NuPIC implementation (in Pony) to show the value of actors there, however, it was already too much to catch up on.
While we are convinced that Elixir (or rather processes running on the BEAM) can help with certain architectural aspects, starting bottom up instead of value-first would probably result in disappointment.
That’s why, after the initial failure with Actor NuPIC and the Thousand Brains paper came out, I’ve created the neural voting experiment in Elixir - to only focus on one aspect: how would highly parallel or even distributed voting work, given e.g. temporal constraints, such as deadlines (for real-time applications), or distribution itself.
As Python programs can run as Erlang C-Nodes, one can potentially simply re-use current Monty code and wire it via built-in Erlang clustering (with the latter being demoed in the latest change to the voting experiment repo).
Perhaps, the next small(est possible) step would be: taking a Monty module, packing it in a shape, connectable via Elixir/Erlang, and implementing one simple voting interaction. Or maybe something even simpler than that.
I’m quite sure, that with each small steps, a deeper understanding of value or future architectural directions will emerge.
As an example, if one fixes this as an architectural constraint, then a bottleneck is pre-programmed, limiting the scalability. Delegating for “raw performance” is not something I’d use Elixir for, because the Python ecosystem already has a good number of well-implemented native packages. Elixir’s Nx for off-loading computationally-intensive tasks only starts to make sense if functionality is re-implemented in Elixir.
Thanks for the thoughtful responses. Clearly, running Elixir subsystems under the control of a Python master program would impose both performance and reliability limitations. However, running the Python master program as a subsystem seems a bit weird and might be a hard sell to the Pythonistas whom I’d expect to be the initial Monty crew.
So, I’d like to come up with a message-based approach, using industry-standard protocols and data formats, that could be used regardless of which language is “in charge”. To that end, I’m wondering about setting up a minimalist test jig (i.e., “Hello Monty”) that would demonstrate the feasibility of interfacing Elixir with Python.
Great ideas and discussion here! I have very little background with this so I am just following along.
I just wanted to chime in quickly to say if you are ready to formalize these ideas more concretely and suggest a first proof of concept approach (like @DLed suggested I think the smallest possible proof point would make sense to start with), you can fill out an RFC and collect feedback from anyone on there. This might be a bit easier to keep track of, especially once it gets into more detailed discussions, than a long conversation thread. But definitely feel free to toss around ideas here for as long as you like
Also, if you have some good crash-course resources for someone like me on the topic of Elixir/Erlang, feel free to share!
The ideas are not exactly the same, but I thought to mention it here as it delves into the idea of modelling environments, sensors, actuators, and agents as processes in Erlang.
from today’s meetup: the moment the people who know Erlang/Elixir hear the phrase “parallelizable and distributed”, a really loud Jennifer Aniston/Erlang neuron fires and doesn’t let go for a while. The unification via the Cortical Messaging Protocol really does enable that. It’s really great to see that this has emerged in the project!
One idea (maybe for an RFC) that could be tried out: the current experiments are step-based. As @vclay mentioned today: nothing from either learning or sensor modules has to actually be calculated synchronously. This is superb and coulb be tried out in various ways, e.g. calculating evidence and gathering results fully asynchronously and adding a timeout to the other voting terminal condition beyond len(union of matches) > delta. Likely, without real asynchronicity this would be hard to test/validate, thus … Elixir.
Another thing I was considering while going through the codebase was this:
Similarly to the Neuro-evolutionary learning model discussed in the aforementioned Handbook of Neuroevolution Through Erlang, Monty seems to “synchronize” the sensory inputs → motor output cycles similar to how feed-forward learning models work (i.e. give the model 1 set of “input”, wait for it to propagate through the model, process the outputs, repeat).
Is that a necessity for the modules / algorithms to function properly? Or is it more of an implementation detail?
I ask this because I was brainstorming how to model the environment, sensors, motors, learning modules etc as separate processes using the cortical messaging protocol and goal states as the message format between them. In thinking about this I realized that in order to implement this there would need to be a sort of synchronization between the environment and all the sensors / motors in order to re-implement what currently exists (at least as far as I understand it), similar to clock ticks in video games (e.g. Minecraft).
So is this truly necessary? Or would it be possible to make truly independent modules that are constantly reacting to received input, bias, goals, and acting upon the environment, receiving feedback?
By default, Erlang messaging is asynchronous, with no timing guarantees. However, IIRC Phoenix Presence uses Conflict-free replicated data types to handle synchronization, etc. So, it seems possible to handle either a synchronous or asynchronous approach.
The Real Question, IMHO, is how the cortex handles this sort of thing. For that, we’d need an opinion from someone in the research side of things. Anyone???
Right, I completely agree with you “real question”, with the added nuance of not just how does the cortex work but do our current algorithms/models assume that the processing is feed-forward.
It would open up many possibilities if the learning and sensor modules could act independently without feed-forward synchronization.
It would be great to have some expert input to these sorts of questions. So, I’d encourage the team members to drop in and comment, educate, elucidate, etc.
Meanwhile, it appears to me that this topic is getting close to the point where an RFC should be started. Might anyone be up for taking a stab at this? (I’d be happy to help out, but I think others may be a lot further along in their thinking and thus in a better position to get things started.)
Good questions! I’d be interested as well to have the perspective of the TBP team. Here is my take:
LMs don’t send request to other modules, they only vote on whatever they want.
Those 2 questions sound the same for me. Those biais/goal state would directly act on the competition for object ID for the LM receiving those signals.
LMs are free to sample whatever seems relevant for them. If some input is already preprocessed by other LMs, then it will probably be useful to use it, naturally creating a feed-forward processing. We could also divide LMs in separate groups and constrain what each group is able to sample to enforce a hierarchy to help the system in creating this feed-forward processing (like the V1 → V2 pathway is facilitated from the beginning in cortical development)
,
