Notes on Monty and the Actor Model

Research and Theory
1

As I understand it, Monty’s overall design is expected to follow the Actor Model:

The actor model in computer science is a mathematical model of concurrent computation that treats an actor as the basic building block of concurrent computation. In response to a message it receives, an actor can: make local decisions, create more actors, send more messages, and determine how to respond to the next message received. Actors may modify their own private state, but can only affect each other indirectly through messaging (removing the need for lock-based synchronization).

So, for example, each Learning Module will emulate a cortical column, sending and receiving messages using the Cortical Messaging Protocol (CMP).

Because a full-scale Monty implementation might have hundreds of thousands of actors, using an operating system process for each one is not going to work. So, support for light-weight processes will be needed, along with some sort of message-passing infrastructure, etc.

Although Elixir can handle all of this (and more!), the current Monty code base is written in Python. So, what’s needed (IMHO) is a way to enable Monty to use both Python and other programming languages.

Existing Work

Wikipedia lists a large number of languages, libraries, and frameworks for programming with actors. These range from experimental and research efforts to production-ready infrastructure. (My favorite, Elixir, is clearly in the latter camp.)

There are several actor libraries and frameworks that claim to support Python, including cell, CloudI, Clutter, Dapr, PARLEY, Proto-Actor, Pulsar, Pykka, ray, and Thespian. However, most of these are either experimental, inactive, or both.

Dapr might be an interesting choice as a message-passing solution. Basically, it sets up a high-performance HTTP-based message broker which can run in a container. So, it isn’t tied to any particular language, operating system, etc.

However, although the project has a fancy landing page, I was unable to contact them, using either Discord or Google Groups. (I posted a GitHub Issue; still waiting for a response…)

Concurrency, etc.

Concurrency isn’t one of Python’s strengths. One can use threads, but this is a famously error-prone approach. Also, Python’s global interpreter lock can cause performance problems.

Python’s multiprocessing module lets a program spawn a separate interpreter for each process. This could be used for experimental Monty implementations (e.g., hundreds of modules), but a better solution (e.g., light-weight processes) will be needed for production work.

4 Likes
2

I asked Microsoft Copilot “Can the Julia language work with the actor model?”.
It responded:
"Yes, the Julia language can work with the actor model! Julia has built-in support for concurrency and parallelism, which includes features that align well with the actor model. For instance, Julia’s Channel and Task constructs allow for message-passing and lightweight processes, which are core components of the actor model.

Julia also provides libraries such as Actors.jl, which is specifically designed to implement the actor model. Using Actors.jl, you can create actors, send messages, and manage communication between them in a structured way. This makes it easier to model systems that rely on independent, message-driven components."

Julia is very, very close to Python

1 Like
3

Here’s a ChatGPT link, for comparison:

4

I suspect that each actor(ish) environment has its own features, idioms, and such. Also, that Monty will want (and perhaps need) certain approaches. However, there will also be some things in common among environments. So, to get the ball rolling, here’s an informal rundown on Elixir’s general approach, idioms, etc.

Message Format

Local messages in BEAM-based languages (e.g., Elixir, Erlang) use native data elements and structures. Erlang’s External Term Format (ETF) is used for distributed messages. Either ETF or language-independent formats (e.g., JSON, YAML) can be used to communicate with processes written in non-BEAM languages.

Elixir messages tend to be short (e.g., 2-4 element) tuples.

  • The first element of the tuple (the tag) serves as a message type identifier.
  • The rest of the tuple contains data relevant to the message.
  • Nested tuples (along with lists, maps, etc.) allow structured and hierarchical message formats.
  • Using pattern matching, processes can efficiently handle different message types, extract data values, etc.

For example:

{:hello, "world"}
{:error, "Something went wrong"}
{:ok, result}

Note that the :foo elements used in these tuples are symbols. This allows the BEAM to match them very quickly (without the need for any string comparisons).

A Code Example

Here’s a (mostly accurate) code example. Within a module, we define a start function and a loop function. The start function, when called, defines initial_state as an empty map and calls loop with it. The loop function then stalls, waiting for the BEAM to hand it a message that matches one of its patterns:

defmodule KeyValueServer do
  def start do
    initial_state = %{}
    loop(initial_state)
  end

  defp loop(state) do
    receive do
      {:update_state, {key, value}} ->
        new_state = Map.put(state, key, value)
      loop(new_state)
      ...
    end
  end
  ...
end

For example, a client might send an update message:

send(server_pid, {:update_state, {:key, "value"}})

Some time thereafter, the BEAM will decide to give the receiving process (server_pid) a time slice, handing control to the first code block that has a matching pattern. The code block will do its defined task(s), then call loop with (a possibly updated copy of) the process state. Whereupon, loop will stall again…

Tricky Business

As might be expected, some tricky business is needed to make all of this work smoothly. For example:

  • The server_pid (process ID of the server ) can address a process in the current BEAM instance (aka “node”), another instance, or even on another computer.

  • Because the very last thing loop does is to call itself, Elixir can perform tail call optimization, turning the call into a JUMP instruction. Thus, the call stack never gets any longer.

  • Because the BEAM performs all of the pattern matching and function dispatching, it can do so very quickly.

  • Because the BEAM performs preemptive multitasking, no process can bog down the system.

  • Because processes don’t share data, the BEAM is free to perform garbage collection on any idle process. This eliminates “stop the world” pauses that can hamper system responsiveness.

  • If a process dies, a recovery strategy can be implemented using a supervision tree. This can kill off and restart enough processes to bring the system back to a “known good state”.

  • …

1 Like
5

Over in Notes on Monty and the Actor Model, @DLed has been showing off demos of Elixir code calling Monty via pythonx (tbp_monty_via_pythonx) and ZeroMQ. Meanwhile, @tslominski has been sketching out possible long-term development strategies.

In an effort to bring things together a bit, here is my proposal for short and medium term development of an actor-based version of Monty… (ducks)

On the Python side

  • Define a Python API for CMP message handling.
  • Implement a “null modem” version of the API.
  • Implement a ZeroMQ-based version of the API.
  • Modify the Learning Module (LM) to use the API.
  • Make the modified LM separately loadable.
  • Test both implementations of the API.

The null modem version simply needs to pass the CMP payload to the (local) receiving LM instance. The ZeroMQ version, in contrast, needs to send and receive payloads using the appropriate “patterns”.

In any event, once all of this has been done, the Python Monty crew can continue to develop new versions of the LM, keeping them readily usable by other languages.

On the Elixir (etc) side

  • Implement the Elixir side of the interface.
  • Test the ZeroMQ-based API implementations.
  • Proceed on to large-scale Monty development.

Now for some API details…

To minimize confusion, I’d suggest adapting Elixir idioms as the basis for the Python API. In Elixir, for example, sending a CMP message might look like this:

send(LM_pid, {:CMP_vote, CMP_payload})

The first issue we encounter is that PIDs are pretty specific to BEAM-based languages (e.g., Elixir, Erlang). The second issue is that symbols aren’t supported by either Python or ZeroMQ. So, the corresponding Python API call might look more like this:

send("...", ("CMP_vote", CMP_payload))

Using strings instead of symbols will impose some overhead (e.g., in pattern matching). Also, the available ZeroMP patterns (e.g., Request-reply, Publish-subscribe, Push-pull) aren’t an exact fit for the BEAM’s “crossbar switch” (any process can send a message to any process) approach. So, some fudging will be required…

The Good News is that “structural pattern matching”, introduced in Python 3.10, uses the match and case keywords to match the structure of objects. So, the Python LM can easily match and dispatch based on the tuple type key (e.g., “CMP_vote”).

As usual, comments welcome…

1 Like
6

thanks for the proposal write-up, @Rich_Morin. I have been involved in several designs for messaging in various systems, and almost always, it makes sense to design the sending and receiving abstraction with as little knowledge about the underlying messaging mechanism as possible, as this de-risks the future well, but also add (unit-)testability by design.

e.g. a sink and source are good approximations:

otp_sink = SinkForOTPLM(LM_pid)
otp_sink.send({"command":"CMP_vote", "payload":{}})
#        ˆˆˆˆ

which would not need any change for the send API usage if we construct the sink e.g. for ZeroMQ or the “Null Modem sink”:

zmq_sink = ZmqPubConnectSink("tcp://localhost:5555")
zmq_sink.send({"command":"CMP_vote", "payload":{}})
#        ˆˆˆˆ

as for tuples, while these are convenient, evolution of APIs might get somewhat problematic with them. If the messages are tuples, the necessary items in the tuple should include perhaps the command / message type, some metadata (e.g. routing information, and the free-form payload). This way, it’s also easily mappable to any kind of messaging technology behind the implementation. The type/metadata/payload pattern can also be seen in event sourcing implementations

7

Just a quick nugget from computational psychohistory (my own pet project). For very large models (perhaps a hundred thousand agents), it’s possible that a single framework/language is not as clean a solution as it might appear at first. An actor’s internal structure (self-modifying, ‘mortal’, answerable to others only through messaging) can be efficiently handled by a simple rule-based declarative language (eg PROLOG). The larger outer model requires much more flexibility and dynamic creation/process/evolution/concurrency capabilities.

So one size does not fit all.

1 Like
8

fun fact, Erlang syntax was born out of PROLOG

1 Like
9

It’s been a while since I reviewed options for the actor model in Python, but when I started an ongoing personal project a couple years ago, I ended up deciding to use Akka and Scala instead (in part because the pattern matching is so rich, and I think in part due to GIL issues). In particular I’m using Akka 2.6’s typed Behaviors DSL, which allows for more functional-style programming.I haven’t looked at Pykka in a while (or Pekko) but I think Akka can be a good reference point.

(I just recently started with the idea of voting with my actors, thanks to this recent comment.)

1 Like
10

First, thanks to all for the thoughtful feedback. Now, for some responses…

I don’t have strong feelings about the use of tuples as the outer data structure for messages. Any ordered sequence of values (e.g., a JSON array, an Elixir list) is equivalent in this use case.

I’m also OK with using a map as the outer data structure. Although this breaks with a few decades of BEAM practice, it has several advantages (e.g., explicit, extensible, self-documenting). Indeed, I’d argue for using maps for almost everything within the CMP itself.

Storing the address field (e.g., PID) in the call’s parameter list is a reflection of Elixir’s functional programming (FP) origins. Pulling this out and stashing it in a “transport object” (e.g., otp_sink) is an object-oriented programming (OOP) approach which has some clear advantages, particularly in the Python space.

OTOH, keeping addresses as data (“data all the things”) makes it easy to change the transport layer by changing a single value. So, we should make sure that any OOP versions have similar flexibility.

More generally, there are only a few things that matter in terms of the way that the CMP transport APIs are defined:

  • The general approach should meet Monty’s current and expected needs.
  • The data structures and types should be common enough that any language or transport layer can handle them.
  • The language implementations (e.g., function/method names) should be similar enough to avoid confusion in programmers that have to work in (say) both Python and Elixir.
11

I’d like to bring up a question about the way that (say) LMs and SMs should emulate cortical columns. This gets into the design of modules, message types, dispatching, etc. So, bear with me…

The TBP’s diagrams show messages being sent to various locations (roughly, levels) in each LM. Clearly, messages going into each location will need to be handled in a particular manner. Also, each message may have an inherent type, based on the payload it contains.

In terms of the actor model, this sounds like a use case for dispatching, based on the message type. However, I’m not at all sure how the message types should be named. For example, naming message types for levels in cortical columns seems like a Bad Idea, since the levels tend to be pretty fuzzy and not closely tied to specific functions.

More generally, trying to encode both the receiving location and the inherent message type in a single token (e.g., :L5_vote) seems like a Bad Idea, because it complects two independent pieces of information.

This seems like a theory question for @vclay et al. Help?