The initial posting left out myriad details; let’s explore a few…
Data Representation, Structures, etc.
In an actor-based system such as Monty, all interaction among the processes is based on the exchange of data structures. So, for example, the Learning, Motor, and Sensor Modules all exchange messages based on the Cortical Message Protocol (CMP).
note: ChatGPT contends that the TBP documentation does not discuss “Motor Modules”, per se. However, in an actor-based system, this seems like an appropriate approach (and thus nomenclature).
Cortical Message Protocol
AFAIK, the Cortical Message Protocol (CMP) is Monty’s only recognized protocol. According to CMP and the State Class,
CMP-compliant messages should contain:
- location (relative to the body)
- morphological features: pose_vectors (3x3 orthonormal), pose_fully_defined (bool), on_object (bool)
- non-morphological features: color, texture, curvature, … (dict)
- confidence (in [0, 1])
- use_state (bool)
- sender_id (unique string identifying the sender)
- sender_type (string in [“SM”, “LM”])
However, this is pretty abstract, in that it does not precisely define the data structure(s) involved. Hmmmmmm. Another chat produced this:
state = State(
location = np.array([...]),
morphological_features = { "pose_vectors": [v1, v2, v3], "pose_fully_defined": True },
non_morphological_features = { "color": [...], "texture": [...] },
confidence = 0.8,
use_state = True,
sender_id = "SM_0",
sender_type = "SM"
)
So, we now have a representative data structure. As long as we’re running Monty in a single Python process, we’re free to ship this data around using native Python data types. However, as soon as we want to use multiple languages, modules, and/or processes, we’ll need to encode the data in some manner, handle addressing, etc. IMHO, the obvious way to do this is to embed the encoded information in a higher-level format.
Data Encoding and Embedding
Although JavaScript Object Notation (JSON) has some annoying limitations (e.g., limited data types, no comments), it is supported by every modern computer language. It is also the base notation for assorted higher-level formats. So, for discussion, let’s assume that we’re using JSON.
The particular “higher-level format” I’d like to propose here is the Model Context Protocol (MCP). MCP, a JSON-based format, appears poised to become the lingua franca of the AI world. It can handle (possibly with a bit of help) most of the administrative issues involved in exchanging messages among AI actors. As ChatGPT put it:
CMP vs MCP: Roles
| Protocol | Purpose |
|---|---|
| CMP (Cortical Messaging Protocol) | Defines what a message contains: State/GoalState, location, features, confidence, sender metadata. It’s the data format / semantic payload. |
| MCP (Model Context Protocol) | Defines how messages are contextualized and routed between modules. MCP tracks the contextual state of an agent, including sequences, dependencies, and hierarchies among CMP messages. It manages state aggregation, temporal context, and multi-module integration. |
Let’s close things out by adding some data type annotations to the links in the previous diagram:
graph LR;
AM_MM["Asst. Monty<br>(LM)"]
EP_MM["Ear Position<br>(MM)"]
LE_HW["Left Ear<br>(HW)"];
LE_LH["Left Ear<br>(LH)"];
LE_SM["Left Ear<br>(SM)"];
RE_HW["Right Ear<br>(HW)"];
RE_LH["Right Ear<br>(LH)"];
RE_SM["Right Ear<br>(SM)"]
SA_LM["Stereo Audio<br>(LM)"]
LE_HW-- Raw -->LE_SM;
LE_HW-- Raw -->LE_LH;
RE_HW-- Raw -->RE_SM;
RE_HW-- Raw -->RE_LH;
LE_LH<-- MCP -->SA_LM;
RE_LH<-- MCP -->SA_LM;
LE_SM<-- CMP -->SA_LM;
RE_SM<-- CMP -->SA_LM;
SA_LM<-- CMP -->AM_MM;
SA_LM-- CMP -->EP_MM;
Legend
-
Edge Types
- CMP: CMP data structures, embedded in MCP
- MCP: MCP data structures, other than CMP
- Raw: raw analog or digital data
-
Node Types
- HW: hardware (e.g., microphone, DAC)
- LH: LLM-based harness
- LM: Learning Module
- MM: Motor Module
- SM: Sensor Module