This comment is not regarding “biological” part of the discussion, but rather the conceptual one, about states, behaviors and reusing them. Yet in first brainstorming part there was a suggestion, that upper neocortex layers’ grid cell-like machinery might encode state space, which is distinct from physical one, but which, as any space, may imply existence of concepts of location, movement, probably path integration. Terminology is always important, but the discussion had shown that some terms meanings here are yet to be clearly defined, so I will describe my understanding of the concepts, which may be wrong or differ from the Monty team’s one.
I understand “state” as a certain relative spatial arrangement of object’s features. A non-compositional rigid object (if such objects can exist in principle, as objects can be at least broken into pieces) within its own reference frame can have only 1 state, so there are no behaviors to model. Behaviors are possible when an object consists of a few parts that can have relative poses and these poses may change, like 2 parts of a stapler can move relatively to each other. (Related post: Some thoughts about scale invariance and model composition)
We need to have behaviors as standalone entities that could be reused across objects. If we assume that there can be a standalone “state space”, then probably this space has to contain these reusable entities, which have to be somehow distinct from each other. Behaviors happen when something, internal or external to an object, forces it to change its state. The way an object is composed defines how exactly it can change with a force applied. Therefore, I would give following general definition to “behavior”: it is a force directed by constraints. The constraints can be physical borders + resistance to a change within the borders. A non-rotating object in zero-g and vacuum is a corner case with no borders, but it will still have at least inertia. From this, it looks to me that the reusable entity, the atom of state space is a simplest constraint, or a primitive degree of freedom (DOF). Movement in physical space is following certain primitive DOF(s) until finding states with different set of available DOFs, this transition from one set of DOFs to another is the movement in state/behavioral space. The space can be represented by a graph with primitive and composite DOFs as nodes. Looks like a state machine.
As for “path integration”, in the video it was explained with example of returning to the starting point directly via a new path. In reality direct path may be not possible due to unknown obstacles (constraints), so I can imagine path integration as the ability to find a new path of arbitrary complexity to get to arbitrary point by avoiding present obstacles, foreseen or discovered during movement. Like on the picture below: something can make certain paths unusable, which the traveler may find out by looking or walking, but knowing AB and BC directions and lengths, the traveler can calculate location of A and hypothetical D point to return to the starting point via it, like CD->DA. Depending on the obstacles there may be many points between C and A, some path segments may even lead away from A but eventually turn back to it.
This picture is probably about what HC-EC does, while a column should work with objects. To me the following example looks both simple and informative: a bolt latch.
It may have one DOF in fully closed or open states (roll up/down), and another one in between those states (slide left/right). For a latch, movement in the state space means movement between DOFs. Changing position in physical space within any DOF can be considered as a cyclic transition to the same state space’s node, like when rotating the bolt up from closed/open state: same vertical DOF is available with each roll movement. Reaching certain positions (which we need to learn) in certain DOFs can lead to transition to another node, with different amount of DOFs, for the latch it is 90 degrees roll, from where one could either roll it further up/down (rotational DOF), or slide sideways (translational DOF). Sliding sideways leads to transition to translational DOF, which eventually can lead to both DOFs and then again to rotational one only:
If we had only orthogonal DOFs, there would be maximum 6 of them in 3D space and predefined amount for any other dimensionality. But different parts of an object may constrain each other in various ways, probably making DOFs non-orthogonal, hence we need to learn DOFs and their combinations for each object class. Also we need to learn the limits within each DOF and the magnitude of resistance to change, to simulate friction, inertia, etc.
Each DOF can be represented by a vector (rotation or translation axis) and therefore can have a pose in object reference frame: this is where and how association of objects and behaviors could be done. A model of an object would contain not only object’s features-at-poses, but also a DOFs graph. Some DOFs could be reused across different objects models, means be included in multiple graphs, allowing to move between alike objects in behavioral (state) space.
Having DOFs as vectors in physical space may allow for path integration: finding new paths in physical space is equivalent to exploring new DOFs that would eventually allow to return back to original state in a novel way.
Does this stuff make any sense?
Thanks.