Hi @Spencer ,
Firstly, welcome to the community, and thank you for the interesting post! I think these are ideas that are definitely worth discussing. After some back and forth, we can see if it makes sense to transition to an RFC or what would be best.
To answer your question about similar work:
-
If you have not already seen the Numenta paper from Lewis et al in 2019, I would recommend checking it out. They built a system that uses grid cells with phase codes to encode the locations of features in objects. These objects were synthetic “grid-world” type objects, where each feature was a randomly chosen SDR. This system was essentially a more biologically plausible precursor to what is now Monty.
-
We have occasionally discussed other phase-codes in our research meetings, such as 1D phase codes for movements in particular directions. These have some appealing properties, but also issues. For example, keeping track of movement in 3D space (path integration) is challenging unless the phase code itself is three-dimensional.
-
More generally, while grid cells are clearly important for biological systems representing environmental reference frames, we have debated how necessary they are for Monty.
-
Lastly, there was an interesting discussion about half a year ago on coordinate systems here on the forums which you might find interesting.
Overall this sounds really interesting. A few initial questions from reading your proposal:
-
I would find it helpful if you could explore the concrete example of using a phase code for location. You could maybe start with 2D space, but then discuss generalizing it to 3D space. Working through how path integration would work, as well as making some drawings would also be very helpful. For example, representing head direction with phase is very different from representing a unique location in a reference frame, so some diagrams would help make sure we are on the same page for any follow-up discussion.
-
Can you unpack a bit more what you are referring to in terms of avoiding the curse of dimensionality? Are the dimensions you are referring to here those of e.g. physical space (constrained to 3D plus time), an abstract reference frame (e.g. movement directions through a family tree), or feature space (e.g. the different dimensions i.e. elements of a 128 dimension SDR encoding a feature like a color)? The description you gave of a low-dimensional manifold embedded in a high-dimensional neural space sounded to me like you were referring to the latter of these examples. However, dimensionality is generally not a curse in SDR encodings, but rather a useful feature. In particular, matching SDRs is a fast computation, and the high dimensionality can confer robustness to noise. On the other hand when we path integrate in reference frames, the dimensionality is relatively low (e.g. 2D or 3D), avoiding excessive computation. Hope that makes sense, but happy to clarify.
Looking forward to discussing more, I think after clarifying the above it should be easier to focus on some of the points you raised.