Hippocampal - Entorhinal Complex

Great question @deric.pinto , yes we definitely believe that the hippocampal-entorhinal complex is important, and we believe we can capture its behavior as a form of learning module.

In particular, part of the underlying Thousand Brains Theory is that the hippocampal formation evolved to enable spatial navigation in animals (i.e. grid cells for reference frames of the environment) and rapid episodic memory. Over time, evolution replicated this approach of modeling the world with reference frames in the form of cortical columns, which were then replicated throughout the neocortex for more general purposes.

As such, we believe that the core computations that a cortical column performs are similar to the hippocampal formation. Since LMs are designed to capture the former, they also implement the core capabilities of the latter. The key difference we believe is the time-scale over which learning happens, with the hippocampal complex laying down new information much more rapidly. In Monty, this would be implemented as a high-level LM that builds new information extremely quickly.

An aside on biology and speed of learning

It’s worth noting that the speed of learning in LMs is an instance where Monty might be “super-human”, in that within computers, it is very easy to rapidly build arbitrary, new associations, while this is challenging for biology. Evolution has required innovations such as silent synapses and an excess of synapses in the hippocampal complex. This degree of neural hardware to support rapid learning cannot be easily replicated in the compact space of a cortical column, so in biology the latter will always learn much more slowly, and very rapid learning is a more specialized domain of the hippocampal formation.

In the case of Monty, we might choose to enable most LMs to have fairly rapid learning, although we believe there are other reasons that we might restrict this, such as ensuring that low-level representations do not drift (change) too rapidly with respect to high-level ones.

Let me know if i can clarify anything further.

3 Likes