Potential Conflict in Thousand Brains Theory: Recognition in New Retinal Regions Without Prior Exposure

Research and Theory
1

Hi,

I’ve been studying the Thousand Brains Theory in detail and wanted to raise a question about a possible contradiction or gap in the current model.

The theory states that:

  • Each cortical column receives input from a fixed, small patch of the sensory array. In vision, this means a specific, consistent region of the retina.

  • Each column learns complete models of objects based solely on the input it receives from that specific retinal patch.

  • Recognition comes from many cortical columns voting on the object being sensed, with each column learning independently.

Now, here’s the issue…

Thought Experiment (or Potential Study Design)

Suppose we generate a completely novel, artificial logo (something the subject has never seen before), about 1 cm² in size.

We run the following experiment:

1. Training Phase:

  • The subject looks at a fixed point on a screen (using eye-tracking to ensure no eye movement).

  • The logo is flashed briefly (just enough to register) only in the upper-left corner of their visual field - so it hits only a small, known retinal patch.

  • This ensures that only a limited number of cortical columns, those connected to that specific patch, are involved in learning the logo.

2. Test Phase:

  • Later, the same subject again fixates on the same point.

  • The same logo is now flashed on the opposite side of the visual field - such that it now activates a completely different set of retinal receptors, and thus a different set of cortical columns (those that did not see or learn the logo before).

Expected Outcome (Based on Real Human Perception)

Despite being shown in a totally new part of the retina, the subject will most likely recognize the logo instantly.

The Problem

According to the theory, the columns receiving this second stimulus:

  • Have never seen the logo before.
  • Should have no prior model of it.
  • Should not be able to recognize it individually.

But recognition still happens. This seems to violate a key assumption of the theory: that each column builds its own model and only recognizes what it has experienced directly.

Questions for the Community / TBP Team:

  1. Is this conflict something the theory has already addressed?

  2. Does the Thousand Brains model allow for some form of model transfer or communication between columns that would enable this kind of recognition?

  3. Or might this point to limitations in the column-specific model-learning framework, at least when applied to high-speed visual recognition?

I’d love to hear thoughts from the team or anyone deeply familiar with the theory. I’m not aiming to discredit the work - just trying to understand whether this is a known open question, or something that requires further consideration or experimental validation.

Thanks for all the incredible work you’ve done in building this model!

2

There’s a short clip where Jeff proposes how hierarchy could solve this problem: https://www.youtube.com/watch?v=BOm1pcA5Vns&t=1482s . I’m sure the researchers will have a more nuanced view, but I wanted to point you to this one as it seems related to your example (and it stuck in my head when I saw it).

Screenshot 2025-08-01 at 15.42.12
Screenshot 2025-08-01 at 15.42.122014×466 75 KB

What stuck in my head is that, with the hippocampus at the top of the hierarchy, it could just memorize the dataset instantaneously. I recommend the clip.

2 Likes
3

I’ve watched the video, and researched a bit more. But still don’t understand it fully.

So I imagine using hierarchy in this way to solve the problem:

  1. The lower level LMs will extract some of the edges, lines, and parts of the shapes as object IDs.
  2. Those object IDs will be passed along to the higher level LMs. The higher level LMs won’t receive the full information from all the retina patches, but they WILL receive ALL the object IDs that were extracted from the lower level LMs.

So let’s say only one high level LM has learned the full shape of the new logo. But even if it can’t see the logo in the retina patch it’s getting as input, it can still recognize the logo because its’ getting as input ALL the object IDs from lower level LMs.

I think this method would solve the problem. Hmm, but does the CMP support sending all object IDs from low level LMs to all higher level LMs?

4

Yes. The connectivity between Sensor Modules and Learning Modules is configurable via sm_to_lm_matrix, lm_to_lm_matrix, lm_to_lm_vote_matrix. There aren’t many lm_to_lm_matrix examples, but that’s where a heterarchy can be specified.

1 Like
5

Tristan already mentioned all the relevant points but in case you are curious in more details, we also discussed this topic again in this recent brainstorming meeting here: https://youtu.be/0xsfu9kAIaw?si=\_opdbHEgEthl8Iwb&t=1756

1 Like