@jhawkins reviews “Interneurons of the neocortical inhibitory system” by Makram et. Al. (2004), a discussion around the role of inhibitory neurons in the Thousand Brains Theory.
3 Likes
Meant to post something to this a bit ago (have watched it a few times now). Some awesome stuff was covered in the video, though it may be a little neurosci heavy for some. I decided I’d try and lay out some of the regulatory basics behind what the team is discussing here. The idea being that, if you understand some of the context underpinning their conversation, you might have a better appreciation for the actual talking points. Here goes…
Viewed simplistically, we can imagine that there are two types of “regulatory networks” in the brain which help modulate our behavior and drive learning:
- GABAergic networks, which are inhibitory.
- Glutamate based networks, which are excitatory.
Excitatory networks tend to trigger first, followed then by a constraining inhibitory force. Through this interaction, these two opposing networks “compress” into one another, creating bottlenecks through which behavior might become regulated. This bottleneck is often called the “excitation-inhibition ratio” (or E/I ratio).
- Example: Jeff had made an example of cloud gazing, being explicity told there was a dog in those clouds, then actually seeing that dog yourself (only after having been told). This is potentially a disinhibition of the GABAergic networks of the brain, thereby allowing “excessive learning” to take place as there is now less constraint placed against the excitatory networks.
Additional random thought…
As an aside, I think it’s interesting to compare this behavior to something like the hallucinatory outputs we sometimes notice in LLMs. They seem similar to me. Also, It should be noted that the brain has significantly more ways to inhibit the E/I ratio than it does to encourage it. There’s probably a reason for this.
Now, an interesting correlation I see between the fields of ML and Neuroscience are found between these two regulatory networks and the ML principles of learning rate and regularization.
For instance, learning rates and regularization tend to pull weights in opposing directions:
- High learning rates pull weights away from zero
- High regularization pulls weights towards zero
This is similar to how GABAergic systems pull neuronal behavior towards inhibition, whereas Glutamate excites.
Re. Double Bouquet presence in L6
I thought DBCs were mainly responsible for the fine-tuning of local circuit E/I ratios. Isn’t L6 more globally networked? Would it actually make sense for bouquet cells to be there?
Edit: Well I’ll be darned…
So last night, I noticed that perplexity had just released a “deep research” function. Curious, I decided to test it with the following prompt:
“I would like to know about the prevalence of double bouquet cells (SBC) within layer 6 (L6) of the neocortex. Specifically, the prevalence of the soma of the cell itself. The existence of an axonal projection, while interesting, is of less importance. If DBCs are not found within L6 in sufficient number, I would like to know why.”
For about two minutes it did its thing, then it returned this: Prevalence and Characteristics of Double Bouquet Cells in Neocortical Layer 6.pdf - Google Drive
From what I understand, it seems accurate. Though its citations are… well they’re weird. To start, it actually seemed to source from a fairly diverse range of relevant papers. But then it only ever links to the top three references. I’m not quite sure how its using the other sources. I think they’re informing the contextual bias of its outputs, but I’m not 100%.
Also, it get hilarious if you let it source from social media. For instance, did you know that r/wetshaving was an excellent way to get information on the phylogenetic considerations on DBC distributions? Neither had I!
All that said, it did seem to like Numenta’s own discourse server, which was cool. In particular, it liked this post quite a bit: Notes: Thomson's "Neocortical Layer 6, A Review" - General Neuroscience - HTM Forum
The tool is interesting, I’ll give it that much. Not quite sure how I feel about its accuracy yet though. The way its links its citations needs some work. Would also be cool to see it shore up additional information/material on certain subjects. The section on developmental gradients, for instance. I’d be curious if anyone here more experienced could point out any serious, glaring flaws the paper. What are your thoughts?