2025/11 - Exploring How Cortical Columns Learn Behavior and Causality

@rmounir gives a short presentation to recap the research mini-focus week. During that week, the team spent several days brainstorming about how cortical columns could learn behaviors. The presentation frames the problem as analogous to HTM Sequence Memory, but including the concept of reference frames and event-dependent movement through state space.

Then @jhawkins introduces two new big ideas about how columns work in a hierarchy and specifically with behaviors. Exploring the benefits of splitting morphological columns and behavioral columns, combined with broadcast signals that use a common egocentric reference frame, allows these two types of columns to correlate causal relationships and learn behaviors.

Summary Video

Main Video

00:00 Exploring How Cortical Columns Learn Behavior and Causality
00:08 Recap of the Mini-Focus Week
00:28 Learning a Sequence of Features
02:19 Learning an Object
06:09 Learning a Sequence of Behavior States
07:12 Behavior Transitions with Actions
12:16 Event-Driven Behaviors and Time-Driven Behaviors
14:11 Separating Morphological Columns from Behavioral Columns
15:11 Mapping Morphological Column Functions onto Behavioral Column Functions
40:24 Broadcast Signals - Attended Area & Inferred Object
53:52 Jeff Recaps the Big Ideas
58:33 How is the Attended Area Broadcast and How Do We Learn Reverse Causality?
01:00:07 Multimodal Learning
01:04:34 What is the Extent of the Layer 1 Projections?
01:06:53 Viviane Summarizes the Key Concepts
01:12:02 Thought Experiment: Learning Visually to Infer Somatosensorily
01:26:41 A Solution to Recognizing a Letter Being Drawn on Your Back
01:40:15 Where Would a Novel Shape Drawn on the Back be Stored?
01:41:45 How is an Egocentric Location Generated and Transmitted?
01:49:53 Next Big Question to Tackle: How Do We Use These Models of Causality to Figure Out Actions to Take?

2 Likes

Dr. Clay talks around 1h21m about doubts over a single column being able to model behaviors on its own. It reminded me of Hawkins’ analogy of the bee brain having roughly the same number of neurons as a human column, yet being highly capable. Coincidentally, this interesting video of a bumblebee learning the behavior of a protective trapdoor made the rounds on Reddit over the weekend:

2 Likes

How do we know it learned rather than just using its senses and natural instincts to find a way into a nest entrance? Typically this would be to push its way underneath an obstacle in the way. I notice it always has a little walk around before trying to get in, perhaps checking for pheromone traces.

1 Like

The brain has two systems, one that acts like a reflex using emotions (I have a feeling) and one that deals with problems intuition can’t solve. We spend most of our time in system one. Like we know here, when the brain lacks knowledge it uses intuition first. It forms a new object based on what it knows about the world and makes guesses what it might do. It might know what a pivot is, what a hinge is and a wheel, and other basic objects and behaviors and it can recognize those. It can manipulate the model of the switch with the switch lighting the bulb informed by the knowledge it already has about lever, hinges and pivots and objects that have them, eliminate the things that can’t happen for this sort of thing and then try the most likely thing from what’s left - reality testing is the most important thing a brain can do once it becomes self-aware - and if it works often enough, update system one so intuition is more powerful by being informed. The brain is constantly switching between system one and system two. The reason it has to function like that is the the brain consumes 20% our calorie intake. Everything the body does, including sweating requires calories, including the increased calorie requirements of increased system two activity in the cortex. Nature has no budget for development. If we hadn’t discovered cooking and fire we would not have the brain we have because chewing uncooked food in the quantities available in the wild can’t extract enough calories to support our modern brain. “Reality bats last.”

The intuition system is “the old brain”, and it provides the motivation to do this project for all of us. Jeff “fell in love with brains.” I know exactly what that means and so do all of you. Kind of cool. The abstract thinking that makes this project possible is completely in the calorie burning cortex, along with the bubbly fun loving personalities we all have here. Switching in DID is a pathological outcome of that emotional switching system. Instead of switching to a single autonomous subsystem that responds to different emotional signals, it switches to one of several autonomous subsystems that deal with the emotions or set of emotions associated with activity in our model of social space. PTSD is also a reflexive response to emotional stimulus, and all of the dissociative disorders function as reflexes because they are learned in the old brain and that’s what it does, shortcuts. That happens before we become self-aware, and forms the pieces of the personality that people see as being different complete personalities because the inner working of people are invisible to us other than our own limited view of our own. All the dissociative disorders are nearly all in response to events involving other people. That’s because the greatest threat to a human being isn’t a bear. It’s someone they trust who is willing to betray them. They don’t perform a protective function. They are a pathological response that in a different context would be beneficial, like a cougar attacking a human, or eating its offspring. They have to be completely starving to do that because they also know that there is no bigger danger on the planet than a human being. Eating your offspring because of stresses unrelated to survival is pathological. Lying to not hurt someone’s feelings or even to fool them is normal, and a mixed bag, but pathological liars lie even when it causes them personal harm. The activity is normal, the context is not. Exposure to the event that caused a dissociative disorder is far less effective than exposure to the emotional state in other events that may seem completely unrelated, other that the feelings. We don’t have that many. In order for a dissociative disorder to develop their has to be a pre-existing vulnerability that we don’t understand. Everyone gets post traumatic stress, but it clears up quickly and on you go. The disorder is when it doesn’t go away because the brain doesn’t know the question of ‘what the hell is happening and what now?’ has been answered.

We are doomed to keep seeing the world top down through our intuition and then forced to think about it “harder” no matter what we attempt to do because that’s how the brain works. We will continue to imagine this from our POV and see goal oriented behavior and not see that it’s an illusion. Then have to readjust to a reality where everything self-organizes from the bottom up, and we can’t help but see everything top down. Tools start with a plan. The rocks that we make them from don’t. The brain doesn’t have strategies or goals. Those are high level concepts we use to describe behavior. If we stop adding that tiny drop of consciousness to everything behavior is constrained by reality, and there’s a limited number of options for behaviors of everything in the Universe. A black hole isn’t even a hole, it’s a sphere. Objects that size and weight in the Universe are all spheres because no other shape is possible. Skinner was right, there is only behavior and some of it is invisible to us because there’s no outwards signs of what led up to a visible behavior. Psychology has been unable to let go of describing the imaginary mechanisms that they insert into that empty space with absolute confidence. There is no location in the brain that contains love, or a soul. There is no ghost only machine, and motivation is an abstraction in the same way ‘survival instinct’ is. We don’t do a thing to survive, we survive because we do a thing. Lots of people don’t want to be alive and just don’t talk about it. Others feel like they are literally not alive and just don’t talk about it. We love that humans are motivated by positive reinforcement, which uses giving someone something they want to motivate them, as opposed to negative reinforcement where the motivation is something that is removed, in this example by mating and perhaps losing your head because it relieves an itch you can’t reach.

It doesn’t take much to make a brain that “functions” and is otherwise not useful for long term survival. Look around. The human brain is teetering on being too complex to be stable, and that’s why we’re in this mess. We are at the limit of what biology can support, and the collective chaos of system one trying to solve global problems using personal feelings exceeds our collective ability to create actual solutions with our overwhelmed system two. Only humans have that two system arrangement, and we are projecting the our concepts and the perceived properties of our awareness in system two onto the underlying system one which is the system that organized system two using intuition and meaning during our childhood. Studies on high emotional states have shown that a person at rest has no idea who they will be when they are suddenly enraged, and that version of them will have no idea how they would act if they were not in the state they are in either. That is true for all our emotional states, and why people fantasize about doing things they would never do in real life. Different emotional state, different version of you that may not be well integrated. “I don’t lose my temper, but sometimes I have to use my temper.” - Dolly Parton

I can’t find it, but I remember an Alan Alda interview with a male neuroscientist in the Bay area in the 90s about consciousness and being human, and the interviewee put the difference between a dog brain and a human as, “if you kick a dog and you kick another professor, 6 months later the dog will remember you and what you did, but it won’t have spent those 6 months trying to get your tenure revoked.” Analytical system two is motivated by the emotional system one. Emotions inform and motivate ourselves and others. In modern society, both systems are usually filled with garbage and random trivia because we’re incredibly bad at civilization. We expect people who never learned to manage their emotions from the humans before them to magically teach their children how to not be a self-centered jerk. That is not something that can be corrected easily later and interferes with motivation for learning, and that makes it extremely difficult to maintain a smoothly functioning, stable and sustainable civilization. Every disorganized system was once a more highly organized system.

System two gets everything it knows from culture. It’s culture that tells us how reality works and what everything means. Nothing in the Universe has intrinsic meaning including words and human beings. If you don’t speak or read Cantonese it has no meaning. Meaning only exists in human minds in the cortex and it’s one of the ways knowledge about the world is associated and organized in our model. For something to be meaningful the meaning has to exist in more than one person, and it has to persist beyond the death of an individual. Humans are social animals and our brains need other brains to be able to function optimally. Our basic ideas about how the world works are thousands of years old and they are not our own. If everyone but 100 people were taken away by Raptors Wednesday, the culture those 100 people are carriers for would continue as long as they survived as a group, continuing through the deaths of the individuals who redundantly carry copies of the story of how we got here, why we’re here, and how to stay alive and pass on the culture.

That goes back to the birth of art and toolmaking and storytelling and when abstract language appeared it led instantly to abstract objects that could last decades and longer that held condensed information we can no longer access because we lack the necessary simple objects to build the complex object with. Information entropy happens much faster than entropy in physics. It quickly takes more information to extract the useful information from an object than is available. That is why culture has redundancy and ‘parity checking’ and the usual mechanisms to prevent data loss. Everyone learns the same story and continually retells it. Culture includes how to do that.

Killing the language kills the culture because the language is culture in action. Culture emerges from brains linked together through language. Bees and other eusocial insects uses an extremely small set of simple signals to coordinate in the same way cortical columns do, and it’s insanely effective. Ants outweigh all other animals on the planet. There are at least 20 quadrillion of them. We feel so smugly successful it’s just silly.

But, eusocial insect colonies use the same mechanisms all brains do to self-organize a hive mind with mini-brains in robot bodies that are communicating using a set of compounds mapped to a single behavior like turn here, grab that, and let go in a sequence as they travel. The behaviors are stored patterns of movement that are preloaded. In humans those patterns are learned and stored in the in the brain stem from the very beginning of neural development and the first one is the heart. Hit the right spot in the brain stem and you have ‘locked in syndrome’ like the the movie “The Diving Bell and the Butterfly”, a memoir of a fashion world fast living dude who had an aneurysm. He was fully conscious but appeared to be in a coma other than blinking one eye. A nurse with a letter board pestered him to write the book a letter at a time by blinking as she moved her finger on a piece of white cardboard with the alphabet and a shared language that allowed her to know what was coming next like any of us can when watching someone writing. It’s a difficult claustrophobic movie, but it’s well worth the watch. If the damage is off a little from that spot, there goes heart and lung movement patterns. Epileptic seizures cause local hypoxia because calcium is released and blood vessels are smooth muscle. Calcium contracts and Magnesium relaxes. When capillaries contract the walls become non-porous and the cells there become oxygen starved. This is how high blood pressure and smoking eventually leads to strokes and heart disease. The walls become stiff (“hardening of the arteries” rather than fatty deposits and blood clots) causing ischemic attacks and eventually strokes as the surrounding nerve tissue dies and leaves white spots on your MRI called White Matter Lesions. Missing chunks in the nerve bundles connecting the old and new brain and the rest together under the cortex. Seizures and migraines cause those over time in addition to high blood pressure. Angina is the heart muscle signaling it’s being deprived of oxygen by clamped down capillaries. A seizure includes an ischemic attack. Usually that produces amnesia, disorientation, aphasia, and the rest, but a condition called status epilepticus can occur where the seizures don’t stop, and if that happens in the brain stem it causes sudden unexpected death in epilepsy, or SUDEP when it starves the brain stem for oxygen and the heart and or lungs can no longer function.

Movement is an autonomous subsystem like emotions, and abstract thought is as well. That’s why it can be ‘switched off’ in a fight. Self-organizing, coordinated, autonomous subsystems acting asynchronously in parallel. It’s how cortical columns are doing it, but they are all the same subsystem repeated, the others are specialized. All the important hormones related to our behavior like dopamine and adrenalin are released from specialized organs by signals from the brain sent in response to activity in its model of the world generated by internal or external changes that stand out against the background. Frogs eyes don’t move, and when they are sitting still everything in the environment that’s not moving disappears. You can mimic it in a white room by fixing on a point on the wall. When an insect enters the field of view the background is largely empty and the insect is the only thing the frog is aware of. The frog’s position and pose never changes and the brain doesn’t have to take the motion of its own body into account. Bogie at six o/clock, fire!

I have a suspicion that, like the attached paper suggests, something that neurons interact with that isn’t made of neurons is probably going to fill in the self-organizational holes we have. All of us including Jeff keep framing this with our personal frame of reference, “if you were to.. if I wanted to” and such. That has tremendous power. Einstein couldn’t see the universe as math, so he reframed it as stories about motion in space. Heisenberg couldn’t imagine the universe as motion in space, so he imagined it as pure math and even today not many people can understand what he saw because we don’t speak calculus that well, and calculus is the language we use to describe the parts of reality we can’t experience directly. Einstein required other people to translate his story into calculus from German in order for his ideas to be fully understood by scientists in their native tongue, calculus. Heisenberg’s work still has to be fully translated from calculus into stories about motion in space and that is much more difficult.

The organizing of the columns and maintaining the illusion of conscious experience being continuous, complete, and error free might be enabled by something that isn’t obvious because we thought we knew what it did already. It’s part of our model, but it’s wrong and we don’t know it. About half of what we learn in getting a degree will be obsolete or completely wrong on the day of graduation. The problem is unsolvable because we don’t know what we don’t know until we know it. The pace of discovery even 50 years ago has become so fast that by the time someone graduates college and enters the workforce their brain has already slowed down from making new connections in minutes as a toddler to days in your 30s. At 250-350wpm reading speed, and with the time available to a professional in a field like neuroscience or physics, if you read a scholarly article on a topic in your specialty every day you will be 5 years farther behind every year.

Our entire system of education is based on fantasies like the idea that the neuroplasticity of an infant persists into old age, and that the knowledge created by the learning that happens in those first 25-30 years continues to expand at the same rate through midlife. That alone has doomed us. We don’t put any energy into education and believe it just happens as part of being alive like everything else. 50% of Americans can’t read above a 6th grade level. That’s a systemic problem. We train people to respond to events in the world of various types but we don’t educate people. Training is for sports. If you train for basketball, you have to be retrained for baseball. If you train to be a lawyer… The result is we lack real knowledge and brains can’t learn independently without it. The model of the world it creates is brittle and unable to respond to unfamiliar situations it doesn’t already have a script for. Huge problem in tech support. Lots of information and little knowledge combined with memorized and physical scripts like troubleshooting trees, and no way to create a troubleshooting tree themselves without further training. We are really bad at civilization. Some of us are autodidactic. Ben Franklin learned to swim in a shallow pond by reading a book. Someone I knew taught himself German at 5 because he wanted to Grimm’s Fairy Tales in the original language. When I met him he spoke 14 languages, 7 fluently and most were from India, and Nepal where he spent 15 years as a 6’ 3” 300lb pale Irish Buddhist priest with with a bad temper and a drinking problem. He learned all those languages getting drunk in taverns with the locals.

The brain continues to refine what it knows and test it against reality, but even Einstein was too old to pick up QM by the time he was finished figuring out relativity from the knowledge he had collected in his youth. That is the source of scientific paradigms getting ‘stuck’ like they do. Brains crystallize as they age. That’s why it’s called crystallized intelligence. Paradigms change when old men with control over where we look and what we are looking for retire or die. A professor in their 70s is limited to the model they finished building 50 years before and have been refining and tuning since then. They know it in and out, but it is not in sync with the complete model of the present day. It has different uses it is superior for, like preserving what was actually knowledge in the pile of information that they devoured when they were younger, and framing it in a way that makes teaching it to others easier. The parts we get right become the shared objects and concepts our collective model of the world in that way.

Anyway, here’s the paper on astrocytes and consciousness

https://pmc.ncbi.nlm.nih.gov/articles/PMC12192557/

1 Like