Clearly, the brain has periodic “waves” that pass through
You’re thinking of oscillatory waves, which is what the project is inspired by. Specifically, oscillatory coupling. I talk about it very briefly in the comments of this post here: Modelling goal-defined behavior through model-based policy recursion
Essentially, we would have nested operating cycles (or windows), inside of which you could have certain model-based policy states operate. These windows would then be synchronized across LM’s by a common temporal reference frame. In the brain, its the medullary nuclei which provides this. For Monty, I assume utilizing the clock cycles of the CPU would provide the same kind of functionality.
The smallest oscillatory wave is known as a gamma wave. It oscillates at a frequency of 30-100 Hz (or about about 30-100 operations per second). In context to your project, I was imagining you could process the highspeed camera throughput with every op of a given gamma window, whereas it’d be maybe every third gamma window for the slow camera throughput. You could then synchronize disparate LM data at every theta window refresh, for example. This way the Monty system could remain stable/synchronized despite the sampling differences of its sensors.
P.S. Yes, I know that 30-100 ops/second is slow for a modern processer (most operate in the range of ~4 billion ops/sec). The example is only meant to demonstrate the nesting behavior.
Lastly, as for simultaneity detection, I’m still working through that. The brain has mechanisms for coincidence detection, though I suspect this is something already being worked on by the TBP team.
@rolly,
I don’t think this will be the case for forever, no. As long as the sensor is able to transmit into CMP format, I see no reason why you couldn’t have any number of arbitrary sensor type.
I’m currently working out all of the different sensor input vectors found on the human body and will be posting something on that soon. But to use our own physiology as an example, we have around 24 different sensor types (depending on how finely you want to class them).
Also, its good to remember, that our entire observable world is made up of around 100,000 of those straws (of which only a small subset are RBG capable). And yet we manage to get along just fine.