Just wanted to let you all know about a few posts I’ve written on various topics I find particularly interesting about TBT to introduce a more general audience. For example, there’s one on catastrophic forgetting which is a fascinating topic that Monty avoids. There’s one on reference frames which helps set up a discussion of Theory of Mind and humor from a TBT perspective.
If anyone has feedback or additional thoughts, I’d love to hear it. Thanks!
I definitely enjoyed the humor one. Punchline delivery depending on speed of inter-column communication and voting convergence… That’s an amusing hypothesis. TBT does bring quite a unique perspective on things I suppose. Makes me wonder what other emergent phenomena of the mind could be intrinsically tied to such contraints.
Regarding catastrophic forgetting, I suspect TBT’s approach could potentially be trading one problem for another; “torturous remembering” - inference slowing to a crawl as the dataset balloons. Monty is very far away from that, but I can certainly picture it approaching O(n) run time with millions of objects and features, unlike vision models and the humain brain. Nonetheless, a compelling obstacle to vanquish.
@grob Thank you so much for writing and sharing these blog posts! I am impressed by your deep understanding of the TBT, and it’s fun to read about how these general concepts might relate to different mental phenomena and abilities. I can definitely recommend these blog posts to anyone who hasn’t read them yet, as they are written really well and accessible!
@AgentRev Good point on the memory consumption. For the number of points per graph, we already have mechanisms in place, which we call constraint object models, to make sure we limit the amount of data that can be stored for one object, and that this represents the most consistently observed features on the object (you can read a bit more about it in our Docs here: Object Models )
Regarding the number of objects in memory we will rely on several mechanisms. One that is already in place is that if you see an object that is very similar to one you saw before, you would just add more points to the existing model instead of learning a totally new model.
One that we currently work on is using compositional models so that we can learn higher-level models with few points composed of more detailed lower-level models. This allows the system to learn reusable parts in detail and then many different combinations of them without storing much additional data.
In the future, we also want to allow for forgetting data that is not relevant for tasks anymore (i.e., has no predictive power because, for instance, your table is not set the way it was 2 weeks ago anymore, and there is no use in remembering that specific arrangement for any downstream task). However, this is not currently work in progress.
Thanks all for checking out the posts! @AgentRev thanks for calling out the humor insight, it feels like a new and elegant way to account for incongruity-resolution in humor. We just need someone to do the experiments to test the hypothesis.
@vclay Good to know, I’m aware this subject was touched upon in your Current Capabilities video, and I’ve also been thinking myself about potential avenues during my dissection of Monty in preparation for the audio module.
The reason why I brought this up here was to identify a relevant caveat applicable to @grob’s article on catastrophic forgetting, since he usually points out both pros and cons in his writings.