Hi @avner.peled , welcome to the forums!
It sounds like an interesting project. I don’t want to put you off combining Monty with LLMs - there could certainly be scope for some interesting demos there, but just a few thoughts that might be helpful.
Re. a system learning about someone’s culture through conversation - as you might be aware, deep learning systems (including LLMs), are notoriously bad at continual learning, and also learning with limited amounts of data. While there are approaches that can be taken with fine-tuning or in-context learning, having a system that learns dynamically and quickly from a conversation with a person is not something that current technology (Monty or deep-learning) would support well. However, it is the kind of thing we think Monty would excel at in the long term.
On the note of long-term research and language, @vclay made a really nice post about how we think about language and Monty. As described there, we think it would be a mistake to try to shortcut language understanding with LLMs, which do not have grounded, structured language concepts, but rather have become statistical text-prediction systems. Of course, I don’t want to stop you from exploring whether Monty could be controlled with voice commands using an LLM as an interface - that definitely sounds like it could make for some interesting demos. However, in terms of a long term solution to the problem you’re describing, I think it will unfortunately have to wait until we have language capabilities in a thousand-brains system like Monty.
If you’d like to contribute to our roadmap (including during your post-doc) to get us there quicker, please do check out our How You Can Contribute Page. We’d love to have you involved.