Over in Does Monty implement a biological mechanism …, I mentioned the notion of providing attention and connectivity hints to Monty during setup. This note suggests some (Sci-Fi!) ways that this might be handled.
For simplicity, let’s consider only a single grayscale camera, generating square (1024x1024 pixel) images. We’d like each LM to concentrate on a relatively small (32x32 pixel) patch. This gives us 1024 LMs, which we can arrange as a 32x32 array.
Conceptually, each LM receives data from the patch “below” it. The LM then exchanges information with other (nearby) LMs, in a manner that diminishes over distance. This allows model (etc) sharing, while keeping the overall connectivity manageable.
Example 1 (constant hops)
In this example, we connect each LM to its (24) horizontal, vertical, and diagonal neighbors, up to three (constant) hops away:
...
... . . . . . . . . . . . . ...
... . 3 . . 3 . . 3 . . . . ...
... . . 2 . 2 . 2 . . . . . ...
... . . . 1 1 1 . . . . . . ...
... . 3 2 1 0 1 2 3 . . . . ...
... . . . 1 1 1 . . . . . . ...
... . . 2 . 2 . 2 . . . . . ...
... . 3 . . 3 . . 3 . . . . ...
... . . . . . . . . . . . . ...
...
Example 2 (increasing hops)
In this example, we connect each LM to its (24) horizontal, vertical, and diagonal neighbors, up to three (increasing) hops away:
...
... . . . . . . . . . . . . . . . ...
... . 3 . . . . . 3 . . . . . 3 . ...
... . . . . . . . . . . . . . . . ...
... . . . . . . . . . . . . . . . ...
... . . . . 2 . . 2 . . 2 . . . . ...
... . . . . . . . . . . . . . . . ...
... . . . . . . 1 1 1 . . . . . . ...
... . 3 . . 2 . 1 0 1 . 2 . . 3 . ...
... . . . . . . 1 1 1 . . . . . . ...
... . . . . . . . . . . . . . . . ...
... . . . . 2 . . 2 . . 2 . . . . ...
... . . . . . . . . . . . . . . . ...
... . . . . . . . . . . . . . . . ...
... . 3 . . . . . 3 . . . . . 3 . ...
... . . . . . . . . . . . . . . . ...
...
To be clear, I have no clue what arrangement(s) might yield the best results. That said, it should be pretty easy to try out and test various layouts…