Some Questions from the Documentation

General
21

The two dictionaries are there so we can evaluate the model. The LM itself doesn’t use these mappings for recognition or anything but we as experimenters are curious about whether the LM has merged information from multiple objects into one graph and if it has created multiple graphs for the same object. In a perfect world none of these may happen (the LM has learned exactly one graph per object) but since we don’t provide labels to the system, either case can happen and we want a way to quantify this.

Related to @HumbleTraveller response, the first case (info from multiple objects in one graph) is actually a desirable quality in some cases such as when we want to learn a model of the general object category (hot air balloon) instead of detailed models of specific instances.

This tutorial on unsupervised learning may be useful: Unsupervised Continual Learning

Here are the current statistics we extract from these two dictionaries: Benchmark Experiments (“Mean Objects per Graph” and “Mean Graphs per Object”)

Here: https://youtu.be/yJBhZkkZ-XM?si=1h_wV7x0JI9kHKdR&t=3096 I show some examples of instances where multiple objects were merged into one graph.

2 Likes
22

Thanks @vclay,

For some reason I was really struggling to find the words for ‘perhaps the code is meant for something else entirely.’ Using it for human eval makes tons of sense, I should have thought of that. Thanks for the clarity!

1 Like