Training Set: learning by observation

Note

The train dataset is available at the Download and resources page.

This describes a training set for training models in an unsupervised fashion from observation only. The video clips are cast in the same environment as the test set (similar objects, occluders, motions, etc), but with completely random events, camera positions, etc. Obviously, it only contains possible events. To help with model training, we do however provide additional metadata about object ground truth positions, depth, etc.

Examples

Here are 4 examples extracted from the train dataset:

_images/train_1.gif _images/train_2.gif _images/train_3.gif _images/train_4.gif

Depth and masks

Each video comes with its associated depth field and object masking (each object has a unique id).

_images/meta_1.gif _images/meta_2.gif _images/meta_3.gif

Metadata

A detailed status in JSON format is also provided for each video.

Example of json file (only 2 frames for illustration purposes, real json files contain 100 frames):

{
    "block":"block_O1_train",
    "possible":true,
    "floor":{
      "maxx":3375.0004882812,
      "maxy":1480,
      "miny":-2770,
      "minx":-3375.001953125
    },
    "camera":"196 304 153 -13 -120 -0",
    "lights":{
      "light_1":"-2978.0249023438 -1332.8410644531 181.63397216797 -20.454223632812 98.856430053711 9.1122478806938e-07"
    },
    "max_depth":2458.3498535156,
    "masks_grayscale":[[0,"sky"],[51,"floor"],[102,"backwall"],[153,"object_3"],[204,"object_2"],[255,"object_1"]],
    "frames":[{
        "object_3":"456.66390991211 -861.45208740234 159.97134399414 -28.465270996094 80.850463867188 -90.793273925781",
        "object_2":"-378.92248535156 -700.74951171875 131.89306640625 34.369613647461 -91.140380859375 16.656764984131",
        "object_1":"500 -550 188.73477172852 -62.817230224609 9.9395532608032 -68.216156005859"
      },{
        "object_3":"413.35668945312 -872.896484375 132.60464477539 -28.465270996094 80.850463867188 -90.793273925781",
        "object_2":"-355.95150756836 -701.56634521484 144.69281005859 34.369613647461 -91.140380859375 16.656764984131",
        "object_1":"500 -550 181.11834716797 -62.817230224609 9.9395532608032 -68.216156005859"
      }
     ]