How to separate trainval of nuScenes into 2 separate ones?

[duynn912]

Hi authors,

Thank you for your work! I am working on Nuscenes Dataset but I failed to separate trainval into 2 separate ones like yours. Train split for train and val split for test with OOD. May you help me explain how you can do that and statistic how many samples of classes in each split

Thank you very much!

[TheCodez]

Hi, generally speaking, you load a frame from nuScenes, check if it contains one of the defined OOD objects and discard the frame if it contains at least one OOD object.
This way, you end up with only scenes not containing any OOD objects.
For evaluation, you simply use the whole eval set from nuScenes.

Training Set Class Statistics

Category	Number
car	413318
truck	72815
construction_vehicle	11993
bus	13163
trailer	20701
barrier	125095
motorcycle	10109
bicycle	9478
pedestrian	185847
traffic_cone	82362

Validation Set Class Statistics

Category	Number
car	80004
truck	15704
construction_vehicle	2678
bus	3158
trailer	4159
barrier	26992
motorcycle	2508
bicycle	2381
pedestrian	34347
traffic_cone	15597
animal	37
debris	620
pushable_pullable	3332
personal_mobility	16
stroller	131
wheelchair	0
bicycle_rack	274
ambulance_vehicle	30
police_vehicle	73

Hope this helps. Let me know if you have any other questions.

[duynn912]

Thanks for your quick reply! I will re-check and contact you if needed.

[duynn912]

As your mentioned information, I cannot use the available pretrained models on nuScenes because they contain scenes with OOD objects. If I would like to do on OOD problem, I need to train a new one which removes the frames containing OOD objects. Is it right? then I will continually train an additional classifier with the existence of both ID and OOD. May you confirm for me the right way to do with OOD.

I still don't understand that you mention the frame, so what does the frame means here? Following the tutorials of nuScenes, I see that we will have scenes with many samples and each samples has multiple objects. To me, your advise is that I remove OOD sample in these scenes?

Best regards,
DuyNN

[TheCodez]

If you follow the evaluation strategy of this paper, then you would have to retrain (or use our provided checkpoint).
Also, if you follow my setup, you don't have any real OOD objects during training, only synthetic ones.

Yes, so a frame would be a single sample. So nuScenes has multiple scenes, each consisting of multiple samples, and you would remove the samples that contain OOD objects.