Multispectral Object Detection

• (1) the fast and accurate detection of objects of interest from images;
• (2) the dynamic and adaptive fusion of information from different modalities;
• (3) low-cost and low-energy multispectral object detection and the reduction of its manual annotation efforts.

In terms of the first challenge, we first optimize the label assignment of the object detection training with a mutual guidance strategy between the classification and localization tasks; we then realize an efficient compression of object detection models by including the teacher-student prediction disagreements in a feature-based knowledge distillation framework. With regard to the second challenge, three different multispectral feature fusion schemes are proposed to deal with the most difficult fusion cases where different cameras provide contradictory information. For the third challenge, a nouvel modality distillation framework is firstly presented to tackle the hardware and software constraints of current multispectral systems; then a multi-sensor-based active learning strategy is designed to reduce the labelling costs when constructing multispectral datasets.