A Neural Network-based Multisensor Data Fusion Approach for Enabling Situational Awareness of Vehicles

Authors: Albert Budi Christian; Chih-Yu Lin; Cheng-Wei Lee; Lan-Da Van; Yu-Chee Tseng

Publication Date: December 30, 2020 (Conference held: 03–05 December 2020)

Conference: 2020 International Conference on Pervasive Artificial Intelligence (ICPAI) Taipei, Taiwan

Abstract: This paper proposes a neural network–based multisensor data fusion framework to enhance situational awareness for autonomous and connected vehicles in V2V communication environments. The study introduces a Mapping Decision Feedback Neural Network (MDFNN) to address the vehicle identification problem by fusing heterogeneous sensor data, including camera images, V2V communication, GPS, magnetometer, and speedometer information. Two MDFNN variants—grid-based and bounding box–based—are designed to improve perception accuracy. Experimental results demonstrate that the grid-based MDFNN achieves 85% identification accuracy, validating the effectiveness of the proposed approach for improving vehicle perception and decision-making in autonomous driving systems.