# FPGA Design for Multimodal Sensor Data Fusion in Autonomous Robots

## Abstract

This research study introduces a novel Field Programmable Gate Array (FPGA) design for autonomous robotics that integrates data from multiple sensors to improve their operational efficiency and decision-making. The proposed technique accomplishes real-time performance by integrating information from multiple sensors, such as LiDAR, cameras, and inertial measurement units (IMUs), using the parallel processing capabilities of FPGAs. Sensor interfaces, control logic, and complex data integration algorithms are integrated into the design. Additionally, the design is adaptable and can be implemented with FPGAs. By employing Kalman filters for state prediction and decision trees for contextual classification, the design enhances accuracy and significantly reduces latency. According to the testing results, the FPGA-based system outperforms earlier existing systems in terms of processing speed and accuracy, capable of processing up to 2000 data points per second with a latency of 10ms. This research enhances autonomous robotics by delineating a comprehensive approach to the efficient integration of sensors and the processing of data in dynamic environments.

## Authors

Muthukumaran Vaithianathan *Samsung Semiconductor Inc., San Diego, USA*

Shivakumar Udkar *AMD Inc., Colorado, USA*

Deepanjan Roy *NVIDIA Corporation Inc., Texas, USA*

Manjunath Reddy *Qualcomm Inc., San Diego, USA*

Senkadir Rajasekaran *Samsung Semiconductor Inc., San Diego, USA*

## Publication Information

**Journal:** 2024 International Conference on Sustainable Communication Networks and Application (ICSCNA) **Year:** 2024 **Pages:** 237-242 **DOI:** [10.1109/ICSCNA63714.2024.10863838](https://doi.org/10.1109/ICSCNA63714.2024.10863838) **Article Number:** 10863838

## Metrics

**Paper Citations:** 1 **Total Downloads:** 150

---

## Keywords

**IEEE Keywords:** Accuracy, Laser radar, Multimodal sensors, Robot vision systems, Data integration, Parallel processing, Cameras, Real-time systems, Field programmable gate arrays, Autonomous robots

**Index Terms:** Autonomic System, Sensor Data, Data Fusion, Multimodal Sensor, Multimodal Data Fusion, Multimodal Sensor Data, Data Processing, Data Integration, Decision Tree, Processing Speed, Dynamic Environment, Real-time Performance, Kalman Filter, Inertial Measurement Unit, Multiple Sensors, Control Logic, Gate Array, Integration Algorithm, Integration Of Sensors, Efficacy Of The System, Lidar Data, Data Fusion Techniques, Variety Of Sensors, Obstacle Avoidance, Multimodal System, Human-robot Interaction, Digital Signal Processing, Processing Latency, System Performance

**Author Keywords:** Multimodal Sensor Fusion, Autonomous Robots, Data Processing, Sensor Integration, Real-Time Systems

undefined
## SECTION I. Introduction

The accelerated advancement of robotics and the concurrent reliance on autonomous systems have had an impact on industries as diverse as logistics, healthcare, agriculture, and manufacturing [^1]. The independence of machines is contingent upon their ability to perceive and respond to their physical surroundings. The acquisition of this insight is contingent upon the integration of multimodal sensors, which generate a variety of data, such as LiDAR depth information, camera images, and inertial measurement unit (IMU) motion measurements [^2]. The challenge lies in the integration of data from multiple modalities to produce a consistent and reliable representation of the environment, as each sensor has its own set of advantages and disadvantages [^3]. Autonomous robots that operate in dynamic environments require data fusion, which enhances the reliability, timeliness, and quality of the information by combining readings from multiple sensors [^4]. When dealing with real-time, large data sets, conventional data fusion techniques are challenged by processing speed, particularly when employing microcontrollers. The performance requirements of real-time processing and analysis may not be met by traditional approaches as the number of sensors and the volume of data they produce continue to increase [^5].

FPGA offers a practical solution to these challenges by enabling the development of programmable circuitry that can be customized to meet the specific requirements of individual applications. FPGAs reduce latency and accelerate data fusion by enabling the execution of numerous data processing tasks in parallel [^6]. Environmental mapping, obstacle avoidance, and navigation are applications that significantly benefit from this feature, as they necessitate rapid judgments. The optimization of data fusion algorithms to specific operating systems and sensor suites is facilitated by the flexibility of FPGAs [^7]. Data fusion systems that are built on FPGAs frequently include sensor interfaces, control logic, and data fusion algorithms [^8]. The data from various modalities is intended to be collected, processed, and fused in an effective manner by all the components working together. For instance, sensor interfaces are accountable for collecting data from sensors and transforming it into a format that can be analyzed. The fusion methods, including Kalman filters and machine learning techniques, may be employed to forecast the robot's condition and identify objects in its environment after the data has been acquired. The control logic is responsible for the coordination of all actions, ensuring that the system operates in real time, and directing the passage of data between the processing units and sensors [^9]. The system's efficacy is further improved by integrating state-of-the-art performance optimization techniques into the FPGA design. Parallel processing, optimizing memory, efficiently allocating resources, and reducing latency are all methods of increasing throughput. The system's resilient performance in real-world applications is ensured by the ability to manage high data rates from a variety of sensors, which is facilitated by these methodologies. To exaggerate the importance of effective multimodal sensor data integration in autonomous robotics [^10]. The development of scalable, efficient, and reliable solutions is facilitated by the integration of the algorithms with field-programmable gate array (FPGA) designs. The efficacy of autonomous systems will expand across a variety of domains as they develop into more duties in environments that are becoming increasingly dynamic. To enhance the performance of autonomous robotics in real-world scenarios, it is necessary to conduct research on field-programmable gate array (FPGA) designs that facilitate the integration of data from multiple sensors.

## SECTION II. Literature Review

B. G. Swamy et al [^11] that the work suggests a digital signal processing (DSP) architecture that is made to facilitate the integration of multimodal sensors in ADAS. The fusion approach is simplified by executing matrix inversions and matrix-to-matrix multiplications using LU decomposition. To optimize performance, the state and covariance sub-modules are constructed with a folded DSP architecture. The architecture's feasibility for low-latency, high-speed sensor integration in vehicles is demonstrated by multiple motion model simulations.

Creno et al [^12] that the primary objective of the proposed initiative is to enhance the decision-making speed of autonomous automobiles using FPGA prototyping. It utilizes heterogeneous sensors, including LiDAR and cameras, to enhance the safety of convoy traveling at greater velocities and enhance vehicle awareness. This method is a data fusion technique. To more effectively comprehend and resolve emergent issues with autonomous vehicle systems, constructs a prototype and employ the field-programmable gate array (FPGA) technique to accelerate processing rates.

Wang et al [^13] the paper explores the current state of multi-sensor integration in autonomous driving, examining the benefits and requirements of combining radar, LiDAR, cameras, GPS, IMU, and V2X to enhance visibility and safety. It divides fusion methods into four categories and examines their applications in environment reconstruction and multi-target tracking. The paper addresses future autonomous driving sensor fusion systems and also provides an overview of current issues.

Duan et al [^14] study offers a succinct summary of the most recent developments in human-robot interaction (HRI) for cooperating robots, with an emphasis on strategies that enhance communication, environmental awareness, and work allocation within factories. These are exceptional at enhancing production due to their capacity to integrate human flexibility with autonomous precision; however, they encounter challenges with adaptability, ambiguity, and decision-making. This study endeavors to illuminate significant research trends and challenges in human-robot interaction (HRI) to promote more organic and cooperative interactions between humans and robots in manufacturing.

## SECTION III. Proposed Work

### A. System Architecture

Efficiency, scalability, and flexibility are prioritized in the development of autonomous vehicles that integrate data from multiple sensors using FPGAs. The primary objective of this design is to be implemented on an FPGA. It comprises three primary components: control logic, data fusion techniques, and sensor interfaces. Fig 1 shows the architecture of the proposed system.

![Figure 1](https://ieeexplore.ieee.org/mediastore/IEEE/content/media/10863820/10863824/10863838/10863838-fig-1-source-large.gif)

*Fig. 1. System architecture*

Data is predominantly collected at the sensor interfaces to integrate a variety of sensors, including LiDAR, cameras, and IMUs. The high-speed data buses that these interfaces employ are indispensable for modern sensors, as they generate tremendous data rates, ensuring a seamless connection between the FPGA and the sensors. The backbone of the design is data fusion algorithms, which organize disparate sensor readings into meaningful insights based on the surrounding environment. Real-time state estimation and categorization can be accomplished through the utilization of Kalman filtering and machine learning algorithms. Kalman filters can approximate the robot's current state by predicting its position and speed by conducting a systematic analysis of its historical states and sensor inputs. The aggregated data can be classified using machine learning techniques, specifically decision trees, to identify critical environmental features, including objects and obstacles. Control logic is responsible for the management of the data fusion methods and the sensor interfaces, in addition to directing the entire system. In this manner, the automaton can respond and make judgments promptly by examining the data in real-time. The framework's modular design facilitates the integration of new sensors or algorithms with ease, making updates and modifications a breeze. Additionally, the FPGA can be easily modified to meet the operational needs of various applications, enabling continuous design optimization to enhance performance across all applications. This design offers a comprehensive and practical solution to the obstacles associated with sensor data integration, which has the potential to improve the perception and interaction capabilities of autonomous robotic systems in dynamic environments.

### B. Sensor Interface Design

The FPGA-based multimodal sensor data fusion system's proper operation is contingent upon the sensor interface design, which serves as the critical connection between the physical sensors and the processing units within the FPGA. Each sensor interface is customized to meet the unique needs of the numerous sensor types that are employed, thereby guaranteeing the most efficient data collection and transmission. Autonomous robots must prioritize data flow optimization and minimize latency to effectively manage data in real time. The sensor interface can accomplish these objectives through the utilization of high-speed communication protocols, using Serial Peripheral Interface (SPI), as appropriate for the sensor. For example, high-speed serial connections are required to transmit point cloud data from LiDAR sensors. To guarantee precise signal representation, the interface circuitry employs analog-to-digital converters (ADCs) to convert the analogue signals from the sensors into digital data that the FPGA can utilize. A robust synchronization mechanism is required to synchronize data streams from various modalities when the system contains multiple sensors. This synchronization is essential for the LiDAR, camera, and IMU data to be combined to provide a comprehensive view of the environment. It is essential for the system to accurately monitor the timing of measurements by timestamping each data payload upon arrival, particularly when sensors operate at varying sampling rates. The sensor interface architecture utilizes filtration algorithms to preprocess raw data to improve the quality of the data and reduce noise before it is transmitted to the FPGA. The data fusion method's dependability is prioritized during this preprocessing stage, resulting in enhanced object classification and state estimates. The interface's modular design also enables the addition of supplementary sensors as required without compromising the system's overall performance, rendering it readily extensible. Simply stated, the FPGA architecture is dependent on the design of the sensor interface. The streamlined data collection, synchronization, and preprocessing of autonomous robotic applications enable the integration of multimodal sensor data to an even greater extent.

### C. Data Fusion Algorithm Implementation

It is imperative for autonomous robotics to convert raw sensor data into actionable insights by incorporating data fusion algorithms into the FPGA-based multimodal sensor data fusion system. The initial phase involves the acquisition of raw data streams from a variety of sensors, such as LiDAR, cameras, and IMUs. The formats and features of these broadcasts will be diverse. Data fusion technologies systematically address diversity by standardizing and filtering out cacophony in incoming data. The reliability and accuracy of the ensuing analyses are contingent upon the preparatory stage. Once the data has been cleansed and standardized, data fusion is based on a combination of Kalman filtration and decision tree methods. The Kalman filter is an optimal estimator for predicting the robot's status based on a sequence of chaotic sensor measurements. It achieves its recursive objective by consistently adjusting its predictions of the robot's position, velocity, and orientation in response to new inputs. The system is able to accurately monitor its own condition in real time, regardless of external factors, as a result.

The system is capable of more effectively identifying and naming objects in its environment by employing a decision tree approach in conjunction with the Kalman filter to organize the combined data. This classification is essential for duties such as obstacle detection and route planning. The decision tree analyzes the cleaned-up data generated by the Kalman filter using attributes retrieved from the sensor data to ascertain the presence and type of objects. Thanks to this machine learning technology, the system is endowed with a decision-generating infrastructure that can adapt to a variety of scenarios and circumstances. Parallel processing is feasible due to the incorporation of these algorithms in the FPGA architecture, which enables the processing of numerous data streams concurrently. This parallelism is essential for the real-time applications of autonomous robotics, as it significantly reduces latency and increases throughput. The FPGA's capacity to incorporate custom hardware logic further enables the efficient refining of algorithms, which in turn expedites the fusion process and provides relevant data for quick decisions. The data fusion method allows autonomous robots to effectively navigate and adapt to their environments by precisely integrating sensor data.

### D. FPGA Architecture & Configuration

The multimodal sensor data fusion system's efficiency and efficacy are extremely dependent on the FPGA's design and configuration. The field-programmable gate array (FPGA) is the primary component of the design due to its exceptional ability to integrate programmable logic with embedded computational resources. Our hybrid architecture is designed to satisfy the unique requirements of integrating multimodal sensors by enabling the optimal management and processing of data. Programmable logic blocks are the fundamental components of field-programmable gate arrays (FPGAs). These blocks can be configured to implement algorithms, filter data, and manage signals, among other functions. The implementation of data fusion techniques in hardware is facilitated by their flexibility, which substantially reduces processing latency in comparison to traditional software-based approaches. The design enables the efficient and rapid processing of incoming data by enabling the rapid transmission of data between the processing units and the sensor interfaces. In addition to logic blocks and integrated digital signal processing (DSP) segments, the FPGA architecture is well-suited for mathematical computations. These digital signal processor segments are essential for the correct execution of mathematical computations by data fusion algorithms, including the Kalman filter.

One significant obstacle in the development of mobile autonomous applications is the preservation of electricity while enhancing performance. This is accomplished by the system's transfer of these computations to specialized hardware. Parallel processing is additionally enabled by the FPGA's architecture. The real-time fusion of data from a variety of sensors is facilitated by the ability to implement multiple processing channels simultaneously. For example, to facilitate the integration of multimodal data, one thread may be responsible for processing camera input while another thread manages LiDAR data. The autonomous robot's ability to operate in dynamic environments is contingent upon its ability to maintain a high throughput and low latency, which are facilitated by its parallelism. Furthermore, the FPGA's high-speed memory connections enable the rapid transmission of data from sensors to processing units. The tremendous volumes of data generated by high-resolution sensors are efficiently buffered and retrieved by these memory interfaces, thereby preventing data flow bottlenecks. The FPGA's architecture prioritizes reconfigurability to accommodate future technological advancements in algorithms or sensors. The ability to adapt to changing conditions not only increases the robotic system's lifespan, but also ensures its continued relevance as sensor technology advances.

### E. Performance Optimization Technique

A critical component of performance optimization initiatives is the enhancement of the responsiveness and efficiency of the multimodal sensor data fusion systems of autonomous robotics, which employ field-programmable gate arrays (FPGAs). The system is designed to operate at peak performance levels to effectively manage the complexity of real-time data processing from a variety of sensor modalities. A variety of methods are employed to achieve this. Resource allocation is an essential method for efficiently executing computational tasks on FPGAs, maximizing the utilization of memory, DSP segments, logic blocks, and other available resources. Parallel processing optimization is also essential. The speed and latency of data are significantly enhanced by the deployment of numerous processes in parallel. Simultaneously with the initial CPU processing the LiDAR data, a second CPU may be responsible for processing the camera's image data. Data collection and fusion are expedited by the simultaneous execution of numerous processes, a critical component of autonomous navigation's reliance on real-time decision-making. Performance is additionally enhanced through the implementation of pipelining.

This approach enables the overlapping of multiple processing stages by dividing large operations into smaller, more manageable components. The overall throughput may be improved if the system is able to process each segment independently, rather than waiting for the complete task to complete. One data segment may be filtered while another is being classified, for example, to ensure that the processing stages are operating continuously. RAM optimization is an additional approach to enhance performance. The retrieval and processing rates of data are enhanced by the efficient utilization of on-chip memory, which reduces access times. Two methods for managing high data rates from sensors in a latency-free manner are data buffering and cache. Additionally, the data integration methods are improved by algorithmic enhancements. The computational burden on the FPGA is reduced by optimizing calculations and utilizing approximations when practicable, resulting in a reduction in execution times. The functionality and responsiveness of autonomous robotics are improved by the integration and processing of multimodal sensor information in real time. This is achieved by combining a variety of optimization methodologies, resulting in an FPGA-based system that operates at its peak performance.

### F. Evaluation Methodology

The FPGA-based multimodal sensor data fusion system evaluation method employs a comprehensive approach to assess the integrated system's performance and effectiveness in real-world contexts. Initially, it implements a sequence of controlled experiments to evaluate numerous critical performance metrics, including processor latency, data fusion accuracy, and system throughput. Several metrics are essential for assessing the autonomous robot's responsiveness and reliability in dynamic environments. It is imperative to create simulated environments that replicate a variety of operational conditions to assess the system in a variety of scenarios, including obstacle detection, navigation, and object recognition. The simulations assess the speed and accuracy of the data fusion algorithms by collecting data from the integrated sensors (LiDAR, cameras, and IMUs) and processing it using the FPGA. The system is compared to real-world data to assess its ability to identify and categorize objects. Field testing in unstructured scenarios can evaluate real-time performance by simulating real-world conditions, such as those encountered during deployment. The system's responsiveness to a variety of environmental changes and sensor inputs is demonstrated by the results of the tests, which facilitates the process of algorithm correction and modification.

## SECTION IV. Results

![Figure 2](https://ieeexplore.ieee.org/mediastore/IEEE/content/media/10863820/10863824/10863838/10863838-table-1-source-large.gif)

*Table 1*

![Figure 3](https://ieeexplore.ieee.org/mediastore/IEEE/content/media/10863820/10863824/10863838/10863838-table-2-source-large.gif)

*Table 2*

Table 1 shows the sensor specification of the sensor used in this study. Data fusion performance values are shown in Table 2. To guarantee that the dataset accurately represented all potential outcomes, the FPGA-based multimodal sensor data fusion system was subjected to rigorous testing in both controlled and real-world environments. The compilation is enhanced by data from three primary types of sensors: LiDAR, cameras, and IMUs. The 3D point clouds in the LiDAR data collection are acquired at 300 Hz and have an accuracy of 0.1 m, covering a 100 m2 area. The video data acquired from the cameras is a series of synchronized frames recorded at a rate of 30 frames per second, with a resolution of 1920 × 1080 pixels. The inertial measurement unit (IMU) generates accurate estimates of angular velocity and acceleration by collecting data at a rate of 1000 Hz.

![Figure 4](https://ieeexplore.ieee.org/mediastore/IEEE/content/media/10863820/10863824/10863838/10863838-fig-2-source-large.gif)

*Fig. 2. Power consumption vs processing speed*

![Figure 5](https://ieeexplore.ieee.org/mediastore/IEEE/content/media/10863820/10863824/10863838/10863838-fig-3-source-large.gif)

*Fig. 3. Throughput vs input data rate*

Fig 2 depicts the power consumption vs processing speed graph and Fig 3 depicts the throughput vs input data rate graph. To further ensure the durability of the fusion algorithms, the dataset includes a diverse array of environmental scenarios, including urban landscapes, forests, and interior habitats. To guarantee that the data integration system was as adaptable and efficient as possible, collected 10,000 samples in each of these scenarios. The efficacy of the multimodal sensor data fusion system that was implemented using FPGAs was evaluated using a variety of output metrics. Processing speed, accuracy, and precision, in addition to the time required to identify and classify obstacles, are essential metrics. A real-time performance objective of fewer than 10 ms was established as a metric for processing latency, which refers to the time it took to transition from data acquisition to data fusion. The following is a definition of accuracy metrics equation (1):

$$
\begin{equation*}
{Accuracy}=\frac{{True Positive}}{{True Positives}+ {False Positives}+ {False Negatives}}(1)\end{equation*}
$$

![Figure 6](https://ieeexplore.ieee.org/mediastore/IEEE/content/media/10863820/10863824/10863838/10863838-table-3-source-large.gif)

*Table 3*

To ascertain the obstacle detection rate, this looked at the proportion of environmental impediments that were accurately identified. Evaluation of the decision tree algorithms was conducted based on their classification accuracy. The proposed system demonstrated an exceptional 93% accuracy, 92% obstacle detection rate, and 90% classification accuracy, as indicated by preliminary results. Processing took only 6.5 milliseconds. In the context of autonomous navigation, metrics such as these illustrate the efficacy of the data integration process in supplying accurate, up-to-date information. Table 3 shows the data resources utilized by FPGA. To emphasize the advantages of the innovative technology, a comparison was conducted between existing methodologies and the proposed FPGA-based multimodal sensor data fusion system. The current systems, which are frequently constructed using microcontrollers, process data in a sequential manner, resulting in a decrease in throughput and an increase in processing latencies. The performance metrics of both systems are summarized in Table 4.

![Figure 7](https://ieeexplore.ieee.org/mediastore/IEEE/content/media/10863820/10863824/10863838/10863838-table-4-source-large.gif)

*Table 4*

The proposed method is unquestionably superior to the current one in all respects. It is more suitable for situations that are continuously altering due to its reduced latency, which improves real-time processing capabilities. Autonomous robots' situational awareness improves navigation and safety in a variety of scenarios by increasing the accuracy and rate of obstacle detection. The findings indicate that FPGA-based architectures have the potential to revolutionize the data fusion techniques of autonomous systems.

## SECTION V. Conclusion

Field-programmable gate arrays (FPGAs) have allowed autonomous robotics to analyze data in real-time with astonishing variety and precise accuracy, thereby significantly advancing multimodal sensor data fusion systems. This technology improves the robot's vision in a variety of environments by utilizing complex data fusion algorithms and leveraging the power of multiple sensors. This system incorporates a diverse array of sensors, such as LiDAR, cameras, and IMUs. The results indicate that the processing latency has been significantly reduced and the accuracy has increased in comparison to previous methods. The inventive FPGA architecture has enabled more efficient parallel processing, which has resulted in improved memory and resource allocation. The results not only demonstrate the efficacy of the proposed technique, but they also present novel opportunities for advancements and applications in environmental interaction, obstacle avoidance, and autonomous navigation. In the final analysis, this desires intelligent autonomous systems that are capable of navigating environments that constantly evolving.

## References

[^1]: Lin P-C, Yankson B, Chauhan V, Tsukada M ( 2022 ) Building a speech recognition system with privacy identification information based on Google Voice for social robots. J Supercomput 78 : 15060–15088. [DOI](https://doi.org/10.1007/s11227-022-04487-3) [Google Scholar](https://scholar.google.com/scholar?as_q=Building+a+speech+recognition+system+with+privacy+identification+information+based+on+Google+Voice+for+social+robots&as_occt=title&hl=en&as_sdt=0%2C31)

[^2]: Lee K, Joe H, Lim H ( 2021 ) Sequential routing framework: fully capsule network-based speech recognition. Comput Speech Lang 70 : 101228. [DOI](https://doi.org/10.1016/j.csl.2021.101228) [Google Scholar](https://scholar.google.com/scholar?as_q=Sequential+routing+framework%3A+fully+capsule+network-based+speech+recognition&as_occt=title&hl=en&as_sdt=0%2C31)

[^3]: Lu L, Kanda N, Li J, Gong Y ( 2021 ) Streaming end-to-end multi-talker speech recognition. IEEE Signal Process Lett 28 : 803–807. [IEEE](https://ieeexplore.ieee.org/document/9394723) [Google Scholar](https://scholar.google.com/scholar?as_q=Streaming+end-to-end+multi-talker+speech+recognition&as_occt=title&hl=en&as_sdt=0%2C31)

[^4]: X. Mao, W. Li, C. Lei, J. Jin, F. Duan and S. Chen, “A brain-robot interaction system by fusing human and machine intelligence ”, IEEE Trans. Neural Syst. Rehabil. Eng., vol. 27, no. 3, pp. 533–542, Mar. 2019. [IEEE](https://ieeexplore.ieee.org/document/8633982) [Google Scholar](https://scholar.google.com/scholar?as_q=A+brain-robot+interaction+system+by+fusing+human+and+machine+intelligence&as_occt=title&hl=en&as_sdt=0%2C31)

[^5]: J.-P. Giacalone, L. Bourgeois and A. Ancora, “Challenges in aggregation of heterogeneous sensors for autonomous driving systems ”, Proc. IEEE Sensors Appl. Symp. (SAS), pp. 1–5, Mar. 2019. [IEEE](https://ieeexplore.ieee.org/document/8706005) [Google Scholar](https://scholar.google.com/scholar?as_q=Challenges+in+aggregation+of+heterogeneous+sensors+for+autonomous+driving+systems&as_occt=title&hl=en&as_sdt=0%2C31)

[^6]: S. Saidi, S. Steinhorst, A. Hamann, D. Ziegenbein and M. Wolf, “Future automotive systems design: Research challenges and opportunities: Special session ”, Proceedings of the International Conference on Hardware/Software Codesign and System Synthesis, pp. 1–7, 2018. [IEEE](https://ieeexplore.ieee.org/document/8525873) [Google Scholar](https://scholar.google.com/scholar?as_q=Future+automotive+systems+design%3A+Research+challenges+and+opportunities%3A+Special+session&as_occt=title&hl=en&as_sdt=0%2C31)

[^7]: B. Dayananda, G. Vandana, P. Srihari and B. Pardhasaradhi, “Real time vital sign monitoring system using awr1642 radar module with remote access ”, 2022 IEEE International Symposium on Smart Electronic Systems (iSES), pp. 191–195, 2022. [IEEE](https://ieeexplore.ieee.org/document/10027025) [Google Scholar](https://scholar.google.com/scholar?as_q=Real+time+vital+sign+monitoring+system+using+awr1642+radar+module+with+remote+access&as_occt=title&hl=en&as_sdt=0%2C31)

[^8]: T. S. Kavya, G. Vandana, P. Srihari, B. Pardhasaradhi, “Doa estimation for micro and nano uav targets using awr2243 cascaded imaging radar ”, 2022 IEEE International Symposium on Smart Electronic Systems (iSES), pp. 528–531, 2022. [IEEE](https://ieeexplore.ieee.org/document/10026927) [Google Scholar](https://scholar.google.com/scholar?as_q=Doa+estimation+for+micro+and+nano+uav+targets+using+awr2243+cascaded+imaging+radar&as_occt=title&hl=en&as_sdt=0%2C31)

[^9]: G. Vandana, B. Pardhasaradhi and P. Srihari, “Intruder detection and tracking using 77ghz fmcw radar and camera data ”, 2022 IEEE International Conference on Electronics Computing and Communication Technologies (CONECCT), pp. 1–6, 2022. [IEEE](https://ieeexplore.ieee.org/document/9865707) [Google Scholar](https://scholar.google.com/scholar?as_q=Intruder+detection+and+tracking+using+77ghz+fmcw+radar+and+camera+data&as_occt=title&hl=en&as_sdt=0%2C31)

[^10]: B. Gopala Swamy, U. Sripati Acharya, P. Srihari and B. Pardhasaradhi, “Systolic-architecture-based matrix multiplications and its realization for multi-sensor bias estimation algorithms ”, Advances in Communications Signal Processing and VLSI, pp. 263–272, 2021. [DOI](https://doi.org/10.1007/978-981-33-4058-9_23) [Google Scholar](https://scholar.google.com/scholar?as_q=Systolic-architecture-based+matrix+multiplications+and+its+realization+for+multi-sensor+bias+estimation+algorithms&as_occt=title&hl=en&as_sdt=0%2C31)

[^11]: B. G. Swamy, B. Pardhasaradhi, U. S. Acharya, P. Srihari, S. Reddy and R. Annavajjala, “FPGA Accelerated Automotive ADAS Sensor Fusion,” 2023 IEEE 12th International Conference on Communication Systems and Network Technologies (CSNT), Bhopal, India, 2023, pp. 35–40. [IEEE](https://ieeexplore.ieee.org/document/10134612) [Google Scholar](https://scholar.google.com/scholar?as_q=FPGA+Accelerated+Automotive+ADAS+Sensor+Fusion&as_occt=title&hl=en&as_sdt=0%2C31)

[^12]: D. Créno, B. Senouci and R. Zitouni, “FPGA based approach for Heterogenous Sensors Data Fusion in Autonomous Vehicles,” 2021 Twelfth International Conference on Ubiquitous and Future Networks (ICUFN), Jeju Island, Korea, Republic of, 2021, pp. 369–371. [IEEE](https://ieeexplore.ieee.org/document/9528721) [Google Scholar](https://scholar.google.com/scholar?as_q=FPGA+based+approach+for+Heterogenous+Sensors+Data+Fusion+in+Autonomous+Vehicles&as_occt=title&hl=en&as_sdt=0%2C31)

[^13]: Z. Wang, Y. Wu and Q. Niu, “Multi-Sensor Fusion in Automated Driving: A Survey,” in IEEE Access, vol. 8, pp. 2847–2868, 2020. [IEEE](https://ieeexplore.ieee.org/document/8943388) [Google Scholar](https://scholar.google.com/scholar?as_q=Multi-Sensor+Fusion+in+Automated+Driving%3A+A+Survey&as_occt=title&hl=en&as_sdt=0%2C31)

[^14]: Duan, J., Zhuang, L., Zhang, Q. Multimodal perception-fusion-control and human-robot collaboration in manufacturing: a review. Int J Adv Manuf Technol 132, 1071–1093 ( 2024 ). [DOI](https://doi.org/10.1007/s00170-024-13385-2) [Google Scholar](https://scholar.google.com/scholar?as_q=Multimodal+perception-fusion-control+and+human-robot+collaboration+in+manufacturing%3A+a+review&as_occt=title&hl=en&as_sdt=0%2C31)

### Additional References