AI-enabled Technologies for Autonomous and Connected Vehicles

This chapter describes one method of approaching fuel-efficient truck platooning using a system called Cooperative Adaptive Cruise Control (CACC). The principal innovation in the system is its lean sensor package, including factory-ready standard ACC system utilizing a dual-beam radar, precision Global Positioning System (GPS), and a Vehicle-to-Vehicle (V2V) communication system. In other words, no imaging sensors such as camera or lidar, and no associated high-performance computing hardware such as Graphics Processing Units (GPU). Extensive test track and public road testing on class-8 semi-trucks, including edge case testing, reveals the efficacy and robustness of this system despite its leanness. Quantitative results are included in this chapter that trace cause and effect through the CACC system.

Environmental Perception for Intelligent Vehicles (EPIV) generally focuses on the awareness and understanding of the driving environment around intelligent vehicles by various vehicle sensors. In recent years, a lot of excellent research has been conducted on developing novel methods and technologies for EPIV. This chapter overviews some of the main research topics in the field of EPIV. First, this chapter reviews various types of vehicle sensors which capture multimodal environmental information around intelligent vehicles and form the foundation for environmental perception. Second, this chapter focuses on data restoration and denoising technologies on camera and LiDAR sensors, which guarantees the quality of the data captured by the vehicle. Third, this chapter deals with methods on semantic segmentation, object detection and tracking with camera and LiDAR data, which play a central role in environmental understanding. Fourth, this chapter introduces technologies on location and mapping with multimodal sensor data, which is essential for the local path planning of intelligent vehicles. Finally, this chapter discusses the research technologies on fusing multimodal environmental data, which represents the frontier of EPIV development.

As an important application of Artificial Intelligence (AI), autonomous driving has developed rapidly in recent years. 3D object detection in autonomous driving has attracted more and more attention because it provides precise range and size information of an object. Some of the detection algorithms rely on point cloud data acquired from LiDAR. Compared with LiDAR, the optical image solution for autonomous driving is cheaper, so that image-based detection approaches are also popular. The chapter overview the recent advance on 3D object detection using 3D point cloud or image directly. For the computing simplicity, the approaches which detect 3D object from the input images without 3D point reconstruction benefit the vehicle computational platform. However, the object location accuracy of such approaches is limited. Therefore, a proposed method named as Descriptor Enhanced Stereo R-CNN (DESR-CNN) is specified in detail. Several 3D object detection algorithms are tested on KITTI dataset. The experimental results demonstrate that DESR-CNN outperforms most of the existing 3D object detection methods based on binocular images.

The recent development of deep neural learning achieved remarkable breakthroughs in object classification and detection. Deep learning has the capability of learning features automatically from data using general-purpose learning procedures. However, because deep neural networks require large amounts of data to train the parameters in the network, it is challenging to develop any object classification or detection system with a relatively small dataset. Transfer learning is an important machine learning technique that transfers the learned features in a pre-trained Convolution Neural Network (CNN) model into a new system. In this study, current state-of-the-art CNN models are reviewed in their architectures and characteristics. For the comparative study of transfer learning, the object classification and the detection systems are implemented using transfer learning with six state-of-the-art CNN models. The object classification model has achieved an accuracy of 97.01% for the three-class classification task using transfer learning. Furthermore, six different Faster R-CNN architectures are implemented for object detection. The performances of the different transferred models are compared in terms of the accuracy and the deploying speed of the new model. Experiments show that transfer learning saves training time and achieves accurate performance by fine-tuning the pre-existing deep learning model.

We discuss the importance of sensing technology in enabling intelligence of future automotive vehicles. We briefly overview efforts of leading technology companies such as Waymo and Tesla which resulted in impressive progress toward highest levels of driving automation. We then describe our efforts in the areas of future radars and lidars, specifically, those which go beyond 2D and mechanical scanning emphasizing importance of AI in improving sensor performance at marginal added cost. We then discuss trends in optical computing with its promise of substantially reducing energy consumption while enhancing edge computing.

There has been significant progress in sensing, perception, and localization for automated driving, However, due to the wide spectrum of traffic/road structure scenarios and the long tail distribution of human driver behavior, it has remained an open challenge for an intelligent vehicle to always know how to make and execute the best decision on road given available sensing/perception/localization information. In this chapter, we talk about how artificial intelligence and more specifically, reinforcement learning, can take advantage of operational knowledge and safety reflex to make strategical and tactical decisions. We discuss some challenging problems related to the robustness of reinforcement learning solutions and their implications to the practical design of driving strategies for autonomous vehicles. We focus on automated driving on highway and the integration of reinforcement learning, vehicle motion control, and control barrier function, leading to a robust AI driving strategy that can learn and adapt safely.

To better connect the Artificial Intelligence and Vehicle Control Communities, the SAE set of terms and definitions for Active Safety Systems is used to discuss modern applications of artificial intelligence to Active Safety Systems. This chapter begins with an introduction to the technology enabling Active Safety Systems and its impact on improving on-road safety. A new dataset is introduced that captures the prevalence of Active Safety Systems in the United States (U.S.) automotive industry for the model year 2021. Three different analyses are performed on this dataset to demonstrate its potential value in studying the offered Active Safety Systems in the U.S. automotive industry. Finally, promising Artificial Intelligence applications to the automotive industry are presented from the fields of deep learning, reinforcement learning, and imitation learning. An example of reinforcement learning applied to Automatic Emergency Braking is provided and demonstrates that reinforcement learning agents can learn policies that are effective in avoiding a collision. This chapter concludes that Artificial Intelligence will play a critical role in the future of automotive safety systems.

Although Model Predictive Control is widely used in motion planning and control for autonomous driving applications, accommodating closed-loop stability with respect to an arbitrary reference trajectory and avoidance of pop-up or moving obstacles is still an open problem. While it is well-known how to design a closed-loop stable MPC with respect to a reference trajectory that satisfies the system dynamics, this chapter discusses how to guarantee stability of a vehicle motion planner and controller when a user-provided arbitrary reference is used. Furthermore, the proposed MPC scheme enables recursive collision-avoidance constraint satisfaction in the presence of pop-up or moving obstacles (e.g., pedestrians, cyclists, human-driven vehicles), provided that their predicted future motion trajectory is available together with some uncertainty bound and satisfies some mild requirement. The proposed motion planner and controller is demonstrated through simulations.

Autonomous driving technologies are expected to not only improve mobility and road safety but also bring energy efficiency benefits. In the foreseeable future, autonomous vehicles (AVs) will operate on roads shared with human-driven vehicles. To maintain safety and liveness while simultaneously minimizing energy consumption, the AV planning and decision-making process should account for interactions between the autonomous ego vehicle and surrounding human-driven vehicles. In this chapter, we describe a framework for developing energy-efficient autonomous driving policies on shared roads by exploiting human-driver behavior modeling based on cognitive hierarchy theory and reinforcement learning.

The decision and control module plays a key role for autonomous driving, which is responsible for generating appropriate control commands that navigate the autonomous vehicles safely and efficiently. Existing decision and control modules for automated vehicles are mainly using a rule-based hand-engineered approach. Although working well in a number of specialized scenarios, such method shows its limitation when dealing with highly automated driving tasks such as dense urban scenarios. Recent advances in artificial intelligence have inspired a line of works about self-learning based decision and control, which enable self-reinforcement of the control policy to potentially super-human performance. In this chapter, we will introduce how to appropriately apply such techniques to automated vehicles. The chapter will begin with the motivations and basics, followed by the key challenges and recent achievements of self-learning decision and control for automated vehicles, focusing on the following key aspects: scalability, performance, interpretability, mixed-model, and emergency handling.

Modern Advanced Driver Assistance Systems (ADAS) have complex logic for determining when and where feature use is appropriate, generally based on geolocation and the vehicle’s sensor suite. This variability can lead to a problem of how to meaningfully measure customer experience from ADAS feature usage data. To provide a broad understanding of customer experience and where the feature should have been active but was not, the data must be viewed relative to the feature availability map. The feature activation and availability experienced by a driver is dependent on numerous design decisions (such as the map and map previewing logic), which may affect the process of understanding the raw data. Therefore, it is critical to compare customer ADAS feature usage data to what the vehicle could have previewed in the best-case customer experience scenario by simulating the feature availability based just on the feature design logic, the vehicle’s location, and the map. This enhanced understanding of customer experience allows for the discovery of corner cases and enables improved feature design. In short, customer ADAS feature usage data can be better understood in the appropriate context, where offline simulations of the designed feature logic provide an appropriate normalization factor.

Safety assessment of Highly Automated Vehicles, including Advanced Driver Assistance Systems and Advanced Driving Functions, is of paramount importance for the acceptance and diffusion of these technologies. On-road testing alone is no option due to the enormous time requirements, so virtual testing is generally considered to be a necessary complement. While more time efficient than on road testing, also virtual testing cannot be performed for all possible situations. Moreover, virtual testing can be even misleading if the considered scenarios are not realistic or do not include the critical situations which can occur in the intended real traffic use. Against this background, we discuss different options and challenges as well as outlooks.

Monitoring driver behavior in real-time is a challenging task as there are several factors that can influence the driver to commit unpredictable mistakes while driving. These factors mainly involve inattentive driver state, absent mind, unreliable cornering, and speeding, resulting in fatal accidents. This chapter identifies the factors that affect driver behavior and performance, and provides an in-depth analysis of various deployed scientific monitoring methods and proposes solutions for early and efficient real-time monitoring of driver behavior. The chapter also reviews real-time smart detection algorithms deployed for the classification of driver state. In addition, the chapter proposes an unsupervised deep learning neural network model that can be deployed in classifying driver states and actions.

The exploitation of communication technologies enables connected and automated vehicles (CAVs) to operate more collaboratively, that is, by exchanging or even negotiating future trajectories and control actions. That way, CAVs (or agents) can establish a networked control system such as to safely automate road traffic in a collaborative fashion. A rich body of literature is available, e.g., on intersection automation, automated lane change or lane merging scenarios. These control concepts, though, are most tailored to the particular application and are in general not applicable to multiple scenarios. This chapter conveys the challenges and perspectives of modeling and optimization-based control techniques for the safe coordination of multiple connected agents in road traffic scenarios. Along these lines, the perspective of generalizing controller design to serve multiple use cases simultaneously instead of designing separate controllers for every use case is discussed. Moreover, the opportunities of learning-based control in case of model uncertainties and mixed-traffic scenarios, involving connected and non-connected agents, are outlined.

This work gives introduction to traffic control by connected automated vehicles. The influence of vehicle control on vehicular traffic and traffic control strategies are discussed and compared. It is highlighted that vehicle-to-everything connectivity allows connected automated vehicles to access the state of the traffic behind them such that feedback can be utilized to mitigate evolving congestions. Numerical simulations demonstrate that such connectivity-based traffic control is beneficial for smoothness and energy efficiency of highway traffic. The dynamics and stability of traffic flow, under the proposed controllers, are analyzed in detail to construct stability charts that guide the selection of stabilizing control gains.

Emerging mobility systems such as connected and automated vehicles (CAVs) provide the most intriguing opportunity for more accessible, safe, and efficient transportation. CAVs are expected to significantly improve safety by eliminating the human factor and ensure transportation efficiency by allowing users to monitor transportation network conditions and make better operating decisions. However, CAVs could alter the users’ tendency-to-travel, leading to a higher traffic demand than expected, thus causing rebound effects (e.g., increased vehicle-miles-traveled). In this chapter, we focus on tackling the social factors that could drive an emerging mobility system to unsustainable congestion levels. We propose a mobility market that models the economic in-nature interactions of the travelers in a smart city network with roads and public transit infrastructure. Using techniques from mechanism design, we introduce appropriate monetary incentives (e.g., tolls, fares, fees) and show how a mobility system consisting of selfish travelers that seek to travel either with a CAV or use public transit can be socially efficient. Furthermore, the proposed mobility market ensures that travelers always report their true travel preferences and always benefit from participating in the market; lastly, we also show that the market generates enough revenue to potentially cover its operating costs.

The chapter presents the research on a real-time deep-learning beamforming prediction model for C-V2X systems. Machine Learning (ML) for modeling and predicting wireless channels of vehicular communications has attracted increasing interest recently. In C-V2X systems, long-haul communication is critically needed, sometimes, without sacrificing congestion factor. Beamforming emerges as such an enabling approach to enlarge coverage by choosing right beams, instantaneously. In both the latest Wi-Fi and C-V2X standards, beamforming selections are to scan all the beams and pick up optimum beams during antenna training phase within a beacon interval (BI). Simulations and experiments conducted by the authors identified such scheme will lead to medium to significant degradation in performances sometimes during the whole BI under certain situations. To respond to this vital situation, this paper presented and studied a deep-learning beamforming prediction model to forecast optimum beams within each BI. In this chapter, a real-time sequence-to-sequence (Seq2Seq) beamforming prediction model is presented and implemented. Experiment data validated the effectiveness of the proposed prediction model under the Dearborn Campus of the University of Michigan, resulting in an enhancement of prediction accuracy of 50–75%.

This chapter covers topics related to big data sources, methods and applications in transportation and mobility research. Big data sources covered include data from vehicle-based and infrastructure-based sensors. Methods from traditional regression to machine-learning and AI are discussed in terms of prediction and inference goals. Finally, example use cases of Big Data and AI are discussed in the context of safety, travel and micromobility.

Connected autonomous vehicles (CAVs) hold the promise of not only improving functional safety but also improving mobility and the efficiency of transportation systems. CAVs can be viewed as a cyber-physical system that contains a large number of minicomputers called electronic control units (ECUs). In order for ECU subsystems to share information and operate efficiently, they are typically networked via various in-vehicle networks (IVNs). Such IVNs include the controller area network (CAN), local interconnected network (LIN), media-oriented system transport (MOST), FlexRay and automotive Ethernet. These IVNs are used to connect safety-critical and non-critical components of the vehicle, including brakes, airbags, engine control, active safety devices, the electronic stability program and adaptive cruise control. Although these IVNs provide some luxury functions and improve the functional safety of the vehicles, the use of in-vehicle communication networks can pose serious security threats to CAVs. Several incidents have been reported showing that intruders are able to access vehicle information, even for safety critical tasks. As the IVNs architecturally are not designed to defend against these attacks, additional methods are needed for security. In recent years researchers are taking advantage of advances in more powerful computing hardware, as well the availability of huge amounts of network data and proposing machine learning-based frameworks to secure these IVNs. To the best of our knowledge, these frameworks lack details such as how to apply machine learning for IVN security. Most of them are focused on the selection of machine learning algorithms to improve attack detection rates. As a result, these frameworks become uninterpreted since they took a lot of time in order to reproduce their result. An efficient successful machine learning system depends not only on the selected machine learning algorithm but also on the quality of data. This chapter aims to bridge this research gap by developing a generalized machine learning pipeline designed to defend against existing and emerging cyberattacks on IVNs. The chapter starts with an overview of IVNs, threat modeling of IVNs followed by machine learning-based defense mechanisms against existing and emerging cyberattacks targeted at these IVNs. The last section of the chapter outlines future directions of using the proposed machine learning approach as a solution against vehicle-based cyberattacks for the next generation of vehicles.