Evaluation of object detection capabilities of autonomous vehicles in harsh weather conditions

Abstract

In the last 20 years, we have witnessed significant advancements in the research on autonomous vehicles (AVs), but challenges remain in their global usage and adoption. According to government data, most AV testing in North America occurs in the desert areas of the southern US where weather challenges are minimal. Little test data is available from areas which encounter harsh weather (e.g., snow, heavy rain, dense fog). Given that object detection is a critical functional requirement for AVs, in this paper, we examine the effectiveness of object detection capabilities of AVs in harsh weather. The first part of our hypothesis is that industry-leading models, mostly trained on datasets from normal weather conditions, will not perform well in detecting objects in harsh weather. The second part of our hypothesis is that object detection accuracy should improve once these models are trained on harsh weather datasets. We used two industry leading models (Faster R-CNN and SSD) with KITTI (normal weather) and CADC (harsh weather) datasets for our research.  Our experiment shows that when trained with KITTI data, both models had an extremely low object detection precision on the CADC dataset, supporting the first part of our hypothesis. In addition, our experiment also shows significantly improved detection precision when these models were trained on the harsh weather dataset, thereby proving the second part of our hypothesis. These results are valuable suggestions to the AV industry as it works to expand the deployment of AVs to harsh weather areas in the future.