In a recent study conducted by a collaboration between King’s College, London, and Peking University, China, it has come to light that autonomous vehicles face significant challenges in accurately detecting children and individuals with darker skin tones. The research, which evaluated eight artificial intelligence-based pedestrian detectors used in the development of driverless cars, reveals concerning discrepancies in detection accuracy across demographics.
The study’s findings reveal a disconcerting reality: detection accuracy for adults outstrips that for children by 19.62%, while a 7.52% disparity in accuracy exists between individuals with light and dark skin tones. Intriguingly, gender plays a less pronounced role, with only a marginal 1.1% difference in detection accuracy.
The study, titled “Unmasking Fairness Issues of Autonomous Driving Systems,” has yet to undergo peer review. It sheds light on a critical aspect of autonomous vehicle development: the potential biases ingrained in AI systems. The researchers noted that common pedestrian detection datasets lack the necessary demographic labels. To address this, the team manually annotated four datasets, resulting in an extensive collection of 8,311 real-world images enriched with 16,070 gender labels, 20,115 age labels, and 3,513 skin tone labels.
This annotated dataset enabled the team to rigorously assess the fairness of pedestrian detectors. Alarming discrepancies in detection capabilities emerged, highlighting the concerning challenges autonomous vehicles might face on the road. Of particular concern was the revelation that “detection performance for the dark-skin group decreases under low-brightness and low-contrast conditions compared to the light-skin group.” Notably, the percentage of undetected individuals rose from 7.14% to 9.86% during nighttime scenarios, underscoring the potential dangers lurking in the shadows.
One alarming implication is that similar software and datasets might be used by various car manufacturers and autonomous vehicle developers. While confidentiality rights prevent confirmation of this practice, the prevalence of open-source models in the industry makes it a plausible scenario. Compounding the issue is the underrepresentation of individuals with darker skin tones in the main open-source datasets used for training AI models. This raises concerns about the inadvertent perpetuation of biases within the technology.
A straightforward solution emerges from the study’s conclusions. It emphasizes the need for “prioritizing group fairness when building software” and posits this as an essential ethical responsibility for software engineers. The researchers also call for increased government involvement, advocating for the establishment of laws and regulations that uphold the rights of all individuals and adequately address these technological concerns.
While this study is not the first to spotlight the potential risks faced by individuals with darker skin tones in the context of autonomous vehicles, it adds to a growing body of evidence. A similar report, “Predictive Inequity In Object Detection,” was presented by scholars from the Georgia Institute of Technology back in 2019. As the development of autonomous vehicles continues to advance, such research underscores the urgent need for a more inclusive and conscientious approach to ensure the safety and fairness of these innovative technologies on the roads.
