AI’s Performance in Covid Detection by Cough Analysis: Insights from UK Study

A recent study led by the UK’s Alan Turing Institute suggests that AI classifiers trained on audio recordings struggle to accurately determine whether an individual has Covid-19 by analyzing the sound of their coughs. The findings challenge previous claims suggesting high accuracy rates in Covid-19 detection through cough analysis, as reported in a paper led by researchers at the Massachusetts Institute of Technology (MIT).

Initial Claims vs. Reality

Contrary to earlier assertions indicating up to 98.5% accuracy in distinguishing cough sounds between Covid-19 positive and negative cases, the study reveals limitations in AI’s ability to fulfill this task effectively. These claims had sparked initiatives to develop apps utilizing AI algorithms for affordable and convenient Covid-19 testing methods. Notably, the UK’s Department of Health and Social Care awarded Fujitsu contracts exceeding £100,000 to pioneer the “Cough In A Box” initiative in 2021, aiming to leverage audio recordings of coughs for Covid-19 analysis via a dedicated app.

Independent Review and Findings

An independent review conducted by researchers from The Alan Turing Institute and Royal Statistical Society, commissioned by the UK Health Security Agency, critically examined the feasibility of audio-based AI technology as a Covid-19 screening tool. Analyzing data from over 67,000 individuals enrolled in the National Health Service’s Test and Trace and REACT-1 programs, researchers assessed the performance of AI models trained on audio recordings alongside Covid-19 test results, cough sounds, breathing patterns, and speech samples.

Confounding Factors and Challenges

The study uncovered significant challenges stemming from confounding variables, where AI models learned to correlate with factors unrelated to Covid-19, leading to inflated accuracy rates. Kieran Baker, a statistics PhD student at King’s College London and research assistant at the Alan Turing Institute, emphasized the impact of recruitment bias within the Test and Trace system, requiring participants to exhibit at least one symptom for inclusion. Additional tests involving paired participants of similar demographics revealed the inadequacy of AI models in accurately detecting Covid-19 biomarkers solely from cough data.

Implications and Conclusions

The study’s findings underscore the complexities involved in utilizing AI for Covid-19 detection based on cough analysis alone. While initial claims suggested promising results, the reality reflects the need for cautious interpretation and validation of AI-driven solutions in real-world scenarios. Addressing recruitment biases and refining AI models remain essential steps in advancing the reliability and effectiveness of Covid-19 screening technologies.