“The works and customs of mankind do not seem to be very suitable material to which to apply scientific induction.”
Alan Turing, progenitor of artificial intelligence
For this AIMed publication of the week, we would like to introduce this commentary in Lancet Digital Health.
The authors first discuss the obvious observation of this domain: interest in application of AI tools in clinical medicine and healthcare continues to escalate, but its widespread adoption from academic research into deployable AI devices has been difficult.
A myriad of issues that engender widespread risk of bias are reviewed: model validation methodologies not being reflective of real-world conditions, data characteristics being flawed, and lack of diversity as a result of not being inclusive enough of researchers and populations from diverse global regions.
A major challenge lies in the “colossal-sized landscape of global AI research” with its heterogeneous centers that lead to a lack of a unifying perspective for everyone. In addition, there are now over 150,000 papers published in this AI in medicine domain (most in the last 5 years). Finally, there is a lack of real-time assessment of studies that renders these publications much less useful (the COVID pandemic illustrates this deficiency).
The authors deserve credit for devising a potential AI solution, in the form of natural language processing, to this escalating AI in medicine problem of real world application of AI tools. The NLP pipeline is capable of performing real time identification and classification of AI research abstracts extracted from MEDLINE, and then outputting these results to an interactive dashboard.
This ambitious project includes four objectives: first, to discover original research in clinical AI model development; second, to identify research at more advanced development stages using mature evaluation methodology; third, to map, in real-time, global distribution and equity in AI research production on a per-author basis; and fourth, to track the main active research themes across clinical specialties, diseases, algorithms, and data types.
More impressively, the development was done with Python and Tensorflow and employed transfer learning using transformer NLP models. The final pipeline and methods are published in the appendix of the commentary. The prospective evaluation showed that the classifier had an F1 score of 0.96 and a Matthews correlation coefficient of 0.94. The dashboard therefore allows all published research “visualized by development maturity, medical specialty, data type, algorithm, research location, publication date, or different combinations of attributes”.
Limitations of this effort include getting all the centers and leaders on board (the “human” aspects of AI in medicine), as well as somehow locating the non-published AI in medicine projects (not all AI in medicine projects are obviously discoverable). The authors should be praised for this global approach nevertheless, and the nascent Alliance of Centers of AI in Medicine (of which Imperial College is a member) can be involved in pushing this laudatory agenda.
Read the full paper here.
The post An interactive dashboard for clinical AI research appeared first on AIMed.
Author: