Better than experts in developing artificial intelligence-based surgical risk prediction models
Oct 28, 2024
|
A research team at Seoul National University Hospital has developed its own Large Language Model (LLM) that evaluates a patient's risk of surgery based on the summary of the pre-anesthetic evaluation. Using this, it is expected that the quality of medical services can be improved through rapid and objective surgical risk assessment.
A joint research team led by professors Lee Hyung-chul and Yoon Soo-bin of the Department of Anesthetic Pain Medicine at Seoul National University Hospital and Lee Hyun-hoon of the National Institute of Strategic Technology announced on the 28th that they developed an artificial intelligence model that predicts the risk of anesthesia before surgery based on 710,000 people's surgical data and verified its performance.
The process of assessing the risk of preoperative anesthesia is critical for patient safety. In the domestic medical field, the American Society of Anesthesiology Classification of Physical Conditions (ASA-PS), which divides the overall health status of patients from grade 1 (healthy patients) to grade 6 (brain death), is widely used as a prediction tool for anesthesia and overall surgical risk.
However, the ASA-PS scheme was subjective in severity criteria, which often resulted in discrepancies in ASA-PS grade classification among medical staff. In order to efficiently provide medical services, preoperative assessment tools were needed to consistently and objectively identify the risk of severe anesthesia.
To solve this problem, the research team developed a giant language model that automatically classifies ASA-PS grades by learning large-scale patient data operated at Seoul National University Hospital in 2004-2023. The model is an artificial intelligence based on natural language processing technology, like ChatGPT, which understands human languages, specifically specializes in medical records and privacy.
Using this macroscopic language model, ASA-PS grades can be classified quickly and objectively based on a 'pre-anesthetic evaluation summary' that briefly describes the patient's health status and underlying diseases. Therefore, the research team explains that it can help improve communication efficiency and patient safety in clinical settings.
As a result of evaluating classification performance based on data from 460 patients, the mean predictive accuracy (AUROC) of this model for all ASA-PS grades was very high at 0.915. The closer this figure is to 1, the more complete the prediction is made.
In addition, the macroscopic language model and anesthesiologist classification performance were specificity (0.901 vs. 0.897), precision (0.732 vs. 0.715), and F1-score (0.716 vs. 0.713), respectively, and the macroscopic language model performed slightly better.
Additionally, the error rate of the giant language model in distinguishing patients with ASA-PS grades 1 to 2 (healthy people and mild systemic disease) and 3 or higher (severe systemic disease abnormalities), which are important for clinical decision-making, was 11.74%, which was better than the 13.48% error rate of the anesthesiologist.
Professor Hyung-cheol Lee and Soo-bin Yoon (Department of Anesthesiology and Pain Medicine) stated "The results of this study show that artificial intelligence technology can be practically used in clinical settings" and that we will continue to work to develop technologies that can contribute to improving patient safety and medical quality through follow-up research"
Professor Lee Hyun-hoon (National Institute for Strategic Technology Specialization) said "We plan to promote global technology commercialization by cooperating globally based on data from specialized research institutes so that the artificial intelligence preoperative evaluation model can be used globally."
Meanwhile, the study was published in the recent issue of the Nature Partner Journal 'npj Digital Medicine (IF;12.4)' in the field of digital healthcare.
|
bellho@sportschosun.com
