Saturday 01 March 2025
The researchers used Med-BERT, a foundation model designed for electronic health records (EHRs), and reformed the disease binary prediction task into a token prediction task and a next visit mask token prediction task to align with Med-BERT’s pretraining task format. The reformulation of the task into Med-BERT-Sum demonstrates slightly superior performance in both few-shot scenarios and larger data samples.
The researchers found that the model’s predictive capabilities were significantly enhanced when the task was aligned with Med-BERT’s pretraining objectives, leading to earlier detection and timely intervention. This approach improves treatment effectiveness, survival rates, and overall patient outcomes. The study highlights the importance of aligning downstream tasks with a foundation model’s pretraining objectives to achieve better performance.
The results show that Med-BERT-Mask outperforms Med-BERT-BC by 3% to 7% in few-shot scenarios with data sizes ranging from 10 to 500 samples. This suggests that using Med-BERT as a foundation model and reformulating the prediction task can lead to significant improvements in disease prediction accuracy.
The study’s findings have important implications for the development of clinical decision support systems, which rely on accurate predictions to guide treatment decisions. By improving the accuracy of disease predictions, these systems can help clinicians make more informed decisions and improve patient outcomes.
Overall, this study demonstrates the potential of foundation models like Med-BERT in improving the accuracy of disease predictions using EHRs. The results suggest that aligning downstream tasks with a model’s pretraining objectives can lead to significant improvements in performance, and have important implications for the development of clinical decision support systems.
Cite this article: “Foundation Models Enhance Disease Prediction Accuracy Using Electronic Health Records”, The Science Archive, 2025.
Med-Bert, Ehrs, Disease Prediction, Token Prediction, Next Visit Mask Token Prediction, Foundation Model, Pretraining Objectives, Clinical Decision Support Systems, Accuracy Improvement, Few-Shot Scenarios
