Joshua Bundy, PhD, MPH, on Risk Prediction Models of AF
Although machine learning can aid in some risk-stratification tools, it is likely not as helpful for predicting atrial fibrillation (AF) risk, according to a new analysis. 1
The analysis was led by Joshua D. Bundy, PhD, MPH, who is an assistant professor in the Department of Epidemiology at Tulane University School of Public Health and Tropical Medicine in New Orleans, Louisiana.
Cardiology Consultant reached out to Dr Bundy for more insights into his team’s research and findings.
CARDIO CON: Which risk prediction models of atrial fibrillation (AF) were most accurate, and what aspects of the models made them the best?
Joshua Bundy: We considered two measures of prediction accuracy: (1) discrimination, which measures the ability of a model to correctly classify those with and without the outcome; and (2) calibration, which measures how well predictions from a model agree with observed outcomes. In our study, the CHARGE-AF Enriched model performed well in terms of discrimination and was the best-calibrated model. More-complex models could perform better because additional variables may provide some additional information. However, such improvements were modest in our study. When we evaluated models composed of variables identified by machine learning, they increased discrimination ability slightly, but this improvement was not statistically significant.
CARDIO CON: What are the main differences between the CHARGE-AF Simple model and the CHARGE-AF Enriched model? Did the differences affect outcomes?
JB: The CHARGE-AF Simple model was developed with several common clinical characteristics in mind, including age, race, height, weight, systolic blood pressure (BP), diastolic BP, current smoking, antihypertensive medication use, diabetes status, and history of heart failure and/or myocardial infarction. Later, the CHARGE-AF team also considered whether additional, less commonly measured biomarkers could improve AF prediction performance. They found that the addition of NT-ProBNP improved performance statistically significantly, which formed the CHARGE-AF Enriched model. Our study corroborated these findings, showing that the Enriched model performed better than the Simple model in terms of discrimination and calibration.
CARDIO CON: For the study, you and your colleagues analyzed the MESA study cohort. What clinical factors among this cohort best predicted AF events?
JB: MESA is unique because the study includes only participants without a history of cardiovascular disease (CVD). The CHARGE-AF models include history of heart failure and/or myocardial infarction, which are traditionally important predictors that were strongly associated with risk of AF. It was not possible to assess these variables in the CVD-free MESA participants. Thus, when looking at the CHARGE-AF models in a sample without history of CVD, it is interesting to note that so-called upstream risk factors, like smoking status and weight, had comparatively larger magnitudes of association in our study compared with those seen in the CHARGE-AF development cohorts.
CARDIO CON: What role does the coronary artery calcium (CAC) score play in the context of predicting AF? Why should it be considered in this process?
JB: Because MESA participants did not have clinical CVD at baseline, it is possible that subclinical measures of disease, such as CAC, may be of particular value. Again, history of CVD was important for prediction in the CHARGE-AF models, so using CAC in lieu of diagnosed clinical disease could improve prediction performance specifically in this population. It is possible that CAC may represent an accumulation of CVD risk factor burden or cardiac structural changes and vascular injury that may directly explain its role in higher risk of AF.
CARDIO CON: What knowledge gaps still exist in predicting AF events?
JB: One of the biggest knowledge gaps relates to the diagnosis of AF. All risk prediction analyses of AF to date suffer from the same limitation, which is that we are only able to relate predictors to diagnosed cases of AF, based on hospitalization and insurance claims data. Many cases go undiagnosed, and we cannot develop models to predict events we cannot see.
Another gap is that we did not have information on the various biologic mechanisms that may cause AF, which are still poorly understood and may impact the choice of predictor variables. Finally, with regard to risk prediction methods, machine learning is becoming increasingly utilized and is proposed to improve clinical risk prediction. However, our study found that variables identified by machine learning did not significantly improve prediction performance. It is important to evaluate such novel findings against established models and methods. Our study suggests the CHARGE-AF Enriched score is still an appropriate standard to which novel findings can be compared.
Reference
- Bundy JD, Heckbert SR, Chen LY, Lloyd-Jones DM, Greenland P. Evaluation of risk prediction models of atrial fibrillation (from the Multi-Ethnic Study of Atherosclerosis [MESA]). Am J Cardiol. 2020;125(1):55-62. https://doi.org/10.1016/j.amjcard.2019.09.032.