Evaluating Performance and Agreement of Coronary Heart Disease Polygenic Risk Scores.

Importance

Polygenic risk scores (PRSs) for coronary heart disease (CHD) are a growing clinical and commercial reality. Whether existing scores provide similar individual-level assessments of disease susceptibility remains incompletely characterized.

Objective

To characterize the individual-level agreement of CHD PRSs that perform similarly at the population level.

Design, setting, and participants

Cross-sectional study of participants from diverse backgrounds enrolled in the All of Us Research Program (AOU), Penn Medicine BioBank (PMBB), and University of California, Los Angeles (UCLA) ATLAS Precision Health Biobank with electronic health record and genotyping data.

Exposures

Polygenic risk for CHD from published PRSs and new PRSs developed separately from testing samples.

Main outcomes and measures

PRSs that performed population-level prediction similarly were identified by comparing calibration and discrimination of models of prevalent CHD. Individual-level agreement was tested with intraclass correlation coefficient (ICC) and Light κ.

Results

A total of 48 PRSs were calculated for 171 095 AOU participants. The mean (SD) age was 56.4 (16.8) years. A total of 104 947 participants (61.3%) were female. A total of 35 590 participants (20.8%) were most genetically similar to an African reference population, 29 801 (17.4%) to an admixed American reference population, 100 493 (58.7%) to a European reference population, and the remaining to Central/South Asian, East Asian, and Middle Eastern reference populations. There were 17 589 participants (10.3%) with and 153 506 participants without (89.7%) CHD. When included in a model of prevalent CHD, 46 scores had practically equivalent Brier scores and area under the receiver operator curves (region of practical equivalence ±0.02). Twenty percent of participants had at least 1 score in both the top and bottom 5% of risk. Continuous agreement of individual predictions was poor (ICC, 0.373 [95% CI, 0.372-0.375]). Light κ, used to evaluate consistency of risk assignment, did not exceed 0.56. Analysis among 41 193 PMBB and 53 092 ATLAS participants yielded different sets of equivalent scores, which also lacked individual-level agreement.

Conclusions and relevance

CHD PRSs that performed similarly at the population level demonstrated highly variable individual-level estimates of risk. Recognizing that CHD PRSs may generate incongruent individual-level risk estimates, effective clinical implementation will require refined statistical methods to quantify uncertainty and new strategies to communicate this uncertainty to patients and clinicians.

References:

• O’Sullivan JW, Raghavan S, Marquez-Luna C, et al. ; American Heart Association Council on Genomic and Precision Medicine; Council on Clinical Cardiology; Council on Arteriosclerosis, Thrombosis and Vascular Biology; Council on Cardiovascular Radiology and Intervention; Council on Lifestyle and Cardiometabolic Health; and Council on Peripheral Vascular Disease . Polygenic risk scores for cardiovascular disease: a scientific statement from the American Heart Association. Circulation. 2022;146(8):e93-e118. doi:10.1161/CIR.0000000000001077
• Natarajan P. Polygenic risk scoring for coronary heart disease: the first risk factor. J Am Coll Cardiol. 2018;72(16):1894-1897. doi:10.1016/j.jacc.2018.08.1041
• Klarin D, Natarajan P. Clinical utility of polygenic risk scores for coronary artery disease. Nat Rev Cardiol. 2022;19(5):291-301. doi:10.1038/s41569-021-00638-w
• Lennon NJ, Kottyan LC, Kachulis C, et al. ; GIANT Consortium; All of Us Research Program . Selection, optimization and validation of ten chronic disease polygenic risk scores for clinical implementation in diverse US populations. Nat Med. 2024;30(2):480-487. doi:10.1038/s41591-024-02796-z
• Lambert SA, Gil L, Jupp S, et al. . The Polygenic Score Catalog as an open database for reproducibility and systematic evaluation. Nat Genet. 2021;53(4):420-425. doi:10.1038/s41588-021-00783-5