Temporal decay in the performance of Cox proportional hazards models for deceased-donor kidney graft survival

Introduction: Deceased-donor kidneys are scarce resources, with demand significantly outweighing supply. The prediction of graft survival contributes to the utility of these organs, with the objective of maximising the lifespan of each transplanted kidney. Presently, the Cox Proportional Hazards (CoxPH) model remains the most widely used approach for predicting graft survival in deceased-donor kidney allocation, informing clinical decision making and the allocation of deceased-donor organs. In Australia, the past 20 years have seen many changes in kidney transplantation, including the introduction of a new allocation algorithm in May 2021, and the disruptive advent of COVID-19. The effects of these on the CoxPH survival models, however, are poorly understood, raising the question of whether it is valid to continue relying on models built on earlier time periods for clinical decisions. To this end, this study investigates the temporal consistency of the performance of the CoxPH model.
Methods: Using data from the ANZDATA registry, we analysed death-censored kidney graft survival data for transplants performed between February 02, 2006, and December 30, 2023. A CoxPH model was developed utilising clinical and demographic predictors based on previous studies: the recipient’s ethnicity, age, waiting time, smoker status, and their diabetes status,the donor’s age, height, and hypertension. The data was partitioned into four distinct cohorts based on the year of transplantation: 2006-2009, 2010-2013, 2014-2017 and 2018-2023. A train/test split of 70%/30% was utilised, and all models were run at 100 iterations. Model performance was evaluated within each era and against future eras using Harrell’s Concordance index (C-index).
Results: Our analysis reveals a progressive and significant decline in the predictive power of the CoxPH model over time. In the earliest cohort (2006–2009), the model demonstrated robust discrimination with a median C-index of 0.63 (IQR 0.011). However, performance steadily worsened in subsequent periods with corresponding increases in variability; within-period evaluation for 2014-2017, before the COVID-19 pandemic, yielded a C-index of 0.62 (IQR = 0.038). When applied out-of-distribution to post-COVID data from 2018-2023, an absolute reduction of 0.14 was observed as the C-Index dropped to 0.46 (IQR = 0.048). This degradation indicates a significant loss in discriminative ability compared to models trained on earlier data. Baseline domain adaptation through transfer component analysis did not yield material improvements in the observed results.
Conclusion: To our knowledge, this is the first study examining the transferability of survival models in kidney transplantation. The observed temporal decay in model performance, even in eras prior to COVID-19 and the launch of the new allocation algorithm, suggests that traditional CoxPH models may no longer adequately capture the complexities of Australia’s kidney allocation framework.