Impact of acute kidney injury labeling strategies in hospitalized patients: suggestion for a dual-alert clinical decision support system
Article information
Abstract
Background
Operational variability in defining acute kidney injury (AKI) undermines the diagnostic reliability and impairs the generalizability of machine learning (ML) models. We evaluated whether the refined criteria based on the Kidney Disease: Improving Global Outcomes (KDIGO) guidelines enhance prognostic validity and predictive performance.
Methods
Two conservative refinements to the KDIGO serum creatinine (SCr)-based criteria were proposed: 1) exclusion of transient SCr decreases from baseline estimation and 2) application of a minimum absolute SCr increase threshold. We generated 441 AKI labeling strategies by combining these refinements with baseline estimation methods. For each strategy, we calculated adjusted hazard ratios (aHRs) for adverse outcomes and assessed CatBoost model performance using the area under the receiver operating characteristic curve (AUROC) and area under the precision–recall curve (AUPRC). To assess real-world interpretability, ambiguous cases were reviewed.
Results
Among 5,115 patients labeled by the standard KDIGO, 4,137 (80.9%) met the refined criteria, and 978 (19.1%) were considered ambiguous. The refined cohort had higher adverse event rates than the standard cohort (44.0% vs. 37.8%; aHR, 1.23). ML models trained with refined labels showed improved discrimination (AUROC, 0.935 vs. 0.926) and precision (AUPRC, 0.729 vs. 0.707) and reduced false-positive alerts by 27% with minimal loss in adverse outcome detection. Ambiguous cases often reflected transient measurement variability or frailty.
Conclusion
This study presents a strategic AKI labeling approach based on KDIGO criteria and suggests a dual-alert system to distinguish refined AKI and ambiguous AKI cases. These approaches may enhance both predictive model performance and clinically meaningful detection.
Introduction
Acute kidney injury (AKI), defined as an abrupt increase in serum creatinine (SCr) or a reduction in urine output [1], is a prevalent and clinically significant complication in hospitalized patients [2,3]. AKI is independently associated with adverse outcomes, such as prolonged hospitalization, progression to chronic kidney disease (CKD), renal replacement therapy (RRT), and increased mortality [4–6]. Therefore, early detection and timely intervention are essential for improving prognosis and optimizing healthcare resource utilization [7,8].
Recently, machine learning (ML) models trained on electronic health record (EHR) data have demonstrated considerable potential for predicting AKI at an early stage [9]. Studies leveraging large-scale intensive care unit datasets, such as the Medical Information Mart for Intensive Care (MIMIC), have reported high predictive performance across various modeling approaches [10–12]. However, the clinical utility and generalizability of AKI prediction models remain limited owing to the variability in AKI labeling strategies and inherent complexities of real-world EHR data [13–15].
The limitations of these models are attributable to a lack of consensus and consistency regarding the operational definition of AKI in retrospective data [16]. Although the Kidney Disease: Improving Global Outcomes (KDIGO) guidelines offer a standardized diagnostic framework, their application in real-world EHR datasets is often constrained by incomplete patient history, irregular presentation timing, and inconsistent laboratory testing intervals [17,18]. Furthermore, heterogeneity in baseline SCr estimation, ranging from preadmission values to empirical approaches, such as mean or minimum values over predefined windows, has led to discrepancies in AKI labeling and hindered model reproducibility [19–22]. In addition, refined labeling strategies that incorporate the clinical context need to be developed. In large-scale datasets, rigid application of standard criteria may fail to capture the nuanced and dynamic nature of kidney injury, leading to potential over- or under-diagnosis [23]. For instance, transient physiological fluctuations, early recovery phases, or small SCr changes in patients with low baseline values may be misclassified without proper contextualization [24]. Therefore, clinically meaningful refinements should be incorporated to ensure the accurate and robust labeling of supervised learning tasks [25–27].
Thus, this study systematically evaluated the effect of operational AKI definition variability on prognostic associations and ML model performance. We constructed 441 distinct labeling strategies by combining different baseline SCr estimation methods, transient drop thresholds, and minimum absolute increases in cutoff values. We comprehensively analyzed data to identify a clinically grounded and reproducible framework for AKI labeling. Ultimately, we propose that distinguishing clearly defined AKI from borderline or ambiguous cases by providing separate alerts in clinical decision support systems may enhance the interpretability and actionability of AKI predictions in real-world practice.
Methods
Study population
This retrospective cohort study utilized structured EHR data from Soonchunhyang University Cheonan Hospital, in the Republic of Korea. The dataset included 67,000 inpatients admitted to general wards between January 2013 and December 2022. The study protocol was approved by the Institutional Review Board of Soonchunhyang University Cheonan Hospital (No. 2020-02-016), and the requirement for informed consent was waived due to the retrospective nature of the study and the use of de-identified data.Patients aged ≥19 years with at least three documented SCr measurements during hospitalization were eligible for the study. The exclusion criteria were: 1) estimated glomerular filtration rate (eGFR) <60 mL/min/1.73 m2 or not calculable at admission (Chronic Kidney Disease Epidemiology Collaboration 2021 equation [28]); and 2) implausible or erroneous clinical records.
Refined acute kidney injury definition
AKI was defined according to the changes in SCr levels based on the KDIGO criteria, excluding urine output [17,29,30]. The detailed definitions of KDIGO-AKI are provided in Supplementary Table 1 (available online).
To explore the impact of AKI definition variability, we systematically constructed 441 refined AKI definitions by modifying the standard KDIGO-AKI labeling based on two parameters:
First, the ‘dropThreshold’ is a parameter that defines the minimum allowable decrease in SCr when estimating the baseline value. If the current SCr level was lower than the median of prior SCr measurements by an amount greater than the dropThreshold, the value was excluded from the baseline estimation to reduce the influence of transient non-pathological fluctuations. Seven thresholds ranging from 0 to 0.6 mg/dL (in 0.1 mg/dL increments) were tested to evaluate their impact.
Second, the ‘AbsoluteIncrease’ is a parameter that specifies the minimum absolute increase in SCr required to define AKI, particularly in patients with low baseline SCr. It was introduced to avoid false-positive classifications due to small, clinically insignificant fluctuations. A total of 21 thresholds were tested, ranging from 0.15 to 0.35 mg/dL in increments of 0.01 mg/dL.
Baseline SCr was calculated using min/median/mean over 2- or 7-day windows [31,32]. If unavailable, the most recent value within 180 days was used. Patients without baseline data were excluded. In total, 441 labeling combinations were generated (7 × 21 × 3). Supplementary Fig. 1 (available online) presents the pseudocode.
Among these, we selected one representative refined definition—referred to as refined AKI—that applied a dropThreshold of 0.3 mg/dL, an AbsoluteIncrease of 0.3 mg/dL, and the minimum baseline SCr method. These thresholds were informed by the KDIGO guideline, which includes an absolute SCr increase of ≥0.3 mg/dL as a key criterion for AKI. The minimum method was adopted both to offset the conservativeness of these refined thresholds—by allowing more patients to meet the criteria—and because it is widely used for baseline estimation in the literature. Fig. 1 provides a visual overview of the refined AKI definition. The primary endpoint was the occurrence of adverse outcomes defined as a composite of in-hospital mortality and RRT initiation.
Operational framework for refined AKI labeling.
KDIGO-AKI: Patients with AKI defined by the original KDIGO (Kidney Disease: Improving Global Outcomes) criteria based on changes in serum creatinine (SCr). Refined-AKI: patients who met refined KDIGO criteria, incorporating both a dropThreshold (0.3 mg/dL) and an AbsoluteIncrease threshold (≥0.3 mg/dL) using the minimum SCr as baseline. Onset SCr: serum creatinine value at the time when AKI criteria were first met. SCr7d baseline: the baseline SCr defined as the lowest SCr value measured within the 7 days preceding the measurement day. If this lowest value shows a decrease greater than the dropThreshold relative to the preceding median SCr within the lookback window, it is excluded, and the next-lowest value that does not exceed the dropThreshold is selected as the baseline.
AKI, acute kidney injury.
Data processing
Structured EHR data included demographics, vital signs, laboratory results, comorbidities, medications, contrast imaging, surgery, hospital stay, mortality, and RRT. For all patients, EHR records were extracted from the first day of admission until discharge. To address temporal irregularities and missingness, time-series data were aggregated into 24-hour intervals [33], with vital signs summarized by maximum values. The laboratory data were processed separately for statistical analysis and model development.
For the statistical analysis, AKI onset-day variables were used for patients with AKI, and the earliest post-admission values were used for patients without AKI. For modeling, patients with AKI were labeled as positive (1) based on data from 1 to 3 days before onset, excluding the onset day. Those without AKI were labeled negative (0). Regarding missing laboratory values [33], they were first forward-filled, and the remaining missing data were imputed using multiple imputation by chained equations for variables with <20% missingness. Binary indicators were added for those with ≥20% [34–36].
Modeling and evaluation
To ensure labeling consistency, the test cohort was constructed from patients consistently classified across the 442 AKI definitions (441 refined and one standard). Stage-specific stratification was applied to AKI cases to preserve class balance, and 20% of patients without AKI were randomly sampled.
The remaining cohort was used to train a CatBoost classifier for predicting AKI within a 3-day window prior to onset [16,37]. The model was trained on all patients excluded from the test set. The model parameters and features are presented in Supplementary Table 2 (available online).
The model performance was evaluated using the area under the receiver operating characteristic curve (AUROC), area under the precision–recall curve (AUPRC), and precision at 70% recall. The predicted AKI events were further assessed for their association with adverse outcomes to evaluate their clinical utility.
Statistical analysis
Descriptive statistics were used to summarize the baseline characteristics. Continuous variables are expressed as means with standard deviations or medians with interquartile ranges (IQRs), depending on the distribution. Categorical variables are summarized as frequencies and percentages.
Groups were compared using the chi-square test for categorical variables and Student t test or Mann-Whitney U test for continuous variables, as appropriate. Cox proportional hazards regression was used to assess the associations between AKI definitions and adverse outcomes, adjusting for potential confounders, including age, sex, Charlson Comorbidity Index (CCI), baseline eGFR, and length of stay.
Survival analyses were conducted using Kaplan-Meier curves with log-rank tests. The proportional hazard assumption was verified using Schoenfeld residuals. To evaluate performance differences, AUROC was compared across combinations of dropThreshold and AbsoluteIncrease under the minimum baseline SCr method, using KDIGO-AKI as the reference. Pairwise differences were assessed with DeLong test [38]. To account for multiple comparisons, the Benjamini-Hochberg procedure was applied to control the false discovery rate (FDR), and q-values were reported [39]. Statistical significance was defined as a two-sided p < 0.05. All the analyses were performed using Python (version. 3.8, Python Software Foundation) and R (version 4.1.1, R Foundation for Statistical Computing).
Results
Cohort characteristics
The final study population, as defined by the inclusion criteria in Fig. 2, comprised 28,229 adult inpatients with a mean age of 60.0 years (standard deviation, 17.3), of whom 55.2% were male. The median hospital stay was 17 days (IQR, 7–151 days). Based on the standard KDIGO-AKI definition, 5,115 patients (18.1%) were classified as having AKI, whereas the remaining 23,114 patients were categorized as non-AKI.
Flow diagram of cohort selection for the AKI study.
AKI, acute kidney injury; eGFR, estimated glomerular filtration rate; KDIGO-AKI, Kidney Disease: Improving Global Outcomes AKI definition.
Applying the refined AKI criteria proposed in this study, 4,137 patients were classified as having refined AKI and 978 as having ambiguous AKI. A greater proportion of patients in the refined AKI group experienced adverse outcomes than those in the KDIGO-AKI group (44.0% vs. 37.8%), suggesting improved prognostic specificity.
Table 1 summarizes the characteristics of patients who met the refined AKI and ambiguous AKI definitions. Compared with the ambiguous AKI cohort, the refined AKI cohort had a higher proportion of males and was older. Maximum systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, and body temperature were higher. For kidney function, blood urea nitrogen and SCr were higher, whereas eGFR was lower. Laboratory measures showed higher C-reactive protein, lactate dehydrogenase, alanine aminotransferase, glucose, triglycerides, total bilirubin, uric acid, potassium, and phosphate, with lower platelet counts and bicarbonate/total carbon dioxide. Distributions of arterial pH, partial pressure of carbon dioxide, partial pressure of oxygen, and pro–B-type natriuretic peptide categories differed. The CCI was shifted toward higher categories. Nonsteroidal anti-inflammatory drug use and surgery were more common in the ambiguous AKI cohort, whereas chemotherapy was more common in the refined AKI cohort.
Baseline demographic and clinical characteristics of refined AKI patients and ambiguous AKI patients
Supplementary Table 3 (available online) presents baseline characteristics comparing patients with KDIGO-based acute kidney injury and those without acute kidney injury.
Clinical outcomes
Fig. 3 summarizes the clinical impacts of the 441 refined AKI definitions. The following representative parameter sets were selected for comparison with the standard KDIGO-AKI: the parameter set of the refined AKI definition proposed in this study (dropThreshold, 0.3 mg/dL; AbsoluteIncrease, 0.3 mg/dL); the most lenient (dropThreshold, 0.6 mg/dL; AbsoluteIncrease, 0.15 mg/dL); and the most stringent (dropThreshold, 0.0 mg/dL; AbsoluteIncrease, 0.35 mg/dL).
Clinical outcomes across refined AKI definitions.
(A) Adjusted hazard ratio (aHR) and 95% confidence interval for adverse outcomes across 441 AKI labeling strategies derived by varying the drop and absolute increase thresholds applied to three baseline serum creatinine (SCr) estimation methods (minimum, median, and mean). The red dashed line indicates the aHR for conventional Kidney Disease: Improving Global Outcomes (KDIGO)-defined AKI (reference), and the blue dashed line represents the aHR for patients classified as non-AKI. (B) For each of the 441 AKI labeling strategies, this panel visualizes. Bold dotted line: number of patients classified as having AKI. Solid line: number of adverse outcome events. Dotted line: number of adverse outcomes missed across all patients. Mixed bold-dot and simple-dot lines: number of adverse outcomes missed specifically among patients identified as having AKI according to the KDIGO definition. Baseline setting: the method was used to estimate baseline SCr levels, including minimum, median, or mean values within the 2- or 7-day look-back period. dropThreshold: the predefined minimum decrease in SCr level used to exclude transient drops from the baseline estimation (range, 0–0.6 mg/dL). AbsoluteIncrease: minimum absolute increase in SCr required to define AKI (range, 0.15–0.35 mg/dL).
AKI, acute kidney injury; CCI, Charlson Comorbidity Index.
Each group was evaluated using minimum, median, and mean baseline SCr estimates and assessed for AKI incidence, adverse outcomes, and adjusted hazard ratios (aHRs).
According to the KDIGO definition, 5,115 patients were diagnosed with AKI, with 1,935 experiencing adverse outcomes (37.8%). Using the minimum SCr method, the lenient, refined AKI, and stringent definitions identified 5,052, 4,137, and 2,854 AKI cases with 1,928, 1,819, and 1,586 events, respectively. Similar reductions were observed for the median and mean baseline methods.
The degree of reduction varied according to the definition of stringency and the baseline method. The stringent definition using the mean method excluded 2,388 AKI cases and 384 events, showing the highest incidence (56.9%) and strongest association (aHR, 1.70). By contrast, the lenient definition with the minimum method showed minimal change (–63 AKI cases, –7 events; incidence: 38.2%; aHR, 1.01). The refined AKI parameter set excluded 978 cases and 116 events (incidence, 44.0%; aHR, 1.23).
Across all baseline methods, lowering the dropThreshold from 0.3 to 0.0 mg/dL increased stringency and consistently reduced AKI incidence, primarily through stage 1 reclassification. Supplementary Fig. 2 (available online) illustrates these trends. The complete results for all combinations are shown in Supplementary Table 4 (available online).
Prediction model performance
Fig. 4 compares the predictive performances of the 441 refined AKI definitions using the CatBoost models trained on a common test cohort. A total of 328 patients in the test group experienced adverse outcomes.
Predictive performance and clinical utility across refined AKI definitions.
(A) Predictive performance metrics: area under the receiver operating characteristic curve (AUROC), area under the precision–recall curve (AUPRC), and precision at a fixed recall of 70% for CatBoost models trained on each of the 441 AKI definitions by combining varying drop and absolute increase thresholds with the three baseline serum creatinine (SCr) estimation strategies. (B) Clinical implications of each labeling strategy showing the number of patients predicted as AKI-positive (predicted 1 count). Number of predicted AKI patients who subsequently experienced adverse outcomes (event ID intersection). Baseline setting: the method was used to estimate baseline SCr levels, including minimum, median, or mean values within the 2- or 7-day look-back period. dropThreshold: the predefined minimum decrease in SCr level used to exclude transient drops from the baseline estimation (range, 0–0.6 mg/dL). AbsoluteIncrease: minimum absolute increase in SCr required to define AKI (range, 0.15–0.35 mg/dL).
AKI, acute kidney injury.
The model trained using the standard KDIGO-AKI definition achieved an AUROC of 0.926, an AUPRC of 0.707, and a precision of 0.447. Among the 1,350 patients predicted to have AKI, 228 experienced adverse outcomes.
When applying the lenient definition, the model using the minimum baseline SCr level method showed an AUROC (0.927) and AUPRC (0.708) comparable to those of the KDIGO model, with slightly lower precision (0.434). A total of 1,393 patients were predicted to have AKI, of whom 227 experienced events. The number of predicted AKI cases increased by 43, whereas the number of events decreased by one. Using the median and mean baseline methods, the AUROCs were 0.933 and 0.935, AUPRCs were 0.723 and 0.728, and precision was 0.520 and 0.527, respectively. The numbers of predicted AKI cases and events were 1,127 (223 events) and 1,105 (219 events), respectively.
With the refined AKI parameter set, the model achieved an AUROC of 0.935, AUPRC of 0.729, and precision of 0.533 using the minimum method with 1,125 predicted AKI cases and 225 events. The performance improved slightly with the median method (AUROC, 0.937; AUPRC, 0.735; precision, 0.565; 1,041 cases, 222 events) and remained stable with the mean method (AUROC, 0.936; AUPRC, 0.733; precision, 0.544; 1,070 cases, 220 events).
The model using the stringent definition and minimum method demonstrated the best overall performance, with an AUROC of 0.954, AUPRC of 0.775, and a precision of 0.689. Of the 875 predicted cases, 223 experienced adverse outcomes. Compared with the KDIGO model, this corresponded to improvements of 0.028 in AUROC, 0.068 in AUPRC, and 0.242 in precision. The performance remained high for the median and mean methods (AUROC, 0.954 and 0.952; AUPR, 0.769 and 0.766; precision, 0.674 and 0.685), although fewer cases were predicted (890 and 873, respectively), with slightly fewer events (220 and 219, respectively).
Supplementary Fig. 3 (available online) presents DeLong p-values for AUROC comparisons against the KDIGO-AKI reference, with the baseline set using the minimum SCr method, together with FDR-adjusted q-values. As dropThreshold and AbsoluteIncrease became more stringent, p-values tended to decrease. The first significant difference by DeLong test was observed at dropThreshold, 0.3 mg/dL and AbsoluteIncrease, 0.33 mg/dL. All combinations with dropThreshold, 0.1 or 0.0 mg/dL were significant. In total, 53 combinations were significant before multiplicity adjustment, of which 44 remained significant after FDR control.
Comprehensive performance metrics for all labeling combinations are presented in Supplementary Table 5 (available online).
Serum creatinine trajectories and subtype risk
Fig. 5 illustrates the differences in SCr trajectories between ambiguous AKI and refined AKI subtypes (Fig. 5A) and clinical risk profiles of non-AKI and refined AKI groups (Fig. 5B), using ambiguous AKI as a reference. Among patients with available SCr measurements up to 7 days after AKI onset, 482 were classified as ambiguous AKI and 1,403 as refined AKI. The proportion of patients whose SCr remained ≥1.5 times baseline—suggesting incomplete renal recovery—was higher in the refined AKI group (16.1%) than in the ambiguous AKI group (5.6%). The aHRs in (B) show that the non-AKI group had a lower risk (aHR, 0.63; 95% confidence interval [CI], 0.53–0.74; p < 0.001), whereas the refined AKI group had a significantly higher risk (aHR, 4.91; 95% CI, 4.17–5.79; p < 0.001).
SCr trajectories and clinical risk by AKI subtype.
(A) Individual SCr trajectories over a 7-day window centered on the day of AKI onset (day 0) and stratified by AKI subtype (ambiguous AKI vs. refined AKI). The green line indicates the median trajectory. Red lines denote patients who failed to recover to <1.5× baseline SCr within 7 days; gray lines represent those who achieved recovery. (B) Adjusted hazard ratio (aHR) with 95% confidence interval (CI) for adverse outcomes, comparing the non-AKI and refined AKI groups using the ambiguous AKI group as the reference. Ambiguous AKI: patients with AKI diagnosed according to the standard Kidney Disease: Improving Global Outcomes (KDIGO) criteria who did not meet the refined definitions. Refined AKI: patients who met refined KDIGO criteria, incorporating both a drop threshold (≥0.3 mg/dL) and an absolute increase threshold (≥0.3 mg/dL) using the minimum SCr as baseline. Non-AKI: patients who did not meet the KDIGO criteria for AKI.
AKI, acute kidney injury; SCr, serum creatinine.
Expert review of ambiguous acute kidney injury cases
Table 2 summarizes the findings of the expert case review. Two independent nephrologists, who were granted access to the institutional EHR system, reviewed a subset of de-identified patient IDs without being provided the algorithmic classification results. These IDs were pseudonymized, and only authorized MDs at the participating hospital were able to link them to actual patients. Each reviewer independently examined the full clinical records, and discrepancies were resolved by consensus to produce the final adjudication. Application of the refined criteria showed high concordance with these independent assessments. The ambiguous AKI group predominantly included patients in terminal clinical states or those exhibiting substantial SCr variability due to non-renal factors, such as sarcopenia, nutritional depletion, fluid shifts, or analytic variability. Several cases showed apparent increases or decreases in SCr relative to baseline; however, these patterns were often attributed to measurement artifacts (e.g., use of non-standard specimens) or biological fluctuations and were ultimately deemed clinically insignificant by expert adjudicators. By contrast, refined AKI cases generally displayed consistent and clinically plausible SCr elevations, frequently accompanied by adverse events or underlying pathologies, such as infection, malignancy, or liver dysfunction.
Discussion
Although AKI is a clinically significant complication, retrospective studies using real-world EHR data often have ambiguities in their definitions. Variability in baseline SCr estimation methods, irregular measurement intervals, and a lack of clinical context can compromise the reliability of AKI labeling, ultimately undermining the validity and interpretability of predictive models.
Previous studies have highlighted the impact of these inconsistencies. Rahimi et al. [22] have demonstrated that AKI incidence and model performance fluctuated by over 30%, depending on whether the minimum, mean, or MDRD (Modification of Diet in Renal Disease)-derived baseline SCr was used in the MIMIC-IV data. In some cases, patients were labeled as either AKI or non-AKI depending solely on the estimation method, limiting the generalizability of the findings. Porschen et al. [40] proposed the pyAKI tool to standardize the application of the KDIGO criteria but also emphasized the risk of overestimation or underestimation of AKI severity due to inaccurate baseline estimation, underscoring the need for context-aware adjustments. Recent efforts such as the extended KDIGO definitions have attempted to incorporate post-AKI recovery patterns into labeling, offering a more context-sensitive approach [26,27]. However, they remain vulnerable to overclassification because of transient physiological fluctuations or measurement errors unrelated to true kidney injury.
To improve the reliability of AKI labeling in EHR-based research, this study introduced two conservative refinement criteria. First, to prevent non-renal overclassification arising from transient SCr drops due to fluid administration or short-term measurement errors, we excluded cases in which the reference SCr level decreased sharply just before AKI onset. This was operationalized using a dropThreshold of 0.3 mg/dL. Hence, AKI cases were reduced from 5,052 to 4,808 (4.8% decrease), and adverse outcomes slightly declined from 1,928 to 1,892 (1.9% decrease), suggesting that non-renal artifacts were effectively filtered without substantially compromising sensitivity.
Second, we addressed scenarios in which baseline SCr levels may be abnormally low due to muscle wasting, dehydration, or volume overload, which can result in small absolute changes in SCr levels that do not reflect meaningful kidney injury. To mitigate misclassification in such cases, we implemented an AbsoluteIncrease threshold to capture only the clinically significant elevations. Under the standard KDIGO-AKI definition, patients with baseline SCr ≤0.4 mg/dL (n = 177) were labeled as AKI with small increases (0.15 mg/dL). Using our refinement criteria, the average increase among these patients was recalibrated to 0.31 mg/dL. Similarly, among patients with baseline SCr ranging from 0.5 to 0.8 mg/dL (n = 1,340), the average threshold was strengthened to 0.32–0.34 mg/dL, thereby reducing AKI mislabeling due to minor fluctuations.
These two criteria were applied across all three baseline estimation methods (minimum, median, and mean), resulting in 441 definitions of operational AKI. Each definition was systematically evaluated in terms of patient characteristics, AKI incidence, adverse outcomes, and model performance. Stringent definitions tend to reduce the AKI population while capturing more severe phenotypes, thereby improving predictive precision. By contrast, lenient definitions increased AKI detection but diluted clinical specificity, suggesting that sensitivity can be preserved at the cost of specificity, depending on the selected criteria. These findings highlight the importance of aligning AKI definitions with study or clinical objectives, as even minor changes in operational thresholds or baseline estimations can meaningfully impact labeling and downstream analysis.
Particularly noteworthy were cases that met the standard KDIGO-AKI criteria but did not satisfy our refinement thresholds. We categorized these as ambiguous AKI and proposed separate handling for such borderline cases. Among these patients, 73% experienced an SCr drop immediately prior to AKI onset, 55% had SCr values <0.6 mg/dL at onset, and 94% recovered within 7 days. Expert reviews have attributed these patterns to terminal clinical states, sarcopenia, fluid shifts, or measurement noise, rather than true renal injury. Importantly, however, ambiguous AKI patients demonstrated worse outcomes compared to non-AKI individuals (Fig. 5B), underscoring their clinical relevance and the potential risk of disregarding them. When ambiguous AKI cases were excluded from model training, the AUROC and AUPRC both increased from 0.912 to 0.935 (+0.023) and from 0.684 to 0.729 (+0.045), respectively, confirming that managing definitional uncertainty can improve predictive performance. Nevertheless, these gains should not be interpreted as justification for excluding ambiguous cases in clinical practice. Instead, we propose a dual-alert framework in which clear-cut AKI cases defined by refined thresholds trigger high-priority alerts accompanied by explanatory features, while ambiguous cases prompt lower-priority alerts supplemented with contextual information such as comorbidities, treatments, or demographic risk factors. This two-tiered strategy can mitigate alert fatigue while ensuring that borderline but potentially meaningful cases are actively reviewed, thereby supporting precision medicine in AKI management.
However, this study had several limitations. First, it was conducted at a single tertiary hospital in the Republic of Korea, which may limit the generalizability of our findings to other populations, healthcare systems, or laboratory infrastructures. Recognizing this issue, we plan to conduct external validation across multiple institutions and prospective cohorts to further establish the robustness and clinical applicability of the proposed AKI definitions. Second, to minimize bias from community-acquired AKI that could not be reliably distinguished, we restricted the cohort to patients with eGFR ≥60 mL/min/1.73 m2 at admission. While this conservative strategy reduced misclassification risk, it also fails to reflect the clinical reality that AKI is more frequent among patients with underlying CKD. We therefore emphasized this limitation and propose future work extending the refined framework to populations with eGFR <60 mL/min/1.73 m2, including sensitivity analyses and potential integration with emerging approaches such as extended KDIGO. Third, baseline SCr was defined dynamically using a rolling look-back window. At each index time point, measurements within the preceding 7 days were used; if unavailable, the window was extended to 180 days. Patients without any measurement within this extended window were excluded from analysis rather than imputed. This minimized bias from unreliable back-estimation and reduced the risk of misclassification due to inaccurate baselines, thereby strengthening internal validity. However, it may limit external generalizability, as excluded patients could represent a clinically distinct group, raising concerns for selection bias. Moreover, by not applying back-calculation methods assuming normal GFR, some AKI cases may have been underestimated. Fourth, although we systematically evaluated 441 combinations for clinical plausibility and statistical performance, we acknowledge that the selection of our representative refined definition (dropThreshold, 0.3 mg/dL; AbsoluteIncrease, 0.3 mg/dL) involved a degree of arbitrariness. While we determined these thresholds through review of KDIGO criteria and hazard ratio analyses, the continuous nature of performance changes across all three refinement approaches made it challenging to identify clear inflection points for optimal threshold selection. For the AbsoluteIncrease parameter, we referenced the established KDIGO criterion of “SCr increase ≥0.3 mg/dL” as one of the key AKI definitions. For consistency and clinical interpretability, we applied the same 0.3 mg/dL threshold to the dropThreshold parameter, which represents the point where hazard ratios first achieved statistical significance across all dropThreshold ranges in our analysis. To address this limitation, we performed additional DeLong testing across various threshold combinations using the minimum baseline SCr method and applied multiple comparison corrections to validate performance differences. However, we recognize that optimal performance metrics alone cannot fully justify threshold selection, as more stringent criteria tend to increase ambiguous cases while potentially excluding clinically relevant borderline AKI. These fixed-threshold choices may not adequately capture inter-individual biological and analytical variability. Future studies should incorporate reference change value-informed cutoffs, symmetric thresholds, and percentage-based criteria to improve calibration across different baseline strata and reduce the inherent subjectivity in threshold selection. Lastly, despite applying outlier filtering, abrupt and biologically implausible SCr elevations were observed in some cases, likely owing to laboratory or data entry errors. Future studies should incorporate more robust anomaly detection techniques and consider integrating biomarkers that are less influenced by muscle mass or hydration status, such as cystatin C, to enhance diagnostic precision.
Importantly, this study was designed as a methodological framework to systematically compare 441 refined AKI definitions rather than to declare one definition as definitively superior. We fully agree that the observed performance differences may partly reflect label distribution or model dynamics, rather than the intrinsic validity of any single definition. Therefore, the higher predictive accuracy observed in certain definitions should not be interpreted as evidence of absolute superiority, but rather as supporting data that some labeling strategies may capture clinically meaningful signals more effectively. Our comparative approach should be regarded as providing indirect evidence to guide the operational utility of different AKI definitions, while acknowledging that methodological constraints prevent definitive conclusions.
In conclusion, the choice of AKI definition is not a trivial preprocessing step but a pivotal methodological decision with far-reaching implications for incidence estimation, prognosis, and prediction model performance. The refined AKI labeling framework introduced here provides a robust foundation for clinically meaningful AKI prediction by quantifying the impact of 441 refined AKI definitions and offers clear guidance for researchers and clinicians seeking to align their definitions with the specific goals and characteristics of their datasets.
Supplementary Materials
Supplementary data are available at Kidney Research and Clinical Practice online (https://doi.org/10.23876/j.krcp.25.248).
Notes
Conflicts of interest
All authors have no conflicts of interest to declare.
Funding
This research was supported by the IITP(Institute of Information & Communications Technology Planning & Evaluation)-ICAN(ICT Challenge and Advanced Network of HRD) grant funded by the Korea government(Ministry of Science and ICT)(IITP-2025-RS-2022-00156439) and the Korea Institute for Advancement of Technology (KIAT) grant funded by the Korean Government (MOTIE) (P0023675, HRD Program for Industrial Innovation). All the authors have agreed to the publication of this manuscript and declare no conflicts of interest.
Data sharing statement
The datasets used in this study and the developed models are not publicly available owing to ethical restrictions, patient confidentiality, or institutional policies. However, the data presented in this study are available from the corresponding author upon reasonable request.
Authors’ contributions
Conceptualization: SJA, NJC, HWG
Data curation, Investigation, Validation: All authors
Formal analysis: NJC, IJ
Funding acquisition: HL
Methodology: SJA, NJC, HWG
Project administration, Resources: YK, BK, HWG
Supervision: HWG, HL
Writing–original draft: SJA, NJC, IJ
Writing–review & editing: All authors
All authors read and approved the final manuscript.
