(144) Comparison of multilevel propensity score estimation methods for tooth-clustered data in the evaluation of treatment effect in the presence of competing risks

Background: Real-world data analysis often requires complicated analytical approaches, particularly in cases where competing risks exist. In dental research, it is common for tooth-level data to exhibit natural clustering within patients and competing events. Retrospective data aggregated at the patient level often presents challenges in estimating treatment effects on tooth outcomes due to the lack of tooth-level information. While there have been several analytical approaches suggested, they have not been evaluated comprehensively in dental outcomes data.

Objectives: To compare multilevel propensity-score-weighted estimators for tooth-clustered data

Methods: Using the Monte Carlo simulation method, the dental data were generated to have small-sized clusters in the presence of competing events. Five hundred patients to have a probabilistic teeth distribution were generated 1,000 times. Covariates for treatment assignment model and outcome model were generated, including one to act as an unmeasured tooth-level confounder. Outcome variables were generated based on an indirect simulation method. Nineteen hypothetical scenarios were established. A national claims database was utilized to devise the simulation scenarios and for model comparison. Six multilevel propensity score estimation methods based on machine learning techniques as well as conventional models were examined.
We computed the treatment effect and the robust variance by a semiparametric random effects model with weights obtained from the proposed models. The absolute standardized mean difference (ASMD) of the unmeasured confounder was evaluated across the models. We assessed model performance in terms of mean bias and relative coverage (RC).

Results: The resulting mean ASMD of the unmeasured confounder was from 0.03 to 0.4, two-third of which were greater than 0.2, and similar across the models in each scenario. The mean bias ranged from 0.15 to 7.5, indicating that the bias of the treatment effect estimate was inflated at least 15%. Causal forest models reduced bias by 0.5 and increased RC up to 2.9 compared to the others when we assigned the unmeasured confounder to have the coefficients in opposite sign in the treatment assignment model and outcome model. Analyzing the real-world data, we found that hazard ratios varied from 0.51 to 0.84 depending on the condition of tooth-level information and model selection, with 95% confidence intervals overlapping across all models.

Conclusions: When a within cluster confounder was not measured, a significant bias for treatment effect estimate is unavoidable even if complicated analytical approaches were attempted. Care should be taken in selection of approaches when considering propensity score model in dental outcomes.