Comparing causal estimands from a sequential nested trial emulation with a treatment decision design to a conventional single point randomized trial: A simulation study

Background: Sequential nested trial (SNT) emulations index patients at regularly spaced intervals (e.g., weekly); this may not be appropriate for all study questions and data. It is possible to conduct a SNT emulation that indexes patients at treatment decision points. The causal estimands implied by such designs have not been described and compared to a conventional trial that randomizes patients once (i.e., a “single point trial” [SPT]).

Objectives: To use simulation to compare the estimates of the population average treatment effect (ATE) and average treatment effect in the treated (ATT) from (a) a SNT emulation with a treatment decision design and (b) a SPT.

Methods: We simulated 1,000 datasets of 5,000 people followed weekly for 104 weeks. Patients contributed indices at treatment decision points (visits) for the first 52 weeks. Disease severity increased over follow-up: more severe disease led to a higher visit rate and risk of the outcome. Treatment decreased the risk of the outcome. For the SPT, patients were randomized to initiate or not initiate treatment at their first index. For the SNT, the probability of treatment initiation increased with disease severity. Indices with no prior treatment initiation were eligible for inclusion, allowing patients to contribute multiple untreated indices and at most one treated index. We considered two scenarios: (1) homogeneous treatment effect and (2) heterogeneous treatment effect by disease severity. We estimated the “crude” per-protocol treatment effects for the SNT using inverse probability of censoring weighted risk ratios (RRs). We then standardized the SNT’s effect estimates to four target populations: the entire and treated SNT (SNT ATE and ATT, respectively) and the entire and treated SPT (SPT ATE and ATT, respectively). We estimated the bias and Monte Carlo standard error of the “crude” and standardized RRs compared to the SPT.

Results: The true RR in the SPT was 0.52 for scenario 1 and 0.81 for scenario 2. Minimal bias persisted in the treatment effect estimates from scenario 1. For scenario 2, the estimates in the SNT standardized to the SNT total and treated populations were less biased than the “crude”, though bias remained (log-RR bias in the Total: 0.017 [0.012, 0.021], Treated: 0.027 [0.022, 0.031], “Crude”: 0.041 [0.037, 0.046]). The estimates were unbiased after standardizing to the SPT: -0.003 (-0.008, 0.002).

Conclusions: Causal estimands from a SNT emulation with a treatment decision design may not be the same as those from a conventional SPT. Investigators should consider drivers of patient visit processes, account for these variables as confounders, and use them to standardize to the target trial population. SNT emulations should also report how their population changes over time to define their target population.