Researchers: Leonard Leibovici^{1, 2}, Dafna Yahav^{1, 2}

Background: Health care policy decisions ought to be based on the best evidence. While considerable efforts have been invested in order to improve internal validity of randomized controlled trials (RCTs), work on external validity is scarce.

Objectives: We aimed to assess the population external validity of four RCTs performed in our research unit by comparing characteristics and outcomes of patients included in the trials to those who were not. In addition, we performed a systematic review of RCTs to explore whether studies with a higher rate of included participants resulted in biased estimates of beneficial intervention effect.

Method: We conducted four observational studies of patients who fulfilled inclusion criteria for the RCTs but were not included due to various reasons. We assessed the numbers of patients excluded from the studies according to reasons for exclusion. We also compared excluded patients with included in each of the 4 RCTs.

Findings: Differences between trial participants and excluded patients were observed. Two of the four trials were characterized by a more complicated cohort of excluded patients. Excluded patients had significantly more comorbidities and more severe infections and higher rates of clinical failure and mortality. In one of the trials, patients not randomized were similar to randomized patients. In the remaining trial, included patients were older and the rate of clinical failure was significantly higher among them.

Conclusions: Nonrestrictive inclusion criteria and access to recruiting the most severely ill patients into the trial population are key elements conferring high population external validity.

Recommendations: Recruitment of patients by approval of an independent physician is one solution for inclusion of severe patients and starting trial treatment without delays. Reporting the number and sub-groups of candidates that were not recruited will also improve our grasp of the external validity of trials.

Research end date: 11/2020