Covid-19’s impact on trial PROs, other endpoints can be addressed via data stratification, sensitivity analysis

The advent of Covid-19 has left pharma companies in the psychiatry space, in particular, scrambling to understand the pandemic’s impact on patient-reported outcomes (PROs) employed in studies and similar subjective measures that gauge patient or caregiver emotions.

To salvage PROs and other results for studies in any therapeutic area that began prior to the pandemic’s start, clients are advised to parse out data collected pre-, during and post-Covid-19, as well as conduct sensitivity analysis. Sponsors are also recommended to stop studies if they’ve reached a certain power, in light of enrollment concerns with the disease.

In terms of other aspects of study design, it is crucial for sponsors to account for missing patient data, although there is some disagreement as to how lenient the FDA would be in terms of allowing for changes in statistical plans after a study has started.

PROs most affected by Covid-19

It is going to be hard to disaggregate Covid-19’s impact on any endpoint that gauges a psychological effect such as mood, said Janet Wittes, PhD, founder, Washington, DC-based biomedical statistics firm Statistics Collaborative. PROs that, for example, question overall mental health states, emotions or feelings of being in control, do not have the scope to properly measure the stresses that come from the daily disruption of Covid-19, making data collected during this period hopeless in terms of parsing out the drug’s true effect, another US statistician noted.

Covid-19’s impact in terms of patient isolation and lockdown measures has undoubtedly impacted easy patient access to clinical trial sites, which in turn will trickle down to PROs, agreed ex-FDAer Laurie Burke, lead author of the FDA Patient-Reported Outcomes Guidance originally published in 2009, and since updated, and founder of the pharma consultancy firm LORA Group. There is a particularly huge PRO impact in psychiatric conditions that affect children and teenagers, who like routine, the statistician said.

A potential approach to salvage studies is to consider analysing the data in terms of pre-, during and post-Covid-19 periods, Wittes and the biostatistician said. This is particularly useful for conditions that have long-term rather than short-term effects, she explained. The stratification windows could also be made more specific to when a stay-at-home order was issued in a particular region, the biostatistician added.

That said, if an effect is not found, that doesn’t mean it is not there, noted a second US biostatistician. Rather, it could mean that the data used to assess Covid-19’s effect does not have sufficient statistical sensitivity to detect this impact because an unplanned variable—i.e., the pandemic—had to be used to stratify the data.

Wittes noted that for an orphan disease study she is consulting on, she has advised the cut-off for data stratification for the pre-Covid term to be 28 February, Rare Disease Day. In all situations, a sensitivity analysis to understand the robustness of the primary outcome would be conducted to see if data collected during the pandemic is particularly different, said the first and second biostatisticians.

In some cases, however, Wittes has recommended that sponsors stop their study completely. For example, if a study was designed with a 90% power, but at the pandemic’s start the study had likely already achieved a roughly 80% power, which could be high enough, it may be better to halt the trial rather than continue through the pandemic, she explained.

Barreling through the Covid period may compromise data integrity, or if the treatment’s effect is different during the Covid period, the information gained may be confounded by the disease and all the protocol changes the disease necessitates, Wittes explained. The first biostatistician agreed that studies are stopping early in accounting for the outbreak’s impact on endpoints and study assessment.

In general, the question remains as to how, ultimately, the FDA will view missing data, say for patients who did not attend clinical trial visits due to lockdown measures, Wittes said. The 'FDA Guidance on Conduct of Clinical Trials of Medical Products during Covid-19 Public Health Emergency,' last updated in May, is quite broad, but the agency is upfront about needing sponsors to speak to the FDA about protocol or statistical changes and to be vigilant in explaining patient absences, Wittes and Burke said. A company cannot make the wholesale excuse that data is missing due to the pandemic and expect that to be acceptable to the agency, Burke said.

If there’s a good reason to make changes to a study’s statistical analysis plan before a trial is unblinded, the first statistician said he does not believe the agency will object, noting the FDA has recently relayed such a message to companies. However, Burke disagreed, noting that based on a webinar the agency hosted following its guidance issuance, it is clear if a trial endpoint or statistical plan is modified in any way, it must be discussed before locking the database and trial unblinding.

If companies can get buy-in from a particular review decision and document the conversation, that can be useful to explain protocol changes, said Rebecca Dandeker, partner, Morgan Lewis, Washington, DC, who represents clients on matters involving FDA-regulated products.

For more insight and data, visit GlobalData's Pharmaceuticals Intelligence Centre.