The Impact of Electronic Administration of the MMSE on Error Rates and Data Quality

Abstract

Background: The Mini-Mental State Examination (MMSE) is a widely used measure of cognition in clinical practice and is a frequently used screening and outcome measure for cognition-enhancing clinical drug trials for Alzheimer’s Disease. Scoring and administration errors on the MMSE in a clinical drug trial can contribute to placebo response and unreliable data. Examples of such errors range from basic addition errors to failure to adhere to structured administration instructions. Prior research from independent clinical review of paper versions of the scale as part of a data quality monitoring program in worldwide studies has demonstrated that MMSE error rates among clinical trial raters tend to be upwards of 30% on initial administration. An enhanced tablet-based electronic (eCOA) version of the MMSE was developed for clinical trials by Bracket and has been utilized in worldwide clinical trials since 2013. The scale is administered in a system compliant with US Food and Drug Administration guidelines on the use of electronic medical records in trials and was designed to be equivalent to the paper MMSE. The eCOA version was augmented with internal logic, automated scoring and addition, standardized instructions, and enforcement of administration conventions. These enhancements were intended to decrease errors and increase the likelihood that the scale would be consistently administered and scored according to standard conventions.

Objective: Our objectives were to (1) determine if error rates using the eCOA MMSE were reduced when compared to standard paper administration and comparable to eCOA error rates from an analysis published in 2015 and (2) consider the impact of utilizing enhanced eCOA versions of scales like the MMSE in clinical practice.

Methods: All data from this analysis and comparative analyses were from multinational studies that trained and certified raters on the MMSE and included an in-study data monitoring program of clinical reviews to detect scoring and administration errors. Error rates on the enhanced electronic versions of the scales used in a late-phase Alzheimer’s study were compared to those previously seen in other studies when using paper versions and the enhanced eCOA MMSE. The dataset for this eCOA analysis was limited to English-speaking sites.

Results: A total of 66/626 (10.5%) of the enhanced eCOA MMSEs reviewed at the inclusion visits contained an error in scoring which required outreach to the rater. This represents a statistically significant reduction (P<.0001) in the error rates compared to paper administration of the MMSE (877/2889, 30.4%) and is consistent with the findings from a prior analysis of an eCOA MMSE dataset from another trial (107/990,10.8%).

Conclusions: Use of enhanced eCOA for the MMSE in clinical trials has significantly improved error rates and data quality compared to standard paper administration. Enhanced eCOA that is designed to minimize errors and enforce administration standards could be a useful tool to reduce common errors and increase the precision of the MMSE use in clinical practice.

Keywords