Piloting and Evaluating an Automated Text-Messaging System in the Veterans Health Administration

Abstract

Background: The Veterans Health Administration (VA) strives to increase access and patient-centered care. “Annie” is VA’s first national automated text messaging system aimed to give patients a self-management tool to take charge of their health and become more engaged in their own care. We examined early patient/provider experiences with Annie as part of a limited field test and subsequently examined if the texting system would improve outcomes.

Objective: To understand early experiences using Annie and subsequently pilot and evaluate automated text messages to improve Veteran self-management and adherence to medications and appointments.

Methods: We conducted a national two-phase study in VA. In Phase 1, five sites conducted limited field testing and engaged 43 respondents (23 patients, 20 providers). Respondents completed surveys and qualitative interviews focused on provider adoption, integration into clinic workflow, patient ease of use, perceptions of program effectiveness, and barriers to use. In Phase 2, seven sites implemented Annie. Four intervention sites received a toolkit of materials and facilitation calls intended to enhance Annie implementation. An additional three sites received Annie without implementation support and two matched comparison sites did not receive Annie. A mixed-methods evaluation is underway, and includes pre and post patient and provider surveys and interviews, medical chart abstraction, and process measure analysis.

Results: Analysis of Phase 1 data revealed that all participating Veterans felt positive about their ability to receive a text message. All but one Veteran used a smart phone and had been using a cell phone for four years or more, with 83% sending and receiving text messages several times a day. Sixty-seven percent of Veterans agreed that Annie was easy to use, 11% felt they needed to learn a lot to use Annie, 46% felt Annie helped them take better care of their health and become more connected with their clinical team, and 91% would recommend Annie to another Veteran. Among providers, 67% agreed leadership and management would support Annie implementation and 60% would recommend Annie to another provider. Analysis of Phase 1 interviews revealed five lessons to support implementation: 1) the importance of identifying a resource person who is able to bridge technology and clinical issues; 2) promoting the evidence, innovation, and patient empowerment associated with Annie to providers; 3) focusing early Annie enrollment efforts on patients comfortable with technology and who do not need intensive follow up; 4) advertising the adaptability of Annie texting protocols; and 5) the value of Annie as a health coaching tool. In Phase 2 of Annie implementation we are collecting data on patient and provider experiences with Annie, including usability, clinical workflow fit, and clinical benefits such as improved medication adherence and fewer missed labs and visits.

Conclusions: Technologies develop rapidly and hold the allure of efficiently doing things that were once cumbersome or not possible to do in a busy clinical setting. The Annie automated text-messaging system may be able to help in this regard by offering support through self-management, coaching, and education outside of clinical encounters. Findings will inform iterative development of Annie and its national rollout.