Development and Usability Testing of a Chatbot to Promote Mental Health Services Use Among Individuals With Eating Disorders Following Screening

Abstract

Background: Eating disorders (EDs) are complex mental illnesses with debilitating, pervasive psychological and physiological consequences when left untreated. Unfortunately, patients may face barriers to receiving treatment, such as stereotypes surrounding EDs, denial of illness severity, lack of motivation for treatment, and lack of knowledge about treatment resources. Barriers such as these result in a large treatment gap: only 20% of those with EDs will ever receive treatment. Digital tools like chatbots show potential to disseminate mental health–related interventions to large populations while offering a user-friendly, cost-effective, accessible, and anonymous means of tackling patient concerns.

Objective: This study developed and evaluated the usability of a chatbot designed for pairing with online ED screening. The tool aimed to promote mental health service utilization by improving motivation for treatment and self-efficacy among individuals with EDs.

Methods: A chatbot prototype, Alex, was designed using decision trees and theoretically informed components: psychoeducation, motivational interviewing, personalized recommendations, and repeated administration. Usability testing was conducted over 4 iterative cycles, with user feedback informing refinements to the next iteration. Postintervention, participants (N=21) completed the System Usability Scale (SUS), the Usefulness, Satisfaction, and Ease of Use Questionnaire (USE), and a semistructured interview. This process aimed to create an optimized chatbot by the final cycle for use in a randomized trial.

Results: Interview feedback detailed chatbot aspects participants enjoyed and aspects necessitating improvement. Feedback converged on four themes: user experience, chatbot qualities, chatbot content, and ease of use. Following refinements, users described Alex as humanlike, supportive, and encouraging. Content was perceived as novel and personally relevant. USE scores across domains were generally above average (~5 out of 7), and SUS scores indicated “good” to “excellent” usability across cycles, with the final iteration receiving the highest average SUS score.

Conclusions: Overall, participants responded well in interactions with Alex, including the initial version. Refinements between cycles further improved user experiences. This study provides preliminary evidence of the feasibility and acceptance of a chatbot designed to promote motivation for and use of services among individuals with EDs. Alex is the first chatbot designed for pairing with an ED or other mental health–related online screen, with the goal of ultimately increasing service utilization.

Acknowledgments: This research was supported by K08 MH120341 from the National Institute of Mental Health.

Availability of Data, Materials, and Code: The data will be made available by reasonable request to the corresponding author.

Authors’ Contributions: EEFC conceptualized and designed the study. OL and BD conducted the investigation process. BD and JS assisted with data curation and conducted formal thematic analyses. JS conducted formal statistical analyses. JS wrote the original manuscript, with contribution from BD. EEFC, CBT, DEW, and SSS designed the data collection instruments, and coordinated and supervised data collection, in addition to reviewing and editing the manuscript with LS, LMF, LAF, and LD.

Conflicts of Interest: None declared.

Keywords