Using Predictive Analytics to Prevent Missed Opportunities and Achieve Higher Immunization Coverage and Timeliness

Abstract

Background: Despite the availability of free routine immunizations in low- and middle-income countries (LMICs), many children are incompletely vaccinated, vaccinated late for age, or drop out over the course of the immunization schedule. According to WHO & UNICEF estimates, average BCG coverage in LMICs is 84% (range: 39-99%) while only 63% (range: 8-97%) children receive the last dose of measles. Without the technology to model and visualize risk on large datasets, vaccinators and policy-makers are unable to identify and target groups and individuals at high risk of dropping out. Thus, default rates remain high, preventing full universal immunization coverage. Predictive analytics algorithm leverages Artificial Intelligence (AI), and uses statistical modeling, machine learning, and multi-dimensional data mining to accurately identify children who are most likely to delay or miss their follow-up immunization visits.

Objective: We conducted feasibility testing of a predictive analytics algorithm to identify which children are likely to delay or miss the follow-up vaccines through risk profiling of clients into high-, medium- and low-risk groups. We will validate the functionality and accuracy of the algorithm though verifying what percentage of clients classified as “high-risk” actually delay or miss the follow-up immunization visit.

Methods: The algorithm was developed using existing immunization data from across Pakistan. There were 44,493 immunization records, collected from 21 immunization centers in over seven cities, used to train the system, which was then successfully piloted on 8,898 longitudinal patient records. After development, the predictive analytics module was incorporated in our existing digital immunization registry (DIR) and is currently deployed in 5 immunization centers in Karachi, Pakistan. Once the child is enrolled in the DIR, his immunization information is captured along with the salient socioeconomic and demographic characteristics. The predictive analytic technology uses data mining, statistical modeling, and pattern identification techniques to accurately forecast future immunization outcomes based on existing immunization and demographic data which are correlated to the child’s likelihood to miss or not show up on time for a vaccination visit.

Results: At the time of development, the predictive power of the system was validated through training the system on 44,493 records collected from 21 immunization centers, and was then piloted on 8,898 records. Out of a total of 8,898, the Recursive Partition Model predicted that 3,352 (38%) children would default out of which 2,500 children did default (75%). Similarly, it predicted that 5,546 (62%) children would return for next vaccine, out of which 3,751 (67%) children did return. This indicates a 70% overall accuracy rate. Over time, through artificial intelligence, as more data is captured, the system will continue to self-learn from accumulated records, recognizing influential variables, self-selecting statistical models, and continually upgrading itself to achieve highest predictive accuracy. Results from feasibility testing in 5 immunization centers are currently pending.

Conclusions: Predictive analytics is an unconventional approach to improving the timeliness of routine immunization and reducing missed opportunities; results of the pilot will provide strong evidence of the potential of predictive analytics to revolutionize immunization service delivery.

Keywords