Improving Data Quality in HMIS

Employing a Systems-thinking Approach

Quicksand adopted a systems-thinking approach to comprehensively study all functions of the Health Management Information System (HMIS). In collaboration with Purple Audacity, we conducted extensive field research across six states to evaluate systemic practices across public health units, including sub-centres, PHCs, CHCs, and District Hospitals. This included analysing outreach data collected at the lowest level facilities collected community health workers (ASHAs and ANMs), as well as facility-wise data from PHCs, CHCs, and district hospitals, meticulously tracking the data collection, reporting, and entry processes involving multiple actors, including clinicians and administrators. Our field research involved meetings with healthcare professionals at outreach and facility levels, including clinicians and administrative staff. Multiple workshops and stakeholder consultations were conducted at different stages with the central HMIS team, the state HMIS officers, project partners, and experts from relevant fields of behaviour design, health care, and large-scale program implementation. We aimed to understand the data journey, assess interactions between data and stakeholders, identify potential errors, and examine the behaviours of all involved actors

Research revealed many disparities in the quality of data collection and compilation across different Indian states, and within facilities in the same state. There was a need to identify states and health facilities based on variables such as technology adoption, performance, procedural differences, and adherence to different kinds of HMIS processes. The research covered six states in total, representing a spectrum of data maturity with a state like Manipur which had just started its digitisation journey, to a state like Tamil Nadu that had a robust state-level data system of its own. Some of the key problems impacting data quality include errors made at the data calculation and aggregation level (estimation of data, poor understanding of indicators, over and under-reporting of data to meet targets), errors at the data entry point (misunderstanding of indicators, typing errors), inconsistent and poor supervision of data before and after entering of data into the HMIS portal. Data quality was also linked directly to the capacity of data workers, which often included non-clinical staff (contractually hired data entry operators) who did not have adequate knowledge or training on the wide variety of health indicators they had to input data on.

In this regard, the project objectives included exploring behavioural aspects of HMIS data reporting and entry issues; designing innovative behavioural and communication interventions; and developing low-fidelity prototypes.

Here are some key highlights from our research process and prototype tools:

Systems thinking lens: The initial problem statement pertained to the point of data entry, but our approach interrogated the flow of data from the central level to the frontline, from the role of HMIS functionaries to frontline workers. This helped us understand all levels of this system and the influence they had, either directly or indirectly, on data quality.

Early intervention design prototypes: Sacrificial concepts were introduced early to gain stakeholder buy-in, enabling multiple rounds of iteration in consultation with the HMIS central team and experts.

Data Journey maps: Detailed journey maps created after initial rounds of research effectively communicated the journey of HMIS data, synthesising research findings and stakeholder responses.

Contextual behavioural design models: We adapted the Manoff Group’s Toolkit for Behavior Integration to analyse behaviour patterns affecting data quality, considering various influencing factors.

Multiple prototypes for a holistic solution

Amongst many findings, we synthesised three key insights that formed the foundation for proposed behavioural and communications interventions that can improve HMIS experience in data reporting and entry for health workers. This led to the development of three intervention routes; building awareness around how data and health outcomes are integrally connected, training for data entry workers, and empowering supervisors with data so that more continuous measures could be taken to improve data quality at the front line level.

Prototype 1: HMIS ‘Mitra’ whatsApp chatbot: A low-fidelity chatbot was designed for HMIS supervisors at public health facilities at block and district levels. This two-way communication tool via WhatsApp provides regular updates, visualised data, stories, reminders, and information - essentially putting the data

Prototype 2: ‘HMIS Saves Lives’ campaign: A brand campaign was developed to build an identity and highlight the critical role of HMIS within the health system, reinforcing the power of data to save lives and complement the clinical work that health workers often prioritise over data tasks. The campaign includes multiple artefacts targeting various health system actors to create a unified HMIS identity.

Prototype 3: Mobile-friendly micro-training for data entry workers: Data entry workers, the first point of digitisation for HMIS data, were identified as crucial for checking and reviewing errors before data was uploaded. A micro-training program was recommended to help these workers understand the connection between HMIS data, patient well-being, and health outcomes. This was also suggested as a way to continuously train data entry staff that saw high levels of attrition.

Challenges and Learnings

In its endeavour to leverage technology to improve health systems, this project highlighted the importance of considering behavioural aspects and systemic barriers that affect technology adoption amongst last-mile users. Among the many successes of the project, these were a few to note:

High-level Approval and Recognition: The project’s solutions and supporting evidence were presented to the Secretary of the Ministry of Health and Family Welfare, and received approval for implementation.

Validation of Behavioural Insights: The project validated the need to consider behaviour change in a multi-layered manner, addressing individual motivations, peer support and systemic factors. It emphasised the necessity of creating systems that enable and incentivise the right behaviours. For instance, we found that health workers often deprioritise data entry due to a lack of incentives and support. The system needed to provide adequate training and build capabilities for non-medical actors responsible for data entry as well.

Designing Two-way Data Flows: The project also highlights the importance of designing systems that facilitate a two-way data flow, empowering grassroots-level decision-making and improving overall system efficiency.  This meant recognising the need for digital systems to not just collect data, but also to provide actionable insights at the grassroots level, making data-related activities a valuable task rather than a chore.

Beyond the design process, however, the project also revealed certain implementation challenges in a public health context. Among such challenges were technical and human resource constraints. The ability of governments to sustain advanced technological solutions is limited on account of inadequate resources - both for human resources as well as setting up and managing technical infrastructure. Significant investments are required, not just for the initial development of applications, but for long-term sustainability, including the need for technical teams with diverse skills (e.g., product managers, data analysts, UX designers). Another notable constraint to implementation was the variation in technological maturity across states as highlighted earlier, and therefore a tailored approach that provided the right level of support basis the existing capabilities in the state.

To aid the government in understanding how the technology solutions being proposed would evolve over a period of time, and the accompanying investments the Government would need to make, Quicksand developed a roadmap for one of the recommended systems, a chatbot, showing its potential evolution from a basic querying system to a sophisticated AI-powered tool. A detailed technical blueprint and project report were submitted to the government, outlining necessary investments and potential benefits.