Data Quality Assessment - A Real Example of Data Quality Assessment in Suchna Project

                                                                                                                                              

 

Executive summery

Data quality assurance is a collective term for the procedures that are used to maintain the integrity of data that is housed within various databases. Often, the process of maintaining data quality requires such tasks as removing obsolete information, cross-referencing relevant information found in different databases, and in general making sure there are no inconsistencies with the information found within a database or a set of databases. This type of data cleansing is an ongoing process that is considered a key element of efficient data administration.

Suchana is a large-scale program to prevent malnutrition in children in Sylhet division, Bangladesh by improving the livelihoods and nutrition knowledge of poor and very poor households. Currently Suchana is being implemented in 70 unions the smallest administrative unit of government, under 20 upazila of two districts of Sylhet division. Suchana is supporting 99482 beneficiaries in phase 3 and 4.

To perform this activity this program, collect and gather huge number of beneficiary details information, activity related information, program related information etc. Those data are collected both online and offline platform. Basically, most of those data are collected by frontliner and it is compiled by union level, upazila level, district, IP-TP level finally program level. As a result, there is possibility to occur error in data which is finally stored. That’s why program management took decision to check this data quality in every four-month interval and identify the gaps. Finally suggest some recommendations to recover those gaps.

It was the first DQA in 2022 which was held in 15-16 February 2022. Here participate program and MEAL personnel of all partners of suchana. There are three group in this mission. They visit three upazila of three implementing IP. They choose only six indicators to conduct this DQA. The team members try to look after the total data source and collection process. They found some finding on VSLA register, demo pond and SAM/MAM identification issue. Besides this they observe some issues on management level regarding data checking and sharing with the team. So that DQA team suggest some recommendations and prepare an action plan to recover this issue. Cordial assistance of all group members this DQA mission is successfully completed

Table of Content

1.    Introduction

2.    Purpose/Objective

3.    Methodology/Process

3.01   Team formation:

3.02   Area Selection:

3.03   Visit Planning:

3.04   Indicator Selection:

3.05   Reporting:

4.    Indicator Wise Performance Score

5.    Findings & Discussion

6.    Challenges in Data Quality Assurance

7.    Recommendations

8.    Action plan (or issue log)

9.    Annexures

1.    Introduction

Generally, the standard of data should be of sufficiently high quality to support management needs for planning and decision making. In fact, high-quality data are the cornerstone for evidence-based decision-making. Through the DQA process, one can determine the status of the data quality standard of the data being reported. As a part of regular data quality check, finding out the improvement areas and working accordingly to meet the gaps are the ultimate objectives of DQA which helps to establish accountability and result-based program monitoring. The five quality standards that have been considered for the quantitative and qualitative part of data are Accuracy, Reliability, Timeliness, Integrity, and Completeness.

 

‘Suchana’ program is a multi-sectoral nutrition intervention led by Save the Children in collaboration with WorldFish, HKI, IDE, icddr,b, CNRS, FIVDB and RDRS as consortium partners. This program is generating huge numbers of data from different levels on the number of different indicators. So, it is very important to maintain the standard quality of Data at the entire process and levels. Accordingly, Data Quality Assessment (DQA) is one of the major components of Suchana MEAL framework. Since 2018, the Suchana MEAL team has been conducting data quality assessments jointly across the consortium members. In February 2022, the seventh round of DQA events was conducted to improve the quality of data by enhancing data management, analysis, and reporting towards strengthening the overall management process of Suchana.

2.    Purpose/Objective

There are three objectives to conduct the DQA are given in the below–

 

ü  To check the quality of reported data at field level

ü  To check the data collection process practiced in the field

ü  Ensuring the data sharing and data use regarding program implementing decision making process.  

ü  Identifying the challenges and find possible solution to ensuring data quality in the program.

 

3.    Methodology/Process

3.1.        Team formation:

Team members are taken from all implementing partners. Suchana SCI MEAL team take initiatives to form the groups through email communication. Finally, in DQA Mission there three group leading by one TL and DTL. Every group consist with 5-7 members from SCI, IP and TP. The team members are in the below table-

Table1: Team member list

Team Members
Team -1	Team -2	Team -2
Mr. Ashoke K. Sarker Project Manager-WF Ms. Fatema Kaniz Manager-Nutrition Nayeem Al Mifthah, Senior Manager-MIS Md. Helal Uddin, Senior Technical Specialist-iDE Md. Md. Abdur Razzaq APC-CNRS Md. Khan Ashfaque Director, MEAL (Observer) Md. Mahabub Hassan, Senior Manager-Livelihoods	Mr. Biswojit Kumar Roy Senior Manager-CR & CB (SCI) Mr. Mohosin Bhuyan  Officer-MEAL (SCI) Mr. Gopal Datta, District Fisheries Specialist (WF) Sk. Golam Moula Technical Officer (HKI) and Mr. Dayem Al Rahman Chowdhury APC(FIVDB)	Mohammad Al Amin Senior Manager Consortium Abdul Baten (DTL) M&E Specialist, Suchana, WorldFish Al-Amin Shovan Senior Manager – MEAL & Research - SCI Hannan Ali Technical Coordinator,IDE Md. Hasanuzzaman Abu Raihan  Sharmin Sazia Manager – Knowledge and Learning (SCI)

3.1.        Area Selection:

 In suchana program DQA is conducted in every four-months interval. Out of 20 upazila 03 upazila were selected for DQA. It is selected randomly based on 3 IPs working area. One upazila from every IPs upazila.  Team-1 visit Moulavibazar sadar upazila of CNRS, Team -2 Visit Jaintapur upazila of FIVDB, Team-3 visit Golapgonj Upazila of RDRS.

Table 2: shows the selected location for DQA conduct

Upazila	IP	District
Golapgonj	RDES	Sylhet
Jaintapur	FIVDB	Sylhet
Moulavibazar Sadar	CNRS	Moulavibazar

 

3.2.        Visit Planning:

Visit plan started through email communication. Senior Manager MEAL lead the total planning, distribution of members and indicator selection. Through email communication the date is fixed on March 15-16, 2022. A planning meeting was arranged at 15^th march. There a meaning full discussion held. After that three group started their journey to filed. Every team sit with the upazila team for informing and field tour planning. After that they started to conduct DQA at Union and community level.

3.3.        Indicator Selection:

A planning meeting was held before DQA conduct at field level. In that meeting SCI MEAL team give a short briefing on DQA process and share the preselected indicators and assessment tools. After that a participatory discussion held among the all-team members selecting the indicators. After those following indicators are selected

1)    Amount of loan distribution under VSLA

2)    Number of Session and participants of PSPM

3)    Number of vaccination campaigns organized at community level (mass vaccination) to reduce mortality rate of poultry birds

4)    Number of IGA- Aquaculture business case developed (received training)

5)    Number of BHHs visited to Demo Pond farmers to learn improved fish culture practices throughout the year

6)    Number of SAM and MAM case identified

3.4.        Reporting:

Every group team leader and deputy team leader were responsible for reporting. They collect the assessment report of different indicators and compile it. After that it is handover to SCI MEAL team.

4.     Indicator Wise Performance Score

The team started the DQA process with a short briefing session at Suchana 3 Upazila office with available colleagues. Then DQA team members were divided into small groups to conduct desk review for intervention-wise data checks at the office with MEKMO, UC, Nutrition officer and GCDO.  The team conducted field-level data and evidence checks on 16th February at different Union along with MEKMO, NO, UC and SCM. After getting back from the field visit, a debriefing session was also conducted at Upazila office and completed the DQA process at upazila level. After observing the total process team members give a score on every upazila separately. The below table it shows the average score of three upazila.

 

Table 3: Average Performance score for data quality standards

Indicators	Performance score for data quality standards
	Accuracy	Reliability	Timeliness	Integrity	Completeness	Conclusion	Total
Indicator 1: VSLA load distribution (amount of tk. and # of participant)	29	15	10	13	13	4	83
Indicator 2: Vaccination campaign	34	16	10	12	14	4	93
Indicator 3: No. of IGA- Aquaculture business case (received training)	34	18	10	15	15	5	96
Indicator 4: No. of HHs visited to Demo Pond operators	34	17	10	13	14	5	91
Indicator 5: PSPM (# of session and # of participants attended)	34	18	10	14	15	5	95
Indicator 6: # of SAM and MAM case identified	34	18	10	15	15	5	95

 5.     Findings/ Important Observation

During the field visit DQA team find some important observations. Those are given in the below table

 Table 4: show the field findings during DQ                

Important observations / Improvement areas to be considered
Incomplete digitization, hard copy reporting, confusion on who is responsible for checking data accuracy.
It found, very often data are checked and analyzed by respective Upazila staff, so GCDO, MEKMO must check and analyze data regular basis at Upazila and Union level.
It was a one-time event and completed as per their plan. They had their internal verification process to check and follow-ups the indicators.
They are maintaining the indicator through recoding at pond register provided by Suchana. The team members have adequate skills and experience and understanding to maintain it. They are recording the numbers through recall method when they are visiting and checking the pond registers.
MEKMO only did the data verification where UC and UzC also need to verify the data on regular basis thus will reduce potential risk on data error
Team needs to discuss the linkage between SAM/MAM identification, treatment, and stunting reduction
Further attention is required on Group level record keeping.
System should update according to received services.

 

6.     Challenges in Data Quality Assurance/Assessment

During the field visit DQA team discussed with the upazila and union level staffs regarding faced challenges ensuring the data quality at field level. Besides this team member observe some issues at field. Following challenges are identified at field level.

·         MEKMO is only person who is verifying the data quality whereas other team members such as UC, UzC did not check the quality of the data this is a challenge to generate high quality data.

·         They are entering and maiming the data as per their progress and schedules. But they aren’t preserving the master rolls and other relevant hardcopies at the local offices similar as many other upazilas. Usually, these types of documents are transferred to their central office as per the organization policy.  DQA team don’t have scope to review or verify the reported numbers with master rolls and other relevant hard copies.

·         Hard copy data is being maintained at the field level by FF/SCM and at the upazila office since digital input system is not fully functional.

·         VSLA group registers have been changed in the last couple of years that can be a source of confusion for VSLA members. IP team members have to a handwritten locally made additional register fulfill the gaps. So, it should be updated based on local requirement and proper documentation. There is possibility of transfer errors since inputs are managed manually; ultimately, it was not possible to track how many VSLA members took out loan in which amounts, only the aggregated amount for the month is available.

·         Numbers must be checked at field level for verification. At upazila level, individual loan information (member ID, amount) wasn’t available except the aggregated amount for the group per month. Team is not sure who is supposed to verify data.

·         There are no written procedures/instructions in place for data collection, cleaning, analysis, and reporting. Hard copies of the data are susceptible to unauthorized changes.

·         There is no centrally prescribed format for data transfer from field level to upazila level. Upazila team was found delivering their own reporting format to the field level staffs which occurs little delay on data verification. 

·         Data safeguard is not established in well manner due to huge, diversified activities and data are coming from field without proper physical regular verification mechanism.

·         Ensure the proper use of Tab (Suchana Apps) where field staff are mostly dependent on manual register for reporting

·         To ensure the data quality because regular data verification system is not available.

7.     Recommendations:

To addressing the above challenges All team members, suggest some recommendations. The recommendations are collected team wise. Finally, all teams’ recommendations are compiled. Suggested recommendations are given in the below-

·         Respective UC and UzC should check the data in their monthly coordination meeting this will give opportunity to MEKMO and other related staffs to verify the data quality. A centrally prescribed format for data transfer from field level to upazila level is needed to ensure the flawless data transfer.

·         Adding more information at the upazila level on individual loans from VSLA can be tracked the frequency of loan and actual amount. VSLA and Demo Pond related data should be analyzed in a periodic basis to find out the reasons of low quality VSLA and demo pond.

·         If possible, preserve one meeting documents in the meeting place.

·          Need more follow-up and on job training regarding data management effectively and efficiently.

·          Data entry into system by respective field staff need to be regularized.

·          Before reporting any data, a strong regular basis verification mechanism is required.

·          Should be shared data findings regular in monthly meeting of IP and TP.

·          Field staff should report based on Suchana MIS data.

8.     Action plan (or issue log)

To address the above issue DQA team suggest for following action plan.

Table 5: Action plan for addressing the findin

9.     Annexures:

The detailed filled-in check lists on different selective indicators are as follows:

Table 5: shows the indicator checklist

Name of indicators	Team 1 Checklist	Team -2 Checklist	Team -3 Checklist
Indicator: Vaccination campaign	N/A
Indicator:  No. of SAM and MAM case identified	N/A
Indicator:  PSPM (# of session and # of participants attended)
Indicator- No. of HHs visited to Demo Pond operators	N/A
Indicator- No. of IGA- Aquaculture business case (received training)	N/A
Indicator- VSLA load distribution (amount of tk. and # of participant)

 To get the full report Click Here