Methodology

While the quality of a country’s health and education systems is determined by many factors, measuring what providers do, know and work with can help improve accountability to the households and families they intend to serve. This evidence also enables policymakers as well as teachers, principals, doctors, and hospital directors to identify key gaps in service provision and target action where it is most needed.

 

What are Service Delivery Indicators?

Service Delivery Indicators (SDI) are a set of key health and education statistics on provider ability, effort, and inputs that offer a snapshot of the experience of the average citizen accessing these services. Using standardized questionnaires and data collection procedures, usually between 200 and 2,000 facilities (schools or health facilities) are surveyed per country, generating results that are representative at national and sub-national levels. The surveys allow for analysis at sub-regional levels (e.g. counties, provinces, districts, etc.), rural and urban areas, public and private providers, and other relevant features (e.g. facility type). Indicator definitions and data collection protocols are standardized so that they can be internationally comparable. SDI are periodically updated to reflect technology and systemic changes while ensuring comparability within and across countries, and over time.

SDI Nuts & Bolts: A Brief Guide for Task Teams

 

What are SDI useful for?

Measurement and transparency are key elements of social accountability, which is essential to ensure efficient use of public resources. Without consistent and accurate evidence on the quality of services, it is difficult for stakeholders, including citizens and policymakers, to assess how service providers are performing, to work towards corrective action, and ultimately to bring about improvements. Moreover, benchmarking within and across countries also helps stakeholders to form a shared understanding of what are key constraints to the delivery of quality services.

By focusing on inputs, provider effort and knowledge, SDI links resources to results and helps shift the policy dialogue towards quality and results. For example, in Mozambique, a national campaign against teachers’ absence was instituted after the launch of the 2014 SDI Education Report. Likewise, in the Democratic Republic of the Congo, 2019 education SDI results are being leveraged in several ways to inform and strengthen the free primary education policy and the Bank’s operations and dialogue. Moreover, SDI build the capacity of local organizations in research and policy analysis. The SDI model is to contract, train and empower local organizations to implement the surveys and conduct the analysis, while also offering data analysis training workshops to researchers, medical statisticians, policy analysts, and civil society activists.

 

 

Beyond capacity generation, the SDI initiative provides relevant inputs for research. So far, SDIs have been featured in about 45 publications, among which more than 15 peer-reviewed academic articles published in leading economics, education, and health journals.

 

Are you interested in learning more about the SDI Initiative or conducting an SDI Survey in your country? Please access our revamped FAQ section or contact us at sdi@worldbank.org

 

Definition of core indicators

Below, you can find brief definitions of core indicators collected for all health and education SDI surveys.

 

School absence rate

Share of a maximum of 10 randomly selected teachers absent from the school during an unannounced visit.

During the first announced visit, a maximum of ten teachers are randomly selected from the list of all teachers who are on the school roster. The whereabouts of these ten teachers are then verified in the second, unannounced, visit. Teachers found anywhere on the school premises are marked as present.

Classroom absence rate

Share of teachers who are absent from the classroom during scheduled teaching hours as observed during an unannounced visit.

The indicator is measured as the share of teachers not in the classroom at the time of an unannounced visit. The indicator is constructed in the same way as school absence rate indicator, with the exception that the numerator now is the number of teachers who are either absent from school, or present at school but absent from the classroom.

Time spent teaching per day

Amount of time a teacher spends teaching during a school day in minutes.

This indicator reflects the typical time that teachers spends teaching on an average day in minutes. It combines data from the staff roster module (used to measure absence rate), the classroom observation module, and reported teaching hours. The teaching time is adjusted for the time teachers are absent from the classroom, on average, and for the time the teacher teaches while in classrooms based on classroom observations. While inside the classroom distinction is made between teaching and non-teaching activities. Teaching is defined very broadly, including actively interacting with students, correcting or grading students’ work, asking questions, testing, using the blackboard, or having students working on a specific task, drilling or memorization. Non-teaching activities includes working on private matters, maintaining discipline in class, or doing nothing, and thus leaving students not paying attention.

Minimum knowledge among teachers

Share of teachers with the minimum knowledge.

This indicator measures teacher's knowledge and is based on mathematics and language tests covering the primary curriculum administered at the school level to all mathematics or language teachers that taught grade three in the previous year or grade four in the year the survey was conducted. It is calculated as the percentage of teachers who score more than 80 percent on the language and mathematics portion of the test. The indicator is representative of the average teacher in the universe of teachers in a given country rather than the average teacher at the average school.

Teacher subject knowledge test score: This indicator measures teacher’s knowledge and it is calculated as the mean proportion of correct answers of a mathematics, language, and pedagogy tests covering the primary curriculum administered at the school level to all mathematics and language teachers that taught grade three in the previous year or grade four in the year the survey was conducted.

Infrastructure availability

Unweighted average of the proportion of schools with the following available: functioning electricity and sanitation.

Minimum infrastructure resources is a binary variable capturing availability of: (i) functioning toilets operationalized as being clean, private, and accessible; and (ii) has working electricity connection and sufficient light to read the blackboard from the back of the classroom.

Functioning toilets: Whether the toilets were functioning was verified by the enumerators as being accessible, clean, and private (enclosed and with gender separation).

Electricity: Functional availability of electricity is assessed by checking whether the electricity connection in the classroom works. The enumerator also places a printout on the board and checks (assisted by a mobile light meter) whether it was possible to read the printout from the back of the classroom.

Equipment availability

Unweighted average of the proportion of schools in which there is a functioning blackboard that is visible and has chalk; and a share of students having a pen or pencil, and a notebook to write on.

Equipment availability is a binary variable equal to one if (i) the randomly selected grade four classroom has a functioning blackboard and chalk, (ii) the share of pupils with pens is equal to or above 90 percent, and (iii) the share of pupils with notebooks in that classroom is equal to or above 90 percent.

Functioning blackboard and chalk: The enumerator assesses if there was a functioning blackboard in the classroom, measured as whether a text written on the blackboard has sufficient contrast to be read at the back of the classroom, and whether there was chalk available to write on the blackboard.

Pencils/pens and notebooks: The enumerator counts the number of pupils with pencils or pens and notebooks, respectively. By dividing each count by the number of pupils in the classroom, one can then estimate the share of pupils with pencils or pens and the share of pupils with notebooks.

Share of pupils with textbooks

Number of mathematics or language books used in a grade four classroom divided by the number of pupils present in the classroom.

The indicator measures, in one randomly selected grade four class, the number of pupils with the relevant textbooks (mathematic or language conditional on which randomly selected class is observed), and divided by the number of pupils in the classroom.

Observed pupil-teacher ratio

Number of grade four pupils per grade four teacher.

The indicator of teachers’ availability is measured as the number of pupils per teacher in one randomly selected grade four class at the school based on the classroom observation module.

Provider absenteeism

Share of a maximum of 10 randomly-selected providers absent from the facility during an unannounced visit.

Number of health professionals who are absent from the facility on an unannounced visit as a share of ten randomly sampled workers who should be on-duty. Health professionals doing outreach are counted as present.

Caseload per health provider

Number of outpatient visits per clinician per day.

Caseload is calculated as the number of outpatient visits recorded in outpatient records in the three months prior to the survey, divided by the number of days the facility was open during the three-month period and the number of health professionals who conduct patient consultations.

This indicator is adjusted for the average absenteeism at the facility-level. For example, if a facility reports having 10 healthcare providers who conduct outpatient consultations, but that facility’s absenteeism on an unannounced visit is found to be 40%, then the number of healthcare providers will be adjusted down by 40% and only 6 healthcare providers will be counted as available for patient care.

Diagnostic accuracy

Percent of correct diagnoses provided in the five clinical vignettes.

The SDI includes five core vignettes: (i) acute diarrhea w/ dehydration; (ii) pneumonia; (iii) diabetes mellitus; (iv) pulmonary tuberculosis; (v) malaria w/ anemia. Healthcare providers are scored on their ability to provide correct diagnosis on each of those vignettes and their overall score is calculated as the percent of vignettes answered correctly.

Treatment accuracy

Percent of correct treatments provided in the five clinical vignettes.

The SDI includes five core vignettes: (i) acute diarrhea w/ dehydration; (ii) pneumonia; (iii) diabetes mellitus; (iv) pulmonary tuberculosis; (v) malaria w/ anemia. Healthcare providers are scored on their ability to provide correct treatment on each of those vignettes and their overall score is calculated as the percent of vignettes answered correctly.

Management of maternal and neonatal complications

Number of relevant treatment actions proposed by the clinician.

The SDI includes two vignettes to assess maternal and neonatal complications. Providers are scored on the number of relevant treatment actions that they propose out of five specific actions for post-partum hemorrhage and seven specific actions for neonatal asphyxia. 

Medicine availability

Percent of 14 basic medicines which were available and in-stock at the time of the survey.

Medicine availability is calculated as the percent of 14 medicines available and in-stock at the time of the survey. The list of medicines included for the SDI is based on a subset of the WHO Essential Medicines list. The medicines included are:

  1. Amitriptyline (anti-depressant)
  2. Amoxicillin (antibiotic)
  3. Atenolol (beta blocker)
  4. Captopril (ACE inhibitor)
  5. Ceftriaxone (antibiotic)
  6. Ciprofloxacin (antibiotic)
  7. Cotrimoxazole (antibiotic)
  8. Diazepam (anti-seizure)
  9. Diclofenac (nonsteroidal anti-inflammatory)
  10. Glibenclamide (anti-diabetic)
  11. Omeprazole (proton pump inhibitor)
  12. Paracetamol (analgesic)
  13. Salbutamol (bronchodilator)
  14. Simvastatin (statin)

The list of medicines in the SDI is adapted based on country standards. Thus, some of these medicines were not included in the surveys in Kenya, Nigeria and Uganda, so these countries have been omitted from this indicator.  

Equipment availability

Availability and functioning thermometer, stethoscope, sphygmomanometer and weighing scale.

Equipment availability is calculated as the availability and functioning of a thermometer, a stethoscope, a sphygmomanometer and a weighing scale (adult, child or infant). Credit is given if all four components are available. 

Thermometer: Credit is given if a facility reports and the enumerator observes that the facility has one or more functioning thermometers (used for measuring patient body temperature).

Stethoscope: Credit is given if a facility reports and the enumerator observes that the facility has one or more functioning stethoscopes.

Sphygmomanometer: Credit is given if a facility reports and the enumerator observes that the facility has one or more functioning sphygmomanometers.

Weighing Scale: Credit is given if a facility reports and the enumerator observes that the facility has one or more functioning adult, child or infant weighing scale.

Infrastructure availability

Availability of an improved water source, an improved toilet and electricity.

Infrastructure availability is calculated as the availability of three components: improved water source, improved toilet and electricity. Credit is given if all three components are available.

Improved toilet: Credit is given if facility reports and enumerator confirms facility has one or more functioning flush toilets or ventilated improved pit (VIP) latrines, or covered pit latrine (with slab).

Improved water source: Credit is given if facility reports their main source of water is piped into the facility, piped onto facility grounds or comes from a public tap/standpipe, tubewell/borehole, a protected dug well, a protected spring, bottled water or a tanker truck. This definition is based on the WHO/UNICEF Joint Monitoring Program for Water Supply, Sanitation and Hygiene.

Electricity: Credit is given if facility reports using electric power grid, fuel-operated generator, battery-operated generator, or a solar powered system as their main source of electricity.