What Share of Students Don't Show up In Learning Outcomes Surveys in India
Learning outcomes surveys typically use a school-based sampling strategy – i.e. they randomly select schools and then conduct assessments of all or some portion of students at each selected school. The advantage of school-based surveys is that they are far cheaper to conduct than household surveys. The disadvantage of school-based survey is that they potentially exclude students who aren’t enrolled, don’t show up on the day of the exam, or attend “unrecognised” private schools (i.e. private schools which don’t have official recognition and thus are not in UDISE, the official government list of schools),
I took a stab at estimating the total share of primary school-age children excluded from school-based sampling surveys. I used data on enrollment for rural 7-10 year olds from ASER, data on enrollment of primary and upper primary children at an unrecognised school from NSSO’s 71st round, and data on attendance of enrolled 7-10 year olds from NFHS 2015-16. The NFHS question asks whether the child ever attended school in the 2015-16 so these figures estimate the share of children that almost certainly wouldn’t be included in a school-based survey. In addition, a child may be excluded because they don’t show up on the day of the exam. Kremer and Muralidharan, in a national survey of rural schools conducted in 2004, found that student attendance was 75% in private schools and 64% in government schools.
The table below gives these figures for the large states in India (excluding Telangana) and for the entire country. Overall, the share of students who would completely excluded from a school-based survey is relatively low at 10%. This surprised me somewhat. In fact, my original motivation for gathering these stats was that I thought they would show that a school-based survey would exclude far more children and thus a household survey would be a better bet. Previous estimates of the share of students attending unrecognised schools have varied between around 46% and 80%. (See footnote 6 in this paper by Kingdon for a review.) And estimates of the share of enrolled students attending school from previous versions of the NFHS were much lower. While the overall share of excluded students is relatively low, the share varies quite a bit state to state from just over 1% in HP and Sikkim to nearly 20% in Bihar.
The upshot for me is that school-based surveys can do a decent job generating representative estimates of school-age children but, for states like Bihar and UP, it will be important to generate estimates of learning levels for children left out of the sampling frame. And, of course, it is also important to make sure that children actually show up on the day of the assessment.
