ERIC Number: ED663607
Record Type: Non-Journal
Publication Date: 2024-Sep-18
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
A Study of Classroom Rostering in North Carolina: Descriptive Results and the Effects of Counterfactual Policies
Peter F. Halpin; Matthew G. Springer
Society for Research on Educational Effectiveness
Background: Research has consistently shown that disadvantaged students have inequitable access to high-quality teaching (e.g., Isenberg et al., 2013; Goldhaber et al. 2018; Hanselman, 2019). This non-random sorting of students to teachers has been documented at multiple organizational levels, including districts, schools and classrooms (e.g., Goldhaber et al., 2015; Clotfelter et al., 2023). Studies that have compared sorting across the different levels have shown that classroom-level inequalities tend to be the smallest in magnitude. However, classroom-level sorting remains a potentially high-impact target for policy intervention for several reasons. First, the mechanism through which classroom-level sorting occurs (i.e., classroom rostering) is under the direct purview of school staff and therefore highly malleable. Second, schools already have the resources required to implement classroom rostering, so changes to rostering policies have the potential to be cost-neutral or even cost-efficient. Third, because every student is rostered each year, rostering policies represent a "high volume, low margin" approach to producing net gains in equitable access and student learning. Purpose and Research Questions: The present study is a secondary analysis of administrative data obtained from the North Carolina Department of Public Instruction (NC DPI). Using this data source, we seek to more fully understand the potential of classroom rostering to reduce inequitable access to high-quality teaching and improve student learning. We approach this overall goal in two main steps. In the first step, we descriptively analyze the assignment of students to teachers within school- grade-year cells. Here we are interested to better understand the student characteristics that are associated with differential access to high-quality teaching. We categorize teaching quality using the tertiles of the distribution of teachers' value-added (VA) in the previous school year, and treat teachers without prior VA as a fourth category. We address the following descriptive questions: 1a) What proportion of school-grade-year cells have sufficient variability in effective teaching for rostering to be a potential mechanism for inequitable access to high-quality teaching? 1b) Which, if any, student characteristics are associated with differential assignment to teachers of different VA categories? Step 2 of the study builds on Step 1 by using the results of Research Question 1b to identify two contrasting subgroups of schools, one group with relatively equitable rostering practices and the other with relatively inequitable rostering practices. In these two contrasting cases, we address two additional questions about the effects of each of three counterfactual rostering policies. The three policies are: Counterfactual Policy 1: Balanced classrooms (no tracking). Counterfactual Policy 2: Assign stronger teachers to students with lower prior achievement. Counterfactual Policy 3: Assign students to teachers to maximize net predicted learning gains while maintaining balance. Clotfelter et al. (2023) recently suggested the first two policies and the third is novel to this study. We study the effects of each policy on two outcomes of interest: 2a) access to high-quality teaching and 2b) student learning. Data Collection and Analysis. We conduct a secondary analysis of administrative data from NC DPI over the 2011-2012 to 2018-2019 school years. We focus on ELA and Math instruction for students in 4th and 5th grades. The panel includes data on 1.2 million students and 57,854 teachers across 1730 schools from all 110 districts in NC. Data cleaning was completed in early Spring 2024, with a working paper expected to be completed in Summer 2024. Step 1. To address Research Question 1a we tabulate the number of teachers in each VA category within school-grade-year cells. To address Research Question 1b we use Poisson regression with fixed-effects for school-grade-year cells (see Springer et al., 2022). We consider a standard fixed effects specification and a specification that additionally incorporates random slopes on coefficients describing differential assignment. This is an extension of Mundlak's (1978) specification for panel data that allows us to study how rostering practices differ over school-grade-year cells, while still focusing on sources of variation that arise within cells. Step 2. We use the predicted random slopes from Research Question 1b to create two groups of schools, those in which there is strong evidence of highly equitable rostering, and those in which there is strong evidence of highly inequitable rostering. The goal is to identify 50-100 schools that unambiguously fall into each group and contrast results across groups. The three counterfactual rostering policies can be implemented by solving standard mixed- integer programs (MIPs) within each school-grade-year cell. Details of the objectives and constraints of these MIPs are currently under investigation and will be reported. The MIP solutions are a set of binary decision variables indicating whether each student is assigned to each teacher -- i.e., a set of counterfactual rosters. Having the counterfactual rosters in hand, Research Question 2a can be addressed by applying the same Poisson regression model described for Research Question 1b, but this time using the counterfactual rosters. It is important to note that Research Question 2b and Counterfactual Policy 3 both require specifying a predictive model for student learning that incorporates complementarity between students and teachers. In the present context, this predictive model is referred to as a match- output function (Graham, 2011). We have previously researched match-output functions for classroom rostering (Halpin et al. 2022) and arrived at a specification in which teachers' leave- year-out VA is estimated separately for each tertile of the within-school distribution of student prior achievement. This is a special case of Delgado's (2022) approach to teacher comparative advantage. We discuss the strengths and limitations of this specification, the latter mainly having to do with imprecise estimation of VA within prior achievement tertiles. We also discuss the conditions under which the match-output function is causally identified (Graham et al., 2016). Results are summarized in terms of the average re-allocation effects on student learning for each of the three counterfactual policies, computed within each school-grade-year cell. Research Question 2b will be addressed by comparing the distribution re-allocation effects in the two contrasting subgroups of schools.
Descriptors: Elementary Education, Disadvantaged, Access to Education, Equal Education, Student Placement, Student Characteristics, Teacher Effectiveness
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Identifiers - Location: North Carolina
Grant or Contract Numbers: N/A
Author Affiliations: N/A