ERIC Number: ED659364
Record Type: Non-Journal
Publication Date: 2023-Sep-30
Pages: N/A
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Embedding an Equity Orientation in Pursuit of Evidence of Educational Improvement
Megan Brunner; Adam Smith; Michelle Tiu
Society for Research on Educational Effectiveness
Background: Evidence of educational improvement is essential for ensuring that we are providing students with meaningful and optimal learning experiences. But what counts as equitable evidence of educational improvement? Our presentation will discuss one organization's attempt to create an environment in which evidence is contextualized in ways that align with commitments to equity. Educational improvement is a complex construct that must be attended to in holistic ways. Such evidence must consider all of the ways a learning approach is at play in sociocultural interactions between students, teachers, and their environments, including institutional and cultural contexts (Cohen, et al., 2003; Kyriakides, et al., 2023). Impact is often measured as outcomes on standardized tests. However, the full impact of a learning approach cannot be captured in isolated data on outcomes or treated solely as cause and effect (Coburn & Turner, 2011; Firestone & Gonzalez, 2007). In alignment with an equity orientation in educational research, studies which aim to understand a learning approach's promise for impact should consider multiple perspectives and data sources. Quantitative and qualitative data should be combined - and well contextualized - to understand complex narratives and unpack nuances of teaching and learning. Further, educator and student voices should be leveraged when considering what evidence of impact is important. Finally, research in partnership with educators and students requires flexibility and agency to ensure studies are designed and implemented in ways that align with their partners' needs. Context and Objectives: We are an organization that works to actively manage teams of researchers, developers, and educators who are engaged in Inclusive Research and Development (R&D). Our model of Inclusive R&D (Angevine, et al., 2019; EF+Math Program, 2023) intentionally centers educators and students as critical partners in the design and study of learning approaches. Our community of R&D teams is working to explore the hypothesis that executive functioning, developed within the context of mathematics learning and combined with an equity-oriented approach to teaching and learning, can narrow opportunity gaps in mathematics outcomes for Black and Latinx students and students of all races experiencing poverty (EF+Math Program, 2019). The R&D teams are iteratively designing and researching learning approaches across grades 3-8 to understand the potential for supporting students' mathematics learning. In the current phase of our work, teams are engaging in preliminary impact studies while continuing to iterate upon their current approaches. As with any R&D work that is exploring a new hypothesis, there is a clear need for evidence of educational improvement to justify continued exploration of these learning approaches. We posit that what comprises that evidence must be responsive to the current state of education research and center the voices of educators and students who are being impacted by the learning approaches themselves. As an Inclusive R&D organization, we are positioned to take a stance on what counts as evidence of educational improvement in alignment with our equity orientation. In this session, we will provide insight to our organizational learnings about conducting impact evaluations of iteratively designed learning approaches. Specifically, we will discuss how our organization's approach promotes equity through flexible, yet explicit goal setting and an inclusive approach to data. Findings: First, our Inclusive R&D organization moves together towards shared goals, even though different approaches or processes may be unique in how these goals are achieved. The organization's goals are set in a way that supports contextualized progress; we are committed to providing guidance to R&D teams and partners that both hold an expectation of rigor and provide areas for adaptation within contexts. For example, pilot study guidance included expectations that data for preliminary studies should be collected, analyzed, and interpreted using methodologies and instruments that are culturally informed and developed with community input. This guidance is explicit in terms of expectations, but flexible for teams' enactment. As teams have had to adapt to the constraints and capacities of their partner districts and educators, their study designs and implementation features have shifted to serve their partners' needs. Teams have found ways to gather data regarding student and educator experiences that create opportunities for comparisons between implementations, while minimizing burden on participants. Educator and student data is leveraged to triangulate with other data sources across studies. Mixed methods approaches not only provide nuanced detail regarding particular aspects of an implementation, but also help to further contextualize data collected via quantitative approaches. Further, our organization is conducting preliminary impact studies, while still engaging in Inclusive R&D cycles. We are both aiming to gather rich data of the approaches' impact on student math learning and understanding how these approaches can continue to be improved. Our organization provides opportunities for educators to be embedded throughout both the design and research of the learning approaches. Partner teachers and students who are participating in the preliminary studies provide valuable feedback through surveys and interviews, sharing how an approach is useful to them and how it can be improved. Educators and students also have opportunities to provide insight regarding the ways growth is assessed throughout the study. Iteration is possible not only for the learning approaches themselves, but also in the ways our community selects measures and interprets data. Our Inclusive R&D approach allows for repeated opportunities to revise what is considered as evidence of impact, so that we can more closely connect research and practice, towards the goal of transforming teaching and learning. Conclusion: Our community is navigating tensions around needing to gather evidence of educational impact while also wanting to explore the deep nuances of practice that can lead to transforming learning experiences for priority students. Disrupting practices around educational effectiveness can be challenging when acting as an individual or small team; our organization has the opportunity to create change in terms of institutional practices and culture. We aim to share our current learnings and strategies with the community to consider ways that we can come together to push for more equitable notions of evidence of impact that are responsive to current educational contexts.
Descriptors: Educational Improvement, Educational Change, Equal Education, Educational Quality, Student Experience, Academic Achievement, Teacher Student Relationship, Learning Strategies, Teaching Methods, Mathematics Achievement, Mathematics Curriculum, Mathematics Instruction, Elementary Secondary Education
Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/
Publication Type: Reports - Research
Education Level: Elementary Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Society for Research on Educational Effectiveness (SREE)
Grant or Contract Numbers: N/A
Author Affiliations: N/A