ERIC Number: EJ1481036
Record Type: Journal
Publication Date: 2024
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2161-4210
Available Date: 0000-00-00
Analyzing Small Sample Sizes after Disaggregation: Exploring Observation Oriented Modeling for Assessing Learning Outcomes
J. Vincent Nix; Yi-Chin Wu; Lan Misty Song; Joseph D. Levy
Research & Practice in Assessment, v19 n1 p72-88 2024
Traditionally, assessment professionals use analyses relying upon null hypothesis significance testing (NHST), but those tools have limitations when analyzing small samples or disaggregated data. This study used common NHST analytical techniques, compared their results, and then explored an alternative technique that perhaps allows for a more precise understanding of learning outcomes; to that end, it juxtaposed results from NHST techniques to Observation Oriented Modeling (OOM) results using data from an eight-module, massive online open course (MOOC). The OOM analysis effectively analyzed small sample sizes after disaggregation, revealing relationships between learners' locations and the disproportionate outcomes among two groups: non-native English-speakers and persons identifying as non-binary. These findings show how meaningful disaggregation and interrogation of collected data can help instructors improve pedagogy and materials, as well as serve the field at large by suggesting alternative analytical tools to glean meaningful results from difficult-to-analyze data.
Descriptors: Sample Size, Statistical Significance, MOOCs, Geographic Location, English (Second Language), LGBTQ People, Data Analysis, Student Evaluation, Nonparametric Statistics
Virginia Assessment Group.; Fax: 504-247-1232; e-mail: editor@rpajournal.com; Web site: http://www.rpajournal.com/
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A

Peer reviewed
