ERIC Number: ED658689
Record Type: Non-Journal
Publication Date: 2024
Pages: 218
Abstractor: As Provided
ISBN: 979-8-3835-6528-5
ISSN: N/A
EISSN: N/A
Available Date: N/A
Investigating Content Multidimensionality in a Large-Scale Science Assessment: A Mixed Methods Approach
Cassandra N. Malcom
ProQuest LLC, Ph.D. Dissertation, University of Oregon
Science, Technology, Engineering, and Math (STEM) skills are increasingly required of students to be successful in higher education and the workforce. Therefore, modeling assessment outcomes accurately, often using more types of student data to get a complete picture of student learning, is increasingly relevant. The Program for International Student Assessment (PISA) is promoted as a summative assessment opportunity that includes a science framework. As with many science assessments, the framework includes Life, Physical, and Earth science, which alone seems to imply multidimensionality, and also there are other sources of dimensionality that seem to be described conceptually in the framework. Using data from the 2015 PISA science assessment, a multidimensional item response theory (MIRT) model was fit to see how a multidimensional model operates with the data. Before developing the MIRT model, a qualitative review of the framework for multidimensionality took place and exploratory analyses were implemented for the quantitative data, including a data science technique to explore multidimensionality and some factor analysis techniques. After fitting the MIRT model, it was compared to several unidimensional IRT (UIRT) models to determine the model that explains the most variation. The qualitative analyses generated evidence of multidimensional science content domains in the 2015 PISA science framework, which should require a MIRT model, but quantitative analyses indicate a unidimensional model is more practically significant. Once quantitative results were triangulated with the qualitative review of the framework for multidimensionality, the implications on equity and history of harm with regards to science assessments were discussed. Findings from the qualitative and quantitative aspects of the study were used to generate recommendations for different stakeholders. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
Descriptors: Student Evaluation, STEM Education, Science Tests, Achievement Tests, Foreign Countries, Secondary School Students, International Assessment, Summative Evaluation, Evaluation Methods, Item Response Theory
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Program for International Student Assessment
Grant or Contract Numbers: N/A
Author Affiliations: N/A