Rater-mediated activities in educational research occur when an expert judge or rater utilizes an instrument to judge persons or items and generates scale scores. Scale scores are from a subjective judgment and must undergo a quality control measure called rating quality. Rating quality in this study is broadly defined as the extent to which…

This dissertation is composed of a study of estimation methods in classical and test theories and the elaboration and application of a cluster-robust variance estimator. Variance estimators derived from generalized estimating equations are known to be robust to most covariance structures and are therefore well suited for psychometric analysis of…

This dissertation includes two independent research projects, one in astronomy education research and the other in planetary science/astrophysics research. In the first research effort, we investigate college students' conceptual and reasoning difficulties on the topic of planet formation pre-instruction. Through an analysis of over 1,000…

Meeting Science, Technology, Engineering, and Mathematics (STEM) workforce demands require students to have positive attitudes and persist through the STEM pipeline. There is limited research on reliable and valid instruments that measures non-cognitive skills towards all STEM fields. Tools that have been developed have mostly used traditional…

Consumers spend an increasing amount of time and money online finding information, completing tasks, or making purchases. The quality of the website experience has become a key differentiator for organizations--affecting whether they purchase and their likelihood to return and recommend a website to friends. Two instruments were created to more…

Our nation is in the midst of one of the largest education reforms in decades centered on the adoption of the Common Core State Standards (CCSS) and aligned assessments. In an era of rising accountability measures and declining literacy proficiency, it is vital to ensure that educational resources, such as benchmark assessments, are appropriately…

The purpose of the researcher in this quantitative study was to examine the relationship between principals' leadership practices, school culture, and student achievement as perceived by elementary school teachers. The researcher established the 5 research questions to: (a) determine the differences between high- and low-achievement schools on the…

Learning standards across the United States have increasingly called for history students to engage in aspects of "historical thinking," a term used to describe the complex disciplinary processes that historians use to make sense of the past. Although students are expected to learn these complex processes, little is known about how to…

In 1997, based on the amendments to Individuals with Disabilities Education Act (IDEA), all states were faced with a statutory requirement to develop and implement alternate assessments for students with disabilities unable to participate in the statewide large-scale assessment. States were given the challenge of creating, implementing, and…

The topic of Newtonian gravity offers a unique vantage point from which to investigate and encourage conceptual change because it is something with which everyone has daily experience, and because it is taught in two courses that reach a wide variety of students--introductory-level college astronomy ("Astro 101") and physics ("Phys…

In large-scale assessments, such as the National Assessment of Educational Progress (NAEP), plausible values based on Multiple Imputations (MI) have been used to estimate population characteristics for latent constructs under complex sample designs. Mislevy (1991) derived a closed-form analytic solution for a fixed-effect model in creating…

The current nursing shortage has challenged colleges to educate nurses at a faster pace than in previous times. Successful completion of the nursing programs and passing the National Council Licensure Examination for Registered Nurses (NCLEX-RN) exam is important for the students, faculty, and nursing programs. The purpose of this retrospective…

The purpose of the study was to (a) examine the psychometric properties of The Assessment, Evaluation, and Programming System for Infants and Children (AEPS Test); (b) provide a process for establishing psychometric properties for other Curriculum Based Assessments (CBAs); and (c) identify and guide evaluation and subsequent revisions of the AEPS…

This study examined the Flynn Effect (FE; i.e., the rise in IQ scores over time) in Estonia from Scale B of the National Intelligence Test using both classical test theory (CTT) and item response theory (IRT) methods. Secondary data from two cohorts (1934, n = 890 and 2006, n = 913) of students were analyzed, using both classical test theory (CTT)…

Three decision consistency and accuracy (DC/DA) methods, the Livingston and Lewis (LL) method, LEE method, and the Hambleton and Han (HH) method, were evaluated. The purposes of the study were: (1) to evaluate the accuracy and robustness of these methods, especially when their assumptions were not well satisfied, (2) to investigate the "true"…