Background: In the context of large-scale educational assessments, the effort required to code open-ended text responses is considerably more expensive and time-consuming than the evaluation of multiple-choice responses because it requires trained personnel and long manual coding sessions. Aim: Our semi-supervised coding method eco (exploring…

Following the onset of the novel coronavirus disease 2019 (COVID-19) pandemic, daily life significantly changed for the population. Accordingly, researchers interested in examining patterns of change over time may now face discontinuities around the pandemic. Researchers collecting in-person longitudinal data also had to cancel or delay data…

Collaboration is a complex skill, comprised of multiple subskills, that is of growing interest to policy makers, educators and researchers. Several definitions and frameworks have been described in the literature to support assessment of collaboration; however, the inherent structure of the construct still needs better definition. In 2015, the…

The purpose of this study is to determine the effect of computer animation technique on academic achievement of students in the "Solar System and Beyond" unit lecture as part of the Science and Technology course of the seventh grade in primary education. The sample of the study consists of 60 students attending to the 7th grade of primary school…

Under the No Child Left Behind Act, large-scale test score trend analyses are widespread. These analyses often gloss over interesting changes in test score distributions and involve unrealistic assumptions. Further complications arise from analyses of unanchored, censored assessment data, or proportions of students lying within performance levels…

Electrochemistry is found to be a difficult topic to learn due to its abstract concepts that involve three representation levels. Research showed that animation and simulation using Information and Communication Technology can help students to visualize and thus enhance students' understanding in learning abstract chemistry topics. As a result, an…

Lord (1980) proposed formulas that provide direct relationships between IRT discrimination and difficulty parameters and conventional item statistics. The purpose of the present study was to determine the robustness of the formulas beyond the initial and restrictive conditions identified by Lord. Simulation and real achievement data were employed.…

Stereotype threat research has demonstrated how presenting situational cues in a testing environment, such as raising the salience of negative stereotypes, can adversely affect test performance (Perry, Steele, & Hilliard, 2003; Steele & Aronson, 1995) and expectancy (Cadinu, Maass, Frigerio, Impagliazzo, & Latinotti, 2003; Stangor,…

A number of vertical scaling methodologies were examined in this article. Scaling variations included data collection design, scaling method, item response theory (IRT) scoring procedure, and proficiency estimation method. Vertical scales were developed for Grade 3 through Grade 8 for 4 content areas and 9 simulated datasets. A total of 11 scaling…

Pretest-posttest experimental designs are often used in randomized control trials (RCTs) in the education field to improve the precision of the estimated treatment effects. For logistic reasons, however, pretest data are often collected after random assignment, so that including them in the analysis could bias the posttest impact estimates. Thus,…

This exploratory study was built on research spanning three decades. Petersen, Marco, and Stewart (1982) conducted a major empirical investigation of the efficacy of different equating methods. The studies reported in Dorans (1990) examined how different equating methods performed across samples selected in different ways. Recent population…

Evaluating the impact of instructional innovations and coordinating instruction, assessment, and testing present complex tensions. Many evaluation and coordination efforts aim to address these tensions by using the coherence provided by modern cognitive science perspectives on domain-specific learning. This paper introduces an alternative…

There has been considerable research regarding the extent to which psychometric sound assessments sometimes yield individual score estimates that are inconsistent with the response patterns of the individual. It has been suggested that individual response patterns may differ from expectations for a number of reasons, including subject motivation,…

This study used innovative assessment practices to obtain and document broad learning outcomes for a 15-hour game-based curriculum in Quest Atlantis, a multi-user virtual environment that supports school-based participation in socio scientific inquiry in ecological sciences. Design-based methods were used to refine and align the enactment of…

A statistical test of the knowledge or random guessing model is presented. A version of the model is introduced in which it is assumed that alternatives can be ordered according to a Guttman scale. Three examples illustrate its application to data from a total of 590 college students. (Author/SLD)