Recent usage of virtual reality (VR) technology in surgical training has emerged because of its cost-effectiveness, time savings, and cognition-based feedback generation. However, the quantitative evaluation of its effectiveness in training is still not thoroughly studied. This article demonstrates the effectiveness of a VR-based surgical training…

The Remote Associates Test (RAT; Mednick, 1962; Mednick & Mednick, 1967) is a commonly employed test of creative convergent thinking. The RAT is scored with a dichotomous scoring, scoring correct answers as 1 and all other answers as 0. Based on recent research into the information processing underlying RAT performance, we argued that the…

Most social scientists agree that case studies are useful for "theory building," but ethnographic methods papers often look to survey research for case selection strategies. This is due to a common but untenable distinction between theoretical and empirical generalization, which obscures how theoretically inclined ethnographers make…

Comparative judgment is gaining popularity as an assessment tool, including for high-stakes testing purposes, despite relatively little research on the use of the technique. Advocates claim two main rationales for its use: that comparative judgment is valid because humans are better at comparative than absolute judgment, and because it distils the…

Recent years have witnessed a host of innovations for conducting research with qualitative comparative analysis (QCA). Concurrently, important issues surrounding its uses have been highlighted. In this article, we seek to help users design QCA studies. We argue that establishing inference with QCA involves three intertwined design components:…

This article discusses how comparability relates to current mainstream conceptions of validity, in the context of educational assessment. Relevant literature was used to consider the relationship between these concepts. The article concludes that, depending on the exact claims being made about the appropriate interpretations and uses of the…

The internal validity of observational study is often subject to debate. In this study, we define the counterfactuals as the unobserved sample and intend to quantify its relationship with the null hypothesis statistical testing (NHST). We propose the probability of a robust inference for internal validity, that is, the PIV, as a robustness index…

This article reviews the evidence on the extent to which experts' perceptions of item difficulties, captured using comparative judgement, can predict empirical item difficulties. This evidence is drawn from existing published studies on this topic and also from statistical analysis of data held by Cambridge Assessment. Having reviewed the…

The Benefit Offset National Demonstration (BOND) tested a $1 reduction in benefits per $2 earnings increase above the level at which Social Security Disability Insurance benefits drop from full to zero under current law. BOND included a rare and large "population-representative" experiment: It applied the rule to a nationwide, random…

This study compares the psychometric properties of reliability in Classical Test Theory (CTT), item information in Item Response Theory (IRT), and validation from the perspective of modern validity theory for the purpose of bringing attention to potential issues that might exist when testing organizations use both test theories in the same testing…

This paper illustrates similarities and differences between two international surveys that assess adults' skills: the Programme for the International Assessment of Adult Competencies (PIAAC) and the Skills Towards Employment and Productivity (STEP) survey. In particular, the paper highlights the issues that can arise for researchers interested to…

Quasi-experiments are common in studies that estimate the effect of instructional interventions on student performance outcomes. In this type of research, the nature of the experimental design, the choice in assessment, the selection of comparison groups, and the statistical methods used to analyze the comparison data dictate the validity of…

Elicited imitation (EI) has been widely used to examine second language (L2) proficiency and development and was an especially popular method in the 1970s and early 1980s. However, as the field embraced more communicative approaches to both instruction and assessment, the use of EI diminished, and the construct-related validity of EI scores as a…

Our article aims to show how illuminating mathematical work as a concept from didactics of mathematics is useful in understanding issues relating to proving and learning of proof, with or without technology. After posing our hypotheses on the relationship of proof with mathematical work, the pedagogical intent of historical elements of geometry…

There is increased emphasis on using experimental and quasi-experimental methods to evaluate educational programs; however, educational evaluators and school leaders are often faced with challenges when implementing such designs in educational settings. Use of a historical cohort control group design provides a viable option for conducting…