There is growing literature on computerized dynamic assessment (C-DA) wherein individual items are accompanied by mediating prompts, but its effectiveness at fine-grained levels across time has not been explored sufficiently. This study constructed a computerized listening dynamic assessment (CLDA) system, where mediation was informed by an…

In this article, we argue that current assessment practices in higher education require urgent examination and should be re-imagined in culturally responsive ways to ensure fairness for all. From sociocultural and social justice perspectives, we highlight examples of cultural and linguistic bias in assessment that disadvantages many First Nations…

This study explores different conceptualizations of creativity within an educational setting, using textual coding and network mapping on data from a creativity-focused course. Two primary conceptualizations emerged: the Freedom-Possession and Playful-Adaptation conceptualizations. Each was associated with unique practices and elements of the…

The rising status of English as a global language prompts a call for a paradigm shift in English language teaching in order to correspond to the new sociolinguistic landscape. However, studies related to English language assessment, which can catalyze the transformation especially in a test-oriented context like Thailand, remain limited. This…

Learning experience designers consistently balance the trade-off between open and close-ended activities. The growth and scalability of Computer Based Learning Platforms (CBLPs) have only magnified the importance of these design trade-offs. CBLPs often utilize close-ended activities (i.e. Multiple-Choice Questions [MCQs]) due to feasibility…

Introductory physics lab instruction is undergoing a transformation, with increasing emphasis on developing experimentation and critical thinking skills. These changes present a need for standardized assessment instruments to determine the degree to which students develop these skills through instructional labs. In this article, we present the…

Students' test-taking motivation has been found to be a predictor of performance. This study tests whether Shanghai students' conceptions of tests and test-taking motivation differ when the consequence of tests have different foci (i.e., none, country, or personal). A between-subjects experiment with vignette instructions systematically assigned…

The purpose of this replication study is to establish external validity of the ADSI-S. A convenience sample (n = 114) tested the validity of the ADSI-S with a population aged 18-20 years. Consistency reliability analysis was conducted, correlations were run to determine known-instrument con- struct validity and a logistic regression examined the…

Raven's Standard Progressive Matrices (Raven 1941) is a widely used 60-item long measure of general mental ability. It was recently suggested that, for situations where taking this test is too time consuming, a shorter version, comprised of only the last series of the Standard Progressive Matrices (Myszkowski and Storme 2018) could be used, while…

We analyze a 12-item version of Raven's Standard Progressive Matrices test, traditionally scored with the sum score. We discuss some important differences between assessment in practice and psychometric modelling. We demonstrate some advanced diagnostic tools in the freely available R package, dexter. We find that the first item in the test…

The present study focuses on studying the psychometric properties of Ethical Climate Questionnaire (ECQ) in Saudi educational milieu. It is a cross-sectional study and the data was collected from 309 educational supervisors from six educational offices of General Department of Education in Jeddah. Confirmatory Factor Analysis (CFA) was carried out…

The responses of non-effortful test-takers may have serious consequences as non-effortful responses can impair model calibration and latent trait inferences. This article introduces a mixture model, using both response accuracy and response time information, to help differentiating non-effortful and effortful individuals, and to improve item…

Research suggests that certain characteristics of survey items may impact participants' responses. In this study we investigated the impact of several of these characteristics: vague wording, question-versus-statement phrasing, and full-versus-partial labeling of response options. We manipulated survey items per these characteristics and randomly…

Validity and reliability are a major focus in teacher education accreditation by the Council for Accreditation of Educator Preparation (CAEP). CAEP requires the use of "accepted research standards," but many faculty and administrators are unsure how to meet this requirement. The Standards of Educational and Psychological Testing…

Answering the call for response process-based validation, this study shows how researchers can evaluate validity evidence of test scores based on the mental "processes" test-takers use, rather than based on correlations with other "outcome" measures. The proposed methods for process-based validation studies are demonstrated…