In this secondary analysis of a previously conducted systematic review, we analyze social validity assessments in intervention research for transition-age autistic youth. Social validity is concerned with the acceptability of the intervention goals, the acceptability and feasibility of the intervention procedures, and the perceived importance of…

The purpose of this instructional piece was to provide a nontechnical synthesis of common internal consistency reliability estimates used in professional counseling and in related fields. The article begins with an overview of coefficients alpha, omega, omega hierarchical, and H, with guidelines for their selection. Next, I provide recommendations…

Few studies have engaged in data-driven investigations of the presence, or frequency, of what could be considered retaliatory assessor behaviour in Multi-source Feedback (MSF) systems. In this study, authors explored how assessors scored others if, before assessing others, they received their own assessment score. The authors examined assessments…

Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers' evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after they scored proposals based on mock overall evaluations…

Item parameter estimation using an item response theory (IRT) model with fixed ability estimates is useful in equating with small samples on anchor items. The current study explores the impact of three ability estimation methods (weighted likelihood estimation [WLE], maximum a posteriori [MAP], and posterior ability distribution estimation [PST])…

Spacing and retrieval practice enhance learning, but students often underuse these strategies. We tested a simple grading incentive, which we call Counting Days, in two RCTs: one randomizing 143 students within a course and another randomizing 71 instructors. The "counting questions" control condition awarded points for each practice…

ACT announced a series of enhancements designed to modernize the ACT test and offer students more choice and flexibility in demonstrating their readiness for life after high school. The enhancements provide students more flexibility by allowing them to choose whether to take the science assessment, thereby reducing the test length by up to…

Representations are an integral component of organic chemistry education and practice. Learning how to visualize and reason effectively with a variety of representations is essential for learners to succeed. However, developing the necessary representational competence skills (e.g., the ability to interpret, translate between, and use…

Since 1991, Chile has provided large, renewable cash grants to indigenous children in lower-income households, conditional on school enrollment. We estimate intent-to-treat effects of grant exposure on indigenous adults and their children, leveraging variation in expected grant exposure across birth cohorts and never-treated adults, and using…

Use of artificial intelligence (AI) to score responses is growing in popularity and likely to increase. Evidence of the validity of scores relies on quadratic weighted kappa (QWK) to demonstrate agreement between AI scores and human ratings. QWK is a measure of agreement that accounts for chance agreement and the ordinality of the data by giving…

Purpose: In grading, one of the most common errors is made in combining two or more different test scores. This study aimed to investigate the agreement of grades calculated by weighting raw scores and standard scores. Research Methods: In this simulation study, data were simulated for midterm and final measurements. Nine conditions [3 (class…

This article presents two ways of quantifying confounding using logistic response models for binary outcomes. Drawing on the distinction between marginal and conditional odds ratios in statistics, we define two corresponding measures of confounding (marginal and conditional) that can be recovered from a simple standardization approach. We…

Writing an integrated essay based on multiple-documents requires students to both comprehend the documents and integrate the documents into a coherent essay. In the current study, we examined the effects of summarization as a potential reading strategy to enhance participants' multiple-document comprehension and integrated essay writing.…

The past few years have seen a shift in many universities' admission policies from test-required to either test-optional or test-blind. This paper uses laboratory experiments to examine students' reporting behavior given their application package and the school's interpretation of non-reported standardized test scores. We find that voluntary…

Background: The Wechsler Intelligence Scale for Children (WISC) employs a hierarchical model of general intelligence in which index scores separate out different clinically-relevant aspects of intelligence; the test is designed such that index scores are statistically independent from one another within the normative sample. Whether or not the…