As evidence from evaluation and experimental studies continue to influence decision and policymaking, applied researchers and practitioners require tools to derive valid and credible inferences. Over the past several decades, research in causal inference has progressed with the development and application of propensity scores. Since their…

Background/Context: Confirmatory factor analysis (CFA) model is a commonly adopted framework to estimate and test a measurement model. Once a well-fitting final CFA model is selected, the selected model may be used to test structural relationships of the latent constructs with other variables, to construct a test with desired reliability and…

This study analyzed the relationship between benchmark scores from the newly published Dynamic Indicators of Basic Early Literacy Skills Math (i.e., Acadience™) math probes and student performance on math and reading sections of a state-mandated high-stakes test. Participants were 420 students enrolled in third, fourth, and fifth grades in a rural…

Applicants must often demonstrate adequate English proficiency when applying to postsecondary institutions by taking an English language proficiency test, such as the TOEFL iBT, IELTS Academic, or Duolingo English Test (DET). Concordance tables aim to provide equivalent scores across multiple assessments, helping admissions officers to make fair…

The Program for the International Assessment of Adult Competencies (PIAAC) is a multicycle survey of adult skills and competencies sponsored by the Organization for Economic Cooperation and Development (OECD). The survey examines a range of basic skills in the information age and assesses these adult skills consistently across participating…

The goal of this study was to provide normative scores and examine the psychometric properties of the Math Computation subtest of the Wide Range Achievement Test--IV (WRAT-IV) for Mexican adolescents after the completion of junior high school. We group-administered this subtest to 1,318 first-year Mexican high school students. We then obtained its…

The maximal reliability of a congeneric measure is achieved by weighting item scores to form the optimal linear combination as the total score; it is never lower than the composite reliability of the measure when measurement errors are uncorrelated. The statistical method that renders maximal reliability would also lead to maximal criterion…

Computational thinking (CT) is one of the systematic tools in problem solving and widely accepted as an important skill in the 21st century. This study aimed to identify the effectiveness of the Chemistry Computational Thinking (CT-CHEM) Module on achievement in chemistry. This study also employed a quasi-experimental design with the participation…

Purpose: Several novel techniques have been developed recently to assess the breadth of a speaker's vocabulary exhibited in a language sample. The specific aim of this study was to increase our understanding of the validity of the scores generated by different lexical diversity (LD) estimation techniques. Four techniques were explored: D, Maas,…

There is a need for developmentally appropriate Computational Thinking (CT) assessments that can be implemented in early childhood classrooms. We developed a new instrument called "TechCheck" for assessing CT skills in young children that does not require prior knowledge of computer programming. "TechCheck" is based on…

School districts across the United States increasingly use value-added models (VAMs) to evaluate teachers. In practice, VAMs typically rely on lagged test scores from the previous academic year, which necessarily conflate summer with school-year learning and potentially bias estimates of teacher effectiveness. We investigate the practical…

This study utilizes an argument-based approach to validation to examine the implications of reliability in order to further differentiate the concepts of score and decision consistency. In a methodological example, the framework of generalizability theory was used to estimate appropriate indices of score consistency and evaluations of the…

The purpose of the current study was to examine dimensionality and concurrent validity evidence of the EARLI numeracy measures (DiPerna, Morgan, & Lei, 2007), which were developed to assess key skills such as number identification, counting, and basic arithmetic. Two methods (NOHARM with approximate chi-square test and DIMTEST with DETECT…

The Mathematics Fluency and Calculation Tests (MFaCTs) are a series of measures designed to assess for arithmetic calculation skills and calculation fluency in children ages 6 through 18. There are five main purposes of the MFaCTs: (1) identifying students who are behind in basic math fact automaticity; (2) evaluating possible delays in arithmetic…

This study analyzed the relationship between benchmark scores from two curriculum-based measurement probes in mathematics (M-CBM) and student performance on a state-mandated high-stakes test. Participants were 298 students enrolled in grades 7 and 8 in a rural southeastern school. Specifically, we calculated the criterion-related and predictive…