Meta-analysis of diagnostic test accuracy (DTA) is a powerful statistical method for synthesizing and evaluating the diagnostic capacity of medical tests and has been extensively used by clinical physicians and healthcare decision-makers. However, publication bias (PB) threatens the validity of meta-analysis of DTA. Some statistical methods have…

This article reviews the evidence on the extent to which experts' perceptions of item difficulties, captured using comparative judgement, can predict empirical item difficulties. This evidence is drawn from existing published studies on this topic and also from statistical analysis of data held by Cambridge Assessment. Having reviewed the…

Cognitive diagnosis models (CDMs) are latent class multidimensional statistical models that help classify people accurately by using a set of discrete latent variables, commonly referred to as attributes. These models require a Q-matrix that indicates the attributes involved in each item. A potential problem is that the Q-matrix construction…

Purpose: To develop models to estimate aerobic fitness (VO[subscript 2]max) from PACER performance in 10- to 18-year-old youth, with and without body mass index (BMI) as a predictor. Method: Youth (N = 280) completed the PACER and a maximal treadmill test to assess VO[subscript 2]max. Validation and cross-validation groups were randomly formed to…

This study compared accuracy of energy expenditure (EE) prediction models from accelerometer data collected in structured and simulated free-living settings. Twenty-four adults (mean age 45.8 years, 50% female) performed two sessions of 11 to 21 activities, wearing four ActiGraph GT9X Link activity monitors (right hip, ankle, both wrists) and a…

The number of malicious files present in the public domain continues to rise at a substantial rate. Current anti-malware software utilizes a signature-based method to detect the presence of malicious software. Generating these pattern signatures is time consuming due to malicious code complexity and the need for expert analysis, however, by making…

We examined the classification accuracy of Curriculum-Based Measurement in writing (CBM-W) Picture Word prompts scored for words written (WW), words spelled correctly (WSC), and correct word sequences (CWS). First graders (n = 133) were administered CBM-W prompts and the Test of Written Language-Third Edition (TOWL-3; Hammill & Larsen, 1996).…

The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

When advice comes from interdependent sources (e.g., from advisors who use the same database), less information should be gained as compared to independent advice. On the other hand, since individuals strive for consistency, they should be more confident in consistent compared to conflicting advice, and interdependent advice should be more…

A common methodological problem in the evaluation of the predictive validity of selection methods, e.g. in educational and employment selection, is that the correlation between predictor and criterion is biased. Thorndike's (1949) formulas are commonly used to correct for this biased correlation. An alternative approach is to view the selection…

Within higher education systems, different institutions deliver different patterns of disciplines. A simple analysis of the structure of that pattern of disciplines across institutions in one higher education system uncovers a surprising relationship. That is, the key dimensions which describe that structure align nearly perfectly with dimensions…

Reading comprehension is based on the efficient accomplishment of several cognitive processes at the word, sentence, and text level. To the extent that each of these processes contributes to reading comprehension, it can cause reading difficulties if it is deficient. To identify individual sources of reading difficulties, tools are required that…

This study had two aims. The first was to explain the process of using the Rasch measurement model to validate tests in an easy-to-understand way for those unfamiliar with the Rasch measurement model. The second was to validate two final exams with several shared items. The exams were given to two groups of students with slightly differing English…

Surveys are frequently used to inform consequential decisions about teachers, policies, and programs. Consequently, it is important to understand the validity of these instruments. This study assesses the validity of measures of instruction captured by an annual survey by comparing survey data with those of a validated daily log. The two…

Metacognitive monitoring processes have been shown to be critical determinants of human learning. Metacognitive monitoring consist of various knowledge estimates that enable learners to engage in self-regulatory processes important for both the acquisition of knowledge and the monitoring of one's knowledge when engaged in assessment. This study…