Data comparison problems are used in teaching and science education research that focuses on students' ability to compare datasets and their conceptual understanding of measurement uncertainties. However, the evaluation of students' decisions in these problems can pose a problem: e.g., students making a correct decision for the wrong reasons.…

This real-data-guided simulation study systematically evaluated the decision accuracy of complex decision rules combining multiple tests within different realistic curricula. Specifically, complex decision rules combining conjunctive aspects and compensatory aspects were evaluated. A conjunctive aspect requires a minimum level of performance,…

Abilities in scientific thinking and reasoning have been emphasized as core areas of initiatives, such as the Next Generation Science Standards or the College Board Standards for College Success in Science, which focus on the skills the future will demand of today's students. Although there is rich literature on studies of how these abilities…

Distractor analyses are routinely conducted in educational assessments with multiple-choice items. In this research report, we focus on three item response models for distractors: (a) the traditional nominal response (NR) model, (b) a combination of a two-parameter logistic model for item scores and a NR model for selections of incorrect…

In theory the formula scoring methods increase the reliability of multiple-choice tests in comparison with number-right scoring. This study aimed to evaluate the impact of the formula scoring method in clinical anatomy multiple-choice examinations, and to compare it with that from the number-right scoring method, hoping to achieve an…

For too long, performance measurement systems in education have failed to document and recognize real differences among educators. But a recent national push to use performance evaluations for critical personnel decisions has highlighted the shortcomings of the current systems and increased the urgency to dramatically improve them. As state and…

Discusses the benefits and shortcomings of three systems for assessing risk factors affecting children: the Illinois CANTS 17B; the Washington Assessment of Risk Matrix; the Child at Risk Field System. Although these systems are used in 23 states, the lack of staff knowledgeable about human development and the uncertain reliability and validity of…

Compares direct and indirect writing assessment strategies and contrasts them in terms of the relationship each has to specific classroom decision-making situations, the components of writing assessed, practical testing matters, characteristics of test exercises, test scoring procedures, and procedures for determining test quality. (HOD)

In the light of reports of bias, the present study investigated the hypothesis that administrative educational decisions assisted by Decision Support Systems (DSS) are characterized by different pedagogical and organizational orientation than decisions made without computer assistance. One hundred and ten high school teachers were asked to suggest…

For six tests, coefficient delta as an index for internal optimality is computed. Internal optimality is defined as the magnitude of risk of the decision procedure with respect to the true score. Results are compared with an alternative index (coefficient kappa) for assessing the consistency of decisions. (Author/JKS)

Native and nonnative speakers' approaches to evaluating student errors are compared, and it is argued that the teacher's assessment is likely to be influenced by such factors as his/her own language competence, familiarity with the learners, teaching priorities, and syllabus as well as frequency and degree of linguistic deviance. (MSE)

This report describes a study comparing the classification results obtained from a one-parameter and three-parameter logistic based tailored testing procedure used in conjunction with Wald's sequential probability ratio test (SPRT). Eighty-eight college students were classified into four grade categories using achievement test results obtained…

The purpose of this study was to provide test-retest reliability data for Holland's Self Directed Search for Educational and Vocational Planning (SDS). Sixty-five subjects were administered the SDS 7-10 months after they completed it during freshman orientation. Results indicated little change in summary codes obtained by subjects. Median…

This paper discusses the use of assessment instruments at the preschool level and systematically reviews several measurement instruments used with young children. The first section briefly discusses the selection of educational objectives, consideration of alternative methods of collecting information about children, the selection of a specific…

During adolescence, one important career-related decision is the choice of a study in higher education. In this article, a new set of measures for different tasks (i.e., orientation, exploration, commitment) that can be distinguished during this career decision-making process was constructed: the Study Choice Task Inventory (SCTI). A sample of 946…