More than three decades after their introduction, diagnostic classification models (DCM) do not seem to have been implemented in educational systems for the purposes they were devised. Most DCM research is either methodological for model development and refinement or retrofitting to existing nondiagnostic tests and, in the latter case, basically…

Two sampling-and-classification-based procedures were developed for automated test assembly: the Cell Only and the Cell and Cube methods. A simulation study based on a 540-item bank was conducted to compare the performance of the procedures with the performance of a mixed-integer programming (MIP) method for assembling multiple parallel test…

An unprecedented assessment project involving half the states is planning a significant shift: Instead of designing one test for all of them, it will offer a choice of a longer and a shorter version. The pivot came in response to some states' resistance to spending more time and money on testing for the common standards. The plan under discussion…

In a typical study comparing the relative efficiency of two item selection rules in computerized adaptive testing, the common result is that they simultaneously differ in accuracy and security, making it difficult to reach a conclusion on which is the more appropriate rule. This study proposes a strategy to conduct a global comparison of two or…

A program of Assessment for Learning (AfL) was implemented with 107 Year 12 students as part of their preparation for a major external test. Students completed extended mathematics tasks and selected student responses were used for peer assessment purposes. This paper reports on two of the AfL elements, namely task selection and peer assessment as…

Describes a process for evaluating career education activities. Discusses the selection of the evaluation sample, selection and development of instruments, data reduction and analysis, comparison standards, and preparation of an evaluation handbook. (JOW)

It is widely believed that item selection methods using the maximum information approach (MI) can maintain high efficiency in trait estimation by repeatedly choosing high discriminating (alpha) items. However, the consequence is that they lead to extremely skewed item exposure distribution in which items with high alpha values becoming overly…

This paper discusses optimal test construction, which deals with the selection of items from a pool to construct a test that performs optimally with respect to the objective of the test and simultaneously meets all test specifications. Optimal test construction problems can be formulated as mathematical decision models. Algorithms and heuristics…

Defines goal of criterion development in Army's Project A as construction of multiple measures of major components of job performance such that total performance domain for representative sample of population of Army entry-level enlisted positions was covered. Describes major steps in job analyses, content sampling, instrument construction, and…

Six methods for assembling tests from a pool with an item-set structure are presented. All methods are computational and based on the technique of mixed integer programming. The methods are evaluated using such criteria as the feasibility of their linear programming problems and their expected solution times. The methods are illustrated for two…

Developed and validated the Ideal Mentor Scale (IMS), a new measure designed to help graduate students consider the qualities they as individuals most value in a potential mentor. Found that two universal qualities were central to students' definitions of a mentor: communication skills and provision of feedback. Three individual differences…

Describes characteristics of experimental predictor battery of tests developed to supplement the Armed Forces Vocational Aptitude Battery for making selection and classification decisions for entry-level enlisted personnel. Explains procedures used to develop new test battery. Basic psychometric properties of each measure, as determine by large…

Examined the influence of item parameter estimation errors across three item selection methods using the two- and three-parameter logistic item response theory (IRT) model. Tests created with the maximum no target and maximum target item selection procedures consistently overestimated the test information function. Tests created using the theta…

Information based item selection methods in computerized adaptive tests (CATs) tend to choose the item that provides maximum information at an examinee's estimated trait level. As a result, these methods can yield extremely skewed item exposure distributions in which items with high "a" values may be overexposed, while those with low…

This paper describes a four-process model for the operation of a generic assessment: Activity Selection, Presentation, Response Processing (Evidence Identification), and Summary Scoring (Evidence Accumulation). It discusses the relationships between the functions and responsibilities of these processes and the objects in the Instructional…