Careful considerations are necessary when there is a need to choose an anchor test form from a list of old test forms for equating under the random groups design. The choice of the anchor form potentially affects the accuracy of equated scores on new test forms. Few guidelines, however, can be found in the literature on choosing the anchor form.…

Complex competences, such as managing students' aggressive behaviour, are challenging to develop during teacher training. Recently, video-based simulations have been considered promising, yet suitable assessment instruments are limitedly available. This paper reports on the design and evaluation of a video-based assessment tool tailored to measure…

With advances in technology, researchers and test developers are developing new item types to measure complex skills like problem solving and critical thinking. Analyzing such items is often challenging because of their complicated response patterns, and thus it is important to develop psychometric methods for practitioners and researchers to…

The computerized adaptive tests (CAT) apply an adaptive process in which the items are tailored to individuals' ability scores. The multidimensional CAT (MCAT) designs differ in terms of different item selection, ability estimation, and termination methods being used. This study aims at investigating the performance of the MCAT designs used to…

Fieldwork education is an integral part of the educational process in occupational therapy and assessing student competency at the end of fieldwork is important. The aim of this study was to design and conduct an Objective Structured Clinical Examination (OSCE) based on the Occupational Therapy Practice Framework (OTPF) for occupational therapy…

For a multiple-choice test under development or redesign, it is important to choose the optimal number of options per item so that the test possesses the desired psychometric properties. On the basis of available data for a multiple-choice assessment with 8 options, we evaluated the effects of changing the number of options on test properties…

In some popular test designs (including computerized adaptive testing and multistage testing), many item pairs are not administered to any test takers, which may result in some complications during dimensionality analyses. In this paper, a modified DETECT index is proposed in order to perform dimensionality analyses for response data from such…

Ten item discrimination indices, across a variety of item analysis situations, were compared, based on the validities of tests constructed by using each of the indices to select 40 items from a 100-item pool. Item score data were generated by a computer program and included a simulation of guessing. (Author/CTM)

A chart of the major action points in the course of developing tests for training evaluation and for qualifying trainees provides a framework for discussing the problems and practices of test development. Performance standards are the most common sources of trouble; the test developer must have unequivocal standards to work from. (AJ)

Elementary performance tests and situational or simulation tests may be combined for comprehensive testing of open skills, i.e., a worker's competency in reacting to unpredictable situations. Elementary performance tests capture the professional skills whereas simulation tests retain realism and complexity and allow variation in responses.…

The process proposed for the development and use of computer-based testing, including simulation and multiple-choice questions, as part of the National Board of Medical Examiners' certification sequence is outlined. Summary reports of first-phase pilot testing in six medical schools are appended. (MSE)

The development and validation of the General Management In-Basket (GMIB) is described. The GMIB is a theory-based generic in-basket simulation, designed to assess supervisory and management skills independent of any job classification. Three of the 15 in-basket items in the GMIB are critical and are scored on a 0-5 scale. The remaining 12 items…

The purpose of the investigation was to obtain some relationships among (1) test lengths, (2) shape of domain-score distributions, (3) advancement scores, and (4) several criterion-referenced test score reliability and validity indices. The study was conducted using computer simulation methods. The values of variables under study were set to be…

A 40-item flexilevel test and a 40-item conventional test were compared using data obtained through (1) computer-administration of the two tests to three groups of college students, and (2) monte carlo simulation of test response patterns. Results indicated the flexilevel score distribution better reflected the underlying normal distribution of…

Monte Carlo simulation procedures were used to study the psychometric characteristics of two two-stage adaptive tests and a conventional "peaked" ability test. Results showed that scores yielded by both two-stage tests better reflected the normal distribution of underlying ability. Ability estimates yielded by one of the two stage tests…