Whether alternative scoring strategies result in improved measurement properties of patient management problems (PMPs) was studied. Nine scoring systems (proficiency, efficiency, select, omit, data gathering, therapy, absolute, goal-oriented, and empiric expert score) were applied to 16 PMPs used in a certifying examination taken by 4,590…

A new test critique procedure developed by the National Board of Examiners in Optometry (NBEO) is described. It encourages candidates to identify potentially flawed items during, rather than after, the test. Therefore, candidates retain the opportunity to challenge the test, while the NBEO is able to retain item security. (TJH)

An edited transcript of the recorded discussion among panel members at a symposium at the 1993 meeting of the American Educational Research Association pertaining to changes in standards across time and jurisdiction is presented. Participants' comments reflect concern about the validity and reliability of standards. (SLD)

A test of skill in reading electrocardiographs (ECGs) was developed as a physician competence (PC) test. The criterion-referenced test ranked 1,825 persons taking the 1983 Certifying Examination in Cardiovascular Diseases (CECD) as expected on the basis of prior education and examination scores on the CECD. The ECG test modestly correlated with…

A validation strategy is proposed for professional licensure tests after reviewing arguments for incorporating a theory of professions into the test development process. Components of such a theory are described, and a methodology is proposed for licensure test design that balances the concepts of content and construct validation. (SLD)

Articles in this special issue attempt to address the most significant aspect of certification and licensing testing programs: validity. Although construct validity has become the prevailing model, the theoretical formulation of the construct of competence remains a topic of considerable debate. (SLD)

Aggregate scoring was applied to a recertifying examination for medical professionals to generate an answer key and allow comparison of peer examinees. Results for 1,927 candidates for recertification indicate considerable agreement between the traditional answer key and the aggregate answer key. (TJH)

Issues associated with incorporation of computerized test development and administration into licensure of health professionals are discussed. Specific issues covered include reporting of test procedures and results, computerized test administration, open-book testing, and the impact of regulation. (TJH)

Examinations were analyzed using the Rasch model to determine differences in judge severity and grading period stringency for (1) essay examination (subjects were 12 judges and 32 examinees); (2) clinical examination (subjects were 18 judges and 217 examinees); and (3) oral examination (subjects were 46 judges and 270 examinees). (SLD)

Results from 627 examinees taking a health sciences certification examination suggest that test items written purposefully to assess higher order cognitive skills do not provide evidence of assessing different levels of cognitive processing. Results do not support continuing use of a hierarchical cognitive classification dimension for the test…

Part III of the National Board Examination--a certifying examination of medical knowledge and patient management abilities--was assessed using 1,866 first-year residents. This 15-year study comparing Part III results with those of Parts I and II and with superiors' ratings indicates Part III's validity and provides a model for future research.…

Possible interpretive arguments for licensure and certification test scores are examined, and evidence to support each type of argument is analyzed. Particular attention is given to a competency-based argument that involves a sequence of inferences from test scores to statements about competence and then to conclusions about expected performance.…

An agenda for research on standards for professional licensure and certification examinations is outlined. The relationship between scores and standards is explored, and major classes of standards are described. Topics for future research are discussed in the areas of judgment basis, process efficiency, group effects, and content and expertise…

Assumptions implicit in the current use of licensing and certification examinations are identified, relevant evidence for these assumptions are reviewed, and implications for current methods of measurement are discussed. To ensure the protection of public health, it is critical to determine whether training and credentialing programs serve their…

A set of guidelines for planning and establishing a certification testing program is presented. The guidelines are organized into three components: (1) content specification; (2) item development; and (3) test design. A mythical new certification testing program, called the Tennis Players' Certification Testing Program, is used to illustrate the…