Used generalizability theory to assess the impact of using independent, randomly equivalent groups of experts to develop scoring algorithms for computer simulation tasks designed to measure physicians' patient management skills. Results with three groups of four medical school faculty members each suggest that the impact of the expert group may be…

Two extensions of the Mantel Haenszel procedure that may be useful in assessing differential item functioning (DIF) are explored. Simulation results showed that, for both inferential procedures, the studied item should be included in the matching variable, as in the dichotomous case. (SLD)

A scoring algorithm for performance assessments is described that is based on expert judgments but requires the rating of only a sample of performances. A regression-based policy capturing procedure was implemented for clinicians evaluating skills of 280 medical students. Results demonstrate the usefulness of the algorithm. (SLD)