Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

Simulation studies are fundamental to psychometric discourse and play a crucial role in operational and academic research. Yet, resources for psychometricians interested in conducting simulations are scarce. This Instructional Topics in Educational Measurement Series (ITEMS) module is meant to address this deficiency by providing a comprehensive…

The impetus for this paper stems from a concern about directions and progress in the measurement of the Technological Pedagogical Content Knowledge (TPACK) framework for effective technology integration. In this paper, we develop the rationale for using a seven-criterion lens, based upon contemporary validity theory, for critiquing empirical…

The use of growth-modeling analysis (GMA)--including hierarchical linear models, latent growth models, and general estimating equations--to evaluate interventions in psychology, psychiatry, and prevention science has grown rapidly over the last decade. However, an effect size associated with the difference between the trajectories of the…

When measuring outcomes in corporate training, the authors recommend that it is essential to introduce a comprehensive plan, especially when resources are limited and the company needs are vast. The authors hone in on five critical components for shaping a measurement plan to determine the success and ROI of training. The plan's components should…

In this chapter, the author proposes a model for assessing institutional effectiveness. The Volkwein model for assessing institutional effectiveness consists of five parts that summarize the steps for assessing institutions, programs, faculty, and students. The first step in the model distinguishes the dual purposes of institutional effectiveness:…

Evaluative statements from alumni have legitimacy with both internal and external stakeholders. Thus, alumni studies can provide valuable evidence of institutional effectiveness and lend themselves to both locally designed and commercial survey instruments for data collection. Under ideal conditions, the results of alumni studies equip faculty and…

Rater biases are of interest to behavior genetic researchers, who often use ratings data as a basis for studying heritability. Inclusion of multiple raters for each sibling pair (M. Bartels, D. I. Boomsma, J. J. Hudziak, T. C. E. M. van Beijsterveldt, & E. J. C. G. van den Oord, 2007) is a promising strategy for controlling bias variance and may…

In practice, the inclusion of correlated residuals in latent-variable models is often regarded as a statistical sleight of hand, if not an outright form of cheating. Consequently, researchers have tended to allow only as many correlated residuals in their models as are needed to obtain a good fit to the data. The current article demonstrates that…

In current psychological research, the analysis of data from computer-based assessments or experiments is often confined to accuracy scores. Response times, although being an important source of additional information, are either neglected or analyzed separately. In this article, a new model is developed that allows the simultaneous analysis of…

Suggests that researchers of counseling effects adopt methods of assessing and representing clients' cognitive structures. A variety of methods are presented, some of which are illustrated in a brief case report. The advantages and limitations of the methods presented are discussed. (Author/JAC)

Measurement of change is essential for documenting the effects of social programs, for refining social-educational programs, and for guiding policy decisions about resource allocation. (CJ)

Discusses the effects of sampling scheme selection on the admissibility of log-linear models for multinomial and product multinomial sampling schemes for prospective and retrospective sampling. Notes that in multinomial sampling, marginal frequencies are not fixed, whereas for product multinomial sampling, uni- or multidimensional frequencies are…

In this paper, the efficiency of conditional maximum likelihood (CML) and marginal maximum likelihood (MML) estimation of the item parameters of the Rasch model in incomplete designs is investigated. The use of the concept of F-information (Eggen, 2000) is generalized to incomplete testing designs. The scaled determinant of the F-information…

Demonstrates how different experimental designs arise from variation of three basic distinctions: block versus treatment factors, fixed versus random factors, and crossed versus nested factors. Argues that understanding how these distinctions influence statistical analysis can reduce amount of experimental design. Presents an example of this by…