The increasing volume of large-scale assessment data poses a challenge for testing organizations to manage data and conduct psychometric analysis efficiently. Traditional psychometric software presents barriers, such as a lack of functionality for managing data and conducting various standard psychometric analyses efficiently. These challenges…

This guide will help researchers design and implement impact studies in education so that the findings are more generalizable to the study's target population. Guidance is provided on key steps that researchers can take, including defining the target population, selecting a sample of schools--and replacement schools, when needed--managing school…

The purpose of this paper is to present results of recent advances in power analyses to detect the moderator effects in Cluster Randomized Trials (CRTs). This paper focus on demonstration of the software PowerUp!-Moderator. This paper provides a resource for researchers seeking to design CRTs with adequate power to detect the moderator effects of…

The field of education is increasingly committed to adopting evidence-based practices. Although randomized experimental designs provide strong evidence of the causal effects of interventions, they are not always feasible. For example, depending upon the research question, it may be difficult for researchers to find the number of children necessary…

Survey sampling is fundamentally an applied field. The goal in this book is to put an array of tools at the fingertips of practitioners by explaining approaches long used by survey statisticians, illustrating how existing software can be used to solve survey problems, and developing some specialized software where needed. This book serves at least…

As evidence-based reform becomes increasingly important in educational policy, it is becoming essential to understand how research design might contribute to reported effect sizes in experiments evaluating educational programs. The purpose of this study was to examine how methodological features such as types of publication, sample sizes, and…

The Programme for International Assessment of Adult Competencies (PIAAC) will establish technical standards and guidelines to ensure that the survey design and implementation processes of PIAAC yield high-quality and internationally comparable data. This document provides a revised version of the technical standards and guidelines originally…

A brief, practical overview of "design effects" (DEFFs) is presented for users of the results of sample surveys. The overview is intended to help such users to determine how and when to use DEFFs and to compute them correctly. DEFFs are needed only for inferential statistics, not for descriptive statistics. When the selections for…

Present research connected with computer-based instruction indicates that many investigators rely on conventional experimental design and data analysis to address the research question. Although this is appropriate in many cases, the limits of hard and software resources and sample size in other cases have made conventional approaches a poor…

This paper presents five designs (three one-group and two two-group) which can be effectively employed to study computer-based methods in non-laboratory settings, including actual classroom implementation, where a non-treatment control design may not be feasible. These designs are sequential analysis, value-added analysis, non-equivalent dependent…

This study explores an approach to the evaluation of expert system shells using case studies. The methodology and some of the results of an evaluation of the prototype development of an expert system using the shell "M1" are detailed, including a description of the participants and the project, the data collection process and materials,…

The potential uses of microcomputers in evaluation research are discussed in this pamphlet. At the beginning, a matrix is provided showing the relationship between the steps in the evaluation research process and common types of computer software. Thereafter, the guide is organized sequentially around the evaluation research activities that are…

Presents an overview of computer assisted instruction (CAI) as an instructional technique; describes studies on CAI effectiveness in compensatory education; discusses elements that need to be addressed to create a positive relationship between CAI, compensatory education, and achievement gains; and reviews evaluation designs appropriate in…

Political communication researchers recently have emphasized conditional relationships as crucial to specifying and understanding the effects of the mass media. In many cases, however, researchers will not predict which variables will modify the form or existence of media effects. They therefore will not include measures of the conditional…

This paper summarizes and highlights some of the critical developments, issues, and trends which impact the work of survey research. The issue of preventing or minimizing nonresponse bias by successfully obtaining cooperation from people being surveyed continues to be of concern to survey research professionals. The most effective response rate…