Several programs of research have sought to assess the replicability of scientific findings in different fields, including economics and psychology. These programs attempt to replicate several findings and use the results to say something about large-scale patterns of replicability in a field. However, little work has been done to understand the…

Research design is a critical topic that is central to research studies in science, social science, and many other disciplines. After identifying the research topic and formulating questions, selecting the appropriate design is perhaps the most important decision a researcher makes. Currently, there is a plethora of literature presenting multiple…

This article presents the results of a systematic review of methods that have been used to measure or assess metacognition in children aged 4-16 years over a 20-year period (1992-2012). It includes an overview of the types of tool and methods used linked with the ages of the participants targeted and how metacognition and associated concepts are…

Engagement is one of the most widely misused and overgeneralized constructs found in the educational, learning, instructional, and psychological sciences. The articles in this special issue represent a wide range of traditions and highlight several key conceptual, theoretical, methodological, and analytical issues related to defining and measuring…

This article draws on previously employed methods for conducting fidelity studies and applies them to an evaluation of an unprescribed intervention. We document the process of assessing the fidelity of implementation of the Math Recovery first-grade tutoring program, an unprescribed, diagnostic intervention. We describe how we drew on recent…

This article examines definitions, rationales, and calculations associated with higher education performance measures: persistence, retention rate, attrition rate, drop-out rate, and graduation rate. Strengths and limitations of these measures are scrutinized relative to online master's programs. Outcomes of a sample of students (N = 96) enrolled…

In this paper we discuss several key challenges encountered when conducting a study of the continuity of participation in the child care subsidy program. While many of these issues are familiar to those who have studied participation dynamics in other assistance programs, and to those familiar with survival analysis, we describe these challenges…

The purpose of this article was to describe a taxonomy of pretend play for children with disabilities based on a systematic review of the literature to characterize pretend play. Thirty-seven studies were identified as measuring pretend play in children with disabilities. Several inconsistencies were found in the measurement of pretend play across…

Good evaluation requires that evaluation efforts themselves be evaluated. Many things can and often do go wrong in evaluation work. Accordingly, it is necessary to check evaluations for problems such as bias, technical error, administrative difficulties, and misuse. Such checks are needed both to improve ongoing evaluation activities and to assess…

The paper by Rupp and Templin (2008) is an excellent work on the characteristics and features of cognitive diagnostic models (CDM). In this article, the authors comment on some substantial and methodological aspects of this focus paper. They organize their comments by going through issues associated with the terms "cognitive,"…

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment…

This special series focused on treatment integrity in the child mental health and education field is timely. The articles do a laudable job of reviewing (a) the current status of treatment integrity research and measurement, (b) existing conceptual models of treatment integrity, and (c) the limitations of prior research. Overall, this thoughtful…

Analysis of 113 studies revealed that researchers most often selected their learning disabled sample on the bases of academic, age and sex, and intelligence components. Less than half used the process or exclusion components or included a description of grade placement. (CL)

The impact of various definitions of the term "policy" on the nature and outcomes of policy analysis is illustrated by noting a variety of definitions which are implied in the literature. These definitions are applied to exemplar policy arenas to note their differential effect. The policy arenas discussed include the community…

Discusses the assignment of status lables (colloquial, illiterate, popular, slang) to indicate the degree of acceptability of a particular word. Article is part of Lexicography and Dialect Geography, Festgabe for Hans Kurath''. (DD)