This paper describes, and lays out an argument for, the use of a procedure to help groups of reviewers to judge the quality of prior research reports. It argues why such a procedure is needed, and how other existing approaches are only relevant to some kinds of research, meaning that a review or synthesis cannot successfully combine quality…

The Forum Guide to Data Quality provides best practices from districts and states for maintaining quality data as these agencies regularly review and revise their methods for working with data, as well as their expectations for all staff who are responsible for education data and their quality. This resource is intended to help districts and…

The quantitative assessment of eye tracking data quality is critical for ensuring accuracy and precision of gaze position measurements. However, researchers often report the eye tracker's optimal manufacturer's specifications rather than empirical data about the accuracy and precision of the eye tracking data being presented. Indeed, a recent…

Purpose: This paper aims to provides a brief review of the dashboard literature, an account of the development of performance dashboards for field data collection at Westat, and more specifically for the first cycle of the Programme for the International Assessment of Adult Competencies (PIAAC). A brief concluding section offers suggestions for…

In the past decade there has been a lot of attention to the quality of the evidence in experimental psychology and in other social and medical sciences. Some have described the current climate as a 'crisis of confidence'. We focus on a specific question: how can we increase the quality of the data in psychology and cognitive neuroscience…

The Education Insights Center produced a series of reports culminating in recommendations for the structure and governance of a preschool through higher education and into the workforce (known as a P20W data system). This brief follows up on that series, with a focus on data quality; the brief was informed by the author's experience using…

To evaluate interim progress toward the State-identified Measurable Result (SIMR), states require access to high-quality data from local education agencies (LEAs) and early intervention service providers. In a review of 2017 Phase III State Systemic Improvement Plans (SSIP), 43 Part C states noted limitations or concerns related to data and…

The Local Child Outcomes Measurement System (L-COMS) is a framework that identifies seven key components of a high-quality child outcomes measurement system at the local level. The L-COMS framework was designed to apply to all local programs regardless of their specific child outcomes measurement approach. Development of the L-COMS originated with…

Every 2 years, Education requires nearly all school districts to report incidents of restraint and seclusion. Generally, restraint is restricting a student's ability to move, and seclusion is confining them alone in a space they cannot leave. The House Committee on Appropriations' explanatory statement accompanying the Consolidated Appropriations…

This article discusses critical methodological design decisions for collecting, interpreting, and synthesizing empirical evidence during the design, deployment, and operational quality-control phases for automated scoring systems. The discussion is inspired by work on operational large-scale systems for automated essay scoring but many of the…

This paper aims to extend existing theorisations around the notion of public narratives by analysing their regulatory effects under intensified market conditions. My analysis suggests that public narratives constitute a liminal space, one that it is not exclusively real or imaginary, factual or normative, but that simultaneously affects and is…

High rates of Information Technology (IT) project failures continues; fail to meet established deadlines, exceeding budget, or not agreed-upon functionality. Failure often results from a fundamental confusion over what is involved in the project. Methods that have provided project success in Service and Manufacturing industries have not been…

Citizen science, participatory research, and volunteer monitoring all describe research where data are collected by non-professional collaborators. These approaches can allow for research to be conducted at spatial and temporal scales unfeasible for professionals, especially in current budget climates. Mobile computing apps for data collection,…

Complex environmental problems call for unique solutions to monitoring efforts alongside developing a more environmentally literate citizenry. Community-based monitoring (CBM) through the use of volunteer monitoring organizations helps to provide a part of the solution, particularly when CBM groups work with research scientists or government…

The "NCTQ Teacher Prep Review" evaluates the quality of programs that provide preservice preparation of public school teachers. This appendix describes the scope, methodology, timeline, staff, and standards involved in the production of "Teacher Prep Review 2014." Data collection, validation, and analysis for the report are…