This paper describes, and lays out an argument for, the use of a procedure to help groups of reviewers to judge the quality of prior research reports. It argues why such a procedure is needed, and how other existing approaches are only relevant to some kinds of research, meaning that a review or synthesis cannot successfully combine quality…

To evaluate interim progress toward the State-identified Measurable Result (SIMR), states require access to high-quality data from local education agencies (LEAs) and early intervention service providers. In a review of 2017 Phase III State Systemic Improvement Plans (SSIP), 43 Part C states noted limitations or concerns related to data and…

The Local Child Outcomes Measurement System (L-COMS) is a framework that identifies seven key components of a high-quality child outcomes measurement system at the local level. The L-COMS framework was designed to apply to all local programs regardless of their specific child outcomes measurement approach. Development of the L-COMS originated with…

This article discusses critical methodological design decisions for collecting, interpreting, and synthesizing empirical evidence during the design, deployment, and operational quality-control phases for automated scoring systems. The discussion is inspired by work on operational large-scale systems for automated essay scoring but many of the…

Citizen science, participatory research, and volunteer monitoring all describe research where data are collected by non-professional collaborators. These approaches can allow for research to be conducted at spatial and temporal scales unfeasible for professionals, especially in current budget climates. Mobile computing apps for data collection,…

The "NCTQ Teacher Prep Review" evaluates the quality of programs that provide preservice preparation of public school teachers. This appendix describes the scope, methodology, timeline, staff, and standards involved in the production of "Teacher Prep Review 2014." Data collection, validation, and analysis for the report are…

Survey sampling is fundamentally an applied field. The goal in this book is to put an array of tools at the fingertips of practitioners by explaining approaches long used by survey statisticians, illustrating how existing software can be used to solve survey problems, and developing some specialized software where needed. This book serves at least…

Mixed-Method studies have emerged from the paradigm wars between qualitative and quantitative research approaches to become a widely used mode of inquiry. Depending on choices made across four dimensions, mixed-methods can provide an investigator with many design choices which involve a range of sequential and concurrent strategies. Defining…

The Early Learning Challenge (ELC) program awarded more than $1 billion in four-year grants to 20 States to implement comprehensive and cohesive high-quality early learning systems that support young children with high needs and their families. A key lever in making these improvements was the enhancement of States' Quality Rating and Improvement…

This discussion constructs a survey data quality strategy for institutional researchers in higher education in light of total survey error theory. It starts with describing the characteristics of institutional research and identifying the gaps in literature regarding survey data quality issues in institutional research and then introduces the…

The implementation of Part C of IDEA has continued to evolve as states have built and modified service delivery systems. The number of children served has increased dramatically with nearly 350,000 children and their families now receiving early intervention services. Meanwhile, challenges to the provision of quality services--such as securing…

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes…

Founded in 1889, Slippery Rock University (SRU) is a state university with 8,500 students and 400 faculty. Facilities and planning (F&P) employs 178 staff, of which 50 are custodians. F&P is responsible for the maintenance of 560 acres and 60 major buildings that comprise approximately 2.3 million gross square feet. Currently work is…

Among the practical applications of statistics is the collection of data from manufacturing processes. Often collected in the form of a time series, data collected from a series of measurements show the variation in those measurements, such as mass of a product manufactured. Limits are set for quality control and if these are exceeded then a…

This commentary highlights the work of the Collaborative for Building After-School Systems (CBASS), a collaborative of mature, city and county-wide nonprofit OST intermediaries, to develop and adopt common youth-, program- and system-level measures that are easy and cost-effective for local systems to implement. By agreeing to adopt and publicly…