I propose two practical advances to the argument-based approach to validity: developing a living document and incorporating preregistration. First, I present a potential structure for the living document that includes an up-to-date summary of the validity argument. As the validation process may span across multiple studies, the living document…

This paper describes, and lays out an argument for, the use of a procedure to help groups of reviewers to judge the quality of prior research reports. It argues why such a procedure is needed, and how other existing approaches are only relevant to some kinds of research, meaning that a review or synthesis cannot successfully combine quality…

Ask students to name the aspects of science class they enjoy most, and working on labs will undoubtedly be mentioned. What often won't be included, however, is writing lab reports. For many students, the process of exploration and data collection is paramount, while the explanation and analysis of findings often takes a backseat. After all, if…

The use of numerical/quantitative data in qualitative research studies and reports has been controversial. Prominent qualitative researchers such as Howard Becker and Martyn Hammersley have supported the inclusion of what Becker called "quasi-statistics": simple counts of things to make statements such as "some," "usually," and "most" more…

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes…

Experimental research involves comparing two or more conditions. One of those conditions in single subject research is often the baseline. Adequate description of the baseline conditions is necessary for evaluating the effects of independent variables, drawing generalizations from studies, and providing information for subsequent replication…

Early childhood education and care (ECEC) can bring a wide range of benefits--for children, parents and society at large. However, these benefits are conditional on "quality". Expanding access to services without attention to quality will not deliver good outcomes for children or long-term productivity benefits for society. This new publication…

This Technical Report is designed to provide technical detail to support the analysis and findings presented in the "Head Start Impact Study Final Report" (U.S. Department of Health and Human Services, January 2010). Chapter 1 provides an overview of the Head Start Impact Study and its findings. Chapter 2 provides technical information on the…

Describes how Geographic Information Systems (GIS) can be used to help evaluators convey complex information simply through a spatial representation. Demonstrates how GIS can be used to plot change over time, including impact and outcome data gathered by primary data collection. (SLD)

The Organisation for Economic Cooperation and Development's (OECD's) Programme for International Student Assessment (PISA) surveys, which take place every three years, have been designed to collect information about 15-year-old students in participating countries. PISA examines how well students are prepared to meet the challenges of the future,…

Large-scale surveys of educational attainment gather data about the proficiencies of a sample of students to support inferences about the distribution in the populations. Several approaches to gathering data have been used, each with its own advantages and disadvantages. Several scales have also been used to bring results together from different…

This paper describes the design and function of Louisiana's statewide student-level database, the Student Information System (SIS). In Louisiana the State Board of Elementary and Secondary Education adopted the dropout definition of the National Center for Education Statistics for use by all Louisiana public schools in the 1989-1990 school year.…

Discusses three interlocking areas associated with effectively and accurately conveying information about school performance to the public: (1) graphical display; (2) nonrandomly gathered data; and (3) statistical adjustment. Illustrates these points with historical data, including test results from the National Assessment of Educational Progress…

The author presents an active-learning exercise for the introductory macroeconomics class in which students participate in a mock Federal Open Market Committee (FOMC) meeting. Preparation involves data gathering and writing both a research report and a policy recommendation. An FOMC meeting is simulated in which students give their policy…

Guidelines are offered that make it more likely that high-quality information will be extracted and coded from primary research reports in meta-analyses. It is also noted that the methodology of meta-analysis results in pressure to change the type of information that appears in primary research reports. (SLD)