Audit correspondence studies are field experiments that test for discriminatory behavior in active markets. Researchers measure discrimination by comparing how responsive individuals ("audited units") are to correspondences from different types of people. This article elaborates on the tradeoffs researchers face between sending audited…

The Benefit Offset National Demonstration (BOND) tested a $1 reduction in benefits per $2 earnings increase above the level at which Social Security Disability Insurance benefits drop from full to zero under current law. BOND included a rare and large "population-representative" experiment: It applied the rule to a nationwide, random…

To draw informed conclusions from research studies, research consumers need full and accurate descriptions of study methods and procedures. Preregistration has been proposed as a means to clarify reporting of research methods and procedures, with the goal of reducing bias in research. However, preregistration has been applied primarily to research…

Many practitioner researchers strive to understand which assessment practices have the best impact on learning, but in authentic educational settings, it can be difficult to determine whether one intervention, for example the introduction of an online quiz to a course studied by diverse students, is responsible for the observed effect. This paper…

This chapter is a call for consequential education research that has transformative potential: intellectually, educationally, and socially. It is about learning to see differently. It is an argument about seeing our work with youth and communities in ways that can help education researchers see ingenuity instead of ineptness and inability, to see…

This paper reviews literature on the rationale, challenges, and recommendations for choosing a nonequivalent comparison (NEC) group design when evaluating intervention effects. After reviewing frequently addressed threats to validity, the paper describes recommendations for strengthening the research design and how the recommendations were…

The integration of ideas, methods, and data from diverse disciplines has been a transformative force in science and higher education, attracting policy interventions, program innovations, financial resources, and talented people. Much energy has been invested in producing a new generation of scientists trained to work fluidly across disciplines,…

This poem, with stanzas in limerick form, refers humorously to the many threats to validity posed by problems in research design, including problems of sample selection, data collection, and data analysis. (SLD)

Describes the nature and consequences of liberally or improperly applying the traditional reading research methodology and provides an argument for tempering judgments about the relative contributions that experimental studies make to the professional literature in reading. (SKC)

An attempt has been made to show how the problem of data bias had been approached, conceptualized and studied. A review of some of the major studies in the area of research artifacts is presented. Following each major topic, suggestions for future research are discussed. The difficulty in complying with the current APA guidelines on ethics is also…

In this review of the current state of humor research, the author traces the basic problems to two major deficiencies: (1) a lack of adequate theorizing concerning the effects of humor in communication, and (2) problems in experimental designs, particularly the study of humor in a laboratory setting. Too little research and insufficient attention…

The decisive power of an experiment, for both social evaluation research and laboratory experimentation as well, is only as strong as the clarity of the basic assumptions which underlie it. A dialectical methodology is proposed for assessing the influence of key assumptions in both settings. (Author/CTM)

This paper represents two intentions: to discuss the need for theoretical advance in generational studies, and to make suggestions about expanding the design repertoire of generational studies in the interests of these theory developments. First, because generational phenomena on the face seem intransigent to experiment, a comment on…

The scientific method is described as intending to be the guarantor of objectivity, neutrality, and diligent application of its rigorous procedures and to produce a body of cumulative knowledge that is systematic and general. The central argument of this essay addresses the discovery that research findings appear to be context-dependent; methods…