Randomized trials play an important role in estimating the effect of a policy or social work program in a given population. While most trial designs benefit from strong internal validity, they often lack external validity, or generalizability, to the target population of interest. In other words, one can obtain an unbiased estimate of the study…

For decades, the notion of teachers engaging in research has been discussed and carried out under the heuristics and methodologies of action research (Manfra, 2019; Pine, 2009). A typical action research project might involve an individual teacher studying the effectiveness of a specific instructional strategy like having students preview content…

Regression discontinuity (RD) analysis is a rigorous nonexperimental approach that can be used to estimate program impacts in situations in which candidates are selected for treatment based on whether their value for a numeric rating exceeds a designated threshold or cut-point. Over the last two decades, the regression discontinuity approach has…

Much of the current rhetoric in areas of child and family research and in early childhood education emphasizes the importance of listening to children in research that has a direct impact on them. Despite this, there remain qualms in some research contexts and amongst some researchers about the reliability, validity and generalizability of…

Previous efforts to investigate the equivalence of rating sources for job analysis ratings have reported conflicting results. In the present research, correlational and generalizability analyses were conducted to examine the equivalency of rating sources for over 70 state civil service job classifications. Incumbent and supervisor ratings (N=697)…

Statistical significance is misused in educational and psychological research when it is applied as a method to establish the reliability of research results. Other techniques have been developed which can be correctly utilized to establish the generalizability of findings. Methods that do provide such estimates are known as invariance or…