Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

This research aims to make a methodological contribution to the mixed methods literature by investigating the value-added of empirical studies that are explicitly framed with the purpose of interrogating divergent findings. Case studies introduce the role philosophical paradigms can play in reconciliation process and reveal the potential to…

Qualitative data offer advantages to evaluators, including rich information about stakeholders' perspectives and experiences. However, qualitative data analysis is labor-intensive and slow, conflicting with evaluators' needs to provide punctual feedback to their clients. In this method note, we contribute to the literature on rapid evaluation and…

Since its founding in 1947, ETS has conducted a significant and wide-ranging research program that has focused on, among other things, psychometric and statistical methodology; educational evaluation; performance assessment and scoring; large-scale assessment and evaluation; cognitive, developmental, personality, and social psychology; and…

College completion rates in the U.S. are stubbornly low despite the large and rising returns to a college degree. Efforts to increase student success in college have largely ignored a potentially key factor: the instruction that students receive in the sequence of courses that add up to a college education. Improving the quality of instruction may…

Student evaluation of teaching is commonplace in many universities and may be the predominant input into the performance evaluation of staff and organisational units. This article used publicly available student evaluation of teaching data to present examples of where institutional responses to evaluation processes appeared to be educationally…

One of the central lessons from research on data use in schools and school districts is that assessments, student tests, and other forms of data are only as good as how they are used. But what influences how they are used? This relatively straightforward question turns out to be fairly complex to answer. Data use implicates a number of processes,…

This article offers new directions in measuring racial isolation in schools. The most widely used measurement approach is to examine the mean on the distribution of school percentage non-White across non-White students (the isolation rate) or the percentage of non-White students in schools with large shares of non-Whites (e.g., 90% or more) at a…

Traditional methods for preparing systematic reviews and syntheses of effectiveness studies rely on a limited set of methodological criteria to include studies that measure and report effects too narrowly to forward the mission of evidence-based practice. This article discusses why and how the criteria for study selection, evidence screening, and…

In this paper, we introduce various graphical methods that can be used to represent data in mixed research. First, we present a broad taxonomy of visual representation. Next, we use this taxonomy to provide an overview of visual techniques for quantitative data display and qualitative data display. Then, we propose what we call "crossover" visual…

This paper introduces five commonly used approaches to analyzing multiple-choice test data. They are classical test theory, factor analysis, cluster analysis, item response theory, and model analysis. Brief descriptions of the goals and algorithms of these approaches are provided, together with examples illustrating their applications in physics…

In the modern world, higher education is transitioning from enrollment mode to recruitment mode. This shift paved the way for institutional research and policy making from historical data perspective. More and more universities in the U.S. are implementing and using enterprise resource planning (ERP) systems, which collect vast amounts of data.…

In this article, the authors clarify and provide additional tests of the key elements of their age-period-cohort analysis of non-marital birth rates in this March 2009 issue of "Social Forces." Where Steve Martin, in his commentary, has suggested specific alternative specifications or interpretations of their findings, the authors have…

The last significant review of research about transformative learning was in 1998 and was mostly focused on unpublished dissertations. In response, this paper reviews an exhaustive body of research conducted since that time, involving 40 studies, published in peer-review journals with a lens of analysis of new findings and insights on…

Qualitative researchers in school psychology have a multitude of analyses available for data. The purpose of this article is to present several of the most common methods for analyzing qualitative data. Specifically, the authors describe the following 18 qualitative analysis techniques: method of constant comparison analysis, keywords-in-context,…