We describe best practices in reporting reliability estimates in counseling research with consideration to precision, generalization, and diverse populations. We provide a historical context to reporting reliability estimates, the limitations of past practices, and new methods to address reliability generalization. We highlight best practices…

By creating and analyzing matched samples, researchers can simplify their analyses to include fewer covariate variables, relying less on model assumptions, and thus generating results that may be easier to report and interpret. When two groups essentially "look" the same, it is easier to explore their differences and make comparisons…

This is the technical appendix for the report "Gaining Momentum: Refining Corequisite Learning Support to Boost Student Success in the First Year and Beyond" which seeks to develop a body of actionable information about how to refine learning support, close equity gaps, and promote strong momentum toward success for community college…

Rural districts have long faced challenges in closing achievement gaps between subgroups of students. This brief report describes key decision points and considerations for decision-makers interested in identifying rural districts that have closed academic achievement gaps. Examining practices in these districts may suggest activities associated…

For testing programs that administer multiple forms within a year and across years, score equating is used to ensure that scores can be used interchangeably. In an ideal world, samples sizes are large and representative of populations that hardly change over time, and very reliable alternate test forms are built with nearly identical psychometric…

The present paper describes the recruitment and site selection process that has been underway since January 2011, with particular emphasis on the use of Mahalanobis distance score to determine matched pairs of sites prior to randomization to treatment and control groups. Through a systematic winnowing process, the authors found that they could…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes…

The work reported in this paper reflects a collaborative effort of many individuals representing multiple organizations. It began during a session at the October 2008 meeting of TILSA when a representative of a member state asked the group if any of their programs had experienced unexpected fluctuations in the annual state assessment scores, and…

In this set of four supplemental tables, the authors compare the baseline test scores of the treatment and matched control group samples observed in each year after KIPP entry (outcome years 1 to 4). As discussed in Chapter III, the authors used an iterative propensity score estimation procedure to calculate each student's probability of entering…

High quality assessment practice is expected to yield valid and useful score-based interpretations about what the examinees know and are able to do with respect to a defined target domain. Given this assertion, the article presents a framework based on the "unified view of validity," advanced by Cronbach and Messick over two decades ago, to assist…

This study examined the relationship between data use and achievement at 13 urban Title I schools. Using multiple methods, including test scores, district surveys, school transformation plans, and four case study site visits, the researchers found wide variation in the use of data to inform instruction and planning. In some cases, schools were …

This article presents propensity score matching as a method to implement randomized conditions to analyze service effects using nonexperimental data. Most social work research is challenged to implement randomized clinical trials, whereas administrative and survey data are often available and can provide valuable information about services…

Reviews the 25-year record of the National Assessment of Educational Progress (NAEP), highlighting criticisms of test design and validity, sampling techniques, standards, and usefulness of results. Provides the newly organized NAEP with guidelines for selecting input variables and specifies nine variables to be studied. A sidebar describes NAEP's…

With the increasing globalization of human resources practices, a commensurate increase in demand has occurred for multi-language ("global") personality norms for use in selection and development efforts. The combination of data from multiple translations of a personality assessment into a single norm engenders error from multiple sources. This…