It is important, when writing opinion essays, for students to consider and integrate both arguments and counterarguments to develop a final conclusion. In this article, the authors explored the effect of criteria instruction and a graphic organizer to promote integration of arguments and counterarguments. The researchers randomly assigned 84…

The assessment of children's social skills is an important task for school psychologists in both applied and research settings. The present study examines the psychometric properties of parent ratings of the Social Skills Rating System (SSRS), in kindergarten through third grades, testing for measurement differences between boys and girls, between…

"Collaboration" is a ubiquitously championed concept and widely recognized across the public and private sectors as the foundation on which the capacity for addressing complex issues is predicated. For those invested in organizational improvement, high-quality collaboration has become no less than an imperative. However, evaluators and program…

This article discusses the use of propensity scoring in experimental program evaluations to estimate impacts for subgroups defined by program features and participants' program experiences. The authors discuss estimation issues and provide specification tests. They also discuss the use of an overlooked data collection design--obtaining predictions…

Encouraging continuous improvement in the quality, scale and breadth of online education, the Sloan Consortium invites practitioners to share effective practices. This report synthesizes effective practices submitted by Sloan-C members to the online collection at http://www.sloanconsortium.org/effective as of November 2010. The synthesis includes…

This report deals with a variety of ways that have been, or could be, used to interpret the scales used in the National Assessment of Educational Progress (NAEP). Policymakers, researchers, and other users of assessment results need to understand the methods used for reporting NAEP. Having a reference is particularly important as the methods of…

Comparative data and results of validation and standard-setting studies are presented for five subject-matter tests for teacher certification in two states. The five tests evaluated were available from the Educational Testing Service (ETS): one National Teacher Examination specialty area test (special education); and four tests from a consortium…

This book is a guide to categorizing, measuring, testing, and assessing aspects of language, and is intended for language teachers, speech therapists and other language-related practitioners, and researchers, in conjunction with other resources on research methods and statistics. The first part is a discussion of basic terminology and the varied…

Large scale surveys usually employ a complex sampling design and as a consequence, no standard methods for estimation of the standard errors associated with the estimates of population means are available. Resampling methods, such as jackknife or bootstrap, are often used, with reference to their properties of robustness and reduction of bias. A…

Free-response (FR) item formats, such as essay questions, are popular in educational assessment. The criticisms against FR items are that they are more expensive to score, take up more testing time, provide less content coverage, and are less reliable than multiple-choice (MC) items. For these reasons, FR items are often combined with MC items.…

The amount of variability contributed to large-scale performance assessment scores by raters is a constant concern for those who wish to use results from these assessments for educational decisions. This study approaches the problem by examining the behaviors of essay scorers who demonstrate different levels of proficiency with a holistic scoring…

When test-takers are offered a choice of essay questions, some questions may be harder than others. If the test includes a common portion taken by all test-takers, an adjustment to the scores is possible. Previously proposed adjustment procedures disregard the test-makers' efforts to create questions of equal difficulty; these procedures tend to…

The usual assessment of speededness for rights-only scored tests does not account for the possibility that examinees respond in a random or patterned fashion to the items at the end of the test as the time limit approaches. This study represented an attempt to determine if Sections 2 and 3 of the Test of English as a Foreign Language (TOEFL) are…

This paper explains the multifacet technology for analyzing performance examinations and the fair average method of setting criterion standards. The multidimensional nature of performance examinations requires that multiple and often different facets elements of a candidate's examination form be accounted for in the analysis. After this is…

This paper argues for a direct, holistic assessment of writing of limited-English-proficient (LEP) students. Holistic writing assessment is the term used for tests that evaluate writing wholly through the production of writing. A holistic writing assessment has at least the following five characteristics: each individual taking the assessment must…