Although International Association for the Evaluation of Educational Achievement-pioneered international large-scale assessment (ILSA) of education is now a well-established science, non-practitioners and many users often substantially misunderstand how large-scale assessments are conducted, what questions and challenges they are designed to…

The IEA's International Civic and Citizenship Education Study (ICCS) investigates the ways in which young people are prepared to undertake their roles as citizens in a range of countries in the second decade of the 21st century. ICCS 2016 is the second cycle of a study initiated in 2009. This technical report follows the publication of several…

The idea that multiple measures should be used when evaluating teachers is widely accepted. Multiple measures are important not only because education has multiple goals, but because each measure is an imperfect indicator of any given goal. For a variety of reasons, states and districts use multiple measures in one particular way: to make…

This publication offers SEAs and LEAs steps for evaluating a Request for Proposal (RFP) that allows input from a wide range of participants and specific subject matter experts, permits flexibility and weighted scoring in appropriate areas and provides an objective and defensible process for determining the vendor finalist.

For mixed-type tests composed of both dichotomous and polytomous items, polytomous items often yield more information than dichotomous ones. To reflect the difference between the two types of items, polytomous items are usually pre-assigned with larger weights. We propose an item-weighted likelihood method to better assess examinees' ability…

Objective: This article brings the discussions on incorporating perceived importance across study areas into the study of client satisfaction and cautions the use of multiplicative scores (multiplying satisfaction and importance scores) as a weighting method. An alternative weighting method is provided. Method: Analyze data from a client…

Evaluating college and university faculty teaching performance is necessary for multiple reasons, including assurance of student learning and informing administrative decision-making. A holistic system of evaluating university teaching is necessary for reasons including the limitations of student evaluations and the complexity of assessing…

The purpose of this study is to study the performance of different methods (inverse probability weighting and estimation of informative bounds) to control for differential attrition by comparing the results of different methods using two datasets: an original dataset from Portland Public Schools (PPS) subject to high rates of differential…

By far, the most frequently used method of validating (the interpretation and use of) automated essay scores has been to compare them with scores awarded by human raters. Although this practice is questionable, human-machine agreement is still often regarded as the "gold standard." Our objective was to refine this model and apply it to…

The purpose of this study was to determine the effectiveness of a teacher professional development (PD) program as measured by the extent that participants have continued to use lessons and materials up to three years after the PD experience. The PD was delivered online and structured by five key characteristics of effective PD. A…

Recently, the literature has seen increasing interest in subscores for their potential diagnostic values; for example, one study suggested the report of weighted averages of a subscore and the total score, whereas others showed, for various operational and simulated data sets, that weighted averages, as compared to subscores, lead to more accurate…

Inverse probability weighting (IPW) estimates are widely used in applications where data are missing due to nonresponse or censoring or in observational studies of causal effects where the counterfactuals cannot be observed. This extensive literature has shown the estimators to be consistent and asymptotically normal under very general conditions,…

This study is an attempt to classify raters according to their weighting patterns and explore systematic differences between rater types in the rating process. In the context of an EFL speaking test, 126 raters were classified into three types--form-oriented, balanced, and content-oriented--through cluster analyses of their weighting patterns…

I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and…

This annual report updates the tuition and fee prices published by all of the public higher education institutions in the WICHE member states. It is the product of an annual survey administered to the State Higher Education Executive Officers (SHEEO) offices in each state, except for states such as Arizona, Idaho and South Dakota, where the data…