In the domain of programming, a growing number of algorithms automatically generate data-driven, next-step hints that suggest how students should edit their code to resolve errors and make progress. While these hints have the potential to improve learning if done well, few evaluations have directly assessed or compared the quality of different…

Early identification of students needing additional support is a foundational component of Multi-Tiered Systems of Support (MTSS). Due to the resource-intensive nature of implementing MTSS, it is critical that universal screening procedures are maximally accurate and efficient. The purpose of this study was to compare the classification accuracy…

Recent literature places more emphasis on assessment comments rather than relying solely on scores. Both are variable, however, emanating from assessment judgements. One established source of variability is "contrast effects": scores are shifted away from the depicted level of competence in a preceding encounter. The shift could arise…

All students deserve to attend programs and institutions that meet high quality standards. But different program goals, investments of time, and eligibility criteria make it difficult to set specific standards that will apply equally well to all types of programs and institutions. This report sets out principles for developing an accountability…

Purpose: Performance evaluation is a topic that has been researched and practiced extensively in business organizations but has received scant attention in higher education institutions. A review of literature revealed that context, input, process, product (CIPP) model is an appropriate performance evaluation model for higher education…

The factorial survey is an experimental design consisting of varying situations (vignettes) that have to be judged by respondents. For more complex research questions, it quickly becomes impossible for an individual respondent to judge all vignettes. To overcome this problem, random designs are recommended most of the time, whereas quota designs…

Purpose: Agreement across methods for identifying students as inadequate responders or as learning disabled is often poor. We report (1) an empirical examination of final status (postintervention benchmarks) and dual-discrepancy growth methods based on growth during the intervention and final status for assessing response to intervention and (2) a…

Purpose: Universidad San Francisco de Quito, USFQ, completed an assessment study to understand its performance in sustainability in 2012. This study aims to recognize the limitations of applying a North American rating system considering relevant criteria to a South American university and to emphasize the importance and lack of benchmarks…

The purpose of this study is to study the performance of different methods (inverse probability weighting and estimation of informative bounds) to control for differential attrition by comparing the results of different methods using two datasets: an original dataset from Portland Public Schools (PPS) subject to high rates of differential…

Policy-makers are frequently interested in understanding how effective a particular intervention may be for a specific (and often broad) population. In many fields, particularly education and social welfare, the ideal form of these evaluations is a large-scale randomized experiment. Recent research has highlighted that sites in these large-scale…

This action research study examines the effect of self-evaluation on grammatical range and grammar accuracy on the English speaking performance of 27 foreign language university and pre-university students enrolled in three different language centers, in three different cities in Colombia. Participants were asked to record themselves answering…

Adult learning policies, like any other policies, need to be effective: they need to reach their objectives and attain the desired impacts, which should be carefully defined. Understanding the performance of policies allows policy makers to change and improve them. A growing body of research and statistics provides important insights into how…

Individual researchers and the organisations for which they work are interested in comparative measures of research performance for a variety of purposes. Such comparisons are facilitated by quantifiable measures that are easily obtained and offer convenience and a sense of objectivity. One popular measure is the journal impact factor, based on…

Education officials and journalists frequently track changes over time in the average ACT® College Readiness Assessment Composite scores and ACT College Readiness Benchmark attainment rates of individual high schools. Using standard statistical methods, I examined how often changes in these statistics are unambiguously positive or negative, rather…

Revolutionary advancements have been observed in e-learning technologies though an amalgamated evaluation methodology for new generation e-learning content development tools is not available. The evaluation of educational software for online use must consider its usability and as well as its pedagogic effectiveness. This study is a first step…