Model evaluation -- the process of making inferences about the performance of predictive models -- is a critical component of predictive modelling research in learning analytics. We survey the state of the practice with respect to model evaluation in learning analytics, which overwhelmingly uses only naïve methods for model evaluation or…

The work described in this report is the first phase of a project to provide easy-to-use tools for authoring and rendering secondary-school algebra-level math expressions in synthesized speech that is useful for students with blindness or low vision. This report describes the initial development, software implementation, and evaluation of the…

Automated scoring models were trained and evaluated for the essay task in the "Praxis I"® writing test. Prompt-specific and generic "e-rater"® scoring models were built, and evaluation statistics, such as quadratic weighted kappa, Pearson correlation, and standardized differences in mean scores, were examined to evaluate the…

This study examined test-retest reliability of the Norwegian version of Children's Assessment of Participation and Enjoyment (CAPE), and Preferences for Activities of Children (PAC) in children with and without disabilities. Totally 141 children, 107 typically developing, mean age 11.1, and 34 with disabilities, mean age 14.2 years participated. A…

Computerized classification testing (CCT) is an approach to designing tests with intelligent algorithms, similar to adaptive testing, but specifically designed for the purpose of classifying examinees into categories such as "pass" and "fail." Like adaptive testing for point estimation of ability, the key component is the…

The study was carried out to establish the competencies of graduate teachers who graduated from teacher education programs between the year 2005 and 2010. Multi-stage sampling was used in selecting 432 respondents comprising educational administrators and graduate teachers working in the three levels of education across two states of Southern…

Assessing the comparability of different groups is an issue facing many researchers and evaluators in a variety of settings. Commonly, null hypothesis significance testing (NHST) is incorrectly used to demonstrate comparability when a non-significant result is found. This is problematic because a failure to find a difference between groups is not…

We used an experimental design to examine the intrapersonal and interpersonal processes through which neutral display rules, compared to positive display rules, influence objective task performance of poll workers and ratings provided by survey respondents of the poll workers. Student participants (N = 140) were trained to adhere to 1 of the 2…

In social policy evaluations, the multiple testing problem occurs due to the many hypothesis tests that are typically conducted across multiple outcomes and subgroups, which can lead to spurious impact findings. This article discusses a framework for addressing this problem that balances Types I and II errors. The framework involves specifying…

This study is aimed at addressing the criticisms that have been leveled at the currently available statistical procedures for analyzing single subject designs (SSD). One of the vexing problems in the analysis of SSD is in the assessment of the effect of intervention. Serial dependence notwithstanding, the linear model approach that has been…

A new process model of the interplay between memory and judgment processes was recently suggested, assuming that retrieval fluency--that is, the speed with which objects are recognized--will determine inferences concerning such objects in a single-cue fashion. This aspect of the fluency heuristic, an extension of the recognition heuristic, has…

Monitoring Trends in Educational Growth (MTEG) offers a flexible, collaborative approach to developing and implementing an assessment of learning outcomes that yields high-quality, nationally relevant data. MTEG is a service that involves ACER staff working closely with each country to develop an assessment program that meets the country's…

Drawing on experience between 2000 and 2007 in developing a validity argument for the high-stakes Test of English as a "Foreign Language[TM]" (TOEFL[R]), this paper evaluates the differences between the argument-based approach to validity as presented by "Kane (2006)" and that described in the 1999 "AERA/APA/NCME Standards for Educational and…

This article discusses CALL development from both educational and ergonomic perspectives. It focuses on the learner-task-tool interaction, in particular on the aspects contributing to its overall quality, herein called "usability." Two pilot studies are described that were carried out with intermediate to advanced learners of French in two…

The reciprocal exchange of employees' work for pay that is central to employment relationships is viewed here through the lens of the psychological contract. A psychological contract involves promised inducements, promised contributions, delivered inducements, and delivered contributions: How an employee cognitively integrates these 4 elements is…