This paper introduces AMMORE, a new dataset of 53,000 math open-response question-answer pairs from Rori, a mathematics learning platform used by middle and high school students in several African countries. Using this dataset, we conducted two experiments to evaluate the use of large language models (LLM) for grading particularly challenging…

Learning data analytics improves the learning field in higher education using educational data for extracting useful patterns and making better decisions. Identifying potential at-risk students may help instructors and academic guidance to improve the students' performance and the achievement of learning outcomes. The aim of this research study is…

Peer-grading is commonly adopted by instructors as an effective assessment method for MOOCs (Massive Open Online Courses) and SPOCs (Small Private online course). For solving the problems brought by varied skill levels and attitudes of online students, statistical models have been proposed to improve the fairness and accuracy of peer-grading.…

We show an approach to semi-automatic grading of answers given by students to open ended questions (open answers). We use both peer-evaluation and teacher evaluation. A learner is modeled by her Knowledge and her assessments quality (Judgment). The data generated by the peer- and teacher-evaluations, and by the learner models is represented by a…

Item Response Theory (IRT) is a ubiquitous model for understanding humans based on their responses to questions, used in fields as diverse as education, medicine and psychology. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving test scoring and better informing public policy. Yet larger…

E-assessments of students profoundly influence their motivation and play a key role in the educational process. Adapting assessment techniques to current technological advancements allows for effective pedagogical practices, learning processes, and student engagement. The "Handbook of Research on E-Assessment in Higher Education"…

Markov chain Monte Carlo (MCMC) methods enable a fully Bayesian approach to parameter estimation of item response models. In this simulation study, the authors compared the recovery of graded response model parameters using marginal maximum likelihood (MML) and Gibbs sampling (MCMC) under various latent trait distributions, test lengths, and…

The use of Bayesian methodology to assign grades in classroom situations is presented. Assigning a grade is viewed from a criterion, as opposed to norm, referenced perspective. Criteria include mastery of some proportion, determined by the teacher, of the subject matter covered in a course. Different levels of mastery are deemed possible and,…

Effects of blind guessing on the success of passing true-false and multiple-choice tests are investigated under a stochastic binomial model. Critical values of guessing are thresholds which signify when the effect of guessing is negligible. By checking a table of critical values assembled in this paper, one can make a decision with 95% confidence…

A universal criterion scale is applied to the direct estimation of validity for various groups of law school applicants and to the testing of the assumptions underlying the more familiar extrapolation methods. Data from American law schools are used to illustrate the procedures. (Author/BW)