The 2011 Education Agenda put students first by focusing on the individuals who most strongly influence student learning on a daily basis. In order to identify and achieve greatness in the classroom, Indiana needs fair, credible and accurate annual evaluations to differentiate teacher and principal performance, to support professional growth, and…

Certain educational programs and jobs require selecting individuals based on their creativity. Tests of creativity are commonly based on divergent thinking tasks. The scoring of the ideational output of such tasks should ideally be done based on fluency (the number of ideas generated), flexibility (the number of categories into which the generated…

Many testing programs use automated scoring to grade essays. One issue in automated essay scoring that has not been examined adequately is population invariance and its causes. The primary purpose of this study was to investigate the impact of sampling in model calibration on population invariance of automated scores. This study analyzed scores…

The construct map is a particularly good way to approach instrument development, and this author states that he was delighted to read Adam Wyse's thoughts about how to use construct maps for standard setting. For a number of popular standard-setting methods, Wyse shows how typical feedback to panelists fits within a construct map framework.…

Two of the largest Massive Open Online Course (MOOC) organizations have chosen different methods for the way they will score and provide feedback on essays students submit. EdX, MIT and Harvard's non-profit MOOC federation, recently announced that they will use a machine-based Automated Essay Scoring (AES) application to assess written work in…

The major purpose of this study is to assess the conditions under which single scoring for constructed-response (CR) items is as effective as double scoring in the licensure testing context. We used both empirical datasets of five mixed-format licensure tests collected in actual operational settings and simulated datasets that allowed for the…

Purpose: The purpose of the present study was to (a) describe and compare the nonword repetition (NWR) performance of preschool-age Spanish-speaking children (3- to 5-year-olds) with and without language impairment (LI) across 2 scoring approaches and (b) to contrast the classification accuracy of a Spanish NWR task when item-level and percentage…

The Common Core assessments emphasize short essay constructed response items over multiple choice items because they are more precise measures of understanding. However, such items are too costly and time consuming to be used in national assessments unless a way is found to score them automatically. Current automatic essay scoring techniques are…

The first edition of the Gray Oral Reading Tests (GORT, 1963) was written by Dr. William S. Gray, a founding member and the first president of the International Reading Association. The GORT was designed to measure oral reading abilities (i.e., Rate, Accuracy, Fluency, and Comprehension) of students in Grades 2 through 12 due to the noteworthy…

The decision to pass or fail a medical student is a "high stakes" one. The aim of this study is to introduce and demonstrate the feasibility and practicality of a new objective standard-setting method for determining the pass/fail cut-off score from borderline grades. Three methods for setting up pass/fail cut-off scores were compared: the…

Nested logit models have been presented as an alternative to multinomial logistic models for multiple-choice test items (Suh and Bolt in "Psychometrika" 75:454-473, 2010) and possess a mathematical structure that naturally lends itself to evaluating the incremental information provided by attending to distractor selection in scoring. One potential…

According to Kane (2006), the argument-based framework is quite simple and involves two steps. First, specify the proposed interpretations and uses of the scores in some detail. Second, evaluate the overall plausibility of the proposed interpretations and uses. Based on experience gained in developing that validity argument, Chapelle, Enright, and…

Students measure out their lives, not with coffee spoons, but with grades on examinations. But what exams mean and whether or not they are a bane or a boon is moot. Senior undergraduates (A. Perrella, J. Koenig, and H. Kwon) designed and administered a 15-item survey that explored the contrasting perceptions of both students (n = 526) and faculty…

ePortfolios and other engaged learning experiences can have extensive impact on students in many facets of their lives, such as subject-area learning, skill and competence development, perspectives on "how the world works," and even students' own identities, confidence, and needs. Assessing these various impacts can be a challenge for…

Previous work has shown that the presence of examples may lead to cryptomnesia, or inadvertent plagiarism, on creative tasks. Various experiential and environmental attributes may magnify this finding. For instance, novices, with limited knowledge, may be more prone to inadvertently plagiarize examples, and increases in cognitive load may result…