The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

This research report presents a summary of research and development efforts devoted to creating scoring models for automatically scoring spoken item responses of a pilot administration of the Test of English-for-Teaching ("TEFT"™) within the "ELTeach"™ framework.The test consists of items for all four language modalities:…

Background/Context: From two decades of research on the grading practices of teachers in secondary schools, researchers discovered that teachers evaluated students on numerous factors that do not validly assess a student's achievement level in a specific content area. These consistent findings suggested that traditional grading practices evolved…

This study examined students' competencies in engineering education at the university level. First, we developed a competency model in one specific field of engineering: process dynamics and control. Then, the theoretical model was used as a frame to construct test items to measure students' competencies comprehensively. In the empirical…

One of the benefits of using collaborative learning is enhancing learning achievement and increasing social skills, and the second benefits is as the more students work together in collaborative groups, the more they understand, retain, and feel better about themselves and their peers, moreover working together in a collaborative environment…

Automated scoring models for the "e-rater"® scoring engine were built and evaluated for the "GRE"® argument and issue-writing tasks. Prompt-specific, generic, and generic with prompt-specific intercept scoring models were built and evaluation statistics such as weighted kappas, Pearson correlations, standardized difference in…

A simple unbalanced block model is proposed for examination marks, as an improvement on the usual implicit model. The new model is applied to some real data and is found, by the usual normal linear theory F test, to give a highly significant improvement. Some alternative models are also considered. (Author)

This study assessed the invariance in the factor structure of the "Test of English as a Foreign Language"™ Internet-based test (TOEFL® iBT) across subgroups of test takers who differed in native language and exposure to the English language. The subgroups were defined by (a) Indo-European and Non-Indo-European language family, (b)…

Results of applying different aptitude-achievement discrepancy models for test scores were compared for 150 second, 150 third, 150 fourth, and 130 fifth graders in several school systems in the Southeast who had been referred for psychoeducational testing for learning disabilities to determine whether different groups of students were identified…

Normal curve equivalent achievement gains estimates were compared with RMC Title I evaluation Models A1 and B1. The comparison focused upon the amount of bias introduced by Model A1 when its underlying assumptions were violated. The model assumes, first, that the local school population is accurately represented by the national norm group; and…

A procedure is demonstrated for constructing tables showing, for each score on a commercial reading achievement test, the percentage of real-world materials that the testee is likely to comprehend with at least a criterion level of proficiency, the percentages of students in a local or national sample who can competently comprehend a given…