The purpose of this study is to describe and evaluate a multilingual automated essay scoring (AES) system for grading essays in three languages. Two different sentence embedding models were evaluated within the AES system, multilingual BERT (mBERT) and language-agnostic BERT sentence embedding (LaBSE). German, Italian, and Czech essays were…

Language assessment is a central aspect not only of language education in the general population, but also amongst heterogeneous, low-incidence populations. One such population are immigrant deaf and hard-of-hearing learners (IDML) who are bimodal-multilingual and whose languages development often includes the spoken, written, and/or signed…

Is it possible to measure wealth and poverty across settings while being faithful to local understandings? The stages of progress method (SoP) attempts to do this by building ladders of wealth in locally relevant terms and using these in comparisons across groups. This approach is potentially useful among pastoralist populations where monetary…

Students are assessed by coursework and/or exams, all of which are marked by assessors (markers). Student and marker performances are then subject to end-of-session board of examiner handling and analysis. This occurs annually and is the basis for evaluating students but also the wider learning and teaching efficiency of an academic institution.…

Mediation analysis plays an important role in understanding causal processes in social and behavioral sciences. While path analysis with composite scores was criticized to yield biased parameter estimates when variables contain measurement errors, recent literature has pointed out that the population values of parameters of latent-variable models…

Recent publications emphasize the need to take greater account of differences in teaching quality between subjects. The empirical analysis of this topic requires a comparison of teaching quality in different subjects to distinguish generic aspects of teaching quality from subject-specific ones. In this paper, we compare teaching quality in…

Although evaluators often use an interrupted time series (ITS) design to test hypotheses about program effects, there are few empirical tests of the design's validity. We take a randomized experiment on an educational topic and compare its effects to those from a comparative ITS (CITS) design that uses the same treatment group as the experiment…

In recent years, the Education for All movement has focused more intensely on the quality of education, rather than simply provision. Many recent and current education quality interventions focus on literacy, which is the core skill required for further academic success. Despite this focus on the quality of literacy instruction in developing…

Recent investigations challenge the construct validity of sustained silent reading tests. Performance of two groups of post-secondary students (e.g. struggling and non-struggling) on a sustained silent reading test and two types of cloze test (i.e. maze and open-ended) was compared in order to identify the test format that contributes greater…

The purpose of this paper is (1) to discuss a set of standards that can be used to evaluate potential assessment instruments; and (2) to use these standards to evaluate the American College Testing Program's College Outcomes Measures Program (ACT-COMP) and the Educational Testing Service (ETS) Academic Profile. Using the work of S. Messick (1975,…

The difference in gain scores produced by traditional pretests and those produced by retrospective pretests when compared to posttest scores on the Science Teaching Efficacy Belief Instrument for preservice teachers was investigated in this study. Results indicated that gain scores using the traditional pretest produced significant improvement in…

Significant relationships between the Holland Vocational Preference Inventory (VPI) and the Strong Vocational Interest Blank (SVIB) were again empirically demonstrated in this study, and conversion equations were developed to use standard scores of SVIB scales, rather than items, to produce estimates of VPI scores. (Author)

The Center for Research on Education Outcomes (CREDO) at Stanford University conducted a large-scale analysis of the impact of charter schools on student performance. The center's data covered 65-70% of the nation's charter schools. Although results varied by state, 17% of the charter school students have significantly higher math results than …

"How New York City's Charter Schools Affect Achievement" estimates the effects on student achievement of attending a New York City charter school rather than a traditional public school and investigates the characteristics of charter schools associated with the most positive effects on achievement. Because the report relies on an…

Examined and compared the validity of Friedman's Developmental Level (DL) and Exner's Developmental Quality (DQ) as measures of cognitive development in children (N=134). Results supported the convergent and discriminant validity of both DL and DQ. The DL and DQ were most strongly related to different types of cognitive ability. (JAC)