In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of…

Minority studies are underrepresented in gifted and talented education programs across the nation and the methods used to identify students for advanced services may be the issue. This study examined the Scales for Identifying Gifted Students (SIGS), a set of nationally normed behavior rating scales, for the purpose of updating the instrument. The…

This study aims to compare the performance of different small sample equating methods in the presence and absence of differential item functioning (DIF) in common items. In this research, Tucker linear equating, Levine linear equating, unsmoothed and pre-smoothed (C=4) chained equipercentile equating, and simplified circle arc equating methods…

Whether high-stakes exams such as the SAT or College Board AP exams should penalize incorrect answers is a controversial question. In this paper, we document that penalty functions can have differential effects depending on a student's risk tolerance. Moreover, literature shows that risk aversion tends to vary along other areas of concern such as…

The Education Value-Added Assessment System (EVAAS), the value-added model (VAM) sold by the business analytics software company SAS Institute Inc., is advertised as offering "precise, reliable and unbiased results that go far beyond what other simplistic [value-added] models found in the market today can provide." In this study, we…

In this study, it is aimed to investigate the impact of different missing data handling methods on the detection of Differential Item Functioning methods (Mantel Haenszel and Standardization methods based on Classical Test Theory and Likelihood Ratio Test method based on Item Response Theory). In this regard, on the data acquired from 1046…

While dichotomous item dominates the application of computerized adaptive testing (CAT), polytomous item and set-based item hold promises for being incorporated in CAT. However, how to assemble a CAT containing mixed item formats is challenging. This study investigated: (1) how the mixed CAT works compared with the dichotomous-item-based CAT; (2)…

The purpose of this study is to determine if the construct of a medical subspecialty examination, as defined by the hierarchy of item difficulties, is stable across physicians who completed a fellowship and recertifiers as compared to non-fellows. Three comparisons of groups are made: 1) Practice pathway board candidates compared to members of all…

Most research on linguistic features of test items negatively impacting English language learners' (ELLs') performance has focused on lexical and syntactic features, rather than discourse features that operate at the level of the whole item. This mixed-methods study identified two discourse features in 162 multiple-choice items on a standardized…

With reports of declining enrolments in mathematics related degrees and low female participation rates in these degrees, the issue of gender differences in mathematics remains relevant. Results of recent studies suggest gender differences in mathematics are nuanced and that small differences in the early years can manifest as larger differences in…

From 2011-2013 the VCAA conducted a trial aligning the use of computers in curriculum, pedagogy and assessment culminating in a group of 62 volunteer students sitting their end of Year 12 technology-active Mathematical Methods (CAS) Examination 2 as a computer-based examination. This paper reports on statistical modelling undertaken to compare the…

Little research has examined individual linguistic features that influence English language learners (ELLs) test performance. Furthermore, research has yet to explore the relationship between the science strand of test items and the types of linguistic features the items include. Utilizing Differential Item Functioning, this study examines ELL…

This paper discusses issues related to the design and psychometric work for mixed-format tests --tests containing both multiple-choice (MC) and constructed-response (CR) items. The issues of validity, fairness, reliability and score consistency can be addressed but for mixed-format tests there are many decisions to be made and no examination or…

Wright and Stone had proposed three features to assess the quality of the distribution of the items difficulties in a test, on the so called "most probable response map": line, stack and gap. Once a line is accepted as a design model for a test, gaps and stacks are practically eliminated, producing an evidence of the "scale…

Sources of test bias are discussed and steps to prevent or reduce bias in tests are listed. Test bias can occur because of the way test materials are written, the conditions of administration, and the interpretations given the results. Steps to prevent or reduce bias arising in the test development process include: (1) using heterogeneous sets of…