The analysis of differential item functioning (DIF) examines whether item responses differ according to characteristics such as language and ethnicity, when people with matching ability levels respond differently to the items. This analysis can be performed by calculating various statistics, one of the most important being the Mantel-Haenszel,…

This article demonstrates how textbooks differ in their description of the term "experimental unit". Advanced Placement Statistics teachers and students are often limited in their statistical knowledge by the information presented in their classroom textbook. Definitions and descriptions differ among textbooks as well as among different…

General chemistry tests from the Examinations Institute of the Division of Chemical Education of the American Chemical Society have been analyzed to identify factors that may influence how individual test items perform. In this paper, issues of item order (position within a set of items that comprise a test) and answer order (position of correct…

Rosner and Kochanski (2009) noticed an inconsistency in the mathematical statement of the Law of Categorical Judgment and derived "the valid equation, the Law of Categorical Judgment (Corrected)" (p. 125). The purpose of this comment is to point out that the law can be corrected in many different ways, leading to substantially different…

Aspects of assessment in physics are considered with the aim of designing assessments that will encourage a deep approach to student learning and will ultimately lead to higher levels of achievement. A range of physics questions are considered and categorized by the level of knowledge and understanding which is require for a successful answer.…

Aim: To establish if there are psychometrically sound standardized tests or test items to assess handwriting readiness in 5- and 6-year-old children on the levels of occupations activities/tasks and performance. Method: Electronic databases were searched to identify measurement instruments. Tests were included in a systematic review if: (1)…

Throughout the world, cut scores are an important aspect of a high-stakes testing program because they are a key operational component of the interpretation of test scores. One method for setting standards that is prevalent in educational testing programs--the Bookmark method--is intended to be a less cognitively complex alternative to methods…

The false memory effect produced by the Deese/Roediger & McDermott (DRM) paradigm is reportedly impervious to warnings to avoid false alarming to the critical lures (D. A. Gallo, H. L. Roediger III, & K. B. McDermott, 2001). This finding has been used as strong evidence against models that attribute the false alarms to a decision…

Temporal discounting, the loss in perceived value associated with delayed outcomes, correlates with a number of personality measures, suggesting that an item-level analysis of trait measures might provide a more detailed understanding of discounting. The current report details two studies that investigate the utility of such an item-level…

A differential item functioning (DIF) detection method for testlet-based data was proposed and evaluated in this study. The proposed DIF model is an extension of a bifactor multidimensional item response theory (MIRT) model for testlets. Unlike traditional item response theory (IRT) DIF models, the proposed model takes testlet effects into…

In this study, we examine the degree of construct comparability and possible sources of incomparability of the English and French versions of the Programme for International Student Assessment (PISA) 2003 problem-solving measure administered in Canada. Several approaches were used to examine construct comparability at the test- (examination of…

Practitioners often depend on item analysis to select items for exam forms and have a variety of options available to them. These include the point-biserial correlation, the agreement statistic, the B index, and the phi coefficient. Although research has demonstrated that these statistics can be useful for item selection, no research as of yet has…

Two different types of anchors are investigated in this study: a mini-version anchor and an anchor that has a less spread of difficulty than the tests to be equated. The latter is referred to as a midi anchor. The impact of these two different types of anchors on observed score equating are evaluated and compared with respect to systematic error…

Background: When studying student performance across different countries or cultures, an important aspect for comparisons is that of score comparability. In other words, it is imperative that the latent variable (i.e., construct of interest) is understood and measured equivalently across all participating groups or countries, if our inferences…

Item response theory (IRT) becomes an increasingly important tool when analyzing "big data" gathered from online educational venues. However, the mechanism was originally developed in traditional exam settings, and several of its assumptions are infringed upon when deployed in the online realm. For a large-enrollment physics course for…