While score comparability across test forms typically relies on common (or randomly equivalent) examinees or items, innovations in item formats, test delivery, and efforts to extend the range of score interpretation may require a special data collection before examinees or items can be used in this way--or may be incompatible with common examinee…

The Programme for International Assessment (PISA) is an important cross-national study of 15-year olds academic achievement. Although it has traditionally been conducted using paper-and-pencil tests, the vast majority of countries will use computer-based assessment from 2015. In this paper, we consider how cross-country comparisons of children's…

There has long been the suspicion amongst staff in Art & Design that the ratings given to their subject disciplines in the UK's National Student Survey are adversely affected by a combination of circumstances--a "perfect storm". The "perfect storm" proposition is tested by comparing ratings for Art & Design with those…

When comparing data derived from tests or questionnaires in cross-national studies, researchers commonly assume measurement invariance in their underlying scaling models. However, different cultural contexts, languages, and curricula can have powerful effects on how students respond in different countries. This article illustrates how the…

The purpose of this document is to provide background information that will be useful in interpreting the 2007 results from the Trends in International Mathematics and Science Study (TIMSS) by comparing its design, features, framework, and items with those of the U.S. National Assessment of Educational Progress and another international assessment…

The microcomputer version of the Ohio Vocational Interest Survey (OVIS II) differs from the machine-scored version in its ability to incorporate data from the OVIS II:Career Planner in its printed report. It differs from the hand-scored version in its ability to include data from the OVIS II:Work Characteristic Analysis in its printed report.…

Five indices are discussed that should logically discriminate between situations in which: (1) the linear equating method (LEM) adequately adjusts for difference between score distributions of two approximately parallel test forms; or (2) a method more complex than the linear equating method is needed. (RL)

Academic psychology has long been composed of two disciplines, one experimental and one correlational. These two disciplines each developed their own method of studying structure in data: multidimensional scaling (MDS) and factor analysis. Both methods use similar kinds of input data, proximity measures on object pairs. Both represent the object…

A study examined the relationship of native language and level of English proficiency to the structure of the Test of English as a Foreign Language (TOEFL). Using all of the information provided by various responses to the test's items (the four alternatives, omitted, and not reached), the items' interrelations were analyzed by three-way…

Six methods of scoring multiple true-false items were compared in terms of reliabilities, difficulties, and discrimination. Results suggest that, for norm-referenced score interpretations, there is insufficient evidence to support any one of the methods as superior. For criterion-referenced score interpretations, effects of scoring method must be…

Ability estimates obtained from applying finite state score theory, item response models, and classical test theory to score multiple-choice items were compared using responses of 1,230 examinees. Scoring models provided essentially the same ranking of examinees, but ease of use and interpretation support the use of the classical test model. (SLD)

New Hampshire has adopted a standards-based statewide assessment, the New Hampshire Educational Improvement and Assessment Program (NHEIAP), which is designed to measure students' learning against proficiency standards at grades 3, 6, and 10. Because of the difficulty teachers had in interpreting the NHEIAP results, a custom-designed software…

A general process for testing the feasibility of applying alternative mathematical or statistical models to the solution of a practical problem is presented and flowcharted. The system is used to compare five models for test equating: (1) anchor test equating using classical test theory; (2) anchor test equating using the one-parameter logistic…

Geometric approaches to representing interrelations among tests and items are compared with an additive tree model (ATM), using 2,644 examinees and 2 other data sets. The ATM's close fit to the data and its coherence of presentation indicate that it is the best means of representing tests and items. (TJH)

Presents major stages in design and development of criterion referenced tests (CRT) with emphasis on differences between CRT construction and norm-referenced test construction. Discussion covers test interpretation; test theory; preparation for test construction (hierarchical analysis, item type selection, and choosing number of items); test…