Contributing to a competency model for engineering thermodynamics, we investigate the empirical competency structure of our exams in an attempt to answer the question: Do we test the competencies we want to convey to our students? We demonstrate that thermodynamic modeling and mathematical solution emerge as significant dimensions of thermodynamic…

A general model for the homogeneous case of the continuous response is proposed. The model is an expansion and generalization of the one proposed by the author in 1974, in which the open response situation is dealt with. In this generalized model, the closed response situation is dealt with, and it includes the model for the open response…

A rating scale can be expressed as a chain of dichotomous items. The relationship between the dichotomies depends on the manner in which the rating scale is presented to the test taker. Three models for ordered scales are discussed. In the success model, which represents growth, the lowest or easiest category is presented first. If the test taker…

This paper developed and illustrated a technique to analyze categorical data when subjects can appear in any number of categories for multigroup designs. Post hoc procedures to be used in conjunction with the presented statistical test are also developed. The technique is a large sample technique whose small sample properties are as yet unknown.…

The second response conditional probability model of decision-making strategies used by examinees answering multiple choice test items was revised. Increasing the number of distractors or providing distractors giving examinees (N=106) the option to follow the model improved results and gave a good fit to data for 29 of 30 items. (SLD)

A new index for assessing the dimensionality underlying a set of test items was investigated. The incremental fit index (IFI) is based on the sum of squares of the residual covariances. Purposes of the study were to: (1) examine the distribution of the IFI in the null situation, with truly unidimensional data; (2) examine the rejection rate of the…

Research related to the multiple choice test item is reported, as it is conducted by educational technologists in Japan. Sato's number of hypothetical equivalent alternatives is introduced. The based idea behind this index is that the expected uncertainty of the m events, or alternatives, be large and the number of hypothetical, equivalent…

Studied effects of students' use of calculators with 2 experimental forms of a university mathematics test taken by 765 and 725 college students, respectively. Calculator effects are not found for overall scores but are seen for some individual items. Analysis at the item level makes the actual impact apparent. (SLD)

This study investigated the convergent validity of expert-system scores for four mathematical constructed-response item formats. A five-factor model was proposed comprised of four constructed-response format factors and a Graduate Record Examinations (GRE) General Test quantitative factor. Subjects were drawn from examinees taking a single form of…

The purpose of this study was to investigate the effect of using multidimensional items in a computer adaptive test (CAT) setting which assumes a unidimensional item response theory (IRT) framework. Previous research has suggested that the composite of multidimensional abilities being estimated by a unidimensional IRT model is not constant…

This study compared student performance on language-usage test items presented in two different formats: as discrete sentences and as items embedded in passages. American College Testing (ACT) Program's Assessment experimental units were constructed that presented 40 items in the two different formats. Results suggest item presentation may not…

A predictive adaptive testing (PAT) strategy was developed based on statistical predictive analysis, and its feasibility was studied by comparing PAT performance to those of the Flexilevel, Bayesian modal, and expected a posteriori (EAP) strategies in a simulated environment. The proposed adaptive test is based on the idea of using item difficulty…

Recent experience with the Computerized Adaptive Test (CAT) has raised a number of concerns about its practical applications. The concerns are principally involved with the concept of having the computer construct the test from a precalibrated item pool, and substituting statistical characteristics for the test developer's skills. Problems with…

This paper focuses on the questions which arise as test practitioners monitor score scales derived from latent trait theory. Large scale assessment programs are dynamic and constantly challenge the assumptions and limits of latent trait models. Even though testing programs evolve, test scores must remain reliable indicators of progress.…

The feasibility of pre-equating, or establishing conversions from raw to scaled scores through the use of pretest data before operationally administering a test, was investigated for the Scholastic Aptitude Test (SAT). Item-response theory based equating methods were used to estimate item parameters on SAT pretest data, instead of using final form…