The object of research is a statistics exam that contains problem tasks. One examiner performed two exam evaluation methods to repeatedly evaluate the exam. The goal was to compare the methods for objectivity. One of the two exam evaluation methods we call a serial evaluation method. The serial evaluation method assumes evaluation of all exam…

This research aimed to develop a valid and reliable test to be used to detect sixth grade students' misconceptions and errors regarding the subject of fractions. A misconception diagnostic test has been developed that includes the concept of fractions, different representations of fractions, ordering and comparing fractions, equivalence of…

The aim of the research reported here was to get some idea of the accuracy of grade boundaries (cut-scores) obtained by applying the 'similar items method' described in Bramley & Wilson (2016). In this method experts identify items on the current version of a test that are sufficiently similar to items on previous versions for them to be…

Increasingly content area projects and programs at the K-12 level, such as in mathematics, involve a programmatic component or project emphasis on developing "teacher leadership". However, there is no consistent definition or framework for this construct and even fewer validated tools for measuring it. This paper describes our efforts in…

In an effort to better align item development to goals of assessing higher-order tasks and decision making, complex decision trees were developed to follow clinical reasoning scripts and used as models on which multiple-choice questions could be built. This approach is compatible with best-practice assessment frameworks like Evidence Centered…

Energy is a critically important topic in the K-12 science curriculum, with many applications in the earth, physical, and life sciences and in engineering and technology. To meet the challenges associated with teaching energy, new tools and assessment instruments are needed. In this work we describe the development of a three-tier assessment…

This paper documents efforts to develop an instrument to measure mathematical knowledge for teaching high school geometry (MKT-G). We report on the process of developing and piloting questions that purported to measure various domains of MKT-G. Scores on the final set of items had no statistical relationship with total years of experience…

The purpose of this study was to construct and evaluate an instrument for determining student preparedness in College Algebra. A 73-item instrument covering prerequisite arithmetic and high school Algebra knowledge for College Algebra was constructed. The instrument was pilot-tested on a freshman population of 595 students. Results of reliability…

Item and test analyses can be used to revise and improve both test items and the test as a whole. Recommendations for item and test analysis practices as they are reported in commonly used measurement textbooks are summarized. A heuristic data set is used to illustrate test and item analysis practices. Techniques developed in this paper are…

Despite the existence of little empirical evidence for their effectiveness, many techniques have been suggested for writing multiple-choice items. The option "none of the above" (NA) has been widely used although a recent review of empirical studies of NA suggests that, while generally decreasing the difficulty index, NA also decreases…

A new indicator of item difficulty, which identifies effectiveness ranges, overcomes the limitations of other item difficulty indexes in describing the difficulty of an item or a test as a whole and in aiding the selection of appropriate ability level items for a test. There are three common uses of the term "item difficulty": (1) the probability…

Development of the test instrument in the Third International Mathematics and Science Study (TIMSS) was based on the expertise of many researchers, including "distinguished scholars from 10 countries" who participated on the TIMSS Subject Matter Advisory Committee. However, a close examination of the TIMSS Science items suggests that not…

The effects of superficial gender-related item wording changes on the performance of male and female examinees were studied through mathematics; discrete English items; and an English passage created in neuter, male, and female gender versions. Units of items were administered to randomly equivalent samples of about 250 examinees taking American…

Generalized expectancies for problem solving referred to by Rotter appear to have significant relevance to the field of psychotherapy, particularly four specific expectancies of interest: (1) looking for alternatives; (2) understanding the motives of others; (3) long-term planning; and (4) discriminating differences in psychological situations. A…

Rasch and classical test analysis methods were compared with respect to their similarities and differences in the identification of noninformative items and implausible person records. Using computer simulated data with known parameters, each model was evaluated in terms of its effectiveness in: (1) identifying noninformative or "bad"…