In previous studies, problem solving appraisal has been identified as playing a key role in promoting positive psychological well-being. The Problem Solving Inventory is the most widely used measure of problem solving appraisal and consists of 32 items. The length of the instrument, however, may limit its applicability to large-scale surveys…

Education systems increasingly emphasize the importance of scientific reasoning skills such as "generating hypotheses" and "evaluating evidence." Despite this importance, we do not know which tests of scientific reasoning exist, which skills they emphasize, how they conceptualize scientific reasoning, and how well they are…

International trended assessments have long attempted to provide instructional information to educational researchers and classroom teachers. Studies have shown that traditional methods of item analysis have not provided specific information that can be directly applicable to improve student performance. To this end, cognitive diagnosis models…

This paper explores the relation between problem solving learning environments (PSLEs) and assessment concepts. The general framework of evidence-centered assessment design is used to describe PSLEs in terms of assessment concepts, and to identify similarities between the process of assessment design and of PSLE design. We use a recently developed…

Research on the graphical facilitation of probabilistic reasoning has been characterised by the effort expended to identify valid assessment tools. The authors developed an assessment instrument to compare reasoning performances when problems were presented in verbal-numerical and graphical-pictorial formats. A sample of undergraduate psychology…

Educators and policymakers envision the future of education in Egypt as enabling learners to acquire scientific inquiry and problem-solving skills. In this article, we describe the validation of a model for problem solving and the design of instruments for evaluating new teaching methods in Egyptian science classes. The instruments were based on…

Flynn wrote a book devoted to the Flynn effect, featuring his theoretical explanation of why the intelligence of worldwide populations has apparently increased from generation to generation. The essence of his theorizing is that because of the societal impact of scientific technology, people of today are much more guided by abstract, rather than…

The models of standard test theory, having evolved under a trait-oriented psychology, do not reflect the knowledge structures and the problem-solving strategies now seen as central to understanding performance and learning. In some applications, however, key qualitative distinctions among persons as to structures and strategies can be expressed…

This article addresses issues in model building and statistical inference in the context of student modeling. The use of probability-based reasoning to explicate hypothesized and empirical relationships and to structure inference in the context of proportional reasoning tasks is discussed. Ideas are illustrated with an example concerning…

Polynomial algorithms are presented that are used to solve selected problems in test theory, and computational results from sample problems with several hundred decision variables are provided that demonstrate the benefits of these algorithms. The algorithms are based on optimization theory in networks (graphs). (SLD)

Described are procedures followed in developing, administering, and scoring a set of mathematical problem-solving superitems and examining their construct validity through a recently developed evaluation technique associated with a taxonomy of the structure of learned outcomes. Data strongly support the validity of the underlying theoretical…

Multiple-choice tests such as the Force Concept Inventory (FCI) provide useful instruments to probe the distribution of student difficulties on a large scale. However, traditional analysis often relies solely on scores (number of students giving the correct answer). This ignores what can be significant and important information: the distribution…

This document reviews several theoretical frameworks of problem-solving, provides a definition of the construct, suggests ways of measuring the construct, focuses on issues for assessment, and provides specifications for the computer-based assessment of problem solving. As defined in the model of the Center for Research on Evaluation, Standards,…

It is only a slight exaggeration to describe the test theory that dominates educational measurement today as the application of twentieth-century statistics to nineteenth-century psychology. Sophisticated estimation procedures, new techniques for missing-data problems, and theoretical advances into latent-variable modeling have appeared--all…

The use of Bayesian decision theory to solve problems in test-based decision making is discussed. Four basic decision problems are distinguished: (1) selection; (2) mastery; (3) placement; and (4) classification, the situation where each treatment has its own criterion. Each type of decision can be identified as a specific configuration of one or…