This report is the fourth in a series considering career-readiness factors within existing high school assessments. The primary goal of this study was to provide a preliminary validation of the career-readiness features identified in prior reports by exploring how different participant groups with different levels of experience in the…

At times, the same set of test questions is administered under different measurement conditions that might affect the psychometric properties of the test scores enough to warrant different score conversions for the different conditions. We propose a procedure for assessing the practical equivalence of conversions developed for the same set of test…

New technology enables interactive and adaptive scenario-based tasks (SBTs) to be adopted in educational measurement. At the same time, it is a challenging problem to build appropriate psychometric models to analyze data collected from these tasks, due to the complexity of the data. This study focuses on process data collected from SBTs. We…

The study aims to examine whether differential item function is displayed in three different test forms that have item orders of random and sequential versions (easy-to-hard and hard-to-easy), based on Classical Test Theory (CTT) and Item Response Theory (IRT) methods and bearing item difficulty levels in mind. In the correlational research, the…

Process of communication can be challenging. At first participants must standardize their concepts of things to hold them close enough to others' concepts, then it's crucial to use appropriate expressions to verbalize those concepts to ensure the mutual understanding. Therefore, it can be problematic when cognitive constructs are hard to…

Recent research has shown how the statistical bias in Rasch model difficulty estimates induced by guessing in multiple-choice items can be eliminated. Using vertical scaling of a high-profile national reading test, it is shown that the dominant effect of removing such bias is a nonlinear change in the unit of scale across the continuum. The…

Effective professional learning evaluation requires consideration of five critical stages or levels of information. These five levels, which are presented in this article, represent an adaptation of an evaluation model developed by Kirkpatrick (1959, 1998) for judging the value of supervisory training programs in business and industry.…

The present study reports the development of a brief, quantitative, web-based, psychometrically sound measure--the Generalized Acceptance of EvolutioN Evaluation (GAENE, pronounced "gene") in a format that is useful in large and small groups, in research, and in classroom settings. The measure was designed to measure only evolution…

The purpose of this study is to determine the language equivalence and the validity and reliability of the Turkish version of the "Web-Based Learning Platform Evaluation Scale" ("Web Tabanli Ögrenme Ortami Degerlendirme Ölçegi" [WTÖODÖ]) used in the selection and evaluation of web-based learning environments. Within this scope,…

With additional features and increasing cost advantages, e-textbooks are becoming a viable alternative to paper textbooks. One important feature offered by enhanced e-textbooks (e-textbooks with interactive functionality) is the ability for instructors to annotate passages with additional insights. This paper describes a pilot study that examines…

Testing organization needs large numbers of high-quality items due to the proliferation of alternative test administration methods and modern test designs. But the current demand for items far exceeds the supply. Test items, as they are currently written, evoke a process that is both time-consuming and expensive because each item is written,…

Scores on essay-based assessments that are part of standardized admissions tests are typically given relatively little weight in admissions decisions compared to the weight given to scores from multiple-choice assessments. Evidence is presented to suggest that more weight should be given to these assessments. The reliability of the writing scores…

Students often gauge their performance before and after an exam, usually in the form of rough grade estimates or general feelings. Are these estimates accurate? Should they form the basis for decisions about study time, test-taking strategies, revisions, subject mastery, or even general competence? In two studies, undergraduates took a real…

Using an empirically-based simulation study, we show that typically used methods of choosing an item calibration sample have significant impacts on achievement bias and system rankings. We examine whether recent PISA accommodations, especially for lower performing participants, can mitigate some of this bias. Our findings indicate that standard…

The main objective of the present pilot study was to construct and standardize a Scientific Aptitude Test (SAT) for the secondary school science students (Grade 9 and 10). The test items (30 questions) were prepared by the researchers based on the five important aspects in scientific thinking such as Analogy, Scientific reasoning, Numerical…