Creativity tests, like creativity itself, vary widely in their structure and use. These differences include instructions, test duration, environments, prompt and response modalities, and the structure of test items. A key factor is task structure, referring to the specificity of the number of responses requested for a given prompt. Classic…

Drawing on Glenn Fulcher's extensive work in performance-based language assessment of speaking, this paper explores the assessment of L2 speaking ability in local language testing contexts. For that purpose, I review Fulcher's influential work that highlights the relationship between the speaking construct, the task, the performance, and the…

A Vygotskian approach to assessment is proposed by invoking the distinction between the development of lower and higher psychological functions. Higher psychological functions are specifically human and develop with the use of cultural tools via mediation. Accordingly, a distinction is made between tests that are based on association, which have…

Linguistic complexity of test items is one test format element that has been studied in the context of struggling readers and their participation in paper-and-pencil tests. The present article presents findings from an exploratory study on the potential relationship between linguistic complexity and test performance for deaf readers. A total of 64…

This study examines the use of subpopulation invariance indices to evaluate the appropriateness of using a multiple-choice (MC) item anchor in mixed-format tests, which include both MC and constructed-response (CR) items. Linking functions were derived in the nonequivalent groups with anchor test (NEAT) design using an MC-only anchor set for 4…

This study tracked gender differential item functioning (DIF) on the English subtest of the Korean College Scholastic Aptitude Test (KCSAT) over a nine-year period across three data points, using both the Mantel-Haenszel (MH) and item response theory likelihood ratio (IRT-LR) procedures. Further, the study identified two factors (i.e. reading…

This study examines Taiwanese English as a foreign language (EFL) graduate students' perspectives on paraphrasing strategies. A two-layer scenario survey was developed to identify the reasoning behind students' judgments that certain paraphrasing is appropriate or inappropriate. The first-layer scenario survey is in a true-false format that…

When a test is adapted from a source language (SL) into a target language (TL), the two forms are usually not psychometrically equivalent. If linking between test forms is necessary, those items that have had their psychometric characteristics altered by the translation (differential item functioning [DIF] items) should be eliminated from the…

The correlation between a short-form (SF) test and its full-scale (FS) counterpart is a mainstay in the evaluation of SF validity. However, in correcting for overlapping error variance in this measure, investigators have overattenuated the validity coefficient through an intuitive misapplication of P. Levy's (1967) formula. The authors of the…

Growing recognition of the central importance of fostering an in-depth understanding of natural selection has, surprisingly, failed to stimulate work on the development and rigorous evaluation of instruments that measure knowledge of it. We used three different methodological tools, the Conceptual Inventory of Natural Selection (CINS), a modified…

Four item response theory linking methods (2 moment methods and 2 characteristic curve methods) were compared to concurrent (CO) calibration with the focus on the degree of robustness to format effects (FEs) when applying the methods to multidimensional data that reflected the FEs associated with mixed-format tests. Based on the quantification of…

This study aimed to provide evidence to guide decisions on the type and timing of assessments in a graduate medical programme, by identifying whether students from particular degree backgrounds face greater difficulty in satisfying the current assessment requirements. We examined the performance rank of students in three types of assessments and…

Alternate forms reliability of the Behavioral Relaxation Scale (BRS; Poppen,1998), a direct observation measure of relaxed behavior, was examined. A single BRS score, based on long duration observation (5-minute), has been found to be a valid measure of relaxation and is correlated with self-report and some physiological measures. Recently,…

This study provides support for the hypothesized effect of leniency on the discriminant validity of grouped questionnaire items. It was found that controlling for leniency resulted in a slight decrement in convergent validity but that discriminant validity was substantially improved. Implications for questionnaire validity and further research are…

Two structural laws for intelligence tests are discussed: one law concerns the sign of correlation coefficients and gives conditions under which all correlations between test items will be positive; and one law concerns the relative sizes of the correlation coefficients between intelligence items. A cylindrical structure extends these laws. (SLD)