There has been increasing interest in applying integrative data analysis (IDA) to analyze data across multiple studies to increase sample size and statistical power. Measures of a construct are frequently not consistent across studies. This article provides a tutorial on the complex decisions that occur when conducting harmonization of measures…

The main purpose of this quantitative-qualitative content analysis study was to compare IELTS and TOEFL listening and reading tests based on the representation of the learning objectives of Revised Bloom's taxonomy. To this end, 12 Academic IELTS listening and reading tests and 12 TOEFL iBT listening and reading tests were analyzed qualitatively…

Despite their significant contributions to research on self-regulated learning, those favoring online and trace approaches have questioned the use of self-report to assess learners' use of learning strategies. An important rejoinder to such criticisms consists of examining the validity of self-report items. The present study was designed to assess…

We examined multimodality (the representation of information in multiple semiotic modes) in the context of international test comparisons. Using Program of International Student Assessment (PISA)-2009 data, we examined the correlation of the difficulty of science items and the complexity of their illustrations. We observed statistically…

Open-ended (OE) items are widely used to gather data on student performance in international achievement studies. However, several factors may threaten validity when using such items. This study examined Finnish coders' opinions about threats to validity when coding responses to OE items in the PISA 2012 problem-solving test. A total of 6…

Teaching and assessing metacognitive activities are important educational objectives, and teachers are calling for efficient instruments. The advantages of questionnaires in measuring metacognitive activities are obvious, but serious validity issues appear. For example, correlations of questionnaire data with think-aloud measures are generally…

Most research on linguistic features of test items negatively impacting English language learners' (ELLs') performance has focused on lexical and syntactic features, rather than discourse features that operate at the level of the whole item. This mixed-methods study identified two discourse features in 162 multiple-choice items on a standardized…

Young children have an impressive amount of mathematics knowledge, but past psychological research has focused primarily on their number knowledge. Preschoolers also spontaneously engage in a form of early algebraic thinking-patterning. In the current study, we assessed 4-year-old children's knowledge of repeating patterns on two occasions…

Recent research (Strunk and Reardon forthcoming) applies Partial Independence Item Response (PIIR) models to teacher bargaining agreements in California to calculate the latent restrictiveness of these contracts. Further research (Strunk and Grissom 2010; Strunk forthcoming) tests the external validity of these estimates. Given that much research…

In addition to validity and reliability evidence, other psychometric qualities of the PE Metrics assessments needed to be examined. This article describes how those critical psychometric issues were addressed during the PE Metrics assessment bank construction. Specifically, issues included (a) number of items or assessments needed, (b) training…

The "PISA 2009 Technical Report" describes the methodology underlying the PISA 2009 survey. It examines additional features related to the implementation of the project at a level of detail that allows researchers to understand and replicate its analyses. The reader will find a wealth of information on the test and sample design,…

Competence-based approaches are frequently adopted as the key paradigm in both formal or non-formal education and training. To support the provision of competence-based learning services, it is necessary to be able to maintain a record of an individual's competences in a persistent and standard way. In this paper, we investigate potential issues…

Counselors conducting survey research have many item format options to contemplate. This study examined midpoint selection and the effect on reliability of including or excluding midpoint options, and using both positively and negatively worded items. Findings indicate that reliability can be affected by both midpoint options and reverse coding.…

This study compared the length of text in English Braille American Edition, the Nemeth code, and the computer braille code with the Unified English Braille Code (UEBC)--also known as Unified English Braille (UEB). The findings indicate that differences in the length of text are dependent on the type of material that is transcribed and the grade…

The purpose of this study is to explore the relationship between a set of item characteristics and the difficulty of TOEFL[R] dialogue items. Identifying characteristics that are related to item difficulty has the potential to improve the efficiency of the item-writing process The study employed 365 TOEFL dialogue items, which were coded on 49…