A log-linear model (LLM) is a well-known statistical method to examine the relationship among categorical variables. This study investigated the performance of LLM in detecting differential item functioning (DIF) for polytomously scored items via simulations where various sample sizes, ability mean differences (impact), and DIF types were…

More robust and rigorous psychometric models, such as multidimensional Item Response Theory models, have been advocated for survey applications. However, item responses may be influenced by construct-irrelevant variance factors such as preferences for extreme response options. Through empirical and simulation methods, this study evaluates the use…

The report is the first systematic evaluation of the sentence equivalence item type introduced by the "GRE"® revised General Test. We adopt a validity framework to guide our investigation based on Kane's approach to validation whereby a hierarchy of inferences that should be documented to support score meaning and interpretation is…

Many theories, methods, and practices are utilized to evaluate teachers with the intention of determining teacher effectiveness to better inform decisions about retention, tenure, certification and performance-based pay. In the 21st century there has been a renewed emphasis on teacher evaluation in public schools, largely due to federal "Race…

The internal/external frame of reference (I/E) model and dimensional comparison theory posit paradoxical relations between achievement (ACH) and self-concept (SC) in mathematics (M) and verbal (V) domains; ACH in each domain positively affects SC in the matching domain (e.g., MACH to MSC) but negatively in the nonmatching domain (e.g., MACH to…

This chapter will present a research synthesis of a series of studies, termed here the Ealing research. The studies use the same general framework to conceptualise tasks and task performance, enabling easier comparability. The different studies, although each is self-contained, build into a wider picture of task performance. The major point of…

The assessment of positive affect (PA) and negative affect (NA) by means of the Positive Affect and Negative Affect Schedule has received a remarkable popularity in the social sciences. Using a meta-analytic tool--namely, reliability generalization (RG)--population reliability scores of both scales have been investigated on the basis of a random…

In this research, we investigated the suitability of implementing "e-rater"® automated essay scoring in a high-stakes large-scale English language testing program. We examined the effectiveness of generic scoring and 2 variants of prompt-based scoring approaches. Effectiveness was evaluated on a number of dimensions, including agreement…

This study employed Bachman and Palmer's (2010) Assessment Use Argument framework to investigate to what extent the use of a second language oral test as an exit test in a Hong Kong university can be justified. It also aimed to help test developers of this oral test identify the most critical areas in the current test design that might need…

Assessments of clinical skills of medical students rely increasingly on standardized patients demonstrating medical cases with faculty rating performance. The common finding of inconsistency of scores across cases is often referred to as case specificity. A multivariate generalizability model reveals that overall case specificity cannot explain…

A proposed model for the validity of measures of professional competence treats validation as the evaluation of inferences drawn from test scores, focusing on evaluation, generalization, and extrapolation. The model is used to indicate strengths and weaknesses of assessments of professional competence: observations of performance, simulations, and…

Located at a meeting place between the West and the East, Hong Kong has been chosen in this comparative investigation to reconfirm a theoretical model of "reciprocal relationship" between mathematics achievement and self-concept using the 8th grade databases from TIMSS and TIMSS-R. During the time between these two projects, Hong Kong…

Positive self-concept and good understanding of science are important indicators of scientific literacy endorsed by professional organizations. The existing research literature suggests that these two indicators are reciprocally related and mutually reinforcing. Generalization of the reciprocal model demands empirical studies in different…

The 7th International Conference on Education Data Mining held on July 4th-7th, 2014, at the Institute of Education, London, UK is the leading international forum for high-quality research that mines large data sets in order to answer educational research questions that shed light on the learning process. These data sets may come from the traces…