Different sources of assessment evidence are reviewed during International Baccalaureate (IB) grade awarding to convert marks into grades and ensure fair results for students. Qualitative and quantitative evidence are analysed to determine grade boundaries, with statistical evidence weighed against examiner judgement and teachers' feedback on…

Identifying inattentive respondents in self-administered surveys is a challenging goal for survey researchers. Instructed response items (IRIs) provide a measure for inattentiveness in grid questions that is easy to implement. The present article adds to the sparse research on the use and implementation of attention checks by addressing three…

Evaluation of college instructors often centers on course ratings; however, there is little evidence that these ratings only reflect teaching. The purpose of this study was to assess the relative importance of three facets of course ratings: instructor, course and occasion. We sampled 2,459 fully-crossed dyads from a large university where two…

Researchers seeking to make valid conclusions about engagement for young children with autism spectrum disorder (ASD) must first determine the reliability of estimates obtained across the conditions sampled. Working from that premise, we conducted a secondary data analysis of shared book readings between caregivers and their children with ASD,…

Although a wealth of literature exists studying the effect of sponsor characteristics on self-reports of mental health, little work assesses a related but potentially powerful effect: a context comprehension effect, that is, a change in the respondent's interpretation of a survey question, given the concept elicited by the interviewer. Further,…

This paper argues that the consideration of educational disadvantage should go beyond the micro-scale contextual level of individual students, and explore eventual connections with hybrid forms of disadvantage in the social field. The paper draws on the capability approach and the concept of dwelling to introduce dwelling in time as functioning.…

School socio-economic compositional (SEC) effects have been influential in educational research predicting a range of outcomes and influencing public policy. However, some recent studies have challenged the veracity of SEC effects when applying residualised-change and fixed effects models and simulating potential measurement errors in hierarchical…

Multilevel modeling has grown in use over the years as a way to deal with the nonindependent nature of observations found in clustered data. However, other alternatives to multilevel modeling are available that can account for observations nested within clusters, including the use of Taylor series linearization for variance estimation, the design…

Using data from the Trends in International Mathematics and Science Study (TIMSS) 2007, this study examined the big-fish-little-pond-effects (BFLPEs) in 49 countries. In this study, the effect of math ability on math self-concept was decomposed into a within- and a between-level components using implicit mean centring and the complex data…

The main purpose of this study is to improve estimation efficiency in obtaining maximum marginal likelihood estimates of contextual effects in the framework of nonlinear multilevel latent variable model by adopting the Metropolis-Hastings Robbins-Monro algorithm (MH-RM). Results indicate that the MH-RM algorithm can produce estimates and standard…

Clustered data (e.g., students within schools) are often analyzed in educational research where data are naturally nested. As a consequence, multilevel modeling (MLM) has commonly been used to study the contextual or group-level (e.g., school) effects on individual outcomes. The current study investigates the use of an alternative procedure to…

Background: Assessment grades are "estimates" of ability or performance and there are many reasons why an awarded grade might not meet a candidate's expectations, being either better or poorer than anticipated. Although there may be some obvious reasons for grade discrepancies, such as a lack of preparation or under-performance, there…

In multilevel modeling, group-level variables (L2) for assessing contextual effects are frequently generated by aggregating variables from a lower level (L1). A major problem of contextual analyses in the social sciences is that there is no error-free measurement of constructs. In the present article, 2 types of error occurring in multilevel data…

Classroom context and climate are inherently classroom-level (L2) constructs, but applied researchers sometimes--inappropriately--represent them by student-level (L1) responses in single-level models rather than more appropriate multilevel models. Here we focus on important conceptual issues (distinctions between climate and contextual variables;…

Identifying English language learners (ELLs) with learning disabilities has become very important in education settings so that appropriate educational services can be provided to this group of students. Linguistic diversity may increase the measurement error and reduce the reliability of assessments. This article discusses the issues with…