The fields of language testing and educational and psychological measurement have not, as yet, developed a set of principles and procedures for linking test scores and score-based inferences to test use and the consequences of test use. Although Messick (1989) discusses test use and consequences, his framework provides virtually no guidance on how…

In this article, we report on our qualitative study involving eight individuals hired to transcribe research tapes in university contexts. We consider issues of data analysis and data trustworthiness and the implications for both when transcription is assigned to someone other than the researcher. We explore the challenges transcribers faced…

This paper continues an exchange between its author and Andrew Davis. Part I addresses the attribution and ontological status of mental constructs and argues that philosophical work on these topics does not undermine high stakes testing. Part II examines the significance for testing of the connectedness of meaningful learning. Part III addresses…

Two methods of assessing preference for stimuli (i.e., potential reinforcers) were compared for adult administrative assistant employees. During Phase 1, a survey method and a verbal choice method of assessing preference for 6 stimuli were administered. During Phase 2, a coupon system was used to determine which categories of stimuli actually…

A structural equation modeling approach to scale reliability evaluation can be employed to estimate generalizability theory indexes in settings where sampling of subjects and conditions is carried out. In one- and two-facet crossed designs, it is demonstrated how this method can be used to obtain estimates of relative generalizability…

Seven approaches to averaging reliability coefficients are presented. Each approach starts with a unique definition of the concept of "average," and no approach is more correct than the others. Six of the approaches are applicable to internal consistency coefficients. The seventh approach is specific to alternate-forms coefficients. Although the…

Direct observation plays an important role in the assessment practices of school psychologists and in the development of evidence-based practices in general and special education. The defining psychometric features of direct observation are presented, the contributions to assessment practice reviewed, and a specific proposal is offered for…

Transcript analysis is an important methodology to study asynchronous online educational discourse. The purpose of this study is to revisit reliability and validity issues associated with transcript analysis. The goal is to provide researchers with guidance in coding transcripts. For validity reasons, it is suggested that the first step is to…

A study was conducted to assess the equivalence of validity and reliability of two forms of the Self-Esteem Rating Scale. A total of 228 responses were obtained from a purposive sample. Several data analysis methods were used to test specific hypotheses, and two methods of equating observed scores on the two forms were used. The results were…

This article offers an overview of learning styles theories, selects seven to consider in more detail on the basis of recent research into validity and reliability, and synthesises these theories into a framework to aid task design in Teacher Education.

The reliability of a laser system was compared with the reliability of a video-based kinematic analysis in measuring displacement and velocity during running. Validity and reliability of the laser on static measures was also assessed at distances between 10 m and 70 m by evaluating the coefficient of variation and intraclass correlation…

This study focuses on six leading communication journals and their use of online citations in articles published between 2000 and 2003. The study uses content analysis to explore if there is a relationship between citation characteristics and their stability. The findings show that online citations in the .gov and .org domains are more likely to…

Response-to-intervention (RTI) technical adequacy standards should follow from model purpose, procedural specification, procedural adherence, outcome determination, and subsequent plans. Therefore, RTI raises atypical measurement questions for practice, and, for this reason, it may require hybridized technical adequacy methods. Due to RTI model…

In this series of studies, we investigated the technical adequacy of three curriculum-based measures used as benchmarks and for monitoring progress in three critical reading- related skills: fluency, reading comprehension, and vocabulary. In particular, we examined the following easyCBM measurement across grades 3-7 at fall, winter, and spring…

The Bologna Declaration brought reforms into higher education that imply changes in teaching methods, didactic materials and textbooks, infrastructures and laboratories, etc. Statistics and mathematics are disciplines that traditionally have the worst success rates, particularly in non-mathematics core curricula courses. This research project,…