A pilot study was conducted to evaluate and improve the rating procedure proposed for use in a research effort designed to assess the essay writing ability of college sophomores. Generalizability theory and the Many-Facet Rasch Model were each used to (a) estimate potential sources of error in the rating, (b) to obtain reliability estimates, and…

This article is a critique of the article by Robin Perry in this issue that describes a study in Florida to compare supervisor and peer performance evaluations of child welfare staff. An overview of prior studies related to the importance of the social work degree is provided. Subsequently, selected technical, design issues, and results in the…

This article is a formal response to those that authored critiques of the author's research. Each author has provided a thoughtful and critical perspective highlighting the perceived merits and demerits of the research questions posed, theoretical assumptions underlying the inquiry, study design and methodology, and interpretations garnered from…

This paper considers some of the methodological implications of undertaking and representing multimedia and digital, ethnographic work. It explores some of the challenges and opportunities of working with and across a range of media, and explores some of the consequences of bringing hypermedia applications to ethnographic work. The paper draws on…

The assessment literature frequently ignores the quandary of the department resistant to assessment in its discussion of implementing departmental plans and instead focuses on idealized implementation processes. The approach articulated in this article builds on the assessment literature, generating the hypothesis that a grassroots approach to…

This article is a response to "Mapping educational research and its impact on Australian schools," Chapter 2 of The Impact of Educational Research, in which researchers Allyson Holbrook, John Ainley, Sid Bourke, John Owen, Philip McKenzie, Sebastian Mission and Trevor Johnson report on their Commonwealth Education Department commissioned…

Multiple matrix sampling is a data collection technique that ensures accuracy and efficiency in group performance. It has been widely used in large-scale curriculum evaluation since the 1980s. However, the design does not always fully embrace the dynamics of local evaluation demands. The purpose of this study is to introduce a modified matrix…

This article discusses methods for evaluating students' content-area literacy skills. Four specific factors that affect content-area literacy are described: vocabulary knowledge, topic knowledge, text structure knowledge, and textbook readability, along with methods of evaluating each of the factors. Most of these methods have not yet been…

A study is reported of the use of a personal journal as a tool for transferring learning from the training room to the workplace. The targeted learning event was an English for Communication training programme. This programme was an intensive four-day course that grouped together participants from the non-UK sites of the aircraft manufacturer…

Given the perennial interest in the construct of age identity yet its persistent resistance to explanation, the purpose of this study was to re-examine the conceptual structure of age identity by using a focus group design modified to reflect contemporary social science theory. The second purpose of the study was to examine how this modified…

This article reports findings from formative assessments of one state's administrator mentoring and induction program during two years of program piloting in 2002/03 and 2003/04. The purpose of this evaluation research was to establish baseline data and to detect problem areas, so that changes could be made in subsequent years. More broadly, the…

The purpose of this mixed-methods study was to document the prevalence of sampling designs utilised in mixed-methods research and to examine the interpretive consistency between interpretations made in mixed-methods studies and the sampling design used. Classification of studies was based on a two-dimensional mixed-methods sampling model. This…

Content analysis scales apply rigorous measurement to verbal communications and make possible the quantification of text in counseling research. The limitations of the Origin and Pawn Scales (M. T. Westbrook & L. L. Viney, 1980), the Positive Affect Scale (M. T. Westbrook, 1976), the Content Analysis Scales of Psychosocial Maturity (CASPM; L.…

This study provides a framework for the evaluation of assessments that may be used in adult continuing education. It provides an example of the analysis of an examination for 33 solicitors seeking specialist accreditation. Resampling was used to generate a group of 1000 results, and responses were analysed using a Rasch model. Results indicated a…

Patriot High School (PHS) adopted a remediation strategy to help its 10th-grade students at risk of failing the Math portion of MCAS, the state's end of year competency exam. The centerpiece of that strategy was a computer-based instructional (CBI) course. PHS used a commercially available CBI product to align the course content with the…