Education Matters' working hypothesis in this report was that the quality of implementation of ILT (Instructional Leadership Teams) and LASW (Looking At Student Work) groups is related to the quality of implementation of other components of reform. Specifically, the authors anticipated that schools in which ILTs and LASW groups were…

This paper describes the design rationale for a prototype of an innovative assessment product, and the process that led to the design. The goals of the Biomass project were to demonstrate: (1) an assessment product designed to serve two new purposes in the transition from high school to college; and (2) the capability needed to produce this kind…

An introduction is provided to the use of confirmatory factor analysis to test measurement invariance and stability in longitudinal research. The approach is illustrated through examples representing one or two constructs in one to three measurement waves. Basic issues in establishing measurement invariance are discussed. (SLD)

Evaluated a peer review process in a graduate instructional design course. Discusses influences on the effectiveness of peer review and proposes peer review as an authentic and efficient means to introduce graduate students to the strategic knowledge needed to apply instructional design skills. (Author/LRW)

Federal policy tools for gathering evidence on "What Works" in education, such as the What Works Clearinghouse's (WWC) standards, emphasize randomized field trials as the preferred method for generating scientific evidence on the effectiveness of educational programs. This article argues instead for extended-term mixed-method (ETMM) designs.…

This technical report and user's guide is designed to provide researchers with an overview of the design and implementation of the 2009 Program for International Student Assessment (PISA), as well as with information on how to access the PISA 2009 data. This information is meant to supplement that presented in Organization for Economic Cooperation…

The paper discussed the difficulties and problems in experimental researches in educational technology, such as misunderstanding in teachers' concept, disjointedness between theory and practice, inadequate understanding of dialectics in experimental educational technology researches, research project selection and theoretical hypothesis formation…

This article reviews research on differential item functioning (DIF) in language testing conducted primarily between 1990 and 2005 with an eye toward providing methodological guidelines for developing, conducting, and disseminating research in this area. The article contains a synthesis of 27 studies with respect to five essential sets of…

Focus group interview is a scientific qualitative research method, which has been extensively applied in academic areas over the past 20 years. However, this method is mostly applied in business studies. It is seldom adopted in other academic areas. Therefore, this study attempts to introduce this method in a systematic approach. This study…

Perhaps surprisingly, many social workers appear to use intercessory prayer in direct practice settings. To help inform practitioners' use of this intervention, this article evaluates the empirical literature on the topic using the following three methods: (a) an individual assessment of each study, (b) an evaluation of intercessory prayer as an…

The authors examined the methodologies of articles in teaching-and-learning research journals, published in 1994 and in 2004, and classified them as either intervention (based on researcher-manipulated variables) or nonintervention. Consistent with the findings of Hsieh et al., intervention research articles declined from 45% in 1994 to 33% in…

In "Utilization Focused Evaluation" (1997), Patton introduced the term "process use" to describe changes in thinking and behavior, whether at the individual, program, or organizational level, as a result of one's participation in an evaluation, irrespective of the evaluation results. In conversations with stakeholders, Patton found that although…

An in-depth understanding of human factors and human error is lacking in current research on seniors' falls. Additional knowledge is needed to understand why seniors are falling. The purpose of this article is to describe the adapting of the Integrated Safety Investigation Methodology (ISIM) (used for investigating transportation and industrial…

The purpose of this study was to construct and evaluate an instrument for determining student preparedness in College Algebra. A 73-item instrument covering prerequisite arithmetic and high school Algebra knowledge for College Algebra was constructed. The instrument was pilot-tested on a freshman population of 595 students. Results of reliability…

Presented are the results of a study using latent profile analysis to describe the self-reported practices of 138 evaluators. Four classes emerged and were labeled (a) indistinct pattern of practice, (b) method-focused, (c) user-focused, and (d) robust pattern of practice. Evaluators in the "indistinct pattern of practice" class had mean item…