This essay describes Rasch analysis psychometric techniques and how such techniques can be used by life sciences education researchers to guide the development and use of surveys and tests. Specifically, Rasch techniques can be used to document and evaluate the measurement functioning of such instruments. Rasch techniques also allow researchers to…

The rise of digital game and simulation-based learning applications has led to new approaches in educational measurement that take account of patterns in time, high resolution paths of action, and clusters of virtual performance artifacts. The new approaches, which depart from traditional statistical analyses, include data mining, machine…

Increasing college participation rates, and diversity in student population, is posing a challenge to colleges in their attempts to facilitate learners achieve their full academic potential. Learning analytics is an evolving discipline with capability for educational data analysis that could enable better understanding of learning process, and…

This dissertation comprises three papers that propose, discuss, and illustrate models to make improved inferences about research questions regarding student achievement in education. Addressing the types of questions common in educational research today requires three different "extensions" to traditional educational assessment: (1)…

There is general agreement that meta-analysis is an important tool for synthesizing study results in quantitative educational research. Yet, a shared feature of many meta-analyses is a failure to report sufficient information for readers to fully judge the reported findings, such as the populations to which generalizations are to be made,…

The Third International Conference on Data Mining (EDM 2010) was held in Pittsburgh, PA, USA. It follows the second conference at the University of Cordoba, Spain, on July 1-3, 2009 and the first edition of the conference held in Montreal in 2008, and a series of workshops within the AAAI, AIED, EC-TEL, ICALT, ITS, and UM conferences. EDM 2011…

This document describes the breadth of the research that the ETS (Educational Testing Service) Research & Development division is conducting in 2010. This portfolio will be updated in early 2011 to reflect changes to existing projects and new projects that were added after this document was completed. The research described in this portfolio falls…

An "Educational Researcher" article by Pamela Moss (1994) asks the title question, "Can there be validity without reliability?" Yes, she answers, if by reliability one means "consistency among independent observations intended as interchangeable" (Moss, 1994, p. 7), quantified by internal consistency indices such as…

This document describes the breadth of the research being conducted in 2008 by the Research and Development Division at Educational Testing Service (ETS). The research described falls into three large categories: (1) Research supported by the ETS research allocation; (2) Research funded by testing programs at ETS; and (3) Research funded by…

A DESCRIPTION IS GIVEN OF ITEM SAMPLING, OR "PSYCHOMETRIC-STATISTICAL INFERENCE," AN APPROACH TO GATHERING AND USING EDUCATIONAL DATA THAT ALLOWS STATISTICAL INFERENCES TO BE MADE SIMULTANEOUSLY WITH PSYCHOMETRIC INFERENCES. THIS IS A PROCEDURE IN WHICH BOTH PEOPLE AND ITEMS ARE SAMPLED AND THE DATA FROM A SAMPLE OF PEOPLE TAKING A SAMPLE OF ITEMS…

This study involved an investigation of the use of Pearson r, tetrachoric r, phi, phi/phi maximum, and Kendall Tau-B coefficients as measures of association for the incomplete principal components analysis of simulated Likert scale attitudinal data, based on a known factor pattern and possessing different types of severe departures from normality.…

The school district fiscal capacity data (1962 and 1967) of the National Finance Project were analyzed for psychometric adequacy and robustness of component composition. The procedures involved: (1) the computation of the Kaiser, Meyer, Olkin Measure of Sampling Adequacy, (2) inspection of the off-diagonal elements of the antiimage covariance…

Recent international studies note that countries whose students perform well on international science assessments report the need to change science education. Some countries use assessments for diagnostic purposes to assist teachers in addressing their students' needs. However, in the United States, standards-based reform has focused the national…