In an article published in an open-access journal, (Pennebaker et al. "PLoS One," 8(11), e79774, 2013) reported that an innovative computer-based system that included daily online testing resulted in better student performance in other concurrent courses and a reduction in achievement gaps between lower and upper middle-class students.…

The emergence of sophisticated AI tools such as ChatGPT, coupled with the transition to remote delivery of educational assessments in the COVID-19 era, has led to increasing concerns about academic integrity and test security. Using AI tools, test takers can produce high-quality texts effortlessly and use them to game assessments. It is thus…

Questionnaires are by far the most common tool for measuring noncognitive constructs in psychology and educational sciences. Response bias may pose an additional source of variation between respondents that threatens validity of conclusions drawn from questionnaire data. We present a mixture modeling approach that leverages response time data from…

In automated essay scoring (AES), scores are automatically assigned to essays as an alternative to grading by humans. Traditional AES typically relies on handcrafted features, whereas recent studies have proposed AES models based on deep neural networks to obviate the need for feature engineering. Those AES models generally require training on a…

Scoring of teachers' noticing responses is typically burdened with rater bias and reliance upon interrater consensus. The authors sought to make the scoring process more objective, equitable, and generalizable. The development process began with a description of response characteristics for each professional noticing component disconnected from…

In this study we investigate the abilities to determine credibility of digital news among 532 teenagers. Using an online test we assess to what extent teenagers are able to determine the credibility of different sources, evaluate credible and biased uses of evidence, and corroborate information. Many respondents fail to identify the credibility of…

Measurement errors attributable to cultural issues are complex and challenging for educational assessments. We need assessment tests sensitive to the cultural heterogeneity of populations, and psychometric methods appropriate to address fairness and equity concerns. Built on the research of culturally responsive assessment, this dissertation…

This study employed a mixed-design approach and the Many-Facet Rasch Measurement (MFRM) framework to investigate whether rater bias occurred between the onscreen scoring (OSS) mode and the paper-based scoring (PBS) mode. Nine human raters analytically marked scanned scripts and paper scripts using a six-category (i.e., six-criterion) rating…

This study examined whether different identity revelation conditions result in different online targeting behavior among peer-assessors through a pretest and posttest quasi-experimental research design. Students from six fifth-grade classes (N = 196) participated in online learning tasks where they generated and selected peer-generated questions…

Stereotypes, expectations, and emotions influence an observer's ability to detect and categorize objects as guns. In light of recent work in action-perception interactions, however, there is another unexplored factor that may be critical: The action choices available to the perceiver. In five experiments, participants determined whether another…

In a test, the testing score would be closer to examinee's actual ability when careless mistakes were corrected. In CAT, however, changing the answer of one item in CAT might cause the following items no longer appropriate for estimating the examinee's ability. These inappropriate items in a reviewable CAT might in turn introduce bias in ability…

Imposed time limits on computer adaptive tests (CATs) can result in examinees having difficulty completing all items, thus compromising the validity and reliability of ability estimates. In this study, the effects of speededness were explored in a simulated CAT environment by varying examinee response patterns to end-of-test items. Expectedly,…

This book compiles several essays about effective evaluation of teaching. Contents of this publication include: (1) Conducting Research on Student Evaluations of Teaching (William E. Addison and Jeffrey R. Stowell); (2) Choosing an Instrument for Student Evaluation of Instruction (Jared W. Keeley); (3) Formative Teaching Evaluations: Is Student…

This study was designed to examine response rates and bias among a sample of community college students who received a district-wide survey by standard mail or e-mail. Findings suggest that predictors of response and types of responses are not appreciably different across paper and online mail-out samples when these samples are "matched" in terms…

This study examined the construct validity of the "e-rater"® automated essay scoring engine as an alternative to human scoring in the context of TOEFL® essay writing. Analyses were based on a sample of students who repeated the TOEFL within a short time period. Two "e-rater" scores were investigated in this study, the first…