Large language models are increasingly used in educational and psychological measurement activities. Their rapidly evolving sophistication and ability to detect language semantics make them viable tools to supplement subject matter experts and their reviews of large amounts of text statements, such as educational content standards. This paper…

Traditional approaches to the modeling of multiple-choice item response data (e.g., 3PL, 4PL models) emphasize slips and guesses as random events. In this paper, an item response model is presented that characterizes both disjunctively interacting guessing and conjunctively interacting slipping processes as proficiency-related phenomena. We show…

In this paper, we describe two empirical studies that demonstrate the application and modeling of keystroke logs in writing assessments. We illustrate two different approaches of modeling differences in writing processes: analysis of mean differences in handcrafted theory-driven features and use of large language models to identify stable personal…

The focus of this paper is on the empirical relationship between item difficulty and item discrimination. Two studies--an empirical investigation and a simulation study--were conducted to examine the association between item difficulty and item discrimination under classical test theory and item response theory (IRT), and the effects of the…

Appropriate placement into courses at postsecondary institutions is critical for the success of students in terms of retention and graduation rates. To reduce the number of students who are misplaced, using multiple measures in placing students is encouraged. However, in practice most postsecondary schools utilize only a few measures to determine…

The action sequences of respondents in problem-solving tasks reflect rich and detailed information about their performance, including differences in problem-solving ability, even if item scores are equal. It is therefore not sufficient to infer individual problem-solving skills based solely on item scores. This study is a preliminary attempt to…

Refined tracking allows students and teachers to more accurately understand students' learning growth. To provide refined learning tracking with longitudinal diagnostic assessment, this article proposed a new model by incorporating probabilistic logic into longitudinal diagnostic modeling. Specifically, probabilistic attributes were used instead…

Common response-time-based approaches for non-effortful response behavior (NRB) in educational achievement tests filter responses that are associated with response times below some threshold. These approaches are, however, limited in that they require a binary decision on whether a response is classified as stemming from NRB; thus ignoring…

Learning progressions can reflect students' continuous in-depth thinking development paths, and their establishment is an iterative process from the construction of hypothetical learning progressions to the verification of that hypotheses. Considering the limitations of the existing verification method of learning progressions based on a rule…

Growth-to-standard models evaluate student growth against the growth needed to reach a future standard or target of interest, such as proficiency. A common growth-to-standard model involves comparing the popular Student Growth Percentile (SGP) to Adequate Growth Percentiles (AGPs). AGPs follow from an involved process based on fitting a series of…

Multidimensional scoring evaluates each constructed-response answer from more than one rating dimension and/or trait such as lexicon, organization, and supporting ideas instead of only one holistic score, to help students distinguish between various dimensions of writing quality. In this work, we present a bilevel learning model for combining two…

The Cross-Classified Mixed Effects Model (CCMEM) has been demonstrated to be a flexible framework for evaluating reliability by measurement specialists. Reliability can be estimated based on the variance components of the test scores. Built upon their accomplishment, this study extends the CCMEM to be used for evaluating validity evidence.…

With the emergence of COVID-19 worldwide, interest in blended learning in higher education is rapidly increasing. Despite the fact that the unipolar view that satisfaction and dissatisfaction are measured separately has been emphasized in higher education, fewer efforts have been made in a blended learning context. Therefore, this study attempts…

The internal/external frame of reference (I/E) model describes the formation of students' math and verbal self-concepts by the joint effects of social comparisons (where students compare their subject-specific achievements with those of their classmates) and dimensional comparisons (where students compare their math and verbal achievements with…

The use and interpretation of educational and psychological test scores are paramount to individual outcomes and opportunities. Methods for detecting differential item functioning (DIF) are imperative for item analysis when developing and revising assessments, particularly as it pertains to fairness across populations, languages, and cultures. We…