This study highlights the role and importance of advanced, machine learning-driven predictive models in enhancing the accuracy and timeliness of identifying students at-risk of negative academic outcomes in data-driven Early Warning Systems (EWS). K-12 school districts have, at best, 13 years to prepare students for adulthood and success. They…

The multiple-choice (MC) item format has been widely used in educational assessments across diverse content domains. MC items purportedly allow for collecting richer diagnostic information. The effectiveness and economy of administering MC items may have further contributed to their popularity not just in educational assessment. The MC item format…

As assessments move towards computerized testing and making continuous testing available the need for rapid assembly of forms is increasing. The objective of this study was to investigate variability in assembled forms through the lens of first- and second-order equity properties of equating, by examining three factors and their interactions. Two…

The emergence of digital-first assessments is prompting reconsideration of, and innovation in, aspects of psychometrics, test validation, and test use. Using the Duolingo English Test (DET) as an example, this three-paper series seeks to address issues concerning the estimation of classification consistency and the reporting of results for such…

In educational measurement, Item Response Theory (IRT) provides a means of quantifying student knowledge. Specifically, IRT models the probability of a student answering a particular item correctly as a function of the student's continuous-valued latent abilities [theta] (e.g. add, subtract, multiply, divide) and parameters associated with the…

This dissertation describes an experiment to investigate how learners with different levels of background in computer science learn core concepts of computer science, in particular, algorithms. We designed a study to focus on cognitive task analysis for eliciting the empirical mental elements of learning two graph algorithms. Cognitive workload…

To meet the needs of complex and/or underserved patient populations, health care professionals must possess diverse backgrounds, qualities, and skill sets. Holistic review has been used to diversify student admissions through examination of non-cognitive attributes of health care applicants. The objective of this study was to develop a novel…

Curriculum Based Measurements (CBM) are a widely-used tool for Response to Intervention (RTI) progress monitoring. In addition, they can be used in the determination of learning disabilities and special education qualification. The most widely used type of CBM is a measure of oral reading fluency (ORF). This type involves having a student read out…

Divergent thinking (DT) tests are the most frequently used types of creativity assessment and have been administered in traditional paper and pencil format for more than a half century. With the prevalence of computer-based testing and increasing demands for large-scale, faster, and more flexible testing procedures, it is necessary to explore and…

This study is part of a larger design study that iteratively improves a robotics programming curriculum as well as a computational thinking (CT) instrument. Its focus was majorly on CT assessment and particularly on an online CT instrument with logging functionality that can store a student's problem-solving process by recording interactions…

Compressed time from matriculation to graduation prevalent in executive graduate degree programs means that traditional methods of identifying deficiencies in student writing during the course of study may not provide timely remediation enabling for student success. This study examined a writing-intensive, 10-month executive graduate degree…

Emotion plays an ever-present role in human existence, impacting nearly every behavior and decision in some way. Research in the behavioral sciences is rife with exploration of emotion and the role it plays in everything from business decision making to health-related behavior. However, affect, the feeling or experience of emotion, is complex and…

Speaking performance tests typically employ raters to produce scores; accordingly, variability in raters' scoring decisions has important consequences for test reliability and validity. One such source of variability is the rater's level of expertise in scoring. Therefore, it is important to understand how raters' performance is influenced by…

A renewed focus on foreign language (FL) learning and speech for communication has resulted in computer-assisted language learning (CALL) software developed with Automatic Speech Recognition (ASR). ASR features for FL pronunciation (Lafford, 2004) are functional components of CALL designs used for FL teaching and learning. The ASR features…

This dissertation reports development and validation studies of a Web-based standardized test of Japanese as a foreign language (JFL), designed to measure learners' off-line grammatical and pragmatic knowledge in multiple-choice format. Targeting Japanese majors in the U.S. universities and colleges, the test is designed to explore possible…