Rating scale analysis techniques provide researchers with practical tools for examining the degree to which ordinal rating scales (e.g., Likert-type scales or performance assessment rating scales) function in psychometrically useful ways. When rating scales function as expected, researchers can interpret ratings in the intended direction (i.e.,…

The New York State Education Department (NYSED) has a partnership with Questar Assessment Inc. (Questar) for the online delivery of the 2022 Elementary-Level (Grade 5) and Intermediate-Level (Grade 8) Science Computer-Based Field Tests. Teachers from across the State work with NYSED in a variety of activities to ensure the validity and reliability…

The purpose of this research is to create a personalized system called CARIA that suggests career recommendations based on students' competencies and the required skills in each career. The focus of this study is on digital technology and digital media careers. The personalized career recommender system uses a novel similarity measure called…

Aim/Purpose: This study examined the experiences of international students writing a typed e-assessment at the on-campus computer lab on the ExamOnline system. Background: After a computer-based typed summative examination was introduced for one of the MSc degree's modules in a UK university, it was critical to ascertain its inclusivity. The…

In any visual search task in the lab or in the world, observers will make errors. Those errors can be categorized as "deterministic": If you miss this target in this display once, you will definitely miss it again. Alternatively, errors can be "stochastic", occurring randomly with some probability from trial to trial.…

In this paper, we propose a new method for selecting cases for in situ, immediate interview research: detector-driven classroom interviewing (DDCI). Published work in educational data mining and learning analytics has yielded highly scalable measures that can detect key aspects of student interaction with computer-based learning in close to…

Computer Aided Detection (CAD) has been used to help readers find cancers in mammograms. Although these automated systems have been shown to help cancer detection when accurate, the presence of CAD also leads to an over-reliance effect where miss errors and false alarms increase when the CAD system fails. Previous research investigated CAD systems…

Question generation (QG) task plays a crucial role in adaptive learning. While significant QG performance advancements are reported, the existing QG studies are still far from practical usage. One point that needs strengthening is to consider the generation of question group, which remains untouched. For forming a question group, intrafactors…

As computer-based testing becomes more prevalent, the attention paid to response time (RT) in assessment practice and psychometric research correspondingly increases. This study explores the rate of Type I error in detecting preknowledge cheating behaviors, the power of the Kullback-Leibler (KL) divergence measure, and the L person fit statistic…

Many students of statistics and econometrics express frustration with the way a problem known as "bad control" is treated in the traditional literature. The issue arises when the addition of a variable to a regression equation produces an unintended discrepancy between the regression coefficient and the effect that the coefficient is…

Although school psychologists often comment on examinee motivation in their reports, systematic evaluation of effort is not common practice. Empirical assessment of performance effort provides critical evidence for the validity of evaluations and will likely lead to more valid assessments, recommendations, and placements. This article focuses on…

Decisions about assessment are often built on myths about teacher professional judgment and subjectivity that prioritize standardized assessment over classroom assessment. Ken O'Connor and Matt Townsley discuss some of the most common myths and explain how to dispel them by developing clear guidelines in which teachers can exercise their judgment,…

Current methods for evaluating faculty teaching fall short, and one way to address this is through campus-wide initiatives that focus on change at the level of academic units. The complex context of higher education makes meaningful teaching evaluation difficult; in particular, four sobering realities of this context must be taken into account in…

Most researchers receive formal training in frequentist statistics during their undergraduate studies. In particular, hypothesis testing is usually rooted on the null hypothesis significance testing paradigm and its p-value. Null hypothesis Bayesian testing and its so-called Bayes factor are now becoming increasingly popular. Although the Bayes…

This memorandum provides information on Advanced Placement (AP) and International Baccalaureate (IB) course enrollment, exam participation, and exam performance during the 2023-2024 school year. During the 2023-2024 school year, 136 AP or IB courses were offered across high schools. Among the 25 comprehensive high schools, all offered AP courses,…