In any visual search task in the lab or in the world, observers will make errors. Those errors can be categorized as "deterministic": If you miss this target in this display once, you will definitely miss it again. Alternatively, errors can be "stochastic", occurring randomly with some probability from trial to trial.…

In this paper, we propose a new method for selecting cases for in situ, immediate interview research: detector-driven classroom interviewing (DDCI). Published work in educational data mining and learning analytics has yielded highly scalable measures that can detect key aspects of student interaction with computer-based learning in close to…

Computer Aided Detection (CAD) has been used to help readers find cancers in mammograms. Although these automated systems have been shown to help cancer detection when accurate, the presence of CAD also leads to an over-reliance effect where miss errors and false alarms increase when the CAD system fails. Previous research investigated CAD systems…

Question generation (QG) task plays a crucial role in adaptive learning. While significant QG performance advancements are reported, the existing QG studies are still far from practical usage. One point that needs strengthening is to consider the generation of question group, which remains untouched. For forming a question group, intrafactors…

As computer-based testing becomes more prevalent, the attention paid to response time (RT) in assessment practice and psychometric research correspondingly increases. This study explores the rate of Type I error in detecting preknowledge cheating behaviors, the power of the Kullback-Leibler (KL) divergence measure, and the L person fit statistic…

Many students of statistics and econometrics express frustration with the way a problem known as "bad control" is treated in the traditional literature. The issue arises when the addition of a variable to a regression equation produces an unintended discrepancy between the regression coefficient and the effect that the coefficient is…

Although school psychologists often comment on examinee motivation in their reports, systematic evaluation of effort is not common practice. Empirical assessment of performance effort provides critical evidence for the validity of evaluations and will likely lead to more valid assessments, recommendations, and placements. This article focuses on…

Most researchers receive formal training in frequentist statistics during their undergraduate studies. In particular, hypothesis testing is usually rooted on the null hypothesis significance testing paradigm and its p-value. Null hypothesis Bayesian testing and its so-called Bayes factor are now becoming increasingly popular. Although the Bayes…

This memorandum provides information on Advanced Placement (AP) and International Baccalaureate (IB) course enrollment, exam participation, and exam performance during the 2023-2024 school year. During the 2023-2024 school year, 136 AP or IB courses were offered across high schools. Among the 25 comprehensive high schools, all offered AP courses,…

Decisions about assessment are often built on myths about teacher professional judgment and subjectivity that prioritize standardized assessment over classroom assessment. Ken O'Connor and Matt Townsley discuss some of the most common myths and explain how to dispel them by developing clear guidelines in which teachers can exercise their judgment,…

Current methods for evaluating faculty teaching fall short, and one way to address this is through campus-wide initiatives that focus on change at the level of academic units. The complex context of higher education makes meaningful teaching evaluation difficult; in particular, four sobering realities of this context must be taken into account in…

Many institutions use placement tests as a method to assess students' readiness for college-level coursework. With the increased use of technology in testing, many institutions have transitioned placement test administration to an online format in an unproctored setting. While unproctored placement tests may provide financial and logistical…

Testing programs have explored the use of technology-enhanced items alongside traditional item types (e.g., multiple-choice and constructed-response items) as measurement evidence of latent constructs modeled with item response theory (IRT). In this report, we discuss considerations in applying IRT models to a particular type of adaptive testlet…

English learners (ELs) comprise approximately 10% of kindergarten to Grade 12 students in US public schools, with about 15% of ELs identified as having disabilities. English language proficiency (ELP) assessments must adhere to universal design principles and incorporate universal tools, designed to increase accessibility for all ELs, including…

Designing a multidimensional adaptive test (M-MST) based on a multidimensional item response theory (MIRT) model is critical to make full use of the advantages of both MST and MIRT in implementing multidimensional assessments. This study proposed two types of automated test assembly (ATA) algorithms and one set of routing rules that can facilitate…