This dissertation consists of three manuscripts. The manuscripts contribute to a budding "methodological reform" currently taking place in quantitative second-language (L2) research. In the first manuscript, the researcher describes an empirical investigation on the application of two well-known effect size estimators, eta-squared (eta…

In the e-learning context, social network analysis (SNA) can be used to build understanding around the ways students participate and interact in online forums. This study contributes to the growing body of research that uses statistical methods to test hypotheses about structures in social networks. Specifically, we show how statistical analysis…

Empirically supported multimedia learning (MML) principles [1] suggest effective ways to design instruction, generally for elements on the order of a graphic or an activity. We examined whether the positive impact of MML could be detected in larger instructional units from a MOOC. We coded instructional design (ID) features corresponding to MML…

Purpose: The purpose of this paper is to explore teachers' noncompliance with secondary-level standardised literacy testing in Tasmania, Australia, particularly their motivations, behaviours and justifications. This paper challenges pervasive views regarding test noncompliance, suggesting a reframing as "advocacy cheating": noncompliance…

The current studies test the hypothesis that the financial burden of college can initiate a psychological process that has a negative influence on academic performance for students at selective colleges and universities. Prior studies linking high college costs and student loans to academic outcomes have not been grounded within relevant social…

The current studies test the hypothesis that the financial burden of college can initiate a psychological process that has a negative influence on academic performance for students at selective colleges and universities. Prior studies linking high college costs and student loans to academic outcomes have not been grounded within relevant social…

Continuously administered examination programs, particularly credentialing programs that require graduation from educational programs, often experience seasonality where distributions of examine ability may differ over time. Such seasonality may affect the quality of important statistical processes, such as item response theory (IRT) item…

The equating of tests is an essential process in high-stakes, large-scale testing conducted over multiple forms or administrations. By adjusting for differences in difficulty and placing scores from different administrations of a test on a common scale, equating allows scores from these different forms and administrations to be directly compared…

Examinees' thinking processes have become an increasingly important concern in testing. The responses processes aspect is a major component of validity, and contemporary tests increasingly involve specifications about the cognitive complexity of examinees' response processes. Yet, empirical research findings on examinees' cognitive processes are…

Extracting information efficiently from game/simulation-based assessment (G/SBA) logs requires two things: a well-structured log file and a set of analysis methods. In this report, we propose a generic data model specified as an extensible markup language (XML) schema for the log files of G/SBAs. We also propose a set of analysis methods for…

The need to build an evidence base for reflective supervision (RS) is threefold: (1) to determine the elements of a supervision session that make it reflective, (2) to demonstrate that change occurs within the supervisee, and (3) to demonstrate that having staff members participate in RS enhances positive program (or client) outcomes. This article…

Although the results of the final examination in Indonesia were the dominant factor in determining high school graduation, the public still did not know how the examination was administered nor how results were used to determine student graduation. Despite its importance, no comprehensive studies about its implementation have been conducted. The…

The first aim of this study was to develop a computer adaptive EFL test (CALT) that assesses test takers' listening and reading proficiency in English with dichotomous items and polytomous testlets. We reported in detail on the development of the CALT, including item banking, determination of suitable item response theory (IRT) models for item…

The present study tests whether presenting video modeling examples from the learner's (first-person) perspective promotes learning of an assembly task, compared to presenting video examples from a third-person perspective. Across 2 experiments conducted in different labs, university students viewed a video showing how to assemble an 8-component…

While Audience Response System (ARS) such as Kahoot! can promote learner self-regulation and enhance comprehension by engaging cognitive processing on multiple levels, learning behavior in questioning with ARS was understudied. The present study explores the use of Kahoot! with student-generated questioning, to report the effect on students'…