This paper describes the development and efficacy of an online tool for assessing the numeracy of undergraduate students. The tool was designed to be easy to administer, provide immediate feedback to students on whether they had the required level of numeracy, and to be consistent with other measures of adult numeracy. When used with students…

Empirically supported multimedia learning (MML) principles [1] suggest effective ways to design instruction, generally for elements on the order of a graphic or an activity. We examined whether the positive impact of MML could be detected in larger instructional units from a MOOC. We coded instructional design (ID) features corresponding to MML…

This paper examines the impact of the informational value of feedback choices on students' performance, their choice to revise, and the time they spend designing posters and reading feedback in an assessment game. Choices to seek confirmatory or critical feedback and to revise posters in a poster design task were collected from a hundred and six…

This paper presents a novel examination of the impact of students' feedback choices and performance on their feedback memory. An empirical study was designed to collect the choices to seek critical feedback from a hundred and six Grade 8 middle-school students via Posterlet, a digital assessment game in which students design posters. Upon…

No significant body of research examines writing achievement and the specific skills and knowledge in the writing domain for postsecondary (college) students in the U.S., even though many at-risk students lack the prerequisite writing skills required to persist in their education. This paper addresses this gap through a novel…

This paper briefly discusses four measurement challenges of data science or "big data" in educational assessments that are enabled by technology: 1. Dealing with change over time via time-based data. 2. How a digital performance space's relationships interact with learner actions, communications and products. 3. How layers of…

Prerequisite skill structures have been closely studied in past years leading to many data-intensive methods aimed at refining such structures. While many of these proposed methods have yielded success, defining and refining hierarchies of skill relationships are often difficult tasks. The relationship between skills in a graph could either be…

The present study aims to demonstrate how the estimation of vocabulary size might be affected by two neglected factors in vocabulary size tests. The first factor is randomization of question sequence, as opposed to the traditional high-to-low frequency sequencing. The second factor is learners' confidence in choosing the correct meaning for a…

This paper presents the online cognitive and algorithm tests, which were developed in order to determine if certain cognitive factors and fundamental algorithms correlate with the performance of students in their introductory computer science course. The tests were implemented among Management Information Systems majors from the Philippines and…

Content balancing is one of the most important components in the computerized adaptive testing (CAT) especially in the K to 12 large scale tests that complex constraint structure is required to cover a broad spectrum of content. The purpose of this study is to compare the weighted penalty model (WPM) and the weighted deviation method (WDM) under…

The effective design of test items within a computer-based language test (CBLT) for developing English as a foreign language (EFL) learners' listening and speaking skills has become an increasingly challenging task for both test users and test designers compared with that of pencil-and-paper tests in the past. It needs to fit integrated oral…

This paper provides a discussion of the results of a cross-sectional examination of linguistic variables that are predicted to influence L2 reading fluency. This study is part of a larger, longitudinal mixed-methods study into reading fluency development using online Timed Reading (TR) with participants from a mid-to-high level private university…

Multivariate normality is required for some statistical tests. This paper explores the implications of violating the assumption of multivariate normality and illustrates a graphical procedure for evaluating multivariate normality. The logic for using the multivariate bootstrap is presented. The multivariate bootstrap can be used when distribution…

When low-stakes assessments are administered to examinees, the degree to which examinees give their best effort is often unclear, complicating the validity and interpretation of the resulting test scores. This study introduces a new method for measuring examinee test-taking effort on computer-based test items based on item response time. This…

Three descriptive statistics (difficulty, discrimination, and reliability) of multiple-choice (MC) test items were compared to those of a new (1980s) format of machine-scored questions. The new method, answer-bank multi-digit testing (MDT), uses alphabetized lists of up to 1,000 alternatives and approximates the completion style of assessment…