This article describes the development and impact of a faculty development program on improving the readiness of Indonesian educators to teach online. While online learning is seen as a potential solution to increasing access to higher education in Indonesia, most higher education faculty in Indonesia lack the necessary skills to teach online…

This research aimed to investigate and develop an online system for educational measurement and evaluation with a focus on authentic assessment. The research process consisted of four main stages: (1) analyzing the current state and carrying out qualitative interviews on authentic assessment, (2) developing an online measurement and evaluation…

To solve the attenuation paradox in computerized adaptive testing (CAT), this study proposes an item selection method, the integer programming approach based on real-time test data (IPRD), to improve test efficiency. The IPRD method turns information regarding the ability distribution of the population from real-time test data into feasible test…

Assessment and institutional research offices have too much data and too little time. Standard reporting often crowds out opportunities for innovative research. Fortunately, advancements in data science now offer a clear solution. It is equal parts technique and philosophy. The first and easiest step is to modernize data work. This column…

Online calibration is a key technology for item calibration in computerized adaptive testing (CAT) and has been widely used in various forms of CAT, including unidimensional CAT, multidimensional CAT (MCAT), CAT with polytomously scored items, and cognitive diagnostic CAT. However, as multidimensional and polytomous assessment data become more…

Algorithm design is a vital skill developed in most undergraduate Computer Science (CS) programs, but few research studies focus on pedagogy related to algorithms coursework. To understand the work that has been done in the area, we present a systematic survey and literature review of CS Education studies. We search for research that is both…

The appeal of a shorter testing time makes a computer adaptive testing approach highly desirable for use in multiple assessment and learning contexts. However, for those who have been tasked with designing, configuring, and deploying adaptive tests for operational use at scale, preparing an adaptive test is anything but simple. The process often…

The systematic literature review (1) investigates whether 'serious games' provide a viable solution to the limitations posed by traditional high-stakes performance assessments and (2) aims to synthesize game design principles for the game-based performance assessment of professional competencies. In total, 56 publications were included in the…

The process by which user experiences (UX) for children are created is uncertain, especially for deaf and hard of hearing (DHH) children. This paper seeks to (I) describe the origins of UX and child-computer interaction and to describe what is being taught to designers today; (II) use the example of digital assessment to extract insights from…

The purpose of this study is to investigate the quality design criteria for developing a Massive Open Online Course (MOOC). Currently, there are limited studies that highlight the required design criteria for the MOOC programming courses. A descriptive analysis was conducted to examine the characteristics of the three important quality design…

E-Assessment, which is a key element in the instructional design process, plays a major role in supporting and enhancing learning. However, the current e-assessment design does not achieve the desired pedagogical objectives in the e-learning environments. In this paper, we propose a hybrid pedagogical framework for e-learning environments, that…

Using item-response theory to model rater effects provides an alternative solution for rater monitoring and diagnosis, compared to using standard performance metrics. In order to fit such models, the ratings data must be sufficiently connected in order to estimate rater effects. Due to popular rating designs used in large-scale testing scenarios,…

Creativity assessments should be valid, reliable, and scalable to support various stakeholders (e.g., policy-makers, educators, corporations, and the general public) in their decision-making processes. Established initiatives toward scalable creativity assessments have relied on well-studied standardized tests. Although robust in many ways, most…

Effectively engaging learners in an online learning environment is a crucial component of instructional design that contributes to improving performance. Gamification, a contemporary instructional strategy, seeks to integrate game elements such as leaderboards into non-game contexts with the aim of increasing learner engagement and performance.…

International large-scale assessments such as PISA or PIAAC have started to provide public or scientific use files for log data; that is, events, event-related attributes and timestamps of test-takers' interactions with the assessment system. Log data and the process indicators derived from it can be used for many purposes. However, the intended…