This article provides a conceptual framework for considering both the theoretical and methodological factors that underpin the successful validation of a competency framework. Drawing on educational assessment literature, this article argues that a valid competency framework relates to an interpretive judgement of the credibility of the claims…

This paper addresses a relatively unexplored area in the field of learning analytics: how analytics are taken up and used as part of teaching and learning processes. Initial steps are taken towards developing design knowledge for this "middle space," with a focus on students as analytics users. First, a core set of challenges for…

With the widespread use of learning analytics (LA) tools, there is a need to explore how these technologies can be used to enhance teaching and learning. Little research has been conducted on what human processes are necessary to facilitate meaningful adoption of LA. The research problem is that there is a lack of evidence-based guidance on how…

We are using stealth assessment, embedded in "Plants vs. Zombies 2," to measure middle-school students' problem solving skills. This project started by developing a problem solving competency model based on a thorough review of the literature. Next, we identified relevant in-game indicators that would provide evidence about students'…

We present a method for assessing science inquiry performance, specifically for the inquiry skill of designing and conducting experiments, using educational data mining on students' log data from online microworlds in the Inq-ITS system (Inquiry Intelligent Tutoring System; www.inq-its.org). In our approach, we use a 2-step process: First we use…

The goal of this research which was carried out in reputable dedicated call centres within the Turkish telecommunication sector aims is to evaluate competence-based curriculums designed by means of internal funding through Stufflebeam's context, input, process, product (CIPP) model. In the research, a general scanning pattern in the scope of…

"Game-Based Assessments: A Promising Way to Create Idiographic Perspectives" (Adrienne Walker and George Englehard) comments on: "How Task Features Impact Evidence from Assessments Embedded in Simulations and Games" by Russell G. Almond, Yoon Jeon Kim, Gertrudes Velasquez, and Valerie J. Shute. Here, Walker and Englehard write…

Information and communication technology (ICT)-enhanced research methods such as educational data mining (EDM) have allowed researchers to effectively model a broad range of constructs pertaining to the student, moving from traditional assessments of knowledge to assessment of engagement, meta-cognition, strategy and affect. The automated…

Student models for adaptive systems may not model collaborative learning optimally. Past research has either focused on modeling individual learning or for collaboration, has focused on group dynamics or group processes without predicting learning. In the current paper, we adjust the Additive Factors Model (AFM), a standard logistic regression…

Student models for adaptive systems may not model collaborative learning optimally. Past research has either focused on modeling individual learning or for collaboration, has focused on group dynamics or group processes without predicting learning. In the current paper, we adjust the Additive Factors Model (AFM), a standard logistic regression…

This article discusses the role of assessment in a response-to-intervention model. Although assessment represents only 1 component in a response-to-intervention model, a well-articulated assessment system is critical in providing teachers with reliable data that are easily interpreted and used to make instructional decisions. Three components of…

When validating assessment models built with data mining, generalization is typically tested at the student-level, where models are tested on new students. This approach, though, may fail to find cases where model performance suffers if other aspects of those cases relevant to prediction are not well represented. We explore this here by testing if…

This article focuses on school self-evaluations (SSE). It addresses whether SSE meets quality indicators and whether differences can be found between the quality judgments of school principals and inspectors. Data stem from two complementary data collections: population data of school inspections and a survey of a representative sample of school…

[Slides] presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation describes how the vehicles for gathering student evidence--task models and test specifications--are developed.

This paper describes the confirmatory factor analysis, validity, and reliability data collection stage of the development of a scale to measure mainstream teachers' self-efficacy beliefs for teaching ELL (English Language Learner) students. Data were collected from 708 K through 12 teachers and pre-service teachers with varying degrees of…