Although educational research and evaluation generally occur in multilevel settings, many analyses ignore cluster effects. Neglecting the nature of data from educational settings, especially in non-randomized experiments, can result in biased estimates with long-term consequences. Our manuscript improves the availability and understanding of…

Multiple imputation (MI) is a popular method for handling missing data. In education research, it can be challenging to use MI because the data often have a clustered structure that need to be accommodated during MI. Although much research has considered applications of MI in hierarchical data, little is known about its use in cross-classified…

This article discusses estimation of average treatment effects for randomized controlled trials (RCTs) using grouped administrative data to help improve data access. The focus is on design-based estimators, derived using the building blocks of experiments, that are conducive to grouped data for a wide range of RCT designs, including clustered and…

The "Information Systems Education Journal" has published uninterrupted since 2003. Over its publication history, it has covered myriad topics related to information systems education including model curriculum, outcomes assessment, online learning, capstone courses, service learning, data analytics, and cybersecurity, just to name a…

How data are collected and how they are analyzed is typically described in the literature, but how the data are encoded is often not described in detail. In this paper, we discuss how data typically gathered in PER are encoded and how the choice of encoding plays a role in data analysis. We describe the kinds of data that are found when using…

This study aims to reveal the main trends and findings of the studies examining the use of learning analytics in distance education. For this purpose, journal articles indexed in the SSCI index in the Web of Science database were reviewed, and a total of 400 journal articles were analysed within the scope of this study. The systematic review…

This paper argues that methods used for the classification and measurement of online education are not neutral and objective, but involved in the creation of the educational realities they claim to measure. In particular, the paper draws on material semiotics to examine cluster analysis as a 'performative device' that, to a significant extent,…

Multivariate analysis of variance (MANOVA) is widely used in educational research to compare means on multiple dependent variables across groups. Researchers faced with the problem of missing data often use multiple imputation of values in place of the missing observations. This study compares the performance of 2 methods for combining p values in…

The present study makes a situation analysis of graduate theses on values education published between 1999 and 2015 in Turkey. It has a qualitative research design, wherein data is collected through document analysis. The form developed for this purpose is comprised of nine sections, each of which focuses on a different aspect: universities,…

In this paper, we describe multimodal learning analytics (MMLA) techniques to analyze data collected around an interactive learning environment. In a previous study (Schneider & Blikstein, submitted), we designed and evaluated a Tangible User Interface (TUI) where dyads of students were asked to learn about the human hearing system by…

Massive open online courses (MOOCs) are being used across the world to provide millions of learners with access to education. Many who begin these courses complete them successfully, or to their own satisfaction, but the high numbers who do not finish remain a subject of concern. In 2013, a team from Stanford University analyzed engagement…

The use of analytic methods for extracting learning strategies from trace data has attracted considerable attention in the literature. However, there is a paucity of research examining any association between learning strategies extracted from trace data and responses to well-established self-report instruments and performance scores. This paper…

Multilevel models are often used to evaluate hypotheses about relations among constructs when data are nested within clusters (Raudenbush & Bryk, 2002), although alternative approaches are available when analyzing nested data (Binder & Roberts, 2003; Sterba, 2009). The overarching goal of this article is to suggest when it is appropriate…

Missing data is endemic in much educational research. However, practices such as step-wise regression common in the educational research literature have been shown to be dangerous when significant data are missing, and multiple imputation (MI) is generally recommended by statisticians. In this paper, we provide a review of these advances and their…

Higher education for the 21st century continues to promote discoveries in the field through learning analytics (LA). The problem is that the rapid embrace of of LA diverts educators' attention from clearly identifying requirements and implications of using LA in higher education. LA is a promising emerging field, yet higher education stakeholders…