Data extraction is a crucial, yet labor-intensive and error-prone part of evidence synthesis. To date, efforts to harness machine learning for enhancing efficiency of the data extraction process have fallen short of achieving sufficient accuracy and usability. With the release of large language models (LLMs), new possibilities have emerged to…

This study analyzed the fit of data collected with the Measure of Engagement, Independence, and Social Relationships for 3- to 5-year-olds (MEISR 3-to-5-years-old) to a proposed theoretical model based on the cross-walk of MEISR 3-to-5-years-old items and codes from 7 chapters of the Activities and Participation component of the International…

There can be many reasons why students fail to answer correctly to summative tests in advanced computer science courses: often the cause is a lack of prerequisites or misconceptions about topics presented in previous courses. One of the ITiCSE 2020 working groups investigated the possibility of designing assessments suitable for differentiating…

Intensive longitudinal data is increasingly used to study state-like processes such as changes in daily stress. Measures aimed at collecting such data require the same level of scrutiny regarding scale reliability as traditional questionnaires. The most prevalent methods used to assess reliability of intensive longitudinal measures are based on…

Purpose: Overall quantitative research aims to observe certain fundamental principles of logic and scientific frame of reasoning. There continues to be challenges on how quantitative research is conducted in the field of information systems. Design/methodology/approach: Structured equation modelling (SEM) research identifies concerns about the…

Monitoring applications (MAs) use digital and online tools to collect and track data on student behavior, and they have become increasingly popular among schools. Empirical research on these complex surveillance platforms is scant, and little is known about the efficacy or impact that they have on students. This study used a multi-method…

K-12 Engineering Education has placed a lot of attention on students' attitudes or predispositions towards science and technology. However, most assessment methods are focused on STEM as a whole or only on technology. In this article, we will discuss the instrument called Technology and Engineering Attitude Scale (TEAS) which focuses on attitudes…

For investigating a comprehensive PLC framework, instruments are needed that capture the multi-layered PLC characteristics and that take into account the complex influencing educational context. Such instruments are currently lacking. This study aims at describing the development and validation of two qualitative classification instruments usable…

In undergraduate research settings, students are likely to encounter anomalous data, that is, data that do not meet their expectations. Most of the research that directly or indirectly captures the role of anomalous data in research settings uses post-hoc reflective interviews or surveys. These data collection approaches focus on recall of past…

This study aims to produce a more personalized recommendation system for online learning using multicriteria in collaborative filtering and data from the Binus Online Learning repository as a knowledge base. The study uses forecasting (regression) and consists of three stages: (1) collecting data on the results of the learning process; (2) adding…

Digital literacy is essential for scientific literacy in a digital world. Although the NGSS Practices include many activities that require digital literacy, most studies have examined digital literacy from a generic perspective rather than a curricular context. This study aimed to develop a self-report tool to measure elements of digital literacy…

In this ITEMS module, we provide a two-part introduction to the topic of reliability from the perspective of "classical test theory" (CTT). In the first part, which is directed primarily at beginning learners, we review and build on the content presented in the original didactic ITEMS article by Traub and Rowley (1991). Specifically, we…

The purpose of this study is to use Indigenous data collection to present construct validity of an instrument designed to test the American Indian/Alaska Native Millennium Falcon Postsecondary Persistence Model (Lopez, 2018). In the following, I describe an alternative sampling technique based on an Indigenous quantitative methodology to examine…

In a world of burgeoning societal issues, future scientists must be equipped to work interdisciplinarily to address real-world problems. To train undergraduate students toward this end, practitioners must also have quality assessment tools to measure students' ability to think within an interdisciplinary system. There is, however, a dearth of…

Currently, there is significant interest being directed towards the development of STEM education to meet economic and societal demands. While economic concerns can be a powerful driving force in advancing the STEM agenda, care must be taken that such economic imperative does not promote research approaches that overemphasize pragmatic application…