The benchmarking toolkit for Technology-Enabled Learning (TEL) developed by the Commonwealth of Learning (COL) is designed to assess TEL practices in higher education institutions. This study evaluated the content validity, internal consistency, and inter-domain relationships of the toolkit using a survey of 355 practitioners across 21…

Educational researchers often need to construct precise and reliable measurement scales of complex and varied representations such as participants' written work, videoed lesson segments and policy documents. Developing such scales using can be resource-intensive and time-consuming, and the outcomes are not always reliable. Here we present…

This research compares the performance measures reported by two bibliographic databases relevant to a set of authors who have published in predatory journals. The reliability of decision-making based on the information provided by uncontrolled bibliographic databases is examined to support rational decisions. A sample of authors who published in…

Introduction: Research in various areas indicates that expert judgment can be highly inconsistent. However, expert judgment is indispensable in many contexts. In medical education, experts often function as examiners in rater-based assessments. Here, disagreement between examiners can have far-reaching consequences. The literature suggests that…

The "LFK" index has been promoted as an improved method to detect bias in meta-analysis. Putatively, its performance does not depend on the number of studies in the meta-analysis. We conducted a simulation study, comparing the "LFK" index test to three standard tests for funnel plot asymmetry in settings with smaller or larger…

In the early years, it has become essential to support the acquisition of computational thinking, which is seen as a 21st-century skill and new literacy. A valid and reliable measurement tool is needed to develop and evaluate educational practices related to these skills. "TechCheck" is a validated unplugged assessment of computational…

The authors contend that children benefit from play as a form of intervention and as a means of fostering their cognitive, social, and physical growth. They review several standardized instruments developed over the last fifty years to assess this benefit of play on child development. They identify twenty-one such play measures, the majority of…

Detecting and mitigating wormhole attacks in wireless networks remains a critical challenge due to their deceptive nature and potential to compromise network integrity. This paper proposes a novel approach to wormhole detection by leveraging propagation delay analysis between network nodes. Unlike traditional methods that rely on signature-based…

We discuss an application of Bayesian factor analysis for estimation of the optimal linear combination and associated maximal reliability of a multi-component measuring instrument. The described procedure yields point and credibility interval estimates of this reliability coefficient, which are readily obtained in educational and behavioral…

The credibility of conclusions arrived at in quantitative research depends, to a large extent, on the quality of data collection instruments used to quantify language and non-language constructs. Despite this, research into data collection instruments used in Applied Linguistics and particularly in the thesis genre remains limited. This study…

Despite over fifteen years of flipped classroom implementation, current literature does not provide any reliable, standardized rubric as a guideline to create or evaluate flipped classroom lessons based on effective flipped classroom design principles. In fact, at the time of this study, when an internet search for existing rubrics was conducted,…

The aim of this study is to develop a valid and reliable measurement tool to identify teachers' tendencies towards professional development models. In line with the purpose of scale development, a survey model was preferred. The scale was designed to be applicable to teachers from various disciplines currently working in any institution…

This study assesses the civic orientation of graduate students via the "Importance of Social Action Engagement" scale and tests that instrument's construct validity and reliability when applied to graduate students. The study contributes to our understanding of the levels of graduate students' willingness to engage with social issues,…

Generative AI tools (GenAI) are increasingly used for academic tasks, including qualitative data analysis for the Scholarship of Teaching and Learning (SoTL). In our practice as academic developers, we are frequently asked for advice on whether this use for GenAI is reliable, valid, and ethical. Since this is a new field, we have not been able to…

Double-scoring constructed-response items is a common but costly practice in mixed-format assessments. This study explored the impacts of Targeted Double-Scoring (TDS) and random double-scoring procedures on the quality of psychometric outcomes, including student achievement estimates, person fit, and student classifications under various…