The author provides information about reliability and areas that educators should examine in determining if an assessment is consistent and trustworthy for use, and how it should be interpreted in making decisions about students. Reliability areas that are discussed in the column include internal consistency, test-retest or stability, inter-scorer…

Multidimensional measuring instruments are often used in behavioral, social, educational, marketing, and biomedical research. For these scales, the paper discusses how to find the optimal score based on their components that is associated with the highest possible reliability. Within the framework of structural equation modeling, an approach to…

This paper describes a process to define a comprehensive list of exemplars for seven core Diagnostic and Statistical Manual (DSM) diagnostic criteria for autism spectrum disorder (ASD), and report on interrater reliability in applying these exemplars to determine ASD case classification. Clinicians completed an iterative process to map specific…

Universal screening is one of the most-common topics and well-accepted best practices within the field of gifted and talented education. There appears to be little disagreement that universally screening all students as part of a gifted and talented identification process results in fewer missed students. But surprisingly, there is little guidance…

The product Bayes factor (PBF) synthesizes evidence for an informative hypothesis across heterogeneous replication studies. It can be used when fixed- or random effects meta-analysis fall short. For example, when effect sizes are incomparable and cannot be pooled, or when studies diverge significantly in the populations, study designs, and…

This article outlines a readily applicable procedure for point and interval estimation of the population discrepancy between reliability and the popular Cronbach's coefficient alpha for unidimensional multi-component measuring instruments with uncorrelated errors, which are widely used in behavioral and social research. The method is developed…

Students who are engaged writers are likely to produce better writing and to enjoy writing more than students who are disengaged writers. Yet, we are unaware of any existing tool that validly and reliably measures writing engagement. In this article, we describe what writing engagement is and why it is important. Then, we present the Writing…

Sample size and statistical power are important factors to consider when planning a research synthesis. Power analysis methods have been developed for fixed effect or random effects models, but until recently these methods were limited to simple data structures with a single, independent effect per study. Recent work has provided power…

This academic article provides the essential steps for designing an effective English questionnaire in social science research, with a focus on ensuring clarity, cultural sensitivity and ethical integrity. Developed from key insights from related studies, it outlines potential practice in questionnaire design, item development and the importance…

Contribution: This article describes the process used to create a questionnaire to evaluate the usability of mobile learning applications (CECAM). The questionnaire includes specific questions to assess user interface usability and pedagogical usability. Background: Nowadays, mobile applications are expanding rapidly and are commonly used in…

Psychometrics is the field of designing tests and assessments to measure certain psychological concepts. It is chiefly concerned with two fundamental properties: reliability and validity. These properties are often influenced by confounding variables: other things that can influence performance but are not what you are trying to measure. Here, I…

One Extension Specialist implemented a STEM pilot robotics program across three middle school settings. A program evaluation to provide guidance and recommendations for future development, implementation, and continued evaluation was conducted as part of a larger study. This process led to the development of a condensed STEM survey that can be…

Nurse educators at the undergraduate level spend significant time developing and revising exam questions. Following the exam administration, course faculty have the opportunity to complete an item analysis and question revision to improve reliability and validity. A challenge faculty face is tracking these exam changes when teaching as part of a…

An integrated system comprising a commercial high-performance liquid chromatography HPLC (except autosampler) equipped with an external multichannel autofeeding module was developed to accomplish interference-free separation and kinetic tracking for the H[subscript 2]O[subscript 2]-thiourea reaction, which is proposed as a lab experiment for…

The research and practice of social scientists who work in a myriad of different specialty areas involve developing and validating scores on instruments as well as evaluating the psychometric properties of existing instrumentation for use with research participants. In this article, the author introduces The MEASURE Approach to instrument…