Studies on automatic scoring systems in writing assessments have also evaluated the relationship between human and machine scores for the reliability of automated essay scoring systems. This study investigated the magnitudes of indices for inter-rater agreement and discrepancy, especially regarding human and machine scoring, in writing assessment.…

This investigation utilized publicly available data from the High School Longitudinal Study 2009 (HSLS:09) by the National Center for Educational Statistics (NCES) to examine any latent structures among variables which may empirically support the validity and reliability of a mathematical sense of belonging (MSB) construct. Using the nationally…

Teachers at St. Scholastica School, a pseudonym, an all-girls Catholic high school, varied in approaches to discipline creating an unequal distribution of student citations and negatively impacting student experience. An Improvement Science approach identified two potential drivers of change: inconsistency in disciplinary policy and variation in…

This study provides preliminary evidence for reliability and validity of the Hawaii Early Learning Profile Strands 0-3 (HELP Strands 0-3), an assessment instrument for young children. First, the degree of interobserver agreement for a sample of representative HELP items was examined; results indicated that HELP scoring was dependable and…

Assessing fidelity of implementation of school-based interventions is a critical factor in successful implementation and sustainability. The Tiered Fidelity Inventory (TFI) was developed as a comprehensive measure of all three tiers of School-Wide Positive Behavioral Interventions and Supports (SWPBIS) and is intended to measure the extent to…

Intoxicated witnesses are common, making it important to understand alcohol's impact on witness accuracy and suggestibility. Participants assigned to an immediate retrieval condition encoded and recalled in one of the three intoxication conditions: sober control, placebo, or intoxicated. Participants in the delayed retrieval condition were…

Aberrant response has an important impact on item parameter estimation, individuals' evaluation, and other statistical analysis. There are various types of aberrant response behaviors in educational and psychological tests, like sleeping, guessing, and plodding. Random response is the most common one. The purpose of this research was to clarify…

In the current study, we present the development of the Global Self-Esteem (GSE) measure. The six-item GSE fulfills a need for a short unidimensional measure of global self-esteem conceptualized as "overall positive view of self." The construct is traditionally measured by the Rosenberg Self-Esteem Scale (RSE); however, several important…

Differences in rater judgments that are systematically related to construct-irrelevant characteristics threaten the fairness of rater-mediated writing assessments. Accordingly, it is essential that researchers and practitioners examine the degree to which the psychometric quality of rater judgments is comparable across test-taker subgroups.…

Many large-scale assessments are designed to yield two or more scores for an individual by administering multiple sections measuring different but related skills. Multidimensional tests, or more specifically, simple structured tests, such as these rely on multiple multiple-choice and/or constructed responses sections of items to generate multiple…

Purpose: This study aims to examine the reliability and validity of the learning style construct conceptualized by Honey and Mumford (1986) in educational settings in the United Arab Emirates. Design/methodology/approach: Two independent samples from the UAE were used: one comprised 1,463 undergraduate students at the UAE University, and the other…

Individuals are exposed to misleading or outright false anthropogenic climate change (ACC) information. The goals of this study are to identify ACC dissenter messages, and to develop an instrument that quantifies the extent to which individuals agree with these messages. The instrument was developed using a sequential mixed methods design. A…

Transactional validity, a common approach in participatory research, is attained when preliminary analyses of research results are discussed with research participants and their feedback is incorporated in the analysis. Member checking is one way of achieving transactional validity, which has been heralded as a stronger version of validity reached…

Many large universities, community colleges and some smaller four-year colleges are turning to hybrid or online instruction for remedial and entry level mathematics courses, often assessed using online exams in a proctored computer lab environment. Faculty face the task of choosing questions from a publisher's text bank with very little, if any,…

Cronbach's alpha is the most frequently used measure to investigate the reliability of measurement instruments. Despite its frequent use, many warn for misinterpretations of alpha. These claims about regular misunderstandings, however, are not based on empirical data. To understand how common such beliefs are, we conducted a survey study to test…