Conventional methods for evaluating the utility of subscores rely on traditional indices of reliability and on correlations among subscores. One limitation of correlational methods is that they do not explicitly consider variation in subtest means. An exception is an index of score profile reliability designated as [G], which quantifies the ratio…

In response to the call for more rigorously validated educational assessments, this study used an iterative multimethod validation process to develop and validate outcomes from the 21st Century Skills Assessment global rating scale. Qualitative and quantitative data sources were used to inform four types of validity evidence: content, response…

In this study, we examined the relationship of special education teachers' performance on the Recognizing Effective Special Education Teachers (RESET) Explicit Instruction observation protocol with student growth on academic measures. Special education teachers provided video-recorded observations of three instructional lessons along with data…

This study aimed to develop a valid and reliable instrument that measures the functional vision of students with low vision. Thus, an assessment tool and performance activities were developed for three vision skill groups (near vision skills, distance vision skills, and visual field) that include functional vision skills. The universe was 1485…

The aim of this study is to show how a many-facet Rasch measurement model (MFRM) can be used for quality control whilst monitoring a musical aptitude examination. The data used in this study was gathered from a musical aptitude examination which was applied in 2019-2020 academic year for selecting teacher candidates to a music education department…

Secondary school staff are often tasked with delivering mental health content to students, yet there has been little research on staff confidence to do so. Given the responsibility placed on staff to support student mental health, reliable and valid measures are needed to facilitate assessment of teacher confidence in the classroom and evaluation…

To examine the reliability and validity of revised Parental Phubbing Scale (PPS) and measure the level of parental phubbing, 701 Chinese children aged 3-6 years and their parents were investigated. The results indicated that: (1) the construct validity of the PPS was supported by the best-fit one-factor model; (2) concurrent validity was…

The aim of this research is to develop a scale for determining pre-service teachers' attitudes towards recycling. The research was carried out using a quantitative research design. The research was carried out in the education faculties of two state universities located in the east and west of Türkiye. 284 pre-service teachers (62 males, 222…

Psychometrics is the field of designing tests and assessments to measure certain psychological concepts. It is chiefly concerned with two fundamental properties: reliability and validity. These properties are often influenced by confounding variables: other things that can influence performance but are not what you are trying to measure. Here, I…

This article presents a ten-item short scale for measuring digital competence. The scale is based on the Digital Competence Framework for Citizens, DigComp2.1 (Carretero et al., 2017). For our surveys, we used five items from the DigCompSat study (Clifford et al., 2020) and created five new ones to address the competence areas defined by…

Internal validity and measurement invariance of the Counseling Center Assessment of Psychological Symptoms-62 item (CCAPS-62) was examined via confirmatory factor analytic models among autistic (n = 1,268) and non-autistic (n = 3,776) college students. Findings reviewed the CCAPS-62 had a strong model fit and was invariant across groups.

The consequence of a change in school leadership (e.g., principal turnover) on student achievement has important implications for education policy. The impact of such an event can be estimated via the popular Difference in Difference (DiD) estimator, where those schools with a turnover event are compared to a selected set of schools that did not…

Research practice partnerships (RPPs) are an increasingly popular approach for bringing together research and practice in ways that can improve and transform education. To date, however, evaluators have lacked measures to evaluate such partnerships. This paper describes a project to develop validity evidence for a survey to evaluate the…

For decades, evaluators have noted that it is difficult for stakeholders to accept negative evaluation results (Carter, 1971; Taut & Brauns, 2003). There is a need for additional research on evaluation to better understand when and why stakeholders reject negative or critical evaluation findings. Drawing on social identity theory (SIT), the…

Background and Purpose: Metavisualization plays a key role in science and technological learning in which visualization is practiced. We conducted two sequential studies with the purposes to first develop an instrument that allows individuals to report or reflect on their metavisualization, and then validate the instrument and investigate the…