This note is concerned with the benefits that can result from the use of the maximal reliability and optimal linear combination concepts in educational and psychological research. Within the widely used framework of unidimensional multi-component measuring instruments, it is demonstrated that the linear combination of their components that…

Researchers often study dynamic processes of latent variables in everyday life, such as the interplay of positive and negative affect over time. An intuitive approach is to first estimate the measurement model of the latent variables, then compute factor scores, and finally use these factor scores as observed scores in vector autoregressive…

This three-article dissertation aims to address three methodological challenges to ensure comparability in educational research, including scale linking, test equating, and propensity score (PS) weighting. The first study intends to improve test scale comparability by evaluating the effect of six missing data handling approaches, including…

This study aimed to introduce an ego-resiliency questionnaire for preadolescents (the ER-P) by restructuring the ER89 using the data of 1398 preadolescents from the Panel Study on Korean Children. The ER-P was proposed as a 10-item second-order instrument with two factors (Optimal Regulation and Openness to Life Experiences). The ER-P achieved…

There is a tradeoff between scale length and psychometric concerns. The two are, in fact, directly linked. Generally, when scales are shortened, reliability is reduced, and when scales are lengthened, reliability is improved, provided the items added to the scale are comparable psychometrically (AERA et al., 2014). Scale reliability, in turn,…

The objectives were to examine the factorial structure of the Academic Procrastination Scale-Short Form (APS-S) and the measurement invariance across gender and educational levels, to determine possible differences in procrastination across gender, educational levels, and grades. The sample was formed of 1486 Spanish primary and secondary school…

Existing measures of test anxiety used with the college student population are old with old norms and old items, and they do not capture the multiple dimensions of the test anxiety construct or assess facilitating anxiety. In the present study, the validity of the scores of a new, multidimensional measure of test anxiety with a facilitating…

Adoption of new instructional standards in science demands high-quality information about classroom practice. Teacher portfolios can be used to assess instructional practice and support teacher self-reflection anchored in authentic evidence from classrooms. This study investigated a new type of electronic portfolio tool that allows efficient…

Among school psycho-social factors with considerable effect on student outcomes are both school and classroom climate. Because how students perceive the classroom climate strongly predicts achievement, measuring classroom climate gains importance and the need for testing the existing results across cultures persists. In this study, we assessed the…

We examine the factor structure of scores from the CLASS-S protocol obtained from observations of middle school classroom teaching. Factor analysis has been used to support both interpretations of scores from classroom observation protocols, like CLASS-S, and the theories about teaching that underlie them. However, classroom observations contain…

The Aberrant Behavior Checklist (ABC) is a widely used measure in autism spectrum disorder (ASD) treatment studies. We conducted confirmatory and exploratory factor analyses of the ABC in 1,893 children evaluated as part of the Autism Treatment Network. The root mean square error of approximation was .086 for the standard item assignment, and in…

The purpose of this article is to provide a commentary on the current state of several measurement issues pertaining to curriculum-based measures of reading (R-CBM). We begin by providing an overview of the utility of R-CBM, followed by a presentation of five specific measurements considerations: (a) the reliability of R-CBM oral reading fluency…

As a property of test scores, reliability/dependability constitutes an important psychometric consideration, and it underpins the validity of measurement results. A review of interpreter certification performance tests (ICPTs) reveals that (a) although reliability/dependability checking has been recognized as an important concern, its theoretical…

Although a few studies report sizable score gains for examinees who repeat performance-based assessments, research has not yet addressed the reliability and validity of inferences based on ratings of repeat examinees on such tests. This study analyzed scores for 8,457 single-take examinees and 4,030 repeat examinees who completed a 6-hour clinical…

Policymakers and school administrators have embraced value-added models of teacher effectiveness as tools for educational improvement. Teacher value-added estimates may be viewed as complicated scores of a certain kind. This suggests using a test validation model to examine their reliability and validity. Validation begins with an interpretive…