Existing tests of interrater agreements have high statistical power; however, they lack specificity. If the ratings of the two raters do not show agreement but are not random, the current tests, some of which are based on Cohen's kappa, will often reject the null hypothesis, leading to the wrong conclusion that agreement is present. A new test of…

This essay reviews the changing functions, and effects, of citation systems in scholarly research as they move from a range of uses primarily oriented around knowledge networking and epistemic validation, to their use as a set of metrics oriented around evaluating and rewarding certain kinds of academic performance (e.g. "impact…

Psychometric models based on structural equation modeling framework are commonly used in many multiple-choice test settings to assess measurement invariance of test items across examinee subpopulations. The premise of the current article is that they may also be useful in the context of performance assessment tests to test measurement invariance…

This study presents the development and empirical validation of score levels and descriptors specifically designed for reporting purposes to provide test takers with more than just a number on a score scale. In the context of a test primarily intended for 11- to 15-year-old students learning English as a second/foreign language, the study examined…

This study explains assessment for learning instrumentation, especially in higher education. The population of this study was 100 lecturers of the Muhammadiyah Makassar University, Indonesia. A total of 50 items from six construct were analyzed and used to determine the reliability and validity of the questionnaire. The result shows that the…

Research shows that when evaluating competing explanations people usually discount an explanation in favor of an alternative explanation and, at the same time, prefer the explanation that is provided before an alternative explanation. In this article, we examine how inconsistencies in one but not the other explanation influence the evaluation and…

Likert scales are used to make relative and absolute judgments about measures of attitude. Despite its ubiquitous use, only few studies have investigated the effects of altering the configurations of the response options. The purpose of this experiment was to explore the effects of response option orientation and directionality in Likert scales…

In this paper an attempt was made to analyze the effect of items direction (positive or negative) on the Alpha Cronbach reliability coefficient and the Split Half reliability coefficient in Likert scale. The descriptive survey research method was used for the study and sample of 510 undergraduate students were selected by used random sampling…

Cognitive assessment in natural group settings facilitates data collection but poses threats to the validity. In this study, tablet-based working memory (WM) tasks, the counting span, and reading span were used in predicting 12-year-old children's (N = 837) scholastic skills and fluid intelligence in a classroom with environmental noise. WM tasks…

This study addresses the development and evaluation of the Physics Problem-Solving Taxonomy (PPST), comprising five levels: retrieval, diagnosis, strategy, conceptual, and creative thinking. The taxonomy draws on Bloom's revised taxonomy in the cognitive domain, the Types of Knowledge Taxonomy, and the Problem-Solving Taxonomy in engineering. The…

Reasoning about mechanism is central to disciplined inquiry in science and engineering and should thus be one of the foundations of a science, technology, engineering, and mathematics education. In addition, mechanistic reasoning is one of the core competencies listed in the Next Generation Science Standards (NGSS) Engineering Concepts and…

This article is focused on the creation of a new scale for measuring preservice teachers' positive affect for science, the Preservice Elementary Teacher Affect Scale for Science (PETAS-S). This new instrument is designed specifically to measure the level of positive affect toward the subject of science in preservice elementary teachers.…

The purpose of this article of reflection is to raise awareness of how poor design of language assessments may have detrimental effects, if crucial qualities and technicalities of test design are not met. The article first discusses these central qualities for useful language assessments. Then, guidelines for creating listening assessments, as an…

Research on teacher-student relationships is mainly based on questionnaires investigating teachers' explicit cognitions. However, it is also important to investigate implicit processes in social interactions, such as internalized feelings. The Teacher Relationship Interview (TRI) is a narrative method aimed at assessing teachers' (implicit) mental…

Deficits in social communication and interaction have been identified as distinguishing impairments for individuals with an autism spectrum disorder (ASD). As a pivotal skill, the successful development of social communication and interaction in individuals with ASD is a lifelong objective. Point-of-view video modeling (VM) has the potential to…