Questionnaire scales that are mixed-worded, i.e. include both positively and negatively worded items, often suffer from issues like low reliability and more complex latent structures than intended. Part of the problem might be that some responders fail to respond consistently to the mixed-worded items. We investigated the prevalence and impact of…

The study aims to verify the validity and reliability of a set of tests for measuring Indonesian students' reading literacy achievement that has been developed by the Center for Language Strategy and Diplomacy Development as well as the Educational Assessment Center, two institutions under the auspices of the Ministry of Education, Culture,…

Bias assessment (systematic errors) is fundamental in industry and service laboratories, where reliable results must be obtained to give correct answers to specific problems. Therefore, knowledge and practice in quality methodologies is of fundamental importance for students. Unfortunately, laboratory lessons often focus on connecting theory and…

The Cross-Classified Mixed Effects Model (CCMEM) has been demonstrated to be a flexible framework for evaluating reliability by measurement specialists. Reliability can be estimated based on the variance components of the test scores. Built upon their accomplishment, this study extends the CCMEM to be used for evaluating validity evidence.…

We investigated two research questions: which students are more likely to respond inconsistently to mixed-worded questionnaire scales, and which country samples have larger shares of inconsistent respondents? We defined an inconsistent response as strongly agreeing or disagreeing with both positively and negatively worded items of the same scale.…

In Comparative Judgement (CJ) exercises, examiners are asked to look at a selection of candidate scripts (with marks removed) and order them in terms of which they believe display the best quality. By including scripts from different examination sessions, the results of these exercises can be used to help with maintaining standards. Results from…

This study aimed to develop a valid and reliable multiple-choice achievement test for the subject area of ecology. The study was conducted within the framework of exploratory sequential design based on mixed research methods, and the study group consisted of a total of 250 middle school students studying at the sixth and seventh grade level. In…

The importance of the opportunity to learn (OTL) for mathematics achievement has been extensively researched. However, there were still unanswered questions regarding OTL's measurement, analytical level, and relationship with motivational beliefs. To fill in the gaps, we aimed to (1) scrutinize the reliability and validity of OTL, (2) investigate…

PISA (Program for International Student Assessment) tests have enabled the OECD countries to see not only the success of their students in gaining the ability to solve some daily problems they may encounter in their lives but also the place in the world rankings as a result of an objective evaluation comparing the achievement results of…

Teachers lack effective curriculum-based instruments to assess their students' scientific competence that would provide information for modifying their inquiry instruction. The main purpose of this study was to develop and validate a Curriculum-Based Scientific Competence (CBSC) test to assess students' scientific competence in a 1-semester Grade…

Disciplinary climate and opportunity to learn (OTL) are considered as effectiveness-enhancing factors that can improve mathematics achievement. In this study, we investigated whether the school-level aggregation of student-reported OTL could yield reliable and valid measures, and then explored the relationships among disciplinary climate, OTL, and…

The study aimed to investigate the effect of child literature based integrative instructional program on promoting 7th graders writing skills at Irbid governorate in Jordan. The sample of the study totaled (87) male and female students selected purposefully. These were randomly assigned into four groups, two experimental groups, the first was…

This research aimed to compare the G and Phi coefficients estimated in Decision studies in Generalizability theory and obtained in actual cases for the same conditions of similar facets by using crossed design. The research was conducted as pure research on 120 individuals (students), six items and 12 raters. An achievement test composed of six…

Data analysis is a significant methodological component when conducting quantitative education studies. Guidelines for conducting data analyses in quantitative education studies are common but often underemphasize four important methodological components impacting the validity of inferences: quality of constructed measures, proper handling of…

The importance of enhancing social and emotional skills in educational settings has gained prominence, with many countries and organizations embracing the Social and Emotional Learning (SEL) framework to equip individuals with the tools needed for shaping a self-identity, emotional regulation, goal achievement, empathy, nurturing relationships,…