This academic article provides the essential steps for designing an effective English questionnaire in social science research, with a focus on ensuring clarity, cultural sensitivity and ethical integrity. Developed from key insights from related studies, it outlines potential practice in questionnaire design, item development and the importance…

A positive working environment and culture are essential for researchers as these enable them to conduct valuable, high-quality research. Yet, university staff frequently report their research culture as less than ideal. To understand researchers' experiences of research culture to inform tangible change, several surveys have been conducted by…

In this article, we describe and illustrate the process by which we developed and validated short, multiple-choice, reliable tests to assess undergraduate students' comprehension of three mathematical proofs. We discuss the purpose for each stage and how it benefited the design of our instruments. We also suggest ways in which this process could…

I am currently conducting research for a PhD Public Health. As a result of my previous experience researching my Master's degree thesis, and what I have learned so far about designing and validating a risk-perception instrument, I have gained some insight related to know-how and possible pitfalls that can occur when undertaking research. These…

Although International Association for the Evaluation of Educational Achievement-pioneered international large-scale assessment (ILSA) of education is now a well-established science, non-practitioners and many users often substantially misunderstand how large-scale assessments are conducted, what questions and challenges they are designed to…

Evaluation is a key component to learning about the effectiveness of a program. This article provides descriptive statistics of the newly developed National 4-H Common Measures (science, healthy living, citizenship, and youth development) based on data from 721 California 4-H youth. The measures were evaluated for their reliability and validity of…

The "PISA 2012 Technical Report" describes the methodology underlying the PISA 2012 survey, which tested 15-year-olds' competencies in mathematics, reading and science and, in some countries, problem solving and financial literacy. It examines the design and implementation of the project at a level of detail that allows researchers to…

Creating surveys is a common task in evaluation research; however, designing a survey instrument to gather average group response data that can be interpreted in a meaningful way over time can be challenging. When surveying groups of people for the purpose of longitudinal analysis, the reliability of the result is often determined by the response…

The ever-changing requirements of working life require individuals to develop their competencies throughout their life cycle. This lifelong learning paradigm requires a renewed vision concerning assessment in which, besides formal learning, informal and non-formal learning experiences are also recognized. To support this lifelong learning…

To determine trends in reporting test reliability, 88 articles addressing 188 instruments in 1980, 81 articles covering 205 instruments in 1985, and 67 articles assessing 195 instruments in 1990 in the "Journal of Counseling Psychology" were reviewed. Articles were examined for the way in which reliability was discussed and reported, and…

In this study, generalizability theory is used to examine the dependability of student rating data for making judgments about courses and instruction. The importance of giving adequate attention to the specification of the universe of admissible observations in generalizability theory is discussed. (Author/CTM)

The Organisation for Economic Cooperation and Development's (OECD's) Programme for International Student Assessment (PISA) surveys, which take place every three years, have been designed to collect information about 15-year-old students in participating countries. PISA examines how well students are prepared to meet the challenges of the future,…

Based on dissertation for EdD degree at University of Texas (1966).

This article illustrates the use of generalizability theory in special education to estimate the reliability of a measure when there is more than one source of error in the universe of inference and how the effects from changing the number of items and/or raters can be evaluated. (Author)

Educators have a need for a procedure to generate alternate forms of tests. The reliability of alternate forms generated from a table of specifications is examined, using 78 high school remedial mathematics students as subjects. (Editor)