The Meaningful Learning in the Laboratory Instrument (MLLI) was designed to measure students' expectations before and after their laboratory courses and experiences. Although the MLLI has been used in various studies and laboratory environments to investigate students' cognitive and affective laboratory expectations, the authors of the instrument…

Monitoring applications (MAs) use digital and online tools to collect and track data on student behavior, and they have become increasingly popular among schools. Empirical research on these complex surveillance platforms is scant, and little is known about the efficacy or impact that they have on students. This study used a multi-method…

Cronbach's alpha is the most frequently used measure to investigate the reliability of measurement instruments. Despite its frequent use, many warn for misinterpretations of alpha. These claims about regular misunderstandings, however, are not based on empirical data. To understand how common such beliefs are, we conducted a survey study to test…

Nowadays, the assessment of student performance has become increasingly technology-based, a trend that can also be observed in the evaluation of scientific reasoning, with more and more of the formerly paper-based assessment tools moving into the digital space. The study aimed to examine the reliability and validity of the paper-based and…

In undergraduate research settings, students are likely to encounter anomalous data, that is, data that do not meet their expectations. Most of the research that directly or indirectly captures the role of anomalous data in research settings uses post-hoc reflective interviews or surveys. These data collection approaches focus on recall of past…

We present considerations for validity when a population outside of a normed sample is assessed and those data are interpreted. Using a career group counseling example exploring life satisfaction changes as evidenced by the Quality of Life Inventory (Frisch, 1994), we showcase qualitative and quantitative approaches to explore how normative data…

In the current article, we consider the influential position of the Programme for International Student Assessment (PISA) and discuss several methodological areas that demonstrate the need for caution when using and interpreting PISA results. We motivate our argument by briefly describing the program's increased influence in educational policy…

How is the usefulness of research assessed as university research becomes more and more commodified? The question is addressed through an analysis of how the results of a particular research project were received in a large private company that had provided the main funding for a research project on gender and top management, a project based on…

Data from the experimental functional analysis of challenging behaviors of 27 individuals with mental retardation were analyzed to assess agreement among three forms of interpretation. Results found that the methods of interpreting function from experimental assessment can give different results and that test-retest reliability of the experimental…

Assesses techniques used by researchers to collect and analyze data on how people with visual impairments or blindness learn, understand, and think about geographic space. Recommendations are made for increasing the validity of studies, including the use of multiple, mutually supportive tests; larger samples; and real-world environments.…

Psychological methods of assessing intelligence have been criticised because of their limited diagnostic-remedial nature and especially their lack of potential for initiating effective and pragmatic intervention programmes. Similarly, the means through which the results of such methods are communicated in order to make them useful and constructive…

Discusses issues relating to the reliability, validity, utility, and information value of unobtrusive testing of library reference services; provides suggestions for practical applications of these criteria; applies study findings to library decision making and planning; and identifies topics for further methodological refinement. (Author/CLB)