This paper aims to explore the philosophical theoretical foundations of two basic research paradigms, namely positivism and interpretivism. In the discussion process, literature in the relevant fields including academic papers and books is reviewed and used as support for the analysis. Firstly, the paper explores the differences between the…

RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two…

Empirical researchers using qualitative comparative analysis (QCA) can work with crisp, multivalue, and fuzzy sets. The relative advantages of crisp and multivalue sets have been discussed in the QCA literature. There has been little reflection on the more frequent decision between crisp and fuzzy sets for which there often is no theoretical…

Increasingly, the broader public, media and policymakers are looking to qualitative research to provide answers to our most pressing social questions. While an exciting and perhaps overdue moment for qualitative researchers, it is also a time when the method is coming under increasing scrutiny for a lack of reliability and transparency. The…

In this article, we explore inconsistencies in the implementation of outdoor learning across Australian early years' education. The benefits of outdoor learning justify regular employment of this pedagogical approach in both early childhood education and primary school settings. Early childhood education services provide daily outdoor learning…

This article considers psychometric properties of composite raw scores and transformed scale scores on mixed-format tests that consist of a mixture of multiple-choice and free-response items. Test scores on several mixed-format tests are evaluated with respect to conditional and overall standard errors of measurement, score reliability, and…

This study compares the psychometric properties of reliability in Classical Test Theory (CTT), item information in Item Response Theory (IRT), and validation from the perspective of modern validity theory for the purpose of bringing attention to potential issues that might exist when testing organizations use both test theories in the same testing…

This article describes the development of a new questionnaire for assessing L2 learners' language attitudes. Drawing on theoretical work in the fields of social psychology and applied linguistics, the author reviewed the concept of (language) attitudes and contrasted them with the concept of (language) motivation. This thorough literature review,…

Fuzzy-set qualitative comparative analysis (fsQCA) has become one of the most prominent methods in the social sciences for capturing causal complexity, especially for scholars with small- and medium-"N" data sets. This research note explores two key assumptions in fsQCA's methodology for testing for necessary and sufficient…

The American Council on the Teaching of Foreign Languages (ACTFL) oral proficiency interview -- computer (OPIc) testing system represents an ambitious effort in language assessment: Assessing oral proficiency in over a dozen languages, on the same scale, from virtually anywhere at any time. Especially for users in contexts where multiple foreign…

For many teachers, the classroom observation has been the only opportunity to receive direct feedback from another school professional. As such, it is an indispensable part of every teacher evaluation system. Yet it also requires a major time commitment from teachers, principals, and peer observers. To justify the investment of time and resources,…

Calls for replication studies are becoming more frequent, and Computer Assisted Language Learning (CALL) has now reached sufficient maturity to offer numerous studies that lend themselves to replication. Realistic and successful replications rely on transparency in terms of data, results, and methodology. Two published studies in the area of…

The 1961 Copyright Office study on renewals, authored by Barbara Ringer, has cast an outsized influence on discussions of the U.S. 1923-1963 public domain. As more concrete data emerge from initiatives such as the large-scale determination process in the Copyright Review Management System (CRMS) project, questions are raised about the reliability…

As a global measure of precision, item response theory (IRT) estimated reliability is derived for four coefficients (Cronbach's a, Feldt-Raju, stratified a, and marginal reliability). Models with different underlying assumptions concerning test-part similarity are discussed. A detailed computational example is presented for the targeted…

Findings on the reliability and cost-effectiveness of the use of cellular phones vis-à-vis face-to-face interviews in investigating health behaviors and conditions are presented for a national epidemiological sample from Lebanon. Using self-reported responses on identical questions, percentage agreement, ? statistics, and McNemar's test were used…