Human ratings are ubiquitous in creativity research. Yet, the process of rating responses to creativity tasks -- typically several hundred or thousands of responses, per rater -- is often time-consuming and expensive. Planned missing data designs, where raters only rate a subset of the total number of responses, have been recently proposed as one…

Policy-makers, research managers, and donors are interested in knowing the outputs, outcomes, and impacts of investment in agricultural research. Several institutions are engaged in rice research and impact assessment of rice research requires area coverage information for rice varieties. The area coverage information for individual varieties is…

Probes are follow-ups to survey questions used to gain insights on respondents' understanding of and responses to these questions. They are usually administered as open-ended questions, primarily in the context of questionnaire pretesting. Due to the decreased cost of data collection for open-ended questions in web surveys, researchers have argued…

Although experimental evaluations have been labeled the "gold standard" of evidence for policy (U.S. Department of Education, 2003), evaluations without an analysis of costs are not sufficient for policymaking (Monk, 1995; Ross et al., 2007). Funding organizations now require cost-effectiveness data in most evaluations of effects. Yet,…

Enzyme-linked immunosorbent assay (ELISA) is a valuable tool in various chemical and biological fields for investigating the presence and quantity of proteins of interest in given samples. The high importance of the assay makes it widely used in biological research and hence a crucial skill to master for up-and-coming scientists. A major…

COVID-19 has changed the way research can be conducted. The present literature review, based on 29 studies (2008-2020), investigates and compiles existing research to identify possibilities and limitations of using digital interviews within social work, sociology and adjacent disciplines. The results show that digital interviews can enhance…

For almost 20 years, Open Educational Resources (OER) are an integral part of the debate about the digitisation of education. However, the empirical landscape of OER research is vivid and largely obscure. This article reviews the state of international empirical research on OER to reveal trends and gaps and, in this manner, identify possible…

Research methods course(s), a standard in psychology programs, often use multiple textbooks to address conceptual and data-analytic information. This study involved transitioning from traditional textbooks to open educational resources in a research methods course. Two psychology instructors, each offering course sections, identified open-access…

This chapter will value of including diverse voices of researchers. An innovative, cost-effective model for assessment will be described, capturing the process implemented at a university in its efforts to center social justice within assessment and evaluation research practices.

Research impact features heavily in debates about 'the measured university' and is now formally assessed by governments in the UK and Australia. Yet clear guidance on how impact can be measured in non-monetary ways is often lacking because of confused thinking and the context-specific nature of outcomes. To help resolve this, we first propose a…

Systematic reviews are increasingly used to inform health care decisions, but are expensive to produce. We explore the use of "crowdsourcing" (distributing tasks to untrained workers via the web) to reduce the cost of screening citations. We used Amazon Mechanical Turk as our platform and 4 previously conducted systematic reviews as…

Weisman and Montgomery completed an overview of reviews to evaluate the status of research concerning functional family therapy (FFT). Despite reporting modest effects on delinquency, substance use, and secondary outcomes, the authors conclude that the research base supporting FFT is tenuous and that it "may not be advisable to continue…

Background: Medical research involving human subjects must be evaluated by a research ethics committee (REC) before a study is initiated. However, knowledge of REC decision processes, particularly in relation to evaluating the risk-benefit balance in various study types, appears scant. Methods: The study protocols and records of a Finnish…

Research has a role to play in society's endeavour for sustainable development. This is particularly true for agricultural research, since agriculture is at the nexus between numerous sustainable development goals. Yet, generally accepted methods for linking research outcomes to sustainability impacts are missing. We conducted a review of…

The rise of social media websites (e.g., Facebook) and online services such as Google AdWords and Amazon Mechanical Turk (MTurk) offers new opportunities for researchers to recruit study participants. Although researchers have started to use these emerging methods, little is known about how they perform in terms of cost efficiency and, more…