RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two…

Automated Essay Scoring (AES) tools aim to improve the efficiency and consistency of essay scoring by using machine learning algorithms. In the existing research work on this topic, most researchers agree that human-automated score agreement remains the benchmark for assessing the accuracy of machine-generated scores. To measure the performance of…

Email has become the most preferred form of business communication. Writing "good" email has become an essential skill required in the industry. "Good" email writing not only facilitates clear communication, but also makes a positive impression on the recipient, whether it be one's colleague or a customer. The aim of this paper…

Psychological interventions provide linguistic data that are particularly useful for testing mechanisms of action and improving intervention methodologies. For this study, emotional expression in an Internet-based intervention for women with breast cancer (n = 63) was analyzed via rater coding and 2 computerized coding methods (Linguistic Inquiry…

In the context of a project developing software for pronunciation practice and feedback for Mandarin-speaking learners of English, a key issue is how to decide which features of pronunciation to focus on in giving feedback. We used naive and experienced native speaker ratings of comprehensibility and nativeness to establish the key features…

Audit trail analysis can provide valuable insights to researchers and evaluators interested in comparing and contrasting designers' expectations of use and students' actual patterns of use of educational technology environments (ETEs). Sequence analysis techniques are particularly effective but have been neglected to some extent because of real…

Microcomputer-based imaging technology reestablishes the direct link between evaluation and environmental design by allowing evaluators and urban designers to visually explore and evaluate alternative simulations of design proposals. Microcomputer hardware and software requirements and the use of imaging technology at the university level are…

Designed for educational evaluators, this paper discusses reasons for the growing concern with linking or interfacing computers for information sharing; compares three major approaches to linking; and discusses the implications of each. An initial overview presents three linking applications in educational evaluation: (1) linking microcomputers…