Epistemic engagement among higher education learners often involves cognitive appraisal operations which generate emotional processes. This study aims to develop and validate a multidimensional "Emotional Presence Scale" (EPS) to measure learners' experience of emotion due to cognitive appraisal in epistemic processes. Based on the…

This study is a conceptual replication of Teimouri et al.'s (2022) investigation into the validity of the second language (L2) grit scale (the L2-Grit scale). There are several concerns about the generalizability of the findings of Teimouri et al. (2022), especially regarding the discriminant validity of the scale and the relation of L2 grit with…

The Career Adapt-Abilities Scale (CAAS) has been validated in several East Asian countries, but the measurement equivalence of its four dimensions and construct validity has not yet been evaluated in Japan. Results from Japanese (n = 298) and South Korean (n = 301) college students indicated good to excellent internal consistency estimates, and…

The performance decision tree (PDT; Fulcher et al., 2011) is a rubric style that is applicable to performance assessment, with origins in Upshur and Turner's (1995) empirically derived binary-choice, boundary-definition (EBB) scale. It is easier for raters to assess performance by evaluating multiple binary-choice descriptors. Additionally,…

Addressing the lack of English tests that measure critical thinking (CT) abilities of EFL learners, this study aims to develop an English Critical Thinking Test (ECTT) designed to measure the consistency, analysis, and inference CT skills. It is a follow-up to a pilot study to accumulate validity evidence for the use of the ECTT. A total of 262…

This paper presents improvements made to a computerized adaptive testing (CAT)-based version of the FCI (FCI-CAT) in regards to test security and test efficiency. First, we will discuss measures to enhance test security by controlling for item overexposure, decreasing the risk that respondents may (i) memorize the content of a pretest for use on…

Background: Research on the test-taker perception of assessments has been conducted under the assumption that negative test-taker perception may influence test performance by decreasing test-taking motivation. This assumption, however, has not been verified in the field of language testing. Building on expectancy-value theory, this study explored…

In this paper, we present a system for visualizing learning logs of a course in progress together with predictions of learning activities of the following week and the final grades of students by state transition graphs. Data are collected from 236 students attending the course in progress and from 209 students attending the past course for…

Placement testing is a crucial issue in Japanese universities. In the majority of language programs, classes are streamed by proficiency levels based on students' placement test score for efficient instruction because university students' proficiency levels vary greatly even in the same program. The Visualizing English Language Competency Test®…

Individual differences are theorised as being key predictors of students' lecture attendance. Understanding students' (lack of) motivation for studying is an essential first step in supporting these individual differences and potentially increasing class/lecture attendance. The current study explored the role of students' deficits in motivations…

In this study, we evaluate the structural validity of Q.16 and Q.7 in the Force Concept Inventory (FCI). We address whether respondents who answer Q.16 and Q.7 correctly actually have an understanding of the concepts of physics tested in the questions. To examine respondents' levels of understanding, we use subquestions that test them on concepts…

This study compared three common vocabulary test formats, the Yes/No test, the Vocabulary Knowledge Scale (VKS), and the Vocabulary Levels Test (VLT), as measures of vocabulary difficulty. Vocabulary difficulty was defined as the item difficulty estimated through Item Response Theory (IRT) analysis. Three tests were given to 165 Japanese students,…

Background: While numerous articles on Criterion® have been published and its validity evidence has accumulated, test users need to obtain relevant validity evidence for their local context and develop their own validity argument. This paper aims to provide validity evidence for the interpretation and use of Criterion® for assessing second…

This paper examines how different test methods may tap different aspects of second language knowledge. It employs multiple-choice (MC) and constructed response (CR) items which yield distinct or convergent information in the computer delivered testing of English in its presentation of this factor. In order to examine the effects of test method, a…

Cloze tests have been the subject of numerous studies regarding their function and use in both first language and second language contexts (e.g., Jonz & Oller, 1994; Watanabe & Koyama, 2008). From a validity standpoint, one area of investigation has been the extent to which cloze tests measure reading ability beyond the sentence level.…