In this paper we use historic score-reporting records and test-taker metadata to inform data-driven recommendations that support international students in their choice of undergraduate institutions for study in the United States. We investigate the use of Structural Topic Modeling (STM) as a context-aware, probabilistic recommendation method that…

As English continues to spread as an international lingua franca, there is a growing diversity in its use around the world. As a result, there are calls for embracing the diversity in the teaching, learning and assessment of the language. At the same time, there is a growing criticism against the widely taken language tests such as the Test of…

To account for natural variability in cognitive processing, it is standard practice to optimize a model's parameters by fitting it to behavioral data. Although most language-related theories acknowledge a large role for experience in language processing, variability reflecting that knowledge is usually ignored when evaluating a model's fit to…

Background: Validity is a notable concept in language testing which has concerned many researchers and scholars in the field of language testing due to its importance in decision making process. Tests' results always introduce consequences to test takers' lives which emphasizes the need to ensure their validity. Detecting and delineating the…

In this research report, we describe the design and empirical findings for a large-scale study of essay writing ability with approximately 2,500 high school students in Germany and Switzerland on the basis of 2 tasks with 2 associated prompts, each from a standardized writing assessment whose scoring involved both human and automated components.…

Developing a diagnostic tool within the diagnostic measurement framework is the optimal approach to obtain multidimensional and classification-based feedback on examinees. However, end users may seek to obtain diagnostic feedback from existing item responses to assessments that have been designed under either the classical test theory or item…

This study investigates two models of L2 vocabulary acquisition at an advanced L2 level: L1 lemma mediation and the revised hierarchical models. Proponents of the L1 lemma mediation model postulate that advanced L2 learners map L2 words to L1 meanings or concepts, whereas proponents of the revised hierarchical model argue that the increasing…

In this study, we assessed the invariance in the factor structure underlying English-language proficiency for two groups of adolescent learners in Japan: students in middle school (ages 13-15 years) and students in high school (ages 16-18 years). Language proficiency was measured using the "TOEFL Junior"® Comprehensive test, an…

Although several studies have explored the effects of task sequencing on second language (L2) production, there is no established set of criteria to sequence tasks for learners in L2 writing classrooms. This study examined the effect of simple ?complex task sequencing manipulated along both resource-directing (± number of elements) and…

Despite calls for engaging assessments targeting young language learners (YLLs) between 8 and 13 years old, what makes assessment tasks engaging and how such task characteristics affect measurement quality have not been well studied empirically. Furthermore, there has been a dearth of validity research about technology-enhanced speaking tests for…

The present study examines the relative importance of attributes within and across items by applying four cognitive diagnostic assessment models. The current study utilizes the function of the models that can indicate inter-attribute relationships that reflect the response behaviors of examinees to analyze scored test-taker responses to four forms…

Testing is an issue of increasing importance. While for many teachers language learning should be communicative; in fact, they expect their students to provide evidence of their knowledge. Thus, there is a clear mismatch on the approach to language teaching and language testing. As a consequence, there is an evident need change the testing…

This study explored the relationships among language proficiency, two selected test-taker characteristics (i.e., topical knowledge and anxiety), and integrated speaking test performance. Data collection capitalized on three sets of instruments: three integrated tasks derived from TOEFL-iBT preparation materials, the state anxiety inventory created…

In this study I examined the dimensionality of the latent ability underlying language use that is needed to fulfill the demands young learners face in English-medium instructional environments, where English is used as the means of instruction for teaching subject matters. Previous research on English language use by school-age children provided…

Previous research in cognitive diagnostic assessment (CDA) of L2 reading ability has been frequently conducted using large-scale English proficiency exams (e.g., TOEFL, MELAB). Using CDA, it is possible to analyze individual learners' strengths and weaknesses in multiple attributes (i.e., knowledge, skill, strategy) measured at the item level.…