Although an increasing amount of research has focussed on the relationship between student language proficiency and English-medium instruction (EMI) programme outcomes, there has been little focus on the broader assessment of progress and learning within EMI programmes, especially in Asia. The purpose of this study was to determine the kinds of programme assessments used throughout all stages of full-degree EMI programmes in Japan. Our inquiry examined the benchmarks currently in place to measure content knowledge, language proficiency and academic skills competence in undergraduate EMI programmes. Data were collected from 13 universities in Japan with well-established EMI degree programmes. In this paper, our findings are explored primarily in a qualitative and holistic manner, weaving the results of our survey together with the extant literature on the topic and our lived experiences as teachers in EMI programmes in Japan. Embedded in this conversation are evidence-based suggestions for improved practice in EMI programmes in Japan.