As a result of the Japanese Ministry of Education's recent edict that students' written and spoken English should be assessed in university entrance exams, there is an urgent need for tools to help teachers and students prepare for these exams. Although some commercial tools already exist, they are generally expensive and inflexible. To address these shortcomings, a new open-source, online test for assessing English ability was developed. The test features the automatic grading not only of reading and listening, but also of speaking and writing. Thus, the general English ability of large numbers of students can be checked quickly online, making the test suitable for use in entrance exams and placements tests. It is based around the Moodle LMS and features several new plugins to automatically grade speaking and writing. This paper details plugin development, shows preliminary samples, and explains how test reliability will be verified by comparing students' scores with human-ratings and widely used tests such as IELTS, TOEIC, and CASEC.