In this study, I compare the accuracy of automatic speech recognition (ASR) transcription against two measures of intelligibility provided by human listeners. The data came from readings of five texts recorded by 15 language learners of French. Human understanding was gauged by (i) asking a group of 36 naïve first language (L1) speakers of French recruited from the general population to provide word-for-word transcriptions of the recordings, and (ii) asking two experienced French teachers (L1 speakers of French) to rate the recordings for intelligibility. The results reveal that ASR transcription accuracy was lower than that of human transcribers. However, all three measures, namely ASR transcription accuracy, human transcription accuracy, and human ratings of intelligibility, were correlated. Implications of these results for using the technology in foreign language teaching and learning are considered in the discussion.