Despite the large amount of research that has taken place with mobile devices for language learning over the years, there are still limited conclusions that can be reached as to what contributes to sustained task engagement. A common problem that has long been seen in the CALL literature is that many tasks and activities that are reported are small scale studies taking place in a single environment with a limited number of subjects, often as a result of teachers investigating the outcomes of their own teaching (see Warschauer, 1997; Hubbard, 2005). Given the nature of the environments in which many teachers find themselves, it is in some way inevitable that studies tend to be of a smaller scale and often undertaken within a single class. It is possible to gain deeper insights into the tasks used through replication studies (Porte, 2012). This paper discusses the potential insights gained from replicating approximately the same basic language learning tasks in varied contexts. Research on vocabulary and listening tasks was carried out in Japan over an 8-year period from 2010 through 2017 for Japanese learners of English. In addition, the same tasks were adapted for Australian learners of Japanese in 2012 and Taiwanese learners of English in 2013, with a total of 420 participants. Data were collected through post-treatment surveys, semi-structured interviews and server logs recording the times spent on the tasks, the scores achieved, and the devices used to engage in the tasks. The same methods of data collection were used in each of the studies, using primarily the same tasks apart from adaptations made for developments in technology and for the different language learning contexts. Carrying out the same tasks in subsequent years with increasingly newer technologies and with teachers and learners from different cultural backgrounds provided insights into the effect of the context, the technology, and role of both teachers and learners in successfully implementing the tasks.