Word problems are a question type commonly seen in math tests and exams. Since word problems are an integral part in teaching and learning of mathematics in schools, understanding what factors that would affect students' ability to solve math word problems may help students understand where their weaknesses lie and help teachers adopt effective approaches to assess students' ability of solving math word problems. Through the data collected from a timed online math quiz and an online questionnaire survey, the present study suggests that math word problems that are not presented in participants' first language will be likely to cause disturbances to their process of solving the problems, but even when the questions are not presented in participants' first language, the difference in the length nor the complexity of sentence structure of the questions would greatly affect their understanding of the context. Participants in general have the perception that their English reading comprehension ability is a critical factor that affects their performance of solving math word problems, but surprisingly, even though the participants could extract accurate information from the question context, they might not be able to apply the appropriate math model to solve the problems.