In the present paper, we assess whether website rating systems are useful for selecting educational apps for preschool age children. We selected the 10 highest scoring and 10 lowest scoring apps for 2-4-year-olds from two widely used websites (Good App Guide; Common Sense Media). Apps rated highly by the two websites had a higher educational potential as assessed by a validated questionnaire for evaluating the educational potential of apps and were more likely to include a learning goal and feedback compared to low scoring apps. However, high scoring apps scored on average just 9/20 for indicators of educational potential, and both high and low scoring apps had poor language quality as determined by psycholinguistic and construction type analyses. We argue that website rating systems should also include quality of feedback, adjustable content, social interactions, storyline and a more fine-grained analysis of language in their assessments.