School closures during the COVID-19 pandemic presented a threat to student learning and motivation. Suspension of achievement testing created a barrier to understanding the extent of this threat. Leveraging data from a mathematics learning software as a substitute assessment, we found that students had lower engagement with the software during the pandemic, but students who did engage had increased performance. Students also experienced changes in motivation: lowered mathematics expectancy, but also lower emotional cost for mathematics. Results illustrate the potential and pitfalls of using educational technology data in lieu of traditional assessments and draw attention to access and motivation during at-home schooling.