Numerous headlines have proclaimed just how far behind students have fallen since the COVID-19 pandemic first shuttered schools in March 2020. Many of these headlines report test score changes in "months (or years) of learning" from recent studies. Despite the accessibility (and therefore popularity) of the "months of learning" metric, it has some major downsides. In particular, the validity of the months of learning translation depends on the accuracy of the original effect size as well as the choices made in selecting the typical learning benchmark. In this brief, we walk through important questions that should be considered when selecting a typical learning benchmark. Our aim is to help researchers make more informed decisions about how to calculate months of learning. While we discuss these questions in the context of understanding important dynamics of the impact of COVID-19 on student learning, the underlying principles discussed apply more broadly to all education research using assessments to benchmark months of learning.