Arithmetic fluency is regarded as a foundational math skill, typically measured as a single construct with pencil-and-paper-based timed assessments. We introduce a tablet-based assessment of single-digit fluency that captures individual trial response times across several embedded experimental contrasts of interest. A large (n = 824) cohort of 3rd-7th grade students (ages 7-13 years) completed this task, revealing effects of operation and problem size in "common" problems (i.e., 5 + 3) often examined in studies of mathematical cognition. We also characterize performance on "exceptional" problems (i.e., 4 + 4), which are typically included in fluency tests, yet excluded from most cognitive studies. Overall, individuals demonstrated higher fluency on exceptional problems compared to common problems. However, common problems better predicted standardized tests scores and exhibited distinct patterns of speed-accuracy tradeoffs relative to exceptional problems. The affordances of tablet-based assessment to quantify multiple cognitive dynamics within chained fluency tests present several advantages over traditional assessments, thus enriching the study of arithmetic fluency development at scale.