An understanding of acid-base reactions is necessary for success in chemistry courses and relevant to careers outside of chemistry, yet research has demonstrated that students often struggle with learning acid-base reaction mechanisms in organic chemistry. One response to this challenge is the development of educational applications to support instruction and learning. The development of these supports also creates an opportunity to probe students' thinking about organic chemistry reaction mechanisms using multiple modalities--i.e., using an app interface or the traditional paper-pencil. This study used think-aloud interviews conducted with undergraduate students in their first semester of organic chemistry to understand how they worked through two acid-base reactions using either paper-pencil or an app. Analysis of the interviews indicates that students from both groups recognize the steps of acid-base reactions, but do not always apply the underlying concepts, such as assessment of pK[subscript a] values or resonance, when determining how a reaction will proceed. The modality seemed to somewhat influence students' thinking, as the app prevented students from making chemically unreasonable mistakes. However, some students relied on the cues it provided, which could potentially be problematic when they are required to respond to assessments that do not provide these cues. Our results suggest that instructors should emphasize the conceptual grounding for the steps that govern acid-base reactions to promote chemical thinking about the relationships between the reaction components and how those influence reaction outcomes, as well as support students to think critically about the chemical information contained within the modalities they are using.