In this dissertation, I establish a research program that uses computational modeling as a testbed for theories of phonological learning. This dissertation focuses on a fundamental question: how do children acquire sound patterns from noisy, real-world data, especially in the presence of lexical exceptions that defy regular patterns? For instance, Turkish infants tune into vowel harmony patterns as early as six months, despite lexical exceptions from disharmonic loanwords. This dissertation demonstrates that phonological learning is facilitated by two essential elements: (1) a restrictive hypothesis space defined by formal language theory and (2) an exception-filtering mechanism. I developed a learning model that harnesses the discrete nature of categorical grammars to filter out lexical exceptions based on statistical criteria adaptmed from probabilistic models. This hybrid model learns interpretable grammars that approximate acceptability judgments in behavioral experiments, demonstrating robust performance across various real-world corpora from English, Polish, and Turkish. Moreover, the dissertation integrates the proposed phonotactic model into learning morphophonological alternations. This approach is not only competitive on real-world corpora but also substantiated by experimental evidence. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]