Successful testing programs rely on high-quality test items to produce reliable scores and defensible exams. However, determining what statistical screening criteria are most appropriate to support these goals can be daunting. This study describes and demonstrates cost-benefit analysis as an empirical approach to determining appropriate screening criteria for a given testing program and purpose. Using a certification exam's item pool and simulation we illustrate how to examine a wide range of screening criteria and reach an acceptable balance between the number of items screened out (cost) and pass/fail classification accuracy (benefit).