In mathematics education, the research on proof by contradiction (PBC) often claims that this activity is more difficult for students than direct proof, or simply difficult in general. Many hypotheses have been offered to support or explain this belief, yet they span a disorientingly wide swath of journal articles, conference papers, dissertations, book chapters, etc. In addition, few attempts have been made to organize these hypotheses or carefully test them. In this paper, we conduct a thorough literature review on PBC, organize existing hypotheses about challenges with PBC into a Hypothesis Framework for (Students' Difficulty with) Proof By Contradiction (HFPBC), discuss the state of research related to each hypothesis, and offer thoughts on the future study of these hypotheses.