The COVID-19 pandemic severely disrupted assessment models that were commonplace in the testing industry for decades. As a response to this disturbance, remote proctoring has emerged as a promising and potentially sound alternative to offer examinations, while adhering to public health authority guidelines. However, validity evidence in support of such a delivery modality with high-stake examinations, in comparison with traditional in-person testing, is still limited. The purpose of this study was to evaluate, compare and share lessons learned from the delivery of a high-stake medical licensing examination that was offered under two proctoring modalities. Over a few months, a total of 2070 candidates (46%) completed the exam in test centers, while 2475 (54%) completed it remotely. The analysis of technical difficulties suggests that the candidate experience was different across the two conditions. However, findings pertaining to candidate performance were encouraging and suggest that these discrepancies did not impact exam or examinee characteristics in any meaningful fashion. We believe that our findings therefore provide some support for the use of remote proctoring as a defensible alternative for completing a high-stakes examination.