An assessment for learning approach is foundational for student learning. The necessity to shift teaching and learning online as a response to COVID-19 has propelled digital assessment into the mainstream within higher education institutions. User experience is a common indicator of effectiveness of technologically enhanced initiatives; however, measuring the pedagogical impact of digital assessment initiatives has not been straightforward. This paper presents a pedagogical evaluation, incorporating staff and student perspectives, of a digital assessment platform (DAP) at a large Australian metropolitan university. Data were collected using surveys, interviews, focus groups and case studies and incorporated findings from student evaluations of courses and a student-staff partnership project. Results highlight strong pedagogical reasons for continuing the digital assessment initiative, while also identifying constraints and opportunities for improvement. The insights generated through this evaluation emphasise the importance of considering technical, pedagogical and contextual factors and the interplay between the factors. In a quality assurance climate, where the demand for this type of institution-wide evaluation is increasing, this study contributes an approach that addresses some of the complexities and challenges of evaluating digital assessment initiatives in higher education. Future research is needed to understand more fully the relationships between pedagogical and contextual factors when undertaking an evaluation.