Evaluating the impact of Academic Literacies teaching (Lea and Street [1998. "Student Writing in Higher Education: An Academic Literacies Approach." "Studies in Higher Education" 23 (2): 157-72. doi:10.1080/03075079812331380364]) is difficult, as it involves gauging whether writers: (1) gain better understanding of what influences written social interactions; and (2) improve their ability to manipulate language to address readers. Self-report can assess the first; the second can only be evaluated by examining texts and their effect on readers. Texts are commonly assessed with rubrics-based tools focussing on textual features, but their insensitivity to communicative context and readers' perception makes these inappropriate for an Academic Literacies framework (Canton [2018. "'It's Hard to Define Good Writing, but I Recognise it When I See it': Can Consensus-Based Assessment Evaluate the Teaching of Writing?" "Journal of Academic Writing" 8 (1): 13-27. doi:10.18552/joaw.v8i1.450]). Consensual assessment used by Amabile ([1996. "Creativity in Context." Boulder, CL: Westview Press]) for creativity, offers a potential solution (Canton [2018. "'It's Hard to Define Good Writing, but I Recognise it When I See it': Can Consensus-Based Assessment Evaluate the Teaching of Writing?" "Journal of Academic Writing" 8 (1): 13-27. doi:10.18552/joaw.v8i1.450]). This paper presents a new instrument based on consensual assessment and empirically tests it. Intra-class Correlation Coefficients (ICCs) found moderate (Koo and Li [2016. "A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research." "Journal of Chiropractic Medicine" 15 (2): 155-63. doi:10.1016/j.jcm.2016.02.012]) to good (Cicchetti [1994. "Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology." "Psychological Assessment" 6 (4): 284-90. https://pdfs.semanticscholar.org/50d7/f68422d0c0424674f6b235ac23be8300da38.pdf]) agreement among raters, which offers proof of concept for capturing the readers' perception of the complex interactions in writing.