Comparative Judgement (CJ) is an increasingly widely investigated method in assessment for creating a scale, for example of the quality of essays. One area that has attracted attention in CJ studies is the optimisation of the selection of pairs of objects for judgement. One approach is known as adaptive comparative judgement (ACJ). It has been claimed in the literature that ACJ produces very high reliability, often higher than can be obtained by conventional marking. Bramley showed by simulation that adaptivity can substantially inflate the apparent reliability in ACJ. The empirical study described here compared an adaptive with a non-adaptive CJ study using GCSE English essays. An all-play-all set of comparisons of a subset of the essays allowed the extent of scale inflation to be quantified: the reported adaptive reliability was 0.97 whereas the deflated value was 0.84. The value from the non-adaptive study was 0.72. However, the scale from the non-adaptive study correlated slightly higher with external variables, suggesting the non-adaptive study was no less valid than the adaptive one.