The purpose of this paper is to highlight some psychometric cautions that should be observed when seeking to develop short form versions of tests. Several points are made: (1) score reliability is impacted directly by the characteristics of the sample and testing conditions; (2) sampling error has a direct influence on reliability and factor structure of scores; and (3) caution should be used when developing short forms of tests when non-normative samples are used. (Contains 3 tables and 32 references.) (Author/SLD)