Discussion of a new method of index term dictionary compression in an inverted-file-oriented database highlights a technique of word coding, which generates short fixed-length codes obtained from the index terms themselves by analysis of monogram and bigram statistical distributions. Substantial savings in communication channel utilization are…

Reports algorithm for calculation of term discrimination values that is sufficiently fast in operation to permit use of exact values. Evidence is presented to show that relationship between term discrimination and term frequency is crucially dependent upon type of inter-document similarity measure used for calculation of discrimination values. (13…

Zipfian laws for frequency distributions of word pairs and longer phrases are derived from text sample analysis. From crossing of Zipfian curves, it is deduced that number of multi-word phrases that occur frequently in text is surprisingly small, of same order of magnitude as number of individual word-types. (8 references) (EJS)