Word Frequency, Information Theory, and Cloze Performance: A Transfer Feature Theory of Processing in Reading

Abstract

HYPOTHESIZES THAT CLOZE EASINESS (the percent of subjects filling in the correct word in a cloze task) is a measure of information as defined by Shannon and Weaver (1949). A theory is developed that information is a function of 2 variables: a) the number of lexical markers associated with the word, and b) the number of lexical markers supplied for the word through what Weinrich (1966) calls transfer features. The theory states that frequent words are low information words because they have few lexical markers. Rare words are high information words because they have many lexical markers; however, the literature of automated indexing (e.g., Carroll and Roeloffs, 1966) shows that rare words that are repeated in a text are closely associated with the topic of the text. It follows that rare words that are repeated in a text are low information words by virtue of extraordinary transfer feature support. The theory is shown to be consistent with a reanalysis of Bormuth (1966) data resulting from administering cloze tests to 675 elementary school subjects over 20 passages at 4 levels of difficulty. The theory is examined in light of 3 current models of reading. It is argued that the theory was consistent with some aspects of the models, that the theory may be more parsimonious than some aspects of the models, and that the theory suggests researchable questions which may refine, modify, or refute some aspects of the models.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 52 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.