Topic Shift in Efficient Discourse Production

Abstract

Speakers have been hypothesized to organize discourse content so as to achieve communicative efficiency.Previous work has focused on indirect tests of the hypothesis that speakers aim to keep per-word entropy constant across discourses to achieve communicative efficiency (Genzel & Charniak, 2002).We present novel and more direct evidence by examining the role of topic shift in discourse planning.If speakers aim for constant per-word entropy, they should encode less unconditional per-word entropy (as estimated based on only sentenceinternal cues) following topic shifts, as there is less relevant context to condition on.Applying latent topic modeling to a large set of English texts, we find that speakers are indeed sensitive to the recent topic structure in the predicted way.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 58 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.