Zipf's law unzipped

Abstract

Why does Zipf's law give a good description of data from seemingly completely unrelated phenomena? Here it is argued that the reason is that they can all be described as outcomes of a ubiquitous random group division: the elements can be citizens of a country and the groups family names, or the elements can be all the words making up a novel and the groups the unique words, or the elements could be inhabitants and the groups the cities in a country and so on. A random group formation (RGF) is presented from which a Bayesian estimate is obtained based on minimal information: it provides the best prediction for the number of groups with k elements, given the total number of elements, groups and the number of elements in the largest group. For each specification of these three values, the RGF predicts a unique group distribution N ( k )∝exp(− bk )/ k γ , where the power-law index γ is a unique function of the same three values. The universality of the result is made possible by the fact that no system-specific assumptions are made about the mechanism responsible for the group division. The direct relation between γ and the total number of elements, groups and the number of elements in the largest group is calculated. The predictive power of the RGF model is demonstrated by direct comparison with data from a variety of systems. It is shown that γ usually takes values in the interval 1⩽γ⩽2 and that the value for a given phenomenon depends in a systematic way on the total size of the dataset. The results are put in the context of earlier discussions on Zipf's and Gibrat's laws, N ( k )∝ k −2 and the connection between growth models and RGF is elucidated.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 123 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.