The semiotic spectrum

Abstract

Because humans cannot know one another’s minds directly, every form of communication is a solution to the same basic problem: how can privately held information be made publicly accessible through manipulations of the physical environment? Language is by far the best studied response to this challenge. But there are a diversity of non-linguistic strategies for representationwith external signs as well, from facial expressions and fog horns to chronological graphs and architectural renderings. The general thesis of this dissertation is that there is an impressively wide spectrum of conventional systems of representation, corresponding to the many ways that the problem of communication can be solved, and that these systems can be described and explained using the tools of contemporary mathematical semantics. As a partial corrective to the countervailing norm, this work concentrates on the class of systems arguably most different from language— those governing the interpretation of pictorial images. Such representations dominate practical communication: witness the proliferation of maps, road signs, newspaper photographs, scientific illustrations, television shows, engineering drawings, and even the fleeting imagery of manual gesture. I argue that systems of depiction and languages embody a parallel technologies of communication. Both are based on semantics: systematic and conventional mappings from signs to representational content. But I also provide evidence that these semantics are profoundly divergent. Whereas the semantics of languages are based on arbitrary associations of signs and denotations, the semantics of systems of depiction are based on rules of geometrical transformation. Drawing on recent research in computer graphics and computational vision, I go on to develop a precise theory of pictorial semantics. This in turn facilitates a detailed comparisonof iconic, image-based representation, and symbolic, language-based representation. A consequence of these conclusions is that the traditional, language-centric conception of semantics must be overhauled to allow for a more general semantic theory, one which countenances the wide variety of interpretive mechanisms actually at work in human communication.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 16 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.