Using Statistics in Lexical Analysis
Abstract
This chapter discusses a number of statistical tools and shows some examples of how these tools can be used to enhance productivity of a human trying to solve one of these problems. It begins by introducing two statistics, mutual information and the t-score. British lexicographers, especially those working on dictionaries for foreign learners, are beginning to depend heavily on machine-readable corpora. They use these corpora and the basic concordancing tool mentioned above to fill in detailed syntactic descriptions. The notion that a language consists of a fixed set of rules and a finite set of lexical items, and that they can be learned by students, who will then “know” the language, is too attractive to be easily dislodged, but it has been responsible for some misconceptions inter alia about the status of infelicities typically produced by foreigners, which are sufficient to identify the speaker as foreign, although not actually wrong.
Cite this
Kenneth Church, William A. Gale, Patrick Hanks, Donald M. Hindle (2021). Using Statistics in Lexical Analysis. Psychology Press eBooks. https://doi.org/10.4324/9781315785387-8
@article{kenneth2021using,
title = {Using Statistics in Lexical Analysis},
author = {Church, Kenneth and Gale, William A. and Hanks, Patrick and Hindle, Donald M.},
year = {2021},
journal = {Psychology Press eBooks},
doi = {10.4324/9781315785387-8}
}Citations in this library
Stored only when both papers are in the corpus — 459 citations exist upstream.