Frequency in Incidental Vocabulary Acquisition Research: An Undefined Concept and Some Consequences

Abstract

The most commonly examined variable in the growing body of literature on incidental second language (L2) vocabulary acquisition is frequency of exposure to target words. Earlier studies even ventured claims for minimum or optimal frequency of exposures needed in order for learners to acquire target words. More recent research addresses more nuanced questions with frequency still figuring prominently among the factors examined. In this article we show that the notion of frequency is less than straightforward, that it has unintentionally been put to use differently by different researchers, and that this difference across studies matters in several ways. We also elucidate specifically why it matters and suggest ways of bringing some coherence to the divergent practices within the research community. Implicit in any research that takes frequency as a variable are notions of identity across multiple tokens, that is, notions of what constitutes the same word encountered more than once. For any given text, the frequency of a word is the number of its tokens occurring in the text. Tokens, however, are not in and of themselves but are tokens by virtue of what they are tokens of, that is, by the type they betoken. We want to suggest that it is in this construed relation of token to type where differences can arise in research practice and that these differences matter. Say, for example, that I encounter the string create and the string creation each once on a page of running text. Have I encountered two tokens of the same single type, of the word create, cast broadly to include its derivational variants? Or have I encountered one token each of two different words, two different types, one the lemma create and the other the lemma creation? This depends on how abstractly or broadly I construe the notion of word as a type. Again, we will show that researchers diverge in exactly these ways of construing word type and, moreover, that such divergence is left implicit. In addition to yielding incommensurable acquisition research results, such implicit differences also make it impossible to address a basic question about the learners' perspective, one that has so far been left unasked. Do learners consider encounters with tokens to be repeated exposures to the same word or disparate exposures to different, unrelated words? While some researchers counting frequency of exposure may consider derivational variations within a word family to be tokens of the same word, tokens of one type, it is not clear whether learners do; whether, for example, they have the morphological wherewithal to detect these variants as repeated exposures to the same word. For learners who do not, we should ask what that says about calculating frequency in ways incommensurate with their perceptions. For learners who do, we should ask what is the evidence that tells us they do. In this article we show how practices in the operationalization of repeated tokens of target words vary across studies reported in the literature, and we trace some of the consequences of these divergent practices for evaluating the significance of this growing body of literature. We then recommend a way of operationalizing the counting of repeated tokens in the hope of standardizing how the concept of frequency is defined in this research community. Incidental vocabulary acquisition from reading refers to words being acquired while learners are reading for purposes other than learning vocabulary. Hulstijn (2001) defines incidental vocabulary acquisition in terms of experimental research design, suggesting it occurs when research subjects are asked to read a text without being made aware they will subsequently be assessed on vocabulary knowledge. Any words acquired during the reading are assumed to have been acquired incidentally. A small sampling of past studies researching incidental vocabulary acquisition from reading show they were based on different text genres including sentence-long contexts (Chen & Truscott, 2010; Heidari-Shahreza & Tavakoli, 2012; Webb, 2007, 2008), specially constructed paragraph readings (Negari & Rouhi, 2012; Rott, 1999), dramatic plays (Dupuy & Krashen, 1993), short stories (Day, Omura, & Hiramatsu, 1991; Hulstijn, Hollander, & Greidanus, 1996; Tekmen & Daloğlu, 2006; Zahar, Cobb, & Spada, 2001), expository texts (Hulstijn, 1992), comic books (Horst & Meara, 1999), academic texts (Vidal, 2011), graded readers (Brown, Waring, & Donkaewbua, 2008; Horst, Cobb, & Meara, 1998; Pigada & Schmitt, 2006; Waring & Takaki, 2003), and novels (Kweon & Kim, 2008; Lehmann, 2007; Pellicer-Sánchez & Schmitt, 2010; Pitts, White & Krashen, 1989). The results of these studies suggest varying degrees of correlation between frequency of occurrence of target words in the reading texts and their acquisition by subjects. Findings generally indicate modest amounts of vocabulary growth with estimates of the number of exposures to target words needed to acquire them ranging from 6 to more than 20 exposures, depending on the study (Waring & Nation, 2004). Target word frequency of exposure has been the main variable investigated. A primary motivation for this article, however, is the near-total absence from this growing literature of any descriptions of how frequency was calculated; that is, how researchers counted the tokens of target words. Only 2 out of 2511 Some articles report more than one study. studies cited above give any indication of how target word frequencies were calculated (Horst & Meara, 1999; Kweon & Kim, 2008). Attempts were made through e-mail to contact the authors of the remaining 20 articles cited above to seek additional information regarding how frequency was operationalized for their studies. We received responses from authors of 13 of the 20 articles. For three articles authors said they counted only the exact word form (M. A. Heidari-Shahreza, personal communication, June 3, 2013; M. Rouhi, personal communication, June 1, 2013; S. Rott, personal communication, July 7, 2013); for four articles the authors also counted inflected forms (J. Truscott, personal communication, May 31, 2013; S. Krashen, personal communication, July 8, 2013; S. Webb, July 13, 2013); for two articles authors counted both inflected and derived forms (T. Cobb, personal communication, June 12, 2013; A. Pellicer-Sánchez, personal communication, June 31, 2013); the data could not be located for three articles (J. Hulstijn, personal communication, January 12, 2013; N. Schmitt, personal communication, June 11, 2013); and finally one author said frequency was operationalized differently depending on the target word (M. Horst, personal communication, July 9, 2013). The responses to the e-mail inquiries indicate that incidental vocabulary acquisition researchers have differentially operationalized repeated tokens of target words. The decision for researchers to differentially count the tokens of target words may be indicative of their stance on L2 readers being more or less equipped at associating related forms of a target word as a single lemma. When Nagy and Anderson (1984) estimated the number of lexical items acquired by native speakers during a school year, they found it helpful to create word categories based on how dictionaries have traditionally treated words. Inflectional variants such as crash, crashing, and crashed have been treated as variants of the same word in many dictionaries under the headword crash. From an instructional point of view, an L2 learner that has mastered the basics of English verb inflection should not have problems associating the three forms. The same holds for plural inflection of countable nouns, such as daughter and daughters. On the other hand, dictionaries may or may not take additional types of relatedness into consideration. Some dictionaries list or cross-reference words with semantically transparent derivatives, associating for example fear and fearful. From the researchers' point of view, the ability to associate inflected and derived variants and vice versa is apparent; however, when considering the point of view of L2 readers, researchers should first ask what work must be done by an L2 reader in order to associate one form of a word (e.g., create) with a related form of that word (e.g., creation). For the L2 reader, is this one or two words? Brown (2013), for example, has drawn attention to the fact that the use of word family as a unit of counting frequency could be problematic in "that the application of word-building knowledge may have a limited role [for learners] and instead the frequency of individual word forms themselves may determine whether [words] are problematic … or not" (p. 1050). Moreover, Ward and Chuenjundaeng (2009) found L2 learners had difficulties associating related forms of words, especially words with classical or latinate etymologies. Previous incidental vocabulary acquisition studies may have adopted unclear criteria for relatedness when identifying recurrent tokens of the same target word. Lack of explicitness or clarity here, however, makes it impossible to examine, for example, the relation of morphological processes to acquisition that could potentially account for certain anomalies in the results found in previous studies. Therefore, we believe it is worth examining whether the notion of word identity across tokens has been operationalized consistently within and across studies in the literature. Since the core notion is left implicit in this literature, we must approach the question indirectly. Thus, we adopt three plausible and widely acknowledged degrees of word form variation and re-analyze data from published studies. From strict to permissive, the three units of word form variation among target word tokens are: (1) exact word form (thus create and created are not tokens of the same word), (2) lemma (thus create and created are tokens of the same word), (3) word family (thus create, created, and creativity are variants of the same word). From the target texts used in the studies, we generate three frequency counts, one for each of these construals of word form variation, and compare these counts with those reported in the published studies. Through this post hoc analysis we uncover a lack of consistency both within and among studies in how word frequency is calculated and trace some potential consequences of this inconsistency. Two criteria were used to identify studies to be included in our analysis. First, only published studies were included. Second, only studies reporting the frequency and/or acquisition rates for individual target words were included. It was only for these studies that we could determine both what construal of word type was used in calculating frequencies and the acquisition results for those words. Horst et al. (1998), for example, report individual target word data. A frequency of 12, for example, is reported for the target word furmity with 16 out of 34 participants having acquired it. Horst et al.'s (1998) study allows us to search the target text and confirm whether the 12 occurrences of furmity include inflected and derived variations of this target word. In contrast, while Kweon and Kim (2008) report that participants showed a 24.83% acquisition of the nouns appearing in a 7–19 frequency band, from their data it cannot be determined exactly which words were acquired and how many participants acquired them. Nor can we examine, as we can with the Horst et al. (1998) study, whether they allow for inflectional or derivational variation among tokens of the same word. A total of 6 out of 25 studies were found to meet our selection criteria. Although this leaves an admittedly small data set for further analysis, this fact in itself makes one of our main points: the inclusion of explicit data on individual word frequencies and acquisition rates in published research is regrettably rare. In recommending that researchers report individual target word data we have not overlooked the fact that publishers may limit the space provided to authors. Upon examination of 22 published articles we found that space did not seem to be the reason for not including individual acquisition results, because 14 of the articles provide the target words assessed but not the number of participants having acquired those target words. Furthermore, with the addition of online accessible supplementary materials, researchers can begin to make this type of data available to readers.22 The 22 articles reviewed are listed in an online supplement to this article. To examine whether practices of frequency counting vary across studies and what consequences follow if they do, we turn to the six studies that did supply the relevant data. Horst et al. (1998) report the frequency and acquisition rates for 45 target words that appear in the 21,232-token graded reader Mayor of Casterbridge (Jones, 1979). Zahar et al. (2001) and a replication of Zahar et al.'s (2001) study by Tekmen and Daloğlu (2006) report the frequency and acquisition rates for the same 30 target words that appear in the 2,383-token short story "The Golden Fleece." Brown et al. (2008) report the frequency of 84 target words appearing in the 5,522-token novel One-Way Ticket (Bassett, 1991b), 5,765-token novel The Witches of Pendle (Akinyemi, 1994), and 5,415-token novel The Elephant Man (Vicary, 1989). Pellicer-Sánchez and Schmitt (2010) report the frequency of 34 target words appearing in the approximately 65,000-token novel Things Fall Apart (Achebe, 2001). Waring and Takaki (2003) report the frequency of 25 target words appearing in the 5,872-token novel A Little Princess (Bassett, 1991a). First, the tokens of the target words from target texts used in the published studies were counted and compared to cited frequencies. Three units of counting were used to generate three frequency counts for each target word: (1) exact word form, (2) lemma, and (3) word family. One of the authors and a research assistant independently counted and compared results for the three frequency counts of each target word to illustrate how differences in researchers' implicit definitions of target word repetition will generate different frequencies for the same target word. Next, we planned to use the individual target word acquisition results from the six studies to show how the use of frequency bands in reporting acquisition results can be misleading. Unfortunately, the acquisition results of target words from "The Golden Fleece" reported in Zahar et al. (2001) and Tekmen and Daloğlu (2006) refer only to participant posttest results (T. Cobb, personal communication, December 18, 2011; A. Daloğlu, personal communication, December 19, 2011) and the remaining three studies do not report acquisition results for individual target words. Therefore, only the individual target word acquisition results from Horst et al. (1998) could be used to highlight the possible disadvantages of using frequency bands to report target word acquisition results. A detailed analysis of the six studies that supplied the specific word frequency and acquisition data, using different construals of word type when counting target word tokens, yielded different frequency counts for target words. The tokens of the nine target words from "The Golden Fleece" that shift in frequency were examined to determine what degree of form variation was used in calculating target word frequency in the previous studies (see Table 1). It appears that inconsistent standards for counting were used by Zahar et al. (2001) and by Tekmen and Daloğlu (2006) in determining frequency for target words. For the words breath and fight only the exact word forms were counted. Yet, the frequency counts the authors cite for six words (centaur, crash, fearful, helmet, oracle, rush) are of lemmas. Furthermore, word family was used to count tokens for the target word ruler. Therefore, no consistent pattern of acceptance in variation of target word tokens could be determined. No clear pattern of accepted variation in word tokens could be found for 39 out of the 45 target words selected from Mayor of Casterbridge used by Horst et al. (1998) (see Table 2). Cited frequencies for five of the words (root, flame, swear, treat, wheat) could be arrived at only by counting tokens showing no inflectional or derivational variation, that is, tokens identical in word form. However, frequencies given for another 10 words (image, weary, cheek, willing, inquire, folk, lean, sorrow, burst, grain) were consistent only with counting tokens of lemmas, and the counts given for yet another 10 words (harvest, confuse, grave, affair, attempt, suffer, magistrate, maid, whisper, hay) could be arrived at only by counting tokens of word families. Furthermore, our counting found four words (stare, lean, inquire, confuse) with zero tokens appearing at the level of exact word form; this indicates that the form of the target word that appeared on the assessment and the form that appears in the target text were not the same. Two target words (lean, swear), both irregular verbs, have cited frequencies that were calculated using two different criteria. Tokens of leant were included for the frequency of the word lean but tokens of swore were not included for the frequency of the word swear. Lastly, we found eight words (ma'am, trade, skimmity, stare, dusk, relative, fellow, effect) having the same frequency for all three units of counting (exact word form, lemma, word family) but differing from the frequencies cited; no apparent reason attributable to variation in target word form could be determined. The tokens of the 35 out of 84 target words from the texts used by Brown et al. (2008) that vary in frequency were examined to determine what degree of form variation was used in calculating target word frequency (see Tables 3-5). From the alternative word forms cited, it appears Brown et al. counted word families to calculate frequency. Specifically, the counts given for five words (loud/ly, police, slowly, kind/ly, true/truth) could be arrived at only by counting tokens of word families. Still, some forms of target words that occurred in the text and were counted by the authors were not cited. For example, cited frequencies for 18 words (bag, bed, book/s, diamond, eye, friend, hole, holiday, horse, lady/ies, leg, nurse, picture, story/ies, tall/er, village, voice, woman/men) could be arrived at only by counting tokens of lemmas. The authors counted inflected forms for all 18 words but the alternative forms were indicated for only five of these words. Furthermore, the authors indicated alternative forms for three target words (day/s, door/s, quick/ly) whose inflected or derived forms did not occur in the text; the cited frequency counts for two of these words still matched ours. We also found three words with derived forms occurring in the texts whose frequency counts did not match those cited; the cited frequency for happy differed because the tokens of happily were not counted by the authors. We could not determine why our counts differed from those cited for the other two words (noise/y, shop/keeper). Our counting also found three words (chair, eye, leg) with zero tokens appearing at the level of same word form; as with other texts analyzed, this indicates that the form of the target word that appeared on the assessment and the form that appears in the target text were not the same. Lastly, we found nine words (cab, chair, difficult, drink, hat, judge, prison, quick/ly, warm) having the same frequency for all three units of counting but differing from the frequencies cited; no apparent reason attributable to variation in target word form could be determined. The tokens of 13 out of 34 target words from the texts used by Pellicer-Sánchez and Schmitt (2010) that vary in frequency were examined to determine what degree of form variation was used in calculating target word frequency (see Table

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 73 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.