Abstract The purpose of this study is to investigate the validity of tweets about scientific publications as an indicator of societal impact by measuring the degree to which the publications are tweeted beyond academia. We introduce methods that allow for using a much larger and broader data set than in previous validation studies. It covers all areas of research and includes almost 40 million tweets by 2.5 million unique tweeters mentioning almost 4 million scientific publications. We find that, although half of the tweeters are external to academia, most of the tweets are from within academia, and most of the external tweets are responses to original tweets within academia. Only half of the tweeted publications are tweeted outside of academia. We conclude that, in general, the tweeting of scientific publications is not a valid indicator of the societal impact of research. However, publications that continue being tweeted after a few days represent recent scientific achievements that catch attention in society. These publications occur more often in the health sciences and in the social sciences and humanities.
Stored only when both papers are in the corpus — 23 citations exist upstream.