Social media platforms like Twitter offered an unprecedented window into the real-time thoughts, conversations, and interests of millions of people. Researchers developed a tool called Storywrangler that leveraged Twitter data to create an "instrument for understanding our world through the lens of social media."
Storywrangler analyzed over 100 billion tweets dating back to 2008 to detect trends in word usage over time. It broke down tweets into "n-grams" - sequences of one, two, or three words - and tracked how the usage frequencies of these n-grams changed on a daily basis across different languages.
This massive database allowed researchers to see how real-world events, from natural disasters to political movements, were reflected in the narratives that unfolded on Twitter. For example, Storywrangler revealed surging interest in climate-related terms during major storms and wildfires. And it captured the rapid rise and fall of hashtags associated with social justice protests. Beyond reacting to news, Twitter also mirrored more subtle cultural shifts, like the waxing and waning popularity of celebrities or diets.
Storywrangler went beyond tracking raw frequencies - it also quantified how widely information spread on social media through shares and reposts. This helped distinguish niche conversations from truly viral ideas. The researchers used "contagiograms" to visualize both the popularity and amplification of n-grams over time.
There were certainly limitations to the Twitter lens. The platform's user base skewed young, urban, and affluent compared to the general population. Bots and organized campaigns could artificially inflate interest in certain topics. And the meanings of words themselves evolved across the years.
But used carefully, Storywrangler offered an unparalleled window into the collective consciousness - recording not just major news events but also the mundane daily conversations of millions worldwide. It aimed to complement more traditional data sources like books and news archives. The researchers hoped Storywrangler would enable more data-driven computational social science to understand our fast-changing, digitally-connected world.
Source:
