What is Tokenization? Let's Explore, Using Novel AI's New Tokenizer as a Use Case

Researchers have developed a benchmark called the LAMBADA dataset to rigorously test how well AI models can leverage broader discourse context when predicting an upcoming word.

AI research from 2019 explored how to automatically generate reasonable suggestions for missing sections of text.

In 2019, researchers explored how artificial intelligence could use hierarchical models to improve computer-generated stories.

Books may seem like straightforward stories, but researchers are finding fascinating mathematical patterns hidden in the text.

Researchers developed a tool called Storywrangler that leveraged Twitter data to create an "instrument for understanding our world through the lens of social media."

New research explores how integrating structured knowledge into AI systems can enhance storytelling abilities.

New research from Stanford University demonstrates how "emotion maps" could improve story generation.

Back in 2018, researchers from Facebook AI developed a new method to improve story generation through hierarchical modeling. Their approach mimics how people plan out narratives.

New research from the University of California, Berkeley sheds light on one slice of these models' knowledge: which books they have "read" and memorized. The study uncovers systematic biases in what texts AI systems know most about.

Prosecraft, a data-driven project designed to analyze word usage and provide statistics on various writing-style markers, has recently come under fire from authors and subsequently shut down its operations.

Researchers from Tsinghua University and Wuhan University propose teaching AI systems to mimic authorial styles.