Data Machina #182
Neural story generation. 2022 top AI papers.Summarisation with Google Flan-T5. Training a LM with 1 Single GPU. Next-gen MLOps. Transformers mechanistic interpretability. Compiling DL models.Chaos ML.
On Neural Story Generation. I guess the main storyline for AI in 2022 was: What a year of mind boggling, amazing, blazingly fast progress in all AI fronts! I reckon 2023 is gonna be even more awesome. But now let me tell you about generating stories with LMs.
Back in November, a team @MaxPlanckInstitute stated that long story generation is vital for the progress of AI. In What is Wrong with Language Models that Can Not Tell a Story? they argue that there are no foundations for long, coherent, story generation with LMs.
Just in mid-December, a team @GaTech introduced a new approach to generate coherent story plot-lines using a large pre-trained LM (GPT- J-6B) See paper: Neural Story Planning
Days later, a team @UWaterloo published Dynamic Story Generation with Large Pre-trained Language Models. A new method for fine-tuning a pre-trained generative transformer that attends to an encoded future plot event.
On that same day, researchers @UCBerkley @MetaAI & @UCLA, very impressively have come up with imo a SoTA approach that automatically generates longer, coherent, relevant, interesting stories of 3500+ word length (see paper & repo)
And just yesterday @jaesungtae released StoryTeller, a multimodal AI story teller. Simply amazing! Given a prompt as an opening line of a story, GPT writes the rest of the plot; Stable Diffusion draws an image for each sentence; a TTS model narrates each line, resulting in a fully animated video of a short story, replete with audio and visuals. Just wow!
Having a lazy New Year’s Day? Here are some pastimes suggestions:
Write a short story with ChatGPT. I fed it with: story elements, story type, characters, and location. And it generated a pretty decent story. Try it. Also see: How to compose a story using ChatGPT 3.5
Deeply interrogate ChatGPT and see if it passes the Blade Runner’s Voight-Kampff Test
Play the Is it AI or Art game? Can you get a 100% perfect score? Don’t get too overconfident
Try to write a short book with ChatGPT. Gautier has done it, checkout: A Machine Learning Approach to Quantitative Trading
Wishing you a happy, healthy and prosperous Y2023! Have a nice week.
10 Link-o-Troned
A Pythonista *Experience*
Scripting aRt
Deep & Other Learning Bits
ResearchDocs
El Robótico
data v-i-s-i-o-n-s
DataEng Wranglings
AI startups -> radar
ML Datasets & Stuff
Postscript, etc
Tips? Suggestions? Feedback? email Carlos
Curated by @ds_ldn in the middle of the night.