Foundation Models & Friends - A Cambrian Explosion. I suspect that the open sourcing of Stable Diffusion by Emad’s team @Stability.ai has triggered a sort of a Cambrian explosion of pre-trained LLMs and foundation models.
It looks like self-supervised learning models pre-trained with massive unlabelled data and billions of parameters are “winning.” This is what happened just in the last 10 days:
Meta AI released Make-a-Video: a state-of-the-art model that uses spatiotemporal attention tensors to generates videos from text. Very cool.
Google Brain published DreamFusion: a denoising diffusion model that generates 3D images using text prompts. Check out the demo. Impressively requires no 3D training data.
A group of anonymous researchers ;-) released a paper on Phenaki- a new causal model that generates variable length videos from text prompts. Check out the demo. Nice!
Facebook AI open sourced MEGA: Moving Average Equipped Gated Attention a model that (they claim) achieves SotA results in Language Modelling, Speech Classification…
Google AI released Talk-to-Books, basically voice querying of book passages and quotes. They’ve open sourced the pre-trained language model and universal encoder here.
OpenAI open sourced Whisper - a multi-lingual, encoder-decoder transformer that transcribes and translates speech audio into text. It’s super easy to pip install and run in the cli. Try it, it’s very good.
Amazon Science introduced a new method to train and fine tune large AI models with 1 trillion parameters. Crazy!
At Tesla AI day, the AI Team presented the new Full Self Driving (Beta) model that interestingly has pivoted to using transformers, attention and LLMs approaches.
It’s really mind boggling the potential of all these new large, pre-trained AI models.
Have a nice Sunday.
10 Link-o-Troned
A Pythonista *Experience*
Scripting aRt
data v-i-s-i-o-n-s
Distributed de-Entangler
Forschung!
Algorithmic Potpourri
Robots & Cyborgs like <you>
Deep & Other Learning Bits
startups -> radar
ML Datasets & Stuff
Postscript, etc
Tips? Suggestions? Feedback? email Carlos
Curated by @ds_ldn in the middle of the night.