Data Machina #193

Craziest week in AI. Downscaling Large Models.Awesome GPT-4. GPT-4 jailbreaks. Standford Alpaca. Transformers.js. Google PaLM API. LLMs & SQL. GenAI Pytools. Dynamic Midjourney v5 prompts.

The Craziest Week in AI, and Downscaling Large Models. There’s this meme in which Captain Haddock asks: What a week, huh? and Tintin replies: Captain, it’s just Wednesday. That’s how I felt this week. Let me try to summarise all that happened in the craziest week ever in AI.

GPT-4 released. Social media has been inundated by AI pundits reporting that GPT-4: 1) Has now an up-to 32K token window size, 2) can engage in visual chat, 3) it’s better at reasoning and math, 4) has powerful OCR capabilities, and 5) has improved at coding. @taranjeet has summarised all the details and things that people are doing with GPT-4 in this awesome-gpt4 repo.

Some interesting, random GPT-4 stuff. @tyler decided to test GPT-4 on a particularly difficult “algorithmic” problem. He wrote about his findings in Can GPT-4 *Actually* Write Code?

@jacksonfall had a great AI business idea. He gave GPT-4 a budget of $100 and told it to make as much money as possible. Jackson just follows orders from GPT-4, which runs the business. Read this fascinating thread here.

@d_feldman tested GPT-3.5 and GPT-4 “reasoning capabilities” with the same prompt, and he concluded that GPT-4 does have a world model.

Not surprisingly -and quite quickly- @alexalbert came up with 3 clever prompt hacks that for the first time jailbreak GPT-4… see also this one, and this one too …

If you don’t have access to GPT-4, you can try it in Replit for free

A Storm of New AI Product Products & Projects. Google and Microsoft are engaged in an arms race to conquer office work with AI. MS introduced 365 Copilot & Business Chat (demo here). I’m no fan of MS but tbh that’s an impressive demo. Google also announced Generative AI for Google Workspace (demo here.)

Google & MS just killed hundreds of startups with those AI products. And I guess all these new AI office tools will super-empower certain office workers, and will also displace (remove?) many corporate middle managers, digital paper-pushers, and infomediaries.

Very interestingly, Google announced PaLM API & MakerSuite: An approachable way to start prototyping and building generative AI apps.

One of Google Brain’s latest papers ReAct: Synergizing Reasoning and Acting in Language Models has ignited a new wave of thinking re: prompt engineering, and LLMs composability, extensibility, and augmentation.

On that subject, @intrcnnctd wrote a great post about prompting and The surprising ease and effectiveness of AI in a loop. Instead of asking GPT to simply do smart-autocomplete on your text, you prompt it to respond in a thought/ act/ observation loop.

In a similar fashion, a few days ago Microsoft launched Semantic Kernel: An open source Framework for integrating LLMs in your apps. SK supports: prompt templating, chaining, vectorised memory, and intelligent planning. That sounds like a fire-shot at LangChain.

Relentless, the team @LangChain announced LangChain + Zapier Natural Language Actions (NLA) which enables you to automate LLMs with 5k+ apps and 20k+ actions. This is the new AI edge on If This Then That. Actual AI Process Automation with natural language, not the rather fake Robotic Process Automation (RPA) sold by the big consultancies.

Anthropic - the self-appointed leader in safe, harmless, and honest AI - announced Claude, a next-generation AI assistant based on Anthropic’s research. You can read more about Claude’s features and request access here (which I have.)

Assembly AI announced Conformer-1, a SoTA speech recognition model that achieves near human-level performance and robustness across a variety of data.

Finally, two other announcements. Midjourney v5 which improves almost every aspect of AI generated images, and Stable Diffusion Reimagine, a sort of img2img AI on steroids based on the new Clipdrop tool.

Downscaling Large Models. Frustrated by the lack of access to huge AI compute and closed large models from the Tech Titans, the AI community is massively pushing towards downscaling large AI models. IMO this has been triggered mostly by:

1. Meta AI open sourcing LLaMA, a smallish 65B param model that performs well

2. The leakage of all LLaMA model weights (see llama-dl repo)

3. Microsoft’s intro on LoRA: Low-Rank Adaptation of LLMs which drastically reduces the number of trainable parameters for downstream tasks

4. The excellent release of llama.cpp, inference of LLaMA in pure C/C++ (repo)

5. The crucial release of Stanford Alpaca, a cheap-compute model, fine-tuned from the Meta AI LLaMA 7B model that performs as well as Open AI text-davinci-003

Here are 7 projects that showcase this trend on downscaling large AI models:

• Local Stanford Alpaca, train it and run it on your own machine

• Meta AI LLaMa in your M1 Mac: How to run it in a few easy steps

• MiniLLM: Run modern LLMs on consumer-grade GPUs, just tiny and easy-to-use codebase mostly in Python

• Alpaca-LoRA: Run a model that performs like Open AI text-davinci-003 in a Raspberry Pi

• Int-4 LLaMa is not enough. A new way to lower LLMs RAM requirements and to easily build Python apps using faster LLM inference

• alpaca.ccp, run a fast ChatGPT-like model locally on your device

• cabrita-LoRA, we translated the Alpaca dataset to Portuguese, and running LoRA training, achieved ChatGPT-like performance, with only 16mb of LoRA weights

But wait! Because just two days ago, Stanford CRFM-HAI published this note in the Alpaca GitHub repo:

We thank the community for feedback on Stanford-Alpaca and supporting our research. Our live demo is suspended until further notice.

I hear from a colleague @StandfordNLP that this is due to “safety concerns and potential licensing issues…” Well, yesterday -just in case- @pointnetwork published point-alpaca or how to distill the model weights from Stanford Alpaca. I love the Internet!

OK if you’re not having enough AI stuff yet, here are 2 suggestions for a lazy Sunday:

• Chat to God now, powered by GPTChat

• Read Impromptu. Amplifying Humanity Through AI (pdf, 223 pages) a whole book written with GPT-4 by Reid Hoffman (early investor in Open AI, LinkedIn, PayPal)

Have a nice week.

10 Link-o-Troned

1. Deducing GPT-4 Architecture

2. ChatGPT as Code

3. Transformers.js - Run Transformers in the Browser

4. AI is Eating the World… ”Generative AI is … Not Enough?”

5. What is Temperature in NLP Models?

6. Large Language Models & SQL

7. awesome-totally-open-chatgpt

8. Training LLMs on Amazon SageMaker: Best Practices

9. Tutorial: Dynamic Prompting for Midjourney & Friends

10. [Fascinating] Dr. Bing’s New Intro to Neuroscience [32 vids]

---

the ML Pythonista

1. A New Way to Accelerate Diffusers with PyTorch 2.0

2. GenAI: Generative AI Tooling for iPython

3. Building a Multi-lang Movie RecSys App with Cohere Embeddings

the ML codeR

1. How I Boosted My Shiny Search App with GPT-3 Embeddings

2. Calibrating Opta’s xG (Expected Football Goals) Model

3. shinyDeepDR - DL for Cancer Drug Response with Genomic Data

Deep & Other Learning Bits

1. Google Vid2Seq: Visual-LM for Multi-event Videos Descriptions

2. LLMs & Embedding- Transformer Token Vectors Aren’t Points in Space

3. ViperGPT: Visual Inference for Reasoning with Python

AI/ DL ResearchDocs

1. Automatic Multi-step Reasoning & Tool-use for LLMs

2. Meet in the Middle: A New Pre-training Paradigm for LLMs

3. An Overview on Recent Developments in Language Models

El Robótico

1. LLM-Grop: Robotics Tasks & Motion Planning with LLMs

2. Q&A with Chief Technologist @AWS Robotics

3. A New Approach to Drone Navigation in Cluttered Environments

data v-i-s-i-o-n-s

1. Car Thefts are Rising. Is a TikTok Challenge to Blame?

2. WaPo Interactive - What Would your 4-day Week Look Like?

3. Generate the Famous GapMinder Dataviz with GPT-4 in 1 Attempt

MLOps Untangled

1. MLOps - Project Scaffold

2. Unified MLOps @Expedia

3. Automate ML Model Retraining & Deployment with MLflow

AI startups -> radar

1. Adept - A New Way to Use Computers with AI

2. Cast - AI for Cutting Kubernetes Costs in Half

3. Fairmatic - AI for Fleet Insurance

ML Datasets & Stuff

1. The COMICS TEXT+ Dataset for Comics Text Detection

2. 25k IMDb Movie Dataset

3. DCASE23 Anomalous Sound Detection Dataset

Postscript, etc

Tips? Suggestions? Feedback? email Carlos

Curated by @ds_ldn in the middle of the night.