Data Machina #202
Generative AI and Vector DBs. Against LLM Maximalism. Cult Cargo AI. LMQL. GPT-4 Tips & Tricks. BLOOMChat. OpenLLaMa. SuperAgent. BindDifussion. Tree of Thoughts.
Generative AI and Vector DBs. Vector DBs have been around for years, and have been used in search, NLP & ML apps. But now the LLMs boom has triggered a new wave of vector DBs. A typical use case is augmenting an LLM with a vector BD for “search.” This in theory sounds like straightforward, but in practice it’s not. Accuracy, latency, similarity, scale and yes…hallucinations are still issues in LLMs. I guess it helps to know how vector DBs work before you integrate them with an LLM.
A refresher on vector DBs. There’s a tendency to put search engines, search libraries, similar search, vector search and vector DBs in the same bucket. Perhaps it may help to have a refresher on vector DBs:
A Comprehensive Guide to Vector Databases A nice intro to vector DBs, main use cases and some -but not all- capabilities required by vector DBs.
Vector Databases - A Complete Introduction A great post describing the differences between vector DBs and vector search libraries, the advantages of vector DBs, and some technical challenges.
Not all vector DBs are made equal. There are many flavours of vector DBs. Some of them have subtle -but important- different key features. Here is a list of imo the most popular vector DBs:
Pinecone: A vector DB for building high-performance vector search apps
Weaviate: An AI-native vector DB
milvus: imo the “best” open source vector DB for scalable, fast similarity search
chroma: An AI native open-source embeddings DB
Vespa: vector, lexical, and structured data search, all in the same query
Quadrant: Advanced, high-performant vector similarity search
pgvector: An open-source vector similarity search for PostgresSQL
Vald: A highly scalable distributed vector search engine
Want to know more? Read this detailed comparison of Milvus, Pinecone, Vespa, Weaviate, Vald, GSI and Qdrant.
Similarity search, kNN and ANN algos. A key feature of vector DBs is similarity search, which finds the k-nearest vectors to a query vector, as measured by a similarity metric. But kNN is computationally super expensive. eBay has developed a blazingly fast billion-scale vector similarity engine that uses two algos called HNSW and ScaNN.
Similarity and measuring distance in vector DBs. Vector DBs come with built-in distance measures to calculate similarity. But there are many different distance measures, each one with pros & cons. Choose carefully. This is a good read: The advantages and pitfalls of 9 common distance measures. Interested in hardcore distance measures? Checkout on distance, divergence, and dissimilarity.
Vector compression and quantisation. Another key feature of vector DBs is to use vector compression techniques to reduce the storage space and improve the query performance. This is a cool post on Vector DBs 101 - Scalar Quantisation and Product Quantisation
Indexing, in-context learning and vector DBs. In-context learning is a few-shot learning prompting technique that allows a model to process examples before attempting a task. Indexify - is a knowledge, memory retrieval, and indexing service that facilitates in-context learning of LLMs. It provides relevant context in a prompt with SOTA embedding models and pluggable vector stores
Neural search and LLMs require more efficient NLP algos. To address that, StanfordNLP released String2String, a new open source library of efficient NLP algorithms for pairwise string alignment, distance measurement, lexical and semantic search, and similarity analysis. And FacebokAI FAISS still is one of the best libraries for efficient similarity search and clustering of dense vectors.
Vector DBs, generative AI content and hallucinations. An example of retrieving information from Weaviate to prompt a generative model and then vectorising and saving thee generated content back into the DB. And also how to avoid hallucinatory results. Check out Generative Feedback Loops with LLMs for Vector Databases
Have a nice week.
10 Link-o-Troned
the ML Pythonista
the ML codeR
Deep & Other Learning Bits
AI/ DL ResearchDocs
data v-i-s-i-o-n-s
El Robótico
MLOps Untangled
AI startups -> radar
ML Datasets & Stuff
Postscript, etc
Tips? Suggestions? Feedback? email Carlos
Curated by @ds_ldn in the middle of the night.