Product Quantization Tutorial

Summary

In this video, we talk about a vector compression technique called Are you struggling with high-dimensional data in your vector database? In this video, we dive deep into Vector similarity search can require huge amounts of memory. Indexes containing 1M dense vectors (a small dataset in today's ... How do we store millions of AI vectors without using massive storage? In this video, I explain how Unlike tree-based indexes used for ANN, a k-NN search with a Authors: Young Kyun Jang, Nam Ik Cho Description: Image retrieval methods that employ hashing or vector 11:20 Scalar Quantization 23:30 Locally-Adaptive Quantization 31:56 100 million vectors × 3072 dimensions × 4 bytes = 1.2 terabytes. That's just the vectors. Not the metadata, not the index. And ... Jegou, Herve, Matthijs Douze, and Cordelia Schmid. " Full-precision embeddings can exhaust memory — learn how FAISS IVF-PQ and linear Integrated Circuits playlist : youtube.com/playlist?list=PL4xnVegekvA1yZaWtAevOvc51Ufz9K17T VLSI Design ...