lance-format/lance
A columnar data format and lakehouse system providing vector search, full-text search, and random access optimized for AI/ML workloads.

Lance is an open lakehouse format for multimodal AI workloads. It provides a file and table format built on object storage that enables high-performance vector similarity search, BM25 full-text search, and SQL analytics on the same dataset. The format supports multimodal data including images, videos, audio, text, and embeddings, with integrations for ML tools like PyTorch, DuckDB, Polars, and PyArrow. It includes built-in versioning for datasets and supports hybrid search combining multiple retrieval methods.