Why AI Needs a Multimodal Data Lake
AI products cross the usefulness line once they can search the messy corpus a company already has: video, images, documents, metadata, raw data, and the embeddings that make it searchable. Chang She is CEO and co-founder of LanceDB, an AI-native multimodal lakehouse for search and retrieval. He knows the research-to-production gap firsthand — where teams rewrite code, copy data, transform formats, and debug brittle pipelines just to move from experiment to serving. This conversation starts in a Waymo on the way to Burlingame, then quickly turns into a sharp systems argument: search infrastructure now dictates AI product quality.
In our conversation, we discuss:
- Why curation, EDA, RAG, and training all collapse into the same search pattern
- Enterprise multimodality in three layers: varied data, varied workloads, and the offline-to-online handoff
- The Kung Fu Hustle rule for AI infra: iteration speed determines who wins
- LanceDB’s shared-data architecture versus the Elasticsearch-style shared-nothing world
- In the alpha zone, why image and video search demands multiple embeddings, full-text indexing, metadata, and raw data together
- How the Bitter Lesson meets real embedding models, and why one latent space misses query-specific nuance
- Why pgvector’s query planner can turn indexed vector search into brute force
- What superhuman queries look like when agents fire dozens or hundreds of searches in parallel
Three takeaways from this conversation:
1. Search is becoming the product moat. Chang’s point is blunt: most teams are not differentiated by whether they can call the same frontier models as everyone else. The leverage is in how well the product can search, retrieve, and reason over the company’s own messy corpus.
2. Multimodal is an architecture problem before it is a file-type problem. The hard part is not just “support video” or “support images.” It is keeping raw data, metadata, embeddings, full-text indexes, experiments, and serving systems close enough that teams can iterate without rewriting code and copying data every time they move from research to production.
3. Agents make search systems weirder and more important. A person might run one query; an agent might fire dozens or hundreds, across different embeddings, filters, permissions, and scopes. Chang has a hypothesis that the winners will be the teams that can make that loop fast, correctly scoped, and flexible enough for many different search lenses.
