Data Discovery in Data Lakes - Ziawasch Abedjan

As data lakes have become a prominent foundation for enterprise and scientific data management, organizations increasingly face the challenge of locating relevant datasets and building ad-hoc integration pipelines across heterogeneous, poorly documented, and rapidly evolving data collections. In this setting, data discovery becomes a critical capability for turning raw, distributed data assets into usable knowledge.

This book examines data discovery and its evolution across industry and academia. It covers the principles, systems, and techniques that enable users to find, understand, and use relevant data across increasingly complex data ecosystems. The book discusses modern approaches to efficient and effective data discovery, including novel system architectures, search and matching methods, metadata use, dataset profiling, and human-in-the-loop techniques.

Beyond core technical concepts, the book offers insight into how data discovery systems are evaluated and benchmarked. It highlights practical challenges faced in real-world deployments, compares emerging academic and industrial approaches, and identifies open research questions that continue to shape the field. The book is intended for researchers, practitioners, and students interested in data management, data integration, data lakes, and the future of intelligent data access.