XLDB attempts to tackle challenges related to extreme scale data sets. The definition of extreme scale is a moving target, with current limits stretching into the exabyte range. Main activities include (a) identifying trends, commonalities and roadblocks related to building, managing and modeling extreme scale systems and analyzing extreme scale data sets, (b) bridging the gap between data-intensive users and solution providers, and (c) facilitating development and growth of appropriate technologies including (but not limited to) databases.
Thousands of people worldwide have participated in XLDB over the years. A subset of the community gathers each year at the annual XLDB event, typically organized at Stanford University in the Spring. There have also been international events held across the world.
Extremely large data systems, today storing many petabytes to exabytes of information, pose challenges beyond those of smaller installations. To address these challenges, in 2007, the Scalable Data Systems team at SLAC responsible for designing and building a 100-petabyte data access system for the astronomical survey called LSST organized an invitation-only workshop with three goals:
- Identifying trends, commonalities, and major roadblocks related to building extremely large databases.
- Bridging the gaps between extremely large database users, researchers, and solution providers.
- Facilitating development and growth of practical technologies for extremely large databases.
Invited participants included leading science and industry users, academic researchers, and solution providers. Since then, the XLDB event has grown to merit larger forums: in 2010 an open conference was introduced, drawing a crowd of 150, and the 2011 event drew a crowd of 300 attendees. Both events reached full capacity and registration had to be cut off early.
These events helped start several initiatives carried on by the community, including collecting use cases, and defining a science benchmark. The open-source SciDB project has been started as a result of the 1st XLDB workshop. A series of satellite workshops in Europe were also initiated, starting with the workshop in Edinburgh in 2011.
General Chair: Jacek Becla
Program Chair: Vaikunth Thukral
Committee: Kian-Tat Lim, Daniel Wang