HPC scientists are producing unprecedented volumes of data that take a long time to load for analysis due both to the size of the data and the I/O bottleneck. In this talk we discuss one possible solution: descriptive metadata management. If we can store information (descriptive metadata) about features of interest in the data then for all subsequent analyses we can use this information to only read in the data containing these features of interest. This can result in a dramatic reduction in the volume of data that scientists have to read in, thereby greatly accelerating analysis. This talk will explore the opportunities, challenges, and state-of-the practice for descriptive metadata management, and will introduce EMPRESSA, a general descriptive metadata management solution that is able to speed up analysis by orders of magnitude by leveraging spatial indexing and RDBMS technologies.
