Applying Chimera Virtual Data Concepts to Cluster Finding in the Sloan Sky Survey Export

@article{citeulike:3153510, abstract = {In many scientific disciplines \— especially long running, data- intensive collaborations \— it is important to track all aspects of data capture, production, transformation, and analysis. In principle, one can then audit, validate, reproduce, and/or re-run with corrections various data transformations. We have recently proposed and prototyped the Chimera virtual data system, a new database-driven approach to this problem. We present here a major application study in which we apply Chimera to a challenging data analysis problem: the identification of galaxy clusters within the Sloan Digital Sky Survey. We describe the problem, its computational procedures, and the use of Chimera to plan and orchestrate the workflow of thousands of tasks on a data grid comprising hundreds of computers. This experience suggests that a general set of tools can indeed enhance the accuracy and productivity of scientific data reduction and that further development and application of this paradigm will offer great value.}, address = {Los Alamitos, CA, USA}, author = {Annis, James and Zhao, Yong and Voeckler, Jens and Wilde, Michael and Kent, Steve and Foster, Ian}, citeulike-article-id = {3153510}, citeulike-linkout-0 = {http://doi.ieeecomputersociety.org/10.1109/SC.2002.10021}, citeulike-linkout-1 = {http://dx.doi.org/10.1109/SC.2002.10021}, doi = {10.1109/SC.2002.10021}, issn = {1063-9535}, journal = {sc}, keywords = {applying, chimera, cinding, in, sky, sloan, survey}, posted-at = {2008-08-25 11:19:36}, priority = {5}, publisher = {IEEE Computer Society}, title = {Applying Chimera Virtual Data Concepts to Cluster Finding in the Sloan Sky Survey}, url = {http://dx.doi.org/10.1109/SC.2002.10021}, volume = {00}, year = {2002} }

TY - JOUR ID - citeulike:3153510 L3 - citeulike-article-id:3153510 N2 - In many scientific disciplines — especially long running, data- intensive collaborations — it is important to track all aspects of data capture, production, transformation, and analysis. In principle, one can then audit, validate, reproduce, and/or re-run with corrections various data transformations. We have recently proposed and prototyped the Chimera virtual data system, a new database-driven approach to this problem. We present here a major application study in which we apply Chimera to a challenging data analysis problem: the identification of galaxy clusters within the Sloan Digital Sky Survey. We describe the problem, its computational procedures, and the use of Chimera to plan and orchestrate the workflow of thousands of tasks on a data grid comprising hundreds of computers. This experience suggests that a general set of tools can indeed enhance the accuracy and productivity of scientific data reduction and that further development and application of this paradigm will offer great value. JF - sc SN - 1063-9535 AD - Los Alamitos, CA, USA TI - Applying Chimera Virtual Data Concepts to Cluster Finding in the Sloan Sky Survey PB - IEEE Computer Society VL - 00 KW - applying KW - chimera KW - cinding KW - in KW - sky KW - sloan KW - survey AU - Annis, James AU - Zhao, Yong AU - Voeckler, Jens AU - Wilde, Michael AU - Kent, Steve AU - Foster, Ian PY - 2002/// UR - http://dx.doi.org/10.1109/SC.2002.10021 ER -