A Comparison of Approaches to Large-Scale Data Analysis Export

@inproceedings{citeulike:4308191, abstract = {There is currently considerable enthusiasm around the MapReduce(MR) paradigm for large-scale data analysis [17]. Although thebasic control flow of this framework has existed in parallel SQLdatabase management systems (DBMS) for over 20 years, somehave called MR a dramatically new computing model [8, 17]. Inthis paper, we describe and compare both paradigms. Furthermore,we evaluate both kinds of systems in terms of performance and de-velopment complexity. To this end, we define a benchmark con-sisting of a collection of tasks that we have run on an open sourceversion of MR as well as on two parallel DBMSs. For each task,we measure each system's performance for various degrees of par-allelism on a cluster of 100 nodes. Our results reveal some inter-esting trade-offs. Although the process to load data into and tunethe execution of parallel DBMSs took much longer than the MRsystem, the observed performance of these DBMSs was strikinglybetter. We speculate about the causes of the dramatic performancedifference and consider implementation concepts that future sys-tems should take from both kinds of architectures.}, author = {Pavlo, Andrew and Paulson, Erik and Rasin, Alexander and Abadi, Daniel J. and Dewitt, David J. and Madden, Samuel and Stonebraker, Michael}, booktitle = {SIGMOD'09}, citeulike-article-id = {4308191}, day = {29}, keywords = {data, data-bases, data-mining, dbms, map-reduce, parallel-databases, storage}, location = {Providence, Rhode Island, USA}, month = {June}, organization = {ACM}, posted-at = {2009-04-14 17:35:27}, priority = {0}, title = {A Comparison of Approaches to Large-Scale Data Analysis}, year = {2009} }

TY - CONF ID - citeulike:4308191 L3 - citeulike-article-id:4308191 N2 - There is currently considerable enthusiasm around the MapReduce(MR) paradigm for large-scale data analysis [17]. Although thebasic control flow of this framework has existed in parallel SQLdatabase management systems (DBMS) for over 20 years, somehave called MR a dramatically new computing model [8, 17]. Inthis paper, we describe and compare both paradigms. Furthermore,we evaluate both kinds of systems in terms of performance and de-velopment complexity. To this end, we define a benchmark con-sisting of a collection of tasks that we have run on an open sourceversion of MR as well as on two parallel DBMSs. For each task,we measure each system’s performance for various degrees of par-allelism on a cluster of 100 nodes. Our results reveal some inter-esting trade-offs. Although the process to load data into and tunethe execution of parallel DBMSs took much longer than the MRsystem, the observed performance of these DBMSs was strikinglybetter. We speculate about the causes of the dramatic performancedifference and consider implementation concepts that future sys-tems should take from both kinds of architectures. CY - Providence, Rhode Island, USA OR - ACM TI - A Comparison of Approaches to Large-Scale Data Analysis T2 - SIGMOD’09 KW - data KW - data-bases KW - data-mining KW - dbms KW - map-reduce KW - parallel-databases KW - storage AU - Pavlo, Andrew AU - Paulson, Erik AU - Rasin, Alexander AU - Abadi, Daniel AU - Dewitt, David AU - Madden, Samuel AU - Stonebraker, Michael PY - 2009/06/29/ ER -