Rainbow: Adaptive Layout Optimization for Wide Tables

التفاصيل البيبلوغرافية
العنوان: Rainbow: Adaptive Layout Optimization for Wide Tables
المؤلفون: Guodong Jin, Xiaoyong Du, Youxian Tao, Yueguo Chen, Haoqiong Bian, Xiongpai Qin
المصدر: ICDE
بيانات النشر: IEEE, 2018.
سنة النشر: 2018
مصطلحات موضوعية: 060201 languages & linguistics, Computer science, Data layout, Process (computing), Joins, Rainbow, 06 humanities and the arts, 02 engineering and technology, Column (database), Computer engineering, 0602 languages and literature, 0202 electrical engineering, electronic engineering, information engineering, Benchmark (computing), 020201 artificial intelligence & image processing, Data compression
الوصف: Popular column stores such as ORC and Parquet have been widely used in many Hadoop-oriented data analysis systems. With the effective column skipping and data compression functionalities provided by column stores, wide tables with hundreds or even thousands of columns are applied by many big data analysis applications to avoid the expensive distributed joins. We found that the performance of such systems can be further improved by optimizing the physical data layout to fit certain workloads and system settings. However, it is nontrivial to perform such optimization manually. In this demo, we present a data layout optimization tool called Rainbow, which leverages workload-driven layout optimization algorithms to adjust data layouts adaptively without intervening the previous data blocks that have been stored. We also provide a Web UI for users to interact with the layout optimization process. Furthermore, Rainbow is open sourced with an accompanying benchmark for performance evaluation of wide tables.
DOI: 10.1109/icde.2018.00200
URL الوصول: https://explore.openaire.eu/search/publication?articleId=doi_________::7474c0c10d1fac3322229f679fc817a1
https://doi.org/10.1109/icde.2018.00200
رقم الانضمام: edsair.doi...........7474c0c10d1fac3322229f679fc817a1
قاعدة البيانات: OpenAIRE