ASTROIDE: A Unified Astronomical Big Data Processing Engine over Spark

التفاصيل البيبلوغرافية
العنوان: ASTROIDE: A Unified Astronomical Big Data Processing Engine over Spark
المؤلفون: Karine Zeitouni, Mariem Brahem, Laurent Yeh
المصدر: IEEE Transactions on Big Data
بيانات النشر: Institute of Electrical and Electronics Engineers (IEEE), 2020.
سنة النشر: 2020
مصطلحات موضوعية: Big Data, Information Systems and Management, Group method of data handling, Computer science, Big data, 02 engineering and technology, Spatial Query Processing, 01 natural sciences, 020204 information systems, 0103 physical sciences, Spark (mathematics), 0202 electrical engineering, electronic engineering, information engineering, 010303 astronomy & astrophysics, Database server, Information retrieval, Distributed database, business.industry, HEALPix, Search engine indexing, Astrophysics::Instrumentation and Methods for Astrophysics, Physics::History of Physics, ComputingMilieux_GENERAL, Spark Framework, Astronomical Survey Data Management, Scalability, business, Information Systems
الوصف: The next decade promises to be an exciting time for astronomers. Large volumes of astronomical data are continuously collected from highly productive space missions. This data has to be efficiently stored and analyzed in such a way that astronomers maximize their scientific return from these missions. Recognizing the need to better handle astronomical datasets, we designed ASTROIDE, adistributed data server for astronomical data. We analyze the peculiarities of the data and the queries in cosmological applications and design a new framework where astronomers can explore and manage vast amounts of data. ASTROIDE introduces effective methods for efficient astronomical query execution on Spark through data partitioning with HEALPix and customized optimizer. ASTROIDE offers a simple, expressive and unified interface through ADQL, a standard language for querying databases in astronomy. Experiments have shown that ASTROIDE is effective in processing astronomical data, scalable and outperforms the state-of-the-art.
تدمد: 2372-2096
DOI: 10.1109/tbdata.2018.2873749
URL الوصول: https://explore.openaire.eu/search/publication?articleId=doi_dedup___::79292276d2772a7885b549ab60a4bb19
https://doi.org/10.1109/tbdata.2018.2873749
Rights: OPEN
رقم الانضمام: edsair.doi.dedup.....79292276d2772a7885b549ab60a4bb19
قاعدة البيانات: OpenAIRE
الوصف
تدمد:23722096
DOI:10.1109/tbdata.2018.2873749