Arc: An IR for batch and stream programming

التفاصيل البيبلوغرافية
العنوان: Arc: An IR for batch and stream programming
المؤلفون: Kroll, Lars, 1989, Segeljakt, Klas, Schulte, Christian, Professor, 1967, Haridi, Seif, 1953, Carbone, P.
المصدر: Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). :53-58
مصطلحات موضوعية: Data analytics, Intermediate representation, Stream processing
الوصف: In big data analytics, there is currently a large number of data programming models and their respective frontends such as relational tables, graphs, tensors, and streams. This has lead to a plethora of runtimes that typically focus on the efficient execution of just a single frontend. This fragmentation manifests itself today by highly complex pipelines that bundle multiple runtimes to support the necessary models. Hence, joint optimization and execution of such pipelines across these frontend-bound runtimes is infeasible. We propose Arc as the first unified Intermediate Representation (IR) for data analytics that incorporates stream semantics based on a modern specification of streams, windows and stream aggregation, to combine batch and stream computation models. Arc extends Weld, an IR for batch computation and adds support for partitioned, out-of-order stream and window operators which are the most fundamental building blocks in contemporary data streaming.
وصف الملف: print
URL الوصول: https://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-262617
https://doi.org/10.1145/3315507.3330199
قاعدة البيانات: SwePub