Parallel Processing Framework on a P2P System Using Map and Reduce Primitives

التفاصيل البيبلوغرافية
العنوان: Parallel Processing Framework on a P2P System Using Map and Reduce Primitives
المؤلفون: Kyungyong Lee, David Isaac Wolinsky, Tae Woong Choi, P. Oscar Boykin, Arijit Ganguly, Renato Figueiredo
المصدر: IPDPS Workshops
بيانات النشر: IEEE, 2011.
سنة النشر: 2011
مصطلحات موضوعية: Tree (data structure), Functional programming, Multicast, Remote procedure call, Computer science, Distributed computing, Scalability, Overlay network, Parallel computing, Latency (engineering), Cluster analysis
الوصف: This paper presents a parallel processing framework for structured Peer-To-Peer (P2P) networks. A parallel processing task is expressed using Map and Reduce primitives inspired by functional programming models. The Map and Reduce tasks are distributed to a subset of nodes within a P2P network for execution by using a self-organizing multicast tree. The distribution latency cost of multicast method is $O(log(N))$, where $N$ is a number of target nodes for task processing. Each node getting a task performs the Map task, and the task result is summarized and aggregated in a distributed fashion at each node of the multicast tree during the Reduce task. We have implemented this framework on the Brunet P2P system, and the system currently supports predefined Map and Reduce tasks or tasks inserted through Remote Procedure Call (RPC) invocations. A simulation result demonstrates the scalability and efficiency of our parallel processing framework. An experiment result on Planet Lab which performs a distributed K-Means clustering to gather statistics of connection latencies among P2P nodes shows the applicability of our system in applications such as monitoring overlay networks.
DOI: 10.1109/ipdps.2011.315
URL الوصول: https://explore.openaire.eu/search/publication?articleId=doi_________::6ff24110d56bb63b5a5432b74ff56c33
https://doi.org/10.1109/ipdps.2011.315
Rights: OPEN
رقم الانضمام: edsair.doi...........6ff24110d56bb63b5a5432b74ff56c33
قاعدة البيانات: OpenAIRE