The Optimization and Improvement of MapReduce in Web Data Mining
| العنوان: | The Optimization and Improvement of MapReduce in Web Data Mining |
|---|---|
| المؤلفون: | Shangwei Song, Chang-Qing Yin, Jun Qu |
| المصدر: | Journal of Software Engineering and Applications. :395-406 |
| بيانات النشر: | Scientific Research Publishing, Inc., 2015. |
| سنة النشر: | 2015 |
| مصطلحات موضوعية: | Data processing, medicine.medical_specialty, Web data mining, Computer science, business.industry, Cloud computing, Large scale data, computer.software_genre, medicine, Data mining, business, Merge (version control), Web modeling, computer, Data Web |
| الوصف: | Extracting and mining social networks information from massive Web data is of both theoretical and practical significance. However, one of definite features of this task was a large scale data processing, which remained to be a great challenge that would be addressed. MapReduce is a kind of distributed programming model. Just through the implementation of map and reduce those two functions, the distributed tasks can work well. Nevertheless, this model does not directly support heterogeneous datasets processing, while heterogeneous datasets are common in Web. This article proposes a new framework which improves original MapReduce framework into a new one called Map-Reduce-Merge. It adds merge phase that can efficiently solve the problems of heterogeneous data processing. At the same time, some works of optimization and improvement are done based on the features of Web data. |
| تدمد: | 1945-3124 1945-3116 |
| DOI: | 10.4236/jsea.2015.88039 |
| URL الوصول: | https://explore.openaire.eu/search/publication?articleId=doi_________::24ccd68a6869bd762f385afb032869f5 https://doi.org/10.4236/jsea.2015.88039 |
| Rights: | OPEN |
| رقم الانضمام: | edsair.doi...........24ccd68a6869bd762f385afb032869f5 |
| قاعدة البيانات: | OpenAIRE |
| تدمد: | 19453124 19453116 |
|---|---|
| DOI: | 10.4236/jsea.2015.88039 |