Dissertation/ Thesis
一個具有成本效益的即時巨量資料處理系統 ; A Cost-Effective System for Real-Time Big Data Processing
| العنوان: | 一個具有成本效益的即時巨量資料處理系統 ; A Cost-Effective System for Real-Time Big Data Processing |
|---|---|
| المؤلفون: | 蔡林峻, Tsai, Linjiun |
| المساهمون: | 電機資訊學院: 電機工程學研究所, 指導教授: 廖婉君, 蔡林峻, Tsai, Linjiun |
| سنة النشر: | 2016 |
| المجموعة: | National Taiwan University Institutional Repository (NTUR) |
| مصطلحات موضوعية: | 雲端運算, 巨量資料, 資源最佳化, 記憶體管理, 效能成本權衡, 效能保證, 網路最佳化, Cloud Computing, Big Data, Resource Optimization, Memory Management, Performance-Cost Trade-off, Performance Guarantee, Network Optimization |
| Time: | 89 |
| الوصف: | The emerging Big Data paradigm has attracted attention from a wide variety of industry sectors, including healthcare, finance, retail, and manufacturing. To process massive heterogeneous data in a near real-time manner, Big Data applications should be run on dedicated server clusters that aggregate huge computing power, memory and storage through fast, unimpeded and reliable network infrastructures. Implementing such high-performance cluster computing is typically not economical for companies that only have occasional demand for Big Data processing. Cloud computing is considered a viable solution to reducing operating costs for Big Data applications due to its on-demand, pay-per-use and scalable nature. The shared nature of cloud data centers, however, may make application performance unpredictable. The strict network requirements and extremely large memory demands of Big Data clusters also lead to difficulties in optimizing the allocation of cloud resources. These difficulties translate into higher hosting cost per application. This dissertation proposes a solution to these problems that allows more concurrent Big Data applications to be deployed in cloud data centers in the most resource-efficient way while meeting their real-time requirements. To this end, we present 1) the first resource allocation framework that guarantees network performance for each Big Data cluster in multi-tenant clouds, 2) the first machine learning model that predicts the most efficient memory size for each Big Data cluster according to given upper bounds on performance penalties, and 3) an adaptive resource consolidation mechanism that strikes a balance between the number of required servers and the overhead of dynamic server consolidation for each cluster. The resource allocation framework takes advantage of the symmetry of the fat-tree network structure to enable data center networks to be efficiently partitioned into mutually exclusive and collectively exhaustive star networks, each allocated to a Big Data cluster. It provides ... |
| نوع الوثيقة: | thesis |
| وصف الملف: | 2435601 bytes; application/pdf |
| اللغة: | English |
| Relation: | http://ntur.lib.ntu.edu.tw/handle/246246/276379; http://ntur.lib.ntu.edu.tw/bitstream/246246/276379/1/ntu-105-D97921014-1.pdf |
| الاتاحة: | http://ntur.lib.ntu.edu.tw/handle/246246/276379 http://ntur.lib.ntu.edu.tw/bitstream/246246/276379/1/ntu-105-D97921014-1.pdf |
| Rights: | 論文公開時間: 2021/2/1 ; 論文使用權限: 同意有償授權(權利金給回饋本人) |
| رقم الانضمام: | edsbas.316C66F7 |
| قاعدة البيانات: | BASE |
| الوصف غير متاح. |