An extreme learning approach to fast prediction in the reduce phase of a cloud platform

Liu, Q; Cai, W; Shen, J; Wang, B; Fu, Z; Linge, N

Abstract

As a widely used programming model for the purposes of processing large data sets, MapReduce (MR) becomes inevitable in data clusters or grids, e.g. a Hadoop environment. However, experienced programmers are needed to decide the number of reducers used during the reduce phase of the MR, which makes the quality of MR scripts differ. In this paper, an extreme learning method is employed to recommend potential number of reducer a mapped task needs. Execution time is also predicted for user to better arrange their tasks. According to the results, our method can provide fast prediction than SVM with similar accuracy maintained.

Citation

Liu, Q., Cai, W., Shen, J., Wang, B., Fu, Z., & Linge, N. An extreme learning approach to fast prediction in the reduce phase of a cloud platform. Presented at International Conference on Cloud Computing and Security (ICCCS 2015), Nanjing, China

• Presentation Conference Type: Other
• Conference Name: International Conference on Cloud Computing and Security (ICCCS 2015)
• Conference Location: Nanjing, China
• End Date: Aug 15, 2015
• Online Publication Date: Jan 5, 2016
• Publication Date: Jan 6, 2016
• Deposit Date: Nov 29, 2016
• Series Title: Lecture Notes in Computer Science
• Series Number: 9483
• Book Title: Cloud Computing and Security
• ISBN: 9783319270500;-9783319270517
• DOI: https://doi.org/10.1007/978-3-319-27051-7_35
• Publisher URL: http://dx.doi.org/10.1007/978-3-319-27051-7_35
• Related Public URLs: https://link.springer.com/book/10.1007/978-3-319-27051-7#about
• Additional Information: Event Type : Conference