Ziyu Lin, Minxing Cai, Ziming Huang, Yongxuan Lai. SALA: A Skew-Avoiding and Locality-Aware Algorithm for MapReduce-Based Join. In proceedings of WAIM2015, LNCS 9098, pp. 311–323, 2015.(EI检索国际学术会议)
WAIM 2015 会议一共收到158篇论文投稿，仅录用34篇full paper，录用率仅为21.5%。
相关链接：蔡珉星同学参加WAIM2015国际学术会议并做大会论文报告
Abstract. MapReduce is a parallel programming model, which is extensively used to process join operations for large-scale dataset. However, traditional MapReduce-based join is not efficient when handling skewed data, because it can lead to partitioning skew, which further results in longer response time of the whole join process. Additionally, some newly proposed methods usually involve large amounts of intermediate results over the network in the shuffle phase of Mapreduce-based join, which may consume a lot of time and cause performance degradation. Here a novel algorithm called SALA is proposed, which employs volume/locality-aware partitioning instead of hash partitioning for data distribution. Compared with other existing join algorithms, SALA has three typical advantages: (1) makes sure that the data is distributed to reducers evenly when the input datasets are skewed, (2) reduces the amount of intermediate results transferred across the network by utilizing data locality, and (3) does not make any modification of the MapReduce framework. The extensive experimental results show that SALA not only achieves better load balance but reduces network overhead, and therefore speeds up the whole join process significantly in the presence of data skew.