Research on the Construction of Smart Campus Social Platform Based on Hadoop
Conguration parameter optimization is an important means of improving the performance of the MapReduce model. The existing parameter tuning methods usually optimize all conguration parameters in MapReduce. However, it is exceedingly challenging to tune all the parameters for the MapReduce model because there are massive conguration parameters in MapReduce. In this paper, a novel conguration parameter tuning method based on a feature selection algorithm is proposed, and it is composed of the feature selection objective function and feature selection process. The objective function is based on the kernel clustering algorithm, in which anisotropic Gaussian kernel is adopted instead of the traditional Gaussian kernel to accurately judge the importance of each parameter in MapReduce. Then, the relationship between the conguration parameters in MapReduce and the features in the feature selection algorithm is dened. Moreover, the importance of each parameter is reected by the kernel width of anisotropic Gaussian kernels. At the same time, the method of gradient descent is introduced to update the kernel width and control the feature selection process of the iterative algorithm. Finally, experimental results show that the proposed algorithm performs suitably for the MapReduce model.
Parameter tuning, Hadoop MapReduce, kernel function, K-means