A Hierarchical Distributed Processing Framework for Big Image Data
This paper introduces an effective processing framework nominated ICP (Image Cloud Processing) to powerfully cope with
the data explosion in image processing field. While most previous researches focus on optimizing the image processing algorithms to
gain higher efficiency, our work dedicates to providing a general framework for those image processing algorithms, which can be
implemented in parallel so as to achieve a boost in time efficiency without compromising the results performance along with the
increasing image scale. The proposed ICP framework consists of two mechanisms, i.e. SICP (Static ICP) and DICP (Dynamic ICP).
Specifically, SICP is aimed at processing the big image data pre-stored in the distributed system, while DICP is proposed for dynamic
input. To accomplish SICP, two novel data representations named P-Image and Big-Image are designed to cooperate with MapReduce
to achieve more optimized configuration and higher efficiency. DICP is implemented through a parallel processing procedure working
with the traditional processing mechanism of the distributed system. Representative results of comprehensive experiments on the
challenging ImageNet dataset are selected to validate the capacity of our proposed ICP framework over the traditional state-of-the-art
methods, both in time efficiency and quality of results.
KeyWords
Big data, Image processing, MapReduce, Distributed system, Cloud computing
|