A Novel Approach of Fair Scheduling to Enhance Performance of Hadoop Distributed File System

Abstract

Nowadays, big data is one of the most challenging issues for managing big amounts of data more effectively and efficiently. It widely used in E-commerce, social media, online business and such types of applications. Hadoop Distributed File System (HDFS) is one of the widely used frameworks which can easily handle and store large amounts of data set frequently. For HDFS job scheduling is more challenging because it plays an important role in time optimization in big data. For resolving this issue in this paper, we introduce a job scheduling algorithm which is more time efficient and accurate than existing fair job scheduling algorithm. We optimize the time cycle of fair scheduling by minimizing iteration. We have accelerated with the different number of jobs in the existing algorithm and proposed an algorithm for experimentally proving the time complexity and time measurement. It is observed that the proposed method is computationally efficient than the existing one and our algorithm has reduced the number of iterations and improved the time efficiency on average 26.719%.