Empirical Analysis of MapReduce Job Scheduling with respect to Energy Consumption of Clusters

Sofia D'souza, K. V. Prema

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The growing popularity of Cloud Computing has led to an increasing number of applications using MapReduce in cloud data centers. MapReduce workloads are mostly interactive workloads or batch processing workloads that consume an enormous amount of energy when hosted on multiple clusters. Currently, the scheduling of workloads is done with the sole goal of the faster execution time of jobs. However, in doing so, there is wastage of energy as the same jobs could be completed within the stipulated Service Level Agreement (SLA) using fewer resources when hosted on small clusters. Therefore, in order to minimize the energy consumption of MapReduce clusters, workloads could be deployed on a minimum number of clusters depending on the type and size with the goal of minimizing energy consumption and not faster response time. In this work, the three schedulers i.e FIFO, Fair and Capacity schedulers are compared with respect to energy efficiency on small-scale and large-scale workloads. Experiments performed on a small cluster using these workloads show significant energy savings with respect to Capacity scheduler compared to other schedulers.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics, DISCOVER 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728137353
DOIs
Publication statusPublished - 08-2019
Event3rd IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics, DISCOVER 2019 - Manipal, India
Duration: 11-08-201912-08-2019

Publication series

Name2019 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics, DISCOVER 2019 - Proceedings

Conference

Conference3rd IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics, DISCOVER 2019
CountryIndia
CityManipal
Period11-08-1912-08-19

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Hardware and Architecture
  • Decision Sciences (miscellaneous)
  • Information Systems and Management
  • Electrical and Electronic Engineering
  • Computational Mathematics
  • Control and Optimization

Fingerprint Dive into the research topics of 'Empirical Analysis of MapReduce Job Scheduling with respect to Energy Consumption of Clusters'. Together they form a unique fingerprint.

  • Cite this

    D'souza, S., & Prema, K. V. (2019). Empirical Analysis of MapReduce Job Scheduling with respect to Energy Consumption of Clusters. In 2019 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics, DISCOVER 2019 - Proceedings [9007941] (2019 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics, DISCOVER 2019 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DISCOVER47552.2019.9007941