A deep reinforcement learning-based optimization approach for containerized microservice scheduling in Hybrid Fog/Cloud environments

Abstract

The deployment of microservices in Hybrid Fog/Cloud (HFC) environments for Internet of Things (IoT) applications presents a significant challenge in efficiently scheduling containerized services across distributed resources. While existing studies have explored microservice scheduling, a comprehensive approach that considers resource constraints, workflow dependencies, and dynamic hybrid environments remains elusive. This paper introduces a novel Deep Reinforcement Learning-based Algorithm (DRLA) for containerized microservice scheduling in HFC environments. DRLA utilizes a multi-constrained Binary Quadratic Program (BQP) model to optimize execution time, resource consumption, and occupancy rates while considering microservice dependencies and resource capabilities. The algorithm leverages two Deep Reinforcement Learning (DRL) agents, DQN and REINFORCE, to learn and adapt to the dynamic nature of the HFC federation. Experimental evaluations using five real-world Business Process (BP) use cases demonstrate that DRLA outperforms existing scheduling approaches such as default Kubernetes, Reward Sharing Deep Q-Learning (RSDQL), and Deep Reinforcement Learning (DRL) schedulers. Compared to existing schedulers, ours delivers optimal or near-optimal solutions, demonstrating significant improvements in key performance values. Indeed, DRLA achieves average optimality gaps of just 0.16% and 0.21%, significantly outperforming the Kubernetes scheduler, which exhibits a gap of 2.88%. It is worth noting that other similar algorithms see a far higher increase in optimality gaps. This highlights DRLA’s excellent performance and ability to schedule containerized microservices in hybrid fog/cloud environments, resulting in near-optimal solutions for a variety of use cases.

Ameni Kallel Chaari

Computer Technologist Teacher

My main fields of interest include Virtualization, Cloud Computing, Internet of Things, with a current focus on dynamic allocation and management of virtualized compute and network resources.

Molka Rekik

Assistant professor

My research interests include cloud engineering, business intelligence, and optimization.

Mahdi Khemakhem

Professor

My research interests are mainly in artificial intelligence including complex systems modeling, heuristics, meta-heuristics, and exact algorithms for combinatorial optimization problems in transportation and networks, resources management, cloud computing, IoT, etc.