
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

39300104
72774
10.1038/s41598-024-72774-5
Article
Efficient deep reinforcement learning based task scheduler in multi cloud environment
Mangalampalli Sudheer ms.sudheer@manipal.edu

1
Karri Ganesh Reddy 2
Ratnamani M. V. 3
Mohanty Sachi Nandan 2
Jabr Bander A. 4
Ali Yasser A. 4
Ali Shahid alikhan@pku.edu.cn

5
Abdullaeva Barno Sayfutdinovna 6
1 https://ror.org/02xzytt36 grid.411639.8 0000 0001 0571 5193 Department of CSE, Manipal Institute of Technology Bengaluru, Manipal Academy of Higher Education, Manipal, India
2 https://ror.org/007v4hf75 School of Computer Science and Engineering, VIT-AP University, Amaravati, AP 522237 India
3 Aditya Institute of Technology and Management, Tekkali, Srikakulam, AP 530021 India
4 https://ror.org/02f81g417 grid.56302.32 0000 0004 1773 5396 Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, P.O. Box 51178, 11543 Riyadh, Saudi Arabia
5 Battery Management System, Research and Development Center, EVE Lithium Energy Company, Huizhou, People’s Republic of China
6 https://ror.org/051g1n833 grid.502767.1 0000 0004 0403 3387 Department of Mathematics and Information Technologies, Vice-Rector for Scientific Affairs, Tashkent State Pedagogical University, Tashkent, Uzbekistan
19 9 2024
19 9 2024
2024
14 2185010 1 2024
10 9 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Task scheduling problem (TSP) is huge challenge in cloud computing paradigm as number of tasks comes to cloud application platform vary from time to time and all the tasks consists of variable length, runtime capacities. All these tasks may generated from various heterogeneous resources which comes onto cloud console directly effects the performance of cloud paradigm with increase in makespan, energy consumption, resource costs. Traditional task scheduling algorithms cannot handle these type of complex workloads in cloud paradigm. Many authors developed Task Scheduling algorithms by using metaheuristic techniques, hybrid approaches but all these algorithms give near optimal solutions but still TSP is a highly challenging and dynamic scenario as it resembles NP hard problem. Therefore, to tackle the TSP in cloud computing paradigm and schedule the tasks in an effective way in cloud paradigm, we formulated Adaptive Task scheduler which segments all the tasks comes to cloud console as sub tasks and fed these to the scheduler which is modeled by Improved Asynchronous Advantage Actor Critic Algorithm(IA3C) to generate schedules. This scheduling process is carried out in two stages. In first stage, all incoming tasks are segmented as sub tasks. After segmentation, all these sub tasks according to their size, execution time, communication time are grouped together and fed to the (ATSIA3C) scheduler. In the second stage, it checks for the above said constraints and disperse them onto the corresponding suitable processing capacity VMs resided in datacenters. Proposed ATSIA3C is simulated on Cloudsim. Extensive simulations are conducted using both fabricated worklogs and as well as realtime supercomputing worklogs. Our proposed mechanism evaluated over baseline algorithms i.e. RATS-HM, AINN-BPSO, MOABCQ. From results it is evident that our proposed ATSIA3C outperforms existing task schedulers by improving makespan by 70.49%. Resource cost is improved by 77.42%. Energy Consumption is improved over compared algorithms 74.24% in multi cloud environment by proposed ATSIA3C.

Keywords

Task scheduling
Cloud computing
Deep reinforcement learning
Makespan
Resource cost
Subject terms

Aerospace engineering
Electrical and electronic engineering
issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

Cloud Computing model provides hassle-free access to all users who need on demand compute, storage, network access with high availability and scalability over the internet. Services of cloud paradigm can be accessed by any user who subscribed to a certain service based on service level agreement from any type of device i.e. (Mobile, desktop, laptop) with the help of a web browser1. There are various services provided by cloud paradigm. There are three major categories of services are available in this paradigm. Infrastructure as a service- provides virtual infrastructure on top of physical machines which reside in datacenter. This service typically helps cloud users to host their applications on cloud platform. Platform as a service- provides development environment on cloud computing platform for the users which gives flexibility for the developers to concentrate on development rather than managing the underlying VMs. Software as a service- provides application software to users for their usage based on need of the customer2.

All these services are to be rendered to the customers based on their needs and different users will access the cloud environment. They may have different workloads at different instances of time to be computed on cloud paradigm. According to the customers need cloud service provider (CSP) have to deploy the cloud services for the corresponding user. Therefore, every user may not have different requirements for application deployment and different deployment models to be used by CSP in this paradigm.

Public cloud- provides all the services publicly over the internet to users around the world without having any restriction on the users who have valid SLA. Private Cloud- Provides all types of services to users only restricted to that organization and users should have valid SLA. This can be configured by CSP with the help of virtual private cloud (VPC). Hybrid Cloud- It combines both the features of public and private clouds which gives customization choice to the customer to have a public access to some of the services and have a private access to the services by isolating them using VPC3.

All the above services are given to the customers or cloud users around world and round the clock only when CSP is properly managing all the requests i.e. assigning virtual resources to the tasks coming to cloud console. Note that all the tasks or requests raised by customers are of different types and they may not have same type of runtime capacity. All these requests may be raised from diversified devices. To handle this type of dynamic situation and mapping these heterogeneous tasks with different processing capacities onto suitable VMs which handles these requests is a challenge for CSP. This is a tedious job which CSP cannot handle manually as number of users varies from time to time and different tasks coming onto cloud platform also varies which impacts the critical aspects of scheduling process. To handle this situation, CSP would employ scheduling algorithm which takes all tasks upcoming onto cloud application and maps tasks to suitable VMs in cloud paradigm which situated in physical nodes and they are situated in datacenters.

TSP in cloud paradigm is a challenge as variable number of tasks which comes to cloud application console should be mapped precisely to the number VMs. It is a highly dynamic scenario and considered to be a NP-hard problem as precise solution in definite polynomial amount of time is not possible. Therefore, every researcher proposed scheduling techniques by using metaheuristic, nature inspired, and bio inspired algorithms like PSO4, HEFT5, ACO6, GA7. All these algorithms are generating schedules in an optimized manner in their perspective yet scheduling procedure in cloud paradigm is a challenge as all these algorithms not converging towards the solutions precisely with the sudden changes of workload in cloud platform. Therefore, it is a crucial challenge for the CSP to employ suitable scheduler in this paradigm. The main reason to choose an efficient task scheduler is if a task scheduler is not mapping the tasks based on run time processing capacities onto VMs there may be a chance of execution delays of task on those VMs which could result in latency, increase in execution time which leads to increase in makespan. It is primary concern for task scheduler as many tasks are dead line constraint based tasks in cloud paradigm and all of them should be executed with timelines as if execution time increases on virtual resource it effects makespan and quality of service of CSP will be reduced.

Another reason to choose an efficient task scheduler is energy consumption. If a task coming up onto cloud application console is not mapped based on its run time capacity, size, data interchange of tasks in between VMs, processing capacity of VMs those tasks will consume relatively more energy which incurs high energy consumption which is a disadvantage for CSP which leads to more power bills and in turn it is disadvantageous for cloud users which increases resource costs. This is a situation where both CSP and users are directly affected by an ineffective task scheduler. Some existing authors proposed task schedulers by hybrid mechanisms by combining nature inspired algorithms with AI mechanisms RATS-HM8, MOABCQ9, AINN-BPSO10 to tackle dispersion of tasks in cloud application console but still these algorithms are also lacking with ineffective mapping of tasks to virtual resources and while increase in the workload.

The above-mentioned algorithms are not able to learn and extract feature information and suffer with adaptability issues with sudden increase of workload. Therefore, to tackle these issues above mentioned we came up with a solution to use deep reinforcement learning approach known as Asynchronous Advantage Actor Critic algorithm. It is a reward-based approach in which for every iteration, when a task is mapped to a VM, a reward will be generated i.e. (Positive or negative). If a positive reward is generated and it will be updated as a best updated solution and if negative reward is generated it should learn from trained mappings and not generate schedules in future. All these mappings will be done by actor network which consists of state and action spaces run in different independent threads. All these threads will be evaluated by critic network which checks for evaluation of generated schedules in line with policies imposed at the time of training and it should guide the actor network not to deviate from the imposed policy in actor network but still this A3C approach also suffers with adaptability with increase of workload. A3C uses fully connected network which consists of more hidden layers. Therefore, we used Residual Convolutional neural network (RCNN) to directly skip hidden layers and connect the network directly with input states. With the help of IA3C, we formulate an adaptive task scheduler which takes tasks as input. Task segmentation is performed to make tasks as sub tasks and based on size of tasks, data interchange happens between VMs which are suitable with respect to processing capacities of VMs, bandwidth requirement, communication time. For effective generation of schedules for these tasks, we used IA3C i.e. Improved Asynchronous Advantage Actor critic Algorithm which is reinforcement learning based approach and multi cloud environment is considered to migrate tasks to cloud environment wherever resource cost is low. Along with the evaluation of resource cost, ATSIA3C aims to minimize makespan, energy consumption in multi cloud environment.

Motivations and contributions

TSP in cloud computing is tedious challenge for CSP as incoming tasks raised from various heterogeneous resources with different processing capacities come onto cloud application console. Mapping all incoming tasks to VMs which are suitable to their runtime capacities is crucial aspect in this model as every time set of tasks which need to be mapped to the VMs are may not be same in size, runtime of tasks and at times more number of tasks comes to cloud platform with a huge volume and at times very less number of tasks comes to cloud application console. Therefore, this is a highly dynamic situation in which scaling of resources also to be done automatically based on number of tasks or requests comes to cloud application console. This cannot be done manually with huge number of users around the world dispersing tasks onto cloud application console. Therefore, CSP should automate scheduling mechanisms in cloud environment by employing an algorithm which should tackle this problem. CSP should choose an effective scheduling algorithm to map tasks based on their size, data interchange between VMs based on their capacity, communication time. Our main motivation to conduct research on this task scheduling algorithm is that a TSP directly impacts Quality of Service of CSP because improper scheduling of tasks by not considering their size, run time capacities may incur delay of execution of tasks there by violating deadlines which also incurs violation of SLA. Delay in executing tasks leads to increase of makespan which is a primary concern of our research. When more delay incurs in execution of tasks, energy consumed for the respective tasks mapped to virtual resources will also be increased there by it will be a huge problem to CSP which leads to increase of pricing of services in cloud paradigm and this also impacts resource cost in that respective cloud environment. Keeping all these constraints in mind, we came up with an Adaptive Task Scheduler using Improved Asynchronous Advantage Actor Critic algorithm (ATSIA3C) proposed to map tasks by segmenting them to sub tasks and carefully map them onto VMs based on their size, data interchange between VMs, communication time while minimizing makespan, resource cost, energy consumption in multi cloud environment.

The Research Questions below motivated and guided us to do formulate the Adaptive Task Scheduling Algorithm using Improved Asynchronous Advantage Actor Critic Algorithm (ATSIA3C).

RQ1. How can a task scheduling algorithm be adaptive and generate optimized schedules if the complex workload comes to the cloud application console?

RQ2. How can an adaptive task scheduling algorithm can optimize energy consumption, makespan, resource cost in multi cloud environments with the help of a deep reinforcement learning algorithm?

Main Contributions of this manuscript are shown below.

An Adaptive Task Scheduling algorithm by using Improved Asynchronous Advantage Actor critic (ATSIA3C) formulated.

For adaptive scheduling process, we carefully segmented tasks and those sub tasks are dispersed onto VMs based on their size, communication time, execution time.

Improved Asynchronous Advantage Actor critic algorithm is used as methodology in this research to model task scheduler.

Simulations conducted using Cloudsim toolkit and schedules generated for proposed ATSIA3C compared over existing MOABCQ, AINN-BPSO, RATS-HM algorithms to evaluate the proposed approach.

For all the simulations, we have used both fabricated workload and realtime supercomputing worklogs i.e. HPC2N, NASA workloads as input trace.

Makespan, Energy Consumption, Resource cost are evaluated as parameters by using ATSIA3C in multi cloud environment.

Related works

Section 2 discusses existing works proposed by various researchers in TSP in cloud computing. We have summarized various TSP algorithms proposed by authors using metaheuristic techniques and Machine learning mechanisms and different types of parameters they addressed to evaluate various scheduling techniques.

Authors in8 proposed task scheduler by using hybrid mechanism with metaheuristic algorithm, machine learning techniques. Initially, in the algorithm, Cat Swarm optimization is used to tackle throughput, makespan. In second phase, deep neural network optimizes generated schedules in the first phase while addressing resource utilization. Finally, a light weight encryption and storage mechanism is proposed and while it is evaluated over existing algorithms, RATS-HM outperforms existing mechanisms over FCFS, RR algorithms for optimizing response time, energy consumption metrics.

A task scheduling algorithm to balance the task load among different VMs is formulated in9. It is designed by hybridizing ABC with Q-learning to speed up ABC. It was simulated on Cloudsim using synthetic and Google Cloud job datasets. It evaluated over existing FCFS, Max-Min, MOPSO, MOCS algorithms to minimize makespan, cost of resources and to improve throughput, utilization of resources. A TS framework designed in10 to minimize completion time of tasks, resource cost in cloud paradigm. It is formulated based on a hybrid approach with metaheuristic algorithm Binary PSO combined with neural networks to speed up the updation of particles to map tasks with VMs. It evaluated over conventional PSO approach and finally AINN-BPSO outperforms conventional techniques for minimization of task completion time, cost of resources.

A hybrid approach consists of HEFT with Q-learning is used to formulate TS algorithm to tackle makespan. QL-HEFT algorithm11 used upward rank of HEFT as reward for generation of schedules. Based on the reward rendered at that time Q-learning optimizes scheduling process with self-learning. This scheduling carried out in two phases task sorting, selection of task phases. It is evaluated over classical HEFT, CPOP techniques. Finally, results of QL-HEFT shown significant improvement of makespan, response time over existing approaches.

A load balanced variation of a task scheduler developed by authors in12 to schedule tasks dispersed over suitable VMs according to the rank modification in calculation of task ranking by adding task length to the existing rank equation. After the task ranking phase, sorting phase to be performed by HEFT to schedule upcoming tasks on to VMs. Extensive simulations are conducted on Cloudsim toolkit with random workloads. To evaluate the effectiveness of algorithm, it is compared against E-HEFT, L-HEFT algorithms to minimize makespan and improve load balancing of tasks, resource utilization.

In13, multi objective TS algorithm is formulated to tackle TSP effectively in cloud paradigm. It is formulated by hybridizing HEFT with decision tree approaches. Scheduling performed in three steps. Initially task priority is calculated based on the rank generated by HEFT algorithm. After that, resource matrix is generated which collects task features and finally resource allocation done by using decision tree which maps tasks to appropriate VMs. It evaluated with existing HEFT with real time scientific workflows and hybrid approach proposed by authors in13 minimizes makespan over existing techniques.

Authors in14 formulated workflow scheduling technique to exploit idle slots of workflows where tasks are not scheduled and those can be filled with task predecessors to speedup task scheduling of unscheduled tasks. It is formulated by modifying the task ranking phase in HEFT algorithm. Simulations input were given with random and real time scientific workflows. TDSA compared over classical HEFT, EFT-MER algorithms. From results, TDSA proved to minimize makespan over existing algorithms while generating optimized schedules for both random, real-time workflows.

A hybridized workflow scheduling formulated in15 to tackle makespan, resource cost. Gravitational Search algorithm(GSA), HEFT algorithms are combined to generate optimized schedules of tasks mapped to VMs. Authors used cost equivalence ratio used to formulate objective functions. Monetary cost ratio, schedule length ratio is used as evaluation criteria for this approach. It is compared against GSA, HEFT. Results shown significant improvement in view of makespan.

TS framework in16 formulated by using deep reinforcement learning techniques to minimize memory usage, resource cost, execution time. Four deep reinforcement learning techniques developed DQN, RL, RNN-LSTM, DRL-LSTM. All these simulations are conducted extensively on MATLAB by using Google Cloud Jobs. Out of all these techniques DRL-LSTM showed significant improvement in resource cost, memory usage, execution time over Improved PSO, SJF, RR algorithms.

Energy consumption plays vital role in cloud model from facet of CSP as well as from a user perspective. If a scheduler is ineffective in terms of mapping tasks to suitable VMs then energy consumption will be drastically increased, which impacts CSP and users. Therefore, to tackle scheduling of workloads which run in parallel in large scale can be scheduled using a machine learning based parallel PSO scheduler17 which works with Deep Reinforcement learning technique to evaluate reward and optimize schedules and PPSO speed up schedules generated by particles in every iteration. It evaluated with classical PSO approach to evaluate its significance. Results revealed that DRLPPSO showed significant improvement over PSO in view of speedup of tasks, Energy Consumption, Task completion time.

Task suitability to a corresponding VM is primary aspect in TS in cloud paradigm as inefficient mapping led to QoS issues in this paradigm. Therefore, to handle these situations authors in18 proposed a task scheduler takes priorities of both tasks, VMs for assignment of a VM. This approach i.e. DRFTSA formulated using Deep Reinforcement learning model called DQN. All these priorities are fed to DRFTSA to generate schedules. Input dataset used is real time Google Cloud Jobs for simulation by categorizing datasets as regular, large. Parameters known as makespan, energy consumption, rate of failures is evaluated by fixing, varying VMs in both regular, large datasets. It evaluated with PSO, GA, ACO. From generated schedules i.e. results, DRFTSA showed huge impact on other approaches by minimizing makespan, rate of failures. The major challenge in TS is assignment of different sizes of tasks with VMs in datacenter. To tackle this challenge using a machine learning approach authors in19 formulated TS algorithm which addresses makespan, throughput, utilization of resources, energy consumption. Entire simulation conducted using MATALAB with python3. Google cloud jobs dataset with different categories of VMs is fed as input to scheduler to disperse tasks onto VMs. This approach is compared against MVO, EMVO algorithms and results show that DRLSTS proved to be dominant over compared approaches.

In20, formulated TS algorithm based on Binary In order traversal combined with ACO, DRL algorithms. Initial task scheduling performed by using Binary In order traversal and resource allocation for tasks to VMs using ACO while Deep Reinforcement algorithm is used to minimize space complexity. It evaluated with ACOPSO, CWSA algorithms to check the efficacy. From simulated results, it proved TSRAACODRL outperformed over compared approaches while minimizing makespan, Power consumption and improving resource utilization.

In21, authors mainly aim at formulation of a task scheduler which is a combination of two heuristics i.e. PSO, Opposition based Learning. The reason they chose these algorithms is to avoid premature convergence in PSO, they introduced opposition learning based algorithm which accelerates performance of PSO. It evaluated over mPSO, Max-min, GA, PSO algorithms to check efficacy of OPSO. Results showed that OPSO improved makespan, task completion time over evaluated approaches. Many authors proposed TS algorithms using metaheuristics, but still fine tune of near optimal solution only can be found when the algorithm is to be adaptable to the workload in cloud application console.

For development of task scheduler in cloud environment using PSO which is a metaheuristic approach, authors in22 formulated an approach to get balance between local and global search. They used a Linearly Descendent Adaptive inertia weight mechanism to obtain balance between local and global search. AdPSO evaluated with existing inertia schemes of PSO to evaluate metrics makespan, throughput, utilization of resources. Results proved that AdPSO shown impact on other inertia weight schemes of PSO.

Task mapping to virtual resources using task priorities with a Deep Reinforcement learning technique is proposed by authors in23. In the initial phase, task priorities calculated using task length, processing capacities of VMs. In next phase, calculated priorities fed to scheduler which uses DQN model to generated schedules. Workloads generated using realtime supercomputing worklogs and different statistical distributions. DRLBTSA evaluated over existing RATS-HM, MOABCQ, FCFS, RR, EDF algorithms and results revealed that makespan, Energy consumption are minimized for both random, realtime worklogs. Another Deep Reinforcement learning technique is formulated by authors in24 for tackling bursty and fluctuation requests in cloud paradigm with heterogeneous resources. For the adaptability of DRL, tuning of hyperparameters is required i.e. tuning of learning rate, activation function while scheduling of tasks are generated using PSO, Firefly algorithms. Proposed DRLPSOFA evaluated over existing PSO, firefly algorithms and after evaluation of cost, makespan metrics over existing approaches DRLPSOFA shown huge impact over compared approaches.

Scheduling of tasks due to number of dependencies in workflows and their complexities is difficult for metaheuristic approaches. This reason motivates authors in25 proposed a TS approach by combining GA with DRL technique to schedule tasks. In the initial level agents in reinforcement learning disperses tasks over VMs. In the second step, resources will be provisioned to each agent and utilization of resources is maximized by corresponding learned agent with RL based approach reward. It evaluated over classical GA, MOHEFT, MCP approaches. From the examined results of proposed approach over existing algorithms there is clear improvement for makespan, resource utilization. To develop cost effective task scheduling in cloud paradigm, authors in26 formulated task scheduling process which hybridizes metaheuristic algorithms by mutating the metaheuristic algorithms. They used different metaheuristic algorithms i.e. PSO, Firefly, Tabu. All these approaches are mutated and formulated to schedule tasks with resource intensity. Proposed MDRLSCIOT evaluated over each other i.e. (mutated approaches combined with DRL). From observed results PSO, firefly mutated DRL obtained minimum makespan and maximum utilization of resources over other algorithms. Task scheduling in cloud paradigm cannot be solved by a metaheuristic approach as mentioned above with different algorithms. In the same way, authors in27 formulated a task scheduler which is a combination of ACO, neural networks. ACO algorithm schedules tasks over different VMs but in complex situations while handling multiple objectives in cloud paradigm with more dependent tasks and complex workflows ACO cannot schedule tasks effectively. Therefore, after dispersion of tasks in cloud paradigm with ACO, neural networks optimizes generation of schedules as they are connected with hidden layers and extracts features effectively when compared with metaheuristic techniques. It evaluated over classical ACO to evaluate execution time, resource cost using LB-ACO. Results showed significant improvement over compared algorithms for parameters mentioned above.

Dynamic TS in cloud environment is still a challenge as many existing algorithms suffer from adaptability of workload, lack of training, unbalanced load in the system. Therefore, in29 authors formulated multi objective task scheduler that addresses energy consumption, response time as primary evaluation metrics in their scheduling of tasks in cloud-edge paradigm. They used A3C algorithm as methodology to design this task scheduler. It is implemented using edge-cloud-co simulator with dataset as input. It is compared over state of art approaches. From simulated results, it was observed A3C showed impact over compared techniques for above specified parameters.

Energy Consumption is a crucial factor to be considered in cloud model as it impacts both the CSP and cloud user in terms of cost incurred to the services consumed in cloud model. In order to address this energy cost authors in30 proposed a task scheduler which tackles energy cost using a Deep Reinforcement learning approach. It is developed by combining Q-learning and other RL approaches. It is simulated on workflowsim and evaluated against crow search based algorithm and task duplication scheduling approaches. Results shown impact of OIRSDRL over compared approaches by tackling energy consumption, task approval rate. For handling dynamic workloads in cloud computing, in31 a cost aware task scheduling framework developed using Deep Learning, Deep reinforcement learning based on selectors. In both the frameworks data labelled used for training on both data applied for DL model and DRL model. In both the approaches, authors aimed at minimization of cost by giving weights to cost parameters and given the training accordingly. It is evaluated with state of art algorithms. Results showed that DRLS minimizes cost effectively rather than other approach.

Adaptive task scheduling algorithm proposed in32 using algorithms GA, ACO. In the initial phase of this approach, local search process starts by using GA population to disperse the tasks. For the effective convergence towards the solutions and adaptability towards dynamic workload ACO speedup the procedure of speed up the scheduling and optimizes generated schedules to get near optimal solution. It compared against ACO, GA algorithms. Evaluated results revealed GAACO minimizes makespan, SLA Violation, improves resource utilization compared with baseline algorithms.

Authors in6 proposed a variation of ACO model which converges towards solution quickly and adaptable to the environment of cloud model. Therefore, a hybrid weighted ACO model developed by them which can converge towards the solution quickly while minimizing makespan, cost in the scheduling process over the baseline approaches ACO, GA. Task Scheduling in heterogeneous systems is a difficult approach when it is tackled by heuristic approaches as data comes onto cloud application console from heterogeneous resources. Therefore, to handle heterogeneous environments and high-performance computing systems a machine learning approach is necessary to handle such a huge workloads and dynamic environment. Therefore, in33 a task scheduler proposed which formulates task scheduler using Markov decision process in the first phase to formulate TSP and DDQN used to allocate tasks to corresponding VMs in an optimized manner to generate schedules. It is evaluated over state of art algorithms and proposed DDQN-TS minimized response time of tasks, distributed tasks effectively by balancing load among VMs for different scenarios of scheduling.

Table 1 Summarization of task scheduling techniques proposed by various authors.

Authors	Technique used	Parameters addressed	Limitations	
8	RATS-HM	Resource Utilization, Energy Consumption.	Resource cost is not evaluated	
9	MOABCQ	Makespan, cost, utilization of resources, throughput, degree of imbalance.	Not conducted simulation in multi cloud environment.	
10	AINN-BPSO	Task Completion time, Cost.	Resource cost is not evaluated.	
11	QL-HEFT	Makespan, Average Response time.	Focused on makespan but not on other operational parameters	
12	LB-HEFT	Load Balance, Makespan, Resource Utilization	Multi cloud environment is not considered.	
13	TS-DT	Makespan, Resource utilization, Load balancing	Multiple objectives are not considered in multi cloud environment.	
14	TDSA	Makespan	Resource cost is not considered.	
15	HGSA	Makespan, cost, schedule length ratio	Multi cloud environment is not considered in simulation.	
16	DRL-LSTM	Resource Cost, memory usage, Execution time.	Energy consumption not considered.	
17	DRLPPSO	Accuracy, Energy Consumption, Task Completion time, speedup.	Hybrid approach used but not considered in multicloud model.	
18	DRFTSA	Makespan, energy consumption	Resource cost is not evaluated.	
19	DRLSTS	Makespan, Energy consumption, throughput, resource utilization.	Not focused on resource cost in multi cloud environment.	
20	TSRAACODRL	Makespan, Resource utilization, Power Consumption.	Not conducted simulation in multi cloud environment.	
21	OPSO	Makespan, Task Completion time	Not focused on Resource cost.	
22	AdPSO	Makespan, throughput, Resource utilization	Not focused on Resource cost.	
23	DRLBTSA	Makespan, Energy Consumption.	Not focused on Resource cost.	
24	DRLPSOFA	Makespan, cost.	Simulation not conducted in multi cloud environment.	
25	SARSAGA	Makespan, utilization of resources	Applicable for single cloud environment.	
26	MDRLSCIOT	Makespan, Resource cost, Resource utilization	Not addressed energy consumption.	
27	LB-ACO	Execution time, Resource cost.	Applicable for single cloud environment.	
28	FTTHDRL	SLA based trust parameters, Resource cost, failure rate	Energy consumption not addressed.	
29	A3C	Average Response time, Average Energy Consumption	Applicable for single cloud environment.	
30	OIRSDRL	Task Approval rate, Load balancing.	Resource cost is not addressed.	
31	DLS, DRLS	Resource Cost	Single objective addressed.	
32	ALTS	Makespan, SLA Violation, Resource utilization	Simulation not conducted in multicloud environment.	
33	HWACOA	Makespan, Cost	Consumption of energy is not addressed.	
34	DDQN-TS	Task Response time, Load balancing	Simulation not conducted in multicloud environment.	
35	MTAC-IBEA	Resource utilization, Energy Consumption, Security	Simulation not conducted in multicloud environment.	
36	AIRL	Success Rate, Utilization Rate, Response time	Resource cost is not addressed.	
37	CNN-MBO	Energy Consumption, Resource utilization, Response time	Simulation not conducted in multicloud environment.	
38	SAC	Average Delay, Completion Ratio	Resource cost is not addressed.	
39	ELHHO	Schedule length, Execution Cost, Degree of imbalance, throughput	Simulation not conducted in multicloud environment.	
40	DNQ-DL	Makespan, Load balance	Simulation not conducted in multicloud environment.	
41	Q-Learning	Makespan, Average waiting time	Resource cost is not addressed.	

From the above Table 1 and literature in Sect. 2 gives overall summarization of existing task schedulers and how they addressed task scheduling problem in cloud paradigm. Many existing authors used either metaheuristic approaches or ML based or DRL based approaches to tackle task scheduling but still adaptability of the scheduler to take decisions dynamically according to the sudden changes occurred in workload is still lacking in this scenario. Therefore, we formulated an adaptive task scheduler which segment all tasks into sub tasks which comes cloud console and fed them according to their size, execution time, bandwidth requirement, communication time onto suitable VM with the improved Asynchronous Advantage Actor critic (ATSIA3C) to minimize makespan, resource cost, energy consumption in multicloud environment.

Problem formulation and system architecture

Section 3 discusses Problem formulation, System architecture of ATSIA3C. Initially, we assumed number of tasks as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{k1}=\{{T}_{1},{T}_{2},{T}_{3}\dots,{T}_{j}\dots{T}_{k1}\}$$\end{document}, number of VMs as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{n1}=\{{V}_{1},{V}_{2},\dots{V}_{n}.,{V}_{n1}\}$$\end{document}, Physical nodes to be indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P{M}_{m1}=\{P{M}_{1},P{M}_{2},\dots P{M}_{m1}\}$$\end{document} and datacenters to be indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D{C}_{i1}=\{D{C}_{1},D{C}_{2},D{C}_{3},\dots D{C}_{i1}\}$$\end{document}. We formulated the problem of task scheduling in such a way that initially tasks are segmented and set of sub tasks according to their size and data interchange among sub tasks to be mapped to VMs according to the processing capacity with multi cloud environment by using Improved Asynchronous Advantage Actor Critic algorithm while minimizing makespan, Resource cost, Energy Consumption. Initially, all the cloud users submits different tasks onto cloud platform. All tasks are captured from the users by the cloud broker. All the tasks with users who are having valid SLA submit to task manager. Task manager will perform task segmentation of all incoming tasks and all sub tasks according to their size, runtime capacity, execution time are grouped together and fed to ATSIA3C scheduler which is integrated with Deep Reinforcement learning technique looks for the corresponding constraints and checks resource availability through resource manager in multi cloud environment to map sub tasks accordingly onto VMs with respect to their processing capacities while evaluating makespan, resource cost, energy consumption. Proposed System architecture of ATSIA3C showed in Fig. 1.For the above process, so as to design mathematical modeling we have used various notations in the below section and all the mathematical notations used in system architecture is mentioned in Table 2.

Fig. 1 Proposed system architecture for ATSIA3C.

Table 2 Mathematical notations used in proposed system architecture for ATSIA3C.

Notation	Meaning	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{k1}$$\end{document}	Assumed number of tasks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{n1}$$\end{document}	Assumed number of VMs	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P{M}_{m1}$$\end{document}	Assumed number of Physical Machines	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$D{C}_{i1}$$\end{document}	Assumed number of Datacenters	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${|T}_{j}|$$\end{document}	Total sub tasks in task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j$$\end{document}	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y$$\end{document}	Task dispersion Matrix	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${x}_{k1}^{lkuv}$$\end{document}	Data Interchange between different Physical machines for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${k}_{1}$$\end{document}tasks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S{D}_{j}$$\end{document}	Size of data blocks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S{T}_{j}^{lk}$$\end{document}	Interchanging amount of data between sub tasks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$SubET$$\end{document}	Sub task execution time matrix.	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s{l}_{j}$$\end{document}	Length of sub task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j$$\end{document}.	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${pr}_{n1}$$\end{document}	Processing capacities of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${n}_{1}$$\end{document} VMs.	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$com{{m}_{time}}_{j}$$\end{document}	Communication time of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j$$\end{document} sub tasks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B{D}_{j}^{lk}$$\end{document}	Bandwidth between two sub tasks \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l,k$$\end{document}for a task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{j}$$\end{document}.	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Ex{{e}_{time}}_{n}$$\end{document}	Execution time of sub tasks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Overal{l}_{Ex{e}_{time}}$$\end{document}	Overall Execution time of all considered sub tasks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f{{i}_{time}}_{k1}$$\end{document}	Finish time of considered tasks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$de{T}_{k1}$$\end{document}	Deadlines of considered tasks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ms{p}_{k1}$$\end{document}	Makespan for all considered tasks	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$En$$\end{document}	Energy Consumption matrix considering execution time	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$To{t}_{En}$$\end{document}	Total Energy Consumption	
\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Re{s}_{cost}$$\end{document}	Resource cost	

In this research, we formulate problem in such a way that tasks are segmented as sub tasks from each task submitted to cloud broker and if we assume \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{j}^{s}$$\end{document} is the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${s}^{th}$$\end{document} sub task of task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j$$\end{document}. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${|T}_{j}|$$\end{document} indicates total number of sub tasks in task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j$$\end{document}. It can be modeled mathematically and represented by using below Eq. 1.1 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$To{t}_{su{b}_{tasks}}=\sum_{j=1}^{n}\left|{T}_{j}\right|$$\end{document}

After task segmentation all sub tasks are fed to ATSIA3C scheduler which need to take decision based on processing capacity of VM and sub tasks of a tasks are interchange the data in between different VMs based on length and execution time of sub tasks. For task dispersion on to suitable VMs, we use a task dispersion matrix which is represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y$$\end{document}. It is calculated by Eq. 2.2 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Y=\left[\begin{array}{cccc}{Y}_{11}&{Y}_{12}&\ldots&{Y}_{1n}\\{Y}_{21}&{Y}_{22}&\dots&{Y}_{2n}\\:&:&:&:\\{Y}_{e1}&{Y}_{e2}&\dots&{Y}_{en}\end{array}\right]$$\end{document}

From above Eq. 2. where.

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Y}_{jn}=\left\{\begin{array}{c}1\,if\,a\,subtask\,of\,{T}_{j}\,mapped\,to\,a\,VM{V}_{n1}\\0\,if\,a\,subtask\,of\,{T}_{j}\,is\,not\,mapped\,onto\,{V}_{n1}\end{array}\right.$$\end{document}

3 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sum_{n=1}^{j}{Y}_{jn1}=\left|{T}_{j}\right|$$\end{document}

Interchange of data between different physical machines of sub tasks \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{j}$$\end{document} is indicated using \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${x}_{k1}^{lkuv}$$\end{document}. It is to be evaluated as follows using Eq. 4.4 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${x}_{k1}^{lkuv}=\left\{\begin{array}{c}1if{Y}_{ju}and{Y}_{jv}=1\,where\,u\,\ne\,v\\0\,otherwise\end{array}\right.$$\end{document}

Where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$landk$$\end{document}are sub tasks in task set \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{k1}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$uandv$$\end{document} are virtual machines in the set of VMs \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{n1}$$\end{document}. In this research, while interchanging data between different sub tasks it considers size of data blocks indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S{D}_{j}$$\end{document}. After that interchanging amount of data between sub tasks can be indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S{T}_{j}^{lk}$$\end{document} i.e. interchanging of data between sub task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S{T}_{j}^{l}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S{T}_{j}^{k}\forall,k=\{\text{1,2},3\dots\left|{T}_{j}\right|\}$$\end{document}. All these sub tasks will run parallelly assigned to different VMs with different processing capacities. The main aspect of our research is to segment tasks and these sub tasks according to size of data blocks in sub tasks and sub task lengths mapped onto VMs according to their processing capacities.

Sub task execution time on VMs using a distribution matrix and it is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$SubET.$$\end{document} It is to be defined as sub task execution time of sub task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{j}$$\end{document} on Virtual machine \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{n1}$$\end{document}. It is expressed using Eq. 5.5 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$sue{T}_{jn}=\frac{s{l}_{j}}{{pr}_{n1}}$$\end{document}

Where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$s{l}_{j}$$\end{document} indicates sub task length or size of data blocks in sub tasks. The sub task execution matrix is expressed as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$SubET=\left\{sue{T}_{jn}\right\}$$\end{document} where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$j=\{\text{1,2},\dots k1\}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n=\{\text{1,2},\dots n1\}$$\end{document}. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$sue{T}_{jn1}$$\end{document} indicates execution time of a sub task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{j}$$\end{document} which is to run on VM \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{n}$$\end{document}. In this research to calculate overall execution time on a VM is calculated by considering communication time of sub task executing on VM and sub task execution time calculated in Eq. 5.Initially, we calculated communication time using below Eq. 6.

6 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$com{{m}_{time}}_{j}=\sum_{l=1}^{\left|{T}_{j}-1\right|}\sum_{k=l+1}^{\left|{T}_{j}\right|}{x}_{k1}^{lkuv}*\left(\frac{S{T}_{j}^{lk}}{B{D}_{j}^{lk}}\right)$$\end{document}

Where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S{T}_{j}^{lk}$$\end{document} indicates interchange of data between sub tasks \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k$$\end{document}. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B{D}_{j}^{lk}$$\end{document} indicates bandwidth between two sub tasks \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k$$\end{document} of task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{j}$$\end{document}.

After evaluating Communication using Eq. 6. we evaluated execution time on a VM \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${V}_{n}$$\end{document}. It is calculated using Eq. 7.7 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Ex{{e}_{time}}_{n}=\sum_{j=1}^{n}{Y}_{jn}*sue{T}_{jn}$$\end{document}

We have already mentioned that all these sub tasks are executed in parallel and therefore overall execution time of all sub tasks are using Eq. 8.8 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Overal{l}_{Ex{e}_{time}}=\text{max}\left(Ex{{e}_{time}}_{n}\right)+\text{m}\text{a}\text{x}\left(com{{m}_{time}}_{j}\right)$$\end{document}

Now after evaluating execution time, communication time and we have calculated finish time of tasks as we have posed deadline constraint in our research. Finish time of all tasks are calculated by Eq. 9.9 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f{{i}_{time}}_{k1}=\sum{V}_{n1}+Ex{{e}_{time}}_{n}$$\end{document}

We have assumed that a deadline constraint is posed as finish time of all tasks should be less than the deadline of tasks i.e. a task or set of sub tasks should be assigned to a VM until the current execution of tasks should be finished before the deadline of tasks. It is calculated using Eq. 10.10 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f{i}_{tim{e}_{k1}}\le de{T}_{k1}$$\end{document}

After evaluating finish time by mentioning deadline constraint we evaluated primary metric i.e. makespan and it is calculated using Eq. 11.11 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ms{p}_{k1}=\text{m}\text{a}\text{x}\left(f{i}_{tim{e}_{k1}}\left({V}_{n1}\right)\right)$$\end{document}

In this research, we considered energy consumption incurred for execution of tasks. We have not considered energy incurred due to communication time. It is expressed as a distribution matrix and it is indicated as below using Eq. 12.12 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$En=\left\{{e}_{jn}\right\}$$\end{document}

Where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${e}_{jn}$$\end{document}is energy incurred by a sub task \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${T}_{j}$$\end{document} on a VM \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n.$$\end{document} Therefore, total energy consumption is calculated using below Eq. 13.13 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$To{t}_{En}=\sum_{j=1}^{k1}\sum_{n=1}^{n1}{Y}_{jn}*{e}_{jn}$$\end{document}

After formulation of both energy consumption and makespan we are interested to calculate resource cost. This is the important parameter to be considered in task scheduling in cloud paradigm as ineffective mapping of tasks to VMs increases resource cost. A task to be mapped to a VM based on its length, runtime capacity of task, processing capacity of VMs in order to optimize memory cost, CPU cost, memory cost. Resource utilization impacts cost of resources as in many cloud datacenters faces overutilization or underutilization of resources which increases or decreases the cost of resources drastically. So, optimization of resource cost plays a major role in cloud paradigm. In order to optimize resource cost at the initial level, a suitable resource should be made available to a task which computes effectively to optimize resource cost. Therefore, to minimize resource cost, we used a multi cloud environment in which all incoming tasks are scheduled using ATSIA3C checks for suitable resource in multicloud environment and schedules tasks onto appropriate virtual resources in which resource cost is lower. Therefore, resource cost is calculated using below Eq. 14.14 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$Re{s}_{cost}=\sum_{n1=1}^{{V}_{n1}}\frac{Tasks\,Running\,Cost\,\left({T}_{k1}\right)*Memory\,consumed\,for\,{T}_{k1}}{{V}_{n1}*P{M}_{m1}}$$\end{document}

Methodology used in ATSIA3C

Section 4 discusses proposed methodology used in our research i.e. improved A3C(Asynchronous Advantage Actor critic) (ATSIA3C) algorithm which is a deep reinforcement learning based approach. It consists of different components i.e. Actor and critic networks. Actor network maps various tasks considered from action space to be mapped to state space i.e. VMs. It generates schedules by imposing policies imposed in actor network. Critic network evaluates actor network actions and guides them according to the policies posed in the actor network. It is a reward based approach as it is deep reinforcement learning model. In this approach actor and critic network threads runs independently as it is an asynchronous approach. For every reward if a positive reward generated from actor network, critic network at that independent thread saves that data and checks for the best possible mapping till that iteration and if it is best mapping solution then it will be updated at the global thread. For a negative reward, it will check with its policy posed in actor network and recommends not to repeat that action in future by learning from the workload appears onto the cloud application console.

Generally, A3C algorithm uses fully connected network which disperses less efficiency as workload in cloud application console is dynamic. Therefore, it is difficult to adapt to the cloud environment to learn and extract complex features from the workload generated from diversified users. This reason motivates us to use improved A3C uses Residual Convolutional Neural Network (RCNN) which easily identify complex relationships and extract features from the generated workload. Residual layer in RCNN skips some of hidden layers in fully connected network and provides by giving direct connection which uses to updation speed of actor critic network parameters, pooling layer improves speed in feature extraction. Entire data which is to be given as input to actor network is given as state vectors which is two dimensional and a flatten function is used to convert two-dimensional state vector to one-dimension vector. This data need to pass through the hidden layers which helps in improvement of learning of the network. The data should pass through the two hidden layers as mentioned above. First hidden layer mentions about input data, another hidden layer consists of number of neurons which is 256 in this case. Kernel size, step size are assumed as 3 and 1 respectively. The one dimensional data is after passing through the hidden layers helpful to extract features effectively. This data is processed through an activation function called as SoftMax at the dense layer to obtain output which lies in between the range 0,1. For a given time\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t$$\end{document}, scheduling interval of time is represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$i{n}_{t}$$\end{document}. For all the tasks which is to be generated through state space \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$st{a}_{t}$$\end{document}, mapping of actions to be through action space which is represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ac{t}_{t}$$\end{document}. This process is carried out through individually at all actor networks and it is evaluated by critic network which leverages a decision either it is a positive or negative reward and reward for that interval of time \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t$$\end{document} is represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$re{w}_{t}$$\end{document}. After generating decision critic network evaluate the action pertained by actor network and a policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau$$\end{document} is to be used to guide the actor network to improve learning and maximize reward for every iteration in the process of scheduling. The tuple for this model is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$<st{a}_{t},ac{t}_{t},\tau,re{w}_{t},{v}_{fn}>$$\end{document}. The above mentioned tuple have to be explained in a detailed manner.

State space

In the above mentioned tuple which is indicated different items in tuple and state space represents set of states which are represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$sta=\{st{a}_{1},st{a}_{2},st{a}_{3}\dots st{a}_{t}\}$$\end{document}. where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$st{a}_{t}$$\end{document} consists of physical host information which indicates features and feature information of tasks. These are represented as a matrix \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$st{a}_{t}=\left\{\left(finf{o}_{t}^{P{M}_{m1}},finf{o}_{t}^{{T}_{k1}}\right)\right\}$$\end{document}. Where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$finf{o}_{t}^{P{M}_{m1}}$$\end{document} indicates feature information of physical machines at an interval \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$finf{o}_{t}^{{T}_{k1}}$$\end{document} indicates feature information of considered tasks at an interval \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t$$\end{document}.

Action space

Action space indicates mapping of set of tasks dispersed in state space to be mapped onto virtual resources. It is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$act=\{ac{t}_{1},ac{t}_{2},ac{t}_{3}\dots ac{t}_{t}\}$$\end{document}. This action space is depends on mapping variable indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${ac{t}_{t}=\{d}_{k1n1}\}$$\end{document} in a time interval \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t.$$\end{document} Mapping of tasks to virtual resources depends on decision variable after segmentation of tasks.

Policy

In improved A3C, policy is to be incurred into this approach which is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau$$\end{document}. It will guide the decisions generated by actor network and evaluated by critic network to maximize reward in every iteration. It is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau=\left(ac{t}_{t}\right|st{a}_{t})$$\end{document}. When it is employed by neural network policy is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau=\left(ac{t}_{t}\right|st{a}_{t};{\eta}_{b})$$\end{document}.

Reward function

This reward function generates outcome of whether the mapping gives positive or negative outcome as a reward. It is represented as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$re{w}_{t}(st{a}_{t},ac{t}_{t})$$\end{document}. It is calculated using below Eq. 15.15 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$re{w}_{t}=\text{m}\text{i}\text{n}(ms{p}_{k1},To{t}_{En},Re{s}_{cost})$$\end{document}

In the mapping decision, if the outcome rendered is negative, the policy should guide the scheduling process and how much deviation occurred in the scheduling process from the expected process is to be known by discount factor reward. It is calculated using Eq. 16.16 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$g{d}_{t}=re{w}_{t+1}+\phi.re{w}_{t+2}+\dots+{\phi}^{t-T-1}.re{w}_{t}$$\end{document}

Value function

Value function indicates expectation of state, action spaces. Value function for state space is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$va{l}^{\tau}\left(st{a}_{t}\right)$$\end{document}. It is calculated by Eq. 17.17 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${vfn}^{\tau}\left({sta}_{t}\right)=exp{t}_{\tau}[re{w}_{t}+\phi.re{w}_{t+1}+{\phi}^{t-T-1}.re{w}_{t}|{sta}_{t}]$$\end{document}

Value function for state and action space is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${q}^{\tau}\left(st{a}_{t},ac{t}_{t}\right)$$\end{document}. It is calculated using Eq. 18.18 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${q}^{\tau}\left({sta}_{t},{act}_{t}\right)=exp{t}_{\tau}\left[{gd}_{t}|{sta}_{t},{act}_{t}\right]=exp{t}_{\tau}[re{w}_{t}+\phi.{q}^{\tau}\left({sta}_{t+1},{act}_{t+1}\right)+\dots+|{sta}_{t},{act}_{t}]$$\end{document}

In Eqs. 17, 18 value functions calculated using reinforcement approach but we can also calculate value function using neural networks with network parameter\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\lambda}_{a}$$\end{document}. It is indicated using Eq. 19. 19 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${vfn}^{\tau}\left({sta}_{t}\right)\approx\,vfn({\text{s}\text{t}\text{a}}_{\text{t}};{\lambda}_{a}), {q}^{\tau}\left({sta}_{t},ac{t}_{t}\right)\approx\,q({sta}_{t},{act}_{t};{\lambda}_{a})$$\end{document}

Training network

In this algorithm, all threads runs in parallel in which each agent independently execute all threads which consists of different tasks input from state space and it maps tasks to corresponding virtual resources. All these threads which consists of agents renders their rewards which are either positive or negative and submits it to global network which evaluates actual and expected mappings and these decisions are guided by policy to render good decision i.e. maximum reward. In training the network, a policy and value functions are to be given for all the independent threads i.e. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau\left({act}_{t}\right|{sta}_{t};{\eta}_{b}^{\sim})$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$q({sta}_{t},{act}_{t};{\lambda}_{a}^{\sim})$$\end{document}. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\eta}_{b}^{\sim}$$\end{document}, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\lambda}_{a}^{\sim}$$\end{document} are control parameters in network. For every iteration in the process for considered set of states \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${sta}_{t}$$\end{document}, action i.e. (schedules over tasks to VMs) to be leveraged is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${act}_{t}$$\end{document} which gives a reward indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$re{w}_{t}$$\end{document} with policy \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau$$\end{document} guides to generate maximum reward by decreasing discount value. In cloud computing paradigm different workloads to be appeared with different processing capacities, therefore, for every iteration for the algorithm runs in this scheduling process if the same policy is applied in the algorithm, different gradient values will be generated. There is a necessity to use gradient ascent process to identify cumulative gradient. It is represented using Eq. 20.20 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\nabla}_{{\eta}_{b}^{\sim}}exp{t}_{\tau}\left[{gd}_{t}\right]={\nabla}_{{\eta}_{b}^{\sim}}{log}\tau({act}_{t}\left|{sta}_{t};{\eta}_{b}^{\sim}\right){gd}_{t}$$\end{document}

Gradient ascent improves its value but too much usage of gradient ascent slow down the learning process of A3C network. Therefore, we have to impose a policy which should balance the workload adaptively and generate schedules according to the workload posed by users. Therefore, advantage function need to be used in this process and it is indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$av({sta}_{t},{act}_{t})$$\end{document} and it should be subtracted from base function indicated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$b\left(t\right)$$\end{document}. It also improves the gradient values and converge towards expected values in an efficient manner. It is calculated by Eq. 21.21 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}{{\eta}}_{{b}}={d}{{\eta}}_{{b}}+{\nabla}_{{{\eta}}_{{b}}^{\sim}}{log}{\tau}{log}{\tau}({{a}{c}{t}}_{{t}}\left|{s}{t}{{a}}_{{t}};{{\eta}}_{{b}}^{\sim}\right){a}{v}\left({{s}{t}{a}}_{{t}},{{a}{c}{t}}_{{t}}\right)=d{\eta}_{b}+{\nabla}_{{\eta}_{b}^{\sim}}{log}\tau{log}\tau({act}_{t}\left|{sta}_{t};{\eta}_{b}^{\sim}\right)\left(q\right({sta}_{t},{act}_{t};{\lambda}_{a}^{\sim})-vfn({sta}_{t};{\lambda}_{a}^{\sim}\left)\right)$$\end{document}

For the subsequent state, reward is calculated and it is updated using Eq. 22.22 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$vfn\left({sta}_{t};{\lambda}_{a}^{\sim}\right)=vfn\left({sta}_{t};{\lambda}_{a}^{\sim}\right)+\gamma(re{w}_{t}+\phi.vfn\left({sta}_{t+1};{\lambda}_{a}^{\sim}\right)-vfn\left({sta}_{t};{\lambda}_{a}^{\sim}\right))$$\end{document}

In the training process, after evaluation of reward for every iteration, temporal error checkpoint to be evaluated using Eq. 23.23 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d{\lambda}_{a}=\frac{\omega{\left[re{w}_{t}+\phi\,vfn\left({sta}_{t+1};{\lambda}_{a}^{\sim}\right)-vfn\left({sta}_{t};{\lambda}_{a}^{\sim}\right)\right]}^{2}}{\omega{\lambda}_{a}^{\sim}}$$\end{document}

Updation of parameters

In the process of Scheduling, for every iteration, all set of tasks are mapped to virtual resources using Improved A3C. After updating all agents and collecting all gradients in independent threads and need to submit at global network and finally need to identify the amount of discount to be generated and how much convergence appeared towards the expected values of scheduling decision. Therefore, for subsequent states, actions can be generated is calculated by Eq. 24.24 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\eta}_{b}={\eta}_{b}+\gamma.d{\eta}_{b},{\lambda}_{a}={\lambda}_{a}+{\Gamma}d{\lambda}_{a}$$\end{document}

Proposed adaptive task scheduler in multi cloud environment using improved asynchronous advantage actor critic (ATSIA3C) algorithm

The above Fig. 2 indicates flow of proposed ATSIA3C. This proposed algorithm schedules tasks both at IaaS level and VM level as VMs are resided virtually on top of physical nodes. Initially the algorithm starts by initializing global and network specific parameters. All the input tasks given through state space are segmented and dispersed onto VMs using Eq. 2 while considering data interchange between tasks, communication time, size of sub tasks to be dispersed onto VMs and its corresponding processing capacity. After giving tasks and VMs as input in state space, mapping of tasks to VMs considered are done in action space. Both state space, action space are performed in actor network. Outcome of mapping i.e. reward is to be evaluated through the algorithm and it should be evaluated in critic network using a policy imposed in ATSIA3C. Then we will check the outcome of critic network and checks whether the generated schedules are in line with the expected schedules to be generated. After this step, parameters are evaluated i.e. makespan, energy consumption, Resource cost. Now after evaluating the outcome of critic network if parameters are improved over the last iterations then update those parameters as best, stop process if not calculate cumulative gradient value and simultaneously check the policy to guide the scheduling process in optimized manner.Fig. 2 Flow of ATSIA3C.

Simulation and results

Section 5 discusses simulation, results of proposed (ATSIA3C)Adaptive Task Scheduler using Improved A3C. All simulations are conducted on Cloudsim toolkit. For entire simulation, initially we used fabricated workloads we have generated using different distributions indicated as un1 which is uniform distribution, nor2 which is normal distribution, ls3 which is left skewed distribution, ri4 which is right skewed distribution. After this we used scientific realtime parallel worklogs from HPC2N, NASA computing clusters which are represented as hp5, na6. The below subsections discusses results, simulations in detailed manner. Subsection 5.1 discusses configuration settings of simulation, 5.2 discusses about makespan evaluation with ATSIA3C, 5.3 presents Resource cost evaluation with ATSIA3C, 5.4 discusses Energy Consumption evaluation with ATSIA3C. Simulation ran for 100 iterations. For evaluating efficacy of our scheduler we compared ATSIA3C with existing algorithms AINN-BPSO, RATS-HM, MOABCQ to evaluate above said parameters. Subsection 5.5 discusses about discussion and analysis of results.

Configuration settings for ATSIA3C

Subsection 5.1 discusses configuration settings used in simulation of ATSIA3C. Extensive simulations are conducted for proposed ATSIA3C using Cloudsim41. Entire simulation setup was ran in a MAC book with 32 GB Memory, 6 TB Storage capacity, with M1 processor in the configuration. We ran simulation using 100 to 1000 tasks for 100 iterations. Initially as mentioned in the introduction of Sect. 5 we used fabricated workload for simulation in first stage i.e.(un1,nor2, ls3,ri4) and after this we conducted simulation in second stage using supercomputing worklogs HPC2N42NASA43 traces. The detailed settings are presented in Table 3.

Table 3 Configuration settings used for ATSIA3C.

Name of the entity	Quantity	
Tasks	1000	
VMs	200	
Task lengths	750,000	
Memory capacity of PH	32GB	
Bandwidth of PH	1100MBPS	
Memory of VM	16 GB	
PH storage capacity	6 TB	
VM bandwidth	5 Mbps	
PH OS	MAC OS	
Datacenters	25	
GPU model	Nvidia GeForce GTX 1080 Ti	

Makespan evaluation using ATSIA3C

Subsection 5.2 discusses makespan evaluation with ATSIA3C by giving input workload as fabricated uniform distribution of tasks represented as un1 in our research. The main reason for selecting makespan as an evaluation parameter is that it directly affects QoS of CSP. Inefficient scheduler impacts increase of makespan which ruins Quality of Service of CSP. Therefore, to address this issue we choose makespan is one of the evaluation parameter to schedule tasks appropriately by scheduling tasks precisely using ATSIA3C in multi cloud environment. For this to happen, we evaluated makespan of ATSIA3C with uniform distribution indicated as un1. The below Table 4; Fig. 3 indicates generated results for makespan using un1. To evaluate efficacy of proposed ATSIA3C we compared it over algorithms RATS-HM, MOABCQ, AINN-BPSO. We considered 100 to 1000 tasks for simulation. Dispersed makespan of all the techniques along with ATSIA3C are mentioned in below Table 4.

Table 4 Calculation of makespan using un1 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(un1)	728.24	794.67	683.18	597.34	
200(un1)	730.12	795.12	687.22	598.02	
300(un1)	733.34	796.87	690.45	599.15	
400(un1)	800.46	923.18	704.68	609.48	
500(un1)	947.12	1389.29	1297.16	689.27	
600(un1)	957.33	1398.31	1300.35	696.87	
700(un1)	984.27	1408.12	1303.56	699.99	
800(un1)	1004.58	1425.88	1402.45	705.47	
900(un1)	1246.88	1476.87	1587.18	745.87	
1000(un1)	1498.27	1532.13	1798.26	799.23	

Fig. 3 Calculation of makespan using un1 for ATSIA3C.

From the above Table 4; Fig. 3 it shown that even tasks are increased to 1000 ATSIA3C minimized makespan over existing algorithms for uniform distribution of tasks.

After evaluating makespan with un1 tasks, we have used fabricated dataset which consist of normal distribution of tasks indicated as nor2. Dispersed makespan of all the techniques along with ATSIA3C for nor2 are mentioned in below Table 4. From the below Table 5; Fig. 4 it is clearly shows that even when diversified tasks comes to cloud application console with different number variations, our proposed ATSIA3C outperforms by minimizing makespan over existing baseline algorithms.

Table 5 Calculation of makespan using nor2 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(nor2)	987.12	1092.25	984.36	783.18	
200(nor2)	988.28	1098.65	992.12	794.56	
300(nor2)	1024.76	1105.89	1046.77	799.99	
400(nor2)	1256.77	1209.37	1345.09	906.43	
500(nor2)	1346.27	1284.16	1443.12	924.07	
600(nor2)	1367.88	1358.12	1489.88	987.17	
700(nor2)	1456.13	1372.89	1495.15	1067.88	
800(nor2)	1458.79	1453.24	1567.22	1078.21	
900(nor2)	1499.35	1524.09	1678.19	1104.56	
1000(nor2)	1508.22	1647.76	1758.22	1146.11	

Fig. 4 Calculation of makespan using nor2 for ATSIA3C.

The below Table 6; Fig. 5 shows makespan evaluation for ATSIA3C using ls3 workload. Dispersed makespan for RATS-HM for considered tasks is 898.06, 915.37, 1521.26 respectively. Dispersed makespan for MOABCQ for considered tasks is 787.34, 1307.64, 1643.75 respectively. Dispersed makespan for AINN-BPSO for considered tasks is 818.04, 1456.46, 1756.32 respectively. Dispersed makespan for ATSIA3C with different tasks are 675.37, 843.12, 921.08 respectively. From the below Table 6; Fig. 5 it is clearly shown that there is clear improvement in makespan for ATSIA3C over other approaches with left skewed distribution of tasks ls3.

Table 6 Makespan evaluation using ls3 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(ls3)	898.06	787.34	818.04	675.37	
200(ls3)	899.23	804.88	832.87	694.87	
300(ls3)	902.18	921.36	856.98	705.36	
400(ls3)	904.87	1056.03	1035.28	745.22	
500(ls3)	915.37	1307.64	1456.46	843.12	
600(ls3)	923.66	1354.19	1498.74	856.98	
700(ls3)	940.16	1472.88	1504.57	867.22	
800(ls3)	1034.67	1487.43	1545.46	875.18	
900(ls3)	1248.92	1503.23	1623.99	894.03	
1000(ls3)	1521.26	1643.75	1756.32	921.08	

Fig. 5 Calculation of makespan using ls3 for ATSIA3C.

Below Table 7; Fig. 6 shows dispersed makespan for ATSIA3C with ri4 workload. Dispersed makespan for RATS-HM with different tasks are 697.26, 732.05, 1473.05 respectively. Dispersed makespan for MOABCQ with considered tasks are 757.96, 889.37, 1587.12 respectively. Dispersed makespan for AINN-BPSO with different tasks is 834.06, 767.11, 1609.35 respectively. Makespan for ATSIA3C with considered tasks is 576.13, 609.52, 1124.07 respectively. From the below Table 7; Fig. 6 it is evident that there is clear improvement in makespan with ATSIA3C even with increase in number of tasks for ri4 workload.

Table 7 Calculation of makespan using ri4 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(ri4)	697.26	757.96	834.06	576.13	
200(ri4)	698.67	760.12	842.37	584.28	
300(ri4)	703.44	772.45	848.74	589.16	
400(ri4)	712.78	805.46	853.19	592.93	
500(ri4)	732.05	889.37	767.11	609.52	
600(ri4)	756.99	924.13	854.37	626.18	
700(ri4)	805.89	946.88	926.18	705.77	
800(ri4)	956.77	1056.88	1125.99	823.12	
900(ri4)	1008.78	1248.89	1256.81	905.67	
1000(ri4)	1473.17	1587.12	1609.35	1124.07	

Fig. 6 Calculation of makespan using ri4 for ATSIA3C.

After evaluation of makespan with fabricated workloads, we evaluated makespan using Scientific realtime workloads captured from HPC2N, NASA worklog traces. Initially, we captured the trace of HPC2N and given it as input to ATSIA3C and other algorithms to evaluate makespan. The below Table 8; Fig. 7 indicates generated makespan for ATSIA3C using hp5. Makespan for RATS-HM using hp5 for 100, 500, 1000 tasks is 1521.15, 1876.25, 2354.03 respectively. Makespan for MOABCQ using hp5 for 100, 500, 1000 tasks is 1678.24, 2453.19, 3135.06 respectively. Makespan for AINN-BPSO using hp5 for 100, 500, 1000 tasks is 1924.17, 3025.27, 3067.18 respectively. Generated makespan for ATSIA3C using hp5 for 100, 500, 1000 tasks is 1236.71, 1562.97, 1688.26 respectively. From the below Table 8; Fig. 7 shows a clear improvement of makespan with hp5 workload over existing algorithms.

Table 8 Calculation of makespan using hp5 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(hp5)	1521.15	1678.24	1924.17	1236.71	
200(hp5)	1534.63	1843.66	1988.65	1321.5	
300(hp5)	1578.18	1905.71	2045.93	1424.56	
400(hp5)	1632.08	2045.57	2642.17	1487.19	
500(hp5)	1876.25	2453.19	3025.27	1562.97	
600(hp5)	1894.16	2567.22	3027.13	1578.32	
700(hp5)	1907.36	2631.18	3032.98	1589.13	
800(hp5)	1967.32	2845.88	3044.43	1608.32	
900(hp5)	2045.78	2868.21	3056.2	1612.57	
1000(hp5)	2354.03	3135.06	3067.18	1688.26	

Fig. 7 Calculation of makespan using hp5 for ATSIA3C.

After evaluating makespan using HPC2N trace i.e. hp5, NASA workload is used i.e. na6 to evaluate makespan for ATSIA3C. We evaluated makespan with na6 trace using other algorithms. Generated makespan for RATS-HM using na6 for 100, 500, 1000 tasks is 967.82, 1035.24, 1456.21 respectively. Makespan for MOABCQ using na6 for considered tasks is 785.19, 1187.26, 1687.15 respectively. Makespan for AINN-BPSO using na6 for 100, 500, 1000 tasks is 893.26, 1256.27, 1867.35 respectively. Makespan for ATSIA3C using na6 for 100, 500, 1000 tasks is 678.45, 936.03, 1035.17 respectively. Below Table 9; Fig. 8 clearly shown that proposed ATSIA3C minimized makespan significantly for na6 worklogs.

Table 9 Calculation of makespan using na6 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(na6)	967.82	785.19	893.26	678.45	
200(na6)	973.18	794.12	895.84	689.76	
300(na6)	976.88	802.36	904.32	732.17	
400(na6)	989.89	924.88	975.98	805.16	
500(na6)	1035.24	1187.26	1256.27	936.03	
600(na6)	1089.36	1198.46	1387.13	940.33	
700(na6)	1135.57	1256.55	1452.98	943.36	
800(na6)	1278.67	1389.12	1577.38	950.98	
900(na6)	1367.22	1396.09	1628.16	964.17	
1000(na6)	1456.21	1687.15	1867.35	1035.17	

Fig. 8 Calculation of makespan using na6 for ATSIA3C.

Resource cost evaluation using ATSIA3C

Subsection 5.3 discusses calculation of Resource cost with proposed ATSIA3C. The main reason to choose resource cost as one of evaluation metric is that in cloud environment various types of diversified tasks arises from various sources and it is difficult to map every task to the same type of virtual resource leads to increase of resource cost which is a burden to the CSP and to the user. Therefore, a multi cloud environment chosen with virtual resources placed in different environments and schedule upcoming tasks to map to suitable resources where resource cost is less for the benefit of CSP and cloud users. Initially, we calculated resource cost of ATSIA3C using various fabricated workloads as input to ATSIA3C. Later, we used HPC2N, NASA computing worklogs as input to algorithm and evaluated resource cost. For evaluating proposed algorithm, ATSIA3C compared with RATS-HM, MOABCQ, AINN-BPSO algorithms. The below Table 10; Fig. 9 shows resource cost of ATSIA3C using un1. Dispersed resource cost of all the evaluated approaches with un1 are mentioned in Table 10.

Table 10 Resource cost evaluation using un1.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(un1)	6.24	7.85	5.92	4.24	
200(un1)	6.33	7.88	5.97	4.35	
300(un1)	6.46	7.92	6.1	4.43	
400(un1)	6.8	8.05	6.4	4.8	
500(un1)	7.52	8.65	7.95	5.18	
600(un1)	7.71	8.72	7.98	5.21	
700(un1)	7.45	8.89	8.03	5.46	
800(un1)	7.67	8.91	8.12	5.51	
900(un1)	7.9	8.94	8.14	5.7	
1000(un1)	8.87	9.62	8.21	6.36	

Fig. 9 Calculation of resource cost using un1 for ATSIA3C.

From the above Table 10; Fig. 9 it is clearly shown that resource cost for ATSIA3C greatly minimized over existing approaches for uniform distribution of tasks.

After calculating resource cost with un1, nor2 workload is given as input to ATSIA3C algorithm and evaluated resource cost. The below Table 11 shows dispersed resource cost with all evaluated approaches using nor2.

Table 11 Calculation of resource cost using nor2 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(nor2)	6.28	5.76	5.08	4.56	
200(nor2)	6.32	5.82	5.21	4.37	
300(nor2)	6.45	6.77	5.78	4.12	
400(nor2)	6.6	7.18	6.84	4.05	
500(nor2)	7.39	9.27	8.12	3.87	
600(nor2)	7.42	9.73	8.02	3.58	
700(nor2)	7.48	9.48	7.32	3.33	
800(nor2)	7.53	9.68	7.11	3.26	
900(nor2)	7.64	9.72	7.09	3.18	
1000(nor2)	8.45	7.87	6.78	2.99	

Fig. 10 Calculation of resource cost using nor2 for ATSIA3C.

From the above Table 11; Fig. 10 it can be clearly observed that resource cost for ATSIA3C over other approaches greatly minimized with input workload distribution nor2.

The below Table 12 shows calculation of Resource cost using ATSIA3C by giving input workload as left skewed distribution i.e. ls3. Resource cost for RATS-HM with considered tasks is 8.37, 7.28, 9.37 respectively. Resource cost for MOABCQ with different tasks is 6.82, 8.93, 10.28 respectively. Resource cost of AINN-BPSO with different tasks is 6.03, 6.98, 11.25 respectively. Resource cost dispersed with ATSIA3C for all considered tasks is 4.27, 5.17, 3.88 respectively.

Table 12 Calculation of resource cost using ls3 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(ls3)	8.37	6.82	6.03	4.27	
200(ls3)	8.42	6.91	6.17	4.8	
300(ls3)	8.21	6.98	6.26	4.92	
400(ls3)	8.19	7.28	6.57	5.08	
500(ls3)	7.28	8.93	6.98	5.17	
600(ls3)	7.38	9.14	7.12	5.23	
700(ls3)	7.62	9.21	7.89	5.87	
800(ls3)	7.98	9.37	8.65	4.94	
900(ls3)	8.58	9.67	9.88	3.92	
1000(ls3)	9.37	10.28	11.25	3.88	

Fig. 11 Calculation of resource cost using ls3 for ATSIA3C.

From the above Table 12; Fig. 11 resource cost calculated for ATSIA3C is significantly minimized over existing baseline algorithms for left skewed distribution as an input to proposed algorithm.

The below Table 13 indicates calculation of resource cost using ri4 for ATSIA3C for 100 to 1000 tasks. Generated resource cost for RATS-HM with ri4 workload for 100, 500, 1000 tasks are 8.78, 10.54, 11.25 respectively. Resource cost rendered for MOABCQ with ri4 workload for different tasks are 9.25, 8.25, 10.13. Resource cost generated with AINN-BPSO with ri4 workload for different tasks are 6.37, 7.92, 8.02 respectively. Finally, resource cost dispersed with ATSIA3C for all considered tasks are 5.18, 6.12, 4.13 respectively.

Table 13 Calculation of resource cost using ri4 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(ri4)	8.78	9.25	6.37	5.18	
200(ri4)	8.93	9.33	6.56	5.26	
300(ri4)	9.08	9.08	6.76	5.58	
400(ri4)	9.25	8.85	6.91	5.75	
500(ri4)	10.54	8.25	7.92	6.12	
600(ri4)	10.62	9.18	7.94	6.36	
700(ri4)	10.74	9.38	7.96	7.17	
800(ri4)	10.89	9.65	7.98	5.85	
900(ri4)	10.94	9.77	7.65	5.72	
1000(ri4)	11.25	10.13	8.02	4.13	

Fig. 12 Calculation of resource cost using ri4 for ATSIA3C.

From Fig. 12; Table 13 it clearly showed that proposed ATSIA3C minimizes resource cost over other baseline approaches for right skewed distribution workload i.e. ri4 in our research.

After evaluation of Resource cost with statistical distributions, calculation of resource cost in multi cloud environment with realtime supercomputing worklogs are performed. We used hp5 i.e. HPC2N worklogs for simulation. Initially, RATS-HM generated resource cost for considered tasks as 9.37, 10.26, 11.37 respectively. For MOABCQ with hp5 workload generated resource cost for considered tasks as 10.33, 11.03, 12.22 respectively. After MOABCQ, resource cost evaluated for AINN-BPSO with hp5 workload for considered tasks as 8.32, 9.57, 10.87. Finally, ATSIA3C dispersed resource cost of hp5 workload with different tasks as 5.39, 6.76, 8.12 respectively. From simulations with hp5 workload as input it is clearly evident that proposed approach shown significant impact on improvement of resource cost against existing approaches even if we have given realtime parallel computing worklogs mentioned in Table 14; Fig. 13.

Table 14 Calculation of resource cost using hp5 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(hp5)	9.37	10.33	8.32	5.39	
200(hp5)	9.41	10.52	8.74	5.12	
300(hp5)	9.56	10.64	8.89	5.26	
400(hp5)	9.88	10.78	8.91	5.38	
500(hp5)	10.26	11.03	9.57	6.76	
600(hp5)	10.64	11.15	9.61	6.88	
700(hp5)	10.79	11.2	9.75	7.04	
800(hp5)	10.98	11.44	9.88	7.15	
900(hp5)	11.03	11.56	9.25	7.25	
1000(hp5)	11.37	12.22	10.87	8.12	

Fig. 13 Resource cost evaluation using hp5 for ATSIA3C.

After evaluating resource cost using hp5 workload, we used another supercomputing worklogs known as NASA worklogs indicated as na6. The below table and figure shows evaluated resource cost using ATSIA3C and other algorithms. Initially, we evaluated resource cost using RATS-HM with na6 workload for considered tasks is 9.57, 10.26, 11.77 respectively. Resource cost for MOABCQ with considered tasks is 10.58,11.83, 10.61 respectively. For AINN-BPSO, dispersed resource cost for considered tasks is 11.34, 12.33, 9.88 respectively. Finally, ATSIA3C resource cost for considered tasks is 6.72, 7.35, 3.24 respectively mentioned in Table 15; Fig. 14.

Table 15 Calculation of resource cost using na6 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(na6)	9.57	10.58	11.34	6.72	
200(na6)	9.65	10.62	11.45	6.84	
300(na6)	9.78	10.76	11.59	6.46	
400(na6)	10.08	10.98	11.73	6.82	
500(na6)	10.26	11.83	12.33	7.35	
600(na6)	10.58	11.08	12.25	7.12	
700(na6)	10.76	11.45	12.17	6.98	
800(na6)	10.83	11.23	11.54	6.43	
900(na6)	10.96	11.08	10.88	5.26	
1000(na6)	11.77	10.61	9.88	3.24	

Fig. 14 Calculation of resource cost using na6 for ATSIA3C.

From the above results in the calculation of resource cost, it is clearly observed that generated resource cost of ATSIA3C is relatively minimum when compared with all the other baseline approaches used in evaluation for different types of workloads i.e. both fabricated data workloads, realtime worklogs. In the evaluation of resource cost, we observed that when the workload increased to 1000 tasks from 100 tasks and in all the cases ATSIA3C adopted to diversified workload and generated schedules while lowering resource costs over the baseline approaches.

Energy consumption evaluation using ATSIA3C

Subsection 5.4 clearly discusses energy consumption evaluation for ATSIA3C. Main reason to evaluate energy consumption in proposed approach as inefficient task scheduling in cloud paradigm effects energy cost which is important perspective from user and cloud provider perspective as if a task is scheduled to VM which is not suitable in terms of its runtime efficiency then definitely it will impact energy consumption by increasing energy costs. Another reason to choose energy consumption as an evaluation parameter is improper scheduling of tasks by without considering the constraints of tasks or deadlines then it will take more time to execute a task on that virtual machine thereby a chance of increase in consumption of energy which indirectly disobeys SLA which impacts QoS of CSP. Energy consumption also depends on cost of cooling required in order to maintain the servers at the optimum temperature level. These are all the reasons motivated us to choose energy consumption as one of the evaluating parameter for this task scheduler. The below Table 16; Fig. 15 indicates evaluation of energy consumption by ATSIA3C and various other compared approaches with input workload i.e. un1(Uniform distribution). Initially generated energy consumption using RATS-HM with 100,500, 1000 tasks 72.25, 91.18, 124.36 respectively. We used MOABCQ to evaluate energy consumption with un1 workload by 100, 500, 1000 tasks is 68.98, 95.77, 153.22 respectively. After these, AINN-BPSO used to evaluate energy consumption with 100,500,1000 tasks with un1 workload is 67.35, 89.21, 132.23 respectively. Finally, using ATSIA3C we evaluated energy consumption with 100, 500, 1000 tasks and it renders 41.21, 53.47, 109.68 respectively for un1 workload. From generated results, it is clearly observed that ATSIA3C shown huge improvement in consumed energy when it was compared over state of art techniques with input workload un1.

Table 16 Calculation of energy consumption using un1 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(un1)	72.25	68.98	67.35	41.21	
200(un1)	73.58	69.35	68.98	45.59	
300(un1)	74.39	70.78	69.44	48.54	
400(un1)	82.18	82.18	72.99	50.18	
500(un1)	91.18	95.77	89.21	53.47	
600(un1)	92.47	96.54	91.46	55.99	
700(un1)	95.67	98.19	93.76	68.36	
800(un1)	98.26	103.34	96.88	74.18	
900(un1)	102.17	123.65	109.77	84.81	
1000(un1)	124.36	153.22	132.23	109.68	

Fig. 15 Calculation of energy consumption using un1 for ATSIA3C.

After evaluating energy consumption using uniform workload, we evaluated consumption of energy using nor2 workload i.e. normal distribution which is a fabricated workload trace. It was evaluated by dispersing 100 to 1000 tasks with different algorithms MOABCQ, RATS-HM, AINN-BPSO and ATSIA3C. Generated energy consumption for RATS-HM with nor2 workload for considered tasks is 90.88, 98.35, 136.28 respectively. For MOABCQ, energy consumption with nor2 workload for considered tasks is 93.34, 89.51, 158.12 respectively. Energy consumption with nor2 workload for AINN-BPSO for considered tasks is 87.54, 99.25, 139.86 respectively. Finally, with ATSIA3C for nor2 workload rendered energy consumption for 100–1000 tasks is 46.29,73.22, 74.36 respectively. From evaluated results for energy consumption using normal distribution of statistical dispersion of workload showed in Table 17; Fig. 16 ATSIA3C shows significant improvement over all the compared approaches in our research.

Table 17 Calculation of energy consumption using nor2 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(nor2)	90.88	93.34	87.54	46.29	
200(nor2)	91.22	93.88	89.78	48.21	
300(nor2)	93.32	92.18	90.63	50.35	
400(nor2)	94.54	93.08	93.78	53.68	
500(nor2)	98.35	89.51	99.25	73.22	
600(nor2)	99.17	92.58	104.28	74.57	
700(nor2)	100.54	105.64	108.82	75.18	
800(nor2)	105.78	108.88	114.58	78.32	
900(nor2)	112.43	124.33	120.67	79.17	
1000(nor2)	136.28	158.12	139.86	74.36	

Fig. 16 Calculation of energy consumption using nor2 for ATSIA3C.

The below Table 18; Fig. 17 shown the evaluation of energy consumption using ATSIA3C which takes left skewed distribution indicated as ls3 in our research. ls3 data distribution involves more small, less large tasks which are dispersed as task distribution for this evaluation of energy consumption. Initially, RATS-HM evaluated energy consumption with 100–1000 tasks with ls3 as 78.57, 88.79, 138.27 respectively. Rendered energy consumption with MOABCQ for 100–1000 tasks with ls3 as 69.78, 112.26, 143.53. Energy consumption with AINN-BPSO for 100–1000 tasks with ls3 as 78.29, 106.19, 128.58. Energy consumption with ATSIA3C with 100–1000 tasks with ls3 as 55.67, 75.42, 87.74 respectively. From the results, proposed ATSIA3C shows significant improvement over state of art algorithms for left skewed distribution dispersed tasks.

Table 18 Calculation of energy consumption using ls3 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(ls3)	78.57	69.78	78.29	55.67	
200(ls3)	79.03	70.35	79.18	58.32	
300(ls3)	80.22	72.66	82.53	60.67	
400(ls3)	83.38	89.13	89.33	67.23	
500(ls3)	88.79	112.26	106.19	75.42	
600(ls3)	89.12	119.38	108.35	77.18	
700(ls3)	90.37	120.24	109.22	79.32	
800(ls3)	92.86	124.98	114.34	80.33	
900(ls3)	103.87	130.58	120.22	81.34	
1000(ls3)	138.27	143.53	128.58	87.74	

Fig. 17 Calculation of energy consumption using ls3 for ATSIA3C.

The below Table 19; Fig. 18 shows Energy consumption evaluation for ATSIA3C using ri4 dispersed tasks in which more large and less small tasks are distributed and given as input workload i.e. indicated as ri4 in this research. Initially, using RATS-HM evaluated energy consumption with ri4 workload and considered tasks are 100, 500, 1000. Generated energy consumption of ri4 for RATS-HM is 69.83, 82.15, 126.87 respectively. After evaluating RATS-HM, we used MOABCQ to calculate energy consumption with ri4 workload for 100,500, 1000 tasks and generated energy consumption is 57.25, 114.67, 140.39 respectively. Now, AINN-BPSO used to evaluate energy consumption with ri4 for 100, 500, 1000 tasks and generated energy consumption is 64.38, 125.88, 108.46 respectively. Finally, we evaluated energy consumption using our proposed approach ATSIA3C with ri4 workload which generates its values for 100,500, 1000 tasks as 40.26, 64.08, 88.35 respectively. From generated results, it is clear that proposed ATSIA3C outperformed over compared approaches for minimizing energy consumption with ri4 workload.

Table 19 Calculation of energy consumption using ri4 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(ri4)	69.83	57.25	64.38	40.26	
200(ri4)	70.24	60.45	67.26	42.89	
300(ri4)	71.57	67.88	71.38	44.71	
400(ri4)	73.68	102.89	80.48	50.34	
500(ri4)	82.15	114.67	125.88	64.08	
600(ri4)	83.57	119.33	109.46	66.19	
700(ri4)	85.68	124.62	118.38	70.56	
800(ri4)	88.19	129.88	102.52	74.88	
900(ri4)	90.35	132.66	104.33	78.17	
1000(ri4)	126.87	140.39	108.46	88.35	

Fig. 18 Calculation of energy consumption using ri4 for ATSIA3C.

For all the above generated results, we used fabricated workloads which are of different statistical data distributions but to evaluate efficacy and robustness of our proposed approach, we used hp5 and na6 workloads to evaluate energy consumption. RATS-HM approach is used to evaluate energy consumption with hp5 workload as input for 100, 500, 1000 tasks and rendered energy consumption for 100–1000 tasks is 85.49, 114.37, 132.78 respectively. Energy consumption for MOABCQ with hp5 workload for 100–1000 tasks is 92.16, 123.38, 134.57 respectively. Energy consumption for AINN-BPSO with hp5 workload for 100–1000 tasks is 78.32, 113.74, 107.21 respectively. Finally, we evaluated our proposed ATSIA3C approach evaluated using hp5 workload for 100, 500, 1000 tasks and generated energy consumption values are 64.38, 70.28, 84.37 respectively. From the generated results of Table 20; Fig. 19 for energy consumption using hp5 workload, we can clearly state that consumption of energy is minimized over state of art approaches.

Table 20 Calculation of energy consumption using hp5 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(hp5)	85.49	92.16	78.32	64.38	
200(hp5)	86.58	94.76	79.18	65.61	
300(hp5)	87.93	96.98	81.64	67.84	
400(hp5)	88.18	100.23	89.87	69.12	
500(hp5)	114.37	123.38	113.74	70.28	
600(hp5)	116.27	125.88	120.21	72.33	
700(hp5)	122.58	127.65	114.08	75.88	
800(hp5)	124.13	128.99	112.33	76.91	
900(hp5)	125.88	129.12	118.87	80.28	
1000(hp5)	132.78	134.57	107.21	84.37	

Fig. 19 Calculation of energy consumption using hp5 for ATSIA3C.

After evaluating energy consumption with hp5 workload, we used another parallel computing workload indicated as na6 to show that proposed approach is efficient and robust over existing approaches. Initially, RATS-HM with na6 workload rendered energy consumption with 100–1000 tasks is 79.87, 82.37, 108.35 respectively. MOABCQ with na6 workload generated energy consumption with 100–1000 tasks is 83.42, 95.36, 128.33 respectively. AINN-BPSO rendered energy consumption with na6 workload for 100–1000 tasks is 75.92, 89.72, 98.34 respectively. Finally, proposed ATSIA3C rendered energy consumption for 100–1000 tasks with na6 workload 49.87, 69.55, 78.36 respectively. From the generated results in Table 21; Fig. 20 it is clearly shown that with realtime parallel worklogs i.e. na6 proposed ATSIA3C minimized energy consumption over baseline approaches.

Table 21 Calculation of energy consumption using na6 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	ATSIA3C	
100(na6)	79.87	83.42	75.92	49.87	
200(na6)	80.24	84.39	76.45	50.06	
300(na6)	80.87	85.09	77.18	51.48	
400(na6)	81.26	89.28	80.58	53.88	
500(na6)	82.37	95.36	89.72	69.55	
600(na6)	82.45	98.18	89.93	70.44	
700(na6)	83.11	99.12	90.12	72.58	
800(na6)	84.57	103.66	91.33	73.64	
900(na6)	88.56	112.19	92.41	74.87	
1000(na6)	108.35	128.33	98.34	78.36	

Fig. 20 Calculation of energy consumption using na6 for ATSIA3C.

From the above generated results, it is evident, our proposed ATSIA3C shown significant improvement over existing approaches by minimizing energy consumption for all types of workloads. In the generated results, it is clearly observed that with the increase of tasks in all types of workloads, energy consumption is minimized for diversified tasks and we can say that it is robust and efficient in the view of energy consumption.

Discussion and analysis of simulated results

Subsection 5.5 discusses analysis of extensive simulated results of ATSIA3C. Our proposed ATSIA3C simulated on Cloudsim by using fabricated datasets i.e. different statistical distributions (un1,nor2, ls3,ri4) and realtime parallel computing worklogs(hp5, na6). All these extensive simulations ran for 100 iterations. Using above mentioned workloads different parameters (makespan, resource cost, energy consumption) evaluated and in the below subsequent tables we have calculated improvement of makespan, resource cost, energy consumption over compared approaches. Below Table 22 Shows makespan improvement over existing techniques with workload un1. Table 23 shows makespan improvement over existing techniques with workload nor2. Table 24 indicates makespan improvement with workload ls3. Table 25 shows makespan improvement with workload ri4. Table 26 indicates makespan improvement with workload hp5. Table 27 shows makespan improvement with workload na6. Table 28 shows Resource cost improvement workload un1. Table 29 shows Resource cost improvement for workload nor2. Table 30 shows Resource cost improvement with workload ls3. Table 31 shows Resource cost improvement with workload ri4. Table 32 shows Resource cost improvement with workload hp5. Table 33 shows Resource cost improvement with workload na6. Table 34 shows Energy Consumption improvement with workload un1. Table 35 shows Energy Consumption improvement with workload nor2. Table 36 shows Energy Consumption improvement with workload ls3. Table 37 Energy Consumption improvement with workload ri4. Table 38 shows Energy Consumption improvement with workload hp5. Table 39 shows Energy Consumption improvement with workload na6.

Table 22 Improvement (%) in makespan using un1 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(un1)	82.02515654	75.16830886	87.43522937	
200(un1)	81.90708377	75.21128886	87.02016821	
300(un1)	81.70152999	75.18792275	86.77673981	
400(un1)	76.1412188	66.01962781	86.49032185	
500(un1)	72.77536109	49.61311173	53.13685282	
600(un1)	72.79308076	49.83658845	53.59095628	
700(un1)	71.11768112	49.71096213	53.69833379	
800(un1)	70.22536782	49.476113	50.30268459	
900(un1)	59.81890799	50.50342955	46.9934097	
1000(un1)	53.34352286	52.16463355	44.44462981	

Table 23 Improvement (%) in makespan using nor2 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(nor2)	79.33989788	71.70336461	79.56235524	
200(nor2)	80.3982677	72.32148546	80.08708624	
300(nor2)	78.06608377	72.33902106	76.4246205	
400(nor2)	72.12377762	74.95059411	67.38805582	
500(nor2)	68.63927741	71.95910167	64.03279007	
600(nor2)	72.16788022	72.68650782	66.25835638	
700(nor2)	73.33685866	77.7833621	71.42293415	
800(nor2)	73.91125522	74.19352619	68.79761616	
900(nor2)	73.66925668	72.47341036	65.81853068	
1000(nor2)	75.99090318	69.55563917	65.18581292	

Table 24 Improvement (%) in makespan using ls3 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(ls3)	75.20321582	85.77869789	82.55953254	
200(ls3)	77.27388988	86.33212404	83.43078752	
300(ls3)	78.18395442	76.55639489	82.30763845	
400(ls3)	82.35658161	70.56807098	71.98245885	
500(ls3)	92.10701683	64.47646141	57.88830452	
600(ls3)	92.7808934	63.2835865	57.18003123	
700(ls3)	92.24174609	58.87920265	57.63905966	
800(ls3)	84.58542337	58.83839912	56.62909425	
900(ls3)	71.58424879	59.4739328	55.05144736	
1000(ls3)	60.547178	56.03528517	52.44374601	

Table 25 Improvement (%) in makespan using ri4 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(ri4)	82.6277142	76.01060742	69.07536628	
200(ri4)	83.62746361	76.8668105	69.3614445	
300(ri4)	83.7541226	76.27160334	69.41583995	
400(ri4)	83.18555515	73.61383557	69.49565747	
500(ri4)	83.26207226	68.53390602	79.45666202	
600(ri4)	82.71971889	67.75886509	73.29143111	
700(ri4)	87.57646825	74.53637209	76.2022501	
800(ri4)	86.03112556	77.88206797	73.10189256	
900(ri4)	89.77874264	72.51799598	72.06101161	
1000(ri4)	76.3028028	70.82451232	69.8462112	

Table 26 Improvement (%) in makespan using hp5 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(hp5)	81.30098938	73.69089046	64.27238757	
200(hp5)	86.11196184	71.67807513	66.45211576	
300(hp5)	90.26600261	74.7521921	69.62897069	
400(hp5)	91.12237145	72.70296299	56.28668859	
500(hp5)	83.30286476	63.71173859	51.66381844	
600(hp5)	83.32559024	61.47973294	52.13915491	
700(hp5)	83.31568241	60.39609605	52.39500425	
800(hp5)	81.75182482	56.5139781	52.82827984	
900(hp5)	78.82421375	56.22217341	52.7638898	
1000(hp5)	71.71786256	53.85096298	55.04274285	

Table 27 Improvement (%) in makespan using na6 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(na6)	70.1008452	86.40583808	75.9521304	
200(na6)	70.87691897	86.85840931	76.99589212	
300(na6)	74.94984031	91.25205643	80.96359696	
400(na6)	81.33833052	87.05561803	82.49759216	
500(na6)	90.41671496	78.83951283	74.50866454	
600(na6)	86.31949034	78.46152562	67.78960876	
700(na6)	83.07369867	75.07540488	64.92587647	
800(na6)	74.37259027	68.45916839	60.2885798	
900(na6)	70.5204722	69.06216648	59.21838149	
1000(na6)	71.08658779	61.35613312	55.43524246	

From the above results, for both the fabricated workloads and for realtime computing worklogs used for ATSIA3C. There is a clear improvement i.e. minimization of makespan for all types of workloads. For un1, nor2, ls3 proposed ATSIA3C minimized more makespan over RATS-HM compared with other approaches but for the workloads ri4, hp5 and na6 makespan is greatly minimized over AINN-BPSO when compared with other approaches. It is clearly observed from the above Tables 22, 23, 24, 25, 26 and 27. In all the cases our proposed ATSIA3C minimized makespan for all the workloads whenever there is increase of tasks from 100 to 1000 i.e. for both fabricated workloads and for realtime computing worklogs.

Table 28 Improvement (%) in resource cost using un1 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(un1)	67.94871795	54.01273885	71.62162162	
200(un1)	68.72037915	55.20304569	72.86432161	
300(un1)	68.57585139	55.93434343	72.62295082	
400(un1)	70.58823529	59.62732919	75	
500(un1)	68.88297872	59.88439306	65.1572327	
600(un1)	67.57457847	59.74770642	65.28822055	
700(un1)	73.2885906	61.41732283	67.99501868	
800(un1)	71.83833116	61.84062851	67.85714286	
900(un1)	72.15189873	63.75838926	70.02457002	
1000(un1)	71.70236753	66.11226611	77.46650426	

Table 29 Improvement (%) in resource cost using nor2 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(nor2)	72.61146497	79.16666667	89.76377953	
200(nor2)	69.14556962	75.08591065	83.87715931	
300(nor2)	63.87596899	60.85672083	71.28027682	
400(nor2)	61.36363636	56.40668524	59.21052632	
500(nor2)	52.36806495	41.74757282	47.66009852	
600(nor2)	48.24797844	36.7934224	44.63840399	
700(nor2)	44.51871658	35.12658228	45.49180328	
800(nor2)	43.2934927	33.67768595	45.85091421	
900(nor2)	41.62303665	32.71604938	44.85190409	
1000(nor2)	35.38461538	37.99237611	44.10029499	

Table 30 Improvement (%) in resource cost using ls3 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(ls3)	51.01553166	62.60997067	70.81260365	
200(ls3)	57.00712589	69.46454414	77.79578606	
300(ls3)	59.92691839	70.48710602	78.5942492	
400(ls3)	62.02686203	69.78021978	77.32115677	
500(ls3)	71.01648352	57.89473684	74.06876791	
600(ls3)	70.86720867	57.22100656	73.45505618	
700(ls3)	77.03412073	63.73507058	74.39797212	
800(ls3)	61.9047619	52.72145144	57.10982659	
900(ls3)	45.68764569	40.5377456	39.67611336	
1000(ls3)	41.40875133	37.74319066	34.48888889	

Table 31 Improvement (%) in resource cost using ri4 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(ri4)	58.9977221	56	81.31868132	
200(ri4)	58.90257559	56.3772776	80.18292683	
300(ri4)	61.45374449	61.45374449	82.5443787	
400(ri4)	62.16216216	64.97175141	83.21273517	
500(ri4)	58.06451613	74.18181818	77.27272727	
600(ri4)	59.88700565	69.28104575	80.10075567	
700(ri4)	66.75977654	76.43923241	90.07537688	
800(ri4)	53.71900826	60.62176166	73.30827068	
900(ri4)	52.28519196	58.54657114	74.77124183	
1000(ri4)	36.71111111	40.76999013	51.49625935	

Table 32 Improvement (%) in resource cost using hp5 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(hp5)	57.52401281	52.17812197	64.78365385	
200(hp5)	54.41020191	48.66920152	58.5812357	
300(hp5)	55.0209205	49.43609023	59.16760405	
400(hp5)	54.4534413	49.90723562	60.38159371	
500(hp5)	65.88693957	61.28739801	70.63740857	
600(hp5)	64.66165414	61.70403587	71.59209157	
700(hp5)	65.24559778	62.85714286	72.20512821	
800(hp5)	65.11839709	62.5	72.36842105	
900(hp5)	65.72982774	62.71626298	78.37837838	
1000(hp5)	71.41600704	66.44844517	74.70101196	

Table 33 Improvement (%) in resource cost using na6 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(na6)	70.21943574	63.51606805	59.25925926	
200(na6)	70.88082902	64.40677966	59.73799127	
300(na6)	66.05316973	60.03717472	55.73770492	
400(na6)	67.65873016	62.1129326	58.14151748	
500(na6)	71.6374269	62.13017751	59.6107056	
600(na6)	67.29678639	64.2599278	58.12244898	
700(na6)	64.86988848	60.96069869	57.35414955	
800(na6)	59.3721145	57.25734639	55.71923744	
900(na6)	47.99270073	47.47292419	48.34558824	
1000(na6)	27.52761257	30.53722903	32.79352227	

From the above results for both fabricated workloads and realtime computing worklogs for the proposed ATSIA3C resource cost in multicloud environment is minimized in a significant manner. We have calculated average of improved percentage of resource cost for all the compared algorithms and among all the approaches ATSIA3C minimizes resource cost over MOABCQ highly when compared with other approaches RATS-HM, AINN-BPSO. It is clearly observed from the Tables 28, 29, 30, 31, 32 and 33 for all the types of workloads whenever there is an increase in the number of tasks from 100 to 1000 resource cost is significantly improved. Therefore, we can justify that ATSIA3C is adapting to the dynamic nature of cloud environment by minimizing resource cost for both real time parallel computing worklogs and as well as for fabricated workloads.

Table 34 Improvement (%) of energy consumption using un1 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(un1)	57.03806228	59.74195419	61.1878248	
200(un1)	61.95977168	65.73900505	66.09162076	
300(un1)	65.25070574	68.57869455	69.90207373	
400(un1)	61.06108542	61.06108542	68.74914372	
500(un1)	58.64224611	55.83168007	59.93722677	
600(un1)	60.54936736	57.99668531	61.21801881	
700(un1)	71.45395631	69.62012425	72.90955631	
800(un1)	75.49358844	71.78246565	76.56895128	
900(un1)	83.00871097	68.58875859	77.26154687	
1000(un1)	88.19556127	71.58334421	82.94638131	

Table 35 Improvement of energy consumption using nor2 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(nor2)	50.9352993	49.59288622	52.87868403	
200(nor2)	52.85025214	51.3527908	53.69792827	
300(nor2)	53.95413631	54.62139293	55.55555556	
400(nor2)	56.78019886	57.6708208	57.24034975	
500(nor2)	74.44839858	81.8009161	73.77329975	
600(nor2)	75.19411112	80.54655433	71.50939778	
700(nor2)	74.77620847	71.16622491	69.08656497	
800(nor2)	74.04046133	71.93240265	68.35398848	
900(nor2)	70.41714845	63.67731038	65.60868484	
1000(nor2)	54.56413267	47.02757399	53.16745317	

Table 36 Improvement (%) of energy consumption using ls3 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(ls3)	70.85401553	79.77930639	71.10742113	
200(ls3)	73.79476148	82.89978678	73.65496337	
300(ls3)	75.62951882	83.4984861	73.51266206	
400(ls3)	80.63084673	75.42914843	75.26027091	
500(ls3)	84.94199797	67.18332443	71.02363688	
600(ls3)	86.60233393	64.65069526	71.23211814	
700(ls3)	87.77249087	65.96806387	72.62406153	
800(ls3)	86.50656903	64.27428389	70.2553787	
900(ls3)	78.30942524	62.29131567	67.6592913	
1000(ls3)	63.45555797	61.13007734	68.23767304	

Table 37 Improvement (%) of energy consumption using ri4 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(ri4)	57.65430331	70.3231441	62.53494874	
200(ri4)	61.06207289	70.95119934	63.76746952	
300(ri4)	62.47030879	65.86623453	62.63659288	
400(ri4)	68.32247557	48.92603752	62.54970179	
500(ri4)	78.00365186	55.88209645	50.9056244	
600(ri4)	79.2030633	55.46802983	60.46957793	
700(ri4)	82.35294118	56.62012518	59.60466295	
800(ri4)	84.90758589	57.65321836	73.03940694	
900(ri4)	86.51909242	58.92507161	74.92571648	
1000(ri4)	69.63821234	62.93183275	81.45860225	

Table 38 Improvement (%) of energy consumption using hp5 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(hp5)	75.30705346	69.85677083	82.20122574	
200(hp5)	75.77962578	69.23807514	82.8618338	
300(hp5)	77.15228022	69.95256754	83.09652131	
400(hp5)	78.38512134	68.96138881	76.9110938	
500(hp5)	61.44968086	56.96223051	61.79004748	
600(hp5)	62.20865227	57.45948522	60.16970302	
700(hp5)	61.90243107	59.44379162	66.51472651	
800(hp5)	61.95923628	59.62477711	68.46790706	
900(hp5)	63.77502383	62.17472119	67.53596366	
1000(hp5)	63.54119596	62.69599465	78.69601716	

Table 39 Improvement of energy consumption using na6 for ATSIA3C.

Tasks	RATS-HM	MOABCQ	AINN-BPSO	
100(na6)	62.43896332	59.7818269	65.68756586	
200(na6)	62.38783649	59.31982462	65.48070634	
300(na6)	63.65772227	60.50064637	66.70121793	
400(na6)	66.30568545	60.34946237	66.8652271	
500(na6)	84.4360811	72.9341443	77.51894784	
600(na6)	85.43359612	71.74577307	78.32758812	
700(na6)	87.33004452	73.2243745	80.5370617	
800(na6)	87.0757952	71.03993826	80.63067995	
900(na6)	84.54155375	66.73500312	81.0193702	
1000(na6)	72.32118136	61.06132627	79.68273337	

When we observe the results from Tables 34, 35, 36, 37, 38 and 39, it is clearly shown that for both types of fabricated workloads and for parallel computing worklogs energy consumption is significantly minimized. We have calculated improvement of average for generated energy consumption over all the considered approaches. Among all the algorithms ATSIA3C highly minimized energy consumption over MOABCQ when compared with other algorithms. It is also observed that whenever there is increase in number of tasks energy consumption is minimized significantly in all the cases for all algorithms considered in the simulation. Therefore, it can be justified that proposed ATSIA3C adapting quickly in the cloud environment and scheduling tasks in an efficient way by minimizing energy consumption.

Conclusions and future work

Task Scheduling problem (TSP) is challenging problem in cloud computing as different tasks with variable run time capacities coming onto cloud application console from diversified heterogeneous resources. Tackling these heterogeneous tasks dynamically and scheduling them onto appropriate virtual resources is a tedious challenge for CSP. Traditional algorithms modeled with the support of metaheuristic approaches suffers with increase of makespan, resource cost, energy consumption with fluctuations (increase or decrease) of workload in cloud application console and not adapting to the workload dynamically as TSP resembles NP hard problem. Therefore, we formulate an Adaptive Task Scheduler using Improved Asynchronous Advantage Actor Critic(ATSIA3C) algorithm which is a deep reinforcement learning based approach segments all incoming tasks onto cloud console as sub tasks and schedule them to appropriate VMs by considering size of sub tasks, execution time, communication time. All extensive simulations of proposed ATSIA3C conducted using Cloudsim. Input to our ATSIA3C given with fabricated workloads which are un1, nor2, ls3, ri4 and realtime parallel supercomputing workloads hp5, na6. For evaluating the efficacy of our ATSIA3C we compared it with state of art techniques MOABCQ, RATS-HM, AINN-BPSO. Results revealed that proposed ATSIA3C shown significant impact over existing techniques by minimizing makespan, resource cost, energy consumption in multicloud environments. As a future work, our aim is to implement and incorporate our proposed approach in a realtime cloud environment i.e. OpenStack to evaluate robustness of algorithm.

Acknowledgements

The authors present their appreciation to King Saud University for funding this research through Researchers Supporting Program number (RSPD2024R918), King Saud University, Riyadh, Saudi Arabia.

Author contributions

S.M., G.R.K., M.V.R. wrote the manuscripts and simualtion, S.N.M., and B.AJ., supervised work, Y.A.A., S.A., and B.S.A. revised and rewrite the manuscript.

Data availability

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Declarations

Competing interests

The authors declare no competing interests.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Marinescu, D. C. Cloud Computing: Theory and Practice (Morgan Kaufmann, 2022).
2. Mangalampalli, S. et al. Cloud Computing and Virtualization. Convergence of Cloud with AI for Big Data Analytics: Foundations and Innovation: 13–40 (2023).
3. Mangalampalli, S. & Ganesh Reddy, K. Cloud Environment Limitations and Challenges. Big Data, Cloud Computing and IoT: Tools and Applications (2023).
4. Pirozmand P An improved particle swarm optimization algorithm for task scheduling in cloud computing J. Ambient Intell. Humaniz. Comput. 2023 14 4 4313 4327 10.1007/s12652-023-04541-9
Pirozmand, P. et al. An improved particle swarm optimization algorithm for task scheduling in cloud computing. J. Ambient Intell. Humaniz. Comput. 14(4), 4313–4327 (2023).
5. Mikram H Kafhali SE Saadi Y HEPGA: A new effective hybrid algorithm for scientific workflow scheduling in cloud computing environment Simul. Model. Pract. Theory 2024 130 102864 10.1016/j.simpat.2023.102864
Mikram, H., Kafhali, S. E. & Saadi, Y. HEPGA: A new effective hybrid algorithm for scientific workflow scheduling in cloud computing environment. Simul. Model. Pract. Theory. 130, 102864 (2024).
6. Chandrashekar C HWACOA scheduler: Hybrid weighted ant colony optimization algorithm for task scheduling in cloud computing Appl. Sci. 2023 13 3433 10.3390/app13063433
Chandrashekar, C. et al. HWACOA scheduler: Hybrid weighted ant colony optimization algorithm for task scheduling in cloud computing. Appl. Sci. 13, 3433 (2023).
7. Xiao Z Multi-objective parallel task offloading and content caching in D2D-aided MEC networks IEEE Trans. Mob. Comput. 2023 22 11 6599 6615 10.1109/TMC.2022.3199876
Xiao, Z. et al. Multi-objective parallel task offloading and content caching in D2D-aided MEC networks. IEEE Trans. Mob. Comput. 22(11), 6599–6615. 10.1109/TMC.2022.3199876 (2023).
8. Bal P A joint resource allocation, security with efficient task scheduling in cloud computing using hybrid machine learning techniques Sensors 2022 22 3 1242 10.3390/s22031242 35161987
Bal, P. et al. A joint resource allocation, security with efficient task scheduling in cloud computing using hybrid machine learning techniques. Sensors. 22(3), 1242 (2022).35161987
9. Kruekaew B Multi-objective task scheduling optimization for load balancing in cloud computing environment using hybrid artificial bee colony algorithm with reinforcement learning IEEE Access. 2022 10 17803 17818 10.1109/ACCESS.2022.3149955
Kruekaew, B. Multi-objective task scheduling optimization for load balancing in cloud computing environment using hybrid artificial bee colony algorithm with reinforcement learning. IEEE Access. 10, 17803–17818 (2022).
10. Alghamdi, M. I. Optimization of load balancing and task scheduling in cloud computing environments using artificial neural networks-based binary particle swarm optimization (BPSO). Sustainability 14(19), 11982 (2022).
11. Dai X Task offloading for cloud-assisted fog computing with dynamic service caching in enterprise management systems IEEE Trans. Ind. Inf. 2023 19 1 662 672 10.1109/TII.2022.3186641
Dai, X. et al. Task offloading for cloud-assisted fog computing with dynamic service caching in enterprise management systems. IEEE Trans. Ind. Inf. 19(1), 662–672. 10.1109/TII.2022.3186641 (2023).
12. Mahmoud H An efficient load balancing technique for task scheduling in heterogeneous cloud environment Cluster Comput. 2021 24 4 3405 3419 10.1007/s10586-021-03334-z
Mahmoud, H. et al. An efficient load balancing technique for task scheduling in heterogeneous cloud environment. Cluster Comput. 24(4), 3405–3419 (2021).
13. Dai X Task co-offloading for D2D-assisted mobile edge computing in industrial internet of things IEEE Trans. Ind. Inf. 2023 19 1 480 490 10.1109/TII.2022.3158974
Dai, X. et al. Task co-offloading for D2D-assisted mobile edge computing in industrial internet of things. IEEE Trans. Ind. Inf. 19(1), 480–490. 10.1109/TII.2022.3158974 (2023).
14. Yao F Pu C Zhang Z Task duplication-based scheduling algorithm for budget-constrained workflows in cloud computing IEEE Access. 2021 9 37262 37272 10.1109/ACCESS.2021.3063456
Yao, F., Pu, C. & Zhang, Z. Task duplication-based scheduling algorithm for budget-constrained workflows in cloud computing. IEEE Access. 9, 37262–37272 (2021).
15. Choudhary A A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing Future Gener. Comput. Syst. 2018 83 14 26 10.1016/j.future.2018.01.005
Choudhary, A. et al. A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing. Future Gener. Comput. Syst. 83, 14–26 (2018).
16. Rjoub G Deep and reinforcement learning for automated task scheduling in large-scale cloud computing systems Concurr. Comput. Pract. Exp. 2021 33 e5919 10.1002/cpe.5919
Rjoub, G. et al. Deep and reinforcement learning for automated task scheduling in large-scale cloud computing systems. Concurr. Comput. Pract. Exp. 33, e5919 (2021).
17. Jiang H Dai X Xiao Z Iyengar A Joint task offloading and resource allocation for energy-constrained mobile edge computing IEEE Trans. Mob. Comput. 2023 22 7 4000 4015 10.1109/TMC.2022.3150432
Jiang, H., Dai, X., Xiao, Z. & Iyengar, A. Joint task offloading and resource allocation for energy-constrained mobile edge computing. IEEE Trans. Mob. Comput. 22(7), 4000–4015. 10.1109/TMC.2022.3150432 (2023).
18. Mangalampalli S A novel fault-tolerant aware task scheduler using deep reinforcement learning in cloud computing Appl. Sci. 2023 13 12015 10.3390/app132112015
Mangalampalli, S. & Shiva Rama Krishna, M. A novel fault-tolerant aware task scheduler using deep reinforcement learning in cloud computing. Appl. Sci. 13, 12015 (2023).
19. Siddesha K Jayaramaiah GV Singh C A novel deep reinforcement learning scheme for task scheduling in cloud computing Cluster Comput. 2022 25 6 4171 4188 10.1007/s10586-022-03630-2
Siddesha, K., Jayaramaiah, G. V. & Singh, C. A novel deep reinforcement learning scheme for task scheduling in cloud computing. Cluster Comput. 25(6), 4171–4188 (2022).
20. Rugwiro U Gu C Ding W Task scheduling and resource allocation based on ant-colony optimization and deep reinforcement learning J. Internet Technol. 2019 20 5 1463 1475
Rugwiro, U., Gu, C. & Ding, W. Task scheduling and resource allocation based on ant-colony optimization and deep reinforcement learning. J. Internet Technol. 20(5), 1463–1475 (2019).
21. Agarwal M Gur Mauj Saran Srivastava Opposition-based learning inspired particle swarm optimization (OPSO) scheme for task scheduling problem in cloud computing J. Ambient Intell. Humaniz. Comput. 2021 12 9855 9875 10.1007/s12652-020-02730-4
Agarwal, M. & Gur Mauj Saran, S. Opposition-based learning inspired particle swarm optimization (OPSO) scheme for task scheduling problem in cloud computing. J. Ambient Intell. Humaniz. Comput. 12, 9855–9875 (2021).
22. Nabi, S. et al. AdPSO: Adaptive PSO-based task scheduling approach for cloud computing. Sensors 22(3) 920 (2022).
23. Mangalampalli, S. et al. DRLBTSA: Deep reinforcement learning based task-scheduling algorithm in cloud computing. Multimedia Tools Appl. 1–29 (2023).
24. Shingne H Shriram R Heuristic deep learning scheduling in cloud for resource-intensive internet of things systems Comput. Electr. Eng. 2023 108 108652 10.1016/j.compeleceng.2023.108652
Shingne, H. & Shriram, R. Heuristic deep learning scheduling in cloud for resource-intensive internet of things systems. Comput. Electr. Eng. 108, 108652 (2023).
25. Duan Y Zhao Y Hu J An initialization-free distributed algorithm for dynamic economic dispatch problems in microgrid: Modeling, optimization and analysis Sustain. Energy Grids Netw. 2023 34 101004 10.1016/j.segan.2023.101004
Duan, Y., Zhao, Y. & Hu, J. An initialization-free distributed algorithm for dynamic economic dispatch problems in microgrid: Modeling, optimization and analysis. Sustain. Energy Grids Netw.. 34, 101004. 10.1016/j.segan.2023.101004 (2023).
26. Shingne H Shriram R Mutated deep reinforcement learning scheduling in cloud for resource-intensive IoT systems Wirel. Pers. Commun. 2023 132 3 2143 2155 10.1007/s11277-023-10709-5
Shingne, H. & Shriram, R. Mutated deep reinforcement learning scheduling in cloud for resource-intensive IoT systems. Wirel. Pers. Commun. 132(3), 2143–2155 (2023).
27. Sharma N Garg P Ant colony based optimization model for QoS-based task scheduling in cloud computing environment Measurement: Sens. 2022 24 100531
Sharma, N. & Garg, P. Ant colony based optimization model for QoS-based task scheduling in cloud computing environment. Meas. Sens. 24, 100531 (2022).
28. Mangalampalli S Fault tolerant trust based task scheduler using Harris Hawks optimization and deep reinforcement learning in multi cloud environment Sci. Rep. 2023 13 1 19179 10.1038/s41598-023-46284-9 37932308
Mangalampalli, S. et al. Fault tolerant trust based task scheduler using Harris Hawks optimization and deep reinforcement learning in multi cloud environment. Sci. Rep. 13(1), 19179 (2023).37932308
29. Cheng, Y. et al. Multi objective dynamic task scheduling optimization algorithm based on deep reinforcement learning. J. Supercomputing 1–29 (2023).
30. Uma J Vivekanandan P Shankar S Optimized intellectual resource scheduling using deep reinforcement Q-learning in cloud computing Trans. Emerg. Telecommun. Technol. 2022 33 e4463 10.1002/ett.4463
Uma, J., Vivekanandan, P. & Shankar, S. Optimized intellectual resource scheduling using deep reinforcement Q-learning in cloud computing. Trans. Emerg. Telecommun. Technol. 33, e4463 (2022).
31. Zhou G Deep reinforcement learning-based algorithms selectors for the resource scheduling in hierarchical cloud computing J. Netw. Comput. Appl. 2022 208 103520 10.1016/j.jnca.2022.103520
Zhou, G. et al. Deep reinforcement learning-based algorithms selectors for the resource scheduling in hierarchical cloud computing. J. Netw. Comput. Appl. 208, 103520 (2022).
32. Meng, Q., Jin, X., Luo, F., Wang, Z. & Hussain, S. Distributionally robust scheduling for benefit allocation in regional integrated energy system with multiple stakeholders. J. Mod. Power Syst. Clean Energy 1–12. 10.35833/MPCE.2023.000661 (2024).
33. Sun C Yang T Lei Y DDDQN-TS: A task scheduling and load balancing method based on optimized deep reinforcement learning in heterogeneous computing environment Int. J. Intell. Syst. 2022 37 11 9138 9172 10.1002/int.22983
Sun, C., Yang, T. & Lei, Y. DDDQN-TS: A task scheduling and load balancing method based on optimized deep reinforcement learning in heterogeneous computing environment. Int. J. Intell. Syst. 37(11), 9138–9172 (2022).
34. Palani, S. & Rameshbabu, K. A secured energy aware resource allocation and task scheduling based on improved cuckoo search algorithm and deep reinforcement learning for e-healthcare applications. Meas. Sens. 100988 (2023).
35. Huang Y Deep adversarial imitation reinforcement learning for QoS-aware cloud job scheduling IEEE Syst. J. 2021 16 3 4232 4242 10.1109/JSYST.2021.3122126
Huang, Y. et al. Deep adversarial imitation reinforcement learning for QoS-aware cloud job scheduling. IEEE Syst. J. 16(3), 4232–4242 (2021).
36. He, H., Li, X., Chen, P., Chen, J., Liu, M., & Wu, L. Efficiently localizing system anomalies for cloud infrastructures: A novel dynamic graph transformer based parallel framework. J. Cloud Comput. 13(1), 115 (2024). 10.1186/s13677-024-00677-x.
37. Shi J Priority-aware task offloading in vehicular fog computing based on deep reinforcement learning IEEE Trans. Veh. Technol. 2020 69 16067 16081 10.1109/TVT.2020.3041929
Shi, J. et al. Priority-aware task offloading in vehicular fog computing based on deep reinforcement learning. IEEE Trans. Veh. Technol. 69, 16067–16081 (2020).
38. Amer, D. A. et al. Elite learning Harris hawks optimizer for multi-objective task scheduling in cloud computing. J. Supercomputi. 1–26 (2022).
39. Zhang S Data-intensive workflow scheduling strategy based on deep reinforcement learning in multi-clouds J. Cloud Comput. 2023 12 1 125 10.1186/s13677-023-00504-9
Zhang, S. et al. Data-intensive workflow scheduling strategy based on deep reinforcement learning in multi-clouds. J. Cloud Comput. 12(1), 125 (2023).
40. Cui D A reinforcement learning-based mixed job scheduler scheme for grid or IaaS cloud IEEE Trans. Cloud Comput. 2017 8 4 1030 1039 10.1109/TCC.2017.2773078
Cui, D. et al. A reinforcement learning-based mixed job scheduler scheme for grid or IaaS cloud. IEEE Trans. Cloud Comput. 8(4), 1030–1039 (2017).
41. Calheiros RN CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms Softw. Pract. Exp. 2011 41 1 23 50 10.1002/spe.995
Calheiros, R. N. et al. CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exp.. 41(1), 23–50 (2011).
42. Mangalampalli S Karri GR Kose U Multi objective trust aware task scheduling algorithm in cloud computing using Whale optimization J. King Saud Univ. Inf. Sci. 2023 35 2 791 809
Mangalampalli, S., Karri, G. R. & Kose, U. Multi objective trust aware task scheduling algorithm in cloud computing using Whale optimization. J. King Saud Univ. Inf. Sci. 35(2), 791–809 (2023).
43. Madni SH Hybrid gradient descent cuckoo search (HGDCS) algorithm for resource scheduling in IaaS cloud computing environment Cluster Comput. 2019 22 301 334 10.1007/s10586-018-2856-x
Madni, S. H. et al. Hybrid gradient descent cuckoo search (HGDCS) algorithm for resource scheduling in IaaS cloud computing environment. Cluster Comput. 22, 301–334 (2019).
