How does a distributed job scheduler work?
5 min
Mar 28, 2018 from Activeeon
Job scheduler is a tool that allows automating and integrating the execution of business & IT processes and tasks (sequential and parallel jobs and orders) across different computing environments and virtual machines. Thanks to its unique console, the scheduler will give you a centralized view of operations of your project.
Workflows & job scheduling
Distributed job scheduling software allows you to easily distribute and execute tasks, jobs and business applications, monitor activity and view jobs results. It ensures more work done with fewer resources, managing heterogeneous platforms and multiple sites. It maximizes utilization of both existing and cloud IT system infrastructures.
Java-based job scheduler and its specifications
Workflow orchestrator: designing efficient workflows to automate the execution of workloads
A workflow orchestration interface provides an easy to use GUI for workflows creation as a representation of your distributed computations. By defining one or more tasks and their dependencies, you are able to set advanced flow controls such as replication, if/else branching and loops.
Example of a distributed computation workflow
Web portals: everything from your web browser
A job scheduler web portal is dedicated to users running workflows. From there you can submit workflows, monitor their execution and retrieve results. Remote visualization of graphical tasks is also possible directly from the browser. Progress of workflow execution is graphically visible in the portal.
The resource manager web portal allows administrators to operate the platform and users to monitor their system resources. It is the place where you can connect compute resources and monitor their usage, and even control dynamic policy-based provisioning of resources.
Resource management: aggregate resources to reduce infrastructure costs and use it to its full extent
A resource manager feature helps aggregate and manage resources from different infrastructures. In ProActive Workflows & Scheduling from Activeeon, resources coming from multiple origins are unified as ProActive Nodes and can be accessed transparently. These include desktop machines (Windows, Linux, MacOS X), all kinds of standalone server machines, cluster nodes managed by common batch schedulers (Slurm, LSF, SGE) and private or public clouds resources (Microsoft Azure, AWS, OpenStack, CloudStack, VMWare, Amazon EC2, etc).
Users can control resources acquisition behavior with job scheduler service according to various policies such as load-based or time-based.
Desktop agents: keep the control of your machines
Desktop machines are often under-used IT resources. Desktop agent smart software offers this computational power in a unified platform, while keeping usage under control for the sake of desktop end-users. It runs as a background service under a restricted user and does not interfere with the day-to-day usage of the desktop machine. Typical configuration consists in the definition of a weekly working plan (like night or weekend) or limitation of memory and CPU usage. Desktop agents are currently available for Windows, MacOS X and Linux.
Job monitoring: unified supervision for system resources and applications
Job scheduling solutions from Activeeon help yor save time and money troubleshooting any inconveniences you may experience during the execution. Distributed job scheduler from Activeeon provides a wide range of monitoring mechanisms accessible through web portals and APIs.
Thanks to job scheduler, you can supervise your jobs and its tasks, and get fine details on its duration, results, execution node, failures and more. Get the state of your execution nodes themselves: disk space left, CPU usage, memory usage, network usage, disk I/O usage, list of processes. Job scheduler also helps supervise your virtual resources: as your infrastructure grows with our cloud connectors, supervision and control on the new virtual resources will grow too, allowing you to get vendor specific information from the IaaS service provider monitoring API.
Fault Tolerance: be sure to get things done
Job failures might occur caused by a brief anomaly in the execution node, so marking a job as failed in such conditions could represent a waste of resources. Our distributed job scheduler will let you configure your workflow and specify policies of re-scheduling that better suit your case.
Network failure is a phenomenon that every system administrator is aware of. Java-based job scheduler from Activeeon handles these scenarios to recover computations executed in a lost node once it is back, saving your time and money. Thanks to scheduler service, outages will not mess your business up anymore. The solution preserves the state of the scheduler, all your job results and its logs.
Energy saving: prepare your IT for future regulations
Energy saving is not only an option: it is also a way to be prepared to regulations regarding the energy consumption in IT departments.
ProActive will help you achieve Green IT with resource policies. Scheduler service will give you control over resource acquisition and will let you promote elasticity with on-demand computing, desktop machines recycling and cloud bursting.
Why go for the job scheduler provided by Activeeon?
- Scheduler service will reduce the overall operation costs and improve the ROI of IT automation
- Fault tolerance: the job scheduler will reduce errors of manual operations
- Manage resource elasticity
- Insure infrastructure high availability
- Distributed job scheduler is fast & simple to use
- ProActive job scheduling software will give you an overview at all running processes and their evolution
Learn more about Workflows & Scheduling.