OAR Job Types
The OAR reservation systems allows you to reserve machines with a job type. In short, a job type in a pre-configuration that is applied to the machines before the job starts. In this page we will describe the available job types in DI-Cluster. If you are not familiar with OAR and the basics of the DI-Cluster usage, please first look at the Getting Started page.
To reserve machines with a job type use option -t <job type name> of command oarsub.
Eg.
$ oarsub -I -t docker-swarm
The available job types are:
docker-swarm
This job type prepares a standard docker swarm with the machines of the reservation. The first machine is set as the manager, while the remaining machines are configured as workers.
For clarity, in the following example we ask for 4 machines with the docker-swarm job type.
$ oarsub -I -l/nodes=4 -t docker-swarm
[ADMISSION RULE] Set default walltime to 3600.
[ADMISSION RULE] Modify resource description with type constraints
OAR_JOB_ID=2864
Interactive mode: waiting...
Starting...
Connect to OAR job 2864 via the node node1
user@node1:~$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
c3rj0gh54quo4dapwc2sm0d3c * node1 Ready Active Leader 19.03.6
naj9370qojsoxw3tm0hifjfdz node2 Ready Active 19.03.6
27t9mtll3y8k45dmizflb47qi node3 Ready Active 19.03.6
fjxhmpp3u2ws1poz21c4ypbbb node4 Ready Active 19.03.6
user@node1:~$
knets
This job type prepares a standard kubernetes cluster with the machines of the reservation. A user is only allowed to have a single knets job a time (i.e., you are not allow to have simultaneous knets jobs). Similarly, to the docker-swarm job, the first node is set as manager, while the remaining are configured as workers.
For clarity, in the following example we ask for 4 machines with the knets job type.
$ oarsub -I -l/nodes=4 -t knets
[ADMISSION RULE] Set default walltime to 3600.
[ADMISSION RULE] Modify resource description with type constraints
OAR_JOB_ID=2864
Interactive mode: waiting...
Starting...
Connect to OAR job 2864 via the node node1
user@node1:~$ kubectl get node
NAME STATUS ROLES AGE VERSION
node1 Ready master 17m v1.19.2
node2 Ready <none> 17m v1.19.2
node3 Ready <none> 17m v1.19.2
node4 Ready <none> 17m v1.19.2
user@node1:~$
Note however that preparing a kubernetes cluster may take some time
knets-calico
This job type prepares a kubernetes cluster that uses calico as the networking framework for the cluster.
spark-hadoop
This job type prepares a spark cluster using hadoop. The job prepares a master and a worker on each machine of the reservation. The master in the first machine, acts as primary, while the other are redundant replicas.