Other Useful Information
This page contains useful information to handle advanced workflows in DI-Cluster
SSH Tunnels
The cluster nodes are not accessible from outside E.g., if you have a web server running inside a job, you will not be able to access it through your browser.
To circumvent this, you need to create a ssh tunnel. The command bellow shows how to do this.
ssh -L [LOCAL PORT]:clusterNode:[NODE PORT] -p 12034 user@cluster.di.fct.unl.pt
- LOCAL PORT - port on your local machine
- clusterNode - hostname of the cluster machine
- NODE export - port on the cluster machine
- user - your username
In a simple example you would do something has shown bellow. Create a ssh tunnel that forwards the 9000 port on your machine to port 8080 on node1 through the frontend. This will log you in the frontend where you can reserve a job on node1, and start a web server on port 8080. Then you can access the webserver in your browser through localhost:9000
.
user@localhost:~$ ssh -L 9000:node1:8080 -p 12034 user@cluster.di.fct.unl.pt
user@frontend:~$ oarsub -I -p "host in ('node1')"
[ADMISSION RULE] Set default walltime to 3600.
[ADMISSION RULE] Modify resource description with type constraints
OAR_JOB_ID=2883
Interactive mode: waiting...
Starting...
Connect to OAR job 2883 via the node node1
user@node1:~$ ... (start webserver on port 8080)
Last modified 24.09.2020