Other Useful Information

This page contains useful information to handle advanced workflows in DI-Cluster

SSH Tunnels

The cluster nodes are not accessible from outside E.g., if you have a web server running inside a job, you will not be able to access it through your browser.

To circumvent this, you need to create a ssh tunnel. The command bellow shows how to do this.

ssh -L [LOCAL PORT]:clusterNode:[NODE PORT] -p 12034 user@cluster.di.fct.unl.pt
  • LOCAL PORT - port on your local machine
  • clusterNode - hostname of the cluster machine
  • NODE export - port on the cluster machine
  • user - your username

In a simple example you would do something has shown bellow. Create a ssh tunnel that forwards the 9000 port on your machine to port 8080 on node1 through the frontend. This will log you in the frontend where you can reserve a job on node1, and start a web server on port 8080. Then you can access the webserver in your browser through localhost:9000.

user@localhost:~$ ssh -L 9000:node1:8080 -p 12034 user@cluster.di.fct.unl.pt
user@frontend:~$ oarsub -I -p "host in ('node1')"
[ADMISSION RULE] Set default walltime to 3600.
[ADMISSION RULE] Modify resource description with type constraints
OAR_JOB_ID=2883
Interactive mode: waiting...
Starting...

Connect to OAR job 2883 via the node node1
user@node1:~$ ... (start webserver on port 8080)

Last modified 24.09.2020