Rainer Weinberger

Astrophysicist

Tag: connections to cluster

Connecting (ssh)

A connection to a compute cluster is established via

ssh <username>@<node>.<institution>

e.g.

ssh USERNAME@login.rc.fas.harvard.edu

Graphics support can be included in the connection via

ssh -CY … or

ssh -X

In my experience, this graphics support is only useful if the connection to the cluster is really fast. In practice it is probably better to get accustomed to not using it. Some things like python plot creation via matplotlib might need to be adjusted to ensure python does not try to use this graphics interface.

 

~/.ssh/Config file

Include something like

Host myCluster

     HostName <node>.<institution>

     User <username>

     ForwardX11 yes

to be able to log in typing

ssh myCluster

instead of

ssh <username>@<node>.<institution>

 

To reuse an existing connection, add

ControlMaster auto

ControlPath ~/.ssh/sockets/%r@%h-%p

ControlPersist 600

to the ~/.ssh/Config entry. This avoids that for each new terminal you have to type in your password.

Data transfer

File transfer can be done via

scp -r ./file <user>@<node>:/my/directory

rsync -r –progress ./file <user>@<node>:/my/directory

both methods work, however, it is advisable to check files after transfers for completeness and correctness. This is done best via a hash, in particular an md5sum

md5sum <file>

For frequent and/or large data transfers between compute clusters, globus (www.globus.org) is a preferred tool. Especially long-distance and intercontinental connections tend to be much faster. This service can check the transferred data automatically, and also resumes automatically after a connection outage.

© 2020 Rainer Weinberger

Theme by Anders NorenUp ↑