Commit d23ab09e authored by Max Jakub Ried's avatar Max Jakub Ried
Browse files

Import from GitHub, and rudimentary conversion to asciidoc.

parents
Pipeline #40611 passed with stages
in 13 seconds
image: asciidoctor/docker-asciidoctor:latest
pages:
script:
# build html
- mkdir --parents ./public
- asciidoctor --destination-dir ./public *.adoc
artifacts:
paths:
- public/
= HPC Cheat Sheet
:source-highlighter: rouge
Jonas Weber <jonas.weber@hhu.de>; Max Ried <max.ried@hhu.de>
Last edit: {docdatetime}
:toc:
== HPC cheat sheet
=== Useful links
https://www.zim.hhu.de/high-performance-computing.html[HPC application]
https://wiki.hhu.de/display/HPC/Wissenschaftliches+Hochleistungs-Rechnen+am+ZIM[HPC
wiki]
{empty}[screen wiki][screen]
{empty}[myJam website][myJam]
=== ☞ Nota Bene
Connections from outside of the University network are currently
disabled for security reasons. You can still connect if you use the VPN.
Its setup is described https://vpn.hhu.de/[here].
=== Connect with terminal
You can connect to the HPC login node via SSH.
[source,shell]
----
ssh <username>@hpc.rz.uni-duesseldorf.de
----
If this is to long for you to type everytime you can open
`+~/.ssh/config+` with an editor of your choice and add the lines:
....
Host hilbert
User <username>
HostName hpc.rz.uni-duesseldorf.de
Host hilbert-storage
User <username>
HostName storage.hpc.rz.uni-duesseldorf.de
....
This way you can just type `+ssh hilbert+` to connect to the login node.
To make it even more convinient, consider adding your ssh public key to
your `+authorized_keys+` file on hilbert. If you have no clue what this
means, just run the following on your local personal machine from where
you already can connect using your password. Assuming you use some kind
of disk encryption, it should be safe to choose an empty password for
your key in the first step.
[source,shell]
----
ssh-keygen -t ed25519
ssh-copy-id hilbert
----
From then on you can login to hilbert without providing your password.
=== Other ways of connecting
==== Mount a directory
If you're inside the University network, you should connect to the
storage backend of hilbert as it's able to achieve speeds around 100
MB/s.
[source,shell]
----
sshfs hilbert-storage:some/path/on/the/HPC some/path/on/your/system
----
You can mount a HPC directory to your filesystem from outside the
University network by running
[source,shell]
----
sshfs hilbert:some/path/on/the/HPC some/path/on/your/system
----
but this is discouraged by the ZIM people.
Example:
[source,shell]
----
sshfs hilbert-storage:/gpfs/scratch/joweb106 Documents/hilbert3/
sshfs hilbert:/gpfs/scratch/joweb106 Documents/hilbert3/
----
You can also use https://filezilla-project.org/[Filezilla], or the file
browser of your Linux distribution. Just try to enter
`+sftp://hilbert-storage/gpfs/scratch/<username>+` into the address bar
of your *file* browser.
Also look at https://wiki.hhu.de/display/HPC/Filesysteme+mounten[this
HPC wiki post]
==== Graphical Session
You can request one
https://view-2018.hpc.rz.uni-duesseldorf.de/enginframe/vdi/vdi.xml?_uri=//com.enginframe.interactive/list.sessions[here].
=== Your home directory
You shoud do your work in `+/gpfs/scratch/username+`.
=== Modules
You can load specific software with modules:
[source,shell]
----
module load module-name
----
List of all modules:
[source,shell]
----
module avail
----
For conda:
[source,shell]
----
module load Miniconda/3
----
For conda with Snakemake (different miniconda module):
[source,shell]
----
module load Miniconda/3_snakemake Snakemake/5.10.0
----
=== screen
One can use [screen] to start a new screen session
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment