Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
general
albi
hpc-cheat-sheet
Commits
d23ab09e
Commit
d23ab09e
authored
May 20, 2020
by
Max Jakub Ried
Browse files
Import from GitHub, and rudimentary conversion to asciidoc.
parents
Pipeline
#40611
passed with stages
in 13 seconds
Changes
2
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
.gitlab-ci.yml
0 → 100644
View file @
d23ab09e
image
:
asciidoctor/docker-asciidoctor:latest
pages
:
script
:
# build html
-
mkdir --parents ./public
-
asciidoctor --destination-dir ./public *.adoc
artifacts
:
paths
:
-
public/
index.adoc
0 → 100644
View file @
d23ab09e
= HPC Cheat Sheet
:source-highlighter: rouge
Jonas Weber <jonas.weber@hhu.de>; Max Ried <max.ried@hhu.de>
Last edit: {docdatetime}
:toc:
== HPC cheat sheet
=== Useful links
https://www.zim.hhu.de/high-performance-computing.html[HPC application]
https://wiki.hhu.de/display/HPC/Wissenschaftliches+Hochleistungs-Rechnen+am+ZIM[HPC
wiki]
{empty}[screen wiki][screen]
{empty}[myJam website][myJam]
=== ☞ Nota Bene
Connections from outside of the University network are currently
disabled for security reasons. You can still connect if you use the VPN.
Its setup is described https://vpn.hhu.de/[here].
=== Connect with terminal
You can connect to the HPC login node via SSH.
[source,shell]
----
ssh <username>@hpc.rz.uni-duesseldorf.de
----
If this is to long for you to type everytime you can open
`+~/.ssh/config+` with an editor of your choice and add the lines:
....
Host hilbert
User <username>
HostName hpc.rz.uni-duesseldorf.de
Host hilbert-storage
User <username>
HostName storage.hpc.rz.uni-duesseldorf.de
....
This way you can just type `+ssh hilbert+` to connect to the login node.
To make it even more convinient, consider adding your ssh public key to
your `+authorized_keys+` file on hilbert. If you have no clue what this
means, just run the following on your local personal machine from where
you already can connect using your password. Assuming you use some kind
of disk encryption, it should be safe to choose an empty password for
your key in the first step.
[source,shell]
----
ssh-keygen -t ed25519
ssh-copy-id hilbert
----
From then on you can login to hilbert without providing your password.
=== Other ways of connecting
==== Mount a directory
If you're inside the University network, you should connect to the
storage backend of hilbert as it's able to achieve speeds around 100
MB/s.
[source,shell]
----
sshfs hilbert-storage:some/path/on/the/HPC some/path/on/your/system
----
You can mount a HPC directory to your filesystem from outside the
University network by running
[source,shell]
----
sshfs hilbert:some/path/on/the/HPC some/path/on/your/system
----
but this is discouraged by the ZIM people.
Example:
[source,shell]
----
sshfs hilbert-storage:/gpfs/scratch/joweb106 Documents/hilbert3/
sshfs hilbert:/gpfs/scratch/joweb106 Documents/hilbert3/
----
You can also use https://filezilla-project.org/[Filezilla], or the file
browser of your Linux distribution. Just try to enter
`+sftp://hilbert-storage/gpfs/scratch/<username>+` into the address bar
of your *file* browser.
Also look at https://wiki.hhu.de/display/HPC/Filesysteme+mounten[this
HPC wiki post]
==== Graphical Session
You can request one
https://view-2018.hpc.rz.uni-duesseldorf.de/enginframe/vdi/vdi.xml?_uri=//com.enginframe.interactive/list.sessions[here].
=== Your home directory
You shoud do your work in `+/gpfs/scratch/username+`.
=== Modules
You can load specific software with modules:
[source,shell]
----
module load module-name
----
List of all modules:
[source,shell]
----
module avail
----
For conda:
[source,shell]
----
module load Miniconda/3
----
For conda with Snakemake (different miniconda module):
[source,shell]
----
module load Miniconda/3_snakemake Snakemake/5.10.0
----
=== screen
One can use [screen] to start a new screen session
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment