Inhaltsverzeichnis
Storage Guidelines HPC
Storage Types
| Storage type | Where/Mount | Mount type | Purpose | Quota | Capacity |
| HOME | /[homes|homes2]/[your_faculty|extern]/[username] | network | Program application data; only small files (very precious data) | ~ 2.5 GiB | - |
| WORK | /work/[username] | network | Permanent data storage; NO scratch files | NO | - |
| BeeGFS | /fast/[your_directories] | network | Scratch Storage of (very) large files | NO | - |
| Local Scratch | For batch jobs: $TMPDIR or optionally: /scratch/[your_directories] | node-local FS (shared by all jobs running on node) | Intermediary scratch storage for small and moderate-large files | NO | ~ 200 - 500 GiB |
Recommendations
BeeGFS(parallel distributed file system): preferred for large scratch files (starting from files sizes of a few hundreds of MiB) AND (semi-)permanent storage of scratch filesNode-local Scratch: fast, local FS storage for small to intermediary sized scratch files up to a size of few GiBs; big amounts of small scratch files should be written here (BeeGFS shows poor performance for files with sizes smaller than a few hundreds of MiB)
General Warning
- Please ensure, that your programs DO NOT write huge amounts of scratch files or data to your WORK (as this will block other users from getting their work done).
- Please avoid (permanent) storage of un-needed scratch files and do not copy them to your WORK.
- Please write your scratch files to node-local or BeeGFS (depends on size of the files; see section „Recommendations“). If needed, copy the program's checkpoint files from node-local scratch to BeeGFS.
date of revision: 03-24-2020 © kraus
