====== Storage Guidelines HPC ====== \\ ===== Storage Types ===== | Storage type | Where/Mount | Mount type | Purpose | Quota | Capacity | | HOME | /[homes|homes2]/[your_faculty|extern]/[username] | network | Program application data; only small files (very precious data) | ~ 2.5 GiB | - | | WORK | /work/[username] | network | **Permanent** data storage; NO scratch files| NO | - | | BeeGFS | /fast/[your_directories] | network | **Scratch** Storage of **(very) large** files | NO | - | | Local Scratch | For batch jobs: $TMPDIR or optionally: /scratch/[your_directories] | node-local FS \\ (shared by all jobs running on node) | Intermediary scratch storage for **small and moderate-large** files | NO | ~ 200 - 500 GiB | \\ ===== Recommendations ===== * ''BeeGFS'' (parallel distributed file system): preferred for **large scratch files (starting from files sizes of a few hundreds of MiB) AND (semi-)permanent storage of scratch files** * ''Node-local Scratch'': **fast, local FS storage** for small to intermediary sized scratch files up to a size of few GiBs; big amounts of small scratch files should be written here (BeeGFS shows poor performance for files with sizes smaller than a few hundreds of MiB) \\ ===== General Warning ===== * **Please ensure, that your programs DO NOT write huge amounts of scratch files or data to your WORK (as this will block other users from getting their work done).** * **Please avoid (permanent) storage of un-needed scratch files and do not copy them to your WORK.** * **Please write your scratch files to node-local or BeeGFS (depends on size of the files; see section "Recommendations"). If needed, copy the program's checkpoint files from node-local scratch to BeeGFS.** \\ date of revision: 03-24-2020 © kraus