Springe direkt zu Inhalt

Storing data

/home

Total size

35 TB

Quota

yes

Type distributed - visible on all nodes
User directory /home/${USER}
Backup An incremental backup to tape of /home is made every night at 7 pm. In addition a full backup is carried out every 6 weeks.
Snapshots 

Copies of data made at various points in time in the directory /home/<username>/.snapshots.  Data from the snapshots can be copied with cp just like any data back into your home directory.

NB: The directory .snapshots cannot be seen with ls -a ! Also the tab completion in the shell will not work with this directory! However, the directory can be used as normal in a path, e.g. ls -l ~/.snapshots .

Appropriate files
  •  job scripts
  • .out- and .err files for job
  • (Mini-, Ana-)Conda and Mamba environments
  • Git repositories
Caveats

Writing a large amount of data very quickly to this file system can cause the system to become very slow and could even cause the whole file system to crash. Please contact us before attempting this.

/scratch

Total size 1.8 PB
Quota yes
Type  distributed - visible on all nodes
User directory /scratch/${USER}
Backup no
Snapshots no
Appropriate files
  • very large files ( > 10s of GB)
  • temporary files (unless very small and/or large numbers)
Caveats
  • This file system is particularly large and has the highest I/O-throughput, but only if the amount of data read or written at one time is large - operations involving only a small amount of data can be very inefficient and slow down the whole filesystem.
  • /scratch is unsuitable for storing large amounts of very small files - in exceptional cases such files may be stored in your /home directory in a subdirectory called tmp.
  • Do not create directories with significantly more than 1000 entries.
  • Use file formats such as TGZ and HDF5 to store multiple small files as a single object.
  • VSCode sessions must prevent directories on /scratch from being monitored by the FileWatcher.

/localscratch

Total size 

167 GB

Quota

no

Type local - visibile only on each node
User directory  /localscratch/${USER}
Backup no
Snapshots  no
Appropriate files medium total volume (< 10s of GB )
Caveats The files system is shared between the users with jobs on a given node.

Quotas

User quotas exist on /home and /scratch.  These can be viewed with the following command:

filesystem_usage