# Scratch

**Scratch Space** or **Scratch Storage** refers to any storage at MSI that is impermanent, meaning all data it contains is deleted periodically or after a given event.

## Global Scratch

Global Scratch (`/scratch.global`) is a large storage space that is available to all MSI users collectively. It has a total storage capacity of 910 TB, and quotas are on a per-group basis at a limit of **40 TB** or **10 million** files.

In order to use Global Scratch, users must access `/scratch.global` and create a directory named after their user there. The following command is a good starting point, creating the user directory, changing into it, and printing the full path:

```bash
mkdir /scratch.global/$USER && cd /scratch.global/$USER && pwd
```

Data on Global scratch will become eligible for deletion **30 days after file creation**. As such, **it is not recommended that users store any important data in Global Scratch**. Users are advised to keep data larger datasets in Tier 2 so they may be restored into Global scratch as needed.

## Local Scratch

Local scratch (`/scratch.local`) is temporary storage space specifically allocated to a job's compute node(s) while it is running. **It must be specifically requested** by the job slurm script by using the `--tmp` flag.

Any data in that allocated storage is immediately deleted after a job has finished running. It is crucial that users include commands to transfer any relevant job-generated data to Tier 1 storage **before** the job terminates.

## Data Backups

No scratch space has any kind of backup system. Any data that is deleted from scratch space at any point is **gone forever**. This goes to reinforce our recommendation to never store important data in scratch storage.

## Quirks and Edge Cases

- Some data transfer methods, such as `s3cmd sync` with default settings, will preserve timestamps on transferred data. Accurate timestamps are important for correctly determining whether items have exceeded a 30-day period in scratch space, so it is recommended that users favor methods that update timestamps after transfers. With `s3cmd sync`, for example, the `--no-preserve` flag can be passed to enable this.