# Managing Tier 1 User Data

Tier 1 user data at MSI commonly spans three storage locations:

- private home directory: `$HOME`
- project space: `$MSIPROJECT`, `$SHARED`, and `$PUBLIC`
- global scratch: `$SCRATCH` or `$SCRATCH_GLOBAL`

These quotas are independent. Use `userquota` for your private home directory. Use `groupquota` for project space and global scratch.

## Interactive Apps missing on Open OnDemand

Open OnDemand stores interactive session job scripts and logs in `~/.ondemand`, so it depends on free space in your private home directory.

Each user is allocated a private home directory on `/users/[0-9]/INTERNETID` with:

- 200 GB storage limit
- 1 million file count limit

If either home directory limit is reached, the Interactive Apps interface may disappear. Reducing home directory usage below the limit restores the ability to create new sessions on Open OnDemand.

## Checking current usage

Users can check quota from a [remote shell](https://ondemand.msi.umn.edu/pun/sys/shell/ssh/agate.msi.umn.edu) or a login node.

### Private home directory

Use `userquota`.

Example output from `userquota`:

```text
11:59:23 [vega0051@ahl02 ~ ]$ userquota
Quota for user 'vega0051'
------------------------
BYTES        |
Usage        | 45.00 GB
Quota        | 200.00 GB
Percent used | 22.5 %
------------------------
FILES        |
Usage        | 176,452
Quota        | 1,000,000
Percent used | 17.6 %
```

### Project space and global scratch

Use `groupquota` to review project space and global scratch for your primary group. If you belong to multiple groups, use `groupquota -g GROUPNAME -r` for a specific group.

Example output from `groupquota` may look similar to:

```text
11:59:58 [vega0051@ahl02 ~ ]$ groupquota
Quota for group 'example-lab'
------------------------
PROJECT SPACE
BYTES        |
Usage        | 7.52 TB
Quota        | 10.00 TB
Percent used | 75.2 %
------------------------
FILES        |
Usage        | 3,841,219
Quota        | 5,000,000
Percent used | 76.8 %

------------------------
SCRATCH GLOBAL
BYTES        |
Usage        | 12.40 TB
Quota        | 40.00 TB
Percent used | 31.0 %
------------------------
FILES        |
Usage        | 4,112,804
Quota        | 13,200,000
Percent used | 31.2 %
```

If `BYTES` or `FILES` usage in `userquota` exceeds 100%, see below on how to identify and reduce home directory usage.

If project space or global scratch in `groupquota` is close to 100%, use `groupquota -a` to see approximate usage by user and `groupquota -a -s files` when file count is the main issue.

## Reducing home directory BYTES usage

If you are not sure which directory is consuming most of your home directory quota, open a new remote shell and execute:

```bash
find $HOME -maxdepth 1 -exec du -sh '{}' \; | sort -sh
```

This measures the sizes of the top-level directories and files and sorts them from smallest to largest.

Once the larger directories have been identified, MSI recommends relocating them to your project's `/shared` directory.

If you do not already have a user directory in `/shared`, create one with:

```bash
mkdir -p $SHARED/$USER
```

Note that `$SHARED` in this example points to your primary project's shared directory. If you want to create a directory in a secondary project, you will need to enter the full path.

An example of moving a directory or file to `/shared`:

```bash
mv /path/to/file.txt $SHARED/$USER
```

## Reducing home directory FILES usage

If you are not sure which directories are consuming most of the file count limit, open a new [remote shell](https://ondemand.msi.umn.edu/pun/sys/shell/ssh/agate.msi.umn.edu) and execute the following commands.

Use these commands to create a directory in `/scratch.global/`, if it is not already present, make a local copy of the `filecount.sh` shell script, and execute it on the home directory:

```bash
mkdir -p $SCRATCH/$USER
s3cmd get s3://helpdesk/filecount.sh $SCRATCH/$USER
bash $SCRATCH/$USER/filecount.sh $HOME
```

If you have a significant number of files, this will take some time to count. The output on the screen will show the 10 directories with the largest number of files. Additionally, the output will be saved to your scratch global directory as a text file similar to `INTERNETID_filecount_TIMESTAMP.txt`.

For example, a directory with 170k files takes about 1 minute to traverse.

## Managing project space and global scratch usage

Project space and global scratch quotas are attached to the MSI group, so coordinate with your collaborators before deleting shared data.

- Use `groupquota -a` to identify which group members are consuming the most storage.
- Use `groupquota -a -s files` when file count, rather than total bytes, is the main problem.
- Move inactive project data off Tier 1 when appropriate. See [Reducing Tier 1 Footprint at MSI](archiving_data.md).
- Remove transient data from global scratch when it is no longer needed. Global scratch is temporary storage and should not be used for long-term retention.

## Further Guidance

If you need help interpreting quota reports or deciding what to move, email MSI support and staff may be able to help.