# Data Transfer Nodes (DTN)

## How do I upload using the DTN?

Follow these steps:
1. Prepare your data for upload in a folder
2. Add a text file titled "upload.opts" to your upload folder
3. Navigate to the directory containing your upload folder (do not proceed to the next step while inside the folder containing your data)
4. SFTP into the DTN
    - Command: `sftp [username]@upload.blackwell.msi.umn.edu`
5. Upload your folder
    - Command: `put -r [upload_folder]`
6. Disconnect from the DTN

You will receive an automated email when your scan has started and when your data is ready to retrieve into Blackwell.

## How do I retrieve data (into Blackwell) using the DTN?

1. Follow the steps above to prepare your data for ingestion
2. Log in to Blackwell and use an `srun` command to move into your project directory
3. Use the `get` command along with your transfer ID (sent via automated email upon scan completion) to move your data into your project space. 

## I uploaded several files to the DTN, but never received an email

Here are some common items to check:
- Did you remember to create an `upload.opts` file? This file tells the scanner that your upload is ready to be scanned.
- Did you remember to fully disconnect from the DTN? Some users have configurations that reduce the number of times they must login which keeps connections alive. Make sure your connection to the DTN is fully terminated. This action causes the transfer software to wake up and look for files to scan.
- Were your files located within a folder? The DTN will not pick up and scan any loose files. 

## When I tried to connect via SFTP, I was immediately disconnected, even though my credentials were accepted.

Check your dot files (such as `.zshrc` or `.bashrc`), or your ssh `.config` file to ensure you aren’t setting anything by default that may interfere, such as requesting a PTY/TTY on the remote host. Blackwell is more fussy about certain settings than our other systems because of its security controls.

## I received an email indicating "Incomplete transfer on upload.blackwell.msi.umn.edu pending removal”, but it doesn’t contain any details…

Try logging into the DTN `sftp upload.blackwell.msi.umn.edu` and using `ls` to list files in the top-level directory. Any files that are not in a folder will generate this message. The DTN transfer software only knows how to interact with files inside a sub-folder.

## Accessing software modules not available in Blackwell

If you need a software modules that is not available in Blackwell, you can either upload your own software through the Data Transfer Network (DTN) or make a request to the [MSI Help Desk](mailto:help@msi.umn.edu) listing the specific module version and the reason you require it for your work.

## How do I access my project directory?

Use `srun --ntasks=1 /bin/bash` then `cd /projects/restricted/<group name>`