MSI Website Open OnDemand MyMSI MSI Systems Status Page

Globus

The University of Minnesota participates in the Globus data transfer service. Globus is a service that allows for fast transfer of data between computer systems, using a convenient web interface. Globus data transfer speeds are typically significantly faster than data transfer speeds using other methods. Globus also contains data sharing capability, which allows users to share data with collaborators at UMN or other institutions.

Storage Collections

There are a few Globus Collections available to users:

  • UMN MSI Home

    • Project spaces on /projects/standard/<project name>

    • Private home directories on /users/{0-9}/<UMN internetid>

    • Global scratch directories /scratch.global

    • UUID 306b37db-de29-4488-ba9e-1e76acf820d0

  • UMN MSI HA Home

    • Provides the same access as UMN MSI Home

    • Also provides access to /projects/regulated/<project name>

    • UUID 22ecb25a-b7d4-4380-a7f8-5cb132057abe

  • UMN MSI Tier2

    • Tier2 Storage

    • UUID f84c5701-cc99-418a-8afe-f026eef1dd87

  • UMN MSI HA Tier2

    • Provides the same access as UMN MSI Tier2

    • UUID fbb22b3e-175c-4694-a6cf-8e79366f990e

  • UMN MSI HA AWS S3

    • Requires a user to have credentials with Amazon’s AWS S3 cloud service

    • Access to Amazon’s AWS S3 cloud service

    • UUID 1c555b59-5b56-4908-9593-40a8baddf805

Warning

The UUIDs above can be used to ensure that the select Collection is an authentic UMN MSI Globus Collection by matching them to the ‘origin_id’ or ‘destination_id’ in the URL at https://app.globus.org/file-manager

Accessing File Manager

  1. Navigate to https://www.globus.org

  2. Select Organization ‘University of Minnesota’ and sign in with your UMN account

  3. On left navigation menu, select ‘File Manager’

  4. On top right, change view to ‘set two pane’

  5. Load the shared collections mentioned above

    • E.g., Loading project space by selecting the collection UMN MSI Home and setting the path to your project space.

Warning

Globus automatically retries failed transfers over a period of 24 hours following an initial transfer request. If a transfer request contains an error such that it will never succeed, it is beneficial to delete the request using the activity panel, to prevent continued failed transfer attempts.

Users are advised against using one endpoint as both source and destination for a single transfer, as this slows the endpoint, and more effective techniques for moving data within MSI are available.

Transfer Restrictions

As with other transfer methods, users may only transfer data for which they have normal filesystem read access, and may only place new data in directories for which they have normal filesystem write access. Globus is bound by filesystem permissions in the usual way.

Additionally, a user may only perform Globus transfers involving data within their home directory and other locations such as their group/project shared directory. Other directories such as /scratch.global are available as well. If a path is unavailable and you would like it to be made available, contact the Help Desk.

On MSI systems Globus will deny transfers involving home directory hidden, aka ‘dot’, files.

Transfer Data to a Local Machine

Users may download directly from a Globus endpoint by adding their local machine as a local endpoint that can be selected from the File Manager interface.

Follow the instructions on installing Globus Connect Personal

External Storage

By default, Globus only has access to certain folders and drives. If you need to transfer a file or folder on a different path without manually copying it over, you can do the following steps. They may differ depending on the OS:

  1. Start Globus Connect Personal if it is not already running in the background.

  2. Click ‘Show Hidden Icons’ on your taskbar to expand the menu. Right click on the Globus Connect Personal icon and select ‘Options’.

  3. On the ‘Access’ tab of the Options popup, click the small ‘+’ button towards the bottom right of the window.

  4. Select the new folder or drive that you wish to give Globus access to.

  5. Click ‘Save’ on the bottom of the screen.

Sharing Data Using Guest Collections

See the official documentation on setting up a Guest Collection

Warning

Globus shared directories remain shared until sharing is removed using the Globus interface. Any material within a Globus shared directory (including newly added material) is shared via Globus. For security, it is recommended that users disable sharing when their need for sharing has ended.

Globus write-sharing is extremely limited on MSI systems by default - the only location enabled for this is if you create a directory named “globus-incoming” within your project “shared” folder on primary storage. Attempts to write to any other MSI directories via sharing will fail.

  • Tier 1 will look similar to /projects/standard/<project name>/shared/globus-incoming

  • Tier 2 will look similar to s3://<bucket name>/globus-incoming

Write sharing is also available for Tier 2 using a similar pattern to primary storage: a subdirectory within a bucket named ‘globus-incoming’ needs to be created before external collaborators can write to the bucket. e.g. /mybucket/globus-incoming