Accessing the Arbutus object storage with AWS CLI

This page contains instructions on how to set up and access Arbutus object storage with AWS CLI, one of the object storage clients available for this storage type.

The awscli client also works with the Object Store service with better support for large (>5GB) files and the helpful sync command. However, not all features have not been tested.

Installing awscli

pip install awscli awscli-plugin-endpoint

Configuring awscli

Generate an access key ID & secret key

openstack ec2 credentials create

Edit or create ~/.aws/credentials and add the credentials generated above

[default]
aws_access_key_id = <access_key>
aws_secret_access_key = <secret_key>

Edit ~/.aws/config and add the following configuration

[plugins]
endpoint = awscli_plugin_endpoint

[profile default]
s3 =
  endpoint_url = https://object-arbutus.cloud.computecanada.ca
  signature_version = s3v4
s3api =
  endpoint_url = https://object-arbutus.cloud.computecanada.ca

Using awscli

export AWS_PROFILE=default
aws s3 ls <container-name>
aws s3 sync local_directory s3://container-name/prefix

More examples can be found here: https://docs.ovh.com/us/en/storage/getting_started_with_the_swift_S3_API/