Accessing object storage with s3cmd

Connection details and s3cmd configuration

Object storage is accessible via an HTTPS endpoint:

object-arbutus.cloud.computecanada.ca:443

The following is an example of a minimal s3cmd configuration file. You will need these values, but are free to explore additional s3cmd configuration options to fit your use case. Note that in the example the keys are redacted and you will need to replace them with your provided key values:

[default]
access_key = <redacted>
check_ssl_certificate = True
check_ssl_hostname = True
host_base = object-arbutus.cloud.computecanada.ca
host_bucket = object-arbutus.cloud.computecanada.ca
secret_key = <redacted>
use_https = True

Using s3cmd's --configure feature is described here.

Example operations on a bucket

• Make a bucket public so that it is Web accessible:

  s3cmd setacl s3://testbucket --acl-public

• Make the bucket private again:

  s3cmd setacl s3://testbucket --acl-private

• View the configuration of a bucket:

  s3cmd info s3://testbucket

Bucket policies

Attention

Be careful with policies because an ill-conceived policy can lock you out of your bucket.

Currently, Arbutus Object Storage only implements a subset of Amazon's specification for [bucket polices]. The following example shows how to create, apply, and view a bucket's policy. The first step is create a policy json file:

{
    "Version": "2012-10-17",
    "Id": "S3PolicyId1",
    "Statement": [
        {
            "Sid": "IPAllow",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::testbucket",
                "arn:aws:s3:::testbucket/*"
            ],
            "Condition": {
                "NotIpAddress": {
                    "aws:SourceIp": "206.12.0.0/16"
                    "aws:SourceIp": "142.104.0.0/16"
                }
            }
        }
    ]
}

This example denies access except from the specified source IP address ranges in Classless Inter-Domain Routing (CIDR) notation. In this example the s3://testbucket is limited to the public IP address range (206.12.0.0/16) used by the Arbutus cloud and the public IP address range (142.104.0.0/16) used by the University of Victoria.

Once you have your policy file, you can implement that policy on the bucket:

s3cmd setpolicy testbucket.policy s3://testbucket

To view the policy you can use the following command:

s3cmd info s3://testbucket