Skip to content

Upload and restore with S3 clients

StorageImpreza documentationRead as Markdown ↗

These examples use an existing bucket that belongs to you and keys allowed to read and write it. Get the endpoint and region from your S3 service, and install the client you choose. The Helsinki endpoint is https://s3.fi-hel-1.imprezahost.com, with region fi-hel-1. Keep TLS certificate verification enabled. Use path-style addressing so the bucket is part of the URL path.

Prepare your session

In Bash, read keys without echoing them or putting their values in command history. Do not enable shell tracing, publish environment dumps, or use someone else's service. For automation, use your secret manager or a private credentials file instead.

set +x
read -r -s -p 'S3 access key: ' AWS_ACCESS_KEY_ID; printf '\n'
read -r -s -p 'S3 secret key: ' AWS_SECRET_ACCESS_KEY; printf '\n'
export AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY
export AWS_DEFAULT_REGION='fi-hel-1'
export AWS_REGION="$AWS_DEFAULT_REGION"
export S3_ENDPOINT='https://s3.fi-hel-1.imprezahost.com'
export S3_BUCKET='YOUR_BUCKET'
export S3_PREFIX="quickstart-$(date -u +%Y%m%dT%H%M%SZ)-$RANDOM"
printf 'S3 restore check\n' > example.txt

Replace YOUR_BUCKET with your bucket name. Use a fresh object prefix for these examples so you do not overwrite existing data. If your service supplies temporary credentials, set AWS_SESSION_TOKEN securely too. Clear unrelated temporary credentials before using permanent service keys.

rclone

Configure an environment-only remote with credentials supplied by the AWS environment variables. Other selects a generic S3 service. force_path_style explicitly enables path-style addressing.

export RCLONE_CONFIG_IMPREZA_TYPE=s3
export RCLONE_CONFIG_IMPREZA_PROVIDER=Other
export RCLONE_CONFIG_IMPREZA_ENV_AUTH=true
export RCLONE_CONFIG_IMPREZA_ENDPOINT="$S3_ENDPOINT"
export RCLONE_CONFIG_IMPREZA_REGION="$AWS_DEFAULT_REGION"
export RCLONE_CONFIG_IMPREZA_FORCE_PATH_STYLE=true
export RCLONE_CONFIG_IMPREZA_SERVER_SIDE_ENCRYPTION=aws:kms
rclone copyto example.txt "impreza:$S3_BUCKET/$S3_PREFIX/rclone/example.txt"
rclone copyto "impreza:$S3_BUCKET/$S3_PREFIX/rclone/example.txt" restored-rclone.txt
cmp example.txt restored-rclone.txt

The server applies encryption automatically and reports aws:kms. Its encrypted-object ETag is not the content MD5. Setting server_side_encryption=aws:kms tells rclone not to compare that ETag with an MD5; keep the download and cmp to verify the restored bytes. Do not use --ignore-checksum.

With rclone 1.60.1, downloading a missing object can return exit status 0 without creating a file. Check that the downloaded file exists and that cmp succeeds; the transfer exit status alone is insufficient.

This sends plaintext to the storage server over TLS. To encrypt before uploading, add rclone crypt above this remote. See the rclone S3 configuration.

restic

restic stores an encrypted repository, not loose copies of the original files. Initialize only a new, empty repository prefix. restic prompts for the repository password; keep a recovery copy outside this bucket. s3.bucket-lookup=path forces path-style addressing. The region comes from AWS_DEFAULT_REGION.

export RESTIC_REPOSITORY="s3:$S3_ENDPOINT/$S3_BUCKET/$S3_PREFIX/restic"
restic -o s3.bucket-lookup=path init
restic -o s3.bucket-lookup=path backup example.txt
restic -o s3.bucket-lookup=path snapshots
restic -o s3.bucket-lookup=path restore latest --target restored-restic

Find example.txt in the restored directory tree and compare it with the original using cmp. Restore into a separate directory to protect existing files. Read restic repository setup and review object lock compatibility before using retention or pruning.

AWS CLI

Use a dedicated local configuration file to avoid changing another AWS profile. The configuration contains no keys. AWS_CONFIG_FILE sets path-style addressing; keys remain in the environment.

umask 077
export AWS_CONFIG_FILE="$(mktemp)"
cat > "$AWS_CONFIG_FILE" <<'CONFIG'
[default]
s3 =
    addressing_style = path
CONFIG
aws --endpoint-url "$S3_ENDPOINT" --region "$AWS_DEFAULT_REGION" s3 cp example.txt "s3://$S3_BUCKET/$S3_PREFIX/aws/example.txt"
aws --endpoint-url "$S3_ENDPOINT" --region "$AWS_DEFAULT_REGION" s3 cp "s3://$S3_BUCKET/$S3_PREFIX/aws/example.txt" restored-aws.txt
cmp example.txt restored-aws.txt

Remove only this temporary configuration when finished and unset AWS_CONFIG_FILE. For configuration details see AWS CLI S3 settings.

s5cmd

A custom --endpoint-url selects path-style addressing in s5cmd. AWS_REGION supplies the region and the AWS environment variables supply credentials.

s5cmd --endpoint-url "$S3_ENDPOINT" cp example.txt "s3://$S3_BUCKET/$S3_PREFIX/s5cmd/example.txt"
s5cmd --endpoint-url "$S3_ENDPOINT" cp "s3://$S3_BUCKET/$S3_PREFIX/s5cmd/example.txt" restored-s5cmd.txt
cmp example.txt restored-s5cmd.txt

See s5cmd credentials and endpoint configuration.

Verify and troubleshoot

A zero exit status from cmp confirms that the downloaded bytes match the original. For backups, also verify that you can decrypt and restore without access to your original machine. Keep the repository password and rclone crypt secrets separate from the S3 keys.

For AccessDenied, check the bucket name, key permissions and any source-IP restriction configured for your service. For a signature or region error, copy the endpoint and region from the service and verify the client's path-style setting and your system clock. For TLS errors, fix the trust chain or hostname; do not disable verification. Quota or retention errors can prevent writes or deletion: see pricing and limits and object lock.

When done, unset AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN in this shell. Processes already started may retain their environment; do not share the shell or expose debug logs containing keys.