Purpose
A tested procedure for moving an existing S3 bucket to XNS with rclone, the
checksum verification that actually verifies something, the cutover checklist, and the same
commands pointed the other way so you can leave again. Every command below was run
S3-to-S3 between two live gateways on 2026-09-09 and the output shown is the output we got.
| Tested | 2026-09-09 |
|---|---|
| Tested by | Kenneth Bell, SCP, Corp |
| Destination | XNS S3 Gateway 1.1.0-2350-gf3a94e00, release channel |
| Client | rclone v1.75.0, Ubuntu 24.04 |
| Estate moved | 401 objects / 311 MiB, including one 256 MiB object large enough to be stored as a multipart upload |
Read this part first
The obvious verification command reports success without verifying anything.
If you take one thing from this page, take this one — it applies to any S3-to-S3 migration,
not just ours.
$ rclone check oldcloud:bucket xns:bucket --checksum
NOTICE: S3 bucket bucket: 0 differences found
NOTICE: S3 bucket bucket: 1 hashes could not be checked
NOTICE: S3 bucket bucket: 1 matching filesThere was one object in that migration. It was not checked. The first line still says
0 differences found. Every large object in a real migration is stored as a
multipart upload, so in practice the objects you most want verified are exactly the ones this
silently skips. The fix is further down; the reason is worth understanding first.
Why multipart breaks checksum comparison
An S3 ETag is an MD5 of the object — but only for objects uploaded in one piece. For a
multipart upload it is a digest of the part digests, with the part count appended.
It therefore encodes the chunk size whoever uploaded it happened to choose, not the content.
The same 256 MiB file, byte for byte identical, uploaded two ways:
| Uploaded by | ETag | Object metadata |
|---|---|---|
| rclone | 14b399585d948d0b4b96fb6d45e15c98-52 | md5chksum present |
| aws-cli | 2d63d8d10d937d8aa49ac706e8b93058-32 | none |
The real MD5 of those bytes is b9375b9ccdbd01f3517c329deb088cde. Neither ETag
resembles it, and the two ETags do not match each other — 52 parts against 32 parts, same file.
rclone handles this for its own uploads by storing the true whole-object MD5 in object
metadata, which is why a bucket rclone filled will verify cleanly end to end. Your source
bucket was not filled by rclone. Its multipart objects carry a composite ETag and nothing
else, so there is no comparable hash on that side and rclone quietly skips them.
The migration
Configure both ends in rclone.conf — the source you are leaving and XNS:
[oldcloud]
type = s3
provider = Other
access_key_id = ...
secret_access_key = ...
endpoint = https://your-old-provider
region = us-east-1
[xns]
type = s3
provider = Other
access_key_id = ...
secret_access_key = ...
endpoint = https://your-gateway:9000
region = us-east-1Create the destination and sync:
rclone mkdir xns:bucket
rclone sync oldcloud:bucket xns:bucket --transfers 8 --stats-one-line --stats 10s401 objects and 311 MiB took 22.1 seconds on our local network. That is a
local-network figure between two hosts on the same rack — it tells you the gateway is
not the bottleneck and it tells you nothing about your own link. Measure yours on a subset
before you plan a cutover window around a number.
Verification that means something
Run the cheap check first. It compares size and modification time, and it will catch a
truncated or missing object:
rclone check oldcloud:bucket xns:bucketThen verify content properly. --download pulls both sides and compares the actual
bytes, which is the only comparison that works when one side’s hashes are composite:
$ rclone check oldcloud:bucket xns:bucket --download
NOTICE: S3 bucket bucket: 0 differences found
NOTICE: S3 bucket bucket: 1 matching filesNote what is absent: no “hashes could not be checked” line. That is the difference
between a verification and a green light.
256 MiB took 11.5 seconds. This reads every byte from both ends, so on a metered provider you
are paying egress to prove your own migration worked. On XNS the read side is $0 under the
published terms, which is why we can tell you to verify the whole estate rather than a sample.
The rule to take away: a non-zero “hashes could not be checked” count is a
failed verification, not a footnote. Read the second line, not the first.
Bandwidth and retries
A migration that saturates the office link is a migration someone kills halfway. Cap it:
rclone sync oldcloud:bucket xns:bucket \
--transfers 8 --bwlimit 8M \
--retries 5 --low-level-retries 20 \
--stats-one-line --stats 10s--bwlimit takes a schedule as well as a number, so you can run wide open
overnight and throttled during the day. sync is restartable by design: it
compares both sides first and moves only what is missing, so an interrupted run costs you the
objects in flight and nothing else.
Cutover
The cutover window is the second sync, not the first. In the tested run, a full sync of 311
MiB took 22.1 seconds; a re-sync after a single 4 MiB object arrived took 2.5
seconds — it moved only the delta.
- Full sync while the source stays live. Takes as long as it takes; nothing is at stake yet.
rclone check --downloadon the whole estate. Confirm zero differences and zero unchecked hashes.- Stop writes to the source.
- Second sync. This is the only step inside the outage window.
rclone checkagain — size and modtime is enough here, you verified content in step 2.- Repoint the application at the XNS endpoint.
- Keep the source bucket for a defined period before deleting it. It is your rollback.
Do not delete the source until you have run the workload against XNS for long enough to trust
it. The cost of keeping it is one more month of someone else’s storage bill; the cost of not
keeping it is the whole migration.
Leaving again
The same command, reversed. We ran it:
rclone sync xns:bucket oldcloud:bucket --transfers 8402 of 402 objects came back, 22.5 seconds, and a checksum comparison of the full round trip —
original files against what came back out — found zero differences across every object.
There is no export fee and no retrieval charge to plan around: $6.00 per TB per month. $0
egress, uncapped. 30-day minimum retention, no early-delete fee. The current terms are
published as machine-readable JSON at
xns.tech/pricing.json. The 30-day minimum is the
one thing to note if you are evaluating and expect to leave quickly.
Where this does not apply
This page covers bucket-to-bucket migration of ordinary objects. It does not cover moving
object versions — rclone sync moves current versions only, so a versioned source
arrives at XNS flattened to its current state, and version history stays behind. It does not
carry object ACLs, bucket policies, lifecycle rules or Object Lock settings; those are
configuration and you reapply them on the destination. If the destination bucket has Object
Lock with a retention rule, see
the restic runbook for how retention interacts with
later deletes.
Related
Plain rclone setup, without the migration specifics, is in
Rclone. Which S3 operations we support and where we differ is at
S3 compatibility — 79.1% of the counted scope, 478 of 604
tests, scope v3.0, run 2026-08-14, with the unsupported list published alongside. What no
egress charge does and does not cover is at Egress fees.
