← S3 compatibility matrix S3 resource guide · Buckets & listing

Buckets & Listing

CreateBucket, DeleteBucket, HeadBucket, ListBuckets, and both ListObjects generations — the container and enumeration layer every S3 client walks before it touches a byte of object data.

Unlimited buckets per account ListObjectsV2 — pagination conformant 3 unclassified conformance gaps


What this area is

Buckets are the top-level containers in S3; listing is how a client discovers what is inside.

Listing is how a client discovers what’s inside one (or which buckets it owns) — the operations a client runs before it can address a single object.

Operations covered here

CreateBucket · DeleteBucket · HeadBucket · GetBucketLocation · ListBuckets · ListObjectsV1/V2 · ListObjectVersions · ListMultipartUploads


How XNS implements it

Bucket names are unique per box across every tenant — a flat namespace, not global like AWS.

CreateBucket (bucket_handlers.go) checks a name against every principal’s partition on the box: a name already held elsewhere returns 409 BucketAlreadyExists; the owner’s own pre-existing bucket is an idempotent 200. A post-create configuration failure (object-lock, default-ACL, ownership) triggers an all-or-nothing teardown (compensatingDeleteBucket, BUG-306) — no half-configured bucket survives.

The gateway is cost_center-scoped, multi-tenant: bucket and object stores resolve by (cost_center, …), and the box’s own primary account is remapped to legacy root ("") so pre-multitenancy buckets stay visible at the bucket root instead of moving under a per-owner folder (“owner-at-root”). A genuine second tenant keeps its own distinct partition. Requests land on the S3 data plane, port 9000 plain HTTP (9443 TLS additive), SigV4-signed.

The BUG-280 disappearing-buckets fix (rel 2.3.1) is relevant here: a metadata-directory read used to leave a leading-slash artifact on certain bucket names, which then failed the name-format check and silently vanished from a page — now trimmed and normalized before validation, at both the listing site and the pagination-token decode site.

CreateBucket
DNS-name validation when a custom S3 domain is set. Per-box uniqueness check, atomic post-create teardown on failure.
DeleteBucket
A background reaper drains contracts (bucket_cleanup.go), scoped to (cost_center, bucket) so one tenant's delete can never sweep another's rows (BUG-240).
HeadBucket / GetBucketLocation
Existence check; location always returns us-east-1 (region-agnostic gateway).
ListBuckets
The box owner's list is match-all across every tenant partition (typed ReadScope{IsOwner:true}); a genuine tenant's list stays scoped to its own cost_center.
ListObjectsV1 / V2
Prefix + delimiter, continuation tokens, both generations. read_handlers.go.
ListObjectVersions
Coupled to the versioning subsystem — see the versioning & object lock page.


How we conform to the S3 protocol

Measured against the public ceph/s3-tests suite — and published.

Bucket and listing behavior — CRUD, per-box uniqueness enforcement, and both ListObjects generations — is verified against the same public conformance suite the rest of the industry is measured by, test by test. We publish the full per-test result, including what still fails, on the compatibility matrix rather than summarizing it here.

Where we deliberately differ. An anonymous or cross-principal caller listing a bucket without credentials gets a uniform AccessDenied, refused outright under the 2026-08 product-security tightening (BUG-361). That's a wider deny than the pre-hardening posture, not a compatibility gap.

The reasoning behind that posture is set out under “What we chose to be different about” on the compatibility matrix, alongside every measured figure and its provenance.


How we compare

Unlimited buckets per account is a real differentiator against most named vendors.

Source: the “Buckets & listing” section of the S3 compatibility matrix, competitor cells from each vendor’s own documentation, researched 2026-06-14.

Competitor figures from each vendor’s official documentation, researched 2026-06-14.
CapabilityXNSAWS S3CephMinIOWasabiB2Storj
Create / Delete bucket
ListObjects v1encrypted keys break lexicographic order
ListObjects v2◐ partial
Prefix + delimiter◐ partial
Buckets per accountUnlimited10,0001,000Unlimited1,000100100


How applications use it

Create the bucket once, then list with prefix + delimiter to browse it like a tree.

boto3

import boto3
s3 = boto3.client(
    "s3",
    endpoint_url="https://relayer.example.com",   # or http://localhost:9000 for local/dev
    aws_access_key_id="AKIA...",
    aws_secret_access_key="...",
    region_name="us-east-1",
)
s3.create_bucket(Bucket="my-bucket")
# List "directories" one level deep using delimiter
resp = s3.list_objects_v2(Bucket="my-bucket", Prefix="logs/2026-08/", Delimiter="/")
for cp in resp.get("CommonPrefixes", []):
    print(cp["Prefix"])
# Paginate a full listing
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket"):
    for obj in page.get("Contents", []):
        print(obj["Key"], obj["Size"])
# Which buckets do I own?
for b in s3.list_buckets()["Buckets"]:
    print(b["Name"])

AWS CLI

aws s3api create-bucket --bucket my-bucket --endpoint-url https://relayer.example.com
aws s3 ls s3://my-bucket/logs/2026-08/ --endpoint-url https://relayer.example.com
aws s3api list-objects-v2 --bucket my-bucket --prefix logs/2026-08/ --delimiter / \
  --endpoint-url https://relayer.example.com
aws s3api list-buckets --endpoint-url https://relayer.example.com


Use cases

Where this matters in practice.

  • Multi-bucket data lake catalogs
  • CI/CD artifact repositories
  • Date-partitioned log buckets
  • Bucket-existence probing in deploy scripts
Multi-bucket data lake catalogs

Each bucket represents a dataset or environment, enumerated via ListBuckets by an inventory job.

CI/CD artifact repositories

Organized by project/branch/build-id prefixes, browsed with prefix + delimiter instead of a full key dump.

Date-partitioned log buckets

logs/2026-08-12/… listed incrementally by day for a downstream ETL job.

Bucket-existence probing in deploy scripts

HeadBucket before CreateBucket to make provisioning idempotent.


Applications that lean on it heavily

Real software, named.

  • rclone / mc
  • Terraform's S3 backend
  • Data pipeline tools (Airbyte, Singer-style taps)
  • CI artifact browsers
rclone / mc

Both drive bucket sync and mirroring from repeated ListObjectsV2 calls; trailing-slash bucket-path normalization specifically targets these clients.

Terraform's S3 backend

Provisions the state bucket with CreateBucket/HeadBucket before ever writing a state object.

Data pipeline tools (Airbyte, Singer-style taps)

Page through source buckets with ListObjectsV2 pagination to discover new files incrementally.

CI artifact browsers

Build prefix-delimited “folder” views directly from CommonPrefixes in a ListObjectsV2 response.


What’s out of scope here

Bucket names are unique per box, not globally unique like AWS.

The boundary, stated plainly

A name is only checked against other tenants' partitions on the same box — two separate XNS deployments can each have a bucket named backups with no collision, which is a different guarantee than AWS provides and matters if you're porting infrastructure code that assumes global uniqueness.


FAQ

Buckets and listing questions, answered directly.

Unlimited on the XNS S3 Gateway — there is no fixed per-account bucket ceiling, unlike AWS (10,000), Ceph (1,000), Wasabi (1,000), B2 (100), or Storj (100).

Yes, with continuation tokens, prefix, and delimiter all supported and conformance-tested. A 2026-07 fix (BUG-280) also corrected a case where legacy-format bucket names could silently vanish from a ListBuckets page after a metadata-directory normalization step.

No. Bucket names are unique per box across all tenants (a per-box flat namespace) — a name already held by another principal’s partition returns 409 BucketAlreadyExists.



See the full conformance picture.

The buckets and listing rows above are one section of the full S3 compatibility matrix — object operations, versioning, lifecycle, encryption, access control, replication, and integration all get the same treatment.


Claims on this page last verified
© Copyright - SCP, Corp | Xa Net Services and Affiliates