Every recipe below is a working script against real S3 — transcribe and query video, parse a document corpus, checkpoint a fine-tune, hand artifacts between agents, keep a vector index that survives a restart. Clone it, run it, keep it. Reading your data back has no per-read charge, so re-running is free on the storage side.
5 recipes
Each one runs end to end
$0 egress
Re-runs cost nothing to read
< 60s on a laptop
Four of the five, once set up
Apache 2.0
Fork it, ship it, no strings
Three commands
You need a storage endpoint, a loader, and a recipe. The first command stands up your own S3 gateway — an AI assistant does the install, the account registration, and the credential provisioning. The rest is ordinary Python.
claude mcp add relayer -- npx @xns-cloud/relayer-mcp@latest
Then ask it to “set up XNS storage.” It checks prerequisites, registers the account, installs and starts the gateway on your Docker host, provisions S3 credentials, and writes ~/.xns/credentials. If you would rather run the compose file yourself, relayer-quickstart does the same job with docker compose up -d.
pip install langchain-xns
# no endpoint, no keys — step 1 wrote the credentials file
from langchain_xns import XNSLoader
docs = XNSLoader("corporate-strategy", suffixes=(".md",)).load()
That is the whole setup. From here, every recipe below is a git clone and a python command away.
The cookbooks
Each recipe ships an architecture diagram, a runnable script, and a section that states exactly where the starter version stops being enough. They are single-process and happy-path on purpose — the point is that you can read the whole thing in one sitting.
Query video by what was said and what was shown. The audio track is transcribed; ffmpeg-sampled frames are captioned by a vision model, so slide titles and chart labels become searchable text. Both are timestamp-labeled into one index.
python pipeline.py media "What was on the slide when they discussed pricing?"
Transcripts, captions and embeddings all cache back into the bucket, so a re-run only pays for what changed.
runnable
Turn a bucket of PDFs and spreadsheets into structured JSON an agent can query, and re-run it nightly without paying to read the corpus again. Docling parses locally — raw document bytes never leave your host; only parsed text reaches the model API.
python pipeline.py documents
Every stage is keyed by a fingerprint of its inputs, so an unchanged corpus makes no model calls at all. Set an invalid API key and re-run to prove it.
runnable
A GPU worker can vanish halfway through a fine-tune. Each completed checkpoint is uploaded with a manifest; a replacement machine finds the newest complete checkpoint, verifies whole-file SHA-256 after download, and resumes. Treat training machines as disposable.
python checkpoint.py demo
The demo simulates training — no torch, no GPU — so you can exercise the real transfer and recovery path on a laptop.
runnable
Two CrewAI agents exchange artifacts through a shared bucket instead of stuffing them into every handoff prompt. One writes work/brief.md; the second is given the key, fetches it, and writes outputs/report.md. The same key contract works across machines.
python crew.py --selftest
The selftest exercises write, read, list, missing-key handling and a presigned download against your gateway without spending a token. Run it first — if it fails, the problem is configuration, not the crew.
runnable
A vector database that survives a restart. Qdrant runs in a container beside the bucket, the collection is snapshotted to XNS, and a cold start restores from the bucket instead of re-embedding the corpus. Throw the disk away and the index comes back.
python index.py cold-start my-bucket "what does the corpus say about X?"
Ingest once, then docker compose down -v and cold-start: the restore path is exercised in the same run, so the claim is demonstrated rather than asserted. Longer than the other four — a container pull and a snapshot round trip do not fit in 60 seconds.
runnable
A running XNS Relayer with ~/.xns/credentials written, and your own model API key where a recipe calls a model. Nothing else.
The gateway these were verified against is a pre-release build. Treat the behaviors each recipe confirms — presigned URLs, read-after-write on a handoff, multipart checksums — as confirmed on that build, not as guarantees for every release. Re-check against the build you run.
langchain-xns
It would connect — override endpoint_url and you are talking to the gateway. Three things then still fall to you, and each one is a line of code you should not have to write.
from langchain_xns import XNSLoader
loader = XNSLoader(
"corporate-strategy",
prefix="2026/", # pushed to the server
suffixes=(".md", ".txt"),
glob="*/notes/*", # fnmatch over the full key
)
for doc in loader.lazy_load(): # streams; never lists the bucket into memory
print(doc.metadata["source"], len(doc.page_content))
Setup writes ~/.xns/credentials. The package reads it, so nothing has to be threaded through your app config or your environment. Arguments win over env vars, which win over the file.
A self-hosted gateway is often reached by host:port during bring-up, where virtual-hosted addressing cannot work. Path-style is the default here, so the first connection succeeds.
Filtering runs against the ListObjectsV2 response, so a key excluded by suffixes, glob, or max_bytes is never downloaded at all — not downloaded and then discarded.
Why read amplification wrecks a storage bill, what a Virtual Private Datacenter is, and how XNS compares to the other no-egress clouds — that case is made on the agentic workflows page. This page is the code that follows from it.
Explore the platform
The platform your pipelines run on — from how it works to what shipped this week.
The recipes are Apache 2.0 and the gateway is yours to run. Stand up storage, point a pipeline at it, and re-read the corpus as many times as the work needs.