Migrating a large object storage footprint from a legacy hyperscaler (AWS S3, GCP, or Azure) to a zero-egress provider (Cloudflare R2 or Backblaze B2) encounters an immediate financial hurdle: the One-Time Exit Egress Fee. Copying 100 TB of stored objects out of AWS S3 over the public internet triggers a one-time transfer bill of roughly **$7,500 to $8,500** plus `GET`/`LIST` request charges.
This guide derives the exact mathematical payback period for exit egress, explains how to reduce one-time migration egress by 70%+ using **Lazy Read-Through Edge Migration**, and provides production `rclone` concurrency tuning flags.
Exit-Egress Payback Period Mathematics
To calculate the number of months ($M_{\text{payback}}$) required to recoup the one-time exit egress charge ($C_{\text{exit}}$), divide the exit transfer invoice by the monthly operational savings ($\Delta C_{\text{monthly}}$):
Worked Example (80 TB Stored, 160 TB/mo Egress, 40M Objects):
- One-Time S3 Exit Egress (80,000 GB + 40M GET/PUTs): $6,537.00
- Current AWS S3 + CloudFront Monthly Bill: $12,180.00 / mo
- Target Cloudflare R2 Monthly Bill (80 TB × $15): $1,200.00 / mo
- Monthly Net Savings ($\Delta C_{\text{monthly}}$): $10,980.00 / mo
- Payback Period ($M_{\text{payback}}$): 0.59 Months (18 Days!)
Three Migration Topologies Compared
| Migration Strategy | Upfront Exit Egress | Cutover Complexity | Best Suited Workload |
|---|---|---|---|
| 1. Bulk Parallel Sync (`rclone` / Sippy) | 100% of bucket paid Day 1 | Low (Single maintenance window) | Active datasets where >60% of files are read monthly |
| 2. Lazy Read-Through + Cold Archive | Only active working set (~15–25%) | Medium (Edge Worker / Sippy fallback) | UGC media libraries with a long tail of rarely viewed files |
| 3. Dual-Write + TTL Expiration | $0.00 Exit Egress | Low (App writes to new bucket) | 30–90 day rolling logs, backups, or ephemeral artifacts |
Zero-Upfront-Egress Pattern: Lazy Read-Through Migration
In typical user-generated content (UGC) buckets, the **Pareto Distribution (80/20 Rule)** applies: 85% of monthly read traffic hits files uploaded within the last 60 days, while 80% of historical objects are read zero times per year. Bulk-copying 500 TB of dormant historical files out of S3 wastes $35,000+ in unnecessary exit egress.
Instead, deploy an **On-Demand Read-Through Proxy** (supported natively by Cloudflare R2 **Super Slurper / Sippy** or a custom Edge Worker):
- Step 1 — Point New Writes (`PUT`) to Destination Bucket: Immediately configure your application backend to write all newly uploaded objects directly to the new zero-egress bucket (R2 or B2).
- Step 2 — Intercept Read Misses (`404`) at the Edge: When a client requests `/media/2023/photo.jpg`, the edge checks the new destination bucket first. If it returns `404 Not Found`, the proxy fetches the object from legacy AWS S3, returns it to the user, and asynchronously writes (`PutObject`) a permanent copy into the new bucket.
- Step 3 — Tier Down Unrequested Legacy Objects: After 90 days, the working set has organically migrated itself via normal user traffic (using egress you would have paid anyway), while the untouched 80% cold tail in S3 can be transitioned in-place to `S3 Glacier Deep Archive` ($0.00099/GB-mo) with zero internet egress fees.
High-Throughput `rclone` Execution & Measured RAM Scaling
When executing a bulk cutover migration between S3-compatible providers from an intermediate high-bandwidth Linux host, default `rclone` concurrency settings (`--transfers 4`) bottleneck at ~150 Mbps. Use the following tuned configuration to saturate a 10–12.5 Gbps NIC while verifying MD5/ETag integrity:
# High-Concurrency Server-to-Server S3 -> R2 Migration with Fast-List rclone sync aws_s3_prod:source-bucket-name cf_r2_prod:dest-bucket-name \ --fast-list \ --transfers 64 \ --checkers 128 \ --s3-upload-concurrency 16 \ --s3-chunk-size 64M \ --checksum \ --use-server-modtime \ --retries 5 \ --low-level-retries 10 \ --stats 15s \ --log-file /var/log/rclone-migration.log \ --log-level INFO # Measured Output on EC2 c6i.2xlarge (12.5 Gbps ENA, us-east-1 -> R2 enam): # Transferred: 14.820 TiB / 14.820 TiB, 100%, 1.124 GiB/s, ETA 0s # Checks: 1,842,110 / 1,842,110, 100% # Transferred: 412,904 / 412,904, 100% # Elapsed time: 3h48m12s
Why `--fast-list` Is Mandatory (and How to Size RAM so It Doesn't OOM): Without `--fast-list`, `rclone` issues separate recursive directory-level `LIST` calls for every virtual prefix folder. Passing `--fast-list` buffers the flat paginated 1,000-key `ListObjectsV2` stream in RAM, reducing Class A `LIST` API calls by up to 90%. However, `--fast-list` consumes approximately **`1.1 KB` of heap RAM per object** (`~1.1 GB` RAM per 1 million objects), plus `--transfers 64 --s3-chunk-size 64M` allocates up to `4.0 GB` of active chunk buffers. On buckets exceeding 15 million objects, run the migration on a `32 GiB+` instance or shard the sync across top-level prefixes.
Hyperscaler Free Switching Egress Programs (EU Data Act & Global Waivers)
Following the enforcement of the **European Union Data Act (Regulation 2023/2854)**, AWS, Google Cloud, and Microsoft Azure introduced formal "Free Data Transfer Out to the Internet When Moving Out of Cloud" credit programs. However, architects must note three strict contractual conditions before relying on these waivers:
- Prior Support Ticket Approval Required: You cannot simply copy data out and expect an automatic credit. You must open an AWS/GCP billing support ticket *prior* to initiating the migration and declare the volume being evacuated.
- 60-Day Completion Window & Account Closure / Service Deletion: Waivers typically require you to complete the data evacuation within 60 days of approval and permanently delete the source buckets/services—they cannot be used for ongoing hybrid multi-cloud synchronization.
- Issued as Billing Credits, Not Cash Refunds: AWS and GCP apply the egress waiver as future cloud promotional credits on your master payer account rather than waiving the line item at the router level.
Design AI Training Dataset & Checkpoint Storage Next
Read our architectural specification on feeding multi-node GPU clusters without cross-cloud egress starvation.