ElastiCache to Redis Cloud: Live Migration with RIOT-X
AWS ElastiCache for Redis is the path of least resistance when you're already on AWS. But as your Redis usage grows — especially for AI workloads that need vector search, Active-Active geo-replication, Auto-Tiering, or fine-grained access control — ElastiCache's limitations become constraints. Redis Cloud offers all of these. The challenge: AWS makes it easy to get data into ElastiCache and hard to get data out.
ElastiCache doesn't support REPLICAOF. It doesn't let you export RDB files on demand. It doesn't offer a native migration-out tool. RIOT-X exists because AWS gave us no other option.
Why ElastiCache Makes Migration Difficult
On a self-managed Redis instance, migrating to another Redis is straightforward: run REPLICAOF against the target, wait for sync, promote the replica. Or take an RDB snapshot, transfer the file, load it on the target. ElastiCache blocks both paths.
| Capability | Self-Managed Redis | ElastiCache |
|---|---|---|
| REPLICAOF / SLAVEOF | Available — point any replica at any master | Blocked — cannot replicate to external Redis |
| RDB export on demand | BGSAVE → copy the .rdb file | BGSAVE blocked — but snapshots can be exported to S3 as standard .rdb files |
| MIGRATE command | Moves keys between instances | Blocked to external endpoints |
| DUMP + RESTORE | Serialize/deserialize individual keys | Available but not practical at scale (key by key) |
| Cluster mode external replication | Supported via replication | Not supported — cluster topology is managed internally |
One thing ElastiCache does allow: exporting snapshots to S3 as standard .rdb files. AWS explicitly states the exported file is a standard Redis RDB file — you can download it from S3 and load it into Redis Cloud, Redis Enterprise, or any self-managed Redis instance. This makes S3 export a viable offline migration path for smaller databases. The lock-in pressure is not in the .rdb format — it's in the blocked commands (REPLICAOF, BGSAVE, CONFIG, SYNC, PSYNC) that prevent live, real-time migration tooling from working natively.
RIOT-X: The Official Redis Migration Tool
RIOT-X (Redis Input/Output Tools) is the official command-line utility supported by Redis, Inc. for getting data in and out of Redis — including live migration from ElastiCache to Redis Cloud. The current version is 1.7.10. It uses the replicate command to read data from a source Redis and write to a target Redis, supporting standalone, cluster-mode, and sentinel topologies.
RIOT-X works at the command level: it identifies source keys using SCAN and/or keyspace notifications, reads data using DUMP (or type-specific commands), and writes to the target using RESTORE (or type-specific write commands). The full documentation is at redis.github.io/riotx.
How RIOT-X Replication Works
RIOT-X's replicate command supports three modes, controlled by the --mode flag:
| Mode | Command | Behavior |
|---|---|---|
| Scan (default) | riotx replicate source:port target:port | One-time snapshot — scans all keys via SCAN, DUMP+RESTORE each to the target. No live sync. |
| Live | riotx replicate --mode live source:port target:port | Snapshot first, then automatically switches to keyspace notification tailing for ongoing changes. |
| Live Only | riotx replicate --mode liveonly source:port target:port | No initial snapshot — only listens for keyspace notifications. Use when data is already synced and you need ongoing replication. |
- Phase 1 — Scan: RIOT-X scans every key in the source ElastiCache using the SCAN command. For each key, it runs DUMP to serialize the value, then RESTORE on the target Redis Cloud instance. In cluster mode, keys are scanned in parallel across cluster nodes. A status bar shows replication progress as a percentage.
- Phase 2 — Live replication (--mode live): After the scan completes, RIOT-X subscribes to keyspace notifications (requires notify-keyspace-events = KEA on the source). For each key change event, RIOT-X reads the updated key and writes it to the target. This keeps the target in near-real-time sync.
- Phase 3 — Cutover: When the live replication lag is minimal, switch your application's connection string from ElastiCache to Redis Cloud. Stop writes to ElastiCache. Terminate RIOT-X.
RIOT-X uses an async queue architecture for high throughput: keys are scanned in a background thread and queued, multiple reader threads process keys in parallel, and the main thread writes results to the target. This overlapping I/O design is significantly faster than sequential key-by-key transfer.
RIOT-X turns ElastiCache's locked-down environment into a migration source by working within the commands AWS allows — SCAN, DUMP, RESTORE, and keyspace notifications. No privileged access needed.
Step-by-Step Migration
Step 1: Prepare the Redis Cloud Target
- Create a Redis Cloud Pro subscription in the same AWS region as your ElastiCache cluster (minimizes data transfer costs during migration).
- Create a database with memory equal to or greater than your ElastiCache cluster's used_memory (check with INFO MEMORY on ElastiCache).
- Enable VPC peering between your AWS VPC (where ElastiCache lives) and the Redis Cloud VPC. This ensures the migration traffic stays on the private network — no internet egress fees.
- Note the Redis Cloud endpoint, port, and password from the database configuration page. The Redis URI format is: redis://default:password@host:port
Step 2: Set Up the Migration Host
RIOT-X runs on a host that can reach both ElastiCache and Redis Cloud. Deploy an EC2 instance (or use an existing bastion host) in the same VPC as your ElastiCache cluster. It needs network access to both the ElastiCache endpoint and the Redis Cloud endpoint (via VPC peering).
- Install RIOT-X following the manual installation guide at redis.github.io/riotx/quick-start/install.html.
- Verify installation: riotx --version (should show 1.7.10 or later).
- Install redis-cli for connectivity testing: sudo apt install -y redis-tools
- Test connectivity to both endpoints: redis-cli -h <elasticache-endpoint> -p 6379 PING and redis-cli -h <redis-cloud-endpoint> -p <port> -a <password> PING
Step 3: Enable Keyspace Notifications on ElastiCache
For live replication (--mode live), ElastiCache must emit keyspace notifications. Create a custom parameter group (or modify your existing one) and set notify-keyspace-events to KEA. Apply the parameter group to your ElastiCache cluster. This tells Redis to publish events for all key operations — which RIOT-X consumes for live sync. See the AWS Knowledge Center for step-by-step instructions.
Important: keyspace notifications add minor overhead to ElastiCache. On a heavily loaded cluster, monitor CPU after enabling. The overhead is typically 1-3% and temporary (only needed during migration).
Step 4: Run the Migration
The RIOT-X replicate command syntax is: riotx replicate [OPTIONS] SOURCE TARGET — where SOURCE and TARGET are Redis URIs.
ElastiCache Single Master → Redis Cloud (snapshot): riotx replicate redis://elasticache-endpoint:6379 redis://default:password@redis-cloud-endpoint:port
ElastiCache Single Master → Redis Cloud (live): riotx replicate --mode live redis://elasticache-endpoint:6379 redis://default:password@redis-cloud-endpoint:port
ElastiCache Cluster Mode → Redis Cloud: riotx replicate --source-cluster redis://elasticache-config-endpoint:6379 redis://default:password@redis-cloud-endpoint:port
ElastiCache Cluster Mode → Redis Cloud (live, with OSS Cluster target): riotx replicate --source-cluster --target-cluster --mode live redis://elasticache-config-endpoint:6379 redis://default:password@redis-cloud-endpoint:port
If ElastiCache has AUTH TOKEN enabled, add --source-tls and --source-pass: riotx replicate redis://elasticache-endpoint:6379 redis://default:password@redis-cloud-endpoint:port --source-tls --source-pass <auth-token>
| Flag | Purpose |
|---|---|
| --mode scan | Default. One-time snapshot migration via SCAN + DUMP/RESTORE. |
| --mode live | Snapshot + live replication via keyspace notifications. Use for zero-downtime migration. |
| --mode liveonly | Live replication only — no initial snapshot. For ongoing sync after data is already copied. |
| --source-cluster | Enable when source ElastiCache has cluster-mode enabled. |
| --target-cluster | Enable when target Redis uses OSS Cluster API. |
| --source-tls | Enable TLS for the source connection (required if ElastiCache has encryption in-transit). |
| --source-pass | Source AUTH password/token. |
| --scan-count | SCAN count hint per iteration (increase for faster key discovery). |
| --read-batch | Number of keys to read in each batch (default 50, increase to 200+ for faster migration). |
| --read-threads | Parallel reader threads (default 1, increase to 4-8 for large datasets). |
| --source-pool | Source connection pool size (increase with read-threads, e.g. 16). |
| --key-pattern | SCAN MATCH pattern to filter specific keys (e.g. 'session:*'). |
| --key-type | SCAN TYPE filter to replicate only specific data types (e.g. 'string', 'hash'). |
| --mem-limit | Skip keys larger than this size — useful for handling big keys separately. |
For high-throughput migration of large datasets, tune the parallel processing: riotx replicate redis://source:6379 redis://target:port --scan-count 2000 --read-batch 200 --read-threads 8 --source-pool 16
Use --read-threads and --source-pool to parallelize the migration without overloading the source. RIOT-X's async queue architecture overlaps scanning, reading, and writing for maximum throughput.
Step 5: Monitor Progress
RIOT-X shows a progress bar with the percentage of keys replicated during the scan phase. Monitor this alongside ElastiCache CloudWatch metrics (CPUUtilization, NetworkBytesOut, CurrConnections) to ensure the migration isn't impacting production.
- Watch ElastiCache CPU — should stay below 70% during migration. If it spikes, reduce --read-threads or --read-batch size.
- Watch Redis Cloud incoming connections — RIOT-X opens multiple connections based on thread count and pool size.
- Compare DBSIZE on source and target — when they match (and live replication is running), you're ready for cutover.
- Check for RESTORE errors — TTL mismatches, maxmemory exceeded on target, or serialization version incompatibilities. RIOT-X logs these.
Step 6: Validate and Cut Over
- 1. Compare key counts: DBSIZE on source vs target. They should match within the live replication lag.
- 2. Spot-check values: Pick 50-100 keys across different data types and compare values between source and target.
- 3. Verify TTLs: Ensure keys with expiry on the source have matching (or slightly shorter, due to elapsed time) TTLs on the target.
- 4. Test application connectivity: Point a staging instance of your application at the Redis Cloud endpoint and verify reads/writes work correctly.
- 5. Cut over: Update your application's Redis connection string to the Redis Cloud endpoint. This can be an environment variable change, a config service update, or a DNS CNAME flip.
- 6. Stop RIOT-X: Once the application is fully on Redis Cloud and no longer writing to ElastiCache, terminate RIOT-X.
- 7. Keep ElastiCache running for 24-48 hours as a rollback safety net, then decommission.
CloudFormation: Automated Migration
For teams that prefer a fully automated approach, RIOT-X provides an AWS CloudFormation template that handles the entire migration infrastructure. You provide your ElastiCache Cluster ID and the target Redis URI — the template automatically discovers VPC configuration, creates an ECS task to run RIOT-X, executes the migration, and logs everything to CloudWatch.
- Deploy the template from the AWS CloudFormation console with the pre-loaded RIOT-X template.
- Enter two parameters: your ElastiCache Cluster ID and the target Redis URI (redis://default:password@host:port).
- The template auto-discovers your VPC, subnets, and security groups.
- Migration runs as an ECS Fargate task — no EC2 instance to manage.
- Monitor progress in CloudWatch Logs under /ecs/riot-migration.
- Stack outputs show migration status, source configuration, and log URLs.
The CloudFormation template turns the migration into a two-parameter deployment. Enter the ElastiCache Cluster ID and Redis Cloud URI — infrastructure, networking, and execution are handled automatically.
Handling Cluster-Mode Migrations
ElastiCache cluster mode distributes keys across shards using hash slots. Use --source-cluster when your source ElastiCache has cluster-mode enabled. Use --target-cluster when your target Redis uses OSS Cluster API. If your target Redis Cloud database does not use OSS Cluster API (which is common — Redis Cloud's proxy handles sharding transparently), RIOT-X can write to the single endpoint and the proxy distributes to the correct shard.
Always specify the source database first and the target database second in the replicate command.
Alternative: RDB Snapshot Import
For smaller databases or when live replication isn't required, the S3 backup path works well. Take an ElastiCache snapshot, export it to S3 via copy-snapshot or the console — the exported file is a standard .rdb that AWS confirms can be used outside ElastiCache. Download it from S3 and import it into Redis Cloud using the database import feature (available on Pro plans), or load it into any Redis instance with redis-server --dbfilename.
| Approach | Downtime | Live Sync | Complexity | Best For |
|---|---|---|---|---|
| RIOT-X live (--mode live) | Seconds (cutover only) | Yes — keyspace notifications | Medium | Production workloads, zero-downtime requirement |
| RIOT-X scan (default) | Minutes (scan + cutover) | No | Low | Development/staging, small datasets |
| RIOT-X CloudFormation | Seconds to minutes | Configurable (SCAN/LIVE/LIVEONLY) | Low (automated) | Teams wanting fully managed migration |
| RDB export → Redis Cloud import | Minutes to hours | No | Low | Small databases, acceptable downtime window |
| DUMP/RESTORE per key (scripted) | Varies | No | High | Selective key migration, partial datasets |
Common Pitfalls
- Forgetting TLS — Redis Cloud requires TLS. If ElastiCache has encryption in-transit, use --source-tls. For the target, use the rediss:// URI scheme (double s) to enable TLS.
- Maxmemory on the target — If the Redis Cloud database is sized too small, RESTORE commands fail when memory is full. Provision the target with at least 1.2x the source's used_memory.
- Big keys — Large sets, sorted sets, or hashes can slow replication. Use RIOT-X's --mem-limit flag to skip oversized keys during the main migration and handle them separately. Run redis-cli --bigkeys on the source beforehand to identify them.
- Live replication is not guaranteed delivery — Keyspace notifications use pub/sub, which does not guarantee delivery. Network failures can cause missed events. For critical migrations, run a post-migration comparison using RIOT-X's compare mode to verify data consistency.
- Keyspace notification overhead — KEA notifications on a cluster doing 100K+ ops/sec generate significant event volume. Monitor ElastiCache output buffer sizes.
- Network bandwidth — A 50GB migration over a VPC peering link saturates quickly. Use --read-threads and --read-batch to control throughput. Schedule during off-peak hours.
- Forgetting to disable keyspace notifications after migration — Remove the KEA parameter group setting once migration is complete to eliminate the overhead.
Post-Migration: Unlock Redis Cloud Features
Once on Redis Cloud, you have capabilities that ElastiCache doesn't offer. Enable them to justify the migration.
- Auto-Tiering — Move cold keys to SSD, cut memory costs by 50-70%.
- RediSearch / Vector Search — Full-text and vector similarity search without a separate engine. Critical for AI workloads.
- Active-Active geo-replication — Conflict-free multi-region writes with CRDTs.
- Fine-grained ACLs — Per-user, per-command, per-key-pattern access control (vs ElastiCache's basic AUTH).
- Redis Modules — JSON, TimeSeries, Bloom, Graph — native data types ElastiCache doesn't support.
- Prometheus metrics on port 8070 — Direct metric scraping without CloudWatch middleman.
The migration is the hard part. Once you're on Redis Cloud, the features you unlock — vector search, Auto-Tiering, Active-Active, modules — make the effort worth multiples of the cost savings.