AWS Launches SOCI Parallel Pull to Slash Startup Times for Massive AI Container Images

AWS has introduced a "Parallel Pull" mode for Seekable OCI (SOCI) on Amazon EKS to eliminate the startup bottlenecks associated with multi-gigabyte AI and machine learning container images. By shifting from sequential downloads to concurrent processing, the update allows massive images to be ready in seconds rather than minutes.

Traditional container pulls operate sequentially, a process that becomes untenable as AI/ML images grow to between 20GB and 48GB. In these scenarios, the image pull becomes the dominant component of total startup time. The new Parallel Pull mode optimizes the "cold pull" scenario—where images must be fully present on the host—by utilizing parallel HTTP range requests to download and unpack image chunks concurrently within containerd. This approach significantly reduces the time expensive GPU instances sit idle while waiting for network I/O.

The Scale of the Image Bottleneck

Modern ML stacks, which include CUDA, deep learning frameworks, and heavy model weights, have pushed image sizes to extremes. These images often reach 20-48 GB when compressed. In standard Kubernetes environments, the overhead of connection setup and the sheer volume of data create a massive delay. This is particularly costly for organizations using GPU-accelerated instances, where hardware is billed while it remains idle during the pull process. This creates minutes during which provisioned accelerators are ready to process real work but are waiting for images to be pulled.

Infrastructure Elasticity and Performance

SOCI is an open-source technology from AWS that enables lazy loading, allowing containers to start without downloading the entire image upfront. The efficacy of this approach has already been proven at scale; SOCI lazy loading was deployed on Amazon ECS Fargate during Prime Day 2025, supporting the launch of 18.4 million tasks per day. For AI/ML companies, moving from minute-long startups to second-long startups directly improves infrastructure elasticity. Faster pulls enable more responsive auto-scaling during traffic spikes and quicker recovery during node failures, ensuring that high-cost compute resources are utilized for actual computation.

Addressing Memory Exhaustion

As parallel pulling increases throughput, it introduces new technical challenges regarding system memory. Disk-Backed Parallel Pull (DBPP) is a research-driven approach to address memory exhaustion issues found in containerd. In standard parallel pulls, out-of-order chunks can accumulate in the runtime heap, potentially crashing the system. DBPP solves this by pre-sizing destination files, allowing these chunks to accumulate on disk rather than in the heap, ensuring stability even when handling the largest container images.