Lumabri Enables Frontier MoE Model Execution via P2P Swarms

Developer JustVugg has released Lumabri, a system that allows massive Mixture-of-Experts (MoE) models to be executed across a peer-to-peer (P2P) swarm. The project decouples model size from local hardware constraints by distributing model weights across a network of peers, allowing users to run frontier-scale AI on consumer-grade hardware.

At the core of the system is the Colibri engine, a lightweight inference engine written in pure C with zero dependencies. Lumabri leverages this engine to enable a swarm architecture where one machine shares a model while others interact with it. Rather than requiring a full upfront download of the model, bytes are streamed from peers on first use. These bytes are then stored in a local mirror, ensuring that subsequent inferences operate at full local disk speed.

The Shift to Disk-First Inference

Mixture-of-Experts models are typically massive, often requiring significant VRAM or RAM to load into memory for execution. Traditional inference requires the entire model, or a significant portion of it, to reside in high-speed memory to be performant. Colibri changes this paradigm by designing for CPU and SSD first, streaming specific "experts" from the disk as needed rather than loading the entire model into memory.

While the system is optimized for CPU and SSD usage, GPU acceleration is supported. According to the project documentation, this acceleration does not alter the byte-for-byte output of the model, maintaining consistency across different hardware configurations.

Lowering the Hardware Barrier

This architectural shift significantly lowers the entry barrier for running frontier-scale models. By combining P2P distribution with a stream-from-disk engine, users can run models that far exceed their local hardware capacity. This removes the necessity for expensive, high-VRAM GPUs for every participant in a swarm, as the network effectively shares the storage and delivery burden of the model's experts.

Future Outlook

As MoE models continue to grow in parameter count, the ability to distribute weights dynamically across a network could become a critical utility for decentralized AI. Observers will be watching to see how Lumabri handles latency in larger, more geographically dispersed swarms and whether the local mirroring strategy remains efficient as model sizes scale further. For now, the project provides a blueprint for running high-capacity models on hardware users already own.