Alibaba Trains Multi-Screen AI Agent Using 100-Device Hardware Array

Alibaba has launched Qwen-UI-Agent, a GUI intelligent agent base model designed to execute tasks across digital devices. The move signals a strategic shift toward AI that can actively operate hardware rather than simply generating text.

To develop the model, Alibaba constructed a real machine mobile runtime featuring more than 100 physical mobile phones and over 150 different applications. This infrastructure allows the AI to be trained and evaluated on actual hardware, utilizing an agent-driven data flywheel powered by reinforcement learning (RL). The resulting model is capable of operating across mobile, PC, and web interfaces, enabling complex cross-device execution, such as transferring information from a smartphone to a computer.

Solving the Sim-to-Real Gap

Most current AI agents are trained using synthetic data or screen recordings within simulated environments. While these models often perform well in controlled tests, they frequently fail when deployed on actual devices. This discrepancy is known as the "sim-to-real gap."

In real-world usage, AI agents must contend with unpredictable variables that simulations cannot perfectly replicate. These include unexpected system pop-ups, network jitter, and dynamic interface changes that occur in real time. By utilizing a physical array of devices, Alibaba is forcing the AI to navigate these frictions, ensuring the agent can handle the instability of live operating systems.

From Chatbot to Operator

This development represents a fundamental transition from "AI as a chatbot" to "AI as an operator." While standard AI integrations on phones typically act as overlays or voice assistants, a GUI-based agent can interact with the user's digital ecosystem as a human would—clicking buttons, scrolling through apps, and managing workflows across multiple screens.

For the industry, this approach could lead to more reliable automation of complex digital tasks. If an AI can master the nuances of physical hardware interaction, it reduces the reliance on rigid API integrations, allowing the agent to operate any application that has a visual interface.

The Path Forward

As Alibaba pushes Qwen-UI-Agent toward broader application, the focus remains on whether this hardware-heavy training method can scale. While 100 devices provide a significant real-world dataset, the challenge will be maintaining this "data flywheel" as app interfaces update and new device hardware enters the market. Observers will be watching to see if this physical training regimen becomes the new standard for developing autonomous digital agents.