OSGym: a new framework for managing 1,000+ OS replicas

Training AI agents that can use a computer—opening apps, clicking buttons, writing code—is one of the hardest infrastructure challenges in modern AI. It’s not a data problem or a model problem; it’s a plumbing issue. You need to spin up hundreds, potentially thousands, of full operating system environments with real graphical user interfaces, all running simultaneously without breaking the bank for a university research lab.

OSGym, developed by a team from MIT, UIUC, CMU, USC, UVA, and UC Berkeley, aims to solve this problem. Before delving into the infrastructure, it’s essential to understand what a computer use agent is. Unlike chatbots that respond to text prompts, a computer use agent observes a desktop screenshot, decides what action to take, and executes it via keyboard and mouse inputs. Think of it as an AI that can operate any software like a human would.

Training such systems requires massive amounts of interaction data generated within real OS environments, making the process expensive and complex. One core issue is that running OS sandboxes at scale demands significant resources. Each virtual machine requires its own bootable disk, CPU, and RAM. Increasing the number of parallel instances leads to resource consumption problems that typical academic budgets cannot absorb.

OSGym addresses this challenge with four architectural optimizations. The first involves decentralized OS state management, where each OS replica has its own state manager, preventing failures from propagating across replicas. The second optimization concerns hardware-aware orchestration of OS replicas, allowing the use of Docker containers instead of full virtual machines to reduce overhead.

The third optimization addresses disk management through a copy-on-write technique, saving disk space and speeding up VM provisioning. The fourth optimization maintains a pool of pre-warmed containers, allowing resource recycling and preventing failures under high load. These enhancements make OSGym an effective tool for training AI agents capable of interacting with computers.