Meta AI Introduces EUPE: A Compact Encoder for Mobile Vision Tasks

Running powerful AI on smartphones isn't just a hardware issue; it's also an architectural challenge. Most state-of-the-art vision encoders are large, and when scaled down for edge devices, they lose the capabilities that made them effective. Furthermore, specialized models often excel in one area, such as image classification or scene segmentation, but struggle with tasks outside their specialization.

Meta AI's research teams propose a new approach with the introduction of the Efficient Universal Perception Encoder (EUPE), a compact vision encoder that can handle diverse vision tasks simultaneously without being large. To understand the significance of EUPE, it's essential to grasp how vision encoders function and why specialization can be problematic.

A vision encoder is the component of a computer vision model that converts raw image pixels into a compact representation used for downstream tasks like classification or segmentation. Modern foundation vision encoders are trained with specific objectives, giving them an edge in particular domains. For instance, CLIP and SigLIP 2 excel at image understanding but often underperform in dense prediction tasks. Conversely, DINOv2 and DINOv3 are strong in dense prediction but lack satisfactory language capabilities.

For edge devices like smartphones or AR headsets that need to manage all these task types simultaneously, the typical solution is to deploy multiple encoders, which quickly becomes cost-prohibitive. The alternative is to accept that a single encoder will underperform across various domains. Researchers have attempted to combine the strengths of multiple specialist encoders through agglomerative distillation methods, but results significantly degrade when applied to efficient models.

The key insight behind EUPE is the principle of 'first scaling up and then scaling down.' Instead of distilling knowledge directly from multiple domain-expert teachers into a small student model, EUPE introduces an intermediate model: a large proxy teacher capable of unifying knowledge from all domain experts. This proxy teacher then transfers its unified knowledge to the efficient student through distillation.