AI ·
Shipping small models
What we learned running compact models next to the user.
By L21 Labs
Placeholder post. Our Edge Inference experiment tests how far compact models go when they run on-device.
Early findings
- Interactive speeds are achievable on recent hardware.
- Removing the network round-trip changes how products feel.