Hi, this is Sankalp. I write about technology, AI, and other interesting things.
VLA Models: Why Hardware Is Harder Than Text
I spend most of my days working with LLMs and building backend systems. So when I first heard about Vision-Language-Action (VLA) models like OpenVLA, π₀, and Octo, my brain immediately slotted them into a familiar box: “multimodal LLM, different output head, done.” Turns out that’s roughly as accurate as saying a Formula 1 car is just a sedan with a bigger engine. The thing that broke my mental model was latency. When you’re streaming tokens from a chatbot, an extra 200 ms is barely noticeable. Maybe the cursor blinks a beat longer. When you’re controlling a robot arm that needs to react at 30 Hz, 200 ms means it’s already overshot, crashed into the table, or yeeted whatever it was holding across the room. You can’t retry a physical collision. There’s no Ctrl+Z in meatspace. ...