“Too heavy of a hammer”: How Yutori is beating AI giants at building the web’s future
In a world racing to frontier intelligence, there remains value in building for highly specialized tasks.
For over thirty years, using the web has meant the same ritual: open a site, click through it, and leave with an answer. And while everything else on the internet has changed, this core ritual hasn’t.
Now, Dr. Devi Parikh wants an AI to do it for you.
While the process has long worked fine for a handful of tasks, it’s far more tedious when combing through thousands of merchant sites, filling out hundreds of procurement forms, or running the same flow across sites that all work differently. And you can’t just script your way out of it.
“If you have to go through the same flows across a large variety of websites, you can’t write scripts for that, because the scripts are not going to generalize across 100,000 websites that have all been built independently,” said Parikh, a former senior director at Meta who led multimodal generative AI research and now runs a startup called Yutori.
To solve this in the age of AI, the industry is now looking at agents that can dynamically “interact with websites, take actions, and complete tasks,” essentially doing what a human does but at superhuman speeds and skipping directly to the results.
OpenAI and Anthropic are already providing browser-use capabilities with their respective frontier models, but for Parikh, that approach is too heavy of a hammer. Instead, with Yutori, she’s offering a purpose-made answer – specialized web agents that do just one thing flawlessly: operate a browser at scale and with reliability.
Today, Yutori is processing over 100 million API requests a month and handling massive automation workloads for FAANG-level enterprise clients, betting that when it comes to the chaotic web, specialized agents can outperform general AI brains.
Building for the future of the web
Back in early 2024, just around the time of Llama 3, there was hardly any chatter of agents. The only thing making headlines in AI was how LLMs were improving across domains like coding, language, and math. At that time, Parikh and her colleagues, Abhishek Das and Dhruv Batra, envisioned agents that could interact with websites and take actions, giving users swamped with web tasks some “room to breathe” (hence the namesake “Yutori” in Japanese).
But the trio noted that the performance of existing systems was far from reliable, barely handling even 2-3 steps when performing a task.
“At the time, if you took any of the frontier models, whether GPT, Claude, or Llama, and put it in a loop to complete these tasks, that just didn’t work. We felt if we can put together a strong team that can train models focused on this problem, then in a year or so we can make a solid dent and unlock various product use cases,” Parikh explained.
Two years and multiple products and model training runs down the line, Yutori is finally making that dent with its agents.
Initially, Parikh said, their agents went up to six to seven actions but broke down when the task demanded longer work. However, after optimizing their training recipes and product architecture, the agent performance improved – now going up to as many as 150 steps when working on a site.
Yutori’s business model also evolved in these two years as well.
The company launched with consumer products “Scouts” and “Delegate,” but started serving API access to the underlying technology – its Navigator models – after seeing massive demand from enterprise giants seeking quick, cheap web automation. Today, it counts Meta, Samsung, Theory Ventures, Cohesion, Utility Profit, and Meritech among its customers.
Weighing intelligence & cost tradeoffs
While much of the AI industry is obsessed with pushing raw intelligence and reaching the frontier, for Yutori, that was only one part of the problem.
To deliver a technology that is truly usable across hundreds of thousands of websites – like for companies aggregating restaurant menus across Europe or those hunting for specific merchant deals – the company also had to solve for two other aspects: reliability of the agent when it performs a task and how affordable its run turns out to be.
If you ask a massive, general-purpose frontier model (trained to do complex math, deep reasoning, and advanced coding) today to navigate to a clunky merchant website, fill out a multi-page form, and check out, it will surely do so, but at too high a cost.
“They’re massive frontier models that will try to do everything, and it’s just too heavy of a hammer to bring. If you’re trying to fill out a form, you don’t need Opus-level frontier intelligence,” Parikh said.
Yutori’s system, she says, matches or beats frontier models on accuracy while running faster and cheaper – because it does only one thing.
“The fact that it’s significantly cheaper isn’t just that it’s linear savings… It’s the difference between whether you can ship the product or not,” Parikh noted. “If it costs you $2.30 to fill out a form on a website, you’re not going to ship that at scale because the unit economics just don’t make sense.”
Initially, the Yutori team assumed the most efficient way for an AI to navigate a website was to read the underlying HTML and DOM code. But the web is entirely decentralized, and there is no strict standardization.
As a result, the team found themselves endlessly cleaning up HTML on a per-website basis just to give the model a clean signal. The approach simply couldn’t scale across the internet’s “heavy tail” of custom-built merchant sites.
Their breakthrough was realizing that websites are built for human eyes, making visual rendering the only true source of truth. Yutori shifted to a visually grounded approach where the model started taking screenshots to decide its next action.
But that, too, was expensive and even slow, given the model took one action at a time before moving to the next.
To offset the compute cost of processing continuous images, they finally came to a clever hybrid architecture.
When the model is confident, it skips the visual step and generates custom JavaScript to execute multiple actions at once, like filling out an entire form and checking multiple boxes in a single inference call. This saves significant time and compute. When the model is less confident, it relies on images to complete the task.
Teaching agents to recover from mistakes
But balancing JavaScript and screenshot analysis for cost efficiency is only half the battle. The other half is making sure the agent doesn’t panic when things go wrong, like encountering a random pop-up or a broken page layout.
Initially, Yutori relied on human contractors to guide the models, tracking their actions through a browser extension to gather training data. But that approach had a ceiling. There is only so much data you can collect with humans in the loop.
Today, the company has transitioned to a Reinforcement Learning (RL) regime. The agents collect their own action trajectories, and automated software verifiers identify which paths were correct, feeding them back into the training loop and improving the models’ ability to act according to the page at hand.
“Over time, we have shifted to rejection sampling where it’s synthetic data at work. The agents are collecting their own trajectories, with verifiers then identifying which trajectories are correct to add back to the training,” Parikh explained. “For a little while those verifiers were humans who would look at these trajectories, the process, and the outcome, but now even that is getting automated. So, we’re now in the RL regime, where these verifiers are also just software, built with coding agents.”
More importantly, Parikh noted, the team focuses heavily on edge cases. “A lot of the magic in these models is in the data,” she said. The agents are explicitly trained to recover from mistakes. If an agent clicks the wrong link or encounters a sudden UI change, it knows how to backpedal and resume the task accurately.
At the most basic level, she emphasized the most important skill to be trained into web agents is instructability, where you can offload entire tasks to agents, while drawing hard lines against sensitive actions like purchases.
“I think this level of control and flexibility where you feel you can trust this and it’s not going to go do something crazy in a way you’re not comfortable with is very important. Some of this is about how agents are trained. Some of it is also just how the product is designed around it and how it shows up in your life, ” Parikh added.
The reality check of building from scratch
While the technology is highly advanced, building it required a shift in mindset for the founders. Parikh, Das, and Batra carry some of the most impressive resumes in AI, having led high-priority work at Meta. But in the startup trenches, past credentials mean little.
“You are starting from scratch,” Parikh noted. “When we build a product and ship it, the users don’t care that [we] were at Meta and were researchers and professors… The product needs to work, people need to like it, and they need to get value out of it; otherwise, they’re not going to use it.”
This direct contact with reality and customer needs has shaped the company’s culture. In contrast to the punishing hustle culture common in Silicon Valley, Yutori operates with a stable, five-day in-office environment, reflecting the founders’ personalities and the very definition of “Yutori.”
Even as giants like OpenAI and Anthropic continue to push into the agentic space with massive resources and talent, Parikh is not concerned. She views the landscape much like the voice-AI sector, where specialized companies like ElevenLabs are shining. Despite massive general models, there is a deep need for vertically integrated startups that offer superior cost-to-accuracy tradeoffs for web navigation.
Looking five years out, she predicts a fundamental shift in how we experience the internet. While top-tier platforms will inevitably build native APIs for AI agents to plug into and read, the majority of the web will not. This will be the opportunity for Yutori to stand out.
“There’s a massive tail of websites that are not going to have APIs anytime in the near future,” Parikh said. “There is a good chance that the way in which this heavy tail gets API’d is by using browser-use and computer-use models like the ones we are training. That’s how it becomes machine-compatible.”