Shortly after Hunter Lightman joined OpenAI as a researcher in 2022, he watched his colleagues launch ChatGPT, one of many fastest-growing merchandise ever. In the meantime, Lightman quietly labored on a crew educating OpenAI’s fashions to resolve highschool math competitions.
At the moment that crew, generally known as MathGen, is taken into account instrumental to OpenAI’s industry-leading effort to create AI reasoning fashions: the core expertise behind AI brokers that may do duties on a pc like a human would.
“We had been attempting to make the fashions higher at mathematical reasoning, which on the time they weren’t superb at,” Lightman informed TechCrunch, describing MathGen’s early work.
OpenAI’s fashions are removed from good right this moment — the corporate’s newest AI methods nonetheless hallucinate and its brokers battle with advanced duties.
However its state-of-the-art fashions have improved considerably on mathematical reasoning. One among OpenAI’s fashions lately received a gold medal on the Worldwide Math Olympiad, a math competitors for the world’s brightest highschool college students. OpenAI believes these reasoning capabilities will translate to different topics, and finally energy general-purpose brokers that the corporate has at all times dreamed of constructing.
ChatGPT was a contented accident — a lowkey analysis preview turned viral client enterprise — however OpenAI’s brokers are the product of a years-long, deliberate effort inside the firm.
“Ultimately, you’ll simply ask the pc for what you want and it’ll do all of those duties for you,” mentioned OpenAI CEO Sam Altman on the firm’s first developer convention in 2023. “These capabilities are sometimes talked about within the AI subject as brokers. The upsides of this are going to be large.”
Techcrunch occasion
San Francisco
|
October 27-29, 2025
OpenAI CEO Sam Altman speaks throughout the OpenAI DevDay occasion on November 06, 2023 in San Francisco, California.(Picture by Justin Sullivan/Getty Photographs)Picture Credit:Justin Sullivan / Getty Photographs
Whether or not brokers will meet Altman’s imaginative and prescient stays to be seen, however OpenAI shocked the world with the discharge of its first AI reasoning mannequin, o1, within the fall of 2024. Lower than a yr later, the 21 foundational researchers behind that breakthrough are probably the most extremely sought-after expertise in Silicon Valley.
Mark Zuckerberg recruited 5 of the o1 researchers to work on Meta’s new superintelligence-focused unit, providing some compensation packages north of $100 million. One among them, Shengjia Zhao, was lately named chief scientist of Meta Superintelligence Labs.
The reinforcement studying renaissance
The rise of OpenAI’s reasoning fashions and brokers are tied to a machine studying coaching approach generally known as reinforcement studying (RL). RL supplies suggestions to an AI mannequin on whether or not its selections had been appropriate or not in simulated environments.
RL has been used for many years. As an illustration, in 2016, a few yr after OpenAI was based in 2015, an AI system created by Google DeepMind utilizing RL, AlphaGo, gained international consideration after beating a world champion within the board sport, Go.
South Korean skilled Go participant Lee Se-Dol (R) prepares for his fourth match towards Google’s synthetic intelligence program, AlphaGo, throughout the Google DeepMind Problem Match on March 13, 2016 in Seoul, South Korea. Lee Se-dol performed a five-game match towards a pc program developed by a Google, AlphaGo. (Picture by Google by way of Getty Photographs)
Round that point, one in every of OpenAI’s first workers, Andrej Karpathy, started pondering how you can leverage RL to create an AI agent that would use a pc. However it might take years for OpenAI to develop the required fashions and coaching strategies.
By 2018, OpenAI pioneered its first massive language mannequin within the GPT sequence, pretrained on huge quantities of web information and a big clusters of GPUs. GPT fashions excelled at textual content processing, finally resulting in ChatGPT, however struggled with primary math.
It took till 2023 for OpenAI to realize a breakthrough, initially dubbed “Q*” after which “Strawberry,” by combining LLMs, RL, and a method known as test-time computation. The latter gave the fashions additional time and computing energy to plan and work by way of issues, verifying its steps, earlier than offering a solution.
This allowed OpenAI to introduce a brand new method known as “chain-of-thought” (CoT), which improved AI’s efficiency on math questions the fashions hadn’t seen earlier than.
“I may see the mannequin beginning to motive,” mentioned El Kishky. “It could discover errors and backtrack, it might get pissed off. It actually felt like studying the ideas of an individual.”
Although individually these strategies weren’t novel, OpenAI uniquely mixed them to create Strawberry, which straight led to the event of o1. OpenAI rapidly recognized that the planning and reality checking skills of AI reasoning fashions may very well be helpful to energy AI brokers.
“We had solved an issue that I had been banging my head towards for a few years,” mentioned Lightman. “It was probably the most thrilling moments of my analysis profession.”
Scaling reasoning
With AI reasoning fashions, OpenAI decided it had two new axes that might permit it to enhance AI fashions: utilizing extra computational energy throughout the post-training of AI fashions, and giving AI fashions extra time and processing energy whereas answering a query.
“OpenAI, as an organization, thinks loads about not simply the best way issues are, however the best way issues are going to scale,” mentioned Lightman.
Shortly after the 2023 Strawberry breakthrough, OpenAI spun up an “Brokers” crew led by OpenAI researcher Daniel Selsam to make additional progress on this new paradigm, two sources informed TechCrunch. Though the crew was known as “Brokers,” OpenAI didn’t initially differentiate between reasoning fashions and brokers as we consider them right this moment. The corporate simply wished to make AI methods able to finishing advanced duties.
Ultimately, the work of Selsam’s Brokers crew turned half of a bigger challenge to develop the o1 reasoning mannequin, with leaders together with OpenAI co-founder Ilya Sutskever, chief analysis officer Mark Chen, and chief scientist Jakub Pachocki.
Ilya Sutskever, Russian Israeli-Canadian pc scientist and co-founder and Chief Scientist of OpenAI, speaks at Tel Aviv College in Tel Aviv on June 5, 2023. (Picture by JACK GUEZ / AFP)Picture Credit:Getty Photographs
OpenAI must divert valuable assets — primarily expertise and GPUs — to create o1. All through OpenAI’s historical past, researchers have needed to negotiate with firm leaders to acquire assets; demonstrating breakthroughs was a surefire method to safe them.
“One of many core parts of OpenAI is that all the things in analysis is backside up,” mentioned Lightman. “Once we confirmed the proof [for o1], the corporate was like, ‘This is sensible, let’s push on it.’”
Some former workers say that the startup’s mission to develop AGI was the important thing think about attaining breakthroughs round AI reasoning fashions. By specializing in growing the smartest-possible AI fashions, relatively than merchandise, OpenAI was in a position to prioritize o1 above different efforts. That sort of huge funding in concepts wasn’t at all times doable at competing AI labs.
The choice to strive new coaching strategies proved prescient. By late 2024, a number of main AI labs began seeing diminishing returns on fashions created by way of conventional pretraining scaling. At the moment, a lot of the AI subject’s momentum comes from advances in reasoning fashions.
What does it imply for an AI to “motive?”
In some ways, the objective of AI analysis is to recreate human intelligence with computer systems. Because the launch of o1, ChatGPT’s UX has been full of extra human-sounding options equivalent to “considering” and “reasoning.”
When requested whether or not OpenAI’s fashions had been really reasoning, El Kishky hedged, saying he thinks in regards to the idea by way of pc science.
“We’re educating the mannequin how you can effectively expend compute to get a solution. So if you happen to outline it that means, sure, it’s reasoning,” mentioned El Kishky.
Lightman takes the method of specializing in the mannequin’s outcomes and never as a lot on the means or their relation to human brains.
The OpenAI emblem on display at their developer day stage. (Credit score: Devin Coldeway)Picture Credit:Devin Coldewey
“If the mannequin is doing laborious issues, then it’s doing no matter obligatory approximation of reasoning it wants so as to do this,” mentioned Lightman. “We are able to name it reasoning, as a result of it seems like these reasoning traces, however it’s all only a proxy for attempting to make AI instruments which might be actually highly effective and helpful to lots of people.”
OpenAI’s researchers notice folks might disagree with their nomenclature or definitions of reasoning — and certainly, critics have emerged — however they argue it’s much less essential than the capabilities of their fashions. Different AI researchers are likely to agree.
Nathan Lambert, an AI researcher with the non-profit AI2, compares AI reasoning modes to airplanes in a weblog publish. Each, he says, are artifical methods impressed by nature — human reasoning and hen flight, respectively — however they function by way of totally completely different mechanisms. That doesn’t make them any much less helpful, or any much less able to attaining comparable outcomes.
A bunch of AI researchers from OpenAI, Anthropic, and Google DeepMind agreed in a current place paper that AI reasoning fashions should not properly understood right this moment, and extra analysis is required. It might be too early to confidently declare what precisely is occurring inside them.
The subsequent frontier: AI brokers for subjective duties
The AI brokers in the marketplace right this moment work finest for well-defined, verifiable domains equivalent to coding. OpenAI’s Codex agent goals to assist software program engineers offload easy coding duties. In the meantime, Anthropic’s fashions have turn into significantly in style in AI coding instruments like Cursor and Claude Code — these are a number of the first AI brokers that individuals are prepared to pay up for.
Nonetheless, basic function AI brokers like OpenAI’s ChatGPT Agent and Perplexity’s Comet battle with lots of the advanced, subjective duties folks wish to automate. When attempting to make use of these instruments for on-line buying or discovering a long-term parking spot, I’ve discovered the brokers take longer than I’d like and make foolish errors.
Brokers are, after all, early methods that may undoubtedly enhance. However researchers should first work out how you can higher prepare the underlying fashions to finish duties which might be extra subjective.
AI purposes (Picture by Jonathan Raa/NurPhoto by way of Getty Photographs)
“Like many issues in machine studying, it’s a knowledge drawback,” mentioned Lightman, when requested in regards to the limitations of brokers on subjective duties. “Among the analysis I’m actually enthusiastic about proper now is determining how you can prepare on much less verifiable duties. We now have some leads on how you can do these items.”
Noam Brown, an OpenAI researcher who helped create the IMO mannequin and o1, informed TechCrunch that OpenAI has new general-purpose RL strategies which permit them to show AI fashions expertise that aren’t simply verified. This was how the corporate constructed the mannequin which achieved a gold medal at IMO, he mentioned.
OpenAI’s IMO mannequin was a more moderen AI system that spawns a number of brokers, which then concurrently discover a number of concepts, after which select the very best reply. A majority of these AI fashions are gaining popularity; Google and xAI have lately launched state-of-the-art fashions utilizing this system.
“I believe these fashions will turn into extra succesful at math, and I believe they’ll get extra succesful in different reasoning areas as properly,” mentioned Brown. “The progress has been extremely quick. I don’t see any motive to suppose it is going to decelerate.”
These strategies might assist OpenAI’s fashions turn into extra performant, positive factors that would present up within the firm’s upcoming GPT-5 mannequin. OpenAI hopes to claim its dominance over opponents with the launch of GPT-5, ideally providing one of the best AI mannequin to energy brokers for builders and customers.
However the firm additionally desires to make its merchandise less complicated to make use of. El Kishky says OpenAI desires to develop AI brokers that intuitively perceive what customers need, with out requiring them to pick particular settings. He says OpenAI goals to construct AI methods that perceive when to name up sure instruments, and the way lengthy to motive for.
These concepts paint an image of an final model of ChatGPT: an agent that may do something on the web for you, and perceive the way you need it to be achieved. That’s a a lot completely different product than what ChatGPT is right this moment, however the firm’s analysis is squarely headed on this route.
Whereas OpenAI undoubtedly led the AI {industry} a number of years in the past, the corporate now faces a tranche of worthy opponents. The query is not simply whether or not OpenAI can ship its agentic future, however can the corporate achieve this earlier than Google, Anthropic, xAI, or Meta beat them to it?
