Jalapeño Shows Power of LLMs for Chip Design - IEEE SpectrumIEEE.orgIEEE XploreIEEE StandardsIEEE Job SiteMore SitesSign InJoin IEEEHow OpenAI Used Its Own LLMs to Design Its Jalapeño ChipShareFOR THE TECHNOLOGY INSIDERSearch: Explore by topicAerospaceAIBiomedicalClimate TechComputingConsumer ElectronicsEnergyHistory of TechnologyRoboticsSemiconductorsTelecommunicationsTransportation
IEEE Spectrum
FOR THE TECHNOLOGY INSIDERTopicsAerospaceAIBiomedicalClimate TechComputingConsumer ElectronicsEnergyHistory of TechnologyRoboticsSemiconductorsTelecommunicationsTransportationSectionsFeaturesNewsOpinionCareersDIYEngineering ResourcesMoreNewslettersSpecial ReportsCollectionsExplainersTop Programming LanguagesRobots Guide ↗IEEE Job Site ↗For IEEE MembersCurrent IssueMagazine ArchiveThe InstituteThe Institute ArchiveFor IEEE MembersCurrent IssueMagazine ArchiveThe InstituteThe Institute ArchiveIEEE SpectrumAbout UsContact UsReprints & Permissions ↗Advertising ↗Follow IEEE SpectrumSupport IEEE SpectrumIEEE Spectrum is the flagship publication of the IEEE — the world’s largest professional organization devoted to engineering and applied sciences. Our articles, videos, and infographics inform our readers about developments in technology, engineering, and science.SubscribeAbout IEEEContact & SupportAccessibilityNondiscrimination PolicyTermsIEEE Privacy PolicyCookie PreferencesAd Privacy Options© Copyright 2026 IEEE — All rights reserved. A public charity, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
Enjoy more free content and benefits by creating an account

Saving articles to read later requires an IEEE Spectrum account

The Institute content is only available for members

Downloading full PDF issues is exclusive for IEEE Members

Downloading this e-book is exclusive for IEEE Members

Access to
Spectrum
's Digital Edition is exclusive for IEEE Members

Following topics is a feature exclusive for IEEE Members

Adding your response to an article requires an IEEE Spectrum account

Create an account to access more content and features on
IEEE Spectrum
, including the ability to save articles to read later, download Spectrum Collections, and participate in
conversations with readers and editors. For more exclusive content and features, consider
Joining IEEE
.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to
all of Spectrum’s articles, archives, PDF downloads, and other benefits.
Learn more about IEEE →
Join the world’s largest professional organization devoted to engineering and applied sciences and get access to
this e-book plus all of
IEEE Spectrum’s
articles, archives, PDF downloads, and other benefits.
Learn more about IEEE →CREATE AN ACCOUNTSIGN INJOIN IEEESIGN INCloseAccess Thousands of Articles — Completely Free
Create an account and get exclusive content and features:
Save articles, download collections,
and
post comments
— all free! For full access and benefits,

subscribe

to Spectrum.
CREATE AN ACCOUNTSIGN INSemiconductorsAIComputingNews
How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip
AI drastically shortened its design time; it will only get fasterMatthew S. Smith14 Sep 20266 min readMatthew S. Smith is a contributing editor for IEEE Spectrum and the former lead reviews editor at Digital Trends.OpenAI’s Jalapeño pairs its compute die with six stacks of HBM4 and an I/O chiplet.
OpenAI
On 25 August, OpenAI fully unveiled Jalapeño, the company’s debut AI accelerator chip. Jalapeño delivers up to 13.4 petaflops of 4-bit compute and accesses 232 gigabytes of the most advanced memory available, linking to it at a blazing 15.4 terabytes per second. Benchmarks cited by OpenAI show that Jalapeño can reduce end-to-end latency (the time between prompt to last token) by up to 3.6 times when compared to Nvidia’s GB300—a chip the company currently relies on—and do so while consuming less power.Whether these figures translate into real-world gains once Jalapeño enters widespread service in OpenAI’s inference fleet remains to be seen, but performance is only half the story. The other half is how the chip was designed—a process which, as you might expect, was accelerated by OpenAI’s large language models (LLMs). Jalapeño moved from first architecture concept to first silicon in under 20 months. Only nine months separated the first RTL—the register-transfer level code defining the chip’s logic—from tape-out, when the finished design goes to manufacturing. That’s a rapid timeline, yet experts believe it could soon look slow as LLMs improve and become more deeply integrated into chip design tools. OpenAI, unsurprisingly, is bullish about the opportunities. “The models are giving superpowers to our engineers,” says Richard Ho, vice president of hardware at OpenAI. “Our engineers are still driving the work. They’re still the final arbiter of what’s going on. But they can do things a lot faster. They can explore a lot more paths.”OpenAI achieved fast results with a small design teamHo says the group that designed Jalapeño averaged fewer than 100 people over the course of the project and continues to stand at roughly 100 today as the team pursues second and third-generation designs. That number includes a broad swath of roles across the hardware team, from system design to software and supply chain, but not those at Broadcom, which partnered with OpenAI on the project.The division of labor between OpenAI and Broadcom was generally split between design and implementation. OpenAI’s team was responsible for end-to-end system design including the inference accelerator, the memory hierarchy, and networking. Broadcom handled “physical design from the gates onward,” Ho says.The partnership with Broadcom dampened some opinions on OpenAI’s speed. David Chin, co-founder at agentic chip design startup Verkor.io, says “the schedule they gave us is quite credible,” but believes that Broadcom’s help was essential to Jalapeño’s rapid timeline. “If you have somebody else start from scratch, it won’t be possible,” he says. Ravi Krishna, also a co-founder at Verkor, called OpenAI’s speed “a relatively impressive result,” but added that he expects that improvements in the capabilities of LLMs could result in even quicker timelines if the project started today.Andrew Kahng, distinguished professor at the University of California, San Diego, also found OpenAI’s speed notable, saying it’s “likely best in class today.” Kahng recalls a 2016 IEEE Design Automation Futures workshop, which he co-organized. The workshop included Richard Ho, at the time an engineer at Google, as a keynote speaker. Ho had strong opinions on design automation and framed the time required to complete a chip’s design as a function of the number of iterations a team could complete in a day. How OpenAI’s LLMs accelerated Jalapeño’s design“Automation itself has existed in chip design for many decades. It’s not a new problem,” says Ankur Srivastava, director of semiconductor initiative and innovation at the University of Maryland, in College Park. Where LLMs differ from prior automation tools, however, is their ability to understand language and code. He says this makes them particularly suited for chip design tasks that “are still in the linguistic domain of the problem.”The team at OpenAI designed a workflow that takes advantage of this strength. OpenAI’s front-end workflow was built around Accelerated Hardware Synthesis (XLS), an open-source high-level synthesis chain of tools originally developed at Google. High-level synthesis is a form of chip design automation that allows engineers to design a chip in a more familiar programming environment. In the case of XLS, chip designers can write in languages such as DSLX (a domain-specific language inspired by Rust) and C++. XLS then converts these to Verilog, a hardware description language used to describe electronic systems.“We were thinking about how to leverage AI to make the project faster, and the AI was much better at software-looking things,” says Chris Leary, member of technical staff at OpenAI. “XLS in some ways looks like software, so it got that benefit.” It helped, too, that Leary was extremely familiar with how XLS should function, as he started it during his time at Google.Kahng agrees that the decision to use AI to accelerate high-level synthesis, such as XLS, makes sense, as it’s “more natural for the LLM to work with” and provides the opportunity for fast iteration. “I see this as a generally useful workflow, and it’s one that ‘has legs’ going into the future,” he says.The same logic led the Jalapeño team to focus on software optimization. When the first chips came back from the foundry in May, the team pointed its internal AI models at designing software to run benchmarks such as SemiAnalysis’s InferenceX. On DeepSeek’s multi-head latent attention kernel benchmark, performance climbed from 0.31 percent of the theoretical ceiling (set by the chip’s compute and memory bandwidth) to 88.94 percent in roughly 40 hours. Ho says this result is repeatable, so the time between when foundries deliver the first chips and when production ramps up can be reduced. “All our schedule assumptions are going to be based on the fact we have this capability now,” he says. Jalapeño is designed for deployment in pods that include 2,048 chips.OpenAIWhile the broad strokes of the Jalapeño teams’ AI-assisted workflow were guessed by Ho and Leary up front, improvements in OpenAI’s models did offer a few surprises. Leary says that the project began with assistance from models like OpenAI’s o3, which was released to the public in April of 2025 (but available to the Jalapeño team earlier). By the time the project had wrapped up, however, the team had access to models that were precursors to GPT-6 Astra, which wasn’t publicly released until 3 September 2026. The newer model can work directly in Verilog without needing XLS’s translation from ordinary programming languages, and it’s close to being able to operate proprietary design tools on its own, Leary says.Ho also confirmed that the team had access to internal LLMs fine-tuned for chip design that are not available to the public. He declined to detail the models used. However, he added that the Jalapeño team partnered with OpenAI’s research team. While not all specific models used to design Jalapeño are publicly available, Ho says the goal is to bring lessons learned from the project into the company’s commercial LLMs. “It’s safe to say that Astra and following models will be very good at chip design,” he says.AI was less useful for backend optimization, but that could changeAs mentioned, the bulk of OpenAI’s work on Jalapeño focused on the “front end” of chip design, which spans the tasks that take a chip from initial concept, through writing RTL code to define the design, and through verification that the design will work when physically implemented. Much of the “backend” design—which includes tasks like routing interconnects, completing and verifying the clock and power specifications, and sending the required design information to the foundry—was handed off to Broadcom, which carried the chip through production.That’s not to say OpenAI’s workflow ignored the backend, though. The Jalapeño team includes physical design engineers who work with their counterparts at Broadcom to provide guidance on the chip’s floor plan and routing, among other things.At IEEE Hot Chips 2026, Ho and Leary put numbers on the gains from AI-guided physical design optimization, including an area reduction of 10 percent for the matrix multiplication units as measured against an optimized human baseline. In other words, OpenAI claims AI-guided optimization helped design more circuits into the same area of silicon than would have been possible before.Broadcom used its own internal workflow. The company’s team did not have access to the internal models OpenAI used to help design Jalapeño, but it did have access to OpenAI’s public, commercial models.Verkor’s Ravi Krishna says that OpenAI’s approach to backend design already feels a bit conservative. He believes that to be an artifact of when the project, which began in October of 2024, took place. “The models from the last four to five months have improved. From April [2026] onwards…is when they really started to be able to handle those tasks better,” he says. Verkor co-founder Suresh Krishna agreed, saying “there’s no reason you couldn’t have an agentic loop that largely accelerates the backend of the process as well.”Ho and Leary also hinted that the workflow used to design Jalapeño may look old-fashioned compared to the team’s next efforts. “As you can imagine with [Jalapeño], we were trying to go as fast as we could. So there’s a trade-off between ‘do we want to take time to do some innovation, or do we want to do things that we know work historically?’” Leary says. “With the second generation, we have a kind of reset opportunity to ask about all the things we want to get set up for.”Ho says the second-generation chip’s workflow has “a lot of places that we are introducing [AI].” He mentions opportunities to do more with AI in verification and physical design. Leary adds that the team now has tools for automatic waveform manipulation and viewing. This automates analysis to identify chip clock signals associated with failures and could improve debugging the hardware while it’s still being designed. Despite these expected improvements, Ho and Leary were clear that they don’t believe chip design can be fully automated. “We’re not saying that anyone can come and just build state-of-the-art, frontier AI/ML accelerator chips using just [OpenAI’s coding platform] Codex,” Ho explains. “We are saying some very specific things about how to be better at Codex and how we are focusing on a small team and fast timelines to reach quality results.” From Your Site ArticlesEnding an Ugly Chapter in Chip Design ›AI Agent Designs a RISC-V CPU Core From Scratch ›AI Alone Isn’t Ready for Chip Design ›Related Articles Around the WebChipMind: LLMs for Agile Chip Design ›[2311.00176] ChipNeMo: Domain-Adapted LLMs for Chip Design ›OpenAILLMschip designMatthew S. SmithMatthew S. Smith is a freelance consumer technology journalist with 17 years of experience and the former Lead Reviews Editor at Digital Trends. An IEEE Spectrum Contributing Editor, he covers consumer tech with a focus on display innovations, artificial intelligence, and augmented reality. A vintage computing enthusiast, Matthew covers retro computers and computer games on his YouTube channel, Computer Gaming Yesterday.The Conversation (1)Martin Maschmann17 Sep, 2026
M
GB300 istn't an IC, it's a rack0
Replies

Hide replies
Show More RepliesRoboticsNewsHumanoid Robots
Video Friday: Two Birotors Make a Quadrotor
5h4 min readRoboticsAIComputingHumanoid RobotsSponsored Article
Rethinking Robot Safety in the Age of AI
16 Sep 20264 min readComputingMagazineFeature
The AI Inference Revolution Is Here
15 Sep 202612 min read
Related Stories
SemiconductorsNews
Stacking Chips Sideways Gives AI More Memory
SemiconductorsAIMagazineFeatureJuly 2026Telecommunications
AI Is Designing Radio Chips That Humans Couldn’t Even Imagine
SemiconductorsAINewsComputing
AI Agent Designs a RISC-V CPU Core From Scratch

OpenAI utilized its large language models to significantly accelerate the design process for its debut AI accelerator chip, Jalapeño. Jalapeño integrates its compute die with six stacks of HBM4 memory and an I/O chiplet, achieving up to 13.4 petaflops of 4-bit computation and enabling data access at 15.4 terabytes per second. Benchmarks indicated that Jalapeño could reduce end-to-end latency by up to three point six times compared to the existing Nvidia GB300 while simultaneously consuming less power. The speed of this development involved moving from the initial architecture concept to physical silicon in under twenty months, with the register-transfer level code defining the chip logic being ready for tape-out in nine months.

The application of large language models was crucial in this rapid timeline, acting to provide substantial boosts to the engineering team by enabling them to explore a wider array of design possibilities more quickly. The process leveraged the inherent strength of LLMs in handling tasks within the linguistic domain of chip design. OpenAI established a front-end workflow centered around Accelerated Hardware Synthesis (XLS), an open-source high-level synthesis chain, which allowed engineers to write in languages like DSLX and C sharp, which the tools could then translate into hardware description languages like Verilog. This methodology benefited from the LLMs' proficiency in processing software-like structures.

Further acceleration was achieved by applying internal AI models to software optimization tasks. For instance, when optimizing multi-head latent attention kernels, internal models managed to increase performance substantially over benchmarks, demonstrating the capability to reduce optimization time considerably. This capability helped reduce the schedule assumptions between chip delivery from foundries and production ramp-up.

The collaboration involved a division of labor where OpenAI focused on end-to-end system design, including the accelerator and memory hierarchy, while Broadcom managed the physical design starting from the gates onward. While Broadcom provided essential support for the overall timeline, the specific models used internally by OpenAI included precursors to GPT-6 Astra and internal models fine-tuned for chip design that were not publicly released. This indicates a strategic effort to incorporate lessons learned into future commercial models.

While AI proved highly effective in the front-end tasks—from concept through RTL definition and verification—the bulk of the backend design, encompassing tasks like routing interconnects and power specification verification, was managed by Broadcom. However, the process was not entirely divorced from the backend; the team included physical design engineers who guided the floor plan and routing aspects. Despite this, observations suggest that the results from AI-guided physical design optimization, such as a ten percent area reduction for matrix multiplication units compared to human baselines, highlight the potential of this approach. Nonetheless, experts anticipate that future advancements in LLMs could lead to even quicker timelines, particularly as the focus shifts to automating physical design and verification, including waveform manipulation. The project underscores that while AI is a powerful accelerator, it does not eliminate the need for human expertise, asserting that full chip design remains a complex endeavor.