The Software Layer Nobody Sees: How DPU Bring-Up Engineering Shapes Cloud Infrastructure Reliability

The data center has quietly grown a third kind of processor. For years the conversation was CPUs, and then GPUs, but the part doing the unglamorous work of moving data, isolating tenants, and accelerating security now increasingly runs on a separate chip: the data processing unit. The market for these units sat at roughly $2 billion in 2025 and is on track to pass more than $21 billion by 2034, a measure of how much infrastructure work cloud operators want pulled off the main processor. The architecture diagrams make this look clean. The reality is that a DPU only earns its place once its silicon, its firmware, and the operating system above it all agree on how the hardware behaves, and getting them to agree is where most of the schedule goes.

Joby Neelamthara Thoman is a Principal Software Engineer with more than 22 years spent designing high-performance networking and data center systems, including packet processing, DPU platforms, and high-speed Ethernet for major cloud and networking platforms. He is a reviewer for the 2026 IEEE International Conference on Computing, Power, and Communication Technologies. Much of his career has lived in the layer this article is about: the bring-up work where a new piece of silicon stops being a design and becomes a product, or fails to.

We spoke with Joby about DPU bring-up, why debug visibility and SerDes integration decide whether hardware ships on time, and what breaks when the layers between silicon and software stop lining up.

DPUs get described as the third pillar of the data center. What does the industry underestimate about building one?

The architecture is the part everyone wants to talk about, and it is the part that is mostly solved. You decide what to offload from the host, you size the engines, you pick the interfaces. That work is hard, but it is bounded. You can reason about it on a whiteboard. What people underestimate is everything that happens after the first chip comes back from the fab, when you have to make the silicon, the firmware, and the operating system behave as one system instead of three projects that happened to occur near each other.

A DPU exists to take networking, storage, and security off the main CPU so that expensive compute does compute instead of bookkeeping. For that to be worth doing, the offload has to be invisible to the workload above it. Invisible is a high bar. It means every path through the hardware has to be correct, observable, and repeatable before anyone upstream trusts it with production traffic. That last 20% of bring-up, the part that turns “it works on the bench” into “it ships,” is where months disappear.

You’ve spent much of your career on bring-up for high-speed networking silicon. What does that work actually involve day to day?

It starts at the bottom and works up. Optics and transceivers, the SerDes lanes underneath them, the MAC, the ASIC interfaces, and then the packet path through all of it. For a long stretch my job was taking capability that existed at the component level and turning it into platform behavior you could ship and support. A transceiver that links up once in a lab is a demo. A platform that links up the same way across thousands of units, every reboot, in a customer’s data center, is a product. The distance between those two things is the work.

It is cross-functional by nature. I sit between the hardware teams, the diagnostics people, and QA, because a bring-up problem rarely belongs to one of them cleanly. Half the job is figuring out whose layer a symptom actually lives in. The other half is building enough visibility into the system that the answer is not a guess. When a link flaps once every few hours, you do not have a theory problem. You have an instrumentation problem, and you solve it by being able to see what the hardware was doing in the moment it misbehaved.

SerDes and PHY integration keep getting blamed for blown schedules. Why is the physical layer so unforgiving?

Because it does not give you clean failures. A digital bug is often binary: the logic is right or it is wrong, and a test catches it. A high-speed link fails in shades. The signal is a little degraded, the eye margin is a little tight, the error rate is low enough to pass a short test and high enough to wreck a long one. You can tune a lane until it looks healthy and still ship a board that comes apart under temperature or aging. The physical layer punishes anyone who treats “mostly working” as working.

This is showing up in the industry numbers. Only 14% of new chip projects now reach first-silicon success, the lowest rate in 20 years. A respin costs months and a lot of money, and a large share of those misses trace back to integration and verification gaps rather than the core logic being wrong. The thing I keep relearning is that signal integrity is not a tuning step at the end. It is a design constraint you carry from the start, and the teams that treat it that way are the ones that hit their dates.

What’s a failure that taught you the most, where the layers genuinely did not line up?

The expensive ones look the same every time. Something intermittent shows up late, and every layer has a plausible reason it is not at fault. Firmware says the hardware is reporting a bad state. Hardware says the firmware is programming it wrong. The OS driver says it is just passing along what it was handed. Everyone is partly right, which is the worst case, because partly right means no single owner and a lot of meetings. I have watched one misaligned assumption between a firmware register definition and what the silicon actually did cost weeks, not because it was hard to fix but because it was hard to see.

What broke that pattern was visibility, every time. The moment you can capture what each layer believed was true at the instant of the failure, the argument ends and the engineering starts. Most teams underinvest here. They build debug hooks as an afterthought, treat them as scaffolding to throw away, and then fly blind during the exact phase where seeing clearly matters most. I would rather ship a week later with real observability built into the platform than save that week and spend a month guessing. That tradeoff is not close, and I think a lot of teams get it backwards.

The work you’re describing sits several layers below anything that gets marketed. Does that kind of deep infrastructure engineering get the recognition it deserves?

Not as much as it should, and I have made peace with that. Bring-up and integration are undervalued relative to how much they decide. The chip gets the headline and the application gets the headline. The work in between, the work that determines whether the chip ever reaches the application, is mostly invisible, including to the people funding it. Recognition tends to arrive only when the work is missing, when a program slips by six months and someone finally asks why.

The teams that stall tend to fall in love with the demo. A demo proves a thing can happen once. Production asks whether it happens every time, under load, after a year, when a component is slightly out of spec. Those are different questions, and the gap between them is exactly the bring-up and verification work people are tempted to cut when the schedule gets tight. Cutting it does not save time. It moves the time to a worse place, after customers are involved.

After more than 20 years across networking and data center silicon, where’s the industry’s blind spot right now?

More companies are designing their own chips than ever, which is the right move for a lot of them. Custom silicon buys differentiation and control. But many of those teams are doing first-time bring-up without the scar tissue that makes it survivable, and the result is the slipping schedules and respins you see across the field. The hard part was never drawing the architecture. It was the integration discipline that does not show up in a press release.

The blind spot is treating bring-up and observability as overhead instead of as the product. The software layer nobody sees, the diagnostics, the bring-up tooling, the verification harness, is the layer that decides whether the impressive chip in the slide deck ever ships at the volume and quality a cloud operator will accept. I would put senior engineers on that work, not junior ones, because the judgment about where a problem is hiding is the rarest skill on the team.

What are you focused on next, and what’s still unsolved?

Making bring-up less of an art and more of something repeatable. Right now too much of it lives in the heads of a few experienced people. If you can build the visibility and the tooling so that a competent team finds a cross-layer problem in days instead of months, you take the single biggest source of schedule risk out of hardware programs. That is the problem worth the next stretch of work, and it is far from solved.

The stakes are only getting larger. Hyperscalers are expected to account for half of $1.2 trillion in global data center capital spending by 2029, and a growing share of that is custom silicon that has to come up cleanly to earn its slot in the rack. The chips will keep getting more capable. Whether they ship on time will keep coming down to the layer underneath them, the one nobody sees, and that is the part I want to make boring.