The TDP Escalation Is Not Slowing Down
For most of the past decade, GPU power consumption was a consideration primarily for enthusiast builders choosing between 550W and 750W consumer PSUs. That framing is now obsolete. The RTX 5090 carries a rated TDP of 575W [1][4][6] — higher than the entire system power budget of a typical office workstation — and that figure sits at the low end of the professional AI accelerator spectrum.
Moving up the stack, the H100 SXM5 operates at ~700W TDP [1], and NVIDIA's GB200 Superchip — a dual-die module combining two Blackwell GPUs — pushes the per-module thermal envelope well past 1,000W. At the rack level, high-density GB200 NVL configurations aggregate these loads into multi-hundred-kilowatt deployments that bear no resemblance to the server infrastructure economics of five years ago.
The trend line is unambiguous. Consumer flagship TDPs have roughly doubled over two GPU generations; datacenter accelerator TDPs have grown faster still. The H100 was already ~2x the power draw of the A100 it replaced. The GB200 Superchip's estimated manufacturing cost of ~$14,200 — including ~$6,500 in HBM3e and ~$2,200 in advanced packaging — reflects a part that is physically large, thermally dense, and power-hungry by design. Higher transistor counts, larger die areas (the B200 logic die runs ~1,600mm²), and more HBM stacks all contribute to load growth that the power delivery stack must absorb.
For procurement teams, the practical implication is straightforward: the 750W PSU that was "headroom" for an RTX 3090 workstation [5] is now a floor, not a ceiling, for any serious professional or near-datacenter AI deployment.
Datacenter accelerators now draw 4–10x the power of mid-range workstation GPUs — collapsing the infrastructure assumptions built for previous-generation deployments.
Source: SLYD GPU Power Calculator [1]; NVIDIA product specifications
Power Infrastructure as a Capex Variable: Quantifying the Stack
When a procurement team prices an AI accelerator cluster, the GPU hardware — expensive as it is — represents only one layer of the true deployment cost. The power delivery and thermal stack adds meaningful capex at every level:
PSU and rack power delivery. For a server containing eight H100 SXM5 GPUs at ~700W each, GPU-side draw alone reaches ~5.6kW. Adding CPU, memory, NVMe, and networking brings a realistic server-level draw to 7kW or above. Redundant 80 PLUS Titanium or equivalent PSUs rated at 3kW–6kW per server unit, busbars, and PDU infrastructure all scale with that load. Infineon's recently published 12kW reference design for AI datacenter PSUs — leveraging silicon carbide (SiC) and gallium nitride (GaN) switching stages alongside silicon — is a direct industry response to this demand profile [2][3]. The migration to three-phase rack PSU architectures is accelerating for the same reason [3].
Cooling infrastructure. Air cooling is thermally and economically viable up to roughly 20–30kW per rack under conventional designs. Above that threshold, direct liquid cooling (DLC) — where coolant flows through cold plates attached directly to accelerator modules — becomes the dominant approach. At H100 and B200 rack densities, DLC is standard, not optional. Full immersion cooling, while still a minority deployment, is gaining ground at hyperscale facilities where rack densities approach or exceed 100kW.
Data center cooling cost is not a flat percentage of facility opex — it varies enormously by architecture and climate. A PUE of 1.3–1.5 is a reasonable planning range for modern AI-optimized facilities; older facilities with legacy air-cooling infrastructure may run PUE above 1.6, materially increasing effective electricity cost per useful compute unit.
Electricity operating cost. Using a simplified but illustrative model: a cluster of ten servers, each housing eight H100 SXM5 GPUs and drawing ~7kW–8kW at the server level including overhead, running at sustained high utilization across all ten servers would consume on the order of 600,000–700,000 kWh per year before PUE adjustment [1]. At a blended commercial electricity rate (which varies substantially by geography and contract structure), annual electricity cost can represent a meaningful fraction of the annualized hardware cost. At enterprise scale — hundreds of servers — the electricity line item is not a rounding error; it directly competes with hardware depreciation for budget share.
| Infrastructure Layer | Typical Scale Point | Key Consideration |
|---|---|---|
| Per-GPU TDP | 72W (L4) → 700W (H100) → ~1,000W+ (GB200) | Determines PSU and cooling architecture floor |
| Server-level PSU capacity | 3kW–12kW per server | Must include headroom for transient peaks; 80+ Titanium efficiency standard |
| Rack-level power | 20kW–200kW+ for AI-dense configs | Drives three-phase AC and busbar decisions |
| Facility PUE | 1.2 (best-in-class) – 1.6+ (legacy air) | Multiplies every kWh of accelerator draw |
| Cooling architecture | Air (≤20–30kW/rack), DLC (30–100kW), Immersion (100kW+) | Capex and opex trade-offs differ substantially |
The PSU Supply Chain Is Upgrading — and That Has Lead-Time Implications
The component-level transition inside AI datacenter PSUs deserves specific attention from procurement and supply-chain teams. Traditional server PSUs were built around silicon MOSFETs and IGBTs. The new generation of high-density AI PSUs — exemplified by Infineon's 12kW reference design and related product introductions in mid-2026 — integrates GaN-based switching stages for high-frequency conversion efficiency and SiC devices for high-voltage stages [2][3]. Current sensing is migrating to isolated magnetic Hall-effect and coil sensor architectures (such as the XENSIV TLE4978) for higher accuracy at elevated currents [2].
This is not a cosmetic change. GaN and SiC power semiconductors are produced on different wafer substrates and at different foundries than standard silicon logic. GaN-on-SiC and GaN-on-silicon wafers are typically 150mm or 200mm, not 300mm, and supply is concentrated among a smaller number of specialized fabs. Lead times for GaN and SiC power devices are measured in weeks to months and are sensitive to capacity allocation decisions that are largely independent of the logic foundry capacity dynamics tracked in mainstream semiconductor coverage.
For AI infrastructure buyers, this means that PSU availability constraints are not simply a function of assembler capacity — they can originate at the compound semiconductor wafer level, several tiers back in the supply chain. Teams building out AI infrastructure on compressed timelines should treat PSU procurement with the same lead-time discipline applied to GPU allocation, not as a commodity component with spot availability.
See our prior analysis on foundry allocation and capacity constraints for a broader picture of where bottlenecks are forming across the semiconductor supply chain in 2026.
The "Modded RTX" Vector: Edge Deployments and the Power Risk Profile
A separate but operationally relevant category is the deployment of consumer or lightly modified gaming GPUs — including RTX 5090-class cards — in near-production AI inference roles. This practice has expanded as the AI inference market has grown and as the cost differential between consumer and datacenter GPUs has remained wide. (For a detailed treatment of RTX 5090 market pricing dynamics, see our analysis of discrete GPU market restructuring.)
The power risk profile of these deployments is distinctive. Consumer RTX cards are designed around ATX power delivery, the 16-pin 12V-2x6 connector standard (capable of up to 600W per connector in specification), and consumer-grade PSUs typically rated to 850W–1,200W for high-end configurations [5]. When these cards are operated at sustained AI inference loads — which can be less bursty than gaming but more continuous — two risks emerge:
First, thermal design assumptions differ. Consumer GPU coolers are optimized for gaming workload profiles with typical duty cycles below 100%. Sustained inference workloads at or near TDP can exceed those thermal assumptions, particularly in poorly ventilated chassis.
Second, PSU derating at sustained load is real. A PSU rated at 1,000W peak may deliver 850W continuously at rated efficiency before thermal derating begins. Deployments stacking multiple high-TDP consumer GPUs in a single system — a common edge inference configuration — need to account for the sum of continuous draws, not peak ratings.
For teams evaluating edge AI inference builds using consumer GPU hardware, the GPU Rental Premium Decomposition analysis provides a useful framework for comparing build-versus-rent economics once power and cooling overhead is properly loaded into the cost model.
Total Cost of Ownership: Integrating Power Infrastructure Capex From Day One
The central analytical point of this piece is simple but frequently underweighted in practice: GPU power consumption and the infrastructure it requires are not footnotes to the AI accelerator investment decision — they are part of the core cost structure.
A rough-order framework for AI cluster TCO should include:
- Hardware cost (GPU, server, networking)
- PSU and power delivery infrastructure capex (including UPS, PDU, busbar, and distribution at rack and row level)
- Cooling infrastructure capex (air handlers, DLC loops, or immersion tanks depending on density)
- Annual electricity opex, adjusted for PUE and local tariff
- Facility power capacity upgrade cost, where applicable (transformer, switchgear, utility connection)
At a 10-server, 80-GPU H100 scale, the non-GPU infrastructure line items can plausibly represent 30–60% of total hardware capex depending on whether a greenfield or brownfield facility is involved and what cooling architecture is required. That ratio grows as GPU density increases, because the physics of heat removal are more expensive to solve at higher densities than the GPUs themselves.
The Chip Cost Calculator can model accelerator-level unit economics. For facility-level power and cooling cost modeling, procurement teams should ensure their infrastructure planning tools carry the same level of rigor applied to silicon procurement.
References & Sources
[1] SLYD GPU Power and Cooling Calculator — GPU TDP reference figures and power/cooling estimation methodology. Accessed 2026.
[2] Semiconductor Today — "Infineon adds two new high-efficiency server power solutions for AI data-center PSUs," 29 June 2026.
[3] Infineon Technologies — AI PSU product and reference design documentation, including 12kW PSU reference design announcement. Accessed 2026.
[4] Various financial press coverage — NVIDIA RTX 5090 pricing and market reception, including GPU power consumption reporting. 2025–2026.
[5] Corsair — Recommended PSU Table / GPU Power Requirements specification reference. Accessed 2026.
[6] Independent hardware review coverage — "NVIDIA RTX 5090 Breakdown: Cooling, Power, AI Performance & More." Accessed 2026.