Investor Shelf

Powering AI: The Semiconductor Ecosystem at the Foundation of Data Centers (Semiconductor Industry Association)

Save
ReportSemiconductor Industry AssociationJune 202645 pages68 min readOriginal source
Key number
4,500 chips
Packaged chips in a single AI server rack (over) p.3
Largest segment
95%
Semiconductors' share of a leading AI server rack's content value (more than) p.3

Summary

  • A single AI server rack contains over 4,500 packaged chips, comprised of approximately 20,000 individual dies.p.3
  • Semiconductors account for more than 95% of a leading AI server rack's content value and more than 50% of the total capital expenditures for building and operating an AI data center.p.3
  • Government and industry will invest over $4 trillion in new data center infrastructure through 2028, of which up to $2.8 trillion will be spent on semiconductors.p.3
  • A leading data center can accommodate over 45,000,000 chips, assuming 10,000 AI compute racks.p.5
  • AI accelerators have a unit price of about $10,000 - $40,000, and CPUs about $7,500 - $15,000.p.11
  • AI Accelerators can handle 10-100x more calculations per second than CPUs.p.16

Key insights

  1. A typical AI compute board combines 1–2 AI Accelerators with 1–2 CPUs.p.12
  2. High-bandwidth memory stacks up to 16 dies each, with 6–12 stacks (96–192 dies) per integrated AI GPU.p.14
  3. A typical data center NVMe SSD includes 256 NAND flash dies.p.17
  4. The power tray is estimated at $50,000 - $290,000 per server rack.p.23
  5. The network and IPMI tray is estimated at $17,000 - $25,000 per server rack.p.25
  6. Active copper cable chips enable short-distance communication of up to 5 meters across trays or racks.p.22

Topics and entities

Summary and figures generated from the document and checked automatically: every number shown appears on the cited page. The full document is available from the publisher. Publishers can request removal.