"According to data collected from 23,000 clusters across thousands of companies using Cast AI's AI agent, average GPU utilization across enterprise servers is at 5%, meaning roughly 95% of provisioned GPU capacity is not being used."
"Unlike traditional cloud computing, where resources can be spun up and down on demand, companies often commit to long-term GPU contracts due to limited supply, leading to overbuying driven by the fear of missing out."
"The act of buying has no correlation with whether you need them or not. You don't buy them because you need them. You buy them because they were available."
New data reveals that organizations are significantly overprovisioning GPU capacity, with an average utilization rate of just 5%. This means that 95% of GPU resources remain idle. CPU utilization is also low, at 8%. The report indicates that while idle CPUs waste minimal costs, unused GPUs can incur substantial expenses. Companies are driven to secure AI chips due to high demand, leading to long-term contracts and overbuying. A healthy GPU utilization rate should be around 50%, prompting CTOs to reassess their current GPU usage.
Read at www.businessinsider.com
Unable to calculate read time
Collection
[
|
...
]