Designing a data center is a multifaceted challenge. It requires meticulous planning for power provision, cooling systems, robust security, consistent reliability, and lightning-fast network connectivity, all of which must harmoniously integrate. However, the rapid adoption of AI and machine learning models is reshaping data center design, driving unprecedented demand for power, cooling, and high-speed networking. AI workloads are far more compute-intensive than traditional applications, requiring entirely new infrastructure solutions to scale efficiently.
AI models, like those used in medical and climate research or large language modelling or financial services, demand relentless computational power. Unlike some traditional data center workloads that experience peaks and troughs, AI model training is continuous. And the current trend of the latest AI algorithms is increasing the computational demands of AI enabled data centers.
Most conventional data centers are not equipped to deal with the enormous compute required to train the neural networks required of AI technology. Specialized infrastructure requirements to meet the updated demands of AI include:
Let’s take a look at each in more detail.
First and foremost is the need for higher density racks. Whereas the average data center rack density a few years ago was 5 kilowatts (kW) per rack, the latest generation of AI supercomputers, like the NVIDIA DGX H100, support up to 40-60 kW per rack.
Just four of these systems in one rack could consume more than 40 kW while only occupying 60% of the space of a typical computing rack. While housing more computational power within a smaller space offers substantial cost-efficiency, it does present some unique challenges.
Traditionally cooled data centers have resorted to widely spaced racks to alleviate heat issues. But at-scale machine learning applications require server racks placed close together – as this optimizes the network configuration that is made up of very expensive high-throughput cables between servers, while also minimizing the overall cost of deployment.
This rising rack density in data centers is putting strains on conventional air cooling technologies. As a result, liquid cooling is gaining traction in AI data centers. It also places new emphasis on sustainable data center solutions, with providers who invest in sustainable practices offering future-proofed services to their customers.
With faster and hotter servers, data centers must also adapt their network architecture and connectivity. Low-latency east-west spine and leaf networks are essential to support production traffic and machine learning processes. Additionally, investments in cache-capable systems like CXL and NVLink are necessary to ensure efficient data transfer between CPUs, GPUs, and DPUs without causing delays.
AI-optimized data centers must also be structurally robust. These data centers need to support the transportation and installation of exceptionally heavy AI computing cabinets, some weighing over 1.5 tonnes or more when fully configured. This structural integrity is essential for the efficient operation of AI infrastructure.
As AI models deal with large and complex datasets, one of the questions being raised is how companies will look for ways to protect their proprietary data. Instead of training public models, companies will leverage proprietary engines that not only protect access to their data sources, but could help them gain a competitive edge in the marketplace. This takes data center infrastructure to a whole new level requiring even greater connectivity, agility, and scalability.
At Verne, we understand the unique challenges and demands that AI models place on data centers. Our data centers are designed to reliably host, support and scale some of the most demanding AI models in the world, with facilities built to handle the high-density and high-performance needs of AI workloads.
As AI continues to reshape industries, we are staying one step ahead to meet these demands, adopting optimized designs that prioritize power, data center rack density, cooling, strength, and security. We know what powering progress looks like as we’ve been delivering high intensity compute for more than a decade.
Our AI-ready data centers offer high-density computing, sustainable power, and advanced cooling solutions to meet the demands of next-generation AI models.
Get in touch today to discover how our AI data center solutions can help your business.