© Vladimir Timofeev | Dreamstime.com
Data Center

With Expanding Data Center Infrastructure, Efficiency and Reliability Move to the Forefront

Nov. 7, 2022
A DCI expansion is a big task to take on. Make sure you're paying attention to the right things.

Whether you are streaming the latest hit series, catching up with loved ones over social media or updating a spreadsheet on your corporate network from your home office, nearly every aspect of our digital lives today involves accessing information stored and processed at a data center located somewhere else in the world.

As the foundation of the digital economy, data center infrastructure (DCI) requires constant infrastructural expansions and upgrades to increase efficiency and reduce costs. In fact, any downtime to data center operations can be costly not just for the operator, but for the digital economy as well.

The Costs of Running Our Digital Economy

All data centers manage their power usage and effectiveness (PUE) and keep their facilities and servers cool at the same time. Consequently, power and cooling systems account for the majority of building costs and a significant part of the operational costs for any data center. As a result, choosing the most suitable power and cooling options are top priorities, whether investing in a new data center or expanding the infrastructure of a current data center.

For a large-scale colocation data center, the average building cost for a new facility is often hundreds of millions of U.S. dollars, with the largest part by far being spent on power and cooling systems. When considering return on investment, every dollar saved by improving the efficiency and reliability of power and cooling systems will translate into a sizeable reduction of the overall cost of the project.

As data center operators must constantly be ready for data center expansion to cope with hard-to-predict and fluctuating customer demand, it is common for many operators to install power, cooling, and monitoring equipment prior to adding modules of servers and other networking devices.

The average annual cost to run an existing large-scale data center nearly always runs into the tens of millions of US dollars, of which typically just under 50% is attributed to power and cooling. When considering operational costs and sustainability, DCI spending on optimization to lower power consumption would help reduce costs and carbon footprints considerably in the long run. This growth driver is even more pronounced in North America and Europe, where sizable DCI growth is attributed to refurbishing legacy equipment and promoting sustainability.

Two Key Issues Affecting DCI Expansion

Large-scale data center infrastructure is supported by integral and interconnected systems that comprise many devices, such as switchgears, power meters, controllers, and environmental sensors. These operational technology (OT) systems, such as critical power supply, cooling, and building automation contribute to stable and efficient operations.

To that end, different hardware belonging to different systems must work together to ensure the best performance. Consequently, decision-makers are often faced with two challenges when it comes to connectivity:

  • Minimizing downtime
    • Adding equipment to existing networks may require installation, testing, rebooting, or reconfiguration, all of which could affect part of or even the entire network and cause downtime. Downtime is costly for data centers, especially for Tier IV data centers that require 99.995% availability and less than 25 minutes of downtime per year. Choosing the right equipment and installation project plan often turns out to be more complex, task-intensive, and time-consuming than originally anticipated. 

  • Connecting systems with disparate communication protocols 

    • Different systems use different communication protocols. For instance, power meters use the RS-422/485 interface and conventional serial protocols, while cooling or HVAC systems use different OT protocols, such as BACnet. As such, connecting these disparate systems to enable effective monitoring and control through the Data Center Infrastructure Management (DCIM) system will require a lengthier and more laborious system integration process, which may also include system reconfiguration, testing, and troubleshooting.

Minimizing Disruptions to Existing Data Center Operations

To address the two key issues mentioned above, and mitigate any adverse impacts, data center operators should consider three factors, namely flexibility, redundancy, and interoperability, when they evaluate any solution for their DCI expansion project.

  • Flexibility
    • A modular approach, such as the Micro Modular Data Center (MMDC), contributes to operational flexibility and system resilience. Operators can take advantage of modular switches to add as many, or as few, expansion modules as needed. Modular design offers the most flexibility, which allows operators to build capacity by adding application-specific and mission-critical modules, which include sensors and controls to monitor and manage cooling, uninterruptible power supply, and overall energy consumption.
  • Redundancy
    • Building suitable redundancy into the DCI helps minimize system downtime, especially during system expansion or optimization. For example, a redundant network temporarily allows the regular network path to be circumvented when new systems are being integrated. As such, the entire system remains operational and available through the backup network.

  • Interoperability
    • When it comes to expansions or upgrades, legacy devices might not be compatible with the new equipment or setup. As replacing legacy devices might be impractical, a workaround must be sought to allow the DCIM system to monitor and control all critical equipment, including legacy devices. Using an industrial protocol converter between systems is a cost-effective solution. For example, serving as a “translator” between two communication protocols, a BACnet-to-Modbus converter helps DCIM systems communicate with both electrical power management systems and cooling systems.

DCI expansion is akin to refueling a commercial jet while it is still in flight. Just as the plane cannot stop moving, the data center cannot stop running. For this reason, a small mistake may cost the operator substantial damages and severely impact the business.

As power supply and cooling constitute the lion’s share of the overall cost of DCI construction, expansion, and operation, a suitable equipment procurement and well-designed project plan that takes flexibility, redundancy, and consistency into consideration from the start could help avoid obstacles from arising further down the road.

Johnny Fang is a Project Manager at Moxa, Inc.