Data Center Design Consideration: Electrical Rooms (1)

Switchboards, switchgear, transformers, generators, and UPSs require space for installation, maintenance, heat dissipation and future expansion (if possible). The wiring, busways, and raceways that distribute the electrical power must be accounted for now or in the future.

Electrical engineers should coordinate with mechanical engineers, architects, structural engineers, and others involved in the design of electrical rooms. Documentation and monitoring of electrical system’s equipment and how it connects to the rest of the facility must be accurately maintained.

We are going to explain the applicable code requirements and evaluate the design criteria for appropriate electrical-room size to accommodate present and future needs. Furthermore, we analyze the requirements for coordinating with structural, architectural, fire protection, and HVAC requirements.

   

Electrical rooms and mechanical, electrical, and plumbing (MEP) spaces are often an afterthought when it comes to building design and planning, either relegated to locations that are left over or deemed undesirable for other planning purposes. This shortsightedness can have unfortunate consequences on the cost, operations, and flexibility of the systems for the future.

NFPA 70: National Electrical Code (NEC) dictates the minimum amount of space needed around the equipment for access, operations, safety reasons, and conduit installation. Together, with the actual equipment sizes, this defines the overall minimum dimensional requirements of the room.





There are three types of general interior electrical spaces that factor into new building design: (1) main equipment rooms, (2) distribution pathways, and (3) local/branch equipment rooms. Code-required working space and dedicated space needs must be met. This article will outline important considerations for these spaces in the early stages of building design as they relate to building type, intended occupancy, size, and future expectations of both the building and the electrical systems.


Working Spaces


Let’s see the different between working and dedicated space as stated by the NEC (see picture below). The working space helps safeguard a clear working zone around all equipment and ensures protection for any workers or occupants within the room. This includes defining minimum width, depth, and height requirements for the working space, which varies due to voltage and the specific equipment. The higher the voltage of the equipment, the greater the depth of the working space. The width should be equal to the width of the equipment and no less than 30 in., while allowing for opening any doors or hinged panels to a full 90 deg. The height should be 6 ft 6 in. from the floor, or the height of the equipment if greater than 6 ft 6 in.

The style and construction type of the electrical equipment dictates whether only front access is required, or if rear and/or side access also is required. For each point of access to a piece of equipment, the minimum working clearances must be provided.

The dedicated space (depicted in red) and the working clearances (depicted in blue) are shown in a new emergency distribution room.


Dedicated Spaces


Dedicated space is a zone above the electrical equipment. It’s reserved to provide future access to the electrical equipment, protection of the electrical equipment from foreign systems, and for installing conduit/other raceways supporting incoming and outgoing circuits. The requirement for dedicated space applies primarily to switchgear, switchboards, panelboards, and motor control centers. The space should be equal in width and depth to the equipment size and extend from the floor to a height of 6 ft above the equipment (or to a structural ceiling, whichever is lower). No equipment or systems foreign to the electrical installation are allowed in this zone by the NEC.

The medium-voltage switchgear sections and unit substation transformers in a large data center installation require additional space and clearances.


The area above the dedicated space may contain foreign systems, provided proper protection prevents damage from drips, leaks, or breaks in these systems. However, it’s good practice to avoid having these systems installed in electrical rooms altogether.

While installations of equipment greater than 1,000 V generally follow the same principles, some of the specifics vary, requiring additional clearance around the equipment due to the increased hazard that these voltages impose (see picture above). Access to this equipment is preferably limited to only those deemed qualified to be there. For this reason, electrical equipment should be installed in rooms or spaces that are dedicated for that purpose and have controlled access.


Main Equipment Rooms


The main electrical room, or service entrance space, should coordinate with the local electrical utility. For example, main equipment rooms have requirements that dictate access to the space from the exterior for servicing, maintenance, and service feeder installation. The type of equipment installed will also further determine the room requirements. The service entrance room is typically located on an exterior wall for both code and practical reasons; it makes installation easier and minimizes the length of the service entrance conductors. Because the service conductors are usually the largest in the facility, this can have a substantial impact on cost.

Using arc-resistant switchgear will also impact space needs. This equipment will be taller and may have a larger footprint. Engineers will also need to account for the potential exhaust gases and arc flash energy by providing a pathway to expel them and relieve the pressure buildup from inside the switchgear.

If an exterior transformer is used to provide the service to a building, feeders from the transformer enter the building and transition to the main service entrance disconnect, typically a switchgear, switchboard, or panelboard. These feeders are often routed underground into the building through the exterior foundation wall via a coordinated opening. Additional coordination with the structural engineer is needed to avoid footings.

The elevation of the service entrance conduits many times do not naturally align with the equipment to which it is routed. Additional space in the form of increased height or footprint commonly is required to allow for the successful transition and termination of these conduits and conductors. Service installations that require medium-voltage equipment and/or transformers installed indoors will require additional elements including more space, higher fire ratings of the rooms (per NEC Article 450), and increased ventilation.

The location of any exterior equipment also needs to be coordinated with other architectural and landscaping elements. Minimum separation distances are often dictated by local codes/ordinances or utility requirements for proximity to screen walls, fencing, vegetation, paths of egress, or building fenestration.

Generator installations offer additional challenges when it comes to defining space needs. Noise, odor, and vibration factor into the location of this equipment within a building. The equipment should be located to minimize disturbances to building occupants and adjacent properties. Many jurisdictions have specific requirements on noise emissions, which will impact equipment placement and other components needed to meet requirements. Increasing the distance of this equipment from sensitive areas is one way of dealing with the concerns, but this comes with added feeder costs and may prove to be more costly than other options.

Sound attenuation and equipment required to meet specific emissions requirements, such as diesel oxidation catalyst, particulate filters, urea tanks, and selective catalytic reduction units, have significant cost implications and require a large amount of space to install.

Tier 4 versus Tier 2 compliance is usually dictated by an owner’s desire to use a generator for utility peak shaving or other non-critical proposes. It is crucial to have a clear understanding of current and future implications in both of these areas from the outset of a project and to discuss them thoroughly with the building owner.

The weight of a generator and the vibration experienced during its operation will have an impact on the building’s structural design. Generators require a lot of ventilation for cooling and combustion needs; getting air into and out of the room is critical and will impact placement.

With regard to fuel storage, most installations require a volume of fuel that dictates an external fuel tank with interconnecting fuel lines. NFPA (National Fire Protection Association) limits the overall capacity of diesel fuel inside buildings to 660 gal. The relationship of the exterior tank and the generator is also important to minimize pumping requirements and allow for gravity-drain return-fuel piping. This requires the fuel tank to be lower in elevation than the generator.

Direct access to the outside is preferable for maintenance and testing. All of this requires close coordination with the architectural, structural, and mechanical disciplines.

NFPA 110: Standard for Emergency and Standby Power Systems requires the emergency power supply for Level 1 installations to be installed in a separate room, separated from the rest of the building by 2-hour fire-rated construction. While NFPA 110 does allow the emergency power supply system equipment (EPSS; equipment consists of all components from the emergency power supply, or EPS, to the load terminals of the transfer switches) to be installed in the same room as the EPS, it is good practice to keep these separated to help enhance system resiliency. EPS rooms are also prone to additional dust, moisture, temperature fluctuations, and excessive noise during operation that limits the ability to have a conversation and may have a negative impact on other equipment if co-located.

For mission-critical facilities (e.g., financial institutions, data centers, and airports) and other highly sensitive installations, the use of a dry-type, pre-action, or another type of fire protection system that does not rely on a normally wet piping installation is highly recommended. In cold climates, this has an added advantage of preventing pipes from freezing, rupturing, and potentially flooding the EPS room.


Continue Reading: Part (2) - Data Center Design Consideration: Electrical Rooms


About us

Strategic Media Asia (SMA) is one of the approved CPD course providers of the Chartered Institution of Building Services Engineers (CIBSE) UK. The team exists to provide an interactive environment and opportunities for members of ICT industry and facilities' engineers to exchange professional views and experience.

SMA connects IT, Facilities and Design. For the Data Center Design Consideration, please visit 

(1) Site Selection,
(2) Space Planning,
(3) Cooling,
(4) Redundancy,
(5) Fire Suppression,
(6) Meet Me Rooms,
(7) UPS Selection,
(8) Raised Floor,
(9) Code & Standards,
(10) Transformers and Harmonic Distortion,
(11) Multi-mode UPS Systems, and 

(12) Electrical Rooms

All topics focus on key components and provide technical advice and recommendations for designing a data center and critical facilities.

Data Center Design Consideration: Transformers and Harmonic Distortion

Data center managers and information technology (IT) engineers in today's critical facilities are in search of reliable and energy-efficient equipment with low total cost of ownership. But after equipment investments are made, it's important to pay attention to possible threats to the operational efficiency.

One threat that is often overlooked is harmonic currents, which can have a significant impact on electrical distribution systems and the facilities they feed. Wasted power and temperature fluctuations caused by these currents can prevent facilities from achieving maximum efficiency, so it's more important than ever for data center managers to evaluate their facilities and to take the time to develop a strategy mitigating harmonic current.

Harmonic Distortion

Harmonics are distortions of the normal electrical current waveform, generally transmitted by nonlinear loads. Switch-mode power supplies (SMPS), variable speed motors and drives, photocopiers, personal computers, laser printers, fax machines, battery chargers, and UPSs are examples of nonlinear loads. Single-phase nonlinear loads are prevalent in modern office buildings, while 3-phase nonlinear loads are common in factories and industrial plants.

A large portion of the nonlinear electrical loads in most electrical distribution systems comes from SMPS equipment. For example, all computer systems use SMPS that convert utility ac voltage to regulated low-voltage dc for internal electronics. These nonlinear power supplies draw current in high-amplitude short pulses that create significant distortion in the electrical current and voltage wave shape (Figure 1). This harmonic distortion, measured as total harmonic distortion (THD), travels back into the power source and can affect other equipment connected to the same source.

All periodic waves can be generated with sine waves of various frequencies. The Fourier theorem breaks down a periodic wave into its component frequencies.

Harmonic currents generated by nonlinear loads increase power-system heat losses and power bills for end users. These harmonic-related losses reduce system efficiency, cause apparatus overheating, and increase power and air conditioning costs. As the number of harmonics-producing loads has increased over the years, it has become increasingly necessary to address harmonics when making any additions or changes to a facility. 

Most power systems can accommodate a certain level of harmonic currents but will experience problems when harmonics become a significant percentage of the overall load. As these higher frequency harmonic currents flow through the power system, they can cause communication errors, overheating, and hardware damage. 

Reducing Harmonics

To determine if harmonic mitigation is necessary, facilities managers should conduct an assessment to precisely measure the harmonics affecting the data center and identify their origin. Solutions for harmonic mitigation vary in complexity and cost and can be deployed individually or in combination. The strategy that makes the most sense for a facility will vary based on the loads it supports, its budget, and the nature of the harmonic-related problems it is experiencing.


Solution 1 - K-rated Transformers in Power Distribution Components


A standard transformer is not designed for high harmonic currents produced by nonlinear loads. It will overheat and fail prematurely when connected to these loads. Therefore, when harmonics were first introduced into electrical systems at levels that showed detrimental effects (circa 1980), the industry responded by developing the K-rated transformer. K-rated transformers are not used to eliminate harmonics, but to manage the heat generated by harmonic currents.

K factor ratings range between 1 and 50. A standard transformer designed for linear loads is designated with a K-factor of 1. The higher the K-factor, the more heat from harmonic currents the transformer is able to withstand. When selecting a K rating, managers should consider the trade-offs between size, efficiency, and heat tolerance. For example, transformers with higher K factors are typically larger than those with lower K factors. The table shows appropriate K ratings to use for different percentages of nonlinear current in the electrical system.

Power distribution units (PDUs) with a K 13-rated transformer are readily available to efficiently handle harmonic currents. Units with K 20 transformers are common, but are typically overkill for most modern data centers.




Units with K 20 transformers are common, but are typically overkill for most modern data centers. The K-rated, dry-type transformer is widely used in electrical environments - either in a PDU or as a stand-alone unit. However, recent advances in transformer design are changing the way facilities managers reduce voltage distortion and power loss caused by harmonic currents. 

Solution 2 - Harmonic Mitigating Transformer


A harmonic-mitigating transformer (HMT) is designed to handle the nonlinear loads of today's electrical infrastructures. This transformer uses electromagnetic mitigation to deal specifically with the triplen (3rd, 9th, 15th, and so on) harmonics. Secondary windings of the transformer are arranged to cancel zero sequence fluxes and eliminate primary winding circulating currents. This transformer also addresses the 5th and 7th harmonics by using phase shifting.

Using these two electromagnetic techniques, an HMT allows loads to operate as they were intended, while minimizing the energy loss and distortion caused by harmonics. Most HMTs exceed NEMA TP-1 efficiency standards, even when tested with 100% nonlinear loads. Wherever a K-rated transformer is specified, an equivalent HMT is a direct substitute.



Solution 3 - Harmonic Mitigating UPS


Much like an active filter, harmonic-mitigating UPS eliminates harmonic distortion by inserting equal and opposite current into the line. They also compensate for reactive power from low power-factor loads and balance loads across 3-phase systems to avoid stranded capacity, as well as to provide clean and continuous power during utility outages or in response to electrical disturbances.

Going Further


Data center managers are increasingly deploying UPSs with energy-saver operating modes to boost efficiency and lower power costs. Recently, harmonic-mitigating UPSs capable of keeping distortion within predetermined and adjustable limits, correcting power factor, and balancing loads while in energy-saver mode have begun to reach the market.

These new systems typically remain within 1% of energy-saver levels while performing these functions, a significant improvement over past technologies. The harmonic mitigation technology in the latest energy-saver UPSs is a built-in feature that requires no additional footprint, saving valuable data center floor space and reducing installation and maintenance costs.

Harmonics continue to be costly for data centers, preventing data center managers and engineers from achieving maximum reliability and efficiency. Fortunately, the latest enhancements in UPS technology offer next-generation harmonic-mitigation capabilities. Though not always required, such systems enable data centers to achieve the highest efficiency possible by actively correcting for harmonic currents as they occur.

About us

Strategic Media Asia (SMA) is one of the approved CPD course providers of the Chartered Institution of Building Services Engineers (CIBSE) UK. The team exists to provide an interactive environment and opportunities for members of ICT industry and facilities' engineers to exchange professional views and experience.

SMA connects IT, Facilities and Design. For the Data Center Design Consideration, please visit 

(1) Site Selection,
(2) Space Planning,
(3) Cooling,
(4) Redundancy,
(5) Fire Suppression,
(6) Meet Me Rooms,
(7) UPS Selection, and
(8) Raised Floor

All topics focus on key components and provide technical advice and recommendations for designing a data center and critical facilities.

Data Center Design Consideration: UPS

Uninterruptible Power Supply (UPS) is one of the key component of a data center. Understanding of the UPS technology is critical and important for the data center design.


Transformer-based or Transformer-free UPS?


There is growing interest in using transformer-free UPS modules in higher power, three-phase, mission-critical power backup applications (e.g., 200 kW to 5 MW). However, many organizations are unclear about which architecture — transformer-based or transformer-free — is best suited for a particular application.


With limited floor space and modular components in data centers, more flexible, smaller footprint UPS units are desirable by companies in the near future. On the one hand, the latest transformer-free systems offer better efficiency, a smaller footprint, and improved flexibility while providing high levels of availability. Driven by data center designer demand, most leading UPS suppliers offer both topologies. On the other hand, transformer-based UPS systems excel at providing the highest capacities and availability while simplifying external and internal voltage management and fault current control.





Currently, large transformer-free systems are constructed using modular building blocks that deliver high power in a lightweight, compact package. This modular design offers advantages when the timing of future load requirements is uncertain by allowing capacity to be more easily added as needed, either physically or via control settings. On the other hand, a modular design means higher component counts, which may result in lower unit mean time between failure (MTBF) and higher unit service rates.





For high-power enterprise data centers and other critical applications, a state-of-the-art transformer-based UPS still provides an edge in availability. Transformers within the UPS provide integrated fault management and galvanic isolation as well as greater compatibility with critical power distribution system requirements that should be considered when designing a high availability UPS system. Technology developments and configuration options allow the latest transformer-based designs to operate at higher efficiencies compared to previous designs, making them more comparable to the transformer-free models in terms of efficiency.


In general, 200 kW is a threshold below which the space, weight, and cost advantages of transformer-free UPS systems outweigh the robustness and higher capacity capabilities of transformer-based systems. The under-200 kW applications can benefit from the high efficiency and excellent input power conditioning through active components offered by transformer-free designs. In addition, the scalability of a modular transformer-free UPS can help avoid over-provisioning while maintaining operational efficiency.



FACTORS TO CONSIDER





Both approaches use a double-conversion process (Figure Above) to provide power protection for mission-critical applications. The primary difference between the two technologies is in their respective use of transformers.

A transformer-based UPS may use a transformer before the rectifier and requires an isolation transformer after the inverter to derive the voltage being delivered to the critical load.

Transformer-free UPS designs use power and control electronics technologies to eliminate the need for an isolation transformer as an integral part of the inverter output section.



TRANSFORMER-BASED UPS DESIGN

Large systems are typically manufactured based on serviceable sub-assemblies and are available in discrete units rated up to 1,100 kVA. Key components of this design include:

• A passive filter (inductors and capacitors) on the rectifier input to reduce input current distortion and improve the power factor.

• A six-pulse (or optional twelve-pulse), semiconductor (SCR)-based rectifier on the input. Optionally, an additional transformer (Xfmr) provides AC-DC isolation for the DC bus and the battery.

• A DC energy storage system (typically a battery) connected directly to the DC bus between the rectifier and the inverter to provide AC output power ride-thru capability during a loss of AC input power. This example uses 540 VDC.

• An insulated gate bipolar transistor- (IGBT) based, pulse-width modulation (PWM) inverter on the output.

• An isolation Xfmr on the inverter output to derive the appropriate output voltage. This also provides a convenient and solid point for referencing the AC output neutral to ground. This neutral ground connection provides excellent common mode noise rejection.

• A passive filter on the inverter output to provide a very low distortion AC voltage supply.

• An automatic bypass switch (static switch) using power SCRs provides instantaneous switchover to an alternate source if a UPS output disturbance occurs.

TRANSFORMER-FREE UPS DESIGN





Transformer-free UPS topologies replace simple passive magnetic-voltage transformation functions with solid-state power electronics circuitry. Figure above shows a simplified block diagram of a transformer-free UPS design. There are a number of key differences listed below between this circuit and the unit depicted in the figure.

• By replacing passive power components (transformers, capacitors, inductors) with power circuit assemblies utilizing PWM power conversion techniques, transformer-free UPS rectifiers are physically smaller and produce low input current harmonics with near unity input power factor.

• Typically, the UPS battery in transformer-free applications is connected to the internal DC bus (about 800 VDC in the previous example) through an integrated bi-directional DC-DC converter. This puts an additional power conversion element in series with the battery.

• Using similar PWM power conversion techniques, transformer-free UPS inverters are physically smaller as well, and produce low output voltage harmonics over a wider range of connected load characteristics.

• The bypass function (static bypass switch) is similar to the transformer-based design. However, without external transformers added, the bypass AC input must be the same voltage as the inverter AC output.

• Transformer-free UPS are typically designed and styled for both computer room in-row lineups and equipment room installations. Complete transformer-free UPS units are typically an assembly of standard frames plus functional control and power modules.

• A transformer-free UPS is lighter and smaller than the power-equivalent transformer-based design with both physical volume and footprint being less. And, according to the fall 2012 Data Center Users Group (DCUG) survey, data center energy costs and equipment efficiency are the top-of-mind issue for DCUG members, with nearly half of the respondents listing it as one of their top facility/network concerns.

However, other external transformers may be required for AC-DC isolation purposes, safety reasons, AC voltage changes, or to provide power distribution flexibility. With the addition of external transformers, the overall facility weight and footprint totals may be higher than with a transformer-based UPS design with implications for end-to-end system efficiency. If transformers need to be added to a transformer-free unit to make it compatible with a facility, a transformer-based unit may be a better solution.

Transformer-free UPS topologies have emerged to meet the demand for more efficient, flexible, smaller footprint, lighter weight UPS systems. The price of these performance feature improvements has been the replacement of a few robust but physically large, passive components, such as transformers, inductors, and capacitors with functional power electronic equivalents packaged in field replaceable, modular sub-assemblies. It is reasonable to expect that the transformer-free units will have service call rates somewhat higher than their transformer-based counterparts.



TECHNICAL FEATURES AND PERFORMANCE DIFFERENCES


In choosing between transformer-based and transformer-free UPS solutions, a system designer should determine where transformers are best utilized and whether they should be internal and/or external to the UPS in view of physical and electrical distribution requirements and tradeoffs. It’s important to review the techniques and tradeoffs utilized in the various rectifier, DC energy storage, inverter, and static bypass functions of these two UPS designs for various UPS system performance functions including:


• Site planning and adaptability to change

• Reliability and availability

• Robustness

• DC energy storage system isolation

• Engine-generator interface

• UPS output interface considerations

• High resistance grounding

• Fault current management

• Arc flash energy

• Isolation

• Maintainability

• External components needed to complete the system design

• Total cost of ownership

• Capital expenses (CAPEX) and operating expenses (OPEX)


When considering the total cost of ownership for these two architectures, it is important to include both the initial upfront or CAPEX as well as the ongoing or OPEX to power, maintain and service various options.


Technological evolution is constantly impacting the relative efficiency of transformer-based and transformer-free solutions. After years of optimizing performance, transformer-based UPS systems have achieved a relatively flat efficiency curve from 30% to 80% loading where typical tier 3 and tier 4 data centers operate. The latest transformer-free designs also have very flat curves down to as low as 20% of capacity, and have efficiencies in the 95% to 96% range in double-conversion mode. A study of the whole system design is necessary to determine the relative efficiencies as the addition of transformers, and the efficiency of those transformers, will have an impact.


In summary, transformers, whether internal or external to the UPS, are necessary to establish circuit isolation and local neutral and grounding points, as well as to provide voltage transformation points. This facilitates, for example, the implementation of very high power density installations based on 600V distribution sources, subsequently stepped down to 208/120V for IT load applications. When transformers are utilized in conjunction with the UPS internal DC link, DC-to-AC output, and AC-to-DC input isolation can be provided, reducing or eliminating the risk of DC faults propagating upstream or downstream of the UPS.



About the Blogger

Strategic Media Asia (SMA) is one of the approved CPD course providers of the Chartered Institution of Building Services Engineers (CIBSE) UK. The team exits provide an interactive environment and opportunities for members of ICT industry and facilities' engineers to exchange professional views and experience.

SMA connects IT, Facilities and Design. For the Data Center Consideration Series, please visit 

(1) Site Selection,
(2) Space Planning,
(3) Cooling,
(4) Redundancy,
(5) Fire Suppression, and
(6) Meet Me Rooms

Keep Your Knowledge Current in Electrical Design for Mission Critical Supply

Within the built environment, mission critical facilities and data centers have particular power requirements that significantly impact how they are designed and operated. Further to the


(1) Site Selection,
(2) Space Planning,
(3) Cooling,
(4) Redundancy and
(5) Fire Suppression,


you are recommended to join the 2-day advanced learning (CIBSE approved CPD Course) in Electrical Design for Mission Critical Supply.

With more than 18 years experience in critical facilities and electrical system design, the Instructors (with Chartered Engineer (CEng) status) will detail about the international standards, best practices and design project experience. You will gain insight into the critical supply system, from power components to distributions and efficiency; from power requirements to sizing, design, testing and commissioning.


-- Concept on primary supply and secondary supply
-- Power flow in mission critical supply system
-- Features of major equipment for critical supply

  > Uninterrupted power supply and power storage
  > Backup generator
  > Automatic transfer switch
  > Static transfer switch
  > Isolation transformer

-- Efficiency assessment
-- Power quality review
-- Configuration diagram of critical supply (N+1 / 2N) design & analysis
-- Review of cable sizing to incorporate harmonics content
-- Earthing system design
-- Testing and commissioning requirements
-- Brief of Systems Merging Appraisal Test (SMAT)


This is an advanced learning section on top of the Data Center Facilities Design & Infrastructure Engineering. For details, please refer to http://www.stmedia-asia.com/newsletter_6.html.



About the Organizer

Strategic Media Asia (SMA) is one of the approved CPD course providers of the Chartered Institution of Building Services Engineers (CIBSE) UK.

The team exits provide an interactive environment and opportunities for members of ICT industry and facilities' engineers to exchange professional views and experience.

SMA connects IT, Facilities and Design. For accreditation details and background, please visit www.stmedia-asia.com/about.html.

Four Monitoring Capabilities for Data Center Power Systems

The primary method that people use to measure the energy effectiveness of their data centers is power usage effectiveness (PUE). PUE compares the power used for computing capacity and the overall power used by the data center (PUE = Total Facility Power / IT Equipment Power). The inverse of PUE is data center efficiency, according to The Green Grid association.


Currently, the industry average PUE is reported to be around 2.0, but a number of new and proven data center infrastructure technologies are available to advance efficiency and reduce PUE to 1.5 or lower without compromising availability. There are 4 key actions data center managers can take with integrating monitoring capabilities into power systems that can help improve efficiency and advance PUE.

Monitoring Power Usage


With power densities and energy costs rising, the ability to monitor energy consumption is essential for advancing PUE. Where power is measured can have an effect on how efficiency is measured.

For a comprehensive picture of data center power consumption, power should be monitored at the UPS, the room power distribution unit (PDU), and within the rack. These data points should be consolidated and reported within an enterprise data center infrastructure management (DCIM) tool that can be shared by both facilities and IT management. DCIM tools (if specified early and fully integrated) can provide an overarching view of the data center, which will provide insights into how effectively or ineffectively the computing power of the data center is being used.


Specifically, measurements taken at the UPS provide a base measurement of data center energy consumption that can be used to calculate PUE and identify energy consumption trends. Monitoring the room PDU prevents overload conditions at the PDU and helps ensure power is distributed evenly across the facility.







The best view of IT power consumption comes from intelligent PDUs inside racks, which feature integrated monitoring and control capabilities to enable continuous power monitoring and feedback to the DCIM tool. Because rack power consumption varies based on equipment within the rack and its load, each rack should be equipped with a PDU—two for dual bus environments—capable of monitoring power consumption to the rack PDU, as well as overload-protected receptacle groups and, where required, at the receptacle level.


These systems can provide PDU, branch-level, and receptacle-level monitoring of voltage, kW, current, and kWh, giving the most direct measurement of power consumption and supporting higher data center efficiency and availability.



Monitoring Rack Conditions


With increasing densities, a single rack can now support the same computing capacity that used to require an entire room. Visibility into conditions in racks can help prevent many of the most common threats to rack-based equipment, such as accidental or malicious tampering.

If specified and installed correctly, rack monitoring units can be configured to trigger alarms when rack doors are opened, water or smoke is detected, or temperature or humidity thresholds are exceeded. Many units are equipped with cameras to capture video of the event. These “eyes inside the rack” can be connected to central monitoring systems where environmental data can be integrated with power data from rack PDUs, while providing local notification by activating a beacon or other alarm if problems are detected.



Monitoring Energy Efficiency


Automating collection and analysis of data from power systems can help reduce energy consumption while increasing IT productivity. Energy efficiency monitoring can track total data center consumption, automatically calculate and analyze PUE, and optimize the use of alternative energy sources.

Using real-time data from the many support systems, the monitoring system can track and trend total infrastructure power use and compare this to capacities. For example, for UPS power output, determine when UPS units are running at peak efficiency, and report Level 1 (basic) PUE. Monitoring at the room or row PDU provides the ability to more efficiently load power supplies, dynamically manage cooling, and automatically calculate Level 2 (intermediate) PUE. Panel board monitoring provides visibility into power consumption by non-IT systems, including lighting and generators, to ensure efficient use of those systems.






Finally, rack-level monitoring provides the most accurate picture of IT equipment power consumption and can support Level 3 (advanced) PUE reporting. The ability to automate data collection, and consolidate and analyze data related to efficiency—which is one of the primary jobs of the DCIM tool—is essential to data center optimization. These functions free up data center staff to focus on strategic IT issues.



Monitoring Batteries


Batteries have long been known as a weak link in the power chain. To prevent data loss and increase uptime, most data centers specify and require a dedicated battery monitoring system, which should be installed when the data center goes online. Using a predictive battery monitoring method can provide early notification of potential battery failure. The best practice is to implement a monitoring system that connects to and tracks the health of each battery within a string.

The most effective systems continuously track all battery parameters, including internal resistance, using a dc test current to ensure measurement accuracy and repeatability. Supported by a well-defined process for preventive maintenance and replacement, monitoring batteries can significantly reduce the risk of dropped loads due to battery failure, optimize battery life, and improve safety. The use of a battery monitoring system has an excellent payback period but, more importantly, provides peace of mind to the data center operator that the power system will react and support a load during a power outage.




INTELLIGENT CONTROLS


Previous generations of infrastructure systems were unable to adjust to dynamic load variations. UPS systems, meanwhile, operated most efficiently at full load, but full-load operation is the exception rather than the norm. The lack of flexibility in power systems led to inherent energy inefficiency.

Integrating intelligent control capabilities into data center power systems can help minimize the amount of energy being lost by enabling data center managers to tailor the performance of the UPS system to the specific efficiency and availability requirements of the site. When proper application of these capabilities is followed, it can enable double-conversion on-line UPS systems to operate at efficiency levels up to 97% while the inverter remains in operation.

Advanced UPS systems can employ an energy optimization mode that powers the load in reverse-transfer mode while keeping the UPS inverter at the ready to pick up the load in the event of a utility disturbance. An even higher degree of energy optimization provides power conditioning as well. The result is increased efficiency without compromising availability.

It is important that data center managers do not compromise on distribution and server power supply performance. Attention to distribution transformer and server power supply efficiencies can also reap huge benefits in improving PUE. Deploying TP1-rated distribution transformers can improve partial load efficiency by as much as 0.5%. High-efficiency server power supplies, while initially more expensive, can pay for themselves over their useful life.

About the Blogger


Strategic Media Asia Ltd (SMA) is one of the Approved CPD Course Providers of the Chartered Institution of Building Services Engineers (CIBSE). SMA exists to provide an interactive environment and opportunities for members of IDC industry and engineers to exchange professional views and experience on data center design, critical infrastructure system, electrical and mechanical facilities, etc.

SMA connects IT, Facilities and Design. For details, please visit our website at http://www.stmedia-asia.com/trainings.html.

Data Center Design Case Study

When the reliability demand is high and the data center has to work around pre-existing building constraints, there are significant design challenges.

With approximately 120,000 sq ft of white space divided into eight Tier III, 2 MW power data halls, the 20 MW data center meets a number of design challenges before construction even began.

The data center was originally the site of an old manufacturing facility. Coupled with the owner’s building requirements, this created a few challenges to the data center’s electrical distribution system design. Here’s a look at how Environmental Systems Design, working together with the other members of the building team, was able to solve them.


Challenge No. 1: The transformer


In an effort to maintain consistency across others data centers, the corporation likes to specify company-standard mechanical, electrical, plumbing (MEP), and fire protection equipment for all of its mission critical facilities. However, much of this equipment is designed for outdoor use, and because this particular facility has very limited outdoor space, significant adjustments had to be made for the transformer and switchgear equipment to work indoors.

For example, the heavy transformer couldn’t be placed on the raised floor designed for the data hall and electrical room space and wouldn’t be able to be removed from the building at the end of its useful life, as the 2500 kVA transformers each weigh approximately 16,000 lbs.

Instead, the facility was designed with the intent of moving in the transformer prior to the raised floor construction. Coordinating with the architect, saw cuts were designed into the pre-cast panels on the side of the building to make it easier for the contractor to remove/replace the exterior wall to get the transformer out for future installations or replacement.

Additionally, because the liquid-filled transformers specified are typically used outdoors, special design considerations were needed for the electrical room to contain the fluid in the event of a leak. A 4-in. metal dam was constructed along the perimeter of the room to contain the liquid, while perforated raised-floor tiles were installed around the transformer to facilitate the flow of liquid to under the floor.


Challenge No. 2: Uniform design


The owner wanted a repeatable, scalable design for each of the eight data halls to both create uniformity across the facility and provide the ability to build out data halls as needed over time. However, due to existing building conditions, including the different quantity and compact nature of the structural columns on the building’s first and second floors, an offsite modular construction design was eliminated.

Instead, the solution was to bring each component of the mechanical and electrical systems into the facility individually and build out the data halls one at a time as identically as possible.


Challenge No. 3: Medium-voltage equipment


Because the switchgear and transformers were located inside the building, medium-voltage (MV) feeders were routed throughout. The MV system itself was daisy-chained so it had to be installed and commissioned in its entirety when the first of the eight data halls were installed.

The active MV system presents a potential risk to workers during construction. Therefore, the MV feeders and conduit were routed through the ceiling of the first floor for safety, stubbing back up through the second floor slab and directly into the equipment. The feeders for the first floor equipment were routed similarly under the first floor slab. Because the eight data halls will be built out over time, this technique permits construction on the floor without subjecting workers to the active feeders.


Challenge No. 4: Switchgear and metering equipment


With very limited space for outdoor equipment, there wasn’t enough room for the utility’s 34.5 kV ground-level switchgear and metering. In addition, this equipment has not been fully vetted for the application and was seen as a risk by the owner. The only option was for the utility’s switchgear and metering to be pole mounted. This meant that the site’s 20 poles had to be spaced 20 ft from each other, taking up 400 ft of linear space on a site with little outdoor area.

The Reliability


Utility Service:

2N, or two separate utility sources/feeders, one general source and the second as backup. The switchgear is daisy-chained together, creating a connection from one switchgear to another down the line so that the power can feed into one switchgear, back out and into the other, rather than having multiple connections to the site.


Backup generators:

N+1 redundant swing generator. While each of the eight data halls has its own backup generator, another redundant swing generator for every two to three halls was designed as well, providing additional backup (a total of four swing generators in all). This provides an extra level of redundancy without the cost of providing an additional backup for each generator.


UPS system and distribution (2N):

Two sides to the electrical system were designed, where each side is a mirror of the other. The benefit of the 2N system topology allows for maintenance or a fault to occur on one of the sides, while still maintaining a completely active data center.



About Strategic Media Asia Ltd


Strategic Media Asia Ltd (SMA) is one of the Approved CPD Course Providers of the Chartered Institution of Building Services Engineers (CIBSE). SMA exists to provide an interactive environment and opportunities for members of IDC industry and engineers to exchange professional views and experience on data center design, critical infrastructure system, electrical and mechanical facilities, etc.

SMA connects IT, Facilities and Design. For details, please visit our website at www.stmedia-asia.com/trainings.html.

Invitation to Data Center Technical Visit (2 Sites) + CPD Course in Critical Facilities and Data Center Design

2-days CPD Course in Data Center Facilities Design and Infrastructure Engineering (23 - 24 October 2014)

(HKIE CPD Course Code: CPD1029)

The course is designed for Building Services Engineers, Facilities / Data Center Managers, IT Management, etc. to enrich and update the knowledge in critical facilities and data centers design & build. It is more than a general introductory program for data center. Topics include:

- IT strategy
- Cabinet layout
- Raised floor system

- Data center network and structure
- Telecommunication backbones, redundancy, sizing and planning
- Fiber and optical system design
- Fiber and optical cable components
- Copper cabling components
- Copper system design and high speed ethernet
- Cable distribution, layout and management
- Earthing / grounding and bounding

- Power (1) – high / low voltage system, switch system, etc.

- Power (2) – UPS, transformers, fuel tanks, generators, etc.

- Cooling (1) – cooling topology, hot / cold aisle, etc.

- Cooling (2) – chiller, CRAC, cooling towers, etc.

- Environmental management system
- Physical security
- Fire protection system

Date: 23 - 24 October 2014 (Thursday - Friday)

Time: 10:00 – 18:00 (14 Hours)

Venue: 20/F, New Victory House, 93 - 103 Wing Lok Street, Sheung Wan, Hong Kong

(Very Near Exit A2, Sheung Wan MTR Station)

Fee: Special rate for HKIE's all membership classes

For details and syllabus, please refer to the CPD Course Calendar of Hong Kong Institution of Engineers, HKIE (http://www.hkie.org.hk/eng/html/cpd/cpdviewer.asp?sn=3248).

Kindly complete and return an Application Form together with a crossed cheque made payable to “Strategic Media Asia Limited” - Room 1605, 16/F, Causeway Bay Plaza 1, 489 Hennessy Road, Causeway Bay, Hong Kong.

Half Day Technical Visit to Data Centers in Kwai Chung and Tseung Kwan O (8 October 2014)

Sponsored by IXTech (www.hkcix.com) and HKCOLO (www.hkcolo.net)

Further to the critical facilities course, a half-day technical visit is targeted to provide an interactive environment and opportunities for members of IDC industry to exchange professional views and experience on TWO data center facilities (Tier II and Tier III+) and services. 

Date: 8 October 2014 (Wednesday)
Duration: 13:45 - 18:00
Assembly Time: 13:45 - 14:00
Assembly Point: Exit D, Kwai Fong MTR Station (Kwai Yan Road, Next to the Bus Stop)

Fee: HK$100, Round-trip transportation will be provided
(Waiver for the 2-day CPD course's participants)

For details of the technical visit, please visit 

http://www.stmedia-asia.com/data-center-tour.html.

Electrical Design for Mission Critical Supply Course

Within the built environment, mission critical facilities have particular power requirements that significantly impact how they are designed and operated.

We are pleased to introduce a new 2-day Program in Electrical Design for Mission Critical Supply. The course content is designed for data center operators, electrical & mechanical engineers, building services managers and infrastructure managers to enrich the relevant knowledge in power system design for mission critical facilities such as data center and other infrastructure.

The course details about the power system design and the components that support typical data centers or critical facilities. It prepares individual to fully understand data centers’ electrical design & build by exploring the international Standards and sharing the speaker’s experience.

You will understand the mission critical supply system, from power components to distributions and efficiency; from power requirements to designed, testing, commissioning and maintenance.


- Concept on primary supply and secondary supply
- Power flow in mission critical supply system
- Features of major equipment for critical supply

(1) Uninterrupted power supply and power storage
(2) Backup generator
(3) Automatic transfer switch
(4) Static transfer switch
(5) Isolation transformer

- Efficiency assessment
- Power quality review
- Configuration diagram of critical supply (N+1 / 2N) design & analysis
- Review of cable sizing to incorporate harmonics content
- Earthing system design
- Testing and commissioning requirements
- Brief of Systems Merging Appraisal Test (SMAT)


For details, please visit the course content at http://www.stmedia-asia.com/newsletter_6.html


The Organizer

Strategic Media Asia (SMA) is one of the approved CPD course providers of the Chartered Institution of Building Services Engineers (CIBSE).

SMA, a critical infrastructure training and event organizer based in Hong Kong, provides an interactive environment and opportunities for members of IDC industry and engineers to exchange professional views and experience on critical infrastructure and E&M facilities.

For details of other data center courses and seminars, please visit our website at http://www.stmedia-asia.com/trainings.html.

Data Center Power System Design

Engineers should take a closer look at the different power strategies being used to distribute power, and how they impact the data center.

Alternating Current (AC) versus Direct Current (DC) is a battle that has been going on for more than a century and continues today in the data center industry. Although AC power is the standard, based on its potential for eliminating conversion losses and improving efficiency, many believe that DC power is the future of data center distribution. Still others believe that the same level of efficiency can be achieved with AC by using more efficient equipment with higher voltage distribution.

Electrical systems usually waste energy in the form of losses due to inefficiencies in the electrical equipment and distribution system. On average, the electrical distribution system losses account for 12% of the total energy consumed by the data center. For a data center with 2000 kW of IT load (2700 kW total load), that equates to an annual cost of USD$280,000


Power System Design Tips


So please review these six key items when planning a data center power distribution system:

• Install or replace existing power and IT equipment with energy-efficient equipment
• Review the proposed IT equipment to determine if the systems can operate on 240 V AC or 380 V DC
• Review all the advantages and challenges of the different power systems
• Determine how much of the existing infrastructure would need to be replaced to change power systems
• Design flexibility into the power system that will allow the data center to adapt in the future
• Design a power system that is modular to eliminate partial loading

Similar to the mechanical systems, modifications can be made to the electrical system to make it more efficient and save energy. The key to a good mission critical facility design is not to degrade the reliability of the facility in the process.

Typical Electrical Distribution Systems


Typical legacy data center electrical distribution system is made up of five major components:

Power is supplied to the data center at medium voltage from a utility/generator power source. The power is stepped down from medium voltage to distribution voltage by a substation transformer. The power then goes through an Uninterrupted Power Supply (UPS) system that conditions the power and provides ride-through capability during an outage until the generator starts. The power is then stepped down to substation voltage by a Power Distribution Unit (PDU). The PDU supplies power to the IT power supply where it is rectified and stepped down DC power, which is the internal operating voltage of the IT equipment.


Utility / Generator --MV AC--> MV/LV Transformers --480V AC--> Switchgear --480V AC-->
UPS --480V AC--> PDU --208/120V AC--> IT Power Supply --12V DC--> Servers


Four components in the legacy electrical distribution system with the highest losses are:

• Substation transformer: Transformer no-load and core losses
• UPS: Rectifier and inverter losses
• PDU transformer: Transformer no-load and core losses
• IT power supply: Rectifier and transformer losses

One method for increasing efficiency is to replace those pieces of equipment with more efficient equipment. Today with ultra-high-efficient transformers that efficiency is above 99.5%. Conventional double conversion UPS systems range from 84% efficient at 25% load to 94% at 100% load. Using flywheel or passive standby UPS topology can increase that range to 94% efficient at 25% load and 99% at 100% load.

Another method for increasing efficiency is to eliminate partial loading of the data center. Eliminating partial loading reduces losses by allowing the equipment to operate at its peak operating efficiency. This can be performed by designing a power system that is modular, grows with the load, or by designing a power system that uses flexible tiers, and matches the reliability and redundancy to the different programs within the data center.

A third method is to eliminate the inefficient electrical equipment altogether. Increasing efficiency by eliminating the equipment that has the most losses is the reason why different power strategies are being investigated for data center distribution.


About the Blogger

Strategic Media Asia (SMA) is one of the approved CPD course providers of the Chartered Institution of Building Services Engineers (CIBSE).

SMA, a critical infrastructure training and event organizer based in Hong Kong, provides an interactive environment and opportunities for members of IDC industry and engineers to exchange professional views and experience on critical infrastructure and E&M facilities.

For more details of other data center courses and industry events, please visit our website at http://www.stmedia-asia.com/trainings.html.