The Situation
Since the Maui High Performance Computing Center was commissioned in 1993, its high-performance computing resources-including a Cray XD1 supercomputer with AMD Opteron processors, IBM Power3 and Power4 configurations, and large Linux clusters-have been housed in a 6,000-square-foot (sf) data center protected by Liebert power and precision cooling equipment. Relying on Liebert technology for environmental control and power protection allowed the center to achieve high availability while continually adding the cutting-edge technology required to stay at the forefront of supercomputing technology.In 2006, the center addressed the need for greater computational capacity by developing plans to implement a terascale high-performance computing platform from Dell. The 5,120-processor Dell PowerEdge 1955 system would increase the computational capability of the center’s high-performance computing platform to 60 teraflops (60 by 1012 floating point operations per second), ranking it among the fastest supercomputers in the world.
The new system represented the biggest single investment in the center’s history, and required a new 8,000-sf data center to house the system. Because planned upgrades will increase overall computational capability to 120 teraflops within two years, the new facility required a dynamic power and cooling infrastructure that could scale to handle the added power demands and heat load. As a national resource for vital government research, MHPCC has a business-critical continuity goal of 100 percent availability. This lofty expectation is consistently threatened by geographic conditions as MHPCC is located in a remote area that experiences frequent loss of commercial power and is subject to earthquakes and tropical storms. “Our entire business is information technology and the necessity for maintaining operational capability is absolute,” says Carl Shelton, MHPCC facility manager.
The Solution
In designing the new facility, MHPCC turned to the Liebert specialists at Emerson Network Power. “We continue to build our critical infrastructure on Liebert power and cooling solutions because of their proven track record of providing the reliability and flexibility it takes to keep our world-class facility running smoothly,” Shelton says. “In 12 years of operation with Liebert technology, we’ve never experienced a single problem related to power or cooling.”Inside
Within the walls of MHPCC’s new data center, the Dell PowerEdge servers occupy 40 racks arranged in four rows in a hot aisle/cold aisle configuration. The room has a 36-inch raised floor to promote ample airflow and pressure. 12 Liebert Deluxe units rated at a combined 360 tons of air conditioning capacity provide adequate cooling and airflow for the projected heat load. The Liebert chiller connects the precision cooling units to the facility’s chilled water system.MHPCC deployed a multi-module UPS designed to balance reliability and scalability, using Liebert Series 610 UPS with a combined capacity of 850 kilovolt-amperes (kVA) of power protection. The UPS modules are connected through a Liebert System Control Cabinet. The enhanced fault current management capabilities of the Liebert Series 610 provide immediate response to utility outages with high overload capacity. Four Liebert PPC precision power centers provide grounding, custom electrical distribution, monitoring and expansion capabilities. A 1.5-megawatt generator provides backup power for extended outages. “Between the UPS and the generator, we’ve been able to weather commercial power outages without incident,” Shelton says.
Because of the facility’s remote location on Maui, fast response to maintenance issues was an important factor. The service business of Emerson Network Power delivered a four-hour emergency response time plan and provides regular preventive maintenance services to ensure systems remain in peak operating condition.
The Results
MHPCC met its project deadline for installation of power and precision cooling equipment and launching the new supercomputing system, which became operational in September 2006. The added capacity complements the Department of Defense High Performance Computing Modernization Program’s array of high performance computing resources. Nicknamed “Jaws,” it is the largest supercomputer in the Department of Defense research program.In initial testing, the new system delivered a clock speed of 42.39 teraflops, ranking it as the 11th-fastest supercomputer in the world on the November 2006 list of Top 500 Supercomputer Sites (as compiled by researchers from the University of Tennessee and the University of Mannheim).
Shelton says he was especially impressed with the responsiveness of Emerson Network Power and willingness to work within a demanding development schedule. “When we said we needed equipment within a few weeks, we didn’t hear any push-back or excuses. They just went to work and started manufacturing it,” he says.
Power and cooling systems have performed flawlessly, supporting continuous operation of the system despite a 6.7 magnitude earthquake in 2006 that caused $200 million dollars of damage on several Hawaiian Islands. “We lost commercial power for about 10 hours, but our UPS and generator kicked in seamlessly and kept us online,” Shelton recalls.
Looking ahead, he is confident that the data center has the dynamic power and cooling infrastructure to meet his business-critical IT infrastructure needs and accommodate planned expansion to the projected computational speed of 120 teraflops. And, when it comes to any potential service issues, Shelton knows he has a partner close by to help out.
“Other vendors quoted us a 24-hour response time for service,” he says. “Emerson Network Power responds within four hours, day or night, quickly handling any maintenance issues concerning our power and precision cooling equipment. The service is impeccable.”
