Data centers serve virtually every industry, either directly or indirectly, and they are critical to the financial industries, communications and transportation providers, governments, military organizations, health care, and many other sectors. Regardless of the specific application, all data center operators must provide for secure storage of data and reliability in the event of interrupted or poor quality power supplies. Certain data centers focus solely upon providing continuity through secure remote access for business users in the event of power outages, natural disasters, or terrorism.
Data centers typically meet these requirements by a combination of backup batteries (commonly called uninterruptible power supply or UPS devices) and on-site activation of emergency backup generators. Power quality is also of key importance for data centers. Voltage irregularities, line noise, harmonic distortions, and frequency variation can negatively affect the operation of a data center. A recent AFCOM survey found that power is the number one concern in data center management. The costs of failure can be high, and substantial liabilities can arise when energy supplies to data centers are compromised.
According to the United States Department of Environmental Protection (EPA), the increasing demand for computer resources in the past five years has led to an estimated doubling of the energy used by data centers nationwide. The EPA estimates that in 2006 the nation’s data centers utilized approximately 61 billion kilowatt-hours (kWh) at an estimated cost of $4.5 billion. By 2011, EPA estimated, energy consumption by data centers could nearly double again to more than 100 billion kWh at an estimated annual cost of $7.4 billion. Such rapid growth, coupled with the aging national power grid, causes concern about the grid’s ability to provide a consistently reliable supply of energy to the nation’s data centers. The 2003 Blackout of the Northeastern U.S. and Canada that rendered 50 million people without power exacerbated such concerns.
As data centers continue to proliferate, issues with energy supply could affect nearly every facet of our existence. According to predictions by AFCOM, power failures and limits on availability could negatively affect 90 percent of commercial data center operations over the next five years. Further, a recent AFCOM survey found that 81 percent of respondents had experienced a failure in the past five years and 20 percent experienced at least five failures within that same period. Of these outages, over 80 percent were due to either external or internal power failures.
Within the last several years, even the best-planned facilities have been subject to unexpected power outages and disruptions in service. On Tuesday, July 24, 2007, a data center co-location provider’s facility suffered an outage of forty-five minutes when it lost grid power and backup generators failed to start. Several well-known and popular websites such as Technorati, Craigslist, and Netflix were unavailable for several hours as a result. On July 30, 2006, a telecommunication hub in Seattle lost power for approximately 45 minutes when a breaker that was supposed to feed backup power failed to engage. On July 24, 2006, the popular website MySpace was offline for hours due to power outages in its Los Angeles data center. On December 22, 1998, the data center for a distributor of microcomputer products suffered a power outage causing its computer system to reset to default settings. This caused a lost connection to the distributor’s critical business application used to track customers, products, and daily transactions for the greater portion of a day.
The losses and liabilities a data center owner or operator might suffer due to a power outage could be significant. Even though stored memory may be unaffected by a power outage, reliability of operations is a major selling point for data centers and a key determinant of success. Data center operators who cannot live up to their promises in providing reliable service may lose customers due to a loss of confidence in the center. Even short outages to critical systems can cause great harm to operations or customers. Further, due to the business-critical nature of many of the computer systems housed in data centers, customers may not be willing to tolerate any outages and may bring their business elsewhere or pursue the data center for damages.
Loss of service to customers may also lead to potential financial liability on the part of the data centers, as customers try to recoup losses from the data center for damaged hardware and lost revenue, In addition, outages may result in liabilities to customers who have specified service level agreements that may result in the data center’s being in breach of agreements with customers. Thus, customers may attempt to seek compensation for losses from the data centers under either contract or negligence theories.
The data center may not, however, be able to recoup from the electric utility the data center’s own damages or damages paid by the data center to the data center’s customers. This may be the case even when the utility is responsible for the outage. This is so because many utilities operate under tariff provisions that deny liability for ordinary negligence and for voluntary outages the utility makes to protect the overall distribution system.
An example of a typical electric utility tariff provision is as follows:
The Company will endeavor at all times to provide a regular and uninterrupted supply of service, but in case the supply of service shall be interrupted or irregular or defective or fail from causes beyond its control or through ordinary negligence of employees... the Company will not be liable therefor. The Company may, without liability therefor, interrupt service to any Customer or Customers in the event of emergency threatening the integrity of its system, if, in its sole judgment, such action will prevent or alleviate the emergency condition.
A UPS maintains a continuous supply of electric power to computers and related equipment in data centers by supplying power from a separate source when utility power fails. The primary types of UPS devices are off-line and line-interactive. An off-line UPS device commences providing electric power “instantaneously” upon the occurrence of a utility power failure. Conversely, an on-line interactive UPS provides continuous power, typically from battery (lead acid) power and is charged on a continuous basis by utility power. This also provides protection to the data center against problems relating to the quality of the power supplied.
UPS units are essential equipment for data centers but can only provide short-term power and are not a substitute for backup power generation, which must be activated to assume the task of supplying power in the event of a continuing electric utility failure to provide electricity. Data centers may elect to install emergency power systems that include both UPS units and standby generators.
Manufacturers’ and installers’ warranties for backup generators may limit their liability to one year or less and may only provide for restoring the generator to working order or replacement of parts. Even maintenance packages may only require that the maintenance provider be on site within a certain period of time with no guarantees as to when power may be restored. Following is an example of an emergency service provision:
Emergency Service shall be available 365 days per year, 24 hours per day. The response time for Emergency Calls shall be within 4 hours of notification by Customer to Maintenance Provider or Maintenance Provider being informed by its Remote Monitoring Service. “Response time” means arrival at the Service Site of an authorized technician or remote diagnosis of the Equipment, and notification to Customer of corrective actions to be taken by Maintenance Provider.
To protect themselves and their investments, data center operators should maintain their data centers in top condition and perform regular testing of backup systems. Personnel should be properly trained to handle power outages and proper procedures should be in place for handling power outages. In addition, data centers should ensure that the appropriate personnel are on-hand to handle a power outage based upon the assumption that an outage will occur.
Data centers should also approach contract provisions with customers realistically, realizing that no plan or equipment can provide 100% protection against outages.
Data centers should ensure that their facilities are insured for losses due to power outages. Courts in some states have been willing to consider an outage as “damages” to a computer system for purposes of insurance coverage. Other states require proof of actual damages to computer hardware. Regardless, a well thought-out contingency plan, regularly tested equipment, properly trained personnel and well-documented procedures can help to avoid potential outages or reduce their severity.
Daniel J. Bauer, associate, Duane Morris , LL.P., helped write and research this column.
