Commissioning critical facilities has become the accepted practice. Almost every data center built, expanded, or modified today undergoes at least some level of formal commissioning. Most projects encompass full-service commissioning, beginning at the design phase or earlier and continuing through formal acceptance and testing, including closeout documentation and operating staff training. But like the rest of the capital project, commissioning ends when the facility gets accepted by the owner and the project is deemed complete.

There is a new trend, however, to extend the concept of “third-party commissioning” to include the commissioning of the facility’s management organization and associated operations and maintenance programs of critical facilities. This is not to be confused with what is typically referred to as “continuous commissioning,” which is focused on the nuts and bolts of critical infrastructure (which is of course a best practice). OPx is applying the commissioning process to the programs and staff required to manage and execute the day-to-day, week-to-week, and year-to-year activities necessary to ensure sustained operations for the life of the facility. Operational commissioning, or OPx, involves engaging third-party validators to oversee and manage the design, development, implementation, documentation, and staff training on the site specific facilities management processes and programs.

THE NUTS AND BOLTS OF OPx

OPx commissioning uses a similar process to ensure that appropriate facilities management processes are delivered to support a facility as is typically employed to commission the facility infrastructure. Ideally, this process begins with a formal “programming” phase to define clear facility management goals and expectations in the form of requirements and key performance indicators (KPIs), which get translated into operations and maintenance (O&M) strategies and processes. This is followed by a development phase, where programs and procedures are constructed that will be used to direct and control how the facility will be operated and maintained to satisfy the programmed requirements.

The acceptance phase is where the developed programs are implemented via a “pilot” program that includes formal testing and debugging of the associated processes and procedures. Discrepancies and issues are tracked to closure just as is done with typical commissioning. And just as formal training and documentation are required when commissioning infrastructure, the facilities management processes and policies must be documented and the respective operating staff trained on how to use and comply with the programs and procedures.

OPx commissioning should result in the delivery of a site specific “playbook that documents the basis for facilities management of the site.” The playbook is organized and structured for ease of use, and in and of itself is a strong training aid for both new management and remedial training of existing management. The playbook begins with a basis-of-operations document, which is to facilities management what the basis-of-design is to the design of the physical facility. It lists, describes, and defines the requirements and performance criteria that the facilities management processes, programs, and procedures will be based upon, along with the acceptable level of performance (KPA) that allows management to objectively measure actual O&M results and compare to expected results. The playbook at a minimum should include:

•Basis-of-operations (Facilities operations performance standards and policies)

• Risk management and reliability assurance program

• Forensic root-cause analysis and incident reporting

• Business continuity / disaster recovery

• Short-circuit, arc-flash, and grounding

• Key performance indicators (KPIs) and performance measurement

• Continuous process improvement (CPI) program

• Facilities management organization and staffing program

• Staff organization, shifts, and minimum coverage

• Staff qualifications and position descriptions

• Escalation protocols and emergency response and contacts

• Staff evaluation, training, and certification program

• Training programs (New hires, remedial, and refresher classes)

• Safety, work rules, O&M, vendor management

• Staff and team evaluation, testing, and certification

• Vendor and contractor site specific training

• O&M program (Standards and procedures)

• Safety procedures

• Normal, maintenance, emergency, and recovery procedures

• Vendor and contractor management and oversight

• Computerized maintenance management system (CMMS)

• Predictive maintenance

• Critical load management and tracking program (Power and cooling)

• Service level agreements (SLAs) (Internal and vendor)

• Financial management (Capital and operating expense budgeting, planning, and accounting)

• Safety and HAZMAT compliance program

• Recordkeeping and document management and control program

• Document management, control, archiving, and destruction policy

• Systems O&M manuals (SOMMs)

• As-built documents, submittals, equip/panel schedules, and record drawings

• Contracts, SLAs, and master service agreements

• Asset management, configuration management, and spare parts/ tool inventory

• Utility and service usage and trend reports

• Training, O&M history, etc.

• Project management and continuous commissioning program

• Code and regulatory compliance program

• Fire protection and life safety testing

• Backflow protection, safety valves, pressure vessel, elevator testing, etc.

• OSHA, HAZMAT, and safety programs

• Continuous process improvement program (Periodic review, update, and auditing)

Perhaps the most important differentiator between traditional commissioning (Cx) and OPx is the continuous process improvement aspect. Unlike traditional commissioning of new infrastructure that has an endpoint, OPx is an ongoing process that addresses the dynamic, constantly changing character of data centers and high technology buildings. It treats each program and the associated procedures as living documents. It recognizes that training must be structured and periodic to keep existing staff competent and to ensure new staff are qualified. It uses a combination of scheduled and random audits to inspect what is expected.

The development of a basis-of-operations playbook and supporting policies, processes, and programs is a very large effort and requires specialized skills and knowledge. The effort is best performed by specialized consultants with the broad expertise necessary, along with close collaboration with the owner’s management and staff. It is a mistake to assume those qualified to manage programs or follow procedures also have the capability and expertise to develop the programs and processes from scratch. Furthermore, those staff should already be fully tasked with other work such as monitoring the construction, startup, and testing of the facility.

Another strong reason to seek outside consultants is they will have experience with a broad spectrum of facility management strategies and processes, which they can bring to the table so the owner can pick and choose the strategies and practices that best match the site and corporate culture. This avoids relying on institutional strategies and “that’s the way we have always done it” syndrome.

If the firm selected to commission the physical facility and infrastructure also has the expertise and resources to perform OPx commissioning, there are many possible synergies available by combining these scopes and efforts since much of the data collection and deliverables between Cx and OPx overlap. The concurrent development of facilities management programs along with the design and construction of the physical facility ensures the programs are site specific, that the programs and documentation are in place on “Day 1,” and that the operations and maintenance staff have the necessary resources and training to sustain continuous operations over the life of the facility.