In one sense, moving a room full of highly sensitive data server /computer equipment is like walking a tight rope. Both performances take nerves of steel.
On the other hand, nerves are no substitute for pinpoint planning. The sequence of a short cycle floor changeout/ hardware move requires pausing the power supply, removing the old raised floor system installing a new floor, reinstalling the data server equipment, along with thousands of cable terminations and then switching the entire system back on again—all in a very short time frame.
Not many construction design teams would attempt such bold and ambitious task, especially in a semiconductor research fabrication facility (Fab), where the stakes are extremely high.
One slight misstep might not only affect current runs at the plant but might also strike at the heart of the strategic development process of time to market.
The following insights come directly out of such a renovation.
The design team at SSOE, which followed the Engineering, Procurement, Construction (EPC) approach, was able to identify and predict the challenges in the execution of construction. SSOE used its experience of working on advanced technology and mission-critical projects to carry out this highly specialized renovation.
The case history shows how to avoid typical pitfalls in the proposal and design phases, how to build the right team, and how the goal of zero unplanned downtime (ZUD) became the team mantra.
The construction design team must be acutely aware of the risks involved with highly sensitive, high value equipment and must have worked previously in such an environment.
The Computer Room is the brain that runs a research Fab, located in Albany , NY. The servers in this computer room are responsible for running the automated semiconductor production tooling, and other equipment, many valued at $20-40 million and up, which are responsible for running processes that could lead to the next breakthroughs in microprocessor chip technology.
Therefore, any unplanned downtime—or worse, a possible shutdown—could ruin the critical development of chips, costing millions and jeopardizing time to market. In this case, the owner required that the computer room stay operational during the construction which included demolition of the entire existing Computer Room and associated mechanical and electrical systems, and rebuild of the room, upgrade of these systems – all with sensitive servers running the entire time.
Putting together a solid design and construction team is job one for leaders in the renovation project.
The contractor must be a strong and knowledgeable leader, and the project manager should have both engineering and construction experience.
The design team must be acutely aware of the risks involved with highly sensitive, high value equipment and must have worked previously in such an environment.
All team members must develop a common understanding of the scope of the work. They discuss details such as architectural, structural, mechanical and electrical issues, key pieces of equipment, and the logistics of renovating a computer room while keeping it operational during construction.
On these types of critical projects, it is important to include early planning and design work during the proposal phase to work out key execution details and schedule issues.
For example, teams should consider manpower and how and where to stage equipment and deliveries during the day in a working facility. Typically, large equipment cannot be moved during the day in an operational facility since it would cause too much disruption. However, work can be scheduled to take advantage of off hours at the site.
For example, on this project, it was identified early in planning between the owner and EPC team that if the owner was to keep the Computer Room operational during construction, ensuring the Fab could operate, that that the new required higher 2 ft. high raised floor system would need to be installed at the very beginning of the project during a limited 24 hour shutdown window.
There was just no way that the floor would be removed and replaced, servers moved out then back in, during the normal working hours in this client’s facility - the disruption would be intolerable.
Building the Best Team Possible
Part of the Engineering, Procurement, and Construction (EPC) approach is to involve the contractor, designer and all other project stakeholders early in the planning phase.
EPC Project leaders should be careful to choose best-in-class team members that have solid experience in working in critical environments.
Although proposals come from various contractors and price ranges, the leaders should not base their decision solely on the amount of the bid, but award the work to the company with a competitive price along with the highest understanding and aptitude for executing the work, knowing that design and construction fees pale in comparison to the potential impact on revenue and the product pipeline from a poorly executed project.
This is best value for the client. In this instance, SSOE was able to draw on a longtime partner, Performance Contracting Group, a world leader in construction of advanced technology and mission critical facilities for their specific experience in working in this high value, high risk environment where failure is not an option
Part of the EPC approach is to involve the contractor, designer and all other project stakeholders early in the planning phase. This allows adequate lead time for pre-ordering equipment, discussions of cash flow, constructability, and other critical operational planning.
The team looks at the end date and then works critical milestone dates and major equipment purchase to achieve that completion. Together, the team was able to identify the need for the raised floor changeout at the start of the project and the means and methods to protect the servers, keep them cool and happy in specially constructed protective mini-environments and accomplish construction with no interruption to the owner’s critical operations in the Fab.
Management at any facility also appreciates knowing important details early. What are the logistical issues when moving equipment? Will security or other staff need to be alerted? Facility personnel would have to be notified to discuss possible sites for temporary equipment storage. Giving the client’s team advance notice in weeks instead of days is always preferable, particularly when their daily processes will be affected.
On Pins and Needles
When preparing for a shutdown, the team should develop a micro schedule of critical sequencing for a disconnect, transport, and reconnect, including drawings and labels for the pieces of equipment in the computer room.
A system crash of the computers that run the automated semiconductor manufacturing tooling and robotics could be disastrous. If processes are not interrupted cleanly, or if the shutdown does not go smoothly, the company could lose hundreds of thousands or millions of dollars due to damage to high value products.
Preparing for a shutdown can be an event in itself, almost like choreographing a ballet. The team should develop a micro schedule of critical sequencing for a disconnect, transport, and reconnect, including drawings and labels for the pieces of equipment in the computer room. Each move by each contractor during the short shutdown window is identified and the sequence is reviewed multiple times.
One nightmarish scenario is the possibility—when turning equipment back on after an electrical shutdown and modification to the supply— that the equipment doesn’t work. It’s a brave soul that flips that switch. Once the shutdown is over, in most cases it can’t be repeated anytime soon without major interruption to the client’s operations. This is why plans for a shutdown are outlined early in the planning of the project, reviewed and refined multiple times, analyzed and scrutinized.
In conclusion, when walking the tightrope of renovating mission critical facilities under deadline pressure, the required game plan for every project phase is planning early, revising, and planning yet again. Gathering the best possible EPC team members and focusing on achieving an operational facility by a certain date with no unplanned down time is the difference between success and failure.
Gavin Kaleta, PE, LEED AP BD+C is a mechanical engineer at SSOE Group, a global engineering, procurement, and construction management firm. With 11 years of experience, Gavin has in-depth knowledge of base build projects for large high-tech clients including PSSS building expansions, Central Utility Building (CUB) expansions, Cleanroom retrofits and Fab interior building development and layout changes. Gavin has extensive experience, developing the project scope of work, managing client meetings, managing the design and change control processes, estimating work and construction costs, overseeing subcontractors in the field, and providing regular project status reporting to both internal and external customers.