Ten years later: IT and life lessons from the South Tower

By Bob Eisenhardt

September 8, 2011, 4:20 AM PDT

Takeaway: Bob Eisenhardt was an IT pro employed in the South Tower on 9/11. He explains what it took to recover — both personally and in IT.

When the Twin Towers of the World Trade Center were attacked ten years ago, 2,753 people lost their lives, including employees of companies housed in the towers, 343 firefighters, 60 police officers, and eight private emergency medical technicians and paramedics. Another 184 people were killed in the attack on the Pentagon. The overwhelming majority of casualties were civilians, including nationals of over 70 countries.

TechRepublic wanted to commemorate that day in a meaningful way. We have asked Bob Eisenhardt, an IT pro and TechRepublic member who was employed in the South Tower, to talk about his own experience and what it took to recover and rebuild his IT shop in the aftermath.

On September 11, 2001 the world changed. I was part of the IT team for Aon Group (a global insurance company) located in the 99 to 105 floors of the South tower. TechRepublic has invited me to commemorate the tenth anniversary with these personal notes and memories.

Since this is an IT site, I will talk about the loss of and recovery of data in the immediate aftermath of the crashes. But I have to stress that, as it is with any disaster scenario, people are the most precious commodity. Aon’s Chairman, Patrick Ryan, demonstrated this when he headed a memorial service at St. Patrick’s Cathedral, with Judy Collins singing “Amazing Grace.” For most of us, this was the first time we all came together again in a single room, and we were mostly still in shock. Recovery would take a long time.

It’s important that your staff know that in evacuations they should not worry about computers, data, retrieving their personal effects from their desks, or going back to get their car. I learned this in the worst possible way: Steve Poulos, the system administrator for Aon Risk Services, returned to the 103^rd floor to retrieve the Risk Service data tapes. By doing so, he ran out of time and did not survive.

Get clear, get safe and go home as required. You should have your staff participate in evacuation drills, taught to use stairs and not elevators, and to have on hand “go bags” (small, easy-to-carry container for basic supplies such as flashlights, small medicals, maps, food and money). IT personnel may consider placing periodic backup data in their go bags if it fits snugly.

Another thing I learned on that day ten years ago was that simple common sense proved to be the ultimate skill. “Down was good, up was not an option, speed was essential.” Thousands of people made snap decisions without knowing the consequences of their actions; survival often meant just following the basics. (To quote Jean Reno’s wonderful advice in the 1998 remake of Godzilla - “Running would be a good idea.” When time came, I ran!)

The technical details

Due to maintenance in the 103 floor data center on Sept. 8-9, the data tapes for Aon Consulting ran cleanly and were taken offsite on Monday morning. We were lucky to have those backups as Tuesday was a very different day.

Technology also aided survivors tremendously in the days that followed the tragedy. In distant Chicago, Aon quickly developed a web bulletin board for survivors to post comments, which also enabled us to do head-counts. With that and the Yahoo-finance message board, survivors and family members could reconnect, and have a space to express their mourning.

September 11 was traumatic to the extreme, and to this day I am truly amazed that the Aon team began restoration literally within hours after the event. On September 12, Michael Edghill was arranging for our data tapes to be shipped to the South Carolina data center. Some of us went to work quickly, others took their time in emotional recovery but, eventually, we all returned. Our professional ethics required that we get to work as soon as possible, and we had it in abundance.

On September 24, we received in our temporary mid-town office over 900 Dell Optiplex desktop towers, large 21″ monitors and 250 laptop systems. The distribution of this equipment required a huge multi-office team effort. We put GHOST imaging to the ultimate test on a bench running 10 systems at a clip. (The only unforeseen problem was that in the beginning we were too easily handing out free keyboards and mice to anyone who needed one, so we later just ran out of them entirely. No good deed goes unpunished). In this intense recovery environment, you worked fast and smart, ate many boxes of pizza, and made lifelong friends.

The 900 plus systems used in rebuilding the network. Photo: Bob Eisenhardt

Not everyone was using imaging systems. I was initially assigned to cover the dislocated Aon Consulting staff move to our Greenwich, CT site. Here my primary job was to keep these emotionally stunned workers productive while staying out of the way of Larry Manno, manager of this location, who already had more than enough to do. By fortunate circumstance, I carried a small laptop that performed truly heroic work for my co-workers. For a few days, I was virtually the only IT staffer who actually had a computer! Print server, email conduit, temporary loaner - this was a wonderful little laptop indeed.

For months prior to 9/11, we were forever complaining about the size of Lotus Notes email files. And yet, after September 11, these same large email files contained a wealth of stored easy-to-access data that provided its own ad-hoc backup protocol! I will never complain about large email files again.

We also learned that Dell does not build substandard equipment. At Ground Zero, workers found an intact Dell Latitude CSX laptop belonging to Louis Sausa. It survived falling 101 floors amidst tons of collapsing building. Now in the hands of the World Trade Center museum, it has not been turned on again, so it remains set to September 10, 2001. I wonder if the BIOS knows what day today is.

For me, the essence of the day is now compressed into 102 minutes, from 8:45 am to 10:02 am. Last year on 9/11, while visiting family in Atlanta, I walked away to a fence at a Soccer game, touched the ground at 9:59 am - the moment the South Tower collapsed. The earth held the remains of many of my friends: Donald Havlish, John Crowe, Laura Snick, Catherine Nardella, and Pamela Gaff, among others. I took a deep breath and then a small hand took mine. My granddaughter, Nicole Skyer, age 7, came to take me back to the soccer game and another day.

Every morning, take a glance up at the sky and remember that you are alive to enjoy this day and that for all the good and bad it may bring, you are here to see it through.

I dedicate this piece to Steve Poulos.

http://www.techrepublic.com/blog/career/ten-years-later-it-and-life-lessons-from-the-south-tower/3438?tag=nl.e101