lundi 15 octobre 2012

CYA: 3 rules to keep a systems crisis from taking down IT

A lire sur:

October 15, 2012

Even the most redundant of infrastructures can be brought down by a lack of readily accessible knowledge

These days, all but the smallest organizations spend mountains of money building redundancy into their infrastructures. As business depends more and more on those systems to function at even the most basic levels, the capital plowed into highly redundant disk arrays, bulletproof backups, and highly available virtualization infrastructures has become an expected cost of doing business.
However, the frenetic pace of break/fix, application rollouts, and systems upgrades often leads to the most dangerous single points of failure of all: people. That huge investment in redundant, self-healing infrastructure can be negated in one fell swoop if the one person who knows how to run some critical part of it quits, is on vacation, or even just went out for lunch without a cellphone at just the wrong time.
[ Get expert networking how-to advice from InfoWorld's Networking Deep Dive PDF special report. | Keep up on the latest networking news with our Technology: Networking newsletter. ]
Often, you don't need an actual service disruption to cause a five-alarm political fire that reaches all the way to the executive suite. In my years as a consultant, I can't count the number of times I've been called in (sometimes at a substantial expense) to help fill a knowledge gap simply because a single member of the IT staff wasn't available for whatever reason.
Looking back on those incidents, there are three items every member of the infrastructure team should have at their fingertips, whatever their role.
1. Build basic documentation
If you've built and been responsible for a critical infrastructure system -- storage, virtualization, or a complex application -- you probably can recite every detail about it, IP addresses and all, without looking at a single piece of paper. But how much of that information does the guy or gal in the office next to you know? Often, very little -- sometimes, nothing at all.
If there's one thing most IT pros really hate doing, it's writing documentation (I count myself in that group). However, even the most basic tabular documentation that shows which physical systems are responsible for which applications, what their names are, and which rack they are in can make a huge difference when something goes wrong while you're not around. I've seen massive damage done in the course of a lunch break simply because an uninformed staff member tried to troubleshoot the wrong system for lack of the most basic shred of current documentation.
Having basic documentation may not allow an unskilled member of the team to fix a complex problem, but it will allow him or her to zero in on where a reported problem might be originating and double-check some of the most obvious causes. Critically, it short-circuits the often politically damaging scramble to answer even the most basic questions from users and stakeholders.

Aucun commentaire:

Enregistrer un commentaire