Responding to a Network Security Breach
No matter how careful you are and no matter how secure your systems are, bad things happen. Electronic components fail. Software can be found to be buggy. People make mistakes. And, just sometimes, people make mischief with intent to do harm to the network. Nature can affect whether your network works or not, too. All of these situations require that you respond to the emergency at hand quickly and efficiently.
Call in the SWAT, um, CERT team
The best way to be prepared for a network security emergency is to have a CERT in place. CERT stands for Computer Emergency Response Team (or CIRT, Computer Incident Response Team) — a team you set up to handle emergencies within your own organization. Whichever way you spell it, these are the Ghostbusters of computer networks; they find the bad stuff and root it out.
The business model of a Computer Emergency Response Team is to respond to the emergencies much the same way firefighters and police do. They respond, identify the situation, isolate the area, and go to work. And because no one ever knows how long it will take to contain an emergency situation, they frequently work long, hard hours under tremendous stress.
CERTs consist of highly skilled people who vary in their areas of expertise but are cross-trained to cover any eventuality. These team members need to
- Have an in-depth understanding of networking, operating systems, and applications so they can recognize when something is awry.
- Be able to identify viruses and eradication techniques.
- Know hacking technique and system vulnerabilities
- Use many cross-platform network tools. Some of these tools are actually hacking tools and others are used to discover system intrusions..
- Be able to work as a team and be cool under pressure.
- Be able to communicate with others who don't have as high a level of understanding as they do. This is important to be able to give status reports and to recommend changes in security to prevent further occurrences.
Because CERT team members are so highly skilled, a company can't afford to hire a CERT team just to sit around for months on end doing nothing. The team members usually have full-time jobs doing something other than emergency response. Rarely do they ever hold more than supervisory or middle-management positions. However, when they are responding to a critical situation, they need to have the authority and autonomy to make executive level decisions. The life of your business may depend upon quick decisions.
You should have a section in your security policies and procedures document that spells out your company's arrangements for a CERT, regardless if it is in-house or outsourced. The roles and responsibilities should be clearly stated as well as who should call the team to action and when they should be called.
Your network has been acting funny lately or you've seen some strange things that make you believe an intruder is in your network. The first item on the agenda is Don't Panic! Call in your CERT as soon as you can. Chances are that the intruder has been in there for some time, but this is just the first time you've noticed. Intruders are like roach infestations — they don't just happen overnight. The CERT will take their time and work systematically to get rid of the nuisance. And unless they notice that files on your servers are being destroyed at an alarming rate, they will not shut down your network.
You need to take some basic steps before the CERT comes in and starts to work. These steps may vary slightly in order, depending on the situation. For example, in some cases you may call in the CERT before you notify company executives because it's more important to get the team working first. In any case, do all the following steps and don't omit anything. First and foremost, however — Don't turn off or reboot any systems.This could hamper the recovery process.
1. Start taking notes.
Don't start typing commands like crazy trying to find the intruder. Let the CERT do that. It's more important at this point that you get a new notebook and write down everything you've noticed and what you did. It's very important that you record the time and date of everything. This notebook may become crucial evidence in legal proceedings later.
2. Notify upper management.
Do not send e-mail messages as they could tip off the intruder. Hopefully, you had prepared a call-sheet ahead of time with the names and phone numbers of those who need to know. The most effective way of handling the notification is for you to call two people and then have them call the rest of the list. Otherwise, you could spend hours on the phone explaining the situation over and over to dozens of people. Time is precious and should be spent on the emergency — not on hand-holding.
3. Call in your CERT.
Do this quietly and without fanfare. You don't want the entire company's work to come to a standstill because you've called a general alarm. When the CERT gets there, brief them and then leave them alone to do their jobs.
4. Enforce a "need-to-know" policy.
Don't tell employees something is up unless they really need to know. The intruder may have an inside accomplice, or it could be a false alarm and not an intrusion. You don't want idle gossip getting outside the office to your customers, the press, or your competitors. You can always say that the company is experiencing "network problems" as most people will accept that explanation without further questions.
5. Someone in the company should be the point person in case the public becomes aware of the situation.
If you have a PR department, that's their job. You don't want a media storm on hand, so only the appointed person or persons are allowed to talk to the press and customers. Sometimes an incident isn't an incident at all but a misconfiguration in the network. Remind the press and the public that things aren't always as they seem.
6. Give support to your CERT.
They probably work long hard hours without breaks. Make sure they get meals and refreshments sent into them. (This may mean more than just Cokes and candy bars.) If relief team members are needed, put them on alert and set up a shift schedule. Enforce the schedule, too. Many team members will be reluctant to give up their posts, but they can easily burn out after a day or two. If the team is in need of more equipment such as spare disk drives and network devices, get them quickly.
7. Contact your legal department.
Let them know the situation. They can decide if any laws have been broken. If they advise you to contact law enforcement, do so.
8. Conduct briefings and meetings after the clean-up.
Tell everyone what happened and why and what you can/have/will do to make sure it doesn't happen again. Don't point fingers; learn from your mistakes.
Make no mistake that this will be a stressful affair for many and that tempers can and will flare. Don't fall into the trap of getting so caught up in the moment that you forget what you're supposed to be doing.