Tuesday, 30 July 2013

"YOU'VE BEEN HACKED!!"... Now what?

Over the past dozen years or so, I have been involved in a number of investigations of information security breaches; some more serious than others.  I claim no particular uniqueness in this, there are many professionals who have far more experience than I... but I thought you might be interested in reading (and hopefully commenting on) some of my conclusions and recommendations as to how an organisation or an individual might emerge from an information security breach with sanity and reputation intact...

This is not an exhaustive "how to" type guide, but a few thoughts through which I hope to stimulate a bit of debate.

The first question to ask yourself is:

"How will I know that a breach has occurred?"

As every article, paper, blog or whatever you have ever read will already have told you, there is no excuse these days for not having a suite of security solutions deployed on your infrastructure (yes, even at home on a single device) which looks out for known viruses, trojans, spyware and other forms of malicious software (or "malware").  Such as solution should also include a firewall of some description which needs to be configured to prevent incoming access from all but known and trusted sources.  These solutions are readily available, inexpensive and not difficult to configure.  The quality of customer support has improved greatly in my experience over the past 10 years as the field has become better understood.

So let's assume that, regardless of whether you use Windows, Mac or Linux, you have done the right thing and installed your choice of anti-malware and firewall solutions... is that going to protect you from all cyber-evil?  Sorry to say that it isn't.  What it will do is protect you against the majority of KNOWN threats and malware.  The problem for us all is the continued emergence of large volumes of new threats in the form of new malware and newly identified security flaws ("vulnerabilities" to use the euphemism du jour) in operating systems, software, software development toolkits, application frameworks and so on.

In a conversation with a colleague from one of the leading anti-malware vendors, I was prepared to learn that there was a pretty large volume of new malware emerging on a daily basis... As someone who has worked in the security field for some years, it was a surprise even to me that upwards of 70,000 new pieces of malware are estimated to be created daily and that new malicious websites appear on the worldwide web every 2 seconds or so.

[NOTE: in this context a "malicious website" is used to indicate a site to which users are drawn by subterfuge with the intention of duping them into allowing the installation of malware on their device.]

So, any reasonable observer will understand that while the "good guys" are pretty well resourced and very clever, it will take a certain amount of time to understand the new malware and create a means of identifying it (other wise known as a "signature" - for the majority of malware this takes the anti-malware vendors under 15 minutes) or to diagnose the security flaw and create a means of fixing it (otherwise known as a "patch").  These signatures and patches then need to be made available to you - the user community - and it is then down to you to make sure they are tested and installed... each step introducing a time delay between the emergence of the threat and you being protected against it.

Then there are the threats which have not yet been identified and diagnosed and therefore for which there is no immunisation or cure.  Amongst the most famous types of these are software security flaws and so-called zero-day malware.  Being realistic, experience over the past years shows clearly that there are more researchers looking for security flaws for nefarious purposes than on what we might call the good side of the force.  This, of course, means that the dark side know about them before those in a position to fix them, which creates a window of opportunity for exploitation.

Zero-day malware is buried in innocuous-looking files awaiting a defined trigger to start doing its thing.  The trigger might be a date and time (hence the name), but it could be anything.

So, to be confident that you will be aware of the occurrence of a security breach as it occurs, more than the basic firewall and anti-malware solutions will be required... you need to be monitoring what is happening on the infrastructure and actively looking for unusual patterns of behaviour which indicate that something untoward might be occurring.

A few years ago, such solutions were the exclusive preserve of the major enterprises and governments, but more and more cost-effective solutions and services are becoming available for the small and medium-sized business - it is these that will give that extra level of protection that is required, but only if they are properly configured, actively monitored, correlated and analysed by experienced professionals.

I'm going to go out on a limb here and walk through a scenario which assumes that such a monitoring solution exists and is reasonably effective - after all, nothing is perfect!

As the late, great Douglas Adams would no doubt have advised, the first thing to remember when a potential security breach is identified, is... 

"Don't Panic!"

... and one of the best ways of ensuring that you don't is to have a plan for responding to security incidents that you have practiced before.  It is also a very good idea to have some friendly experts on call who know you, know your business and are able to mobilise a professional response team at short notice to support your efforts.  And before you declare this as a blatant sales pitch, please consider this... your business is not going to stop just because you have a security incident.  Your customers will still need to be served, your suppliers engaged with and paid and your stakeholders are certainly not going to allow you to ask the world to stop while you figure out what to do!

In this day and age, the economic environment does not allow for an organisation to carry extra expert staff just in case an incident occurs...

Anyway, back to the plot.  You are now aware that you have a problem and you have not panicked... well done!

Security incidents generally fall into two main categories: accidental and malicious...

In either event, the full extent of the impact needs to be understood in order for damage to be reversed where possible or at least managed.  If the cause was accidental, someone is likely to require training.  If the cause is malicious, someone might need to be prosecuted.  In either event a key, fundamental principle is:

Preserve the Evidence

Instinct might move you to delete malware or files that have suddenly appeared on your infrastructure, or to unplug and reformat affected had drives... please don't... doing so will destroy evidence which might provide investigators to identify the source and effects of the incident.  

Imagine you are in charge of dealing with a break-in to your office... you would not (I hope) simply allow the office to re-open and normal operations to continue whilst the police crime scene officers attempted to find fingerprints and other forensic evidence on the desks and cupboards your staff are continuing to use, would you?  Unlike a physical crime-scene, you can take steps to allow the business to continue whilst preserving all of the forensic evidence, by seizing, isolating and making secure copies of all such evidence and ensuring the chain of custody is properly recorded.

Another piece of sound advice I have heard many times across the years is:

Don't confuse haste for speed!

Of course, it is important to understand what has happened in the shortest possible amount of time, but it is far better to be fast and right than hasty and wrong.  It is absolutely vital to follow trails of evidence from end to end, noting the tangents along the way for them to be either followed up by others or returned to later.

Every action, every piece of evidence, every decision, every conclusion and every individual involved in the investigation needs to be documented in a comprehensive timeline, catalogue of evidence and chain of custody in case it is required by legal eagles or law enforcement at a later date.

It is also important not to jump to conclusions... what appears to be obvious can often be deliberate misdirection - don't forget that if you are dealing with a genuine cyber attack, the chances are that your adversary is a thoroughly sneaky individual who really does not want to get caught.  The professional hacker is not only devious and paranoid, but in many cases highly skilled in creating diversions and distractions to put you off their scent.  There are no shortcuts when investigating cyber attacks - so every conclusion you draw should be tested again and again to ensure it is valid.


You know the saying... "if it walks like a duck, quacks like a duck..." and so on?  Well, here's a final thought on the subject of cyber attacks and incident management... there is a scenario where what appears to be a duck might, in fact, not be.  And this is probably something to verify as early in the investigation as possible as it could save everyone a great deal of time, effort and money... it is always worth checking whether any security (penetration) tests could have been taking place at the time of the alleged incident - try not to laugh this off... I have seen it happen that those responsible for scheduling penetration tests have not thought to inform those responsible for information security that a test was scheduled resulting in a significant amount of stress and money being expended on a redundant investigation!