Understanding the Costs of Incident Response
James Gimbi
Today we’re kicking off a series of posts, Understanding the Costs of Incident Response, that will walk through the various costs that organizations encounter during the incident response process. Over the next few weeks, we’ll look at each of these “buckets” of costs and breakdown what the expected costs would be based on the typical ransomware scenario. Cyber controls play a crucial role in limiting the frequency of attacks. However, what often goes unnoticed is their profound impact on the severity of an attack when it inevitably occurs.
Let’s consider two organizations: one, a typical ransomware victim where there are some controls deployed but others may be missing, misconfigured or there are legacy systems in place; the other, a well-prepared peer where controls are modern and deployed thoroughly and correctly.
Now, we understand that companies of different sizes or industries can have very different cyber risk profiles - a manufacturing company’s cyber risk profile will differ significantly from that of a hospital or a law firm. However, when we look at these seemingly identical companies who operate in the same space, engage in similar activities, and with a similar number of employees, the disparity in outcomes between the two becomes glaringly apparent.
Implementing robust and comprehensive controls can help not only limit the frequency of attacks by potentially preventing them in the first place but can also have a substantial impact to the severity and help reduce the overall cost of an attack by mitigating the damage that can be done should an attacker gain entry.
Over the next several weeks, we will share a series of posts that break down the costs across the full IR process that make up the $2.875 million estimate in the typical ransomware scenario shown in this post and then look at key security controls that can help organizations better prepare themselves to face the aftermath of a cyber-attack and make informed decisions that strike the right balance for their own unique risk profile.
- Investigation
- Ransom
- Recovery
- Data Mining + Notification
- Business Interruption
- Counsel + Litigation
- The Impact of Proper Security Controls
Investigation
Part 2 of 8
In the first post for our Understanding the Costs of Incident Response series, we shared a comparison of incident response costs between two peers: a typical ransomware victim where security controls are absent or misconfigured and a well-prepared peer. Now we’ll start looking at each of the major cost buckets across the incident response process.
Investigation Costs
The first costs that any victim will incur will be for the investigation – which in our example, accounts for ~$150k of the estimated $2.875 million claim arising from a typical ransomware event. These costs are what a lot of people think of when they think of cyber incident response but are just the tip of the iceberg.
Just think about it: Your infrastructure has been crippled, your network and storage have been ravaged by the attacker. And now, with limited IT resources, you’re tasked with collecting artifacts and evidence from potentially thousands of endpoints spread across multiple sites. It’s an arduous undertaking that requires speed, precision, and resourcefulness.
The scope of the attack needs to be determined: Is the adversary still present within the network? If so, how can we remove them effectively? Assessing the extent of compromised data and implementing containment measures that minimize further compromise are vital objectives. It’s at this point that many organizations begin engaging external specialists across multiple disciplines – forensics, recovery, data mining, legal counsel – to start answering key questions and building a plan for getting back to operational.
Now, it can be tempting for organizations to take a DIY-approach due to the costs associated with outside help. The harsh reality is that this process can be far more intricate and time-consuming than many initially realize and without the right support can become the biggest bottleneck in the entire incident response process. We see it everyday.
The investigation stage will set the foundation for the rest of the incident response process. Getting the actionable information you need to understand the scope of the incident is critical to building a game plan for how you can recover safely, but also as quickly as possible.
Ransom Payment
Part 3 of 8
The next big piece in understanding the costs of incident response can sometimes be the most controversial: ransom payments. Do you pay or not pay? Is the ransom demand proportionate to the severity of the recovery or the data that’s compromised? Will paying really make recovery any easier? Attackers have become increasingly sophisticated, targeting backups and wiping them clean, leaving organizations with limited options.
Adding to the difficulty answering these questions is the fact that ransom demands have risen significantly over the past couple of years – ransoms that were $50k jumped to $500k and we often see ransoms going into the millions, making this one of the costs that can be the most difficult to plan for.
So, what happens after a victim decides they are willing to pay the ransom? The first step is to engage professional investigators and negotiators. These experts, dedicated full-time to such negotiations, possess invaluable knowledge and a wealth of data to maximize positive outcomes. They employ a structured framework and strategic approach, leveraging personas and understanding different attacker groups to minimize the ransom amount.
Another crucial action these negotiators take is establishing real “proof of life.” It is essential to verify that the attacker can indeed decrypt your data after payment. After all, paying a ransom without assurance of decrypting your systems and recovering your data is a nightmare scenario. Negotiators also strive to minimize the initial demand, especially when facing more sophisticated groups that may demand a significant percentage of your annual revenue. By leveraging their expertise, negotiators can often achieve substantial reductions in the ransom amount.
It’s also worth noting that some ransomware actors actively target specific information within their victim’s environment to support their initial ransom demand. Attackers are not oblivious; they are adept at maximizing their profits. This underscores the importance of having seasoned professionals lead these negotiations to helpguide the conversation and find the best strategy to minimize the ransom amount.
While paying the ransom may be the only way forward in some cases, we want to dispel the misconception that paying it is like hitting the “easy” button and quickly solves the problem. You may get a decryptor, but there’s no guarantee that it’s going to work well. Keep in mind, a decryptor is software written by criminals who don’t have your best interests in mind – it’s not going to be fast; it’s going to be buggy and there’s no support or documentation. There’s still going to be significant effort and costs put into recovery, which will be the topic of our next post.
Recovery costs
Part 4 of 8
Today, let’s delve into the critical phase of the recovery effort following a cyber ransomware incident. Once the investigation has provided actionable insights into the necessary actions and safety precautions, the hard work of recovery begins. There are three primary paths to business-as-usual, each with its own unique set of challenges.
First, if you’re fortunate enough to have viable backups, the restoration process can commence. Even this seemingly straightforward approach takes more time than victims expect, especially when dealing with a vast number of servers and limited bandwidth. The painstaking task of validating and re-establishing communication between numerous services can significantly prolong the recovery timeline, which can be amplified if documentation is missing or if asset ownership is not well understood. But despite the turbulence, restoring from backups is far and away the fastest and most complete recovery path for most ransomware victims.
Another option involves leveraging the decryption tool provided by the attacker, assuming it functions as intended. As mentioned in our last post, there’s no guarantee that a decryptor will work well – they commonly have poor performance, bugs, and lack any formal support or documentation you can turn to in a pinch. We typically only recommend this route if there is no other way to restore or reconstruct critical operational data.
Lastly, if backups are unavailable and acquiring a decryptor is not an option, organizations may need to embark on the challenging journey of rebuilding their environment from scratch. This arduous process can involve various tactics, such as data reconstruction efforts, where individuals extract information from sources like email inboxes, hoping to approximate the original data. These endeavors are far from foolproof and rarely achieve a perfect recovery state. However, rebuild efforts can be an opportunity to simplify an IT footprint and can reduce technical debt.
It’s crucial to acknowledge that some degree of data loss is inevitable in most cases, regardless of the chosen restoration method. The key consideration is whether the lost data holds critical importance to the organization’s operations and stakeholders. MOXFIVE recommends utilizing the 3–2–1 rule - three copies of the data, on two distinct forms of media, and one offline or offsite backup.
We also recommend having a defined recovery plan in place with clear roles and responsibilities for team members and a prioritized list of servers and applications to work from as your one source of truth. Having an incident response and recovery plan that your team is familiar with can not only keep everyone focused on the right priorities and reduce stress but can reduce recovery and business interruption costs as well.
In the next phase of our discussion, we’ll explore the multifaceted aspects of managing third-party relationships and regulatory obligations during the recovery process.
Data Mining + Notification
Part 5 of 8
For the fifth post in our series on Understanding the Costs of Incident Response, we look at a critical aspect of incident response that often comes with substantial costs and complexities—data mining & notification. Attackers are adept at maximizing the pain inflicted on victims, making data exfiltration a common component of their attacks.
Beyond in-house recovery efforts, victims who have experienced a data breach need to quickly determine whether third parties might be affected. There may be distinct legal and contractual obligations to employees, beneficiaries, customers, business partners, regulators, and more.
Satisfying these obligations requires a thorough understanding of accessed, damaged, or stolen data. Unfortunately, organizations rarely have the context or pre-existing controls to streamline that impact analysis. Rarer still is an authoritative inventory of affected third parties and their contact information. This is where a data mining exercise become essential.
Data mining helps victims unravel the complex coil of affected data and the third-party obligations that data invokes. Combing through the troves of information that attackers may have stolen – whether structured data in spreadsheets or unstructured content like scanned documents and images – is typically time consuming and resource intensive. While this process can be reduced by high fidelity investigation and negotiation findings, the cost ultimately depends on the structure and volume of impacted data.
Once the affected parties are identified, organizations also need to create a comprehensive notification plan. This often includes formal notification letters, call centers to handle inquiries, and a thoughtful communication strategy. It is important to note that notification is usually required even if a ransom is paid to suppress the release of data. MOXFIVE encourages organizations to engage experienced third-party breach and privacy legal counsel to understand their obligations.
Traditionally, data mining has been a very manual process that demanded large analytical teams to give each document human attention. Now, effort and cost can be dramatically reduced by working with purpose-built AI/ML tools and dedicated teams. On a recent case, MOXFIVE worked with a large medical client to process over 15 TB of data in one week, cutting the notification timeline from six months to just one and overall costs by 50%. This client would not have been able to meet regulatory deadlines without this more modern approach.
Data breached are minefields fraught with nuanced obligations, technical challenges, and hidden financial setbacks. Organizations must not only focus on recovering their systems but also on managing relationships with affected parties, navigating business interruptions, and quantifying losses. This multifaceted challenge underscores the importance of proactive cybersecurity measures, incident response planning, and building a resilient organizational framework that can weather the storm of data breaches and cyber-attacks.
Business Interruption
Part 6 of 8
One of the largest costs resulting from a cyber incident can sometimes be particularly challenging to calculate: the cost of business interruption. In this blog, we’ll delve into the intricacies of calculating this cost and emphasize its vital role in incident response planning.
Business interruption can be defined simply as the income loss experienced as a direct result of the incident and is often a significant element in insurance claims. For example, let’s consider a manufacturer or a professional services organization that falls victim to ransomware. There’s no way to recover the lost revenue if they can’t produce or bill their clients on a given day due to downtime from the ransomware attack. Absent a time machine, those hours are lost forever and irreplaceable and would directly contribute to business interruption calculations.
While that example is intuitive, assessing actual business interruption costs can be deceptively complex. Let’s contrast that example with an online retail business that suffers a similar attack. Downtime caused by the attack prevents the business from processing orders, which you would expect to lead to lost sales. However, some number of customers that couldn’t place orders during downtime will simply place their order when the website is back online. In another example, reputational damage might cause the loss of a significant deal. But that, too, is very difficult to definitively tie to the incident. These nuances make precisely quantifying lost revenue challenging, complicating insurance claims. In fact, accurately determining reasonable business interruption costs often requires the involvement of forensic accountants.
While interruption costs will vary from business to business, one principle is always a factor: the more complex and time consuming the first four response costs are (recall: investigation, ransom payments, recovery and data mining/notification), the more significant business interruption costs will be. Proactively investing in response and recovery capabilities will also serve to minimize your organization’s exposure to interruption costs.
As ransomware incidents continue to threaten businesses of all sizes and sectors, proactively assessing security posture, implementing cybersecurity capabilities, and establishing comprehensive incident response plans are paramount. By preparing for potential business interruptions and costs, companies can better safeguard their finances and mitigate ransomware’s detrimental effects.
Counsel + Litigation
Part 7 of 8
When an organization faces a cybersecurity incident, the immediate concerns often revolve around investigating the breach, mitigating the damage, and getting IT services back on track. However, there’s a hidden cost that lurks beneath the surface - legal expenses. In this blog, we will explore the nuances of counsel and litigation costs in incident response and why they should be an integral part of an organization’s cybersecurity planning.
From the start, specialized attorneys and law firms play a pivotal role throughout the response process. Often called privacy counsel or “breach coaches,” these specialists help organizations handle intricate risks and obligations that in-house counsel may not be equipped to tackle. External legal teams provide several essential services during a response effort, including:
Attorney-Client Privilege
Engaging service providers through external counsel may protect incident-related communications and documents with attorney-client privilege. This legal safeguard helps shield misleading or potentially damaging material from discovery processes in downstream legal proceedings. But this privilege umbrella is not automatic! Teams should carefully follow counsel’s guidance for written communications and document production.
Navigating Legal Obligations
Law firms commonly support dozens, if not hundreds,] of data breaches and cyber-attacks each year. This experience helps external counsel navigate regulatory and contractual obligations across the complex landscape of jurisdictions and verticals. In-house counsel are not likely to be as familiar with current regulator expectations and how best to shape a technical response effort to support them.
Communications
Careful communication strategies are critical during an incident. Expert counsel can help shape effective comms strategies for internal and external stakeholders without introducing new risks. They may engage specialized public relations companies to work at their direction and often coordinate mandated notifications.
Stakeholder Liaison
Expert counsel are often effective liaisons for clients that need to balance response efforts with engaging third parties like law enforcement, insurance providers, and business partners. Counsel can support third-party needs without disrupting sensitive response efforts and can help victims understand what third party support they can realistically expect.
Litigation support
Unfortunately, legal costs don’t stop with the technical response effort. The meteoric rise of cyber extortion attacks, coupled with ever more rigid and complex data breach regulations, is fueling a surge in data breach litigation. Litigation can extend the already lengthy tail of incident response costs and many organizations find themselves dealing with legal engagements for months or even years after the incident.
Just like business interruption costs discussed in our last post, legal costs scale with the size and complexity of the incident. In fact, larger incidents may require engaging multiple law firms with niche material or jurisdictional expertise. Contemplating legal costs and challenges should be part of every organization’s incident response plan. Organizations should understand the legal providers and services covered by their cyber insurance policy and spend time focusing on data governance to get ahead of costly notification obligation surprises.
After a breach, organizations may find themselves navigating uncharted legal territory, where the potential for litigation looms large. Seasoned legal cybersecurity specialists can help navigate the intricate web of legal obligations and minimize the impact of litigation. Engaging expert counsel should not be a back-end afterthought; it’s a front-end strategy that can protect victim organizations’ reputation and save them from costly missteps.
The Impact of Proper Security Controls
Part 8 of 8
To round out our series on Understanding the Costs of Incident Response, we will discuss the impact proper security controls can have on both the frequency and severity of cyber incidents. In our first blog, we looked at two organizations: one, a typical ransomware victim with legacy or absent controls deployed; and the other, a well-prepared peer with modern, thoroughly deployed controls. In these two scenarios, the estimated incident response costs for the well-prepared peer are significantly reduced: an estimated $650k vs. $2.9million for the typical victim. How did the well-prepared peer achieve such savings when facing the same incident profile?
Controls can limit the impact of cyber incidents in two ways – by preventing attacks in the first place (preventative controls), or by reducing the severity and cost of a successful attack (mitigatory controls). Some controls focus solely on one front, while others contribute to both. Let’s consider how five common controls fall into this spectrum.
Preventative
Email security products detect and filter social engineering attempts, malicious attachments, and harmful links to prevent initial compromise, the first stage of the attack lifecycle. While important, they will not limit the “blast radius” of a successful attack.
Preventative Lean
Multifactor authentication (MFA) requires a second factor beyond passwords, like possession of a key or bio-identifier, before access is granted. This addresses risk presented by user password hygiene and account compromise. MFA plays a huge role in preventing unauthorized access to sensitive services like VPN and email. While MFA can also limit the impact of a successful attack, prevention is the primary use case.
Preventative + Mitigatory
Endpoint Detection & Response (EDR) has two core features: the first is next-generation antivirus that prevents malicious activity from running on endpoints, a fundamental prevention control. But EDR also supports incident response efforts, allowing cyber response teams to quickly react to malicious behavior and extinguish attacker access. Endpoint protection is a great example of a control that plays both preventative and mitigatory roles.
Mitigatory Lean
Privileged Access Management (PAM) tools command accounts that have elevated privileges in your IT environment, preventing account abuse and poor internal password hygiene. While PAM can prevent an attacker from establishing a foothold (de-facto prevention), the primary use case is limiting the blast radius of a successful attack by preventing privilege escalation and, often, lateral movement.
Mitigatory
Backups allow organizations to quickly recover from a destructive event like a ransomware attack. Immutable backups can be one of the most significant cost savers during an incident, as they can eliminate the need for ransom payments and protracted recovery efforts. While backups are indisputably an elementary control for any security program, they are a wholly mitigatory control.
No control discussion is complete without emphasizing the importance of robust implementation and adoption. Almost no cyber control is “set it and forget it”, and poor deployments give false confidence that can be worse than missing the control altogether. As a for-instance, almost all our incident response clients have some sort of backup system, but nearly 60% of those backups are unusable during the incident due to incomplete collections or poor defense of the backup system itself. Do not let legacy products or complacency undermine the effectiveness of your risk investments.
True cyber resilience demands a defense-in-depth strategy with a combination of preventative and mitigatory controls. Organizations that focus on implementing a holistic approach tailored to their specific needs instead of a one-size-fits-all solution can significantly reduce the frequency and costs associated with cyber incidents.
Planning isn’t passive! We recommend building a thoughtful roadmap with several phases to be sure your cybersecurity program has the breadth and depth to stand resilient in the face of today’s threats.