The following story illustrates the challenges a chief information officer (CIO) might encounter when building a cyber resilience and response plan. While Martin Kinsley is fictitious, the nightmare scenario he experiences — rapidly spreading malware and data loss — is all too real for many organizations. Companies often believe their business-critical data is safely backed up — only to be met with permanent data loss. Read on to see what challenges Martin faces in his cyber resilience efforts and discover what choices he makes in response. What would you have done differently?

“So many people rely on us to get where they need to go,” said regional airline CIO Martin Kinsley to his team of IT leaders. The meeting was focused on cyberattack prevention, but he never missed an opportunity to discuss customer service. As he wrapped up the Friday afternoon session, he took care to emphasize the airline’s people-first values.

“Any mistake can affect people on a deeply personal level,” Martin said. “Missing flights means missing business meetings, birthdays, weddings — those are moments our passengers can never get back.” He felt proud as he wrapped up the meeting and returned to his office, as he’d worked tirelessly to convince the rest of the leadership team that security and cyber resilience needed to be a priority. While he knew the business continuity plan was a work in progress, the airline’s customer satisfaction scores had never been higher.

The IT team didn’t always receive the credit it deserved from headquarters leadership, but Martin was aware that the success of the team’s client-facing systems and infrastructure helped the airline maintain its multicity contract with a major air carrier.

No Rest for Weary Security Leaders

As Martin worked through his outstanding emails at the end of the week, he thought about how he had earned the short getaway that awaited him. With just a few tasks standing between him and a three-day vacation, he was ready for the break.

Just as Martin prepared to close his inbox, a new message came in from the airline’s security information and event management (SIEM) solution. He read the subject line — “High Alert: Network Security Incident” — and quickly realized the message was serious.

Most of Martin’s team was still in the office, so he asked them to assemble in the conference room immediately. He tasked them with investigating the notification and tracking down the cause of the security incident. Help desk calls began to roll in at the same time, and soon the team had the answer: Malware had infected hundreds of airport terminals.

Martin expelled a heavy sigh.

The help desk advised customer service agents to power down the terminals, but it was too late. Every endpoint was already infected and encrypted.

Cyber Resilience in the Face of Chaos

The malware had spread rapidly across his airline’s remote agent and passenger terminals over the past few hours. The infected terminals were now essentially bricks. Help desk employees were fielding calls from frantic airport employees complaining about angry passengers. To make matters worse, remediation attempts had failed since the malware’s encryption was airtight.

Not only was the malware spreading like wildfire, but the damage it inflicted was also focused on the largest airport in their region — a major international hub. Thousands of passengers at that airport were effectively grounded on the busiest weekday for airlines (and they were definitely unhappy about it).

All six of their airline terminals shared a network, which was segregated from headquarters’ networks. Forensics would come later, but Martin was fairly certain whatever strain of malware they were dealing with had the escalated account privileges necessary to spread damage to every one of the airline’s terminals.

Martin reached in his pocket to text Marina Petrov, the airline’s CEO, who was already fielding calls from his office with airport supervisory staff about emergency policies for granting vouchers and hotel rooms to angry travelers on the ground. He quickly typed to Marina that he was afraid the incident was getting even worse.

The team’s frantic efforts to contain the malware had failed, and they were now in full-on remediation mode.

By late Saturday morning, Martin’s prediction had come true. The malware had reached every endpoint on the airline’s terminal network — executing malicious code at all of their regional airports. The team’s frantic efforts to contain the malware had failed, and they were now in full-on remediation mode.

Marina stood in the door of the conference room where the IT team had created an impromptu security operations center (SOC). Martin informed her that — since all of the terminals had been infected and encrypted — his team had no choice but to start from ground zero, which meant establishing backups.

Failed Backups Lead to Business Continuity Disaster

“You’re saying the last usable backup we have is six months old?” Martin asked frantically. When system admin Fei Zhou nodded, Martin felt faint. His long-awaited weekend getaway had been replaced by the worst weekend he could have imagined. In fact, it was now early Saturday afternoon, and it had been nearly 30 hours since he’d seen his bed or taken a shower.

“The network-attached storage has been running idle since before I was hired — and no data’s been backed up,” said Fei.

After looking up and down through the directory and backup logs, Fei discovered the backup had stopped working the same week her predecessor left the company. She also saw the admin credentials had changed for the centralized network management tool.

Martin bit back the urge to ask why Fei hadn’t bothered to test backups (or do any other kind of digging) during her nearly five months at the organization. It was a definite failure on her part, but the current situation wasn’t any one person’s fault. It was a series of failures caused by everyone on the IT team.

Martin winced when he realized his worst-case fear of permanent data loss was achieved. He discovered the latest release of the reservation software was issued three months ago. So, his team could restore the backups — but they’d need to perform manual updates. The manual update process was long and grueling, but Martin and his team maintained their composure as they worked together. All the while, Martin scolded himself for not checking the backups himself.

Lasting Financial and Reputational Repercussions

“Too little, too late,” Marina said. Her words echoed in Martin’s ears after a Tuesday morning meeting with the leadership team. Martin’s team had demonstrated heroic behavior over the weekend, working tirelessly to restore backups to each of the terminals and manually update the reservation software. It was a painstaking process, but their hours of work paled next to the IT failures that had caused the issue in the first place.

The airline’s operations were just beginning to return to normal four days after the malware hit. News reports were scathing — and the chief financial officer (CFO)’s tentative projections of just how much the incident would cost were beyond grim. Union negotiations around pilot overtime revealed staffing costs into the millions. This number didn’t even begin to cover the costs of accommodating travelers over the weekend or the reputational damage the airline had suffered.

The average cost of a data breach is well over $3.62 million.

Martin knew the media would eventually forget the information technology incident, but he couldn’t say the same for the airline’s customers. Would they ever trust the company to get them where they needed to be again? Marina and the CFO had also alluded to rumors of heavy federal fines and a loss of contracts.

While the average cost of a data breach is well over $3.62 million, Martin knew this disaster was going to be far above average, even without the leak of data. He was certain the next chapter would include better information security safeguards and regular backup testing — but Martin had few other certainties about the airline’s future.

Embracing Proactive Cyber Resilience

Rapidly spreading malware that causes permanent data loss is all too common in the real world. In the past year, countless high-profile organizations have experienced long-lasting repercussions as a result of ransomware and malware spreading through their networks.

As Martin realized too late, his experience was the product of countless technical and human failures across the IT department. Although tasks were left undone for months on end, it wasn’t because his IT team wasn’t putting in hours or effort. Martin wanted to lead his organization toward a state of cyber resilience, but he lacked the expertise and resources to create an end-to-end strategy.

To avoid an expensive disaster, security leaders like Martin should consider onboarding resilience consulting services to design a business continuity plan and establish a central incident management hub instead of relying solely on a series of SIEM and network monitoring applications.

A resilience orchestration tool can help security teams automatically contain and respond to an incident across complex network structures.

A resilience orchestration tool can help security teams automatically contain and respond to an incident across complex network structures. And with the help of incident response experts, CIOs like Martin can also contain malware in the event of a breach and ensure that the security operations team does all the right things.

In addition, security leaders can invest in automated, cloud-based backup services to protect sensitive data and implement disaster recovery-as-a-service (DRaaS) tools to prevent a lasting IT outage.

A worst-case scenario can become a reality at any time — but it doesn’t need to result in regulatory repercussions or long-term damage to an organization’s reputation. With the systems and processes for proactive security response, CIOs can achieve confidence in their cyber resilience plans and remediation ecosystem.

Read more articles about incident response