“Pipedream” Malware Targets ICS: What Critical Infrastructure Owners Need to Know

Troubling new malware designed to facilitate attacks on a wide array of critical infrastructure – from oil refineries and power plants, to water utilities and factories – is raising concerns for its versatility. The malware, named Pipedream by Dragos and Incontroller by Mandiant, who have both tracked and researched the toolkit, is potentially capable of gaining full system access to multiple industrial control systems (ICS) and supervisory control and data acquisition (SCADA) devices.

Fortunately, there is no evidence yet that the malware has been successfully deployed in the wild, but the threat it poses to critical infrastructure is severe enough to warrant an advisory from multiple federal government agencies. This joint advisory was issued by the Department of Energy (DOE), the Cybersecurity and Infrastructure Security Agency (CISA), the National Security Agency (NSA), and the Federal Bureau of Investigation (FBI). It says:

APT actors have developed custom-made tools for targeting ICS/SCADA devices. The tools enable them to scan for, compromise, and control affected devices once they have established initial access to the operational technology (OT) network. Additionally, the actors can compromise Windows-based engineering workstations, which may be present in information technology (IT) or OT environments, using an exploit that compromises an ASRock motherboard driver with known vulnerabilities. By compromising and maintaining full system access to ICS/SCADA devices, APT actors could elevate privileges, move laterally within an OT environment, and disrupt critical devices or functions.

There have only been a handful of known and credible malware threats designed to specifically target critical infrastructure. The first such example is Stuxnet, which was uncovered in 2010 and was developed and used by the U.S. and Israeli governments to destroy nuclear enrichment centrifuges in Iran. In 2016, Industroryer (also known as Crash Override) was used by Russian actors to target electrical infrastructure and force blackouts in Kiev, Ukraine. Triton or Trisis was discovered in 2017, and again used by Russians to target Saudi Arabian oil refineries. Most recently, Ukrainian security officials detected a new variant of Industroyer linked with the current Russian offensive, just a few weeks ago.

Since Stuxnet opened the door to malware targeting critical infrastructure more than a decade ago, these are the most prevalent instances to be uncovered. And without even recording proof of it being deployed in the wild, Pipedream/Incontroller already stands apart because it can manipulate such a wide variety of industrial control programmable logic controllers (PLC) and industrial software used across industries.

In their joint advisory, DOE, CISA, NSA, and the FBI urge critical infrastructure organizations to implement a series of detection and mitigation recommendations to strengthen their security posture against the Pipedream/Incontroller threat. Among the 13 recommended steps outlined, XONA already naturally provides organizations with eight of them, including:

  • Isolating ICS/SCADA systems and networks from corporate and internet networks using strong perimeter controls, and limiting any communications entering or leaving ICS/SCADA perimeters.
  • Enforcing multifactor authentication for all remote access to ICS networks and devices whenever possible.
  • Limiting ICS/SCADA systems’ network connections to only specifically allowed management and engineering workstations.
  • Implementing robust log collection and retention from ICS/SCADA systems and management subnets.
  • Ensuring all applications are only installed (accessed) when necessary for operation.
  • Enforcing principle of least privilege. Only use admin accounts when required for tasks, such as installing software updates.

For more information on how XONA natively includes these protections for its customers and to learn how our technology can protect your organization, visit our resources page or reach out to schedule a demo.

Understanding ISA/IEC 62443 Standards for Industrial Networks, OT, and Critical Systems

There are many significant technology-enabled changes taking place in industrial environments today. Smart factories and Industry 4.0. The Industrial Internet of Things (IIoT). The convergence of information technology (IT) and operational technology (OT). All of these things are introducing digital technologies at a fast pace to improve operations, increase productivity, enhance oversight, and increase profitability.

For all the good the technologies offer, there’s also a dark side that opens up the digital environment to vulnerabilities that can enable cyberattacks, theft of intellectual property, and even cyberwarfare.

The threats and concerns of attacks on industrial systems are clearly evident by the recent Biden Administration and the Cybersecurity and Infrastructure Security Agency (CISA) warning that Russia has been conducting “preparatory activity” for cyberattacks, including scanning websites and hunting for software vulnerabilities, and could attack any critical infrastructure segment in the U.S.  The Administration urges owners of critical infrastructure to conduct cyber risk assessments, implement multi-factor authentication, keep software and malware protection up to date and educate employees on the threats. The 62443 standards provide a framework of controls to mitigate the risk of these types of attacks.

It is this deep concern about security vulnerabilities that led several industry regulators to collaborate on the development of a series of standards that create a flexible framework to address and mitigate current and future security vulnerabilities in industrial automation and control systems (IACSs). The main collaborators in this effort are IEC TC65 / WG10, ANSI / ISA-62443, and ISO / IEC-JTC1-SC27. The standards they came up with, known collectively as ISA/IEC 62443, are applicable to all industry sectors and critical infrastructure.

Due to the comprehensive nature of ISA/IEC 62443, the standards are very broad and are presented in 14 separate documents, organized as shown below in Figure 1. They cover a wide range of topics from terminology, concepts, and models to security technologies for IACS, and much more. The standards are written for various audiences, including plant operators, integration and maintenance service providers, and component/system manufacturers.

In terms of the broad aspects of the standards, XONA provides capabilities for security requirements in the three areas highlighted in green, below.

Documents for ISA/IEC 62443

Figure 1 – Documents for ISA/IEC 62443

Trust us when we say the standards are very broad and deep. It took us weeks to scrutinize every requirement to determine if, and how, XONA supports the standards. The truth is, no single component or system manufacturer can claim to cover every single requirement—the needs are just too diverse. The standards were written to go across multiple technology providers, which explains why system integrators are one of the target audiences of the document: someone needs to put the diverse pieces together to help a company achieve full coverage.

Practitioners and customers often ask if we “comply” with ISA/IEC 62443. This is a bit of a misnomer, as 62443 is not a regulation mandated by a government or industry agency, such as NERC-CIP is for the energy industry. Instead, 62443 is a set of recommended standards that can help companies with industrial automation and control systems protect and secure those systems. Our customers seek to confirm compliance as they have adopted 62443 as a corporately mandated cyber security standard That said, XONA security capabilities and features meet the foundational and security level requirements of the relevant 62443 standards and fulfill the compliance requirement.

Meeting ISA/IEC 62443 Standards

Leading industrial organizations worldwide trust XONA for secure user access and analytics for their critical systems. XONA provides granular user-to-asset access controls and user session analytics via a zero-trust architecture. By integrating with OT asset management and security information and event management (SIEM) platforms, XONA adds the essential user-to-asset access control and analytics components needed in industrial infrastructure today.

Given these capabilities, it’s a natural fit for XONA to address various aspects of 62443, specifically around access control, identification and authentication control, use control, data confidentiality, and least privilege. These fall within XONA’s functionality and areas of expertise for securing industrial networks and systems. An important consideration is to select technology that aids in meeting and staying compliant and not undo any security countermeasures.

Because of the complex and detailed nature of these requirements, we’ve created a datasheet that explains which of the requirements XONA meets and how. Download it now:

Download Now

If you’d like to discuss how we fulfill those requirements and can help your organization improve user access control, schedule a demo today.

 

US Officials Warn – Heightened Risk of Ransomware Attacks on Municipal Utilities

U.S. Critical Infrastructure must guard against malicious ransomware attacks by implementing standards-based encryption and multi-factor authentication at all access points to OT assets 

U.S. officials warn of potential ransomware attacks in response to increased sanctions on Russia and have asked state and local officials to consider how ransomware attacks could disrupt the provision of critical services. “Right now, the biggest concern we have are preparations for potential impacts to US utilities and industrial critical infrastructure.” (Dragos)

The threat of Ransomware attacks is emerging as a critical cyber risk for electric utilities in the United States as evidenced by the recently passed Infrastructure Investment and Jobs Act (“Act”) Public Law 117–58.  The Act specifically provides grant funding for municipal utilities to deploy advanced cybersecurity technologies to protect against, detect, respond to, or recover from a cybersecurity threat to enhance the security posture of electric utilities. 

Utility owners should consider implementing a Zero-Trust secure operational gateway for user access with Multi-Factor Authentication (MFA) for encryption and authentication at the critical assets to block hackers from gaining access to their industrial control system. Regardless of how a hacker attacks the networks, or OT access points, encryption at the OT asset mitigates the ransomware attack. 

The XONA Critical System Gateway (CSG) was explicitly designed to provide Zero-Trust secure user access for the OT environment. Our CSG directly addresses the requirement for encryption and authentication through hardware token-based multi-factor authentication (MFA), user session recording, user-to-asset monitoring, OT protocol isolation, encrypted screen remoting, and auditable connection logs. 

XONA CSG provides a simple and secure solution that can be deployed and functioning in less than a day to harden OT access connections securing critical infrastructure. 

“Shields Up” Strategy – the New Reality for U.S. Critical Infrastructure

U.S. Critical Infrastructure must guard against malicious cyber-attacks by implementing encryption and authentication at all access points for connected OT assets or continue to face an increased level of cyber risk.

Russian hackers are attempting to broadly penetrate Ukrainian infrastructure to disrupt critical services such as electricity, transportation, finance, and telecommunications.

US Government urges US Critical Infrastructure owners to harden their systems and implement a “shields up” strategy.  As tensions escalate, Russian cyberattacks could seek to disrupt US electricity, gas, and other systems, warn the FBI and Department of Homeland Security.  Biden says, ‘we are prepared to respond if Russia launches cyberattack against the US.’

OT systems need to implement a Zero-Trust secure operational gateway for user access with Multi-Factor Authentication (MFA) for encryption and authentication at the asset connection to stop the attack before gaining access to an industrial control system.  Regardless of how a hacker attacks the IT systems,  networks, or OT access points, encryption at the OT asset mitigates the attack.

The XONA Critical System Gateway (CSG) was explicitly designed to provide Zero-Trust secure user access for the OT environment. Our CSG directly addresses the requirement for encryption and authentication through hardware token-based multi-factor authentication (MFA), user session recording, user-to-asset monitoring, OT protocol isolation, encrypted screen remoting, and auditable connection logs.

8 Immediate Risk Mitigation Steps to Protect Critical Infrastructure Systems

  • Identify all data communication protocols communicating on the OT network (East-West) and from OT Network to IT Network or Internet (North-South)
  • Ensure all communication from IT/Internet to OT network is encrypted
  • Ensure no data-in-transit for any user sessions not associated with a multi-factor authenticated session.
  • Isolate all data communication protocols to OT network
  • Ensure all user access session data to critical OT systems is logged and recorded.
  • Ensure plant-level controls for allowing remote access through software “lockbox and virtual wait lobby including visual and audible alarms.
  • Monitor all non-read only user access sessions
  • Verify acceptable risk level for access to critical assets through asset monitoring, threat (IOC) feeds, and vulnerability detection tools.

XONA CSG provides a “shields up” solution that can be deployed and functioning in less than a day to harden OT access connections securing critical infrastructure.

The Ideal Simple and Secure Connection Solution for OT Remote Access

Industrial companies worldwide are adopting capabilities that allow for remote operations. The pandemic has led companies to consider how they can reduce an onsite workforce while continuing with normal operations. Likewise, the worker shortage is leading companies to think in terms of a flexible workforce that may include remote staffing and flexible resourcing. In addition, industrials must think about emergency preparedness, control procedures, and the need to operate reliably with reduced onsite staff.

While the benefits to remote and mobile access are multifaceted, the risks to critical systems are real.

When workers and third-party vendors use remote communication technologies to directly access critical OT systems, the attack surface can be huge. A malicious actor can insert himself into the communication channel, overtake a legitimate user’s credentials, and utilize the data protocol such as RDP on the remote user’s device (i.e., a man in the middle attack) as a launching point to get into the OT network. The attacker can then move laterally to find vulnerable systems.

Quite obviously, this situation is completely untenable, and it is the scenario XONA Systems was created to solve. XONA prevents direct access to the network assets and includes multifactor authentication and an encrypted connection to mitigate the threat of man in the middle attacks on protected networks.

Remote Access Without Compromise of Essential Security

Operators need to provide remote and mobile workers with secure and managed access to operational controls as if they were there in person. This type of remote access absolutely requires a zero-trust architecture, control room managed access, and other operational safeguards. XONA’s Critical System Gateway (CSG) is purpose-built to fill this exact need.

XONA has a very simple architecture, shown below, that implements a zero-trust strategy to control, log and monitor the connection between the remote end user and the trusted asset in the OT network. As it relates to access, the questions of who, what, where, when, and why all are predetermined using a CSG appliance. Having this very well-defined architecture in place, companies with critical OT systems can now enable remote or mobile access without compromise of essential security.

Here’s what it takes to implement this remote access solution.

On the user side:

A remote user can use a variety of device types. The device itself does not need to run any sort of software agent; however, it must be able to support the use of a physical key for multi-factor authentication (MFA). The user can connect via a virtual private network (VPN), but it isn’t necessary. The user can utilize any modern browser to gain access to the XONA CSG by directing the browser to a predetermined IP address, which connects into a specific port on the CSG.

On the OT side:

The target asset on the operational side can be anything that an operator would normally interact with—a SCADA system, PLCs, HMI servers, etc. These devices are defined, and a specific protocol is assigned and isolated by the XONA CSG appliance.

The XONA CSG:

The appliance has two ports—one to connect the user side and the other to connect the OT side. Once those connections are made, an administrator can configure the software to set up user accounts and profiles that determine who can access which assets (systems) or applications on the OT side, and when. These profiles are used to authenticate and authorize specific users or groups to each system.

In all, it can take as little as 30 minutes to install the XONA solution.

What XONA CSG Brings to Remote Access for OT Systems

CSG is the world’s first – and of course, best – zero-trust remote operations platform for critical infrastructure and other industrial facilities. XONA delivers a trusted solution with very unique features for a changing world.

Zero-Trust Architecture (ZTA) – First and foremost is XONA’s ZTA for access control. By definition, zero trust means that every entity, be it a person or a machine, inside or outside a network, must be authenticated, authorized, and continuously validated before gaining and maintaining access to a protected system, application, or data. The XONA platform adheres to the ZTA requirements outlined by NIST, which recommends a Policy Enforcement Point (PEP) for enabling, monitoring, and eventually terminating connections to a protected resource.

The XONA CSG provides this PEP between the IT and OT enterprise or for any connected assets. The CSG directly mitigates cyber risk and physical security gaps that are prevalent in the OT environment. These security features are extended to include any remote access to connected assets.

Multi-Factor Authentication (MFA) – MFA is required to connect a user’s device to the XONA CSG. The gateway allows entities to seamlessly authenticate with WebAuthn-, U2F-, or OTP-compliant hardware tokens. It can also integrate with a company’s legacy MFA solution.

Protocol Isolation – For those OT devices that communicate using any of three major protocols – RDP, VNC, or SSH – XONA is able to isolate those protocols to the OT network, so they are not utilized by the remote user on the untrusted user side. What the user interacts with are image files rather than the actual OT protocol, but they still have complete control of the OT device as if they were sitting in front of it. XONA uses proprietary technology to do this and is the only remote access company to have such a capability.

Agentless Access – For convenience, the XONA solution is clientless and browser-based, which means that the remote/mobile user can use any device without having to use plug-ins, agents, or client-based software. This kind of simplicity is especially important for third party access as well as emergency use situations where a user device doesn’t have to be pre-configured.

Logging and Recording – To help ensure security and compliance, XONA has user access logging, event logging, and screen recording of every action that is performed. These logs/recordings can be used for compliance purposes and for worker training.

Moderated Secure File Transfer – XONA provides the ability to send or receive files back and forth between the user and the OT asset. The file transfer can be configured to be bidirectional or unidirectional for a number of reasons, such as patching the asset or to pull log files from the asset. With moderated file transfer, the file is stopped at the CSG and an administrator must approve or deny the movement of the file from that point. It’s another layer of security checks and balances.

These and all other XONA features map to relevant NERC CIP controls and are compliant with other standards such as IEC 62443 and NIST 800-53.

In short, the XONA CSG is the ideal platform to serve as a simple and secure connection solution for secure remote or mobile plant operations.

Download the “The Power of XONA: Supporting Operational Technology’s Cybersecurity Mission” white paper to learn more >

Understanding the Unique Challenges of Securing OT Systems in 2022

As industrial organizations continue to embrace change by leveraging the latest technologies into their daily operations and production cycles, they have also been tasked with embracing remote and hybrid work environments – all while maintaining operational continuity.

Utilizing advanced technologies has enabled these organizations to reduce expenses, expedite production time, and elevate customer service levels. At the same time, the global pandemic has accelerated remote and hybrid operations that allow employees, contractors, consultants, and vendors to “operate on-site” anywhere in the world, as well as via a variety of digital devices.

Unfortunately, along with the many benefits of delivering new value and improving productivity through technology and shared operations come escalating OT security risks that can impact – and even severely harm – workers, reputations, and operations. Cyberattacks on OT systems are no longer a niche exploit and can be catastrophic. Today, no organization in the OT environment is immune.

Accelerating OT Infrastructure Targeting

There has been an explosive growth in OT infrastructure targeting in the past few years. IBM Security’s 2020 X-Force Threat Intelligence Index reports a 2000% increase in the number of events targeting OT assets since 2018. Even more daunting is the rapid evolution of OT attacks from immediate critical infrastructure disruption – such as the Colonial Pipeline ransomware attack – to the Oldsmar, FL municipal water treatment’s network hacking attempt to cause physical harm by increasing the sodium hydroxide in the water intake. The new reality is that today’s threat actors are targeting weaknesses in the OT environment through open ports, lack of proper OT network segmentation, lack of MFA on access points, and back doors opened by third party vendors.

Recently, the technology research and consulting company Gartner predicted that the financial impact of OT attacks will reach $50 billion by 2023, including a variety of costs from insurance, regulatory fines, litigation, and compensation. They also forewarned that most CEOs would be personally liable for such incidents.

To combat the range of risks before an incident occurs, industrial organizations must adopt a forward-thinking OT security strategy that addresses these upward trends of the modern world.

Protecting Critical OT Assets

No longer can organizations wait to put processes, procedures, and technologies in place to protect their critical OT assets and remain secure and operational. Manufacturers, energy producers, utilities, and other organizations that deal with the public sector need to turn to a simple to deploy zero-trust access control platform with capabilities that include:

  • Secure “clientless” browser-based multifactor authentication (MFA)
  • Secure operational link for Industrial Internet of Things (IIoT)
  • Role-based third-party vendor management
  • Secure application access for monitoring and session logging
  • Application screen recording for forensics and training
  • Centralized management, visibility, and control of authorized user access

Securing OT Demands a Platform Approach

Since security considerations must extend beyond the on-premises system, a user access control and analytics platform is essential in mitigating cyber risk and physical security gaps prevalent in covering the operating system, the network infrastructure, and the IIoT.

The development of a unified security strategy should also include asking the following questions to help identify and evaluate solutions that are simple, proven, and cost-effective:

  • Does the vendor have a deep understanding of the nuances in cybersecurity, safety, and reliability challenges being faced by the OT industry?
  • Does the vendor have an established ecosystem of strategic partners, technology alliance partners, and resellers committed to reducing risk, cutting costs, and improving public safety?
  • Is the vendor able to implement robust and compliant network segmentation between IT and OT networks?
  • Does the vendor offer a centralized management platform designed to provide a single point of management and a 360-degree view across all remote sites?
  • Is the vendor able to meet even the most stringent compliance standards, including NIST 800-53, FIPS 140-2, and Risk Management Framework (RMF) guidelines?

Getting the answers to these and other essential questions will help guide critical infrastructure operators in taking the first steps toward improving their functional resilience and protecting their critical assets through a secure operational link between IT and OT.

Please click here to learn more about taking proactive steps to harden an OT environment>

Consequential, Certain & Disruptive: 3 Cybersecurity Risks that Will Impact Operations in 2022

2021 was a challenging year for manufacturers, energy producers, and utilities. A chaotic pandemic year created an opportunity for threat actors to take advantage of disruption to infrastructure integrity and IT to OT operational dependencies, something they achieved with frightening rapidity and effectiveness.

As many organizations transitioned to a hybrid workforce, novel integrations between IT and OT systems created new vulnerabilities that threat actors exploited, leading to surging ransomware attacks, infrastructure compromise, and other problematic repercussions.

According to one industry survey, 63 percent of respondents indicated that their organization experienced an ICS/OT cybersecurity incident in the past two years. With the average ICS/OT cybersecurity incident costing companies nearly $3 million, organizations have plenty of reasons to improve their defensive posture in the year ahead.

It’s critical that they do. Manufacturers, energy producers, and utilities should not expect heightened cybersecurity risk to subside alongside the pandemic. Instead, they should expect OT-related cybersecurity threats to be a certainty — and more expensive, consequential, and disruptive in the year ahead.

Expensive

As last year’s Data Breach Investigations Report glibly notes, “money makes the cyber-crime world go round.” In 2022, that price is going up.

For example, in 2020, the average ransomware payment exceeded $200,000, nearly four times the amount from just a year prior. In 2021, several high-profile ransomware payments netted multi-million dollar payouts as organizations and utilities worked to restore system access as quickly as possible.

Organizations should expect ransomware demands to continue increasing in the year ahead. Meanwhile, opportunity cost, regulatory implications, and other factors are making cybersecurity failures increasingly expensive. Therefore, timely and effective investments in holistic defensive capacity are essential to mitigating the financial implications of a cybersecurity incident.

Consequential

In 2021, cybersecurity failures halted manufacturing operations, exposed sensitive data, and eroded brand reputation – significantly raising the stakes for companies of every size in every sector.

Moving forward, companies should expect that the consequences of a cybersecurity incident will be more severe than ever before. For example, ransomware gangs are increasingly looking to leverage their network access to acquire and leak sensitive company data. Data exfiltration incidents surged in 2020, something that will inevitably continue in 2022.

Most prominently, when utilities and energy producers are compromised, public safety is often at risk as threat actors can disrupt critical services. It’s clear that without proper cyber protection, the consequences of failure are likely to become more extreme each year.

Disruptive

In November 2021, the Federal Bureau of Investigation (FBI) released a memo to companies completing “time-sensitive financial events,” warning that ransomware gangs are targeting these companies, looking to capitalize on the urgent and public nature of their operations. This warning most prominently applies to the financial sector, where mergers and acquisitions are time-sensitive, and public events, which can be derailed by a ransomware attack.

However, given the prominent attacks on critical infrastructure in the past year, it’s likely that threat actors will look to exploit companies and municipalities with time-sensitive operations, hoping to capitalize on the high-stakes nature of their sector to maximize payment opportunities.

Implementing Solutions That Work

Recognizing the immense challenges posed by today’s cybersecurity threats, manufacturers, energy producers, and utilities should turn to a simple to deploy zero-trust access control platform that can keep companies secure and operational, especially when IT and OT platforms are united.

Taken together, it’s clear that cybersecurity needs to be a top priority for every company in 2022, and they should start preparing today to meet tomorrow’s challenges.

Getting to Resilience

When I turned 7, I got my first BMX bike. Of course, within a week my best friend and I built a ramp with plywood and cinderblock. I remember the first jump vividly. I sped down the street like a miniature Evil Knievel and hit the ramp at a pretty good clip. A moment after I caught “big air,” my front tire hit the road, and I went over the handlebars – leaving a fair amount of skin on the road.

Clearly, the operational process of pedaling the bike up a ramp and into the air and landing was not done the right way. The data was clear. All I had to do was look at the blood on my knee and my stinging hands and recognize that I needed guidance. Fortunately, there was another older kid on his bike who was watching the whole thing and with the wisdom of Socrates said, “you have to lean back when you jump.”

This was the moment I learned about resiliency. I not only found out that I could endure adversity, but I now had knowledge to recover and make sure that the next time I went off that ramp I would likely stay on the bike…though wearing knee pads also would probably not be a bad idea.

Over the last 18 months, we have all learned more about resiliency. Large corporations have gone remote practically overnight, and our critical industrial sectors have had to adjust as well to limited travel schedules, while also needing to protect OT assets and interdependent IT systems from nefarious threat actors.

Recent shutdowns of these systems due to cyber-attacks and the cascading effects on society cannot be understated. Most of us have now experienced first-hand the fragility of operational processes that don’t have proper logical access safeguards in place. We all need the “older kid” who knows how operational processes work, so we are not crashing the bike or leaving it unlocked in an open area.

There are a lot of folks, including politicians and many in the media, talking about the problems with aging insecure infrastructure and the need for more money and resources for upgrading systems and putting in cybersecurity tools.

Unfortunately, this money is often spent on politically aligned companies who implement expensive and complex technology – resulting in solutions that are not effectively integrated and handed off to people who are not trained or much too busy with other tasks such as operating a power plant. This approach will not make our critical infrastructure resilient, and many times, it can lead to misconfiguration and exposure of critical systems to cyber-attack.

Getting to resilience requires the older kid experience with simple solutions that can make managing critical operations less expensive and more secure. The right resources are in almost every control room – the challenge is to put operational processes and technology in place that enables more effective operational management and reduces cyber risks simultaneously.

The Colonial Pipeline Incident Fallout and Building Zero-Trust

Colonial is an archetype of critical infrastructure.

Back in March, a hacking group known as DarkSide began a campaign on Colonial Pipeline’s IT network and billing systems. On May 7th, Colonial publicly announces the attack, shuts down servers and some pipelines and pays DarkSide $4.4M in ransom.  On May 12th, Colonial restores operations and announces fuel delivery timelines amidst panic buying at gas stations.

While Colonial was able to get operations back up and running after the 6-day shutdown, the incident’s economic ripple effects were stark.

  • Gas Stations: Last week, 71% of gas stations in North Carolina, 55% in Virginia, 54% in South Carolina and 49% in Georgia were dry.
  • Air Travel: American Airlines altered schedules and announced adding refueling stops for long-haul routes out of Charlotte, NC.
  • Department of Transportation: The DoT announced a regional state of emergency for 17 states, easing restrictions for transport of fuel.

Clearly, the closure of the 5,500-mile pipeline system has been the most disruptive cyberattack on record.

Colonial’s OT network uses automation systems to control and monitor the flow of fuel from refineries and tank farms into Colonial’s pipeline, and from Colonial’s pipeline into the tanks and transportation facilities belonging to suppliers and distributors.

According to CNN, people briefed on the matter were concerned they wouldn’t be able to figure out how much to bill customers, and the billing system is central to the unfettered operation of the pipeline.

The interdependency between the IT billing system and OT automation system is clear. Colonial automated fuel monitoring, and control data from the OT network is fed into the IT billing system so they know how much to bill customers.

The Problem – lack of proper access controls for critical systems

Colonial said it shut down the pipelines as a precaution to prevent the infection from spreading. The reality is that there are cascading dependencies when you automate processes and IT systems are dependent on OT systems and vice versa.  In addition to billing systems, Colonial’s IT network includes HR/payroll systems, supplier data, business analytics, pipeline schematics, etc… which are not interdependent on the pipeline automation system.

I don’t doubt that Colonial was taking a precautionary measure to “prevent spreading” – but this statement illuminates a bigger problem. Why would an attack on a critical billing system spread to other IT systems or the OT network? The likely answer is that this critical system was not properly segmented with separate logical access controls including multi-factor authentication and granular system or application authorization. There appears to be a lack of appreciation or recognition of the difference between a “critical” system and a “confidential” or “sensitive” system within Colonial’s IT operations.

IT systems that are interdependent on OT systems become critical infrastructure systems and must have separate logical access controls based on zero-trust. 

The Solution – Zero-Trust access platform for both critical IT and OT systems

While corporate IT networks must be connected to the internet, there are critical systems that need additional authentication and authorization. For example, it is no problem to give keys to the janitor to clean your office, but would you give him the combination to the safe under your desk? This is the concept of “zero-trust.”

For critical IT systems such as Colonial’s billing system, a zero-trust access layer including multi-factor authentication (MFA) and granular role and time-based authorization should be required. In addition, full user session logging, monitoring and recording of access to these systems is paramount.

The risk of ransomware is mitigated when a separate “zero-trust” user access layer is deployed between the “sensitive” corporate network and the “critical” billing systems.

There also needs to be a secure operational link between critical IT systems and OT network. This can be accomplished by additional segmentation, logging and monitoring.

The corporate IT network needs to have a separate zero-trust user access platform for connecting to the OT network. There may be OEMs that need access to control systems, and this access should also be controlled through MFA, user-to-asset connection control, logging, monitoring and recording.

Summary

Critical Infrastructure systems need to be identified in every large organization and measures need to be taken asap to ensure that the systems – whether on the IT network or OT network – are protected with a separate “zero-trust” user access platform.  A system housing credit card data is not critical infrastructure.  17,000 gas stations don’t run out of gas when a few hundred or thousand people need new credit cards.  We must understand relative risk and impacts and employ separate granular authentication and authorization to critical systems. We can mitigate risks from threat actors such as DarkSide as well as from other nefarious and skilled actors through a zero-trust methodology.

Taking an IT-Focused Approach to Securing OT Remote Operations at Municipal Utilities May be Risking Lives

The Oldsmar, Florida, water breach is two months behind us, but the lessons learned will continue to reverberate for thousands of budget-constrained municipal utilities in North America, as well as other regions across the world.

Lesson #1: Technology Budget Constraints

Oldsmar, like many other municipal utilities, occasionally needed remote access to their site, so they chose TeamViewer because it “didn’t cost anything extra.” Reading between the lines, the key point here is that the IT department had already purchased TeamViewer for their needs and had extra licenses that OT could use. The IT department probably had secure infrastructure around TeamViewer, but they could not forklift this infrastructure over to the water treatment plant because it would be too expensive to replicate for a few “critical” HMIs and other systems. TeamViewer in itself is not the issue – the problem is with the complex and expensive proposition of scaling IT cybersecurity architecture to OT.

Lesson #2: Cybersecurity Resource Constraints

Senior plant managers have mechanical and/or electrical engineering backgrounds and are not well versed in IT protocols, 2FA, firewalls, VPNs and Jump Servers, etc. They don’t have time or the expertise to manage IT cybersecurity stacks. If they have to remote into a plant at 2am and check systems, they want something that just works. Some utilities may invest in integrating a cybersecurity tool, but plant managers will not know if everything is properly configured and just want it to work. The need for easy access to the plant could drive behavior away from complex secure remote access through IT infrastructure and over to “give me the free ‘easy’ button.”

Lesson #3: IT and OT Cultural Differences – Confidentiality vs. Availability

A utility’s IT network of consists of billing, accounting and HR systems, which contain PCI and PII data that must be kept confidential. IT operations and cybersecurity personnel need to make sure that access to these systems is limited and controlled through several integrated secure authentication and authorization mechanisms. IT operations is hyper-focused on providing secure access to sensitive and confidential data for its users.

The OT network consists of process and automation controls and distributed control systems for valves, pumps, meters, etc., as well as human machine interface (HMI) computing systems and SCADA applications that interact with these real-time systems. The safety and availability of these real-time systems is paramount.

The very culture of OT operations is keeping systems running. IT is focused on protecting confidential data. These differing priorities mean that cybersecurity in the OT context needs to be built-in with unique features for both senior managers and technicians.

The Final Lesson: IT Remote Access Solutions Can Increase Risks to Public Safety in OT Environments

The nature of OT requires a very secure and simple remote operations platform that doesn’t break the bank. IT/OT converged networks can create complexity where insecure protocols such as RDP can be exposed into the IT network and out to the internet. Critical OT systems that have exposed protocols can be found with tools such as Shodan. Complex IT cybersecurity infrastructure and Security Operations Centers are focused on IT networks and not built to look for issues within OT networks. While larger utilities do implement OT-specific cybersecurity stacks, smaller municipalities cannot usually afford these, as was the case with the breach in Oldsmar.

In addition, there are specific operational needs that require OT-specific secure remote operations platforms. OT-specific user access and operations can reduce risks to public safety by including unique features such as:

  1. User access screen recording on HMIs and other OT systems – this can help diagnose user errors and help with training junior technicians to mitigate automation and control issues that could lead to disastrous consequences
  2. Granular role-based access controls such as a Remote Access Manager and File Transfer Manager – these roles can be given to specific individuals for specific tasks, thus limiting access privileges and mitigating risks associated with oversubscribed access to non-IT OT managers
  3. Live user connection monitoring – which provides senior managers visibility to technician input to walk through processes and provide real-world training

Summary

Enterprise IT remote access technologies such as VPNs and Jump Servers, when used with multi-factor authentication, intrusion detection systems and firewalled network segmentation can reduce risks associated with confidential data compromise; however, these integrated enterprise technologies cannot be forklifted and replicated for OT. Often, an OT staff will deploy a subset of these technologies to enable remote access, which then opens up the OT network to compromise. OT has very specific needs to ensure operational availability and public safety. They cannot afford the vulnerabilities associated with incomplete enterprise remote access tools or complex full stacks, which are too expensive to acquire and maintain in resource-limited OT environments.

To learn about XONA’s user access solution built for OT that puts all of these lessons into action, schedule a demo now.