By Gregory Hale
Seventeen Zero Day vulnerabilities ended up fixed in the OPC UA (Object Linking and Embedding for Process Control Unified Automation) protocol, researchers said.
Developed to provide secure communications between systems on industrial networks, OPC UA has been growing in popularity since its launch in 2006.
“It is common for monitoring and control systems based on different vendors’ products to use mutually incompatible, often proprietary network communication protocols,” said Security Researcher Pavel Cheremushkin and Senior Security Researcher Sergey Temnikov with Kaspersky Lab ICS CERT, who discovered the vulnerabilities. “OPC gateways/servers serve as interfaces between different industrial control systems and telemetry, monitoring and telecontrol systems, unifying control processes at industrial enterprises.”
Cheremushkin and Temnikov found 17 zero-day vulnerabilities in the protocol’s implementation, leading to denial-of-service threat attacks, as well as remote code execution. In addition, several flaws were found in commercial products built on the protocol. All vulnerabilities were reported to the developers and fixed by the end of March.
“Several years ago the OPC Foundation made the decision to provide an open source version of our OPC UA Technology,” said Thomas Burke, president and executive director of the OPC Foundation. “There are numerous other open source versions of the OPC UA Technology that are not developed and authorized by the OPC Foundation. Kaspersky Labs performed an analysis on the open source version of the OPC UA code from the OPC Foundation. Even though the code base is open source, the OPC Foundation does provide development and maintenance of the code base, specifically to address anomalies inclusive of security vulnerabilities.
“Kaspersky Labs is a member of the OPC Foundation and more importantly a member of the OPC UA Security Working Group and has worked closely with the OPC Foundation to ensure the vulnerabilities identified in the report were fixed and reported to OPC vendors and OPC users.
“These kinds of reports are an essential part of the process needed to maintain the robust and secure OPC UA eco-system that ICS operators have come to depend on. The OPC UA Security Working group is the OPC Foundation’s rapid response team that includes more than 25 security experts from OPC member companies who are actively engaged in evaluating and addressing security issues when they arise.
“We also made sure that our SDK / toolkit vendors were aware of these security vulnerabilities to make sure that their code base that potentially could have had some of the same problems were corrected if they exist, and that their customers were aware that a new version of their software was available and should be used if the software they had shipped had these security vulnerabilities.
“Again, it’s important to note that this was the open source version of the OPC Foundation deliverables that had the security vulnerabilities that has now been corrected. We will continue to work with Kaspersky Labs, and other security evaluation companies to make sure that we have a secure open source code base.”
OPC UA protocol is widely used by major vendors in modern industrial facilities, including manufacturing, pharmaceutical, oil and gas as well as other industries. Its gateways are installed by industrial enterprises for communication in automated process control and telemetry, as well as monitoring and telecontrol systems, allowing these enterprises to unify their management processes. The protocol is also used in Industrial Internet of Things (IIoT) and smart city components.
Kaspersky Lab ICS CERT experts analyzed OPC UA architecture and its products. They examined its open-source code (available on GitHub), including a sample sever, and discovered current implementations of the protocol had code design and writing errors. These errors should not exist in such widespread critical infrastructure software, the researchers said. Overall, 17 Zero Day vulnerabilities in the OPC Foundation’s products were identified and reported to the developers, who fixed them accordingly.
In addition, Kaspersky Lab ICS CERT analyzed third-party software based on this industrial protocol, including solutions by leading industry vendors.
In most cases, the discovered flaws were caused by the developers not using some of the protocol implementation functions properly. In other cases, vulnerabilities were the result of incorrect modifications applied to the protocol’s infrastructure.
As a result, researchers discovered the insecure implementation of functions in a commercial product, despite the fact the original OPC Foundation implementation did not include errors. As a result, such modifications in the protocol’s logic, made by vendors for unknown reasons, were leading to risky functionality.
All vulnerabilities found in the OPC UA protocol implementations could result in heavy damage to the industry.
“Very often software developers put too much trust in industrial protocols, and implement the technology in their solutions without putting the product code through security checks. Therefore, vulnerabilities in the example used can affect complete product lines, so it’s highly important that vendors pay close attention to such widely available technologies,” Temnikov said. “Moreover, they should not be deceived by the idea that they can design their own piece of software. Many think this could be more efficient and secure than existing software, but even a brand new piece of software may still contain numerous vulnerabilities.”
To identify vulnerabilities in the implementation of the OPC UA protocol by the OPC Foundation consortium, Kaspersky researchers reviewed:
• The OPC UA Stack (ANSI C, .NET, JAVA)
• OPC Foundation applications that use the OPC UA Stack (such as the OPC UA .NET Discovery Server mentioned above)
• Applications by other software developers that use the OPC UA Stack
OPC UA Stack
“OPC Foundation developers provide libraries that are essentially a set of exported functions based on a specification, similar to an API,” the researchers said. “In such cases, it is often hard to determine whether a potential security problem that has been discovered is in fact a vulnerability. To give a conclusive answer to that question, one must understand how the potentially vulnerable function is used and for what purpose – i.e., a sample program that uses the library is necessary. In our case, it was hard to make conclusions on vulnerabilities in the OPC UA Stack without looking at applications in which it was implemented.
“What helped us resolve this problem associated with searching for vulnerabilities was open-source code hosted in the OPC Foundation’s repository on GitHub, which includes a sample server that uses the UA ANSI C Stack. We don’t often get access to product source code in the course of analyzing ICS components. Most ICS applications are commercial products, developed mostly for Windows and released with a licensing agreement the terms of which do not include access to the source code. In our case, the availability of the source code helped find errors both in the server itself and in the library. The UA ANSI C Stack source code was helpful for doing manual analysis of the code and for fuzzing. It also helped us find out whether new functionality had been added to a specific implementation of the UA ANSI C Stack.
“After building our fuzzer, we got the first crash of the program within a few minutes.
“An analysis of memory dumps created at the time of the crash enabled us to identify a vulnerability in the UA ANSI C Stack which, if exploited, could result at least in a DoS condition,” the researchers said.
OPC Foundation Applications
“Since, in the previous stage, we had performed fuzzing of the UA ANSI C Stack and a sample application by the OPC Foundation, we wanted to avoid retesting the OPC UA Stack in the process of analyzing the consortium’s existing products, focusing instead on fuzzing specific components written on top of the stack,” the researchers said. “This required knowledge of the OPC UA architecture and the differences between applications that use the OPC UA Stack.
“As a result of using our ‘dumb’ fuzzer, we identified eight more vulnerabilities in OPC Foundation applications,” researchers said.
Third Party Applications
Having completed the OPC Foundation product analysis stage, researchers moved on to analyzing commercial products that use the OPC UA Stack. From the ICS systems researchers worked with during penetration testing and analyzing the security status of facilities for some of our customers, they were able to select several products by different vendors, including solutions by global leaders of the industry. After getting approval, they began to analyze implementations of the OPC UA protocol in products.
In that testing, the researchers found two more new vulnerabilities.
“Throughout our research, experts from the OPC Foundation and representatives of the development teams that had developed the commercial products promptly responded to the vulnerability information we sent to them and closed the vulnerabilities without delays,” the researchers said.
“In most cases, flaws in third-party software that uses the OPC UA Stack were caused by the developers not using functions from the API implemented in the OPC Foundation’s uastack.dll library properly – for example, field values in the data structures transferred were interpreted incorrectly.
“We also determined that, in some cases, product vulnerabilities were caused by modifications made to the uastack.dll library by developers of commercial software,” the researchers said. “One example is an insecure implementation of functions designed to read data from a socket, which was found in a commercial product. Notably, the original implementation of the function by the OPC Foundation did not include this error. We do not know why the commercial software developer had to modify the data reading logic. However, it is obvious that the developer did not realize that the additional checks included in the OPC Foundation’s implementation are important because the security function is built on them.”
OPC Foundation is opening the source code of its projects certainly which means it is open and committed to making its products more secure.
At the same time, the researchers said their analysis has demonstrated the current implementation of the OPC UA Stack is vulnerable, but also has some fundamental problems.
“First, flaws introduced by developers of commercial software that uses the OPC UA Stack indicate that the OPC UA Stack was not designed for clarity,” the researchers said. “Unfortunately, an analysis of the source code confirms this. The current implementation of the protocol has plenty of pointer calculations, insecure data structures, magic constants, parameter validation code copied between functions and other archaic features scattered throughout the code. These are features that developers of modern software tend to eliminate from their code, largely to make their products more secure. At the same time, the code is not very well documented, which makes errors more likely to be introduced in the process of using or modifying it.
“Second, OPC UA developers clearly underestimate the trust software vendors have for all code provided by the OPC Foundation consortium. In our view, leaving vulnerabilities in the code of API usage examples is completely wrong, even though API usage examples are not included in the list of products certified by the OPC Foundation.
“Third, we believe that there are quality assurance issues even with products certified by the OPC Foundation,” the researchers said.