Keeping systems up and running and productive is the primary mission of every manufacturer and even with the best cyber defense mechanisms in place, cyber incidents will likely occur. The question now remains: Is your organization prepared to properly identify what went wrong and recover?
Preparation and planning are essential to an organization’s ability to respond to a cyber incident. The ability to identify the source of an incident and analyze the extent of the compromise is necessary to rapidly detect issues, minimize loss, mitigate exploited vulnerabilities, and restore computing services, according to a report in the Industrial Control System Cyber Emergency Response Team Monitor (ICS-CERT Monitor).
Cyber incidents are tense, complicated, and not often part of routine operations. When properly maintained, operational preparedness measures can ensure the availability of information necessary to recover from an incident quickly while minimizing the impact.
A dedicated incident handling team should be led by a senior technical staff member who has the authority to make key decisions in a timely manner. In addition to the lead and forensics analysts, a control systems incident response team should include control systems subject matter experts and stakeholders from corporate IT (network and host management), public relations, legal counsel, and law enforcement, if necessary.
The team should end up trained in proper incident handling techniques and should practice using the tools to establish and maintain proficiency. Control system environments have special needs that need evaluation when establishing operating procedures. An overall incident preparedness checklist should end up created and reviewed annually using a “table-top” exercise. Documentation should be accessible to operations personnel to help facilitate analysis of the incident and identify priorities for recovery. There should also be an incident response information gathering checklist. This checklist should identify the types of information that should be collected to aid analysis by external CERTs or partners.
It is also important to establish an “out-of-band” communications policy. Any communications regarding an incident or potential incident should not go through the standard communication channels, e.g., corporate email, VoIP systems, as these may already suffered compromise and will tip off the attacker you are aware of their presence in your network. In addition, any files relating to the incident or handling policy should be stored off the network under the control of the incident response team.
Logging is an important aspect of incident response. System and network device logs are essential to incident investigators. The types of logging a user should think about include firewall, proxy, domain name server (DNS), dynamic host configuration protocol (DHCP), web app, audio visual (A/V), intrusion detection system (IDS)/intrusion prevention system (IPS), and host and application logs. Additional logging to think about is flow data from routers, switches, and packet captures. This type of network data will be helpful when responding to a control system event because network-related logs are sometimes all that is available. If the control system endpoints do support logging, these, too, should undergo review for a better understanding of what took place. Log integrity is essential during an incident investigation; therefore, logs should continuously end up stored on a separate system, frequently backed-up, and cryptographically hashed to allow detection of log alterations.
Other critical components of incident response are forensic data collection, analysis, and reporting. These elements are essential to preserving important evidence. Organizations should consult with trained forensic investigators for advice and assistance prior to implementing any recovery or forensic efforts. In addition, ICS-CERT subject matter experts are available to aid in incident response activities.
For additional information and resources on cyber incident response for industrial control systems, go to ICS-CERT’s fact sheet entitled “Preparing for Incident Response”.