As industrial networks move towards being more open IP-based networks, those responsible for developing and maintaining them are faced with the growing challenge of managing these technologies traditionally developed for the IT world.
And, as the importance of high-performance, highly-available and secure OT (Operational Technology) networks has never been greater we take you through the top 5 faults you might find with your industrial networks and how to address them...
- Not knowing your network
- Lack of back-up and recovery plans
- No network segmentation
- Lack of network monitoring
- Poor security practices
Not knowing your network
When it comes to troubleshooting an industrial network, one of the main keys to success is having a thorough understanding of how your network assets are connected. Understanding the underlying network topology and configuration is the first step to begin properly troubleshooting networking issues.
With up to date and accurate network topology diagrams, things such as troubleshooting become a much simpler exercise. Suppose you have connectivity issues between a couple of PLCs. Without a proper topology diagram and understanding of where in the network these PLCs are, it will make it very difficult to systematically troubleshoot the issue and understand the path that packets should be taking from those PLCs.
There are many ways to create network topology diagrams. The manual way involves making use of switch discovery features or neighbour discovery such as CDP (Cisco Discovery Protocol) or Hirschmanns topology discovery. In addition, open standards such as LLDP (Link Layer Discovery Protocol) can help you discover how your switches are connected to each other and on which ports. With this information you can use a program such as Visio to create an accurate network topology diagram.
More automated ways of doing this include the use of software such as Hirschmann Hivision or Solarwinds network monitoring. These tools make use of SNMP (Simple Network Management Protocol) to automatically discover switch to switch connections and other useful information. These tools rely on SNMP being enabled on your switches. In networks where there are multiple VLANs being used, these tools require program configuration to ensure they discover the whole topology and its connections.
Lack of backup and recovery plans
As networks become more complex, so do the configurations that reside on your network switches. Unless you have a large flat open network of unmanaged switches (which has many issues in itself), your switches will have a configuration on them. Should a managed switch fail, its replacement will require that configuration to be loaded onto it.
Many industrial switches nowadays come with easy recovery options for this that include USB memory stick and SD card backups. When either of these are present, when a configuration is saved it will be written to this storage device. If the switch failed, the USB memory stick or SD card can simply be plugged into the replacement, powered on and the configuration will be instantly restored.
If this functionality is not available on the managed switches you use, there are other ways to help streamline the backup and recovery process of your switch configurations. Many switches now support centralised backup storage of configuration. Whenever a config is changed, this will get written to a centralised storage server somewhere. Whenever one of these backups are required, you can be assured that they will have the most up to date copy of the config on.
Restoring configs from file is normally a straight forward process. Most switches will come shipped with a default IP address that will give access to the web management, from here, you can import in a configuration file to restore it. For a lot of Cisco switches, the config file can be copied / pasted back into a replacement switch through the use of Putty or other terminal emulation software.
No network segmentation
A big flat open network is a bad thing, particularly in the industrial world where reliability and security are of such importance. A flat network is one in which there is no segmentation through the use of separate IP address ranges and VLANs (or less commonly complete physical isolation of these IP address ranges).
Whilst segmenting an industrial network is certainly a challenge, it provides many benefits both from a management and security prospective. In a flat network, any issues experienced at layer 2 (for example a broadcast storm) will affect the entire network and have the potential to bring the entire process to a standstill. With proper segmentation in place, an issue such as a broadcast storm would only affect the local network segment and not the wider network.
But by far the biggest benefit from proper network segmentation is the added security controls. Securing assets on a flat network can be very challenging indeed. A segmented network of multiple VLANs / subnets on the other hand now has a centralised point in which proper security controls can be put in place, most commonly; access control-lists. Any traffic between your VLANs has to pass through your layer 3 infrastructure which in turn can be your firewalls. From here you can configure access control-lists to only allow through the required traffic and to block everything else.
Lack of network monitoring
A proper network monitoring solution can serve two main purposes. The first is to give complete visibility of the entire network and its assets within it. By having this, it is much easier to respond to issues on the network and can save valuable time from having to do lengthy amounts of troubleshooting. The ability to be able to respond to a failed link, device or switch in the network can be the difference between minor outages and major ones.
In addition to this, cyber security network monitoring most commonly labelled as IDS (Intrustion Detection Systems) serves to constantly monitor all network traffic and alert you to potential threats. Even in a small network, thousands of packets are passing between devices every minute and it is possible to have a hold on everything that is happening within the network. An IDS solution such as Claroty CTD (Continuous Threat Detection) is used to monitor 100% of all network traffic across a very wide range of industrial and non-industrial protocols. This kind of monitoring allows for a much more proactive approach to cyber security, as opposed to reacting to a major breach, by which time it’s far too late!
Poor security practices
The world of cyber security can be confusing, overwhelming and difficult. Whilst it is true that it is an ever-changing journey, there are plenty of best practices that be followed to really enhance the security of your network and its assets within it.
Here are some:
- Secure your switches. Your switches are the entry point into the network. Common cyber security threats are simply not possible if a would-be attacker cannot get onto the network in the first place. These are a few things you should always do as a minimum to secure your switches:
- Reconfigure all unused ports into an unused / empty VLAN and then shut them down
- Ensure switch management is only possible over secure protocols (HTTPS and SSH) and ideally keep the management interfaces into a dedicated and secured management VLAN
- Make sure you are using strong usernames and passwords and avoid common usernames such as administrator, admin or root
- Consider local access control-lists to lock down ports to just specific MAC addresses
- If using protocols such as SNMP, opt for the more secure version (v3) that allow for encryption
- If the switches support them, consider configuring things such as BPDU guard, dynamic ARP inspection and DHCP spoofing to protect against some of the most common cyber security attacks
- Use firewalls. Proper network segmentation will create the necessary boundaries to begin implementing access control-lists and other firewall features. The goal is to limit traffic allowed to only the required protocols and to deny anything else. For real critical areas of the process, consider local firewalls that are specific to that area. Many industrial based firewalls exist that understand specific industrial protocols (such as the Hirschmann Tofino)
- Never overlook wireless security. Wireless devices have given us much more flexibility in how we can do things, but this comes at a cost. In a home environment a WPA2 key is normally sufficient enough to protect our network. In an industrial network there is more at stake and using even more enhanced security features such as certificate-based authentication and RADIUS authentication should be strongly considered