New threats prompt renewed security scrutiny for product storage sites

March 3, 2003
Described here is an actual but, for security reasons, unnamed underground petroleum product storage facility in the Middle East that is protected against nuclear, bacteriological, and chemical (NBC) attack.

Robert I. Williams
SCADA Systems Engineeringand Training Consultants
Mold, Flintshire, UK

Described here is an actual but, for security reasons, unnamed underground petroleum product storage facility in the Middle East that is protected against nuclear, bacteriological, and chemical (NBC) attack.

The description and subsequent discussion in this article will provide useful information for current US and European reevaluation of security vulnerability of crude oil, products, and natural gas storage facilities, especially that of the supervisory control and data acquisition (SCADA) systems that monitor and control these facilities.

Although underground protection may not be technically and economically feasible for every existing and new oil and gas industry project, some of the concepts outlined here apply in the ongoing search to improve physical and electronic security as such facilities.

Oil and gas pipeline SCADA systems from part of the overall industry security vulnerabilities, and some of these vulnerabilities are elaborated on in this article.

Open transport network (OTN) fiber optic communications systems' technical descriptions are included in this discussion because use of such systems will minimize some of these security vulnerabilities as well as enhance security by providing continuous monitoring of remote pipeline facilities by intrusion detection alarms and closed-circuit television (CCTV) monitoring.

Integrating telecommunication requirements, such as SCADA, CCTV, company telephone extension systems, emergency phone lines, and corporate data channels, can enhance security because fiber optic cables are inherently more physically and electronically secure when compared with previous pipeline communication systems that depended on microwave or UHF radio. The buried fiber is also less susceptible to malicious electronic interference.

Product storage

Storage of petroleum products for military purposes, as for airbases and command centers and other strategic purposes, requires application of highly secure physical barriers and integrated electronic security systems based on intrusion detection, CCTV, authorized card access systems, etc., especially when such installations are subject to invasion or terrorist attack.

Process control and security SCADA systems utilized on such Middle Eastern storage sites are described but without incorporating specific location or design detail to avoid compromising the security of these sites.

Aviation fuel, gasoline, and diesel storage facilities are all underground with each product stored in ultra large, steel-lined tanks at a depth designed to withstand a nuclear bomb attack as well as military or terrorist attacks that use chemical or bacteriological warfare means.

During peacetime, the stored products are delivered by pipeline from the nearest refinery. The products are returned to their original refinery or distribution terminal by a separate pipeline where they are redistributed to the marketplace at truck terminals or airport distribution facilities.

Supply and distribution pipelines are of various lengths, depending on the distance to the nearest refinery: 30 to 300 km with up to five pump stations due to the mountainous terrain in some storage facilities' locations.

Both supply and distribution pipelines are monitored and controlled by the pipeline SCADA system for leak detection and batch-tracking modeling. Supply and distribution pipelines are equipped with custody-transfer metering and proving systems designed according to industry standards and the product-supply petroleum company's standards.

During wartime, it is assumed that the pipeline supply and distribution capability will be compromised. Each storage site, therefore, is equipped with secure, low-profile tanker truck loading facilities.

Click here to enlarge image

Fig. 1 shows a generalized cross-section of such a facility with product storage at the lower levels, process facilities at the mid-level, and an operations control center in a totally isolated and NBC-protected area at the higher level.

Storage tanks are equipped with vapor-recovery systems and highly accurate level measurement and averaging temperature probes. Two submersible pump systems are provided, one to pump out the product for pipeline distribution and one to pump out accumulated water from a bottom-level sump after precise water-content measurement based on capacitance probes.

Hand sampling means are also provided to verify product density and other quality parameter measurements.

Piping tunnels interconnect the product storage and process levels; each storage tank is equipped with a pump manifold area for re-circulating product for return to the distribution pipeline and for periodic filtering and quality testing.

Pipe tunnels and pump manifold areas are equipped with fire and gas-detection systems and with foam-based fire extinguishing systems. Each process area is then treated as a separate fire zone.

NBC protection requires suitable detection of and protection from these attacks as well as sustaining the facility for a period of time, e.g., 30-40 days. An NBC attack may consist of a nuclear bomb attack on the facility, which is protected by the depth of the facility underground. A direct bomb attack on the main entrance or emergency exit is protected by the angled tunnel entrance that will reflect a significant portion of the blast energy back out of the tunnel (Fig. 1 inset diagram). The next defense is a blast barrier that acts as a check valve using the blast energy to seal the barrier and not allowing sufficiently damaging energy to be relayed through into the underground site.

Chemical and bacteriological detectors are above the facility and at HVAC (heating-ventilation-air conditioning) air-inlet tunnels into the facility. The SCADA system relays these detector alarms, including test and malfunction alarms, to the main operations control center (OCC).

Ventilation systems keep a constant and regulated flow of air from outside the facility through all process equipment areas and piping and cable tunnels. An intricate network of ventilation ducts, dampers, and regulators are also monitored and controlled by the distributed control system (DCS) system. Air supplied is regulated via the DCS at the inlet points, and airflow rates are controlled by fan-speed control loops.

Although the site is normally powered from the public utility company, it is independently supplied from an electric generation plant consisting of several diesel-engine-driven generators with separate and independent backup generators for the NBC-protected operations control complex.

Click here to enlarge image

SCADA, DCS, fire and gas, emergency shutdown (ESD) systems, and fiber optic multiplexing equipment are in nonhazardous area electronic equipment rooms throughout the facilities. All process-area instrumentation systems are intrinsically safe with explosion-proof control devices for heavy-duty valve actuators. Fig. 2 shows the DCS, ESD, and fire and gas systems architecture including the pipeline SCADA system interface.

Each ESD system is designed to be standalone and isolates each tank manifold area or process area only. ESD isolation stops all product operations for that process area and other affected areas, e.g., metering, and also ensures all ventilation fans and fire dampers as well as access doorways are closed.

A fire detected and confirmed at any process or utility area initiates a global shutdown via the ESD and DCS systems.

Electronic equipment rooms are equipped with ionization and thermal rate-of-rise detectors with automatic extinguishant-gas release based on confirmed fire (two or more detectors in alarm in the same zone). The operations control center and computer rooms are similarly protected with all zones containing personnel equipped with evacuation alarm before extinguishant-gas release.

Physical security

Physical access to the site is limited to one main entrance, after the initial construction period, when all normal personnel access is via security card access-controlled personnel turnstile.

A secondary exit is provided for emergency evacuation, if such a contingency became necessary considering the operations control center can sustain personnel during and after an NBC attack. All security entrances, above and belowground, are monitored by intrusion detectors and CCTV monitoring systems with infrared lights and pan-tilt-zoom (PTZ) for security guard control.

Click here to enlarge image

Fig. 3 outlines the security SCADA system architecture. The system consists of data acquisition and control RTUs at the various security areas throughout the underground facility, communicating over a fiber optic network.

The main security control center is equipped with the CCTV video multiplexing equipment, a large bank of TV monitoring displays and operations-control consoles for display and presentation of various intrusion alarms and other security administrative functions, such as access-card checks and validity verification.

Physical intrusion alarms systems can be further evaluated by operating the PTZ controls for the individual cameras for the area reporting a suspicious security situation.

A secondary security-monitoring center operates at the main entrance guardhouse.

Operations control complex

The operations control complex consists of the following:

  • Operations control center (OCC).
  • Emergency-response center (ERC).
  • Security control center (SCC).
  • Computer communications equipment (CCE).
  • Nuclear, bacteriological, and chemical air filtering equipment.
  • Uninterruptible power systems, including backup batteries.
  • Standby generator and fuel supply.
  • Accommodation, restaurant, and recreational facility.
  • Offices, meeting rooms, etc.
  • NBC purging and isolating entrance chambers.

The OCC control room consists of operator workstations for the underground facility, off-site and pipeline DCS and SCADA systems. This continuously manned operations center monitors and controls all process, fire and gas, HVAC, and ESD functions.

The operations control center is designed to maintain operations during and after an NBC attack. During an NBC scenario, all air supplied is filtered by specially designed micro-filtration equipment to remove chemical or bacteriological elements. A separate pure- water storage system is also provided.

SCADA security

In general, long-distance crude oil or product pipeline facilities, such as pump stations, are unmanned because they are designed to be monitored and controlled remotely and self-protective via the ESD system. Maintenance visits are generally infrequent except when required for routine or corrective maintenance.

Personnel on site may be local operators, technicians, engineers, or managers, none of whom would be expected to respond or be equipped to respond to a determined terrorist attack.

Pump-station security generally consists only of a chain link fence with one or more padlocks. Multiple padlocks are used to allow for pump-station access by different operations and maintenance departments.

SCADA systems can operate independently of any data connections to management information systems or other networking computer systems. This isolation could provide the required defense against any external terrorist hacking attack to prevent the SCADA system from operating normally.

Corporate policy will have to evaluate the threat of an external attack against the need securely to disseminate real time data to allow for customer and corporate access and for timely and accurate invoicing of product inventory. Computer technology utilizing firewalls and other means can reduce this risk to a tolerable level.

Database-access restrictions and disk-mirroring techniques can mitigate these security vulnerabilities.

Computer-security considerations have universally adopted the term "intrusion detection" to refer to hackers or illegal internet access attempts as opposed to the "intrusion detection" term for personnel-detection devices used by physical security systems.

Combined computer-intrusion detection and firewall software systems require an initial investment in installation and configuration as well as ongoing management commitment to ensure overall security protection efficiency.

SCADA communications systems utilize telephone, microwave, UHF/VHF, VSAT (very small aperture terminal) satellite, or fiber optics as the main data transmission medium.

Telephone lines are generally leased lines and are a potential security problem. Radio communications are vulnerable because antenna and towers are highly prominent and susceptible to physical attack.

Radio signals can also be interfered with so that the SCADA system could be inundated with communication errors and disrupt normal operations.

In a multiple-drop polling mode, the SCADA system will attempt a number of retries, typically three, before it continues scanning the next RTU. It will generally attempt to scan an affected RTU two or three more times before it will flag a communication failure at an RTU and effectively remove that RTU from the scanning program.

Operator or maintenance engineer intervention is generally required to return the RTU to the scan sequence.

In the meantime, pump-station control and ESD systems remain unaffected and, provided pipeline hydraulic conditions are normal, pipeline operations can continue for a company-defined period.

Fiber optics in this security respect has a higher degree of protection because they are not easily intercepted and interfered with during transmission.

In most cases, fiber optic cables are installed along the pipeline right-of-way in a separate trench at a suitable distance from the actual pipeline. (See Fig. 1 in accompanying article, p. 59.)

From this respect, the cables are just as vulnerable for physical damage from terrorist attack as the pipeline itself.

A total optic cable failure would affect all pump stations downstream of that failure. With fiber optics, the damage would be immediately diagnosed and reported and maintenance diagnostic capability can identify the location of the damage.

VSAT communications also utilized for long-distance communications may have security advantages in that only one location would be affected.

Pipeline operations control centers are where terrorist attacks can have a significant effect on pipeline operations.

The SCADA master station is in one or sometimes two key locations. All communications and real time data acquisition are concentrated at these locations. Communication availability is the only restriction to the ultimate operations control center locations. It is not necessary to staff both control centers continuously, and some pipeline companies alternate between their control centers frequently, even monthly.

This is a recommended security consideration for implementation on new pipeline facilities and for upgrading consideration for other facilities. Any such physically diverse designs should carefully analyze the communication requirements and their associated vulnerability to terrorist attacks.

Most pipeline control centers within the author's experience have always included restricted access to the SCADA control and equipment rooms. This is to protect operators from operational distractions, however, not as a direct security precaution.

VIP visits to SCADA operations control centers shall be a thing of the past as would be observation windows in the control rooms. A negative effect for SCADA system suppliers and their potential customers will be the curtailment of necessary site visits to verify SCADA systems' project references and system availability, as claimed by the system supplier.

Emergency or contingency plans must include terrorist threats against the pipeline, pump stations, and operations control centers. Over-reaction to implied security threats must be balanced against the unknown while still remaining focused on pipeline throughput efficiency to preserve the pipeline company's profitability.

OTN

Fiber optic open transport network (OTN) is one of the evolving technological solutions for communication networks that cover wide geographical area. The main requirements for the pipeline fiber optic network are high availability, security, redundant network configurations, high network resilience, voice and data interfaces, and overall network monitoring and configuration management.

The Eastern Province of Saudi Arabia gas-gathering pipeline system uses OTN fiber optic communications systems for voice (telephone and emergency line) and monitoring and control data transmissions for approximately 100 wells. Overall distances involved exceed 400 km with some hub distances of more than 60 km.

A similar crude oil pipeline in West Africa uses OTN fiber optic communications for 20 nodes at pump stations, pressure-reduction station, and block valve stations.

OTN fiber optic communication systems offer security advantages for long-distance oil and gas pipelines as well as for other industries for which these systems are installed, such as railroad, airports, electric power distribution, and mines.

Click here to enlarge image

Typical pump stations or production facilities or distribution terminal user equipment are connected to each of the OTN nodes, N1 through N6 in Fig. 4, through one of a number of interface card slots. The Fig. 4 insert shows a typical node equipment connection.

The interface cards perform the conversion to digital signals to be injected into and retrieved from the time-division multiplexing message frame sent on the fiber optic ring. Available interface modules include SCADA, management information, company telephones, mobile radio-paging systems, and security surveillance video cameras.

Fig. 5 shows a typical pipeline security CCTV surveillance installation, based on explosion-proof surveillance cameras.

The OTN nodes in the network (Fig. 4) are interconnected by means of dual point-to-point optical fiber links. These fibers form two counter-rotating rings.

In normal operation, all data of the connected equipment are transmitted on one ring, while the second ring is standby. The latter is kept in synchronization, however, in order to monitor its continuous availability.

Click here to enlarge image

This explosion-proof CCTV camera installation at a natural gas facility is connected to an OTN fiber optic network (Fig. 5).

An OTN node is based upon a 19-in. mountable chassis (Fig. 6), which is equipped with several common modules and capacity of up to eight interface modules. The common modules are the power supplies, two transceiver modules, and a common logic card.

The common logic card implements the time-division multiplexing and sends the received information to the appropriate interfaces and from the interfaces to the fiber optic module for transmission. Detected failures, such as cable breaks, can be corrected within 50 to 120 ms, without the intervention of the network management system.

Click here to enlarge image

OTN node multiplexer equipment contains channels for SCADA, company telephones, corporate data network, emergency telephones, and CCTV (Fig. 6).

OTN can be used for point-to-point, multipoint, or multidrop connections by means of the appropriate interfaces. In a self-healing back-up mode, each node is programmed to loop back the data when it detects failure in the network, as shown in Fig. 4.

Fig. 4a shows that the network fails over to the standby fiber ring when there is a fault in the main fiber ring. When the fault (Fig. 4b) involves a complete node, the adjacent nodes loop back the data traffic in the network. When the fault shown (Fig. 4c) involves the complete fiber cable, the adjacent nodes again respond with data loop back without involving the network management system.

The network management system's diagnostic feature is notified of the various failure modes and the information is alarmed on the monitoring displays. Optionally, nodes can be equipped with an optical by-pass relay that can take the node into and out of the ring. F

Click here to enlarge image

The author
Robert I. Williams (riwilliams @scada-online.co.uk) is an oil and gas industry international consultant specializing in automation, instrumentation, and SCADA applications for SCADA Systems Engineering and Training Consultants, UK. As an instrumentation and control systems engineer, Williams was involved in such major projects as the UK North Sea first gas fields (early 1970s), the trans-Alaska crude oil pipeline (1972-78), and the Piper Alpha replacement platform, Piper Bravo, as overall systems integration consultant (1990-92). He was also a SCADA consultant to the Romanian crude oil network rehabilitation project (2000-02) and is currently on the Pakistan White Oil Pipeline project with China National Petroleum.Corp. as the EPC contractor. His 5-year Middle East assignment (1994-2000) was for the Ministry of Defense and Aviation for one of the major countries in the region. Williams is a University of Wales, UK, electronic engineering BSc graduate and a senior ISA member (1970), Institute of Electrical Engineers chartered engineer (1969), and a California professional engineer (1973).