Philip S. Carpenter - Serrano Services & Systems, Leander, Tex.
Morgan Henrie - MH Consulting Inc., Anchorage
Ed Nicholas - Nicholas Simulation Services, Wichita, Kan.
Expanding the outputs from automated software-based pipeline leak-detection system performance testing to include false positives can improve assessment and optimization of PLDS. Marked improvement is available when these new outputs are combined with appropriately high-level measures of PLDS efficiency and performance and a methodology that allows for calculating these efficiencies based on the results of the software-based testing.
The challenges of consistently testing PLDS performance inhibit assessment and optimization of the system (OGJ, Mar. 21, 2005, p. 53). The lack of available standards for clearly and easily summarizing system performance also inhibits PLDS assessment.
Systematic, software-based testing automatically develops probabilistic performance maps by imposing a wide range of artificial leaks on sets of recorded real-time data and then using the modified data to drive the system in an off-line mode. The resulting maps assessed leak-detection performance in greater detail than is possible through physical discharge testing.
The performance maps’ level of detail, however, limits users’ ability to summarize leak-detection system performance on a high level clearly and easily. This circumstance limits the maps’ suitability for comparison with other leak-detection systems and utility in optimizing performance of the leak-detection system.
This first of two articles examines performance mapping and presents details of a case study. Part 2 will define the various efficiencies that must be considered to effectively gauge PLDS performance.
This article weights all outcomes (value of true positives as well as cost of alarm errors) equally. The costs associated with validation of false positives plus the incremental costs that occur when the PLDS fails to detect a leak (leaving it to be detected later, by other means) would ideally be compared with the incremental value produced when the PLDS detects a leak promptly.
The methodology presented here improves assessment of PLDS performance capabilities. The efficiencies described supply a basis for comparison of competing leak-detection systems when implementation decisions are being made, as well as a potential mechanism for optimization of performance for an existing system. Because the system efficiencies express complex performance issues in simpler terms, they may also provide an important basis for discussions between pipeline operators and regulators.
Performance mapping
Fig. 1 shows the simplest pipeline suitable for monitoring by a PLDS. Oil enters on the left side and exits on the right. Meters at both the entrance and exit of the segment measure the flow of oil. Pressure, temperature, gravity, and other transducers measure additional data often used to correct for inventory changes.
Pipeline leak-detection systems operate with various principles. API 1130 identifies several variants or approaches, including line balance, volume balance, modified volume balance, real-time transient model (RTTM), acoustic or negative pressure wave, and statistical systems.1
A modern PLDS may combine several of these techniques. Regardless of the approach or approaches taken, the leak-detection system processes the raw, measured data to develop a leak-detection signal. A comparison of the size of the resulting signal to an internal threshold of some sort evaluates the signal. If the flow imbalance exceeds this threshold, the system alerts the pipeline operator.
It is inherently desirable that the effective threshold of the PLDS be as low as possible for detection capabilities across the broadest operating range. System noise inherent in both the input measurements and internal processing of the PLDS, however, produces uncertainties in the resulting leak-detection output signal.2
Tuning the PLDS to a leak-sensitivity range well above the range of output leak-detection-signal uncertainty will generally allow it to work in a deterministic fashion. In this range, any leak alarm the system produces will likely be significant. On the other hand, any LDS tuned to leak-detection sensitivities of the same order as the potential measurement errors will begin to exhibit errors in its output.
Such errors include false positive-Type I errors (false alarms or LDS system alarms that do not correspond to any real pipeline leak) or false negative-Type II errors (real leaks missed and never alarmed by the system). An LDS operating in this range is operating probabilistically.
The noise inherent in the output signal places practical limitations on how low the system threshold can be set, and an inherent tradeoff exists between the additional coverage or sensitivity gained and the cost in false positives encountered as the threshold is lowered.
A modern LDS is so complex that predicting its performance based on first principles is impractical or impossible. This circumstance requires testing the LDS to evaluate system performance. In fact, a pipeline LDS should be periodically tested to determine its performance.1 Evaluation can use commodity discharge testing, but this type of testing carries significant cost and risks. The narrow range of practical tests also limits commodity discharge testing. These constraints limit PLDS performance testing to a small set of data that may not fully represent the system capabilities or limitations, often precluding the possibility of developing a detailed performance picture for the PLDS.
Off-line testing of the leak-detection system provides a useful alternative in these circumstances, allowing for efficient, automated or semiautomated evaluation with prerecorded data incorporating a wide range of inputs configured as required to map the full system performance. This approach also allows for developing a comprehensive and detailed range of simulated leaks.
A software-based application applies modified input data sets as inputs to the PLDS.1 The data sets consist of real world recorded data normally used for input to the LDS. For any specified configuration of the LDS, the performance analyzer application runs in two modes: a mode in which archived input data are run unaltered through the LDS to determine the baseline performance (including generation of false positives); and a mode with archived data perturbed to reflect the presence of a leak of specified size.
The special application automatically reads and modifies the data sets and then inputs them into an offline version of the actual leak-detection system. Only the constraints of the computer hardware and operating system limit the PLDS’s operating rate.
Case study
A case study of an RTTM-based mass balance PLDS illustrates the test methodology. This system uses flow measurements at points of flow entrance or egress from the pipeline system or segments, as well as internal pressure, temperature, tank level, and other measurements in conjunction with its internal pipeline model to develop a mass balance for the system.
User configuration files that permit specification of a set of both static and dynamic parameters configure the system’s leak threshold. Dynamic parameters include persistence specifications for the leak in terms of both time and volume.
In addition to the output alarm state, the PLDS provides diagnostic capabilities of trends for pressure, flow, and other inputs, as well as a calculated leak-detection signal. The calculated system balance (normally zero) is alarmed as a leak if the flow rate is above threshold.
Because of the way that this PLDS handles its data processing, it will start viewing leak-related imbalances that persist for much longer than 24 hr as part of the normal baseline. If it does not detect them within this period, it will probably never detect them. This detection time window implies that the effective threshold of detection for the system at 24 hr be used as the most sensitive threshold reference in this analysis.
The test pipeline is a portion of a large-diameter crude oil system with custody-transfer measurements available at the system boundaries. Internal flow measurements from less reliable ultrasonic flow meters supplement the custody-transfer measurements. All pump stations also measure pressure and temperature. Storage locations supply additional gravity and tank-level measurements. The protected portion of the system measures 350 miles. This case study analysis used roughly 3 months of continuous pipeline data.
The performance analyzer assessed a range of PLDS cases. Configuring the PLDS so that its effective leak-detection threshold increased monotonically as the case identifier changed through A, B, C, etc. distinguished these cases. Modifying the threshold determined an optimum configuration for the PLDS.
Fig. 2 shows typical results for a single case configuration, displaying the cumulative probability of detecting leaks for a range of specific sizes (expressed as a percentage of design throughput) as a function of time out to 24 hr. As expected, the performance is probabilistic.
The probability of detection increases with both leak size and time after initiation of the leak. The probability of detection begins to decline beyond this period, as noted previously. The results shown reflect the system as it is currently configured (Case C).
Moving to the right side of Fig. 2 allows re-plotting the probability of detection at 24 hr (as good as the system will ever get) as a function of leak size for each case configuration of the test PLDS. Fig. 3 shows this limiting probability of detection, PDET, for selected Cases A, C, and G. Using a criterion of 95% probability of detection within 24 hr gives each case an effective 95% confidence, 24-hr threshold, T24-hr, expressed in terms of the leak rate, qL.
For instance, the PLDS as currently tuned behaves as though it has an effective 95% confidence leak-detection threshold of roughly 0.35% of design throughput. This threshold does not necessarily correspond directly to any particular internal tuning parameter but is instead an effective threshold for the PLDS based on its internal configuration parameters coupled with both leak detection and its inherent signal uncertainties. Inspection shows that for the range of Cases A through G, the 95% confidence leak-detection threshold for the PLDS ranges between 0.2% and 0.75% of design throughput.
Lowering the PLDS threshold results, in principle, in early detection of an increased incidence of pipeline leaks.
For each of the configuration Cases A through G, it is also possible to determine the false positive rate, FPR, by running the performance analyzer against recorded data with no imposed leaks. Fig. 4 shows resulting annualized false positive rates plotted as a function of 95% confidence leak-detection threshold. A rate of roughly 40 false positives/year, or roughly 6 alarms/month, characterizes baseline Case C. Lower-threshold cases produce considerably more false positives.
The false-positive rate consistently declines as the effective threshold is raised. The data adhered well to an exponential fit, with r2 of about 0.95. Any attempt to detect more leaks by lowering the threshold results in more PLDS alarms. The higher incidence of alarms leads to either increased operator, controller, or support personnel time with the PLDS trending tools to diagnose the alarms, or to more time spent on field callouts.
References
1. “Computational Pipeline Monitoring,” American Petroleum Institute 1130, October 1995.
2. Carpenter, Philip S., Nicholas, Ed, and Henrie, Morgan, “Accurately Representing Leak Detection Capability and Determining Risk,” Pipeline Simulation Interest Group, November 2005.
The authors
Philip Carpenter ( [email protected]) is a consultant in Leander, Tex., with 20 years of experience in pipeline hydraulics, mechanical design, leak-detection systems, integrity analysis, systems simulation and analysis, and project and engineering management. He holds an MS in engineering science and a BS in aerospace engineering from the University of New York at Buffalo.
Morgan Henrie is an independent consultant in Anchorage who has been associated with leak-detection systems maintenance, design, project implementation, SCADA security, and state-of-the industry analysis for 20 years. Henrie, a PhD candidate, holds several degrees, including an MS in project management.
Ed Nicholas is an independent consultant in Austin who specializes in pipeline leak detection. He has worked on real-time pipeline applications for 25 years. Nicholas holds a BS in physics from Houston Baptist University, an MS in applied physics from the California Institute of Technology, and 3 additional years of full-time graduate work in engineering at the University of Houston.
