SPECIAL REPORT: Monitoring, best practices maintain APC applications

Benefits associated with advanced control are due to reducing variability in key process variables through multivariable model based control and subsequently operating at the most profitable constraints through a constrained optimization scheme that is often a part of the APC application (Fig. 1).

Fig. 1 shows that once the application is commissioned, various reasons may cause the APC application to move away from an economic optimum. This results in deterioration of the benefits from the advanced controller.

Based on our experience in commissioning and supporting many APC applications during the past 20 years, some of the main reasons for this deterioration include:

• Lack of APC resources.
• Lack of operator training.
• Changes in process dynamics.
• Inability to quantify APC performance.
• Inability to link APC benefits to process performance.
• Lack of APC best practices.
• Inability to visualize model quality.
• Inability to visualize and assess performance of the supporting cast: PID controllers, valves, and inferentials.
• Inability to rationalize and explain APC behavior in real time.
• Lack of management support.

This article gives a broad overview of the most commonly encountered issues and best practices when companies try to sustain performance of their APC asset base. We have shown examples wherever possible to illustrate these issues.

APC

Implementation of an APC application in an operating facility has an immediate positive effect in many ways.

Fig. 2 shows that APC reduces variability in key operating parameters by moving that variability to available manipulated variables.

This reduction in variability may, in and of itself, be valuable, but reducing variability creates an opportunity to shift the operating point of the process closer to limits and constraints. This shift is normally the key contributor to value delivered by APC applications (Fig. 1). A recent survey of industrial practitioners and vendors indicated that the economic assessment of advanced control applications is still ad-hoc.³

Moving and holding to this new operating point changes the performance requirements of many other aspects of the business. If these changes in performance are not achieved, benefits delivered by the APC application will be less than its potential.

These elements include:

• Instrumentation. Faulty instruments will increase instability or cause part or all of the APC application to be turned off, or otherwise reduce the value that the application delivers. Ensuring the application delivers full value requires improved identification of instrumentation issues and modified prioritization of repair efforts to account for this effect.

• Computer systems. Issues with computer hardware, software, or networks will prevent communication between the APC application and the process. This will directly reduce application uptime. Adding APC and control monitoring systems also increases the number of systems that must be maintained. It may also increase the complexity of the system maintenance problem.

• Process. Many aspects of the process will reduce the application’s effectiveness. Some of these include unmeasured or infrequently measured limits, unmeasured disturbances, process behavior changes (such as heat exchanger fouling, or catalyst decay), and process modifications.

To enable an application to continue to deliver high value, the operator must identify when aspects of the process are limiting the value delivered and either adapt the application or repair the process. Changes in process economics may invalidate some of the application’s control objectives.

• Control system. The control system itself can cause poor control performance. When constraints become active for the first time, because the process has moved to a new operating region, poor quality models that were once unimportant become important. Also, improvements in the system’s fault tolerance are required for the application to run when other problems (instrument, process, computer, or other) exist.

• Staff. The skills, knowledge, and capabilities of staff assigned to support APC applications have a large effect on the success of the applications. Having work processes and tools in place to better support that staff is important to allow for succession and reducing the work when new people take on the job.

Satisfying these more stringent requirements is difficult. Having tools that automatically generate performance statistics will allow properly trained staff to use good work practices and achieve high application performance.

Best practices

Precise activities and people performing those activities involved in monitoring and maintaining an APC application depends upon the organization’s particular requirements. These requirements include desired core competencies, value that the applications deliver, and number of applications needing support.

When an organization’s requirements lead to having a control engineer performing the monitoring and maintenance tasks, tactical high-value job activities might include:

• Daily monitoring of application performance and issue identification.
• Regularly challenging the application limits, seeking ways to allow it to do more.
• Performing regular maintenance on the application to improve fault tolerance and control performance

Strategic high value job activities would include:

• Documenting procedures and activities to enable job succession better.
• Supporting regulatory control.
• Examining and challenging performance benchmarks and target operating points.

Table 2 shows APC application performance categories and adaptive or improvement action that should be taken.

Monitoring APC applications

There are four different aspects of analyzing a multivariable-constrained control scheme that is model-based:

1. Utilization. This is the first level of analysis. If the controller is unused, there is likely a fundamental problem. Similar comments apply to the use of individual manipulated variables (MVs) and controller variables (CVs). If critical variables are being turned off, it affects the controller’s ability to deliver performance or benefits.
2. Performance. A controller may be on, yet performing poorly. In such cases, one must analyze performance of the various CVs, constraint variables, etc. An MV may be on, but at its limits, which means it is not being effectively used.
3. Diagnostics. If performance is not meeting expectations, there could be several reasons: model plant mismatch, improper tuning, instrumentation issues, etc. If crucial CVs are being under or overpredicted, responsiveness of the application to address any disturbances or target changes will be diminished.
4. Remediation. Having understood the reasons for unsatisfactory performance, one often must prioritize and make those changes to the controller or process that will result in the improvement of performance without excessive cost or effort.

Fig. 3 shows a representative schematic of the typical workflow involving the assess, diagnose, and remedy steps for any APC application

Performance monitoring

Measurement of APC performance is a nontrivial problem. Three key aspects of APC performance include variability, constraint performance, and model performance.

Variability

Variability is an important measure because it is the basis on which APC applications are implemented. It is important to ensure that the multivariable controller reduces variability up to the level anticipated in the initial estimates.

A multivariable controller is able to derive benefits by operating the unit against the right constraints. It is important, therefore, to monitor active constraints and ensure there are sufficient degrees of freedom for the controller.

Lastly, the model is at the heart of the predictive controller and is the single most important tuning parameter. Tracking model performance and ensuring that it is satisfactory ultimately ensures good health of the controller.

Variability is a commonly used approach for establishing variability measurements relative to a historical baseline data set. Before commissioning, a historical baseline is selected. Typical estimates of the reduction in process variability due to multivariable control are about 50%.

After commissioning, the variability (i.e., standard deviation of key controlled variables) is compared to the baseline and improvements noted. Postcommissioning variability then becomes effectively the new benchmark for further comparison. The expectation from any application is that it achieves at least the postcommissioning variability.

Seasonal changes account for long-term patterns in underlying variability. For example, summer conditions may often mean a distillation column is cooling limited and variability is higher. In technical terms, there are two benchmarks that are considered here: open-loop variability, prior to controller commissioning; and closed-loop variability, postcontroller commissioning.

A shortcoming of using absolute variability as a benchmark is its dependence on underlying process disturbances. One can compare variability for APC applications at two points in time in a fair manner assuming that nothing else has changed—disturbance magnitude and dynamics are the same.

This can lead to both false positives and false negatives. Variability may decrease due to the overall process being more stable or variability may increase because of process changes being made elsewhere.

A more accurate representation of closed-loop performance is based on closed-loop performance requirements such as settling time, rise time, overshoot, integral of the absolute error, integral square error, etc. These measures are invariant to the scale of the disturbance and are thus normalized. Some commercially available APC technologies directly incorporate such a measure into their controller tuning stage.

Desired variability can be established through historical benchmarking (postcommissioning) or design considerations (closed-loop response requirements) as previously discussed. One can calculate relative variability at an individual CV level and aggregate it to the controller level (see Equation 1 in attached equation box).

Constraint performance

An operator or engineer interacting with an APC application would like to know if:

1. The right constraints are targeted.
2. The key constraints are not being compromised.
3. There are adequate degrees of freedom to achieve 1 and 2.

Quite often, one comes across an APC application that is being used but the limits on critical MVs have been tightened. This leads to a situation in which the controller is effectively full open due to a lack of degrees of freedom.

The reason for tightening MV limits may be due to a lack of performance or a lack of training. In either case, the real measure of controllability of any application is effective utilization rather than merely having individual variables or the application being used (on).

A measure called effective utilization (Equation 2) measures the relative degrees of freedom available to a multivariable controller.

This measure can be calculated over any time scale and captures the degrees of freedom or manipulated variables available to the controller. It can be calculated at the variable level and aggregated to the controller level with a weighting scheme. One should consider availability of the process unit when computing this measure.

Model performance

Key questions a practitioner is interested in answering are:

• Which models are at fault?
• What is the effect of model error on performance?
• What model parameters does one need to update—gain, time constant, or time delay—to recover performance?

In the ideal case, one would like to answer these questions based on operating data without having to perform intrusive testing. Commonly used measures of model quality are based on prediction errors and a visual comparison of measurements with unbiased predictions (or simulation).

A disturbance model other than the integrated, white-noise model is rarely used in commercial APC technologies; therefore, engineers focus mainly on comparing the error in the model’s deterministic portion.

Fig. 4 shows a vivid example of the scale and complexity of the problem in the form of a 40 MV x 94 CV x 3 feed forward (FF) variable model matrix from a crude column APC application (Fig. 4).

The color-coding indicates positive (red) and negative gain (blue) models.

An operator or APC engineer who is interacting with this application daily must solve the puzzle of which MV-CV models are causing a particular behavior—particular constraint tradeoffs or variability increases—and the appropriate actions needed to rectify that model behavior.

As a start, one can measure the relative variability of the model prediction compared to the CV measurements (Equation 3).

The prediction is the raw or unbiased prediction for each CV. This allows tracking of the process predictability in relative terms and flag significant deviations from benchmark performance. A good benchmark comes from the postcommissioning data for which model performance is considered satisfactory.

Table 3 shows some guidelines for the key variables for any APC application. Variables listed are at the overall application level and related to individual CV-MV performance measures through a weighted average based on the relative importance of each CV and MV. Variability is relative to design performance or desired variability. Model performance is relative to benchmarked model performance.

These targets take into account that there will be short-term variations in the different performance measures due to operating point changes and disturbances. It is essential that the ownership team agree on these goals for the APC applications.

Performance monitoring

A team involving vendor and ownership personnel—consisting of process control engineers, operators, and managers—often commissions an APC application. The process control engineer with appropriate APC training is eventually responsible for maintaining and troubleshooting the controller as necessary.

The operator uses the application on shift basis to ensure that the process is operating at the right state, it is stable, and the key constraints are being met. The process control engineer is involved with the application daily to ensure that any issues raised by operators are addressed, known issues with the application are addressed, and operators are running the controller as recommended.

The technical and process control personnel and plant managers are often concerned with adequate uptime of the application, given the level of effort and resources that have been invested in commissioning the application. If it is a well running application, operators, shift supervisors, and plant managers will treat the APC as they would any other plant asset—it is essential to the smooth operation of the facility and needs to be running well.

Based on profiles of the ownership personnel, the kind of information each member wants differs. The composition and roles of APC ownership discussed here are representative; each organization will have its own roles and responsibilities based on resource availability and organizational structure.

• Operator. An operator wants to know, from a real-time interface, what the controller is doing at any given instant and why it is doing so. In more technical terms, this is the knowledge of active constraints and the cause-effect behavior between MVs and CVs that cause constraints to be active or given up on.

• Shift supervisor. A shift supervisor is concerned if the operators must pay too much attention to one or more controllers and they are having to take the application offline. In other words, the level of operator interactions, process stability, and controller uptime is of concern to this group.

• Plant manager. The plant manager has a reason to be concerned if the controller uptime is unsatisfactory and the controller is not achieving process stability.

• Production engineer. The responsibility of the production engineer is to achieve the production targets that have been set by the planning and scheduling team and the plant manager. Safety is a key priority for this individual. He or she tends to be concerned with the APC application if it is engaging in behavior that, for example, cuts feed rates and compromises the production targets or is causing off-spec incidents resulting in emissions or off-spec product. Operators or shift supervisors usually provide this information.

• Process engineer. A process engineer is involved in the application’s design to ensure that key variables are included and the controller targets the “optimal” constraints. At some sites, a real-time optimizer (RTO) layer may sit above the APC layer and send targets and set points.

The process engineer wants to ensure the APC application is achieving the target changes that may be sent by the RTO every few hours. The process engineer may also become involved in troubleshooting if the underlying process behavior is causing a problem with the application.

• Technical, process control manager. This group is concerned with application uptime and benefits. One of the other groups, if necessary, will bring to its attention issues requiring additional resources. If the process, for example, has undergone some changes recently and a full step test is required to update the plant models, this group will be involved in justifying the resource allocation.

• Process control engineer. Any issues observed by the rest of team are important to the process control engineer. In addition to controller uptime, active constraints, and process stability, the process control engineer is also the one who is concerned about the “why” part.

This means that things like process variability relative to a benchmark, cycling in the controller, degrees of freedom available, constraint give ups, underlying PID controller performance, inferential and analyzer performance model performance, operator interactions, tuning problems, and process nonlinearities that affect application behavior all become important to this individual because he has the responsibility for addressing these issues.

• Instrument and electrical technician. This person only gets involved if an issue raised by one of the other people needs to be addressed to recover APC performance. An example is a sticky valve or a malfunctioning sensor or analyzer. This group needs information on the diagnostics pertaining to the instruments and valves that have been flagged.

Table 4 shows a representative responsibility matrix for the different aspects of an APC application. In this article, we have reviewed a few key measures pertaining to use, performance, and diagnostics.

Table 5 outlines the typical time scales for monitoring different facets of the APC application, as we outlined in the roles previously discussed.

References

1. Qin, S., and Badgewell, T., “An Overview of Industrial Model Predictive Control Technology,” Control Engineering Practice, Vol. 11 (2003), pp. 733-64.
2. Brisk, M.L., “Process Control: Potential Benefits and Wasted Opportunities” in 5th Asian Control Conference. Vol. 1, 2004, pp. 20-23.
3. Bauer. M., and Craig, T., “Recent Economic Assessment of Advanced Process Control—A Survey and Framework,” Journal of Process Control, Vol. 18 (2008), pp. 2-18.

The authors

Rohit Patwardhan is the product manager for the control performance monitoring suite of solutions at Matrikon Inc., Edmonton. He has worked in the area of automation with specific focus on process control for the last 8 years. Before joining Matrikon, Patwardhan obtained his PhD in process control. His research focused on designing and monitoring model predictive controllers.

Blaine McIntyre is director, control performance solutions at Matrikon Inc., Edmonton. He is responsible for helping customers realize the full potential from their process control assets has worked in the field of process control and advanced process control for 19 years. During this time, he has maintained systems, implemented and managed new projects, developed control maintenance strategies, and managed process control personnel.