Special Report: Dynamic programming determines optimum catalyst life strategy

Sept. 2, 2002
A well-established mathematical algorithm known as dynamic programming, combined with advances in personal computing power, now allows straightforward optimization of fixed-bed catalyst life strategy.

A well-established mathematical algorithm known as dynamic programming, combined with advances in personal computing power, now allows straightforward optimization of fixed-bed catalyst life strategy. This article uses an FCC feed hydrotreater example to demonstrate dynamic programming.

Dynamic programming is a systematic method that balances present-day rewards against future penalties that are a direct result of today's actions. An Excel-based framework allows straightforward input of the pertinent relationships and event data.

The output graphically depicts the profit function and all variables as functions of time. The output also clearly identifies optimum turnarounds and catalyst changeout times. Dynamic programming methodology offers systematic solutions for the best catalyst temperature policy, optimum scheduling for the next catalyst replacement, and whether the replacement should be fresh or regenerated catalyst.

A typical refinery situation

It has been 2 weeks since the most recent catalyst replacement. The scheduling department is pushing hard on operations to maximize unit throughput and severity.

Operations, with backing from the technology department and the catalyst vendor, is pushing back. Maximizing severity or choosing particularly difficult-to-convert feedstocks generally accelerates the rate of catalyst fouling, thus dramatically limiting cycle length.

Operations is hoping for a deservedly long operating cycle, but the refinery linear programming (LP) model is predicting significant current profits if the unit is operated to push constraints.

What is the middle ground? How does one decide between immediate profits and long cycles? The glib answer is: "choose the policy producing the greatest present value based on life cycle economics." So how does one get there?

The process technology department usually takes on this strategic task by simulating catalyst cycles with varying input parameters such as unit feed quality, feed rate, and operating severity.

Engineers can make legitimate present-value comparisons by evaluating these cases wherein the catalyst replacements or regenerations for all scenarios coincide at some future date. This article demonstrates a different, and much less tedious, approach of dynamic programming.

Dynamic programming

If the process unit is at the end of its useful life, setting operating targets for the final couple of weeks is easy. Operations runs the unit hard, up to its physical constraints, without regard to catalyst life. This might be the first time that operations and scheduling agree; profits are maximized given the state of the catalyst and processing unit.

Two weeks before this point in time, the decision compared to the final 2-week period of operation may be different. If the catalyst is in good shape, one would utilize the catalyst's potential to its fullest; i.e., "burn up the catalyst" to maximize profits.

But what if the state of the catalyst is so poor that continuing to operate in a severe mode for even 2 weeks would make the catalyst inoperable during the final stage of operations? The decision would then be to back down on the severity to maximize profits during the final 4 weeks.

Click here to enlarge image

This process of determining optimum cycle operating strategy is the technique known as "dynamic programming." At any given point in time, working systematically backwards from the end of a process unit's life, all decisions affect the future in a known way and the cumulative (present) value is readily calculated. This statement is true for any and all states of a catalyst's condition. Fig. 1 shows how the algorithm can construct a general solution matrix.

Example problem-FCC feed hydrotreater

An FCC feed hydrotreater adds value to a refinery in several ways:

  • It upgrades FCC feed so that the FCC can produce higher-value products.
  • It reduces the amount of organic nitrogen compounds, thus allowing greater FCC operating flexibility for varying throughput, feed selection, or conversion.
  • It reduces the amount organic sulfur compounds, which quantitatively reduces FCC sulfur oxide (SOx) emissions.

A set of refinery, process, feed, and catalyst-dependent relationships describe these value-added concepts. These relationships contain independent variables such as feed rate and reactor temperature, and dependent variables such as sulfur and nitrogen levels in the hydrotreater liquid product.

The values of some variables will be constrained by physical limitations (maximum temperature, maximum hydrogen consumption), environmental regulations (maximum sulfur in the hydrotreater product), as well as other factors such as safety, scheduling, etc.

An example FCC feed hydrotreater problem with a single independent variable and several dependent variables will demonstrate the dynamic-programming method. The independent variable is weighted average bed temperature (WABT), with lower and upper bounds of 620° F. and 760° F., respectively.

The important dependent variables are related to:

  • Product upgrading—hydrotreater product organic nitrogen level and API gravity increase.
  • Environmental limitations—hydrotreater product organic sulfur level.
  • Heat—release control—H2 consumption.

The profit function is the net value of upgrading the feed through the hydrotreater after subtracting the variable and fixed costs. Catalyst deactivation is characterized as first-order with respect to catalyst activity with an exponential WABT adjustment.

Click here to enlarge image

Fig. 2 depicts graphical results of the deactivation equation. This relationship between the catalyst deactivation rate and process variables is generally the most arduous function for the process engineer to develop.

Creating general solution matrix

The accompanying box shows the basis and relationships used in the example FCC feed hydrotreater problem.

Click here to enlarge image

Table 1 shows the final six stages (12 weeks) of the hypothetical example. The calculations are for discrete catalyst states expressed as a percentage of fresh catalyst activity. Each element of the top array represents a cumulative value for a stage in time and a catalyst state.

For example, the hydrotreater's cumulative value for Stage 256 (Oct. 30, 2011) for a catalyst activity of 60% is $8.38 million.

The calculation process starts at Stage 260, the final 2 weeks of operation. Each cell, beginning with an activity state of 100, is sequentially calculated.

The algorithm uses Excel Solver to search over the entire operating range of the independent variable, WABT, to find the maximum hydrotreater profit. This procedure is repeated for each of the 11 states (100-0) of this final stage.

Because this is the last stage of operation, the answers are trivial; in all but the lowest activity states the optimum temperature is 760° F., the maximum WABT allowed due to metallurgical constraints.

Catalyst activities of ≤20 are insufficient to meet the product sulfur specification of 0.4 wt %. The hydrotreater is simply shut down for states equal to 0, 10, and 20.

Click here to enlarge image

A search through the permissible range of WABT for a temperature that maximizes the sum of the current stage value and maximized profit from the succeeding stage (260) determines the maximum profit for each of the catalyst activities in the next stage (Stage 259). The deactivation model calculates catalyst activities for each combination of WABT and current catalyst activity.

The elements in the array titled "Next stage activity" contain the catalyst activities calculated for successive stages. Since these are discrete elements, interpolation within the "Profit function" array is necessary to complete maximum profit calculation.

Elements in the array titled "Weighted average bed temperature" contain the respective optimized WABTs. Product sulfur contents are similarly represented in the "Product sulfur" array. Similar arrays will exist for each of the variables in a dynamic programming problem.

Example cumulative profit calculation

The maximum (two-stage) profit for a state of fresh catalyst (100) in the next to last stage (Stage 259) is $3.70 million. The Excel spreadsheet determined that the catalyst temperature is at its upper limit for maximum profitability.

Catalyst deactivation will result in the next stage's catalyst state = 86.5. To recreate the calculation for the maximized cumulative profit for the State 100 cell in Stage 259, one must add:

  • Single-stage profit resulting from the conditions of Stage 259 when the catalyst has a state = 100 and WABT = 760° F. This single stage profit value is $1.859 million.
  • Profit previously calculated for Stage 260 when the catalyst has a state = 86.5. This profit is determined by linear interpolation of the Stage 260 profits between state = 80 and state = 90.

The cumulative profit (Stage 259, State 100) is then $1.859 million + $1.823 million + ($1.844 million - $1.823 million) x (86.5 - 80) / (90 - 80) = $3.70 million.

This generalized procedure repeats for each state in Stage 259; i.e., choose the value of the independent variable (temperature) that maximizes the sum of the present profit (Stage 259) and the profit in the succeeding stage (Stage 260).

In Stage 258, the sum of the profits from Stages 259 and 260 are already stored in the elements of Stage 259. The calculations for each stage, granted that there are a large number of them, are only performed once.

Because some of the decisions being optimized take place at a future time, discount cost factors are normally used. All of the cumulative profits are calculated on a present value basis.

If the interest rate is high enough (e.g., 15%) and the last stage is placed far enough in the future (e.g., 10 years), the end effects of the programming become negligible. This "end-effect reduction" is graphically evident when creating an optimum operating strategy.

Optimum operating strategy

Assuming the unit is at Stage 255 (Oct. 16, 2011) and has a catalyst state of 100, Table 1 shows that the optimum path is maximum temperature for at least the next 2-week period.

This high-severity operation deactivates the catalyst to a state of 86.5 in Stage 256. Interpolation within the WABT array between States 80 and 90 in Stage 256 shows that the optimum policy is, again, to maximize temperature and further deactivate the catalyst to a state of 75.1 in Stage 257.

Click here to enlarge image

Table 2 tabulates the entire six-stage cycle. Fig. 3 shows the operating strategy for starting states of 100 and 42.2, respectively. One can start this solution from any date and for any catalyst activity.

Click here to enlarge image

This demonstrates a powerful feature of dynamic programming: Once the general problem is solved, all the optimized paths can be readily determined for any combination of stage and state.

Catalyst changeout

As catalyst activity declines, replacing the catalyst will become optimal at some time; i.e., cumulative profit for low-activity states is maximized only when the unit is shut down for a catalyst replacement.

One calculates this "catalyst-replacement profit" by subtracting all the costs associated with a catalyst replacement from the accumulated profit in the succeeding stage element that represents fresh catalyst activity (State 100). For example, the catalyst replacement is $3.20 million in Stage 255 ($9.04 million - $1.5 million - $4.34 million = $3.20 million).

The catalyst-replacement option replaces the maximum operating value as the optimum when either of these cases is satisfied:

  • The catalyst-replacement option is worth more than the unit's maximum operating value.
  • A process constraint would otherwise be violated.

There are times when the optimal path is to simply shut down the hydrotreater. One such time is represented by the low-activity states of Stage 260; it simply makes no sense to spend money to replace the catalyst.

Fully developed example solution

The previous example problem is now fully developed to include FCC maintenance turnarounds, discounted cash flow factors, and a life of 260 stages. The state step size is also reduced to 2 from 10 to improve the algorithm's precision.

The general problem was previously solved, and the general matrix of all optimized solutions is available. The user now maps the current date and current catalyst activity onto the general matrix to begin construction of the optimal path.

Fig. 4
Click here to enlarge image

A stage-by-stage interpolation algorithm enhances this step. Figs. 4 and 5 depict example catalyst life strategies.

Fig. 5
Click here to enlarge image

In the first example (Stage 0 and State 100), the refinery has just started up the FCC feed hydrotreater after a catalyst replacement and full maintenance turnaround in December 2001. The optimal strategy is to replace catalyst about every 17-18 months (36 stages) and during forced shutdowns that correspond to the planned FCC turnarounds every 4 years.

The optimal operating strategy implies that the start-of-cycle WABT should be relatively low, increasing slowly during the first part of each cycle. The temperature increases are more frequent in each cycle when the maximum sulfur limit is reached.

Cycle repeatability demonstrates the insignificant influence of the end effects on the optimal strategy for at least the first 5 cycles.

A slight variation to the simulated problem is to start the catalyst activity much lower than expected due to some glitch in the unit start-up.

Fig. 5 depicts the optimal path for a unit with a fresh catalyst activity of 40%. In this case, the best course of action is to replace catalyst after only about 20 weeks. Start-of-cycle WABTs for each of the next 2 cycles are much lower than they were in the previous example life strategy.

These lower start-of-cycle WABTs allow the respective cycles to be long enough to optimally take the unit to the FCC turnaround in stage 103 (Dec. 18, 2005).

A significant change in relative economics, fresh catalyst activity, or catalyst cost, or a change in the valuation of FCC feed quality can all have a major impact on optimum hydrotreater operations. The operator should continuously update the optimum strategy, therefore, by rerunning the catalyst life strategy model with current data and relationships.

Potential applications

Dynamic programming can optimize virtually any process that declines in value over time as a known function of process variables. Examples include:

  • Semiregenerative reformers.
  • Hydrotreaters.
  • Hydrocrackers.
  • Toluene disproportionation reactors.
  • Xylene isomerization reactors.

Sensitivity studies allow the program user to explore other alternatives such as the use of regenerated catalyst or a new catalyst formulation. Other potential uses are to evaluate projects that alter process constraints or change feed composition. Exploring changes in turnaround timing or evaluating different refinery economic scenarios are other potential applications.

One of the more interesting and challenging applications of this technique is simultaneously optimizing reactors or process units in series. In a two-stage hydrocracker application, the catalyst functions, costs, and deactivation rates are separately characterized, but the global profit function is fundamentally related to the operation of both stages.

This complex two-stage hydrocracker problem is ideally suited to dynamic programming optimization.

Computer processing concerns

My original exposure to solving the optimum catalyst cycle strategy with dynamic programming was on a mainframe computer. The method was fairly straightforward, but interpreting the results from mountains of computer printouts was tedious. Meanwhile, the early desktop computers that replaced mainframes lacked the speed and memory capacity required for dynamic programming.

Today's desktop computers, however, are powerful enough to crank through the repetitive calculations in a reasonable amount of time. The fully developed FCC feed hydrotreater problem was solved after about 2½ hr of processing time on a 450-Mhz PC with 128 Mb of RAM.

Although complex and highly nonlinear relationships will involve considerably more computing time, the most recent personal computers are up to the task.

Model details and limitations

Although I have not found a real limitation on the number of independent variables, three or more variables can be a handful. The number of permissible constrained variables is much larger (100). Limits on the constrained variables can be changed at any time.

One can use a new set of economics (component or stream values) for each stage, making seasonal operation programmable. The algorithms easily handle predetermined events such as turnarounds or known catalyst changeouts.

The key to getting good value out of a complex dynamic programming model is to provide simple and precise relationships for the profit function, deactivation rate, and constraints. All functions must be smooth and absent of local maxima within the constrained range. Yields must be weight balanced.

The author

Click here to enlarge image

Gary Kuchcinski is currently an independent consultant for refinery operations. Previously, he worked for Amoco Corp. and BP PLC for 26 years before retiring in August 2000. He worked at the Naperville, Ill., research and development lab and at Amoco's Texas City, Tex., refinery where he was responsible for technical consulting in the hydroprocessing and catalytic reforming areas. Kuchcinski most recently worked on a contract basis for UOP. He holds a BS (1970) in chemical engineering from the University of Toledo and an MS (1972) and PhD (1974) in chemical engineering from Purdue University.