Petroleum firms share information openly to meet Year 2000 deadline

A plant such as Texaco Ltd.'s 180,000 b/d Pembroke refinery in Wales, shown here, presents a massive Year 2000 puzzle. A refinery's control system involves hundreds of devices-such as valves, transmitters, and flowmeters-that contain embedded systems, each of which has the potential to cause a failure when it first encounters a date in the next millennium. Photo courtesy Texaco Ltd.

• Common Embedded System Uses [161,318 bytes]
• Year 2000 Safety Decision Tree [250,479 bytes]
• This Year 2000-compliant Fieldvue "smart" valve, manufactured by Fisher Controls, contains embedded technology that controls flow rates and communicates with the overall plant control system. Each such embedded system must be checked for Year 2000 readiness and repaired, if necessary. This installation is at Monsanto Co.'s Chocolate Bayou petrochemical plant at Alvin, Tex. Photo courtesy Fisher Controls [33,009 bytes].

With the Year 2000 deadline bearing down, petroleum companies are in the midst of solving a technological problem with a scope never before seen in this, or any, industry.

This high-tech, data-intensive business is fraught with potential Year 2000 pitfalls, and companies are taking a careful, measured approach to finding and remediating these problems. Their approach includes the painstaking process of inventorying software and equipment, searching for potential date problems, fixing any noncompliant components that are found, and testing the remediated systems to determine whether their efforts have been successful.

But petroleum companies' Year 2000 programs have also included an open sharing of information with competitors that seems unprecedented (see related story, p. 24). It is this sharing, and the realization that the interdependence of the industry necessitates such cross-pollination, that makes oil companies-especially the majors-quite confident that they will be prepared for the Year 2000 when it arrives (see related story, p. 26).

Date traps

For an industry that can visualize underground formations in four dimensions, drill for and extract oil in the world's most treacherous ocean environments, and make clean-burning transportation fuels from heavy oil, the problem of a date abbreviation in computers might seem easy to solve.

Technologically speaking, it is. But the Year 2000 problem still presents many challenges. These result from: the complexity of a highly integrated and automated industry, the difficulty of finding all the potential date problems because of this high degree of complexity, and a lack of time and resources available to solve the problems that are uncovered.

"Major facilities like refineries and chemical plants are operated using highly sophisticated control technology," states Royal Dutch/Shell's website. "There may be thousands of embedded (computer) chips in the plant run with several layers of networks (see table, p. 29).

"In an extreme case, if safety-critical criteria at a plant cannot be met, then the plant can be shut down. But what happens, for example, to the continuous gas supply from North Sea fields into Britain-a stream heavily controlled by IT (information technology) systems? Will it just be switched off if the safety-critical issues are not resolved?"

Addressing the problem

Thomas McAndrew, senior partner in Computer Sciences Corp.'s consulting and systems integration division in Boston, says companies must ask themselves three questions in order to tackle the Year 2000 problem and solve it comprehensively:

• How do I manage a program that could topple the enterprise?
• How do I bring my enterprise together as one?
• How am I going to mitigate the risk?

Shell's Year 2000 team started by identifying and prioritizing "safety-critical and business-critical systems." The U.K. Health and Safety Executive has published a safety decision tree to help companies make these determinations (see diagram, p. 30).

Many operating units in the Shell group of companies began carrying out audits in 1997 to determine the scale of their Year 2000 problems and to develop a plan for solving them by yearend 1998. Meanwhile, an inventory of the potential effects of the problems was tallied in order to assess the magnitude and cost of remedial action. Upon completion of the audits, operating units began carrying out remediation plans.

Shell's policy is to assume that every system is at risk unless it is demonstrated to be otherwise.

Another approach taken by Shell is to involve all levels of the organization in finding Year 2000 problems.

Nick Thompson, manager of the Shell Information Services Year 2000 support team, said, "There are so many things that can be affected that we need all the help we can get. Everyone should be on the lookout for potential problems.

"We are saying to all staff in all fields of operations that they could be the person who identifies potential problems first. Nobody has the inside edge on this issue, because it is a weird and unique event. Everyone should try to think around this, and, if they come up with something, do not be shy about raising it because of the false assumption that there are people who have spotted it already."

Remediation

The process of fixing Year 2000 problems involves three key steps: identifying components that have a date sensitivity, testing those components to determine whether they are Year 2000 capable, and fixing those that are not. Any items that are remediated-software or hardware-must then be checked to determine whether the solution is effective.

Complicating the issue is the fact that 2000 is a leap year, even though there has been some doubt about whether this is true, given the complex leap-year rules for years ending in two or more zeros. Nevertheless, software programs and control systems must also be capable of making this adjustment at the end of February.

The Year 2000 problem places enormous identification, testing, and remediation burdens on a petroleum company. These burdens can be divided into sections to facilitate problem solving.

British Petroleum Co. plc divides its Year 2000 project into three main sections: IT systems, engineering process control, and intercompany relationships. Shell divides its program into asset integrity (includes control and monitoring systems on platforms and in process plants), business computing (IT, telecommunications, and infrastructure), and commercial integrity (relationships with outside parties).

Amoco Corp. makes further distinctions, however. Its Year 2000 program addresses:

• Mainframe business computing systems.
• Personal computers.
• Plant process control systems.
• Facility management systems.
• Supply chain, including customers, suppliers, and business partners.

Software

In the area of software, firms must search for potential date problems in each of the many computer programs that they use.

There are a number of commercially available tools on the market designed to test for and fix Year 2000 problems in software, but they are not infallible. There is no easy solution.

For some companies, fixing software problems will involve implementing an expensive, Year 2000-compliant, enterprise-wide software system. But, because most firms also use software developed in-house, it will also be necessary for them to examine certain software code, line by line, and correct any problems that are found.

BP's website addresses the difficulties involved in solving Year 2000 problems in software systems: "Compiling the inventories and assessing them for business impact have taken longer than initially envisioned. We find that even a limited level of customized (software) systems forms a tough Year 2000 problem.

"...Even some recently written applications have been found at risk. For example, we have only recently fixed credit card processing at some retail sites, due to waiting for new software releases."

BP believes companies may find it necessary to evaluate 80% or more of all software code in their library. And programmers typically make three mistakes for every 100 changes to program code, says the firm.

A typical 250,000-line software application may require 7,500 changes, so a potential of 250 errors have to be found and corrected in testing, according to BP.

Other estimates of error introduction by software code writers are much lower: 100 errors per million lines of code, says Systeme Evolutif Ltd., London.

Control systems

In the hardware arena, oil companies are presented with an even more complex challenge.

In a technologically intensive industry, companies operate thousands of pieces of equipment that may contain date sensitivity. The main source of this sensitivity is what is called "embedded technology"-computer chips that are "embedded" in equipment and that communicate with a main control unit.

Such devices include flow meters, transmitters, and "smart" valves, among myriad others (see photo, p. 32). They are found in all sectors of the industry-on drilling and production platforms, along the lengths of pipe- lines, and throughout process plants.

A typical offshore platform or onshore gas plant uses 50-100 embedded systems, said Shell U.K. Ltd. Corporate Affairs Director John Mills at a Petroleum Economist conference in April. And these systems can contain as many as 10,000 microchips.

In a process plant, devices containing embedded chips are interconnected like an electronic spider web, making the problem even more complex and the possibility of a failure even more likely.

"We have found that up to half of these systems are critical, in terms of production and the impact of our activities on the environment," said Mills.

Steve Rowlands, Year 2000 project manager for CMG U.K. Ltd., London, says the potential for major problems in control equipment is great: "It could take out a major part of the operation.

"The best example I can think of is valves that control pipelines. (These) valves are designed to 'fail safe.' Because they have a maintenance cycle built in, they will fail safe if they haven't been examined within a certain period of time. (In 2000), they (may) suddenly decide that they haven't been checked in 99 years.

"Probably the oil industry is one of the most safety-conscious industries going. They build their systems to be safe," said Rowlands, suggesting that these precautions can present myriad Year 2000 traps.

For Shell, after identifying critical embedded systems, the next step is to agree on a comprehensive plan of remedial action. According to Mills, "This can involve chasing vendors for evidence that they will be able to cope with the Year 2000, testing the equipment, and developing contingency plans in the event of a system failure.

"This action must be agreed, planned, and then scheduled with each of at least 60 vendors," said Mills. "It has to be extremely well coordinated and planned, as no shutdowns are scheduled. And the logistics of this exercise are, of course, considerable."

Because of the number of embedded systems involved, Shell sees downstream as the most difficult and costly area to resolve, with respect to the Year 2000 problem.

"A modern refinery relies heavily on automated control systems," said Mills. "The basic systems are usually supplied by large, specialized companies that are actively developing Year 2000-compliant upgrades. However, most control systems have many local user applications added as enhancements. These all have to be checked for hidden date dependency."

BP says embedded devices form a central part of its Year 2000 work: "We have been working with suppliers to upgrade many systems to the level that they have certified as compliant and (are) testing the site-specific applications.

"Safety is paramount. Our reviews have not detected any new safety risks, because, in the main, the ultimate safety systems are either not date-sensitive or do not rely on programmable systems.

"The volume of faults found which might cause a plant shutdown has been low-up to 5%-and we aim, through timely remedial work and robust contingency plans, for minimal disruption to our production facilities."

In its survey of process control systems, Suncor Energy Inc. identified critical process controls that were noncompliant for Year 2000, plus critical process controls that could not be assessed for compliance without significant disruptions to operations. These systems will be replaced during scheduled maintenance shutdowns, says Suncor.

David Trim, a consultant with Shell Information Services' Year 2000 team, believes embedded systems in downstream operations present the trickiest piece of the Year 2000 puzzle: "As far as process control is concerned, as yet, we still do not know the full story. We are finding enormous levels of complexity, and this has to be understood.

"For instance, one problem is finding people who can understand the whole picture in process control: at one level, you have IT people, and at another, the engineers. We have to understand the total picture."

External disruptions

It is not enough for a company to simply find and rectify its own Year 2000 problems. It also must work with outside parties, such as customers, goods and materials suppliers, technology vendors, service providers, joint venture partners, subsidiaries, and financial partners.

An issue of great importance in Year 2000 programs is the possibility that a company can experience a failure as a result of another firm's computer systems.

A company does have some control over these external disruptions-gained through the exchange of preparedness assurances-but less than it has in testing and fixing its own systems.

Exxon Corp.'s 1997 annual report somewhat softened its Year 2000 preparedness update by following it with admonishments regarding the possibility of external disruptions: "Comprehensive plans for achieving Year 2000 compliance were finalized during 1997, and implementation work was under way at yearend. While plans are in place, significant work remains to be done.

"Most required systems modifications are expected to be completed in 1998. Also during 1998, attention will continue to be focused on the compliance attainment efforts of vendors and others, including key system interfaces with customers and suppliers.

"Notwithstanding the substantive work efforts described above, the corporation could potentially experience disruptions to some aspects of its various activities and operations as a result of noncompliant systems utilized by unrelated third-party governmental and business entities. Contingency plans are therefore under development in order to attempt to mitigate the extent of such potential disruption to business operations."

In Shell's case, said Mills, "As a minimumellipsewe must satisfy ourselves that the contractors providing (drilling, engineering, seismic, and maintenance) services are paying enough attention to theellipseproblem. This inevitably requires considerable effort and, above all, a cooperative approach.

"Both operators and contractors must realize that they are in this thing together. No company in this industry can isolate itself from the problem. As an industry, we must be more open about our remedial plans and more willing to discuss the progress and pitfalls of meeting them."

The testing challenge

In addition to the challenges of finding and fixing Year 2000 problems, the changeover to the new millennium involves testing on a scale never seen before. BP says fixing the date problems in software programs before the deadline is less than half the job.

"Very little of this testing can be automated," said BP in an internal memo on the Year 2000 problem. "Tools can reduce the effort by perhaps 2%, but much of the testing involves business units."

McAndrew said, "There is no single tool that identifies all of your date sensitivity, let alone fixes it, so you need people and processes as part of your renovation program." He estimates that 40-45% of a company's effort will be taken up by testing.

BP believes testing will consume 60% of the time and money spent on Year 2000 programs. Other estimates have ranged as high as 70%.

McAndrew says this won't happen, not because it shouldn't, but because there isn't time. Programs aren't progressing quickly enough, plans are not pragmatic or achievable, and he is not seeing contingency plans, he says.

According to Rowlands, "The real challenge is in the embedded stuff. It's not a problem in terms of diagnosing and fixing them. It's fixing them in time and testing them."

Companies know what equipment they've got and where the chips are, says Rowlands. So the first order of business is to get a statement from the manufacturer of each device stating whether that device has a date problem.

Here, the liability issue arises again. It is not adequate to simply take the manufacturer's word that a device is Year 2000 compliant. Proof is necessary.

On a case by case basis, says Rowlands, companies must decide which equipment or systems to test themselves and which to send to the manufacturer for testing.

"The manufacturer is going to have to supply you with some kind of fix or new system," he said, and the nature of this will depend on the system involved. On some systems, faulty parts can simply be replaced. Others will have to be shipped off for repair or replaced.

In most cases, the user's goal should be to fix these systems onsite. If a company has 1,000 miles of pipeline, says Rowlands, it wouldn't want to shut the whole thing down and send off each device. It would want to do it on a maintenance basis.

Test methods

The choice of testing methods is another complicated issue.

"You have to do some specialist testing," said Rowlands. "You have to find a way of simulating the rollover."

According to an unpublished paper by Kevin McGinty, Jennifer Hale, and J. Michael Burke, of Science Applications International Corp. (SAIC), there are three common methods of fixing Year 2000 problems: windowing, sliding windows, and date expansion.

"Windowing applies logic to identify if a date occurs before or after the year 2000," said the three SAIC authors. "Each company selects an appropriate period of time for their particular business, such as the years 2000-30. This period of time, or window, is compared against a two-digit year that is passed into it. If these digits match, or are within the range of the window, then it is assumed that the first two digits of the year are 20, i.e., 2020. Any other date not in the window is assumed to be 19, i.e., 1975.

"The sliding window is a variation that uses the same range but moves incrementally forward. For example, in the year 2000, the window years are 00-30; in the year 2005, the window years become 05-35. This continues until the problem reintroduces itself in the year 2075.

"Date expansion is the third and most costly, but undeniably the most permanent, solution," said the authors. "All data and all programs are modified to accept the eight-digit date fields (mm,dd,yyyy). Most companies avoid this last option because of the significant cost involved and the belief that all existing systems using one of the windowing techniques will be fully replaced before future date logic problems will surface."

The type of testing used should depend on the criticality of the item involved, says McAndrew. At the bare minimum, it is necessary to perform what is called a "time-shift" test, which tests all the different scenarios that could cause the Year 2000 problem.

"True time-shift testing is an enormous undertaking," said McAndrew. Simply changing the system date handles only about 10% of the potential problems, he warns.

According to the BP memo, "Even the act of testing is perilous. For example, so-called time-machine tests (where the system clock is advanced to the year 2000) can bring down networks and cause e-mail to be automatically deleted.

"The magnitude of the testing task scares some managers into taking shortcuts-including the omission of entire testing procedures.

"Just scheduling time to test Year 2000 fixes is a major concern," the memo continues. "Many disaster recovery facilities are being used as test facilities, and scheduling their use for Y2K work is quite difficult. There is little open time left in their schedules during the heaviest periods of expected testing activity."

Preparedness

It is difficult to assess the progress that has been made toward solving a problem as complex and far-reaching as the Year 2000 bug.

Trim said, "The exercise of just trying to understand the extent of the challenge that we face is quite awesome.

"Frankly, some groups in Shell are much further advanced than others in trying to grasp that. Some have completed an audit only to realize they have missed whole areas they needed to consider."

Nevertheless, the petroleum industry appears confident about its ability to copy with Year 2000 date-change glitches. As a general rule, petroleum firms started early and plan to finish early.

Like many companies, including BP, Amoco Corp., Chevron Corp., and Kerr-McGee Corp., Shell U.K. has set itself a target date of yearend 1998 for completing all remediation work.

"That's an ambitious program," said Mills, "but it has to be. "We have deliberately built into our targets a year for testing, and for important contingency planning.

"Our goal is achievable," he said. "It is to develop and implement plans for all perceived threats, and to achieve assurance levels that reflect, in each case, the business criticality of that threat.

"The aim is to reduce the impact of the Year 2000 problem to manageable levels in all areas-a target we are confident can be achieved."

Embedded technology is an issue that concerns many companies when it comes to preparedness. Although it is a recognized, very high-priority problem, said Rowlands, companies seem to have it under control.

Having said that, he admonished that it is not until after the testing phase that companies can prove that what they thought was correct was actually correct.

The plans are in place, he said, and companies are working against their own deadlines. Their goal, in general, is to have the systems in place by yearend 1998, and to iron out any remaining problems in 1999.

"There are cases where they won't roll out the whole problem until mid-'99. That represents a risk they will have to manage.

"They can always do things smarter and more efficiently, but they seem to have the thing under control. Control is about juggling balls," said Rowlands. "You're lucky if you can keep them in the air.

"If they could do something better, it perhaps is communicating to the outside world and allaying fears and communicating that they are in control."

Trim disagrees, saying that more guidance is needed from government and: "More needs to be done at the highest levels. We need an emergency-response type of thinking."

Cross-pollination

Despite the liability issues involved in the Year 2000 problem, petroleum companies seem to share a spirit of cooperation when it comes to this issue. The general consensus is that sharing ideas is the surest method of preparing for potential failures.

"In Shell," said Chris Fay, chairman of Shell U.K. Ltd., "we view Year 2000 as a cooperative, not a competitive, issue.

"No company is an island, and we all rely heavily on each other as customers, suppliers, and co-venturers. Given that this is one project that cannot cross its deadline, it is crucial that all concerned work together with real urgency to minimize the Year 2000 impact on all of us."

Shell's Mills agrees. "We recognize that our responsibilities lie first and foremost in ensuring that our own systems work as smoothly as possible in the run up to Day 1 of the new millennium, and indeed beyond," said Mills, "but our wider responsibilitiesellipsego beyond that immediate goal.

"We must ensure that critical supply chains are not broken and that the basic infrastructure on which our business depends is not affected. We need to work closely with contractors and small businesses supplying goods to us to ensure that they understand and are addressing the problem."

Shell has published guidelines and held workshops to help its smaller suppliers, which have less resources at their disposal, to solve their Year 2000 problems.

Governments and industry associations are also working to increase the sharing of information among companies (see Part 1 of this series, OGJ, Sept. 28, 1998, p. 29).

Ukooa has an active program of Year 2000 education, as does the U.K. Health and Safety Executive (HSE).

API has formed a Year 2000 Task Force to help its members address Year 2000 problems. The general goal of the group is to share common problems and solutions.

The Federal Energy Regulatory Commission and the Gas Research Institute also offer Year 2000 awareness programs.

In the U.S., President Bill Clinton has promised to introduce legislation that provides companies immunization from lawsuits if they share Year 2000 information. The bill would grant limited immunity to companies who share information on the Year 2000 problem while still allowing liability suits for products such as computers and software programs that fail because of the glitch.

Orrin Hatch (R-Utah), Senate Judiciary Committee Chairman, is hoping to speed the bill through the Senate and on to the House before the congressional session ends this month.

The U.S. Justice Department has already determined that such sharing of information does not violate antitrust laws. The bill has come under fire, however, from the Association of Trial Lawyers of America.

"No one will ever find every embedded microchip, every line of code that needs to be rewritten," President Clinton told an audience at the National Academy of Sciences in July. "But if companies, agencies, and organizations are ready, if they understand the threat and have backup plans, then we will meet this challenge."

Copyright 1998 Oil & Gas Journal. All Rights Reserved.