WEB SERVICES TO SOA-2: BP describes maturity model for IT in its E&P organization

This is not to say that Level One IT projects fail to deliver any real value. They do. However, benefits tend to be localized, and little cross-project value or reuse of technology occurs (Fig. 6)

Success depends largely on having a seasoned technical architect and a capable software development team, rather than any structured organizational competence. While an overall portfolio management process may be available to optimize sanctioned applications, there is scant guidance around implementation.

It is, of course, at Level One that an IT organization realizes the need for SOA and web services and begins to move in that direction. However, no formal SOA work occurs at this point. Most of the heavy lifting will take place in Levels Two and Three.

Level 2: Managed services

At Level Two, where many IT organizations are today, basic web service standards are being established for the first time.

By the end of this phase, web services will be the preferred method for all projects in the organization to connect all applications and-or services. In effect, projects will have to request a formal exception if they wish to do it any other way.

During Level Two, therefore, considerable time is spent building or otherwise acquiring common business functions intended for reuse. These include not only specific business and technical functions but generic services required by all applications, such as security.

It would be highly inefficient for every project to continue writing code for user authorization and access, as they do today. It’s best to do it once and do it rigorously for the whole organization.

Security services may be among the most challenging technical hurdles at this level, but the value in getting them right is enormous. With potentially thousands of employees, contractors, and partners to keep track of day-by-day, no company can treat SOA security as an afterthought.

Legacy applications can also be elevated from Level One to Level Two by wrapping their core functions with standard web service interfaces. Ultimately, as the library of discrete services grows, projects will favor reuse of existing components before building new ones, off-the-shelf services will take precedence over custom-built services, and application vendors who provide native services will be preferred over those who do not. To find reusable components, project teams will refer to a central catalog where all services must be registered. Initially, this may be little more than a web page on the corporate intranet identifying what the service is, what it does, and who maintains it.

Divergent data models still exist at this level, but services that show the greatest potential of being reused across the organization should be aligned with key data models by the IT team.

It is recognized that not all services created at Level Two will necessarily be adopted by other projects or business units. However, the service development process is disciplined enough and sufficient web service standards are available to enable adequate interoperability, both internally and with external partners.

For some period of time-perhaps several years-an organization could remain at Level Two as past IT initiatives continue untouched, new services are under construction, and new projects slowly begin adopting them (Fig. 7)

To efficiently manage Level Two activities and encourage a culture of reuse, basic IT metrics on the cost of building services and amount of reuse among projects must be tracked and reported centrally. In addition, a comprehensive set of documents that capture SOA policies, standards, and governance processes must be created. General documents intended for internal SOA marketing and training should include:

1. SOA overview presentation-provides a reasonably technical introduction to web services and the SOA vision for IT professionals.
2. Maturity Level Two training pack-includes overview presentations and related “deep-dive” documentation on Level Two.

In terms of process-related documents, the SOA Maturity Model itself (whether the one presented in this article or another version) is required to facilitate the transition to Level Two. It should include a definition of SOA maturity levels, as well as the criteria used to assess the maturity of any given IT initiative. Other SOA process documents we’ve identified include the following:

1. Service life-cycle management-describes the entire process of creating services, from conception to delivery, including requirements planning, design, development, testing, deployment, and documentation.
2. Operational support plan-describes the handover-to-support process as well as the standard process of supporting individual services.
3. SOA governance framework and policies-defines the management structures required to ensure that SOA architectural decisions will actually get implemented.

Important technical SOA documents, which contain more details than the above, should include:

1. SOA architectural principles-outlines a high-level set of principles and “beliefs” within which the entire SOA concept and architecture operates.
2. SOA architectural decisions-lists the key architectural decisions that have been made and the reasoning behind each one, for future reference.
3. SOA standards and guidelines-identifies detailed technical standards for the development and deployment of services and web services in the organization.
4. SOA development tools-contains guidance and recommendations on best-practice usage of commercial service development technology; in general, the development environment should be chosen based on the most appropriate solution for a company’s specific issues; the only mandated “tool” required at Level Two is a common service catalogue that complies with standards.
5. Interim security model-describes the interim security model to be used until an enterprise security framework is established.
6. Interim scalability and SLA model-describes the interim scalability model and how service level agreements (SLAs) will be managed and met.

As the foundations of the SOA are put in place at Level Two, a number of benefits can be expected. The use of basic integration standards and a cohesive architectural framework will accelerate the evaluation, design, and execution of internal and external solutions. Availability of a small number of common services will begin to reduce both the costs and time involved in deploying applications, significantly in at least a few cases. Establishing an organizational approach to SOA will help educate key IT and business stakeholders, as well as primary IT vendors, on the company’s plans and processes for pursuing the SOA vision. The organization is now starting to think strategically rather than around the tactical, disjointed projects of Level One.

Shared services developed at maturity Level Two will form the essential basis for Level Three process orchestration and all subsequent SOA-related benefits.

Level 3: Process orchestration

At Level Three, the primary activity is the formal “orchestration” of broad business processes based on a pool of fine-grained services rather than the writing of code as in Levels One and Two.

This means the organization must have begun modeling current processes that span multiple areas of the business, decomposing those processes into discrete activities, mapping those to available services, and managing services with a common drag-and-drop orchestration tool.

At this level, a broad range of standards and definitions exists for implementing SOA and consuming reusable services across projects. A more formal services registry has been implemented, providing a single source of information on all available services.

Although new services continue to be introduced and some are not intended for broader use, the focus is on increasing the amount of sharing and reuse among as many different programs as possible.

To ensure integration in critical business areas, a small number of common data models exist-either industry standard models, or at least major data models used by other projects. However, no standard for all possible data types is available at this point.

Additional SOA metrics captured at this level include the spread of web services as the preferred integration technology, the number of key business processes being modeled and executed in the orchestration environment, and estimated cost savings generated by service reuse.

SOA solutions at this level enable much greater business agility. In different regions or business units, even when a relatively similar overall process is called for, unique operating conditions may require flexibility in IT implementation. Even if the same component services are involved, they may need to be orchestrated differently.

Once it has reached Level Three, an organization can rapidly “rewire” existing services used in a particular business process, without necessarily buying or building any new technology. Services can be upgraded or replaced entirely without any noticeable impact on the process, because the user’s interface remains unaltered (Fig. 8).

As we noted earlier, Level Three is still a good state for most large companies, including BP. We have a few advanced IT projects dabbling with Level Three capabilities, but at present there are still too few services available to orchestrate a complete business process. By the end of this year, we expected to have sufficient services completed to begin orchestration of a few critical processes in E&P.

The SOA Maturity Model currently includes all documents we have completed for Level Two. However, documents for Levels Three through Five are still in development. We have identified seven standards that appear critical to successful implementation of Level Three:

1. Process modeling tool standard-defines the standard for exchanging models and identifying compliant tools that enable process modeling and creation of executable business process definitions.
2. Orchestration tool standard-identifies a commercial product and defines usage guidelines for executing business process models at runtime.
3. Data model standard-defines the standard for exchanging models and identifying compliant tools for capturing and extending existing data models, or proposing new models.
4. Intranet portal standards-identifies a specific product and defines usage guidelines for the implementation of an intranet portal.
5. Web services gateway standard-identifies a product and defines usage guidelines for a service gateway that mediates services between providers and consumers.
6. Services registry standard-identifies a product and defines usage guidelines for a central repository of service information.
7. Publish/subscribe standard-identifies a product and defines usage guidelines for an environment that allows publish/subscribe connectivity.

In addition to the above standards, Level Three requires two, more detailed technical documents, to update Level Two documentation with new information:

1. Extended SOA standards and guidelines-incrementally extends the allowable definition of a service where new SOA industry standards become available for implementation.
2. Enhanced security model-updates the maturity model to incorporate the enterprise SOA security model, framework, and patterns.

Finally, an SOA test bed or “sand box” must be provided-a preconfigured environment (target platforms) equipped with the standard tools listed above, enabling projects to test new composite business applications and services.

Maturity Level Three further boosts business agility through more extensive reuse and better integration of services across the organizational project portfolio. Greater reuse of common components and interoperability standards enables the company to deploy or rewire business solutions more quickly, at lower costs.

Business processes are no longer “hidden” inside big, monolithic applications, thereby supporting greater awareness and ownership of processes by the operating units. With greater knowledge, they can begin to analyze, automate, and optimize processes more intelligently and responsively than the long, costly IT initiatives of the past.

Level 4: Quantitatively managed

In moving to Levels Four and Five, it’s important to reiterate that significant areas of maturity remain unidentified.

Nevertheless, we have sought to avoid the kind of arm-waving statements that are common at higher levels of transformational change programs, such as “everything just gets better, faster, and cheaper.”

While many details may be uncertain at present and subject to revision, the purpose and value of each higher level is not fuzzy at all.

For the most part, by the time an organization reaches Level Four few basic enterprise-level services will be missing and orchestration will be well understood (Fig. 9).

Also, the message semantics (based on XML industry standards) for service/data interchange will be fully defined and governed. Services intended only for local use will be rare, and the introduction of new services will be seen as “adding to the pool” or “extending the portfolio” rather than creating independent work.

Industry-wide-or at least consistent vertical segment-wide-data models will be available for most sectors of the business. Because external partners and suppliers have access to the same models, data can be shared more transparently than at any previous level of maturity.

At Level Three, metrics were focused largely on IT processes associated with the spread of reusable services across projects and business processes. The organization was still putting critical pieces in place, and learning how to use them.

The primary shift at Level Four is starting rigorous quantitative measurement of the effectiveness of business processes now being orchestrated within the SOA environment. Quantitative targets, therefore, are set for cost, quality, and performance both of the business processes themselves and of the services supporting them. These goals are determined by the needs of the business users, those who implement the processes, and the organization.

Quality and performance are understood in statistical terms, monitored, and managed throughout their life span-a practice currently known as business activity monitoring (BAM). The idea is to benchmark now well processes are functioning, identify significant causes of process variation, and seek to rectify them, where appropriate, to ensure repeatability and prevent future issues.

At Level Five, all this quantitative feedback on current processes will act as a baseline for greater automation and process optimization.

The only new technology considered essential to success at Level Four is some type of BAM tool. Organizations will only need to create a small number of SOA documents at this stage:

1. Business activity monitoring tool-identifies a selected commercial product and defines usage guidelines for monitoring and measurement of processes.
2. Industry data standards-identifies multiple vertical industry standards that will be adopted, along with gaps in existing data models that need to be filled (known examples for the upstream energy industry include WITSML, PRODML, and PIDEX-all XML-based standards for data exchange in specific areas).

Monitoring the benchmarking of business process efficiency at Level Four will enable rapid identification of opportunities for improvement. Then processes can be quickly reconfigured using new or reusable services, and resulting enhancements can be compared with a known baseline. With pervasive common services and supporting data standards, information from diverse locations throughout the organization can be located, aggregated, reported, shared (internally and externally), and analyzed with ease and efficiency.

Level 5: Optimized

Here we reach the summit of the SOA vision.

By this point, it may seem there is almost nothing left to accomplish. Nevertheless, at this maturity level, considerable added value will be created through continuous process improvements-both incremental and innovative. Primary activities revolve around automated process monitoring and optimization.

This completes the shift from companies being suboptimized through traditional IT approaches to a truly virtualized company where the optimized business processes seamlessly span internal organizations and external partners. Quantitative improvement objectives for the organization will be established and continually revised to reflect evolving business needs.

All resulting changes will be measured and compared with objectives. Broad transformation of existing business models will take place through this ability to monitor, evaluate, and reconfigure processes “on the fly” (Fig. 10).

At Level Five, detailed external security standards and services will enable complex, seamless integration of the global organization’s entire value network. Internal staff (even those using mobile devices) and authorized external partners, vendors, and customers can access shared services over the internet. Commodity business functions can be outsourced based on cost, performance, and delivery without being constrained by past technological barriers. Based on carefully formulated trust models, diverse external resources can be leveraged to accomplish a variety of strategic endeavors.

We have identified only two more SOA documents that will be needed here:

1. Business process efficiency standard-describes a standard (and possibly commercial tools) for automating the optimization of business processes; possibly including “what-if” analysis.
2. External security standards-details the standards required to interact with third parties by way of complex, potentially lost-lasting transactions; and outlines the process of building trust models with external organizations.

Automated optimization of business processes will increase the global organization’s ability to respond quickly and intelligently to change, boosting competitive advantage. Tighter integration with, and outsourcing of commodity processes to, external parties will further lower costs, and free internal time and talent to pursue more important core activities.

Conclusion

Any large corporation or IT group within an organization that commits to the vision and value of SOA could run into three major roadblocks along the way.

The first roadblock, which we’ve called the “chasm”-the general lack of a common, compelling SOA vision and a practical, proactive SOA maturity model-is the subject of this article. For the most part, it’s an “internal” organizational roadblock.

IT managers and staff don’t know exactly how to get where they want to go. Once they have a common vocabulary to talk about SOA and a clear, comprehensive roadmap showing the way forward, they can be off and running.

The second roadblock, by contrast, is one common to all major technology initiatives, namely, “How do we engage the business teams to adopt this vision and what governance do we need to ensure consistent application of the approach and standards?”

Clearly, a lack of understanding and involvement from the project team working in the business will be a significant impediment to the success of an SOA program. Further, an absence of corporate governance will mean a disjointed and inconsistent application of the architecture.

SOA changes the fundamental relationship between the business and IT and, if that relationship is not addressed, much of the true value of SOA will not be realized. Therefore, a comprehensive engagement and education effort, along with a governance process that fits with the organization’s culture, is a key to the success of an enterprise-level SOA program.

The third and final roadblock is largely “external.” For companies beginning to take well-defined, incremental steps toward SOA, the general immaturity of SOA industry standards and dearth of commercial, reusable services could cause SOA early adopters to stall out.

Even if an organization has the will and resources to forge ahead aggressively, it may be forced to wait for SOA tools to mature or evolving standards to crystallize before it can move to the next higher maturity level. On the other hand, it could go ahead and build proprietary services in-house, but some or all of them will have to be torn out when standards become available.

What’s more, if a corporation’s major IT vendors, vertical application developers, and business partners are not embracing SOA as well, everyone could be held back. No one can do this alone.

It behooves every IT organization, therefore, to launch a coordinated SOA program, accelerate its move from Level One to Level Two, and to inform all of its partners and suppliers of its commitment to the vision. In addition, it will benefit all of us to work together, across traditional organization and industry lines, to develop, test, and establish solid SOA standards.

As we noted above, SOA is not a natural evolution of current practices or environments. It is a totally new and revolutionary way of creating business solutions and optimizing business processes. Only with widespread cooperation among forward-thinking global businesses will we begin to realize the full promise and potential of SOA.

Acknowledgments

Thanks to Joe Hardman of IBM for invaluable contributions to the development of the SOA maturity model. Thanks to Jay E. Valusek, consultant, and Jim Bettencourt of SAIC for assistance in preparing these articles for publication.

The authors

Russell D. Foreman is program manager, solutions architecture, for E&P digital and communications technology, in BP’s upstream IT organization. He has more than 25 years of experience in upstream oil and gas working for operators and service companies in technical, operations, managerial and R&D positions. He was instrumental in the development of three upstream industry standards: WITS, WITSML, and PRODML. His current remit is to deliver common IT architecture solutions to BP’s E&P segment. He has a BS in petroleum engineering from the University of Tulsa.

James W. Jones is the director of service-oriented architecture in the Enterprise Architecture Group of BP. He has over 32 years with the company and has had various roles in advanced computing technology application in all parts of the company. These included process control, computer simulation, process instrumentation, refineries and chemical plants, enterprise architecture, and trading and supply. He is now in the central BP enterprise architecture team. He has a BS in chemical engineering and a BA degree from Bucknell University.

Danny J. Ducharme is the lead enterprise architect for E&P Digital and Communications Technology, BP’s upstream information technology organization. He has over 25 years’ experience in information technology, strategic planning, computer integrated manufacturing, operations and engineering management in semiconductor and related high-tech industries. At BP, he established an enterprise architecture team and core capability for E&P to create and maintain blueprints of the processes, systems, solutions, and standards across the segment. He holds an MBA in technology management and bachelor’s degrees in electrical engineering, physics, and mathematics.

Dean Forrester is senior technical architect for SAIC Consulting. He has 13 years of experience with companies including EDS, Halliburton, and SAIC in the design and implementation of IT systems, including major “next-generation oil field” programs. He earned degrees in physics and instrumentation, systems analysis, and design, and an MBA from Napier University.