Increased complexity presents big data challenges for industry

July 1, 2012
According to Goldman Sachs, more than $1.2 trillion will be spent in the next five years on oil and gas projects around the world.

Charles Karren, Oracle, Redwood City, Calif.

According to Goldman Sachs, more than $1.2 trillion will be spent in the next five years on oil and gas projects around the world. These projects represent the "future growth of the energy sector and will be key determinant of oil and gas supply to the global economy."

With the increasing number of unconventional plays, such as shale gas and tight oil, the oil sands in Canada, and the ultra deepwater in the Gulf of Mexico, West Africa, and Brazil, the upstream sector has become an even more complex place to manage not only operationally—but financially as well.

Since the start of the latest oil and gas boom, drilling and projection engineers have been tackling "Big Data" challenges as they explore, drill, and produce hydrocarbons. These new capital-intensive projects have significantly increased operational expenses due to labor and equipment demands and long project lead times, which increase company risk and exposure.

Oil and gas financial executives also increasingly confront "Big Data" challenges when it comes to managing an enterprise portfolio plan and ROCE (Return on Capital Employed), as well as balancing risk and reward and minimizing earnings variation. In addition, these oil and gas financial executives need to make this all happen in a compressed timeframe with increased regulatory scrutiny by numerous government agencies.

Several leading oil and gas financial officers across the world have been deploying a new generation of tools purpose-built for big-data financial environments to help manage this complexity and capitalize on new opportunities, while maximizing long-term financial returns for their companies.

What is Big Data?

"Big Data" can mean many different things to different people. However, most analysts agree that Big Data contains four key elements: volume, velocity, variety, and value. Here's how they play out for financial departments in the oil and gas industry:

  • Volume – Recently, Jay Pryor from Chevron was quoted as saying: "Chevron manages as much data as Google." This includes financial spend and operational data.
  • Velocity – With some companies drilling thousands of wells in a compressed period, more financial and spend data is being compiled by financial departments than ever before.
  • Variety – Oil and gas companies are grappling with unfamiliar and new data types, and a lack of integration between semi-structured and unstructured data, such as drilling approvals, contracts, and regulatory approvals. Data standards are also lacking in some areas.
  • Value – Data is only effective if you can get the right information to the right people at the right time.

The challenge for financial executives

Financial officers in upstream operations need the ability to make proactive decisions. They need to do this by reducing lag time between data capture and data analysis and between data analysis and the implementation of the resulting decisions.

We all know production in a well naturally declines over time. In conjunction with operation and production personnel, E&P financial executives need a way to rapidly assess several important factors including, but not limited to:

  • Price of oil and/or natural gas
  • Revenue calculations from production volume
  • Expected production improvement
  • Duration of production increase
  • Cap-Ex
  • Total allocation forecast for all OCTG items in dollars and metric tons
  • Materials management
  • Effectiveness of the oil field services vendor who does the work
  • Cost, including lost production while well is off-line
  • Opportunity cost of enhancing a particular well vs. other wells in the field

These factors (as well as many others) are interrelated and can have substantial operational and financial implications. The E&P financial executives must be able to analyze multiple scenarios and calculate the expected ROI for each scenario to optimize decision-making. With the new plays, this data has become Big Data as it has become more volumetric and more varied than ever before.

This is why E&P financial executives are paying attention to Big Data and are now implementing solutions to help them manage Big Data and become even more proactive.

Meeting the challenge of financial Big Data

Independents working the Shale Plays

A leading super-independent pursuing a significant increase in shale activity was having Big Data issues as it worked to manage information from 43,000 wells. The issues impacted its ability to effectively deploy and manage personnel in the field. It needed a more timely and accurate way of managing and updating personnel assignments for oil and gas wells, leases, and territories to improve analysis and decision-making.

To solve these issues, this super-independent deployed a Master Data Management solution that allowed it to build teams independently of geopolitical boundaries. Specifically, the solution:

  • Delivered metrics and the ability to mine data for needed answers;
  • Improved the ability to look back at history to assess changes in production, and use that information to explain current performance and plan for the future;
  • Implemented version-controlled data to help explain questions like, "Did that swing in production happen because an asset manager is responsible for more or fewer properties than he was last month?" or "Did that swing in production occur because his wells are not producing at the same level as before?"

Super-major meets new regulatory environment

The second-largest oil company in the United States faced financial reporting challenges stemming from the need to manage version control of financial reports coming from 12 different applications and regularly processing 2,000 different reports.

The super-major improved its ability to meet Sarbanes-Oxley requirements by simplifying data collection, increasing the flexibility and timeliness of its reporting process, providing visibility into its daily financial activity, gaining control of its corporate-wide financial systems and, as a result, dramatically streamlining its closing efforts.

The key to the company's success in managing its financial Big Data is a single consolidation application within a global control framework. This has allowed the oil company to be a leader in financial reporting.

Probabilistic forecasting solution helps manage oil field costs and risks

Another super-major is pursuing several deepwater plays and drilling to depths never thought possible thanks to technologies that can exploit hard-to-reach reservoirs and increase recovery rates at existing fields. However, as oil becomes more challenging to reach, lead times become longer and the cost of exploration and production increases.

Offshore platforms, for example, can cost US$1 billion or more to bring online, and take from 5 to 10 years to go from discovery to production. Given the enormous capital at stake, it's not surprising that this super-major invested heavily in systems and tools that help the company get a handle on oil field costs and timelines, and better understand and manage the risks involved. One of the company's most important tools employs state-of-the-art probabilistic techniques—also known as Monte Carlo analysis—that predict costs and time with remarkable effectiveness.

As a result, this Super-Major gained a clear understanding of oil field costs and schedule risks. In addition it gained the ability to:

  • Enable productive conversations among engineers about likely project outcomes;
  • Focus on risk areas that can generate the greatest cost and time overruns;
  • Protect the reputation of the company as an organization that delivers projects on time;
  • Support accurate financial planning at the corporate level.


Oil and gas companies are looking for applications that unite goal-setting, modeling, planning, monitoring, analysis, and reporting into a single, integrated system. Understanding this information supports closed-loop financial processes by delivering an accurate representation of past results and enabling swift, confident planning and execution for the future. It is this differential performance of successfully deploying Big Data technology that allows companies to raise performance to such a level that their peers, potential partners, and even financial markets will take note. OGFJ

About the author

Charles Karren is the director of energy industry strategy for Oracle. He is part of Oracle's Global Industry business unit and is responsible for strategy, planning, marketing, and industry relations for the oil and gas segment. A graduate of the University of California-Berkeley, Karren also leads Oracle's oil and gas industry relationships, including collaborating with PPDM, Energistics, NPRA, PESA, the SPE, and also serves as Oracle's liaison to the National Petroleum Council in Washington, DC.
More Oil & Gas Financial Journal Current Issue Articles
More Oil & Gas Financial Journal Archives Issue Articles
View Oil and Gas Articles on