Can we have a discussion about the use of the term Intelligence, particularly as it is being applied in the Oil and Gas sector?
As someone who has worked in the intelligence profession for over 30 years, in both government and the private sector, the word intelligence is often misused. The terms data, analytics and business intelligence are frequently and often incorrectly interchanged confusing clients and the industry as a whole.
At its simplest definition, intelligence or intelligence gathering is the collection of information and analysis to drive decision making. Therefore, intelligence should be actionable, if it doesn’t further planning, visibility or decision making, it is just a collection of data and information.
Specifically, oil and gas (O&G) intelligence should drive your decisions – should we invest capital (and how much) in those assets,why are my competitors achieving better well results, or how much takeaway capacity will be available in those new pipelines and when will they come online. If you do not have confidence and clarity when making these decisions, you did not rely on intelligence.
This problem is only exasperated in the O&G “big data era” we are now in. The emphasis seems to be on obtaining more data and forcing it through an opaque model, often tinged with the terms machine learning, or even worse AI. It is common that very little thought goes into collecting better data or enriching the data quality of existing datasets. Very few providers understand how to establish intelligence collection practices - coupling the right data sources and analysis –often filling the holes in datasets with on the ground human insights.
Before we delve into the data, let’s start where we should – focusing on the problem we are trying to solve. All too often, analysts go straight to the data sources, losing sight of the question they are actually trying to answer. One example I often hear from my O&G clients; “we want to obtain satellite imagery.” When asked why, answers run the gamut from “it’s different” to “it’s really cool” – regardless of the fact that it often does not provide useful information,or there are far superior and less expensive means to obtain the same or better results. As intelligence practitioners,we must keep the problem front and center, determine the best strategies to solve it, and only then find the right sources of data best suited to aid in that analysis.
Which brings us to data. First and foremost, more is not always better. Numerous studies have been conducted examining the relationship between the amount of information available to the analyst, the accuracy of their judgements using this information and their overall confidence in their accuracy. One such example related to picking winning racehorses is highlighted in the figure below:
The experiment demonstrated that as the actual number of items of information increased, a saturation point was reached, and additional items only decreased the analysts’ accuracy. More importantly, the analysts’ confidence continued to steadily increase, demonstrating an overconfidence in their judgments with additional items of information.
This illustrates that more is not necessarily better and that we should instead focus on selecting the right data sources and verifying the accuracy of that data. In order to achieve this,we must certify the providers of our data sets, including human intelligence;how is it sourced, cleansed or reported. As clients have learned, “free” is not synonymous with accurate, nor are the largest and most expensive data sets any guarantee they will meet their needs or strategies.
Finally, intelligence collection should not always be limited to only numerical data sets. To drive decision making, “on the ground” human or technical intelligence gathering is often needed to fill in the limitations of data – particularly if you are only relying on publicly available data sets. Conducted properly, the techniques and skill sets to select and optimize these sources is far beyond “I know a landman”. While it is beyond the depth of this post, proper intelligence practices must consider such things as circular data, false confirmations, and our cognitive biases that can create subconscious errors in our data selection and processing.
Now, more than ever, The O&G industry needs better decision making. If we start making the distinction between data, analytics and intelligence – we will be well on our way.
