Open Grid Europe (OGE), one of Europe’s leading transmission system operators with approximately 12,000-km of pipelines, is using new technologies such as artificial intelligence (AI) based on Microsoft Azure to transition the company away from natural gas to hydrogen. Germany's goal is to be No. 1 in hydrogen over the near term and OGE will play a key role moving Germany toward its carbon neutral goal.

In a Microsoft customer report, OGE says it began developing its PipeMon+ (pipeline monitoring system) in 2005.  While its main purpose is to monitor existing pipelines during construction work, it can also be used as a safeguard in areas with a high population density or that are in close proximity to hospitals, schools, or kindergartens.

The company notes: "the monitoring PipeMon+ provides is based on cathodic protection, whereby a corrosion protection system feeds a protective current into soil. This current searches for any microscopic defects in the pipeline coating, by which it flows back through to the protection system. Sensors set at intervals along the pipeline gather data on the current and other electrical parameters ten times per second. Now, if an excavator damages a pipeline, the current flow and the measurement values will change, triggering an alarm."

“In the past, PipeMon+ ran on a distributed cluster in our own local data center, which meant we had to check all alarms manually,” says Martin Trzeja, Project Manager for Analytics & Reporting at OGE. “Every time we added a new sensor to our network, we were afraid the entire system would collapse.” OGE’s corrosion protection department was forced to proceed with caution—they could only see data from the past three days and the on-premises solution’s lack of storage capacity meant that each additional sensor shortened this timeframe.

In 2019, OGE launched a transformation initiative. All IT systems would, where technically feasible, be migrated to Microsoft Azure. PipeMon+ became one of the pioneering cloud projects. “Simply put, we needed a stable solution with reliable scalability that would allow us to integrate as many new sensors into our system as we wanted. In other words, a solution that would give us technical flexibility,” Trzeja says.
                                                                                                                                                                                                                                                                                                  Thus OGE developed an artificial intelligence solution which filtered out false alarms and shortened response times.  According to Trzeja "we had only six months until the license for our previous solution expired in June 2020. The migration had to be completed by that point, and many were skeptical about whether we would make it.” Two weeks before Christmas 2019, the project team met for the first time with its infrastructure partner to establish an overview of the services that would be required and to decide upon the system architecture. The great advantage was that the cloud services were available right away, which meant that the Azure environment was set up and signed off on by Christmas. Nothing stood in the way of kicking off the project and starting the migration right after the New Year.

Following the first tests of the AI models developed using Microsoft Azure Databricks and MLflow, the new solution was used to monitor the first section of pipeline in April. “The major challenge was that we had to keep the old system fully operational during the migration. The system was being constantly inundated with new data and we couldn’t afford to lose any of it,” Trzeja says. “At the same time, we were using Microsoft Azure Data Box to migrate around 50 terabytes of data collected over the past three years—a terrific achievement by the entire project team.” By the time the old system was deactivated in June 2020, the migration was complete, and all core functions were integrated. The team managed by Trzeja, together with Patrick Krzyzak, OGE’s project manager for PipeMon+, had met its deadline for developing an entirely new architecture based solely on native Azure services and readying it for rollout. Training the AI algorithms continued into October—now they reliably filter out false alarms and ensure that genuine alarms are detected quickly.