Key challenges
- Most organisations have moved past basic AI pilots, but more advanced AI applications demand that governance practices keep evolving, meaning they cannot be static or one-size-fits-all.
- There is also a double bind of governance and innovation. AI hype frequently collides with incomplete data foundations, forcing leaders to sneak in governance improvements alongside pressing AI use cases.
- While significant investments in tooling have been made, these often remain under-utilised by business users, who prefer more intuitive, domain-relevant tools. For example, data cataloguing tools were named as major investments intended to bring structure and discoverability to data assets. However, these catalogues were often designed with technical users in mind, leaving business data owners struggling to navigate them. Calls for simplification are tempered by the recognition that no tool fully satisfies both technical and business users.
Persistent Governance Gap
Governance tools frequently cater to the technically proficient, leaving business-side data owners out of touch and unengaged. Despite structured approaches and top-down commitments, the gap between policy and practice persists.
Key challenges
- Convincing data owners that governance is their concern, not just a technical compliance exercise, is an ongoing battle. Some have found success by embedding governance in domain-led use cases, like trading, operations, or asset management, creating a bottom-up push.
- Technical users navigate complex platforms, but business users default to familiar, often legacy, workarounds. Tooling mismatches were a shared pain across participants.
- Organisations are experiencing cultural volatility. AI has reignited interest in data management, but leaders noted that enthusiasm comes in waves and sustaining momentum once the AI hype wanes is a core challenge.
The Realities of Talent and Skills
While data science and AI roles are comparatively well-filled, the scarcity of data engineers is a bottleneck to progress across the energy sector.
Skills for success
- AI and data science are no longer the bottleneck; data engineering is. This gap is felt most acutely as leaders attempt to scale beyond pilots, which is the state most energy sector organisations find themselves in.
- Technical capability alone is insufficient and soft skills matter deeply. Leaders stressed the need for data professionals who can also manage relationships and influence across business functions to achieve long-term success.
- Some see the energy sector’s real-world impact as a differentiator in attracting talent, while others flagged this as a modest advantage when compared to the allure of big-name tech businesses.
Sector-Specific Reflections
Leaders have highlighted that the energy sector’s reliance on unstructured data from physical assets and shipping documents, as well as the vast volumes of asset telemetry data, sets it apart from industries like finance or retail. This makes data governance more crucial yet more complex. While tools may offer seamless governance for structured data, they often falter when confronted with the messy realities of physical infrastructure data. Meanwhile, the sheer volume of high-frequency operational data adds cost and storage challenges that few other sectors face, underscoring the unique data management bottlenecks and concerns in the energy space.
While AI can be a catalyst for new governance practices, its success depends on robust domain-led data foundations and the often-underestimated discipline of data engineering. As the energy sector grapples with unique unstructured data challenges and entrenched cultural hurdles, data leaders must balance the promise of AI with a grounded, iterative approach to governance and collaboration.
Next Steps for Data Leaders in the Energy Sector
- Prioritise domain-led use cases, starting with data governance initiatives that directly support high-impact business areas rather than trying to cover all areas simultaneously.
- Evaluate data governance tooling for user experience and assess if it serves both technical and business users. If not, consider layering user-friendly interfaces or training to boost adoption.
- Use AI projects as leverage to improve governance practices aligning governance improvements with tangible AI use cases to secure buy-in and incremental progress.
- Recognise that data engineering is the bottleneck and invest in internal development programmes, cross-training, or targeted hires to address the gap in data engineering talent.
- Test and validate AI-powered governance claims against your real-world data before rolling out at scale.
- Accept that enthusiasm for governance can be cyclical. Establish continuous engagement strategies to sustain momentum when the AI hype fades.
- Build flexible governance frameworks that allow experimentation while ensuring quality, security, and data ownership are not compromised.
Related Hub Content (Subscribers Only)
Why Businesses Don’t Engage with Data Governance, and How to Change This
Boosting Data Catalogue Adoption and Utilisation in the Public Sector
From Traffic Rules to Gamification: How Vattenfall Eases the Integration of Data Governance
Approaches for Selling Data Governance to Business Stakeholders
