Beyond AI Experimentation: What Data Leaders are Learning About Scaling AI

This article is a summary of all the conversations, presentations, and roundtable discussions held at the recent DataIQ Peer Exchange in Boston, US, where data and AI leaders examined what it takes to move organisations forward while scaling successfully.
Beyond AI Experimentation: What Data Leaders are Learning About Scaling AI

This write-up covers all the discussions at an exclusive in-person DataIQ peer exchange hosted in Boston, US. The Peer Exchange brought together senior data and AI leaders from a diverse range of organisations including finance, utilities, and enterprise sectors. You can find all upcoming DataIQ events here and register your interest in attending. 

Across discussions on AI readiness, intelligent workflows, human oversight, governance, and business ROI, it was repeated that technology will not determine which organisations succeed. 

AI is forcing leaders to confront questions that have existed for years around data quality, decision ownership, operating models, organisational culture, and accountability. But it is also raising the stakes. When machines become part of operational workflows and decision-making, weaknesses that people previously compensated for become harder to ignore.  

The DataIQ Peer Exchange conversations in Boston highlighted priorities for leaders seeking to progress from AI activity to sustainable business value. 

 

Measure outcomes over activity 

Data and AI leaders must understand and communicate that organisations can have extensive AI activity without generating meaningful returns. 

Leaders repeatedly challenged measures such as:  

  • Numbers of pilots 
  • Licences 
  • Use cases 
  • Technology adoption 

Instead, leaders argued for starting with: 

  • The business outcome 
  • Determining the metric that should change 
  • Establishing clear accountability 

This was evident in Scott Richardson, Chief Data and Analytics Officer at First Citizens Bank’s discussion of AI readiness and ROI. He argued for working backwards from customer and business value, concentrating resources on opportunities that can materially affect performance rather than allowing individually attractive ideas to proliferate. 

Demonstrating progress means showing what has changed in the business over what the data or AI function has delivered. 

 

Redesign work before introducing AI 

Workflow redesign emerged as a prerequisite for successful implementation. One discussion explored the importance of understanding how work happens before introducing agents or automation, including clarifying decision rights and escalation paths. 

Aravind “Jag” Jagannathan, Single Family Vice President and Chief Data Officer, Freddie Mac, shared an approach based on decomposing processes into triggers, decisions, actions, and exceptions. This provided a way of identifying where intelligence could improve a workflow instead of starting with the technology and searching for somewhere to deploy it. 

Inefficient processes do not automatically improve when AI is introduced. Genuine transformation requires leaders to reconsider how the work should operate when technology and people can perform different parts of it. 

 

Accountability must evolve with autonomy 

As AI becomes more operational, an increasingly important question is not simply what a system can do, but who is accountable for what it does. 

The Boston discussions highlighted unresolved questions around ownership between business, data, AI, and technology functions. Leaders argued for distinguishing who develops capabilities, who manages risk, who operates deployed solutions, and, critically, who owns the resulting business outcomes. 

At Freddie Mac, the discussion shifted from conventional data ownership towards decision ownership, bringing the people responsible for business decisions into workflow redesign and connecting accountability more directly with outcomes. 

The question becomes more important as organisations give AI systems greater autonomy. Leaders discussed the need for defined decision boundaries, escalation routes, and clear human accountability before agents are entrusted with consequential decisions. 

 

Trust is an organisational challenge 

During roundtable discussions on trust and human oversight, leaders argued that oversight needs to reflect the risk and context of individual use cases rather than following a universal model. In some environments significant automation may be appropriate, while elsewhere continuing human involvement remains essential. 

Trust depends on participation. Employees engage with AI when they are involved in development, testing, feedback, validation, and implementation rather than being presented with a finished system. 

Leaders discussed concerns among experienced employees about what AI could mean for the specialist knowledge on which careers have been built. Ignoring questions of future relevance and job security risks creating resistance regardless of the technical quality of the solution. 

 

AI readiness extends beyond technology 

AI is exposing enterprise weaknesses as data needs to be accurate, consistent, and organised for machine consumption. Scott Richardson highlighted how information that people can interpret, correct, or work around may not be suitable when a machine depends on it, potentially requiring organisations to revisit data foundations they considered complete. 

Procurement, legal processes, third-party risk, model governance, architecture reviews, sponsorship, and product management can affect how an AI initiative reaches production and generates value. Improving the speed of technical development merely exposes the next organisational constraint if the surrounding processes have not evolved with it. 

Freddie Mac’s experience similarly highlighted the importance of modernising legacy data foundations, pushing data quality responsibility towards its source, and building broader business understanding across data teams. 

AI readiness is a test of enterprise readiness. 

 

From experimentation to execution 

The Boston discussions suggest that the next phase of enterprise AI will require a change in leadership emphasis. The questions are about whether organisations can decide where it belongs, redesign work around it, establish accountability, earn employee trust, strengthen the underlying data, and demonstrate measurable results. 

Data and AI leaders are closer to the commercial and operational core of organisations than ever before. Technical capability remains fundamental, but business understanding, product thinking, organisational influence, governance, and the ability to translate investment into outcomes are just as important. 

This was examined further in the recent An Unconventional Truth report which can be downloaded for free. 

 

Go deeper with DataIQ 

DataIQ members can access the full learnings from the Boston Peer Exchange on the exclusive DataIQ Hub. The conversation continues at DataIQ Congress on 3 November 2026, under the theme The End of AI Theatrics: Accountability, Execution, and Value in 2026. Congress will bring together senior data and AI leaders focused on operationalising AI and demonstrating business value. 

The End of AI Theatrics report that builds the core of the DataIQ Congress can be downloaded for free. Read it now and be fully informed.  

The next DataIQ Peer Exchange is Data and AI Literacy: Approaches to deliver programmes that last, taking place on 28 January 2027 at Great Scotland Yard Hotel, London. 

We just need 30 seconds of your time

Interested in:

By clicking “Continue” button, you are agreeing to the DataIQ terms of use and privacy statements.