Improving your data management capability from levels 1/2 to levels 3/4
Background
All data has an origin. From direct inputs by customers in web forms to transactions in all channels, from digital tracking to third-party validation, the range and sources of data continues to expand. Each new operating system or external route to market adopted by an organisation adds to the range and complexity of these data sources.
For data management, this creates a perpetual challenge in trying to map these sources, apply standards and governance protocols, create links and potentially integrate disparate data into a unified asset. Delivering this to core business functions, such as sales and marketing, and value-adding activities, such as analytics and data science, is the justification for the time and resource invested into data management.
But organisations face an escalating problem in moving data from cost-centre to value-adding resource. Across ten CARBON™ assessments completed by DataIQ Leaders, data management emerged as the one area on which scores emerged as below or just above average – see Table 1. As these results show, strong leadership (scoring as high as 4.2 out of 5 in some organisations) and robust skills (scoring as high as 3.64) are being undermined by performance on dimensions that should be core to the organisation and which are outside of the direct control of the data management function – strategy (where scores are as low as 1.58 and only peak at 3.02) and technology (as low as 2.02).
If an organisation does not have the right strategy and approach to technology in place, then issues with data will only multiply, from creating data sets in unregulated “shadow” IT environments to allowing data capture to take place without basic rules and standards in place. All of these problems will then be loaded onto the data management function to resolve, leading to a significant drag on its capability and output.
One dimension of this impact was documented in research carried out by IDC on behalf of Alteryx in early 2018 – see Figure 1. It found that data professionals are wasting 14 hours per week on preparing (5.5 hours), governing (4.6 hours) and searching (3.9 hours) because the data they need is hard to find, has not been standardised or is not integrated. In addition, they spend a further 10 hours building data assets that already exist elsewhere (but were not discoverable or disclosed). All of this has a direct financial implication – IDC estimates that it costs European organisations €1.1 million each year for every 100 employees.
Figure 1 – Time spent getting from data to insight (source: IDC/Alteryx)
Understanding the scope and scale of the problem
There is no doubt that with the growing recognition of data as a corporate asset, combined with the leadership of the CDO, the problem of data silos is surfacing to a greater extent. Obligations around privacy and data governance, especially the General Data Protection Regulation (GDPR), have also driven up awareness of the issues. Yet at the same time, the digitalisation of business is seeing the increasing introduction of new systems which originate data or where data needs to be processed in order to deliver their servi
Consequently, even those companes with the best of intentions around managing their data struggle to build a single consistent, coherent and governed view of their customers. Where there is no overarching strategy to become customer-centric, the problem of data silos multiplies further – the UK’s largest insurance provider, operating across 27 separate brands, holds records by policy, not customer. A mobile telephone network operator holds 16 separate copies of its call records, none of which match.
Table 2 summarises the experience of DataIQ Leaders members as revealed during a recent Breakfast Briefing discussion. It shows the diversity of data management approaches, but with common features such as functions or brands creating their own silos within both data and technology. As organisations grow, so do the challenges – in one case, the fundraising platform saw its daily event logs rise from 20,000 to 2 billion per day, for example.
It is tempting to assume that start-ups do not face the problem of data silos as they do not have legacy IT systems. However, unless a clear strategy is estblished and enforced from the outset, even new businesses can replicate the faults of the past by adopting a “do your own thing” approach, leading to dozens or even hundreds of instances of data arising.
Drivers and blockers of data integration
Driver 1 – Regulation:
Regulation is one of the most powerful drivers for investment into data integration and the elimination of data silos. GDPR is the most recent example since the mandated requirement to provide access to data subjects to their records practically writes the business case for a single view. But it is only one piece of regulation – vertical industry sectors will have their own specific requirements. The Financial Services Compensation Scheme is one example where SCV is at the heart of understanding what level of investor deposits is at risk within and across deposit-taking institutions.
Blocker 1- Risk Management:
This is not to say that regulated companies always and automatically recognise this requirement and invest accordingly. While Know Your Customer and Anti-Money Laundering laws are easier to comply with in the absence of data silos, they do not specify a solution and there are many competing ways to address compliance. Global organisations can also arbitrage regulatory environments around the world and choose to locate their data management or data processing in territories with lower standard. This can go as far as balancing the revenue potential within maintaining localised data sets versus the potential regulatory fine, although GDPR is likely to see a levelling-up that will reduce or remove this option as other countries follow suit. Tactical decisions may lead to the suppression of centralised data, despite a need for common information sets. Silos will always exist because they represent opportunities.
Driver 2 – New technology:
New methods of linking and matching records are constantly emerging, with the latest generation leaving data within existing host systems, thereby reducing the systems engineering costs. To reduce the risk of future silos being created, propagating and enforcing a standardised data model can be an important step. Organisations can plan for legacy by embracing data governance at the outset of each new system – if they don’t, it will be eventually become a problem. As one DataIQ Leader put it: “All of our systems were cutting edge once.” A new technology stack, based around NoSQL and Hadoop, is now a viable option.
Blocker 2 – Unstable technology:
Software vendors try to dis-intermediate the IT department by selling directly to end-users in lines of business, so there is a need for stakeholder management or even lock-down by the data management team to prevent these “shadow” IT projects. It is impossible to forecast what the technology landscape will look like even at just three years out, despite the need to set out a roadmap towards that horizon. Adopting x-as-a-service is treated as operational expenditure, which has a different decision point from capital expenditure infrastructure investments.
Driver 3 – Centralise governance:
Identify all data sources and build links which can either support an ultimate migration or integration without initially having to move data sets around. Centralise governance and connectivity so that data required by specific functions can be pulled together, while allowing systems to revert to their core use, thereby delivering a performance gain. Infrastructure is left looking the same, but it has actually been enhanced.
Blocker 3 – Localised culture:
Companies want to move forward, but don’t want to let go of the past. Culturally, there can be a view that data represents a seat of power which lines of business do not want to see centralised away from them. Creating a data and analytics centre of excellence also provides a place where business processes can be seen end-to-end, which is also very exposing for those functions that have bottlenecks or performance issues. Centralised governance can also conflict with the desire of data scientists to access all available data, but controls do need to be placed on their usage to mitigate risk.
Tips to improve your data management CARBON™ score
- Establish a data management strategy: At Level 1 (Aware) and Level 2 (Repeatable), data management strategy is either not formally set or only takes limited input from stakeholders. Improving to Level 3 (Defined) sees the strategy ensuring all stakeholders are consulted. At Level 4 (Managed), the strategy is reviewed and updated. Alongside this, measuring how the organisations is performing against this strategy is central to improving maturity.
- Develop management reporting: At Level 1 (Aware) and Level 2 (Repeatable), there is either no reporting to senior management from the data management leader or too many layers between them. At Level 3 (Defined), the leader still reports one level below the most effective tier, but in Level 4 (Managed), there is regular reporting to the most senior layer of management.
- Map skills for now and the future: While skills mapping is absent at Level 1 (Aware), by Level 2 (Repeatable) current needs are defined within the data management function. Progressing to Level 3 (Defined) sees clear role definitions being put in place with a short-term view into the future – evolving to a rolling horizon with regular reviews builds a Level 4 (Managed) capability.
- Align the data strategy and technology strategy: At Level 1 (Aware), strategies are not in place and there is no separate technology provision for data management. At Level 2 (Repeatable), data management is recognised by IT as having specific requirements, but remains under its control. Giving data management its own technology stack takes the organisation to Level 3 (Defined), while ensuring it continues to have all of its technology needs met in line with the data strategy takes it to Level 4 (Managed).
- Benchmark data and technology strategies: It is a characteristic of Level 1 (Aware) organisations that they make no use of benchmarking, although informal tracking is used by those at Level 2 (Repeatable). Reaching Level 3 (Defined) sees internal benchmarks being introduced while at Level 4 (Managed), certification by vendors provides a method for tracking how well technology is being introduced and used.
