Learning to be lean
Recent years have seen a rush by organisations to invest in data and analytics resources, to hire data scientists and to reap the promised rewards of the “Fourth Industrial Revolution”. Often, the assumption has been that the presence of highly-skilled data engineers, statisticians, modellers and the like will, of its own self, lead to positive results.
Less well understood has been the need to impose working practices that ensure projects are prioritised, progressed and completed at pace. One major retail group noted that a previous CDO had suffered from failing to deliver an integrated data warehouse on time, but that even its current incumbent has not yet seen this project reach fruition, leading to some discontent among business stakeholders. This underlines the issue with technology projects using waterfall development when they are applied in a business that needs to move fast.
In addition, the same company’s teams of analysts are all still involved with “project one” because no definition of “done” had been agreed with stakeholders during the briefing phase, meaning that nothing can reach final sign-off as changes to requirements and mission creep continually shift the parameters of the projects.
Both of these experiences point to the importance of having a clear methodology, agreed procedures and keeping discipline across both the data and analytics function and its stakeholders in the business.
Making priorities visual
A common experience for data and analytics functions is the expansion of demand from the business from year two onwards. As the function achieves early wins and tells its success stories around the organisation, more stakeholders come forward looking to have their own projects developed.
At this point, prioritisation becomes a significant issue, as noted in this DataIQ Leaders briefing. A useful tool, adopted from lean management, is the Kanban board – a visual method of organising demand and capacity. Projects are listed on the board with their progress clear to all teams. When a practitioner or team becomes available, they pull a task from the board.
An important dimension of this process is to assign points to projects at the point where they are accepted by the function. These points should reflect the estimated time and resource required so an individual or team can choose a project that matches their capacity. Certain projects will not attract a practitioner or team easily – these are put onto the deadlock list and assigned from the top down so that every practitioner will work on both favoured projects and those which are less appealling.
The points value of projects undertaken by each practitioner is monitored to ensure balance, as well as to avoid over-commitment to long-term projects by one team or repeated exposure to the same type of project by individuals.
An important correlative is the need to hold out resources for unplanned, business-critical requirements that are certain to arise and which do not fit easily into the agile planning cycle. This “emergency squad” picks up new tickets to action and turnaround. Rotation of members of this squad is important to avoid burn out (or addiction to the faster pace of these projects).
6-4-2 planning cycle
The early stages of adopting agile analytics can prove particularly challenging for two reasons. Firstly, analysts have not necessarily been taught how to do what they do as a formal working method. They are naturally drawn to the work and have the skills for it – it is important not to dampen that enthusiasm, but to apply a structured process. Secondly, estimating the time and resources required for projects can initially be difficult and time-consuming. While this skill will develop, it must be recognised as a management overhead. It is also important to bring the understanding of the function as a whole into this process, rather than leaving it entirely to product managers.
Product managers are responsible for creating a rolling six-week roadmap of development. Every four weeks, this is reviewed – slow moving or unadopted projects are dropping into the backlog as necessary. Every two weeks, sprints are initiated to start new projects or new phases of longer-term projects. Daily scrums and stand-ups surface specific tasks and their progress, with the scrum master responsible for balancing the workloads of individuals and teams. Over time, this 6-4-2 approach will come to feel natural.
Managing people, not tasks
For the managers of data and analytics functions where agile is being adopted, one of the most difficult issues is the new focus it creates. Whereas historically managers have concentrated on workflow, loading and progress, agile removes much of this.
With practitioners and teams self-selecting their tasks under the watchful eye of the scrum master, analytics managers instead concentrate on stakeholder and product manager relationships, alongside career and skills development of individuals in the function. This shift towards people management may not suit a manager who has a preference for remaining hands-on with projects, but is a significant opportunity for those with a leaning towards it, not least because this is a highly-transferrable skills set.
Some practitioners describe this management style as “servant leadership” since the role is about encouraging collaborative working, rather than setting out a solution and deciding who will work on it, as well as identifying and removing obstacles to projects. Coaching and mentoring play a much bigger part in this approach than for conventional managers.
Minimum viable bureaucracy
Self-selecting of tasks, two-week sprints and people management are new behaviours for the data and analytics function. As such, they do not suit a command-and-control approach. Instead, the aim should be for light-touch management and oversight where controls are understood, but not at the forefront.
One contested area of agile analytics is the need for documentation. While agile typically prioritises the creation of a minimum viable product over detailed documentation, in the analytics space, there is a much stronger need to capture and share how projects have been developed. The aim here is replicable analytics which allow other teams to absorb learnings, then to repeat or adapt techniques as necessary.
