Taking a more disciplined approach to predictive analytics

Predictive analytics, under the guise of the rather more prosaic label of mathematical and statistical modelling, has been around for years, especially in financial services and mail order retailing. Now, as the value of data becomes much more widely recognised, marketers are increasingly aware that using such techniques is the key to understanding consumers and predicting their behaviour. And if you can predict consumer behaviour, you’re in the right place to optimise your marketing budget allocation.

This ability to optimise spend on activities that maximise returns has contributed to a resurgence in the popularity of predictive analytics. However, for many the use of predictive analytics is still relatively new and getting the process right, from setting the right objectives through to measuring the results, is not straightforward and requires an experienced hand.  

As with any analytical exercise, it is important to start with a clear objective of what you are trying to predict. This is vital as it determines the nature of your predictive analytics exercise. It shapes how the data is to be sourced (both that which you are predicting and that which is being used to predict) and how you are going to measure the results, over what timeframe and using what management information (MI) or reporting. Does new reporting need to be designed to support the development, testing and evaluation of the model to demonstrate its return on investment?

Another key consideration when building a predictive model is sampling. Sampling past data that is representative of the future you are trying to predict is vital to delivering accurate results. Clearly it is no use, for example, trying to build a model designed to have general usage across all age groups and not selecting all those age groups in the sample. Secondly, the sample needs to be large enough to give a decent spread of data. If the sample is not large enough, models can be over-fitted – using too many or irrelevant factors that can lead to unnecessarily complex implementation and prediction errors, meaning spurious relationships that don’t exist in reality can be wrongly identified. For example, it is easy to find a perfect correlation between two random variables if they are only sampled at two points. The consequence of this may be some poor communications decisions that waste resources, miss opportunities and are irrelevant to the recipient.

It is also critical during these early planning stages to choose the right modelling technique. Without the right model in place you may be left with something that, at best, is difficult to implement and, at worst, an approach that doesn’t adequately represent the market or consumers you are attempting to model, generating ineffectual results that fail to reveal useful insight. For example, rule-based predictions may be easy to implement but they’re not as accurate as other approaches. Linear models are more precise, but can be more difficult to implement and they require a sound understanding of their limitations, as a linear model doesn’t take account of less obvious inter-dependencies between variables.  Finally, despite the increased accuracy of more complex non-linear approaches, they may be difficult for non-statisticians to execute and interpret. Marketers need to work closely with their analysts to establish carefully both the objectives and the modelling technique to ensure implementation is possible and accurate and that measurable results are generated at the end.

The ability to predict future behaviour is based on observations of past behaviour. This can only be achieved by setting an appropriate observation period during which behaviour and attributes are gathered, then using these to predict behaviour during some future outcome period.

However, once built, predictive models must be tested on un-seen data, or hold-back samples. If the model has been built correctly, the expectation is that it will predict accurately across the hold-back sample and at an equivalent level to that observed during the model-building phase. If it doesn’t, this is a sign that the sample is not robust enough, or that the model has been incorrectly-specified or over-fitted.

Once the predictive model is built and implemented, detailed monitoring is critical for evaluating the ongoing accuracy of the model as the market changes or consumer response to the stimulus being predicted changes. Effective monitoring will undoubtedly trigger required changes in the model. This may lead to re-alignment of certain predictive variables, where the model is tweaked to maintain its predictive performance, or an entire re-building of the model to take into account new consumers or a changing market.

Predictive analytics leaves ample room to test and learn. For example, how does the new approach compare with the old approach and what is the incremental benefit? Which data performs better than others in explaining behaviour? Does the model work better in some cases than others? However, taking a test-and-learn approach requires those involved to think through the problem and the approach pro-actively. It is impossible to compare one approach with another without setting up a control group – critical for understanding what would have happened if you hadn’t used the model or if you had used a more basic or cheaper approach. Once the modelling exercise is underway, taking a test-and-learn approach can lead to even better results in the future.

There is a great deal of skill required to succeed with predictive analytics: it is not simply a case of pressing a button on specialised statistical software that then spits out a model. The skill comes from designing the conceptual approach to modelling a market or set of consumers which may require a number of overlapping models that work together.

Identifying the most predictive data, removing correlations and transforming variables so they fit the particular modelling approach you are using also requires a level of expertise.  As does looking at how data interacts in order to be predictive, rather than looking in isolation at variables that may have little impact on their own, but together are highly-predictive of behaviour.

The ability to predict future behaviour through analytics holds real value for marketers as it allows them to focus on activities that deliver results. It also results in a better experience for the consumer, too, as brands can see how individuals respond and learn from that to deliver communications and content they want to receive. This in turn helps builds relationships between brands and their consumers. 

However, as with any technique that on the surface of it appears difficult, predictive analytics can be successfully implemented by ensuring there are people in the team who have the right level of skill and expertise. A degree in mathematics or statistics alone isn’t usually enough to understand when you should and shouldn’t follow certain lines of enquiry. This comes from exposure to lots of different techniques and challenges and a commercial understanding of what you are trying to achieve. And it is this that ensures you will take a disciplined approach from the outset, choose the right modelling technique, monitoring and measuring every step of the way. Get it right and marketers have a real opportunity to predict future behaviour and consumer intent and to reap the rewards of their improved relevance.