It doesn’t seem possible to read about marketing and technology or attend a conference these days without the topic of big data arising before long. Everyone’s talking about it, from vendors and commentators to agencies and marketing professionals — but what does it really mean for Marketing, how seriously does it need to be taken, and what are the first steps to adoption in your organization?
What is Big Data?
The commonly accepted definition of big data is known as the Three Vs:
- Volume: Clearly, the number one characteristic of big data is that there’s a lot of it. IDC estimates that between 2005 and 2011, the digital universe exploded from 130 exabytes to 1,800 exabytes (one exabyte = 1 billion gigabytes (Gb)). 90% of the data stored in the world today has been created in just the past two years, according to IBM.
- Velocity: Rapidly changing and rapidly occurring data needs to be handled in near-real-time. Clicks, visits, purchases, Tweets,
posts and video uploads generate an endless stream of data to process, which is increasingly being met with affordable technology to do so.
- Variety: The many different sources and types of data being generated today contribute to the overall challenge, particularly unstructured video feeds, photos and social media posts. Previously, structured data came in rows and columns that could be held and processed in conventional SQL databases, but now different tools are needed to handle new types of data.
In addition to these three, IBM added another characteristic:
- Veracity: Meaning “conformity with truth or fact”, this dimension of big data has less to do with the inherent characteristics of the data — but with how it needs to be used. This may hint at the lack of consistency or quality in big (and other) data sources, which need to be taken into account.
Many of the key drivers behind big data relate to other technology megatrends currently being experienced.
- Social media makes a substantial contribution to all of the aspects of big data above, and of course has itself experienced prodigious growth. 65% of all adult Internet users access social media sites.
- Spend on cloud data management is growing from $81.3B in 2011 to $148.8B in 2014.
- Mobile usage continues to expand across the world and generates huge amounts of data covering calls, location and data usage.
Should We Care?
According to Aberdeen Group, organizations of different sizes that have implemented big data initiatives report a 12% year-over-year improvement in operating profit. On average, these performance improvements are more than 26% better than for companies that haven’t embarked on big data efforts.
The danger with the concept of big data is that it becomes a bandwagon onto which everyone jumps, especially vendors looking to sell software and solutions and the media who need something new to talk about. On one level, it’s gratifying for data to achieve a similar profile to social media or search marketing, having previously been seen as something of a back-room undertaking. Getting caught-up in a whirlwind of hype is counterproductive, though, and it risks damaging perceptions of the role that data can play. A leading big data commentator, Gregory Piatetsky-Shapiro, sums it up best when he says, “The phenomenon is real. The potential is real. What is new is just a buzzword that captures the promise that big data will do things better. But it will not produce miracles.”
On balance then, the phenomenon and benefits of big data are real and should be taken seriously, albeit with a note of caution. So what are some of the key steps to getting started?
What to Do
Approaching a big data initiative can seem overwhelming, so here are a few suggestions to help you get underway.
- Define objectives. As with any new undertaking, it’s easy to get caught up with thinking about defining budgets, selecting technology and forming a project team.
- Avoid rushing into these activities, though, and start by defining the objectives of a big data initiative and what needs to be achieved.
- Ask yourself what questions need to be answered, where is information needed and when, how often should it be updated, delivered, and shared and what is the business impact?
As Shawn Rogers of EMA Research puts it, “Make sure
- Think small. Are you really dealing with “big data” at all? Standard tools are entirely capable of managing very large data volumes, and these may exist in your organization already. A well known UK retailer, building a single-customer-view database of 20 million consumers from 30-plus data sources and daily updates, only required one terabyte of main storage. Consider whether all of the Vs of big data are actually involved — in an IDC study, relatively few organizations surveyed actually professed to be dealing with the velocity and variety aspects of big data.
- Break it down. Even where a true big data scenario does exist, though, all available data doesn’t necessarily have to be consolidated for analysis to take place. Look at the important data streams from which the most value can be obtained — such as click streams, social media posts and transactions — and deploy dedicated social media monitoring, web tracking, dashboards and analytics tools appropriate to each data type. Keep the objectives defined earlier in mind and avoid boiling the ocean!
- Obtain agreement and buy-in. It’s always easier to swim with the current, so look for ways to link up with existing initiatives and executive priorities. Build on existing reporting structures and presentation, placing new metrics alongside existing ones and avoiding the introduction of new processes that replicate existing ones. In addition, make sure everyone in the organization knows when positive results arise from data-driven activities, and use these successes to drive and maintain momentum.
- People. IDC estimate that 50% of new marketing hires will soon come from a technical background, with many of these adopting the new role of “data scientist”. Whether this is genuinely a new discipline or just a new name for an existing role, thought needs to be given to where these staff will be found: graduates, in-house training, overseas hires? Also, data professionals skilled in working with big data technologies can command salaries up to 20% higher than other analysts and developers, according to E-skills UK. These costs must be factored into headcount budgets.
- Data management and preparation. The old adage about data processing remains true in the era of big data, except that it becomes big garbage in, big garbage out!
- Tackling all the usual data quality issues is particularly crucial where multiple sources are being consolidated, with particular emphasise on consistency, duplication and completeness.
- Workflow and process around data collection and capture are also key considerations — especially where velocity and variety come into play — along with how data is being maintained and updated.
- Execution. Once the objective setting, planning and agreement are in place, the focus for execution should be on the data that can most easily be turned into actionable insight and deployed.
- Look for the readily available sources of transaction and interaction data and determine the path of least resistance to achieving the greatest impact.
- Develop milestones for undertaking the big data journey: crucially assign responsibilities and avoid attempting to do everything at once and being paralyzed by the scale of the initiative.
- Work with IT. The rise of cloud computing and concepts such as the chief marketing technology officer makes it tempting to think that Marketing can undertake big data initiatives by itself. However, a true big data initiative needs specialist input and cross-organizational collaboration, and is unlikely to be something that Marketing can undertake with an external vendor alone. Deep expertise across enterprise architecture, structured and unstructured database development, and analytics are all crucial to a big data initiative. Without IT involvement, it probably isn’t big data — and if it is big data without IT involvement, it will probably fail!
- Keep Talking. In addition to the tools and expertise that almost certainly already exist within the organization, there’s likely to be a wealth of insight waiting to be tapped into as well.
- Analysts in marketing insight teams may already be conducting work that can be re-used, built-on or used for inspiration.
- Other functions including Sales, Marketing Operations and Customer Service are also likely to be good sources of customer insight, given their close proximity.
- And of course don’t forget to speak to customers and prospects themselves, who are usually delighted to share their opinions and insight, without a data model in sight!
It’s Really About New Uses & Insights
True big data, then, is not so much about quantity, but also about different types of information from many different sources arriving in real-time. Not to mention the need to ensure that this data is fit-for-purpose. Only when all these factors are present is a genuine paradigm shift both necessary and inevitable. Much can be achieved by managing and making use of rich data sources through conventional means and it’s important not to get swept up in the euphoria of buzzwords and trends, but the future of data driven marketing undoubtedly lies in the harnessing of all of these elements together.
Start with small, focused initiatives, clearly define objectives and milestones, obtain cross-organization buy-in, and ensure that the necessary technical expertise is engaged — and big data will deliver the benefits that are being sought. “Big Data is really about new uses and new insights, not so much the data itself,” says Rod A. Smith, IBM technical fellow and vice-president for emerging Internet technologies. Good advice.