To understand the impact that artificial intelligence (AI) will have on market research, it is first important to be clear about what exactly AI is and what it is not. Artificial intelligence is the intelligence displayed by machines, often characterized by learning and the ability to adapt. It is not the same as automation.
Automation is already widely used throughout the insights industry to speed up a number of processes. From recruitment to data collection and analysis, automation is simply the set of rules that a machine follows to perform a task without human assistance. When complex logic and branching paths are introduced, it can be difficult to distinguish from AI. But, there is an important difference. Even in it’s most complex forms, when a task is automated, software follows the instructions it has been given. The software (or machine) does not make any decisions or learn something new each time the process runs.
Learning is what distinguishes artificial intelligence from automation. And it is what offers the greatest opportunities for those that embrace it.
Examples of AI in Use Today
Already, there are a number of ways in which artificial intelligence can empower researchers with insights and analysis that would not have been previously possible. Most notably is the ability to process large, unstructured datasets.
Processing Open-Ended Data
Dubbed Big Qual, the process of applying statistical analysis techniques to huge volumes of written data aims to distill quantitative results.
The Google Cloud Natural Language API provides a demonstration of this in action. Using the first paragraph of this article as an example, the program recognizes “AI” as the most salient entity in the paragraph (i.e. the most central to the text). It is also able to recognise the category of text, syntactic structure and offer insights on sentiment. In this case, the first and third sentence had a negative sentiment, while the second was overall more positive.
When applied on a large scale, this can reduce the time it takes to analyses qualitative responses from days to seconds — particularly in the case of open-ended data.
Proactive Community Management
A second way in which artificial intelligence is being used today can be observed in community management. As any community manager will testify, one of the largest threats to a long-term community is member disengagement. This can result in a high churn rate, increased management effort and lower quality results.
Fortunately, behavioural predictions, driven by AI, reduce the risk of disengagement. Behavioural predictions involve analyzing a vast array of data points about community members such as number of logins, pages visited, time between logins etc. to build individual engagement profiles. Once built and compared against disengaged members, the AI is able to identify which members are at risk of becoming disengaged. This allows community managers to provide additional support and encouragement to these individuals, reducing that risk.
Over time, this means that a community managed with the assistance of an AI will, on average, lose less members to disengagement than a community without AI support.
Market research and insight LinkedIn groups may be about leveraging the knowledge of others to your advantage. but remember. this is a mutual beneficial relationship
Machines Making Decisions
Give a machine enough information and it will be able to make a decision. And that’s exactly what Kia did over 2 years ago when the brand employed IBM’s Watson to help decide which social media influencers would best support its Super Bowl advert.
Using Natural Language Processing (NLP), Watson parsed the vocabulary of social media influencers to pinpoint which exhibited the traits Kia were looking for — openness to change, artistic interest and striving for achievement. What is perhaps most interesting about this example is that the decisions Watson made are ones that would be difficult for a human to make, demonstrating the possibility that perhaps AI can understand us better than we can ourselves.
The Future of AI in Market Research
Of course, progress never stands still. We are still very much in the absolute infancy of artificial intelligence, and it is a technology that will have a much wider impact on market research in the years to come. Although there is little way to predict exactly what this impact will be, the ideas highlighted here are already in development — and may arrive sooner than we think.
Virtual Market Research & Forecasting
Recruitment is expensive. In fact, depending on sample size and the length of a task, it can quickly eat away at a research budget. One proposed idea to reduce this cost and stretch insights budgets further is to build a virtual panel of respondents based on a much smaller sample.
The theory is that sample sizes naturally restrict a brand’s ability to understand the behaviour of every potential consumer and customer. Therefore, taking this sample, representing it as clusters of behavioural traits and building a larger, more representative pool of virtual respondents from the clusters offers a more accurate prediction of behaviour.
There are abundant limitations to this method, such as the likelihood that virtual respondents will be limited to binary answers in the first instance. However, there is still value in this — especially when combined with the ability to run a massive number of virtual experiments at once. This could be used to find the most appropriate price point for a product, or understand how reaction to a change in product attribute might impact sales.
Chatbots & Virtual Moderators
As FlexMR CEO Paul Hudson highlighted in a paper presented at Qual360 North America, a question still hangs over whether artificial intelligence could be used to gather conversational qualitative research at scale. Today’s research chatbots are limited to pre-programmed questions, presented in a user interface typical of an online conversation.
However, as advances in AI continue to develop, so too may these online question delivery formats. The ultimate test will be whether such a tool could interpret answers from respondents in a way which allowed following questions to be tailored and interesting points to be probed. This will signal the evolution from question delivery format to virtual moderator.
Conducting Secondary Research
Not all research is primary research. In fact, for many smaller organisations, secondary or desk research is the most cost-effective option. But it can also be a valuable tool in larger insight teams when seeking to enter new markets, develop new products, analyse competitor performance or review supply chains.
Desk research. while valuable. is constrained by available resource. AI is poised to change this.
The natural limitation to desk research is resource. Desk research, while valuable, can be time consuming meaning that insight does not always reach the hands of decision-makers before a decision is taken. Artificial intelligence is capable of reading, learning themes and identifying trends much faster than a human, making this a potential application of the technology. The current barrier to adoption of this is sourcing material for the AI to process. However, as an increasing amount of content is archived on public and private networks, this barrier is slowly eroding.
Unfortunately, it seems that — today — despite the broad scope of AI much of the debate surrounding artificial intelligence in market research today centres around the topic of coding and how the technology could speed up the analysis of qualitative data. While this is an important topic, the range of applications AI could have within the insights industry is much broader. For insight professionals to make the most of this powerful technology, still very much in its infancy, it is important that this greater range of applications are fully explored.
The original version of this article appeared on the FlexMR Insight Blog and can be accessed here.
