On Thursday 9 th May 2024, Team Russell and Team Hudson duelled in a panel debate modelled off the popular TV show Room 101. This mock-gameshow-style panel, hosted by Keen as Mustard Marketing’s Lucy Davison, was held at Quirks London and featured two teams of insight experts fight to convince the audience that their most-hated aspects of market research should be banished forever to the industry’s own ‘Room 101’.
Round 1 of this panel debate took place a month earlier in the Market Research Society’s Annual Conference 2024, where after 6 rounds of heated debate, both teams tied with two banishments apiece — this led to a fiery battle in Round 2, where it went to another nail-biting tie-breaker — let’s see which aspects of market research the audience banished and which team won this time.
Market Research Room 101 — Round 2
Significant Difference
Statistical difference is a classic scientific practice that helps researchers understand whether one group’s answers are different from another group’s answers through statistical analysis. This is what Paul Hudson wanted the audience to consider and see if it was really a beneficial scientific practice to use in market research or whether we should stop using it in favour of other, better, practices. Paul argues that it’s built more for science rather than consumer behaviour.
Danny’s team immediately countered, saying that science is important. Market research is part-science and as such statistical differences have their place in our analytical toolkits. To Paul’s disappointment, the audience sided with the other team, and saved Significant Differences from Room 101.
Respondents
James Sallows is a returning competitor and this time James asked the audience to consider one of the everyday terms we use and how it impacts our behaviour in research projects. This term is ‘Respondents’. James argues that it is a reflection of our lack of respect for the people that make our industry. Just because they’re anonymous doesn’t mean they’re not still human and deserve to be treated as humans. When we distance ourselves from them, we put ourselves in danger of treating them as numbers rather than people.
While the other team believed that it’s just a term like many others we use and not to look too deep into it, the audience agreed with James and banished the term to Market Research Room 101.
The One-Way Mirror
The One-Way Mirror is a long-standing staple of market research, starting in the physical realm to allow researchers to observe participants without participants knowing they were there, this was one aspect of market research that was transferred to online research, as researchers now observe participants behave in online forums, focus groups and more from specific observer areas. Louise Sharpe brought this to the attention of the audience and asked us to consider the value it holds in modern research practices — Louise believes that face-to-face conversations are more valuable, and the mirror only serves as a psychological barrier.
Louise argued that if you’re in the room you can engage with participants and this gives both stakeholders and participants a more active role in the research. However, the other team argued that we can still provide that feeling in both stakeholders and participants in other ways, the one-way mirror isn’t as much as a barrier as we might think, to which the audience agreed and saved it from banishment.
Generational Cohorts
Team leader Danny Russell presented his most-hated aspect of market research, Generational Cohorts. He argued that classifying people on one variable is outdated, we don’t do it with any other variables so why Generations? Tea Hudson put up a good fight, countering that demographic segmentation is useful and we need a common language to translate what we find.
Danny concluded that we might as well do your segmentation based on foot size for all it’s worth, and doing it just because it’s simple isn’t a good argument. The audience agreed with Danny and voted to banish Generational Cohorts to Room 101.
Political Polling
Shehnaz Hansraj has a particular frustration with the current state of political polling, especially since they’re so popular with media outlets and business stakeholders. Shehnaz believes that these polls are so often inaccurate and can lead to respondent complacency. However, the other team argued that if we banish political polling from the industry, it will still exist because there is still a profitable market.
Even as Shehnaz maintains that political polling is a distraction, they’re just guessing, the audience voted to save political polling.
Banning Surveys
This is perhaps the most controversial topic up for debate, as surveys are still one of the most popular and used quantitative research tools today. However, Zoe Dowling brought this to the audience at Quirk’s London and there was an interesting debate; the main points brought up in this discussion were that there are significant challenges associated with collecting valuable insights through survey methods, with fraud and biases, and Zoe argued that we have spades of Big Data, as well as qualitative and behavioural methodologies to use to gain the same valuable data.
In the end, the audience votes to keep the stalwart survey, as they are critical tools to help us ask questions and understand people’s feelings and perceptions on any subject.
To Banish or Not To Banish?
With a fresh set of aspects up for debate, this panel uncovered some great questions about modern market research practices, mindsets, tools and methodologies. These discussions are useful, even if they don’t have a tangible impact on the industry, they make us question the well-established practices and thinking that dominate our industry to better understand if they still fit into modern research projects, or whether there are better ways of gaining actionable insights.
While at Quirks London, FlexMR held a prize draw where entrants submitted their own most-hated aspects of market research for consideration! Our board became full of fantastic opinions, with some humorous and others very poignant, for example:
Self-doubt; synthetic data; 400 slide PowerPoint decks; NPS scores; confirmation biases; fraud; AI/ChatGPT; method snobbery; jargon; academic researchers; observers in qualitative interviews; The Debrief; pricey niche demographic samples; storytelling; “agile”; perfectionism; vanity metrics; CATI surveys; siloed teams; non-targetable audiences based on segmentation; My Boss (mentioned twice!); bots; the fear of seniority and not seeing everyone as a peer; vague KPIs; red herrings; sample aggregators; the belief that only big numbers validate research results; low response rates; and insights scepticism.
Everyone has their own opinion based on their own experiences, and these unique perspectives are what make the market research industry so great. So, what would you banish to Room 101 if you had the chance?
This article was originally published on the FlexMR Insight Blog and can be accessed here.
