Better insights, better service: how AI is transforming mystery shopper data

Woman making a purchase

IntelliShop, a customer experience research and advisory firm, is gaining newer and faster insights from its mystery shopper reports after partnering with NovaceneAI.

In 2011, Office Depot executive Kevin Peters swapped his suit for a pair of faded jeans, a T-shirt, and a baseball hat. Detailing the experience in the Harvard Business Review, Peters described what he discovered when he went incognito at 70 Office Depot locations across the United States – and learned first-hand some of the changes that the company needed to make to boost its performance and sales.

It’s not always realistic, however, for top executives to go undercover and learn about their companies’ strengths and weaknesses. Instead, many businesses opt for services like those offered by IntelliShop – a customer experience research and advisory firm that offers a variety of services to help businesses perform better, including sending mystery shoppers to clients’ organizations.

These mystery shoppers can provide a goldmine of information for a company: what the service was like, how quickly they were greeted, and if they would ever return as a customer, to name a few. But for IntelliShop’s analysts, sifting through this data can be an onerous and time-consuming process. The firm needed a faster and more reliable way to analyze feedback, and has recently started using the NovaceneAI Platform to transform this raw data into advanced insights quicker and more efficiently.

Analyzing data ‘the old-fashioned way’

Matt Urich, client services manager at IntelliShop, says that the firm collects its data from mystery shoppers who complete an operational audit on the company they’re evaluating.

In this audit, IntelliShop’s mystery shoppers answer questions about their experiences and also answer sentiment-based questions – for example would they return to the store, and why or why not.

“It provides us an opportunity to get some really unique insights that are difficult to get from general survey research,” Urich explains. “Oftentimes, these responses can be quite verbose – so analyzing them is extremely time consuming, particularly for our larger projects which have thousands upon thousands of instances.”

Up until now, IntelliShop relied on verbatim coding “the old-fashioned way” to analyze these insights.

“We’d have to read a bunch of narratives and try to get the 20 or so most common issues,” Urich explains. “We’d have to do a lot of the heavy lifting.”

He adds that the clients’ expectation for turnaround is quite ambitious, so analysists would have to conduct this work under tight timelines, which left little time for providing deeper analysis to the client.

“We want to provide our clients with a more complete picture of their operations and what the issues are. To do that, sometimes you really have to delve into those open-ended questions – and that part of the analysis is often the most time-consuming and laborious,” Urich says. “It was essential for us to find a way to speed up that analysis.”

Shifting to an AI solution

The company turned to AI to help analyze its data quicker – and to discover key insights from the data that can help bring more value to their clients.

Urich says IntelliShop partnered with NovaceneAI because of its expert team, and its platform provided the capabilities that IntelliShop needed.

“The platform had lots of things we were looking for. We could easily integrate our data into the platform, and it was fast, easy, and helped us keep the pace of our clients,” Urich says.

While they’re still in the early stages of working with the platform, Urich says he’s been impressed with the tool’s speed, summary capabilities, and its ability to capture “nuggets” of information that helps strengthen their analyses for clients.

Beck Langstone, AI Solutions Analyst at NovaceneAI, says that the team has worked extensively on the platform that helps clients, like IntelliShop, categorize responses that have different sentiments – for example, “the restaurant was clean, but the service was bad.”

“When you get into verbatims, there’s often those nuggets of good insight. The platform allows our clients to make those more in-depth discoveries,” she says.

Marcelo Bursztein, Founder and CEO of Novacene, adds that the partnership is a great opportunity for NovaceneAI to further its research into advanced analytics.

“It’s the perfect use case for all the R&D that we do here,” says Bursztein. “Everyday, our experts work with text analytics, and we always welcome opportunities for real-world applications of the technology.”

A ‘transformative’ tool

Urich says NovaceneAI has provided exceptional support to the company as it begins to incorporate AI into its operations – adding that the NovaceneAI team has been consistently working to improve the technology as it continues to evolve.

“They are very eager to understand what their clients’ needs are, and how to adapt the platform to those particular needs,” he says.

Urich also describes the platform as an innovative technology that could change the way IntelliShop analyzes its data now, and in the future.

He says he’s impressed with the tool’s capabilities – including the insights that it’s able to capture, and the level of information it has been able to provide IntelliShop.

“Based on the potential we’ve seen, I think the NovaceneAI Platform could be transformative for our business,” Urich says.