Survey data analysis – Text Analytics for Net Promoter Surveys

As a consumer insights manager at a national retailer, you are responsible for understanding the wants and needs of your customers.  You’ve followed the sage advice of the Net Promoter Score (NPS) experts, and you’ve implemented an online NPS survey capturing “Would recommend..” that allows customers to leave open-ended comments.

In our prior posts we’d discussed how to join internal “structured” data such as retail location, transaction date/time, payment type, check amount, etc. to your survey results, segmenting the NPS scores to identify  the customer segments requiring the most attention.  But how do you make sense of a 100,000+ open-ended customer comments?  

You can segment your open-ended or “unstructured” customer comments using your “structured” data in the same manner as your NPS scores.  For example, we could focus our review of customer comments to the customers paying with a credit card within the Eastern region reporting an NPS score of 50 or less. Unfortunately, national retailers will likely find that they still have hundreds to thousands of comments to review even within very focused segments.

We can use text analytics to help make sense of survey comments without having to read every individual comment. Term frequency (TF), or the percentage in which a selected term occurs, helps gain an understanding of the important concepts underlying the comments, just like you would use the mean of a series of numbers to gain an understanding of a numerical series.  Term frequencies can be conveniently analyzed using a selectable tag cloud graphic as we’d illustrated in a previous post.

When TF analysis is combined with segmenting the comments using our “structured” data segments, we can compare the term frequencies between various segments to gain an understanding of the relative importance of key concepts within each segment.

We can further refine term frequency analysis by normalizing our TF terms using inverted document frequency (IDF) analysis. TF-IDF analysis goes a step beyond TF analysis by considering both the frequency of occurrence of a term with a individual comment (TF) as well as term’s frequency of occurrence within either all comments or a segment of comments.

We can then use TF-IDF analysis as the starting point for determining the relative similarity of comments. This additional step allows us to call out a small set of comments, for example, the top 20 comments, that are the most representative of the larger group. These top 20 comments serve as an “executive summary” of the overall set of comments, summarizing the key topics and issues without the need to individually review the entire series of comments.

We would welcome the opportunity to help you to reveal the hidden insights within your NPS survey results.  Please follow us on Twitter http://www.twitter.com/mTAB to receive updates to our ongoing survey analysis series as well as more information pertaining to our mTAB survey analysis service.

Qualitative Research – Tips on how to use Unstructured Research Data

In my previous blog article ‘mTAB Understands Qualitative Research Needs!’  I discussed the need, or advantages, to utilizing qualitative, or unstructured, data when reviewing research results; now I want to help you learn how to implement this suggestion.

I have always visualized structured response data in my imagination as a city with sky-rise buildings and lots of traffic; there are mathematical equations creating a functioning city and it works as well as we need it to. In my imaginary city I visualize unstructured data as a reservoir of gold in the underground tunnels that the city is built on. Burgeoning cities have a tendency to expand, and expansion will generally follow the basis of the original model, but improvements are made by utilizing the underground reservoir.

Yes, fanciful, I know. Keep following me, structured response data gives you an answer to exactly what you’ve asked, but how did you decide what to ask? Have you based your survey response questions on your desires, or your consumer’s desires? Unstructured data allows for responses that are unsuspected and can lead to great improvements.

Let’s use an example: Car manufacturing. The survey asks for demographics, model preferences, etc. Generally an unstructured data response will be positioned as a means of gathering more information about a previous structured data question; e.g. ‘why did you answer yes to this question?’ This is brilliant because the survey is looking to dig deeper into reasoning, however, the full potential of unstructured data is not being utilized.

To harness the power of unstructured data one must think outside of their normal understandings. If you work in the automobile industry you immediately take for granted the need for automobiles. This assumption is the basis for the rest of your survey. Now, if you realize your previously held assumptions about your field and bring them into question, you may find that reservoir! Survey respondents don’t want to write a book while filling out your survey, so you’ve got to choose your open-ended non structured questions wisely. Some examples may be: why do you drive a car instead of taking mass transit? If money was not an issue what car would you buy, and why? What is the most important aspect of a car in your opinion? If you’ve ever considered fuel-alternative transportation, what factors concerned you? Are there any questions that you would want to ask us?

Asking your customers questions that you haven’t assumed answers to will result in responses that you couldn’t have anticipated, and insight you may have overlooked. By incorporating unstructured data responses in your survey you may just find the seedling of an idea that will greatly improve your product, and also give your customers the satisfaction of knowing that you’re interested in their thoughts.

Please be sure to follow my blog posts regarding qualitative survey analysis.  Keep an eye out as well for text analytic posts that we’re planning to run in the near future.

Enhancing Your Net Promoter Score Survey Analysis

Imagine that you are a retailer tracking your net promoter score (NPS).  Your NPS survey is administered by way of an invitation printed on your in-store POS receipt.

The point of tracking NPS is to identify how to make improvements in your service and products.  Changes in your process can then be validated by continual tracking of the NPS metric.

So how can we best identify opportunities for improvement from the NPS tracking graphic below?

We can simply observe that the NPS score has gone up or down relative to our last observation.  Assuming at least one open-ended question in addition to our net promoter survey question, we can review the open-ended or “unstructured” data from customers providing low (or high) NPS scores.

Text analytics tools, for example tools providing classification and sentiment analysis of unstructured survey responses, along with search term and tag cloud drill down tools, can significantly enhance the understanding of the unstructured survey responses.

Additional structured data, coupled with the appropriate tools for the analysis of structured and unstructured survey questions, will significantly enhance our analysis and thereby greatly increase the value of our net promoter program.

POS receipts typically include a transaction identifier that can be used to link a wealth of structured data to the survey results.  Listed below is a partial listing of possibilities:

  • Store Location
  • Transaction date/time
  • POS terminal and sales associate
  • Inventory of items purchased
  • Items purchased on sale or promotional items
  • Payment type
  • Total amount of purchase (total check)
  • (assuming you have a frequent buyer program) buyer profile

Without adding questions to our NPS survey, we can now break down scores by region, district, zone and store, by day-part, by payment type, by check amount, by store department and potentially by frequent vs. infrequent shoppers.

If you would like to review your NPS program with industry experts  please sign up for a web meeting or simply visit our web-site to learn more about our survey data basing and analysis services.

Customer Retention Rate – What does it mean?

t is one of the most profitable methods for selling goods and services. Once a company has attracted a consumer to ‘buy’ and ‘try’, a satisfied customer is the best means to retain that business for the foreseeable future.

There are many reasons to keep customers satisfied; most importantly to earn their business again. All marketers know that it costs much less to convert an existing satisfied customer into a repeat sale than to ‘conquest’ a new sale from another brand. If you browse enough web sites, you will see that…

  • Acquiring new business can be 5 times more expensive than retaining customers.
  • Increasing customer retention by just 2% can translate into a 10% cost reduction
  • Some retailers indicate their top 15% of ‘loyal’ customers comprise 50% of their sales revenue.

Those are some valuable customers and the statistics speak to why a high customer retention rate is so important.  Substantial sums of money are spent on survey data and questionnaires because a Satisfied Customer = A Valuable Customer !

So, how likely is your company to retain customers in the future? Past and current behavior is the best predictor of future trends. It is of the utmost importance for marketing analysts, brand managers and advertisers to understand both the current retention rate and defection from the analysis of ownership and/or survey data. The Customer Retention rate is often referred to as Loyalty.  Loyalty is represented by the percent of current owners that repurchase the same brand. Those who don’t repurchase the same brand are defectors and they represent lost business.

Also of importance to brand health is the measurement of conquest. Instead of looking at the current owners and “where the business went” (retained or lost), a look at new customer’s behavior will supply a measurement of where the business came from, or where new sales were ‘conquested’ from a competitive brand. Conquest is also important when considering a brand’s customer retention rate.

The graph below shows overall brand health by plotting the brand customer retention rate (loyalty of current owners) against the conquest rate (new sales from a competitive brand).

  • Brand A is the healthiest of brands given its strength in customer retention of current owners while attracting new buyers from the competition.
  • Brand B is successful in attracting brand switchers, but needs to work on current owner’s satisfaction to ensure a higher customer retention rate.
  • Brand C is relying on customer retention, but is in danger of slipping into ‘Decline’ if a downward trend in loyalty occurs.
  • Brand D is the weakest relative to its competitors and needs to identify a strategy to move its position.

Where is your brand’s health in the customer retention and conquest relationship? Is your survey data capturing these elements? What does it look like in terms of specific demographics or geography?  How does it compare over time?

PAI would welcome the opportunity to demonstrate how PAI’s mTAB™ service would benefit your understanding of customer retention rates, defection and conquest.  Please visit the PAI website to schedule a no-obligation review of mTAB for an analysis of consumer behavior data.

Understanding your survey’s Net Promoter Score calculation

Net Promoter Score (NPS) is a popular customer loyalty tracking metric that is frequently included within consumer surveys.

NPS is based upon on a simple premise; growth of your business is directly related to your customer’s willingness to “promote” or recommend your product or services to others.  Think of NPS as a “bottom line” tracking metric that summarizes your customer’s experiences and loyalty into one understandable and explainable measure.

The NPS calculation starts with a survey data containing the question “On a scale of 0 to 10, what is the likelihood that you would recommend this (product or service) to a friend or relative?”

The score behind the NPS is calculated by subtracting the percentage of the bottom 7 box responses, in other words those indicating they would be very unlikely to recommend, from the percentage of the top 2 box responses, or respondents that would be very likely to recommend, as depicted within the illustration below.

NPS scores can take on any value within the range of 100% to -100%, with 100% being the desired or objective NPS.

The point of collecting and tracking NPS metric is to take action when the NPS data suggests room for improvement.   An NPS score in isolation has limited value; perspective is required to bring meaning to the data.   Perspective can be obtained by tracking of the NPS score over time, by comparing the NPS score with NPS scores of similar products or services, or by segmenting the survey data by respondent subgroups and comparing the NPS scores of the subgroups.

Segmenting of the respondent data is made possible through secondary data that can be linked to the respondent (example: receipt transaction number linking to CRM data) or through additional structured and unstructured data presented to the respondent within the survey questionnaire.   It is extremely important to include the appropriate survey questions that will provide for a meaningful segmentation of the NPS results.

For example, you would want to know the NPS differences between heavy user and light user segments, especially if heavy users comprise the vast majority of your current sales volume.   If you don’t have a way of determining segmentation from your survey or secondary data sources, then you may be missing out on a key opportunity to gain additional insight from your NPS data.

There are many other factors that direct relate to the value of NPS metric such as sampling, sample size, and the percentage answering the NPS survey question.

PAI would welcome the opportunity to demonstrate how PAI’s mTAB™ service would benefit your understanding of the meaning implications of your NPS metrics.  Please visit the PAI website to schedule a no-obligation review of your current NPS program.