Organizing and Interpreting Complex Sets of Survey Data

Large surveys make the task of data analysis much more complex. In order to uncover meaningful information from your large survey efforts, you will need to define a logical process for organizing and interpreting survey data at the start. While there are numerous techniques you could employ, the following offers a series of reliable techniques for working with complex sets of survey data. Continue reading

Mind Your Measurement Scales in Market Research

Welcome to the Making Molehills out of Mountains University (MMoM U) Market Research Data Analysis 101 or MARDA 1 as we like to call it in the halls of academia.  Today we discuss the four different types of scales used in measuring behavior.  Open your books and let’s get started…

The four scales, in order of ascending power are:

  • Nominal
  • Ordinal
  • Interval and
  • Ratio

Nominal Scale

Nominal is derived from the Latin nominalis meaning “pertaining to names”.  But, seriously, who cares? That tells us nothing except how much academics love showing off.  The Nominal Scale is the lowest measurement and is used to categorize data without order.  For your market research data analysis exercise a typical nominal scale is derived from simple Yes/No questions.

How the nominal scale (and all these scales) is used statistically is for the next lecture.  For now, just know the behavior measured has no order and no distance between data points. It is simply “You like? Yes or no?”

Ordinal Scale

From the Latin ordinalis, meaning “showing order”… Enough of that.  An Ordinal Scale is simply a ranking.  Rate your preference from 1 to 5.  Careful!  There’s no distance measurement between each point.  A person may like sample A a lot, sample B a little, and C not at all and you would never know.  Here we have gross order only, learning that the subject likes A best, then B, then C.  Determining relative positional preference is a matter for the next scale.

Interval

Ah, the Interval Scale.  It’s the standard scale in market research data analysis.  Here is the 7 point scale from Dissatisfied to Satisfied, from Would Never Shop Again to Would Always Shop,  etc.  The key element in an Interval Scale is the assumption that data points are equidistant.  I realize savvy market analysts might say, “Hold on Professor. What about logarithmic metrics where the points are not equidistant?” To which I say, “Correct! but the distances are strictly defined depending on the metric used, so don’t get ahead of yourself. This is MARDA 101.”

For now, understand that with the Interval Scale, we can interpret the difference between orders of preference.  Now we can glean that Subject 1 Loves A, Somewhat Likes B and Sorta Kinda Doesn’t Like C.

Subject 2 Somewhat Likes A , Sorta Kinda Doesn’t Like B and Hates C.  Both subjects ranked the samples A, B, & C on an Ordinal Scale but for very different reasons as discovered by using the Interval Scale. Got it?  Good.

Moving on.

Ratio

Similar to the Interval Scale it’s not often used in social research.  Like Interval, it has equal units but it’s defining characteristic is the true zero point.  Ratio, at its simplest, is a measurement of length. Even though you cannot measure 0 length; a negative length is impossible, hence, the true zero point.

To sum up, I leave you with the the chart below, indicating various measures for each scale.


Difference
Direction of Difference
 Amount of Difference
Absolute Zero
 Nominal  X
 Ordinal  X  X
 Interval  X  X X
 Ratio  X  X X X




The Difference Between Causation and Correlation Research

Correlational research can tell you who buys your products, but it may or may not tell you why. For example: Let’s say that you are trying to sell instant meals. If your research tells you that working mothers buy more instant meals, you cannot draw the conclusion that being a working mother causes people to purchase instant meals. The purchase of instant meals could be due to a third factor such as being too busy to cook, having extra money, or trying to control calorie intake.

More complex correlational research studies can help you narrow down the causation, but you still cannot draw a conclusion from them with absolute certainly. In fact, the more factors you add, the more confused you may become. Let’s say you discover that only single working mothers purchase instant meals and that married working mothers rely on their spouses to cook. Should you try to market instant meals to the spouses of working mothers, or could there be yet another factor involved that is still eluding you?

Knowing for sure

If you really want to know whether being a working mother causes people to purchase more instant meals, you need to conduct a controlled experiment. This means that you would need to have a large pool of subjects, randomly assign them to be working mothers, and monitor their purchasing habits. Unfortunately, we cannot ethically force people to be or not to be working mothers. This is a lifestyle choice, and an experiment where we let the subjects choose their own treatment would not be an experiment at all.

A good way to dig deeper in your research without dealing with ethical issues is to ask an open-ended question. It may take you more time to read through all of the responses, but with the right software you can make the process much faster. You can find out how often a certain keyword or phrase is used to determine why people purchase instant meals. If a common response is, “I am a working mother,” you can be reasonably certain that being a working mother causes people to purchase instant meals. While this is still technically a correlational research study, it can give you more useful information than a survey in which you only allow simple, discrete responses.

Closing the Information Loop with Actionable Data

Analyzing survey data and turning it into actionable data is the key to maintaining your current customers and adding to that customer base. But what is actionable data? Basically, actionable data is your survey results input into an improvement program to modify your business and make it better.

In order to effectively transform your piles of information into actionable data and make analyzing survey data work, you need to look at the actual questions you ask on your survey. If you don’t ask the right type of questions your data could be misleading and as a result, you will misuse it with no benefit to your company.

Close Ended Questions

These are simple clear questions that ask a survey taker to rate specific aspects of the products, services and experiences they engaged with. They respond on a numbered scale or check boxes of different graduated measurement to reply to the questions. This type of questioning has two useful advantages.

  • Easy Analyzing: Analyzing the survey results is very simple, since all the data is coming to you as a check mark, number or yes/ no answers. The guess work is taken out of the analysis and ambiguity is kept to a minimum. The simple responses make calculating the results and transposing them into percentile information or mathematical equations very easy.
  • Easy to Take: The actual survey itself is easier for your subject to navigate through. Clicking on responses they identify with is much easier than writing down specific detailed narratives. If it’s easy more people will take it and that will give you more data to use in your growth process.

Open Ended Questions

Open ended questions ask your survey takers to describe something in detail. While analyzing survey data, the more detail oriented the survey answers are, the more focused and exact our improvement efforts can be. The problem is that open ended questions rarely get answered and if they do it’s with one or two words. Not really the detail you were looking for and very misleading if you use it. Open ended questions should be used sparingly and only as an addendum or supplement to the closed question parts.

Filtering and Branching

Two survey construction techniques that, when used correctly, will make analyzing survey data easier by pre-sorting and categorizing certain elements of the survey as determined by you.

  • Filtering: In the survey, it serves the useful purpose of weeding out the subjects that you don’t need results from and lets you focus on subjects whose opinions you want. When it comes to analyzing, you can implement filters like location, age, or whatever to help with the process. This allows you not waste peoples time (including yours) and will increase the amount of useful data.
  • Branching: Similar to filtering, branching just divides up your subjects and sends them down different survey roads, if you will. This allows you to view specific results easily and categorize your raw data. Branching serves as a pre-sorter that will streamline you process for analyzing survey data.

Conduct Interviews: Yes, more surveys. I know it sounds weird but if you think about it, it makes sense. Interviews effectively expand the scope of your resulting data. The pre- and post-survey interview helps to make sure the subjects understand the topic or field the survey is concerned with and they add dimension to the scope of the main survey.

Analyzing survey data and turning it into usable, actionable data will positively effect your business. Invest the resources into well thought out data gathering and analysis programs and reap the rewards.

Technology’s Impact on Data Collection in Market Research

The widespread adoption of the internet, and now smartphones, has revolutionized all industries with very few exceptions. If you have been in the market research industry for the past few decades, you know first hand how drastically these technological advances have changed the way researchers conduct business.

In the mid to late 80s, gathering survey data by phone and mail had become the standard over surveying consumers door-to-door. Today, however, the declining popularity of landlines and traditional mail has led researchers to place more focus on online alternatives. With its ability to reach consumers at multiple touch points and near instant results, it’s easy to understand why the internet has won data collection dominance. While market researchers have a wide selection of options for collecting data, having reliable research and effective analysis tools has become more vital than ever.

We know that market research technology will continue to evolve. Present and future trends point to social media and user generated feedback where we can analyze what consumers are saying rather than just observing them. The ability to adapt to these new trends will be an important factor in staying competitive and delivering the products and services that consumers want and need.

In spite of all the changes in market research over the past two decades, the objectives remain the same: glean insights from consumers, and respond in a manner that will increase sales and market share.

Steve Jobs’ effect on the market research community

Can Mr. Jobs be credited with inventing the idea of market research?

Well, no!
But he can surely gain credit for shaping the future of how it is conducted! For so long, if you wanted to find out what people thought of your product, the only way was to get out on the street and ask them. Of course, we then moved on to mailing out questionnaires and interviewing by phone, but each of these requires a significant time investment on the part of the interviewee.

With the digital age upon us, things have moved on rapidly and peoples thirst for information seems never ending. High speed connections are everywhere and you are never far from a screen of some kind.

With this technology explosion, researchers have worked hard to keep up, fielding surveys by cell phone based text messages, emails, web pages and any other means they can conjure up. Consumers are encouraged to participate in research efforts through links on retail receipts, direct TV/radio advertising and most recently through the use of ‘QR Codes’ that are placed in magazines, newspapers, on product packaging, stickers and even clothing, encouraging the inquisitive among us to pick up their smart phone, scan the code and follow the link to find out more!

So what, you say, does all this have to do with Steve Jobs? Well, it was only last week that it was noted that “around 92% of Fortune 500 companies are currently testing or using iPads as a corporate solution”. And to make that statistic even more impressive, the iPad has only been on the market for just over 18 months!

Whether it be in the local mall, small town tourist office or at a specialist research event, tablets are the go-to device of the moment for data collection.  Apps are springing up to make this easier than ever and with their portability and ease of use, this trend will only increase.

The unquenchable thirst for knowledge that I mentioned earlier, now seems to be driving more and more people to not only want to collect data via tablets, but also to want the ability to delve in to the data further, to find the nuggets of information hidden within it.  Developers are reacting quickly to this need and in the very near future, it is safe to assume that the number of tablet based survey analysis tools will grow drastically.

And all any of us can really say is: Thanks Steve!

To learn more about how to prepare your survey data for analysis, feel free to download our whitepaper Ten Essential Prerequisites for Survey Data Analysis.

Qualitative Research – Tips on how to use Unstructured Research Data

In my previous blog article ‘mTAB Understands Qualitative Research Needs!’  I discussed the need, or advantages, to utilizing qualitative, or unstructured, data when reviewing research results; now I want to help you learn how to implement this suggestion.

I have always visualized structured response data in my imagination as a city with sky-rise buildings and lots of traffic; there are mathematical equations creating a functioning city and it works as well as we need it to. In my imaginary city I visualize unstructured data as a reservoir of gold in the underground tunnels that the city is built on. Burgeoning cities have a tendency to expand, and expansion will generally follow the basis of the original model, but improvements are made by utilizing the underground reservoir.

Yes, fanciful, I know. Keep following me, structured response data gives you an answer to exactly what you’ve asked, but how did you decide what to ask? Have you based your survey response questions on your desires, or your consumer’s desires? Unstructured data allows for responses that are unsuspected and can lead to great improvements.

Let’s use an example: Car manufacturing. The survey asks for demographics, model preferences, etc. Generally an unstructured data response will be positioned as a means of gathering more information about a previous structured data question; e.g. ‘why did you answer yes to this question?’ This is brilliant because the survey is looking to dig deeper into reasoning, however, the full potential of unstructured data is not being utilized.

To harness the power of unstructured data one must think outside of their normal understandings. If you work in the automobile industry you immediately take for granted the need for automobiles. This assumption is the basis for the rest of your survey. Now, if you realize your previously held assumptions about your field and bring them into question, you may find that reservoir! Survey respondents don’t want to write a book while filling out your survey, so you’ve got to choose your open-ended non structured questions wisely. Some examples may be: why do you drive a car instead of taking mass transit? If money was not an issue what car would you buy, and why? What is the most important aspect of a car in your opinion? If you’ve ever considered fuel-alternative transportation, what factors concerned you? Are there any questions that you would want to ask us?

Asking your customers questions that you haven’t assumed answers to will result in responses that you couldn’t have anticipated, and insight you may have overlooked. By incorporating unstructured data responses in your survey you may just find the seedling of an idea that will greatly improve your product, and also give your customers the satisfaction of knowing that you’re interested in their thoughts.

Please be sure to follow my blog posts regarding qualitative survey analysis.  Keep an eye out as well for text analytic posts that we’re planning to run in the near future.

Collecting Survey Data vs. Analyzing Survey Data

To grow in today’s tough economic market, every hour and dollar spent needs to yield positive results that can be used to facilitate growth. Collecting survey data and analyzing the data are two different, but very important steps you must take to help identify crucial areas of growth.

Collecting Data

Data collection comes from a variety of sources and interactions, such as face to face interviews, mail surveys, web survey, and offers, and the amount of information that can be collected and compiled can be staggering. However, simply collecting this data is not enough.  While it is a crucial step, it is what you do with the data that is key to growing your business.

Analyzing your Collected Data

In order to utilize this information to target areas of growth; you will need to have a strategic survey analysis plan in place.  Having a wealth of information on hand is one thing, but being able to effectively use the information is another.

Compile & Validate your Information

Before you do anything, compile the data, including any questions, answers, and profiles of participants.  Make sure it is fairly easy to move and group your results. Check to see that all of the questions were understood.  See if participants made comments about particular questions or left specific questions unanswered, and then consider what you will do with this data.

For example, if your survey question asked the participant to provide a rating from 1-5 and you find the question was answered with something other than <blank>, 1, 2, 3, 4, or 5, how you will interpret the findings?

Identify Patterns

Once you have compiled and validated your results, break the data down by grouping it into classes and recording how many data points fall into each class.  This is where you check for patterns based on gender, race, age, religion or any other information you collected from participants, and where your raw data is transformed into valuable information.

Establish Simple Frequency Distributions for Each Survey Question – Frequency distributions demonstrate how many observations on a given variable have a particular attribute.  You may choose to look at gender, income levels, age ranges, and etc. For example, a survey is taken of 50 people.  The frequency distribution might indicate 25 people selected (a); 5 were female and 20 were male. The other 25 people selected (b); 12 were female and 13 were male.  Distributions may be displayed using percentages, or in terms of a bar chart.

Ultimately, the more detailed your analysis, the more valuable the data becomes.

Turning Survey Data into Actionable Insights

What is most important to your customer base, and what drives their overall satisfaction? Undoubtedly, it can be difficult to know what customers are looking for from your business. Many factors go into whether or not a prospect will browse your products, make a purchase and, most importantly, return for future business. With proper survey development and survey data analysis, you can accurately uncover these key drivers.

To identify what drives customer satisfaction, the first step is to ask your customers. The information gathered from surveys can pinpoint where to focus and will help with the strategic planning process.

Keep the following objectives in mind as you create the questions for your surveys:

  • Find the Top Attributes. A variety of elements could be important to any given customer, but with a wide-spread survey you can discover which are important to the largest number of people. These top attributes will be the key drivers that you’ll want to focus on improving and maintaining. If you run a shop, you may find the most people desiring “Pleasant atmosphere,” “Good return policy,” or “Large selection.” If you’re an online company it may instead be “Easy to navigate” or “Availability and helpfulness of customer service.”
  • Get a Rating. Once you know what it is that your customers deem important, you’ll also need to know how well you’re doing in each aspect. Have your customers rate how well they find your service in each attribute. The most important ones to focus on are the top-rated elements that your customers have found lacking from your business. That way you don’t have to pour money and resources into every aspect of your company. Instead, you can target specific points that your customers actively desire, and potentially even move money away from the unwanted attributes.
  • Calculate Importance. While everyone in your survey may say that they want “Good customer service,” that may not be of greatest importance to them. It may not, for instance, affect whether or not they return to your business. Make sure that your survey asks customers to list desired attributes by their importance, so that you have the most specific information available.

Driver analysis surveys can be very simple or complex; however, analyzing the results is an intricate process.  As a result, many organizations struggle to understand how to turn survey data into actionable insights. You don’t need to spend a fortune trying to identify what makes your business lucrative and desirable to customers. The availability of survey data analysis software has made it possible to quickly visualize data in a useful, easy to understand way. These tools offer an affordable solution to glean the most important data from the people who use your services and make well informed business decisions.

Survey Data Analysis – Online survey tools often fall short

Survey data collection is just as important, if not more so, than the reports generated by that data.

Most market researchers, however, are more concerned about what the results of a given survey are saying rather than the data collection tools used to provide those results. Especially with all of the new web-based, do-it-yourself (DIY) data gathering tools popping up recently, there are many different options one can employ when conducting research. I won’t name any of them here because, while they allow for quick and dirty methods of obtaining data, they don’t do a very good job of trending and reporting.

Let’s first look at the quick and dirty aspect. Using DIY tools allow the researcher to quickly develop and deploy a survey on the internet, but without careful planning and long-range thinking, this may be the beginning of a giant mess. These tools do not restrict or recommend how to setup the underlying single file database. There are no established naming conventions for variables, nor are there any data validation methodologies employed by the software. I’m not trying to say that these tools should have these things built-in, because I think it would be too restrictive and lessen their appeal. Instead, this aspect is not being given enough respect by those responsible for setting up the data collection.

While this strategy works for small one-off surveys, it becomes more problematic when the survey is longer, more complex or is conducted multiple times per year (waves) or multiple years in succession.

If your survey involves more than one data gathering effort, then you will want to make sure that the variable naming and definition remain constant across those efforts. Also, you will want to make sure that the data integrity is intact, or that some validation/verification is being performed to make sure you don’t get invalid data. This can either be done on the front-end (during the data gathering effort) or on the back-end (through scripting or some other verification method).

These aspects of survey reporting are something to consider when employing your next data collection effort for market research. Though DIY survey creation is an exciting step for marketers there are still restrictions on using them for large, trending data sets. Whether or not you choose to use DIY tools or something more robust, planning out the data structure and integrity is critical to gaining key consumer insights.

If you’d like to learn more about how to prepare survey data for analysis, please download our whitepaper Ten Essential Prerequisites for Survey Data Analysis.