John Kavuma, Author at Löger Group

Modern professional business to business and business to consumer research is a delicate field of practice with high demands imposed on the various stages of planning, recruiting, distributing, collecting, recording, assessing, and presenting large scale research projects. It is with a good reason why business analysts are increasingly called upon to manage the logistical project management part, and why data scientists, data analysts, and data engineers have emerged as important actors for managing big data. As with most activities, a good start is half the battle, and in market research it starts with the planning and preparation stage. The planning stage is the most underrated part where most of the improvements are to be gained. In this blog post we are going to provide a general overview of the characteristics (skills, other requirements) of the main phases of primary research methods for both Business to Business and Business to Consumer research. Special attention goes out to explaining which Business to Business research methods are most appropriate under which industry or product technology conditions, and a special section is dedicated to Business to Consumer testing studies by J. Kavuma.

Interviewing Method

The goal of research interviews is to adopt a qualitative structural approach to collecting business analysis data. You are going to gather business information regarding a concrete field to be assessed. As such you need to be certain that you are selecting the appropriate industries as well as industry insiders to interview depending on the goal of your interview. The research analyst or business analyst will formulate questions and save the feedbacks from each of the interviewees.

Drivers of successful research interviews

There are a series of elements to take into consideration when doing research interviews:

Awareness – This applies not only regarding the participant in the interview and their know-how about the subject matter and competencies about the domain to be assessed, but also your understanding of their interests, their predispositions and attitudes, and the obstacles that they need to overcome in their daily work.
Track record – Experienced research interviewers with an established track record guide the interview while at the same time granting enough space to the participant to communicate their ideas.
Competence – The skill of the research analyst and of the participant are important to consider. If it’s face-to-face, the reply to each question should be recorded as soon as the information is given. If necessary, the interviewer should pause for recording accurately what has been said, even if this means slowing the interview down. Repeating a respondent’s answers can slow the respondent down, thereby helping the interviewer to record the answers properly. Another important point is that answers should never be interpreted or condensed.
Compliance – Frequently participants are hesitant to express their views because of not entirely knowing the consequences to them. It's relevant for the research analyst to provide clarity regarding the end goal of the interview and reassure their answer’s anonymity.
Transparency – If the goal of the research interview and the questions are known, the interviewee can share their viewpoints in a clear manner.
Rapport – Building rapport with your participant interviewee at the start of the interview will aid in helping them feel at ease, which builds trust, and leads to the interviewee to be more willing to share information.
Confidentiality/security – As market research agencies work for a number of clients, each study should be treated with complete confidentiality and should never be discussed, particularly not with any clients one could be in contact with. Further, the work should not be discussed with friends or family. Survey material should not be left lying around for anyone to read.

Preparing for effective research interviews

There are a couple of steps to take when planning for research interviews:

Goal setting – Identifying your research purpose and the way you will handle a discussion that moves into unanticipated directions.
Determining research participants – Make sure that the information that you are attempting to obtain from them is appropriate. For instance, it’s irrelevant to recruit participants for an FMCG product, when the subject matter is social.
Research design – Fashion research questions based on the purpose and goals that you have established and ensure that those queries are not dubious or inappropriate. Also ensure that the research questioning process goes smooth so that the research participant's viewpoints can be received as well as research data can be collected through a logical sequence.
Logistical planning – What is going to be the interview location? What are the circumstances under which you are going to conduct the interview? How are you going to craft your questions in a way that will compensate for the lack of physical interactions and facial expressions in case of a phone interview?
Local Language Translations-The English questionnaire and the stimulus material are then translated into the local language at the respective fieldwork office. The local language questionnaire, layout, and structure are aligned exactly with the English questionnaire in order to avoid any errors. The translated questionnaire and stimulus material are then checked by another person (i.e. other than the translator) to eliminate any errors. All local language questionnaires are stored both electronically (hard disk and back up diskette) and in the library with a hard copy, to be used as guidelines for future studies. The local language questionnaire is sent to the executive for approval. If pilots have been conducted, the executive makes the necessary changes in the English questionnaire and informs the local office. The local office then changes the local language questionnaire accordingly.
Pilots-In case of complicated, lengthy questionnaires, the local Field manager/Field Coordinator is responsible for conducting a pilot. The pilot is aimed at identifying any problems referring to the length, logic, comprehension, and flow of the questionnaire.

High performance research interview process

To have an effective research interview progress, take certain actions at the start of the research interview:

Explain the goal of the research interview, the reason for interviewing them, your own background, and what your responsibilities are within your position
You might describe how your role as a research analyst is not to make forward looking decisions, but to identify the needs and requirements.
As part of ethical considerations, if applicable, you might explain to them that the participant’s answers will be anonymized by not associating their information to the individual, regardless of the negative or positive nature of the information.
Describe the trajectory of the interviewing process and what is required from the interviewee.

The following steps during the research interview

Preserve focus and adjust if the interviewee goes off track. You will have to improvise and act fast to capitalize on any extra information thrown your way unexpectedly.
Keep track of the level of motivation the interviewee has. You can do that by being aware of their body language, and if at any point doubt is cast on the extent to which the participant has sufficient knowledge of the subject at hand, ask who the appropriate person might be to speak to instead. Deal with possible concerns from the side of the interviewee as best as you can.
Proactively listen by maintaining frequent eye contact and rephrasing their answers to get validation about your level of understanding. An interviewer, who gives full attention to the respondent, will clearly show the respondent that every opinion is important, and that his/her help is appreciated. The respondent should not feel, at any time, that the interviewer has a different opinion on the topic as him/her (by the displaying of astonishment, shaking the head, etc.).
Document the flow of information and repeat your notes to double-check whether you have understood their answers properly. The questions should not be read too fast or too slowly; also the interviewer should pause, and give the respondent sufficient time to reply. If the respondent has misunderstood the question, it should be repeated in its original form. Make sure that the question is never rephrased.
Asking the Questions Exactly as Worded in the Questionnaire. The words of questions should not be changed according to the interviewer’s preference because it is essential that all respondents be asked the same questions in the same way. Research has shown that even small differences in the wording of a question can make very large differences in the answers. Therefore, the exact wording must be used with no paraphrasing at any time. As variations are likely to produce different answers to the original question, only the exact wording of the questions as shown on the questionnaire is to be used, in order to ensure consistency.

For example, the questionnaire might read.

“What brands of shampoo have you used in the past 4 weeks for washing your hair?”

The following variations are possible:

“What brands of shampoos have you used in the past weeks?” (The timing is omitted)

“What products have you used in the past 4 weeks for washing your hair”. (Instead of brands, we now have products).

The closing section of the research interview should consist of the following:

Verify whether you have covered all areas that needed to be discussed
Share your contact information in case the interviewee is inspired to provide more information afterwards
Compile a summary of your research findings and inform about the data assessment step that follows.
Inform the interviewee that you might follow-up about more information after the research interview has been completed. If it is necessary to arrange a recall interview (e.g., placement/recall survey) it is advisable to arrange for a time range rather than a specific time. For example, rather than scheduling a recall for 6:00 p.m. one should suggest a time between 5:00 to 7:00 p.m. This protects the interviewer if he/she is delayed. Once an appointment is made it should not be broken unless alternative arrangements can be easily made in advance of the appointment. It is always useful to make a telephone call (where possible) a day or two before the appointment to reconfirm it.

The research interview process ends with the next follow-up steps:

Structure the information in such a way that it enables you to cross-tabulate it with other interviews to identify disparities or discrepancies.
Share the collected data with the research participant to verify the research findings once more.

Research Interview considerations

Research Interviews have a set of benefits:

Research interviews promote engagement if you're a competent facilitator and can build rapport from the beginning of the session.
Research interviews are straightforward and allows real time adaptability due to its interactivity.
They enable grasping of the contextual explanations surrounding the information that you have collected.
They allow you to monitor the interviewee’s body language, behavior, and motivation to cooperate.
Research interviews enable questions post-interview
Research interviews enable the interviewee to show their ideas and sentiments about a specific field under assessment

Research Interviews also have several deficiencies:

They need considerable planning, in terms of preparation of the queries, making sure that you are moving towards achieving your goal, the required effort to recruit the participants etc
Research interviews need a significant dedication from both the research analyst and the interviewee
Training is needed, and therefore it may be sensible to get education on how to conduct successful research interviews to collect business analysis intelligence
The participant’s information may be misinterpreted subjectively by the research analyst
As opposed to for example surveys, closed questions aren’t going to be as effective as open questions during research interviews, in order to elicit as much objective and well-rounded perspectives from the participant as possible.

Observation method

There are certain technologies for which strict standards need to uphold in terms of securing their confidential data, intellectual property etc due to the high-degree of innovation, high strategic stakes involved or outright national security concerns. Or there may be industries that involve complex decision making, slow pace of innovation, low degree of standardization, and complex supply chains. Examples are direct business to consumer sectors or industrial production environments. In such cases, provided that you gain the trust of the participants, close observation of the processes and systems can yield the most useful insights in the most effective way.

Purpose of observation as a research method

The power of research observation is based on the following points:

It’s an approach to obtain circumstance-based information within the natural context of the person’s daily work. The major observational methods are mystery shopping research, electronic observation and audit.

It can aid in detecting the difference between valuable causal variables or correlation variables and less useful association or coincidental variables.
It can aid in learning more about the end-user experience of the application of your product, service, process, system etc, offering opportunities for improvement
Observation research allow the research or business analyst to document unorthodox behavior of relevant parties who tend to operate under the radar
Observation research as a real-life research method weeds out potential “noise” and is therefore sufficiently reliable to be called upon when making critical decisions, for example regarding setting performance parameters or making financial decisions. It is also helpful to make decisive decisions when confronted with unresolved dilemma’s born out of alternative research methods.
It helps in making critical evaluations about the functioning of the processes or for troubleshooting when parts, components or systems fail when used by the person who is observed
Observation research also supports the development of training programs, offering you a sense of how the person observes is performing their work and the obstacles they face

The various styles of observation research

There are primarily two types of observation research:

Active – An active type is characterized by an involvement of the research analyst with the end-user’s activities to answer pre-determined how and why questions about the use of the system.
Passive – The passive approach is for when the research analyst doesn’t have pre-determined questions to answer, and they are just accompanying the person’s daily work routines to take notes with an open mind. After the observation, the research analyst will ask the end-user a set of descriptive what questions about the tasks they were involved with

Research Observation objectives and preparation

There are a couple of aspects that you need to take into consideration when planning for observation

Goal setting – When you are devising your goal, you ought to think about what you are trying to succeed in
Preparation of the approach – The following step is to prepare for the approach to make sure that the observation progresses as natural as possible without your presence affecting the normal behavior of the end-user. This is important because you don’t want your presence to create a distorted picture of reality.

Flow of the observation session

A couple of things need to take place before the session:

Provide reassurance that you are not doing your research to judge their performance, but rather to identify the obstacles of the end-user to come up with solutions.
Describe to the end-user how the information will be processed after the observation is completed.

The various actions to be taken during the observation:

Observe the actions of the end-user from as many perspectives as possible
Don’t forget to take quick notes while you are doing the actual observation
Pose questions in case you are using an active approach

The steps to be taken post-session

Audit and verify your data; A technique to raise the quality of your data is to observe more than one person and in multiple locations to identify potential inconsistencies, especially when observing within industries characterized by high complexity and decentralization of information (see figure 2)
Search for similar patterns, deviations and trends which may help to identify windows of opportunity for changes and transformations.
Structure, assess and disseminate the information as deem fit.

Research Observation considerations

The strengths to benefit from can be summed up as follows:

The value derived from the practical industrial insights is very high. These insights have the potential to take away possible misguided assumptions that you may have about the way they do their work.

Research observation is useful to find out about informal processes that end-users have constructed to manage some of the deficiencies of the system that they mostly have to rely on.
It enables you to measure and compare performance- as well as social related indicators such as productivity from one end-user to the other or environmental policies that are still in the development stages
Research observation can be an excellent way to cross-reference and validate research finding from interview and surveys, especially as a follow-up step.
It works especially well in consumption settings or industrial production environments characterized by complex decision making trajectories such as purchasing behavior, process engineering, manufacturing.

Some of the cons of using observation as a research method are:

Research observation can demand much attention and energy from the end-users, so discuss pre-cautionary measures with the management to minimize disruption and distraction to the end-user workflow as much as possible.
Make sure that the end-user understand how you are going to use the data and for what specific end-purpose.

As already mentioned before, the risk of research observation is that data accuracy about the observation will decrease because of the act of observation.
It is not the best positioned research method within the more knowledge-based work environments where either a high degree of ICT, automation is involved or where strict EHS (Environmental, Health, Safety) practices need to be adhered to for example in semiconductor fabrication or data security processes. In these cases, specialized third party business intelligence or business information providers may be better positioned to deliver the needed research and provide the require intelligence. e

Surveying or Questionnaire method

Planning Questionnaires and Surveys like a pro

The goal of surveys and questionnaires is to gather data about business information from the stakeholders, be it about their business or opinions. It represents a systematic technique using a set of questions to make inferences about your field of study or to verify whether you are on the right track with achieving your goals. Questionnaires should not be very time consuming for the participant. This will increase the willingness of the participants to contribute and complete the survey. Surveys and questionnaires are useful research methods for a variety of industries and are therefore one of the most adopted research methods.

Figure 1. Sample Telephone Q&A Interview session. Source: Inhouse Data Portal EngineerOnomics [LÖGER Group]

Research Question types

There are a few main types of research questions that you’ll use in a survey:

Most of the questions that you will use in a survey are:

Closed ended – Most of your questions will be of this closed nature. You will provide the participants the chance to select from a set of answers, rate something a range or scale, or provide a Yes or No answer.

Open ended – This kind of questions offer the respondent the chance to offer their viewpoints in a free exchange of information. Applying these types of questions should be done occasionally because it’s usually harder to assess the data that has been retrieved from the respondent.

Planning for successful research surveys and questionnaires

There are series of steps to be taken when planning for conducting surveys and questionnaires.

Determine your goal – Define why the survey is being prepared in the first place. You might establish that a survey is the most effective way to obtain opinions on the current state of affairs in a particular industry, regarding specific technologies or other forms of applications

Identify a target group – Who ought to become the respondents why should they contribute? It may be a group of end-consumers who have been using a new system that should be in conformity to your pre-determined goal.

Select the type – Select the type of survey and the kind of questions that you want to ask so that you can get the max benefit from the process. Variants of surveys are panel sampling, telephone surveys, online surveys, focus groups etc.

Establish the sample group size timeline – You need to establish a response rate which represents the amount of people you need a response from out of the sample group. An insufficient number of respondents will compromise the representatives, quality, and validity of your data, for example because it tends to introduce the likelihood of bias.

Choose the research distribution and collection methods – This ask of you that you establish how you will allocate the surveys logistically, what infrastructure to use to gather the information.

Qualify Recruits – Do a pre-screening of the recruited respondents to make sure that you weed out potential participants to that do not fulfil the criteria of your target group. You can use a pre-screening form for that.

Select the sample – Select your sample from the list of respondents and ensure that it meets your established target levels to obtain sufficient respondents providing the information and data.

Establish beforehand whether you are going to complement with interviews (afterwards) – After the survey is completed, ensure that you gather contact information in the survey for follow-up contact.

Plan the research questions attentively – The question needs to be as neutral, clear-cut and flow as logically as possible. It is better to avoid framing questions in a way that influences the outcome beforehand or questions that are confusing to the participant.

Test – It make a lot of sense to test your survey with a least five people, getting their feedback on the user-friendliness of especially the substance of the questions, and why they answered the way they did. This will make sure that the data collected is not inconsistent and clear-cut.

Distribution and project management

This can be done in person – you could use this in a workshop if you wish – or by sending the survey via e-mail or other online tools, such as SurveyMonkey. You can also use your in-house data collection portal and project management material such as a tracker to track how the different stages evolve from recruiting to data collection. Subsequently make appointments and monitor progress.

There are some factors to take into consideration with regards to research logistics

Urgency – How immediate is the need to access the data?
Security – Can any of the data collected become a concern in terms of security issues?
Location – In person surveying requires due diligence of the location that your study covers as well as the specific geographical area of your participants including any cultural consideration that you need to take into account.

Structuring questionnaire and survey results

The steps to be adopted after the survey is completed are:

Collate the data.
Summarize the research results.
Review and evaluate the research findings and assess what value you can derive out of the data. The utilization of data visualization such as graphs and charts are very useful tools to document the research results.
Convert, codify, and categorize the information to learn about possible consistent patterns in the answers of the respondents
Unpack the data even further by narrowing down the information to core essential insights, and more detailed granular data.

Survey and questionnaire considerations

Strengths of using surveys and questionnaires:

They have relatively lower costs
They can be employed relatively fast
Research surveys generate vast amounts of data that is useful when seeking experiences, knowledge, and opinions of others
When your survey is brief and concise, the respondent can finish it fast, and your target numbers will rise
They make sense from a geographical point of view as you can gather data from a sizeable group of respondents spanning a wide geography
Research surveys enable you to quantify qualitative data as well as to directly collect quantitative data, especially when you are using a Likert scale or a 1-5 range scale.
They yield breakthrough insights and commercially invaluable data

Cons to using surveys and questionnaires:

They need a person with special skills in designing the appropriate questions and avoiding skewed outcomes
They may occasionally result in a low response rate, and you might need to re-issue the survey
Open-ended questions require more extensive analysis which can be more time consuming
Research surveys and questionnaires may harbor dubious questions, so they required diligent preparation to minimize the chances of respondents understanding questions differently than you envisioned
Research surveys and questionnaires frequently necessitate some kind of follow-up action, which may demand more time from you.

Figure 2. Matching research and learning methods with the subject industry.

This infographic represents a taxonomy that serves to simplify the choice of the appropriate research method based on the concept of “fitness” or match along two dimensions: the learning requirements of your industry as well as the strengths of a particular research method. The learning requirements of your industry is based on mainly seven (7) techno-scientific characteristics that determine the eventual extent of fitness: Degree of uncertainty/predictability; degree of standardization/specificity; degree of complexity/linearity; degree of interdependency/modularity; degree of global/local learning; degree of tacit/codifiable learning; degree of EHS or privacy standards. These industry learning dimensions correspond with a set of dimensions associated with research methods: Degree of interactivity, contextuality, quantifiability, quality, quantity, accuracy, capacity. The industry typologies have to be seen as ‘ideal’ scenarios and the boundaries between the resultant research methods as fluid. For example, although the infographic has positioned the research method interviewing as mostly fitting Type IV industries, it does not preclude the possibility that interviewing could be an effective method for other industry types under equally matching circumstances.

Types of direct Business to Consumer Studies Usually Conducted

Benefit Ranking Study

A technique to assess which of the various potential benefits, that could be provided by a product, would be most/least attractive/motivating to the consumer. Used in concept development to identify which benefit should be the focus of the concept.

Blind Product Tests

Blind product tests for certain product categories e.g. soft drinks, chewing gum are frequently conducted at central locations, whereas for most household and personal care products, which require a longer period of usage – e.g. washing powders, household cleaners, personal cosmetics etc. – are the products tested by respondents at home. In the latter case, interviewers will conduct a short interview at the placement stage to establish eligibility and to explain the purpose of the study. This will be followed-up by a recall interview on completion of the specified usage period.

The products / brands are usually presented to the consumers without their names, i.e., “blind’ (as this avoids a bias caused by pre-conceived ideas about the product / brand being tested).

We conduct both single product tests (where all respondents test one product only) and parallel placement product tests (where different groups of respondents each test different products).

As in all blind product tests, the purpose of a Parallel Placement Blind Product Test (PPBT), is to assess the consumer’s acceptance of a product after a minimum usage period under realistic home conditions and without the influence of any marketing mix. The product therefore is produced in a plain white pack, without a name but with usage instructions and content declaration, and placed with consumers who can judge it based on past experience. Respondents are interviewed face-to-face in their homes.

When two or more products are tested, each product must be tested by a different group of respondents. At each starting point, an equal number of interviews are conducted per product. Each interviewer must conduct an equal number of interviews for each product in order to avoid any area bias or interviewer bias. There must be equal number of interviews / questionnaires conducted daily and each product has to be placed in the specified sequence.

When the interviewers place the product, they must ensure that the product matches the product code specified in the questionnaire. If the respondent refuses to use a product, he/she is not offered another product and such respondents are normal “losses”. Interviewers need to find respondents who accept specific products before they are allowed to place another.

Brand Equity Monitor

This type of study is conducted annually (or bi-annually) for tracking purposes. The aim of such study is to establish and monitor the usage, strengths and weaknesses of key brands.

Business-to-Business Surveys

These types of surveys are usually conducted for clients whose main interest is in selling products and services to other businesses. Consequently, they are interested in the perceptions of businesses towards the products and services offered rather than those of end-consumers.

Business research thus differs from consumer research because:

The respondent is interviewed as a representative of his/her company and not as an individual
Interviews are almost invariably done in the respondent’s workplace
The universe sizes are much smaller
Creating a representative sample is harder.

Brand Price Trade Off Studies

The Brand Price Trade Off study (BPTO) is an adaptation of the conjoint approach for pricing studies. The underlying assumption of the BPTO is that there are two factors that influence the purchase decision – the brand (representing a summarized version of all variables other than price) and its price. The BPTO measures price sensitivity of brands in a competitive context. Therefore, the first stage in a BPTO exercise is to identify the competing brands which constitute the ‘market’ and the range of prices that need to be tested. All respondents recruited for the study are presented with photographs or packs of brands representing the competitive set. All brands in the competitive set are presented at the lowest price for each brand. The respondent is then questioned as follows:

“Suppose you were to visit your usual shop and find these brands of ____________ (MENTION PRODUCT) at the prices shown on this card, which of these brands would you be most likely to buy?”

The price of the selected brand is increased by one level and the respondent is asked to make a second choice. This process is repeated until the respondent refuses to make any purchase at the prices quoted or when the maximum price to be tested has been reached.

Concept Tests

The purpose of these tests is to identify the concept, which most effectively communicates the benefits of a product. Usually therefore, more than one concept is tested at the same time in parallel legs. Occasionally, however, each respondent can be asked to look at more than one concept and to give his/her overall preference. The information resulting from the concept test is then used to develop the final advertising campaign. The main selling line/message of the product/service being tested should be included in the heading of the concept. Other information that must be included in the concept is the brand name, intended benefits of the product; why/how the product/service will provide these benefits, and information about the price/size/variants available.

These tests are carried out either door-to-door or in central locations amongst respondents who use products within the specified product category. The respondent is asked to read the concept carefully and is then asked questions such as purchase intent, likes/ dislikes etc.

When two or more concepts are tested, each concept is usually tested by a different group of respondents. Therefore, when using the door-to-door method, an equal number of interviews must be conducted per concept at each starting area.

An equal number of interviews must be conducted by each interviewer for each concept so as to prevent any interviewer bias creeping in. The daily number of interviews/questionnaires conducted must also be the same while a consistent sequence shall be upheld for each concept.

The interviewers must ensure that the concept matches the concept specified in the questionnaire.

Concept and Use Tests

The purpose of such tests is to test how interested consumers are in trying a product, to identify potential purchasers, why they are interested in the product and, finally, whether it fulfils the expectations set by the concept. It is often the final step before test-marketing the product. Again, the method is usually face-to-face interviews conducted at the respondent’s home. However, at times the concept stage can be carried out at a central location followed by the post-usage interview at the respondent’s home.

When two or more products are tested, each product/concept must be tested by a different group of respondents. In case of a door-to-door study, an equal number of interviews are conducted per product at each starting point. Also in this case, an equal amount of interviews/questionnaires should be conducted on a daily basis with every product positioned in a pre-determined sequence. The interviewer ought to do the same number of interviews with respect to individual products or concepts to prevent interviewer bias.

Variations to the above studies include Concept and Look tests whereby respondents are shown the product / pack and Concept and Sniff tests used for the fragrance category of products.

Diary Panels

This form of research is based on a fixed sample of households rather than stores. A representative sample of homes is recruited throughout the country and they are asked to fill in a diary on a weekly basis giving details of all the daily purchases they have made during that week. The diary idea is used in order to reduce errors resulting from poor memory and confusion.

Early Brand Evaluation Study

The purpose of these studies is to assess awareness, trial, and purchase intent for new brands/products/line extensions. It is usually conducted in the early stages of the product launch and in several waves in order to track the progress of a brand. The interviews for such tests are conducted under normal in-home conditions.

Habits and Practices Study

A very comprehensive study to understand all aspects of consumer behavior in an area of interest. It may include the placement of a diary, in which respondents are asked to record their habits/practices/product usage in detail for a limited period.

Hierarchy of Needs Study

A technique to determine the important unmet needs of a particular category across geographies. Respondents are shown 100 attributes and asked to rate them on both importance and how well current brands meet each attribute.

Identified Test

This is a variation of a product test used when it is necessary to identify the brand being tested (e.g., when the client wishes to assess the effectiveness of the brand name or whether the package / bottle is easily identifiable).

Mystery Shopping /Customer

Mystery Shopping research is a technique of quality assessment in sectors such as retail and services. It is a tool for assessing the quality of both goods and service provision. Companies use this method for self –assessment and / or to benchmark against the competition.

Name and Pack Tests

The aim of these studies is to test packaging/name options for various product categories. These studies are usually carried out in a central location.

Omnibus Studies

These are syndicated studies run at regular intervals (for e.g. 3/6/12 months) and based on a fixed representative national sample of adults. Companies can participate in one or more waves of the omnibus. There are, however, fixed limitations regarding the sample size/timing/processing of tabulations.

Pre-/Post-studies

This type of study can be applied to any study (advertising evaluation, promotional, etc.) as exactly the same questionnaire is administered via the same methodology to a comparable sample of respondents before (pre) and after (post) a particular event for measuring a change affected by that same event. For e.g., pre- and post-advertising effectiveness studies will test the effectiveness of a specific advertising campaign.

Ranking Study

A technique where respondents are asked to give their order of preference for the available options overall, and/or evaluating against particular attributes or characteristics.

Round Robin Test

A form of paired comparison test of more than two products using the model A vs. B, B vs. C, C vs. A. It is useful for early screening as the scores are aggregated and therefore require only a small number of respondents for each pair evaluation.

Sampling Effectiveness Study

A study run to assess the effectiveness of a sampling wave in generating usage and purchase. It compares a test sample (or leg) versus a control leg.

Sequential Monadic Test

Two products are placed with the same respondent, but at different times in this product test.

Shopping Habits and Store Type Acceptance Study

Respondents are asked about their shopping habits, but also to indicate the importance they attach to particular attributes of a store.

TV Copy Testing

The focus of such tests is to find out spontaneous reactions of consumers to proposed advertising for a brand (pre-launch). These tests are normally conducted in a central location.

Syndicated Research

This research usually involves the collection of information about sales, purchasing, distribution etc., of a wide range of products on a regular and a continuous basis and then the sale of this information to any client who wishes to buy it, usually on an annual subscription basis.

It also covers some branded products such as the Sales Force Image Study (SFIS) / Consumer Brand Profile Study whereby the same information / data can be provided to different clients (although some of the data can be tailored to each client’s specific requirements).

Taste Test

Allows a respondent to try the product and evaluate its characteristics. Most commonly done for products such as chewing gum and soft drinks, these are also conducted in a central location.

Usage and Attitude Studies

These are full-scale market studies to measure the awareness, usage, purchase/ repurchase intent, attitude, advertising awareness and recall for specific products / brands. The interviews for such studies are conducted under normal in-home conditions. Over-quota interviews are often required to achieve a minimum base size (normally 100) for attitude questions regarding a specific brand.

Tracking Studies

These studies are done on a regular basis (for e.g., every 3/6/12 months) to track consumer usage and behavior vis-à-vis a product category. Interviewing for such studies is done door to door. The Tracking Study for Wrigley’s are good examples of such studies.

Business Intelligence and Estimation Methods

The goal of estimation is to identify the expenses and extent of energy required to run an organization. The research and business analyst and other stakeholders will predict the costs of implementing and upholding a solution and identify what value the solution brings to the organization. The research analyst and stakeholders will also predict the potential of risk impacts. Business Intelligence based forecasting techniques are most used in commodity type vertically integrated industries where the high-degree of commercial information and strategic planning capabilities available inside the organizational boundaries is often outpaced by erratic and dynamic supply-demand developments occurring in the external business landscape, accompanies by great financial costs.

The various existing methods available for making estimations:

Top down – This method delineates the solution in parts or a hierarchy, and predicts each on the highest level
Bottom up – This method unpacks the lowest level components of a hierarchical delineation to predict the costs and extent of effort required for specifying and summarizing each of these elements, so that an overall estimate can be determined.

Parametric estimation. This method utilizes the relationship between variables to calculate the cost or duration. This estimation method establishes the unit cost or duration and the amount of units needed for the projects, for example 25 hours of development per hour of in-class training.

Rough order of magnitude – This method is useful in case of a limited amount of information and is reported at a general high level. It is suitable in the early stages of an initiative to identify the duration or costs.
Rolling wave – This method applies repeated estimates throughout the activity. The predictions are directed towards the next stages of activities. For instance, you may predict that it takes around 10 hours to make a prototype and in the next wave of work you may re-evaluate the actual figures against the prediction and adjust your estimation.
Delphi – This method applies several rounds of predictions among subject matter experts. It uses historical data to generate the estimations. These data may derive from other project and know-how to establish the prediction for each stage until there is common agreement on the final numbers.
PERT – This method applies three predictions: the ideal scenario, the worst case scenario, and the most likely value. To be more precise, the three are calculated using the weighted average. You accumulate the optimistic and pessimistic values, on top of which you add the most likely value, which is then multiplies by four, finally dividing the number by six to obtain the average value.

Confidence intervals and prediction accuracy

To increase the accuracy about an estimated effect value, the raw estimate technique is calculated as a ratio of the width of confidence interval compared to its primary value and then expressed as a relative value or percentage. The rough order of magnitude estimate will likely be approximately 50% accurate. A final prediction uses more data and information to estimate and tends to have a 10% reliability when set against the actual figures. One practice is to use rolling wave estimates and compare these estimates to the actual values for ensuing stages of the project. A rolling wave estimate will offer more definitive estimates for each stage of work. It is important to do data collection of the real empirical values to keep continuously improve your estimation outcomes.

Precision versus reliability

There is a difference between precision and reliability:

Precision – With confidence levels, predictions are frequently delivered within a scale of values for a specific part of work. A confidence interval is expressed as a percentage, for instance an estimate of 60 hours with a confidence level of 90%. The estimate in this case would be in the range of in between 54 and 66 hours. When various techniques are applies, the variance between them, or standard deviation, can be applied to achieve a conclusion on the prediction number.
Reliability – Reliability essentially implies that the research results will be replicated when the same research is repeated with the same variation of estimates.

Estimate confidence

There are a couple of points to take into consideration about estimate confidence:

Team predictions have proven to be more accurate compared to individual estimates. Every team member contributes information and know-how to the estimation exercise, which acts as a corrective mechanism against subjectivity.

In case an organization needs to have more certainty about their confidence in a prediction, they may hire an external independent party to conduct or evaluate the estimate of the internal team. Specialized market research companies are well-positioned to have your estimation verified independently and objectively.
The organization can contrast the independent estimate with the internal prediction to figure out whether corrections need to be made.

Estimation considerations

Estimation as research or learning method has a couple of pros:

Estimates are advantageous to validate project management related elements such as budgeting, time frame, size of the parts being predicted, and the work schedule.
Comparing the actual values with the estimates creates excellent opportunities for learning and improving the predictions, with better estimate positively associated with a higher degree of team knowledge and experience / with estimate quality increasing along with the increase of team knowledge and experience
it helps to ensure success by updating various estimation methods

Cons of estimation as a research method are:

Sophisticated forms of now-how and experience are needed within a team that have a solid background of having jointly worked closely together in projects and on the same tasks
The use of estimation methods can lead to oversimplifications, which in some cases can be prevented by complementing it with additional research methods.