#max diff analysis

RELEVANT MAX-DIFF

 MaxDiff is a survey research approach for obtaining preference/importance scores for multiple items (brand preferences, brand images, product features, advertising claims, etc.).

 Sometimes researchers want to show only the MaxDiff items that are relevant to a given respondent.  

 Relevant max diff analysis let us to show a customized and dynamic list for each respondent based on a variety of conditions and logic.  This means  we can customize the MaxDiff items (potentially a different number of items per respondent) to show respondents based on relevance, awareness, importance, preference, etc.

 Example link:

 https://knowledgeexcelsurvey.com/relevantMaxdiff/cgi-bin/ciwweb.pl?studyname=relevantMaxdiff

 Analysis can involve logit, latent class, or HB and you need to specify in the analysis interface how to treat missing items for the respondent.  There are three methods of treating missing items:

 1.Missing items are missing at random and inferred from the population information (default method),

2.Missing items are inferior to included items,

3.Missing items are unavailable and should have near-zero probability of choice for a respondent (we insert a large negative utility such as -20 for the missing utility).

 All three methods lead to a full set of utility scores for all the items (for the group, each latent class, or the respondent if using HB estimation).  If using HB estimation, you can also export the scores with missing items per respondent replaced with blanks, such that the missing items have missing utility scores.

 There are two ways to create the combinations of items to show respondents (the design) for MaxDiff questionnaires: Pre-generated and On-the-fly.

On-The-Fly Design Generation

On-the-fly (during survey) MaxDiff designs contain the combinations of items to show each respondent and are generated in the moment that the respondent takes the survey.  On-the-fly designs are required when the number of items in the constructed (dynamic) MaxDiff list is different across respondents for the same MaxDiff exercise.

We should take special care that respondents always get a reasonable number of items in their consideration list.  If too many items are included, each item might not be shown enough times for stable HB estimation.  If too few items are included, there may be too few items to populate even one MaxDiff question and the entire MaxDiff exercise would be skipped for the respondent. We should ensure that our consideration set has a minimum number of items per respondent.

Researchers also recommend to add a few items to the consideration set that are not relevant/important to the respondent.  However, given the method we use for augmenting each respondent's choice data with full information when missing items are treated as inferior, this may not be necessary

 Pre-Generated Designs

When the number of items to show each respondent is the same, we can generate the combinations of items to show respondents prior to data collection. This is called a Pre-Generated design. The default is to generate 300 versions (blocks) of the design prior to fielding and respondents are assigned to different versions (often times with multiple respondents receiving the same versions, which is just fine from a statistical standpoint).  

There are at least two MaxDiff approaches that involve customizing a subset of items to show each respondent that could involve pre-generated designs, because the subset of items always has the same constructed list length: Express MaxDiff and Bandit MaxDiff.  For example, 30 items could be drawn from a master list of 100 items for each respondent to evaluate, so a pre-generated design accommodating 30 item slots could be employed.  We should note, however, that prohibitions may not be specified if using pre-generated designs for Express MaxDiff or Bandit MaxDiff.  Rather, on-the-fly design may be used with Bandit and Express MaxDiff for implementing prohibitions between items.