Why is low sample mortality critical in HUTs (Home Use Tests)?

Extra placements are the worst possible cure for high sample mortality. Here’s why.

Placement tests have challenges in many countries due to high levels of sample mortality: loss of participants between placement and completion.

Developing markets tend to have more respondents who forget or misunderstand the instructions for product usage or drop out for countless reasons. This compromises research in every aspect imaginable:

TECHNICAL

> Very different sizes of demographic sub-cell quotas

> Some insufficient cell sizes for analysis

> Unequal bases for statistical comparison of product performances

> Difficulty of obtaining return of partially tested products for laboratory analysis

FINANCIAL

> Wasted research budget with an excessive number of placements and attempted callbacks

> Wasted production budget for the expensive test products and materials placed

SECURITY

> Heightened risk of losing project confidentiality when products stay in the homes

TIMING

> Longer fieldwork period that delays results and management action

WORRISOME FINDINGS

Some international clients who ran numerous HUTs actually requested over-sample as high as 100%, expecting a 50% loss during the test, based on their previous experiences in developing markets.

Anyone experienced in product testing can recall projects that exceeded field deadlines and required a rushed shipment of extra test products because of wasted placements and overlooked or poorly controlled quotas. The usual solution is to place products in many more homes initially,…a mediocre solution at best.

THIS IS THE RESEARCHER’S BUSINESS CHALLENGE: Should we agree with the client, or suggest ways to do it better, risking friction?

CASE STUDY

A very traditional global CPG manufacturer recently requested 2250 placements to obtain 1250 completed interviews after two phases of week-long product use with three in-home face-to-face interviews. Yes, all three interviews expensively face-to-face (!)

Even though the test had demographic rotations of 11 products, we felt certain a much smaller placement would suffice, based on our 40 years of product testing experience.

For additional economy, Abaco proposed that callbacks be done by telephone, after establishing a solid eye-to-eye relationship with the respondents during placement.

In Brazil they say “Gato escaldado tem medo de água fria” A scalded cat is afraid of cold water.

The previously scalded client did not authorize use of telephone in any phase: it must all be face-to-face. Nevertheless we promised good results with fewer placements, despite risking some discomfort at client headquarters.

OUTCOME AND LESSON LEARNEDagain

After compromising on an initial 1670 placements, a full third fewer expensive placements than originally requested, Abaco maintained its customary, proven field methods and quality controls. We achieved a 85% completion rate with very high matching among 90 sub-cells, showing that only 1450 placements would have sufficed.

This client could have economized more than 750 home visits and 220 sets of costly test products from the agreed sample. The lesson here is embarrassingly obvious: emphasize field quality to minimize waste.

The simple, logical solution is to invest the maximum in pre-planning, training, dedicated field effort, and strict controls. With a higher CPI but smaller samples, the payout is substantial overall economy and much higher data quality.

STILL THE DOUBT REMAINS:

Would we have a more or less satisfied and loyal client if we had simply obeyed that initial request?  Only time will tell.

– Alan S. Grabowsky, President

Leave a Reply

Your email address will not be published. Required fields are marked *