Posts Tagged ‘Online Sample’

So when are we dumber than a guppy?

The answer is simple:  when we neglect to collect the information we require for our own survival.

When we fail to gather information about our sample sources.

On a typical project we’re operating at a deficit of information.  We are too willing to accept sample sources’ black box mentality, their unwillingness to share information critical to our ability to assess them.

When a sample source dries up as its respondents get burned out, we either allow those sources to backfill with unnamed partner sources, or we switch to another source entirely.  We are focused on reaching our desired number of completes and keeping costs down rather than underlying sample consistency.

The real risk with sample inconsistency is shouldered by the end users of research, not the designers and executors.  It is they who should be the most alarmed.

Ron Gailey, formerly with Washington Mutual and now with Coca Cola, released data in 2008 that illustrates the pain that can be imposed on client companies.  A superb researcher, Ron collected 40,000 interviews for 29 studies, only to find out that demand for WaMu’s financial products had suffered a meltdown in his surveys.  It was not the kind of data that a market researcher wants to bring to management. Especially when it is unsupported by reality.

Ron tore into the data to find out what was going on.  With the help of a high quality panel source, he discovered the panel tenure of respondents driving the survey results was increasing.  Longer term respondents were
more conservative about purchasing than were new ones.

After all that research, Ron had no choice but to pray that business decisions that had relied on the data would not hurt WaMu.  A total of 29 studies were affected.  The scope of the damage was unknowable.

In order to optimize, we must know more about our sample sources.

At Mktg, Inc. our Sample Source Auditors division collects data on panels through various optimization techniques.  We also audit the consistency of panels over time and/or the compatibility of panels around the world.

We grew up measuring all samples against the census and adhering to a strict standard of probabilistic sampling.  Those days are over.

Now our samples are increasingly different from the census, and we’re operating with a lack of information.  If the data we produce is the product of inconsistent sample, then we are advising our clients to take unmeasured risks.

We are out there without a probabilistic framework.  If we don’t adopt a new model for sampling, we’re going to be someone’s lunch.

The guppy makes its decisions based on potential risks.  If we are exposed to the risk of having our research invalidated due to inconsistent online samples, then we had best begin to track that consistency.

The risks of not knowing are too great.  Ask your suppliers if they have passed the guppy test.  It is better to eat than be eaten.

Like many marketing researchers, I didn’t plan this career path from a young age.  I brought transferrable skills from another field.

As far as I know, I’m the only person in our industry with a doctorate in Ecology.

My background in the “hard” sciences gives me a unique perspective on the research industry.

Ecology is not for the faint of heart.  Nor is business.  Things are born, grow, and get eaten by predators.  All in a day’s work.

The marketing research industry is a pond, and you are a guppy.

Guppies have buddies who never make it to the pet store.  These little guys mean business.   And it’s their approach to business that interests us most here.

Guppies do quite a bit of research in their purchasing decisions.  They forage around looking for a good meal in all the likely places.

Their decisions are much like ours.  In these days of busy schedules we shop where we can optimize our time and energy.

Guppies do the same – they optimize.  While food is abundant and efficiently obtained they limit their travels.  There is a risk for guppies in swimming around:  they could become someone else’s lunch.

When the aquatic supermarket is well stocked guppies will eat as they keep an eye out for their own safety.  In the millions of guppy generations that have preceded this one, this process of information gathering has
become essential to guppy survival.  There is no need to take an excessive risk by wantonly traveling all over the pond when staying put will do.

Guppies are good mathematicians.  They constantly calculate the risk/return of their decisions.  There is no hope in the pond for a guppy that doesn’t have a well designed optimization formula in its head.

Optimization is a fairly complex thought process, but it has been shown that even insects can do it.

Once the formula is in place, then the hard work is gathering the data to keep it up to date.  Our little fish friend constantly monitors food abundance, how much time it takes to gather lunch, the energy it must
expend and the amount of attention it attracts in its foraging.  All this goes into its computer-like mind, and a constant flow of decisions follow.

There wouldn’t be a guppy left on the planet if they couldn’t do it.  They are born knowing it’s a fish eat fish world out there.  Information is king, and they gather and monitor it constantly.

Guppies know optimization isn’t a luxury, it’s a necessity.

When food becomes more scarce, the guppy must forage further and further.  Perhaps swimming through the weeds to the bottom of the pond will serve best.  Our little friends take the information they have gathered about
risk from the cues they glean from the environment.

When it seems prudent down they go, through the water to the bottom where they hope to find their next meal.  Such “switching” behavior isn’t taken lightly.  Who knows what lurks in the darkness beyond the lily pad?

The essence of market research is information gathering and interpretation.  We help our clients optimize decisionmaking in their own ponds.

So why am I telling you this tale?

It’s because our guppy friends have something to teach us.  About survival.

In the marketing research industry, we’re ignoring the most basic, instinctual survival system, a system so simple and basic that even a guppy is doing a better job than we are at it.

We’re ironically overriding our instincts and skills as information gatherers and denying ourselves critical information we need for survival.