Archive for August, 2009

She’s dead.

By now she would have been too old anyway. So I guess the fact that she’s dead isn’t a tragedy.

I loved dinosaurs when I was a kid. I dreamt of discovering one of my own. Secretly, I still hold them in high esteem.

The subject of their demise has fascinated generations. They say an asteroid did it.

Probably did.

I guess we would all have to agree that an asteroid crashing into earth and kicking up a cloud of dust so thick that the sun disappears from sight is the consummate unpredictable event.

Yet there are scientists who have loud, deep voices and speak in planetariums that will tell you the probability of one coming soon.

How soon?

It’s the kind of mystery that kept me up at night. After all if the asteroid was coming there probably was a lot I wanted to experience before I was twelve.

I guess it would be good to know.

Planning for the unexpected is what insurance companies tell you they are all about. But no insurance company could survive the payout.

Maybe that’s why we need a space shuttle.

I guess there’s not much we can do about a catastrophic global disaster – the ultimate unpredictable event.

Market research depends on the reliability of our sample frame. And if the sampling frame is going to change, we must be able to predict that change. Sample must be reliable and predictable.

Consistency includes the concept of measuring change in our sample frame. If we are really good at it, the more data we compile, the better we will be at predicting that change.

No, asteroids do not fit into our marketing plan, but changes in seasons and cyclic shifts in the economy could find a place.

Throughout the history of market research, we have grounded ourselves to Census Bureau statistics. They changed through time, even though the big count was every ten years. If it didn’t change, the founding fathers could have done it back in the early 1800’s and called it a day.

No the census changed, albeit at a crawl, but the changes provided us with trends. Slow change sounds more like a trend. An asteroid making a big bang does not.

We need metrics in our research that measure change in our sampling frame on a more microscopic basis than the old macroscopic shifts we have counted on all these years. No, we will not capture the asteroid stuff, but seasons make sense.

Now that we are in the age of online research, where fast is the guiding philosophy, it is past time to come up with a family of progressive metrics that give us anchors for our research.

If we don’t we will go the way of my favorite dinosaur. It was discovered by Elizabeth Gomani from Southern Methodist University, who calls Madagascar home. That’s where the big guy laid his head down to rest and that is where Elizabeth found him and wrote her doctorate about him.

My favorite dinosaur is in the picture below. The big one on the right. Sweet as a lamb, couldn’t hurt a fly, unless she stepped on one. She was about the size of a bus.

So when are we dumber than a guppy?

The answer is simple:  when we neglect to collect the information we require for our own survival.

When we fail to gather information about our sample sources.

On a typical project we’re operating at a deficit of information.  We are too willing to accept sample sources’ black box mentality, their unwillingness to share information critical to our ability to assess them.

When a sample source dries up as its respondents get burned out, we either allow those sources to backfill with unnamed partner sources, or we switch to another source entirely.  We are focused on reaching our desired number of completes and keeping costs down rather than underlying sample consistency.

The real risk with sample inconsistency is shouldered by the end users of research, not the designers and executors.  It is they who should be the most alarmed.

Ron Gailey, formerly with Washington Mutual and now with Coca Cola, released data in 2008 that illustrates the pain that can be imposed on client companies.  A superb researcher, Ron collected 40,000 interviews for 29 studies, only to find out that demand for WaMu’s financial products had suffered a meltdown in his surveys.  It was not the kind of data that a market researcher wants to bring to management. Especially when it is unsupported by reality.

Ron tore into the data to find out what was going on.  With the help of a high quality panel source, he discovered the panel tenure of respondents driving the survey results was increasing.  Longer term respondents were
more conservative about purchasing than were new ones.

After all that research, Ron had no choice but to pray that business decisions that had relied on the data would not hurt WaMu.  A total of 29 studies were affected.  The scope of the damage was unknowable.

In order to optimize, we must know more about our sample sources.

At Mktg, Inc. our Sample Source Auditors division collects data on panels through various optimization techniques.  We also audit the consistency of panels over time and/or the compatibility of panels around the world.

We grew up measuring all samples against the census and adhering to a strict standard of probabilistic sampling.  Those days are over.

Now our samples are increasingly different from the census, and we’re operating with a lack of information.  If the data we produce is the product of inconsistent sample, then we are advising our clients to take unmeasured risks.

We are out there without a probabilistic framework.  If we don’t adopt a new model for sampling, we’re going to be someone’s lunch.

The guppy makes its decisions based on potential risks.  If we are exposed to the risk of having our research invalidated due to inconsistent online samples, then we had best begin to track that consistency.

The risks of not knowing are too great.  Ask your suppliers if they have passed the guppy test.  It is better to eat than be eaten.

Like many marketing researchers, I didn’t plan this career path from a young age.  I brought transferrable skills from another field.

As far as I know, I’m the only person in our industry with a doctorate in Ecology.

My background in the “hard” sciences gives me a unique perspective on the research industry.

Ecology is not for the faint of heart.  Nor is business.  Things are born, grow, and get eaten by predators.  All in a day’s work.

The marketing research industry is a pond, and you are a guppy.

Guppies have buddies who never make it to the pet store.  These little guys mean business.   And it’s their approach to business that interests us most here.

Guppies do quite a bit of research in their purchasing decisions.  They forage around looking for a good meal in all the likely places.

Their decisions are much like ours.  In these days of busy schedules we shop where we can optimize our time and energy.

Guppies do the same – they optimize.  While food is abundant and efficiently obtained they limit their travels.  There is a risk for guppies in swimming around:  they could become someone else’s lunch.

When the aquatic supermarket is well stocked guppies will eat as they keep an eye out for their own safety.  In the millions of guppy generations that have preceded this one, this process of information gathering has
become essential to guppy survival.  There is no need to take an excessive risk by wantonly traveling all over the pond when staying put will do.

Guppies are good mathematicians.  They constantly calculate the risk/return of their decisions.  There is no hope in the pond for a guppy that doesn’t have a well designed optimization formula in its head.

Optimization is a fairly complex thought process, but it has been shown that even insects can do it.

Once the formula is in place, then the hard work is gathering the data to keep it up to date.  Our little fish friend constantly monitors food abundance, how much time it takes to gather lunch, the energy it must
expend and the amount of attention it attracts in its foraging.  All this goes into its computer-like mind, and a constant flow of decisions follow.

There wouldn’t be a guppy left on the planet if they couldn’t do it.  They are born knowing it’s a fish eat fish world out there.  Information is king, and they gather and monitor it constantly.

Guppies know optimization isn’t a luxury, it’s a necessity.

When food becomes more scarce, the guppy must forage further and further.  Perhaps swimming through the weeds to the bottom of the pond will serve best.  Our little friends take the information they have gathered about
risk from the cues they glean from the environment.

When it seems prudent down they go, through the water to the bottom where they hope to find their next meal.  Such “switching” behavior isn’t taken lightly.  Who knows what lurks in the darkness beyond the lily pad?

The essence of market research is information gathering and interpretation.  We help our clients optimize decisionmaking in their own ponds.

So why am I telling you this tale?

It’s because our guppy friends have something to teach us.  About survival.

In the marketing research industry, we’re ignoring the most basic, instinctual survival system, a system so simple and basic that even a guppy is doing a better job than we are at it.

We’re ironically overriding our instincts and skills as information gatherers and denying ourselves critical information we need for survival.