We clearly believe in our Real ID product and seek all comers to provide tests between our method and any other sampling method out there. The industry needs to determine the strengths and weaknesses of these methods. So how about it? Who is going to step-up to the plate and take a whack at it?

Chuck Miller and Melanie Courtright presented a paper at CASRO Online titled “Respondent Validation: So Many Choices!” which neatly presented a comparative analysis of different outside verification services. We think that this paper was timely and appropriate. But we are still haunted by a lingering question….

….if respondents represent such a dear commodity then why not use a sequence of multiple services in real time to verify respondents. In this fashion, if a respondent fails to verify on one service we can check them out in a sequence of services with the intent of preserving as many respondents as we can.

In the spirit of disclosure, we have considered providing just such a service. When we tested it recently the recovery rate achieved 94%. Most often a single service will recover somewhere in the low 80’s. Surely, the fact that someone is not verified by one service should not be sufficient to eliminate them from the sample frame. We do not know why different services would verify different people, but a better question might be that there is no good reason to expect that all services would verify the same people.

We asked Melanie why she had not considered the option and she lamented that these services are very difficult to work with. We agree.

We are moving away from our fundamentals. We always validated respondents. Essential to any telephone data collection program was the traditional 15% verification. Yes there were those who chose to do less but the concept still existed. If a respondent admitted a failure to qualify then we invalidated the data, re-trained the interviewer and then replaced the respondent.

Our approach to online quality thus far has paralleled these efforts. We verify respondents to make sure they are who they say they are, and by some algorithm seek to eliminate those who behave in certain ways—inconsistencies, straight lining, satisficing and other ills.

We appear to forget that we are measuring populations. Fundamental to our research is an understanding of the population that we are studying. The data we collect is contextual. That it has the greatest meaning when it is considered against background data of the population we are studying. In this sense we are doing little or nothing to achieve quality. We are eliminating respondents with seemingly little concern for the impact that their removal might represent. We huff and puff about the issue but charge ahead with seeming abandon.

Blending sample has always been a black box, proprietary approach. For years we tried to find out if there was substance to OTX’s claim as the blending kings of the industry. Was it real or just another routing approach? We still don’t know.

But when giant sample provider Research Now talks, the industry is likely to listen. In a precedent setting demonstration of pure transparency, Research Now presented the science behind its blending methodology to CASRO. We applaud Research Now for its courage and precedent setting actions. The industry has a new standard to uphold.

In a spirit of proper transparency, we co-presented with Research Now at CASRO and the MRS.

It figures, but no matter where the CASRO Online conference is held, and no matter what it is called, it’s one of our favorites. The tenor of the conference is serious. The content is usually cutting edge and this year was no exception.

We are not in favor of infomercials and lean towards the hard core. The crowd in attendance seemed to concur and the speakers rose to the occasion; our kudos to CASRO and the conference committee that selected the speakers.

GMI Pinnacle joins Mktg’s Real ID® as tools for achieving representative samples.   Chief Scientist Mitch Eggers presented his new sample frame tool at CASRO Online in Las Vegas. For those of you who were not at the IIR conference where Pinnacle first reared its head, CASRO provided a first look at GMI’s innovative method.   Essentially, geared to the GSS,  Mitch has craftily created a means for “trimming” respondents from sources who stray from any one of sixty metrics managed by the N.O.R.C and the University of Chicago.

Eggers has been working on his method for a few years now and presented it after making industry rounds in a series of meetings with various market research companies.   His tool removes outliers from various sources by asking a few (3 or 4) questions of respondents.  The idea is to improve the fit with the GSS variables that his company has chosen.   So far we have not seen all of the variables Mitch has chosen to employ but we salute him for the degree of transparency that he has allowed.

This year’s CASRO/ARF/MRS has represented a turning point in content. GMI is out with GMI Pinnacle, their new sample management tool. Research Now acquired Peanut Labs and allowed Mktg, Inc. to establish new blending technologies. Real ID® and GMI Pinnacle were tested by Phoenix International in their first head to head test and the results were reported at the ARF Convention In March. Phil Garland of SurveyMonkey stunned the MRS with breaking news that SurveyMonkey would consider allowing market researchers to access their sample data base. There is plenty to write about.

When a blog isn’t a blog—that is when you don’t write it. This winter has been all consuming for us at Mktg, Inc. The concept of total sample frame management has taken fire and we have been consumed by the heat. Since our presentation at ESOMAR last Berlin we have been going nonstop. The usual break that we might have expected during the holidays never took form and thus this blog was neglected. Sometimes one gets caught up in other priorities. But we are back. Not because we have less to do but we miss the feedback that this forum has provided us.

Using the fabulous sounds and images of Animusic we caught the attention of a rapt audience at ESOMAR. If you haven’t seen this incredible animation we urge you take a look at the video below. The folks at Animusic deserve credit and we hope that they are listening. Their stuff is landmark and deserves a second look.

As balls went flying through the air we conjured up descriptions of how this fit into our scheme for behaviorally targeted model based routers. We put out a challenge to the industry that all should join in on the fun and participate in the development of new standards.

Buying behavior segments of social network respondents differ dramatically from other sample sources. Phone, panel and river samples all are somewhat similar with purchasing segments at about the forty percent level but social networks appear to range into the high sixties making them a difficult source for blending.If you are one of those who still believes that a well executed telephone sample approaches a probabilistic standard that best approximates the American public then you will find that social network is dramatically different from the old standard.

This implies that if you choose to use only social network sample as a source for research studies beware! Although social networks appear to represent a large portion of the American public they do not appear to be representative of a proper sampling of that population.