On November 28, 2005, we published, "Critical Look: Personal People Meters" by BIGresearch VP Joe Pilotta, who regularly writes about SIMM (simultaneous media consumption) for iMedia Connection. In the article, Pilotta argued that there are differences between hearing and listening or between consumers being exposed to media and how much attention consumers are paying to the media in question.
Pilotta's context was Arbitron's Portable People Meters, and he cited some results from a RAJAR Validation Test in support of his argument.
(Note: the iMedia Connection article mistakenly used the Arbitron technology's old name, "Personal People Meters.")
Arbitron Vice President of Communications Thom Mocarsky contacted iMedia Communications to dispute Pilotta's characterization of the RAJAR Validation Test:
"In his November 28, 2005 article, 'Critical Look: Personal People Meters,' Mr. Pilotta misrepresents the findings of the RAJAR Validation Test when he states: 'It should be noted that [the Arbitron] PPM accurately discriminates 59 percent of the sounds' and, 'With an accuracy of 59 percent, PPMs "hear" with only one eye.'
"Mixed metaphors aside, this interpretation is inaccurate and misleading. In its validation test, RAJAR created a wide range of listening situations, many of which were not supposed to be credited as valid radio listening for the person carrying the meter.
"According to the paper 'Testing Audiometers' presented at the ASI radio conference in Warsaw earlier this month, Mr. Paul Kennedy, Research Director, RAJAR wrote: 'when the test was designed, we judged that the human ear should be able to register some listening in 55-65 percent of sessions.'
"This means that RAJAR expected an ideal audiometer would credit as 'listening or hearing' no fewer than 55 percent and no more than 65 percent of the tested sessions. By crediting 59 percent of the sessions as 'listening or hearing,' the Arbitron Portable People Meter achieved a near perfect score, coming within one percentage point of the middle of the expected range.
"Mr. Pilotta will be disappointed if he continues to rely on the RAJAR test to uncover any weakness in the abilities of the Arbitron PPM. The RAJAR test showed that the Arbitron PPM does what is designed to do -- reliably and objectively register a respondent's exposure to encoded media -- and shows that the PPM does its task very well."
I contacted Joe Pilotta to ask for his thoughts about Arbitron's statement. Here's what he had to say:
"The fact is that Arbitron's PPM hears 59 percent of the audio accurately. No matter how you slice it, there is a 41 percent chance of error, regardless of how one calibrates its success (i.e., it discriminates between 55 percent and 65 percent).
"According to the July 2005 document presented to RAB, the test was conducted in November 2004. The July 2005 document also referenced a RAJAR validation test from February 14, 2005, with goals of between 50 percent and 70 percent. I don't think 50 percent and 70 percent are equivalent measures; neither are 59 percent and 100 percent.
"I applaud Arbitron's technical accomplishment, as they reached their stated goal. However, my point still stands, and it appears Arbitron concurs."
Brad Berens is Executive Editor for iMedia Communications.