T.O.P. versus the Arbitron Atlanta Approach


A discussion paper by Geoff Mills [MFA, BA, GradDipBusAdmin, DipTeach] April 2003


There has been much discussion around the OAAA (USA) about the outdoor measurement system being developed by Arbitron in Atlanta. Though clearly there is a lot that is excellent about the Arbitron approach, I believe it still has some inherent problems that advertisers simply won't buy into...

1. For a start, all of Arbitron's people-meter electronic recording relies totally on GPS systems which...

  • Can't accurately measure mobile, transit, indoors or other non-traditional media - this has been the biggest problem with all previously proposed systems including Australia's ROAM, etc. Without the support of the mobile, transit, indoor and small format sectors of the industry, it will be very difficult to establish a 'total industry mindset'...and an industry divided is NOT going to beat television and radio at attracting advertisers.
  • Standard Civilian GPS (without resorting to the considerably more expensive and time consuming Differential GPS) can't provide accurate enough data. There are two formats, 'personal GPS using the SiRF chipset or similar' and 'vehicular GPS' using either of two popular technologies. The vehicular GPS technology is quite accurate and stable, but of course, can only measure 'vehicular traffic movement'. The 'personal GPS technology' that Arbitron are using isn't stable indoors (as the people at Wherify have recently discovered with their 'Personal GPS watches') and at best is only accurate to 10 metres (about 30 feet)...and in 'blowout' conditions is only accurate to about 190 feet (60 meters) with a 95% degree of confidence. Not to mention that most GPS devices require 'line of sight' to the various satellites they are receiving signals from...so inside a concrete building, tunnel, city canyon, etc...they often just don't get readings at all!

    Which when recording the position of a person (as opposed to a vehicle) begs questions like...

    • Is the survey respondent sitting in front of bus shelter A or bus shelter B on the other side of the road, or at the park bench 100 feet away?
    • Where did they go? - when they disappear into a city canyon, a highrise building or under a concrete tunnel, etc.
    • Did the bus on School Road pass close enough to the survey respondent for them to be able to see the advertisement on the side of the bus? What about the back of the bus?

2. Secondly, Arbitron are ultimately a statistical research company who have developed sophisticated algorythms and analysis processes to estimate population behaviour based on field trip surveys, log books, government generated census data and complicated travel pattern projections, etc. Though I'm sure there are differences, it appears remarkably similar to the ROAM process of traffic projections, etc (with a small amount of GPS data to try to add credibility). 

TV and radio people-meters don't just record the performance of some media companies and then estimate the performance of others. The survey sample's activity is monitored against ALL TV or radio media in the area and from THESE results, the total population estimates for reach and frequency are accurately calculated. If the out-of-home advertising industry believes it can do anything less, than it's fooling itself. Advertisers just won't buy it.

Isn't ANY form of outdoor ratings better than none at all?

NO. Advertisers 'trust' the television and radio people meter surveys for two (2) reasons...

  1. Advertisers know that to create the ratings some 'exact' percentage of survey respondents (e.g. 8.1%, 16.9%, etc), had to be actually 'exposed to' the advertising on the various channels/stations timeslots being rated. Though statistical models are then involved in projecting the total population reach and frequency figures, these projections are based on ACTUAL measured activity of a small but statistically accurate sample of consumers. In other words...the final ratings represent a ratio of REAL recorded performance. With the television and radio people-meters, they don't record the performance of some media companies and then estimate the performance of others...the sample group's activity is monitored against ALL media companies, and from THESE percentages, the total population estimates for reach and frequency are proportionally projected. If the out-of-home advertising industry believes it can do anything less...than it's fooling itself! The advertisers won't buy it.
  2. Advertisers know that ALL media companies within the industry sector (television or radio) accept the published ratings. Since these companies are in direct competition with each other and the ratings reveal 'winners and losers', the advertisers can feel confident that the various media companies will ensure that the ratings and the measurement of sample groups is 'honest'. 

My concern with the Arbitron process is that the amount, type and accuracy of the GPS data collected will fail to convince the market, that the Arbitron ratings are little more than 'statistical projections' based on other statistical projections...something that advertisers have already shown they don't trust.

In addition, I fear that the Arbitron ratings will have even greater difficulty convincing the market that their process has accurately measured mobile, transit and small-format media performance. This is likely to result in factional fighting within the out-of-home industry as to who's 'paying' for the service...(personally, I'm not likely to want to pay for a service that my clients and customers don't believe is valid or credible) and the Arbitron system could then become a measuring tool paid for by a few stakeholders...with the others in the industry trying to down-sell and undermine its importance.

Unless ALL the 'calculations' are based on 'actual recorded consumer activity' of well-defined sample groups across ALL media, advertisers are likely to dismiss the Arbitron figures as they have dismissed other 'estimates' in the past.

Arbitron are receiving an excellent worldwide response with their Portable People Meter (PPM) system. But PPM has absolutely NOTHING to do with Out-of-home advertising as WE define it. PPM is a clever process that imbeds inaudible signals into the audio tracks of radio and television broadcasts which can be detected by personally carried devices on survey participants. In this way, radio and television are obtaining better ratings because the system doesn't rely totally on 'home' reception or 'recollections of exposure' (as Nielsen's system predominantly does). Meanwhile, the cohesive radio and television industries are slowly gaining yet another strategic advantage over the out-of-home industry.


So how is TOP going to be different?

T.O.P. doesn't rely on statistical modeling to provide its 'core measurement component'.

T.O.P. (like television and radio people meters) relies on actual 'hits' by survey participants. Someone in the survey group has to be legitimately exposed to the out-of-home advertising media, before that media site will record a 'hit'.

EXAMPLE :: Let's say there are 15,000 participants in the T.O.P. survey of the Greater Sydney area (for example). Then let's further assume that these 15,000 survey participants represent a statistically accurate 'demographic sample' of the total Greater Sydney adult population of approximately 1.5 million.

If 591 persons in our sample (or 3.93% of all participants) are recorded as regular daily 'line-of-sight proximity hits' for a Supersite on Parramatta Road, this would translate to an estimate of a reach of 59,100 and a frequency of 23 hits per month (a total of 1.359 million hits per month)

If an additional 2,160 individual 'line-of-sight proximity hits' were recorded by our survey participants on a casual or transient basis throughout the survey period of a seven (7) day week, this would equate to an estimated total of 216,000 additional 'total population' hits for the site per week (or 936,000 additional hits per month). 

The end result summary for this Supersite might calculate out to something like...

Reach (Total Population) ::  116,261 persons
Av. Frequency (Tot. Popn) ::  16.4 hits per month
Total Hits (Total Population) ::  2.295 million hits per month

Of course, complete breakdowns by demographics, time of day, day of week, etc, would be easily and accurately provided and CPM rates and 'qualitative data'...including :
• What percentage of survey respondents recall 'seeing' the advertisement
• What (if any) was the survey respondents intended action in relation to the displayed advertisement
• What percentage of those exposed to the advertising were in private transport, public transport, taxi, pedestrians, etc...
would also be available from a T.O.P. survey.

The difference is in the fact that TOP actually measures 'the exposure of advertising media to the survey participants' and then from these measurements, total population figures are calculated.


The TOP Method ::

• Media/plant/sites are equipped with a TOP Gun (a simple RF transmitter that broadcasts a field closely approximating the actual 'visibility field' of the media).
• Survey participants are supplied with a TOP Watch (a wearable device that not only records 'hits' from all TOP Guns it encounters, but also records its location via GPS every 20 seconds).
• The resultant data is cross-checked, corroborated and correlated into the survey results for that region/period.

 The hardware costs of the 'TOP Guns' (that broadcast a 'viewable field' for each site) are estimated to be less than $20 per unit (the actual component parts can be purchased at an electronics shop for less than about $10 but this excludes the assembly, power supply management and programming of each chipset. If several thousand devices were required, the per unit costs could drop significantly.)

Ultimately, it's a simple problem with a simple solution...

THE PROBLEM ::

  • You want to be able to tell what percentage of your survey group actually passed within the 'viewable field' of (i.e. were actually 'exposed to') a specific piece of Out-of-home advertising (irrespective of whether that advertising was stationary or moving, indoors or outdoors).
  • You also want to be able to determine WHEN the survey respondents actually encountered that advertising, HOW OFTEN they actually encountered that advertising during the survey period and for HOW LONG each encounter actually lasted.
  • Ideally, you also want to record 'qualitative data' like how well the survey respondents actually recall seeing that advertising, and even potentially what action (if any) they actually intend to take in relation to the advertising that they were exposed to.
  • Oh yes...and this is the hard part...you want to be able to record all this information for ALL out-of-home advertising media.

The point is, at its core...you want to record 'actual hits' not guesses or estimates, irrespective of how well intentioned and well conceived those estimates are.

THE SOLUTION ::

1. You determine exactly what percentage of all out-of-home media you need to collect this data for. (It is unlikely for example that you actually need 100% of sites recorded since two billboards side-by-side could effectively be returning the same data.)

2. You equip as many out-of-home advertising sites as you can, with a low-powered, low-cost RF transmitter that broadcasts only as far as the respective advertising can be effectively seen/read by the naked eye. The only data that these transmitters need to transmit, is the unique ID key number of that site/plant/media and a universal time code - you'd better encrypt the data so that it doesn't get lost in the 'soup' of RF signals out there.

3. You equip your survey respondents with a device that can detect when the respondents pass through any of these transmitter equipped sites and record the ID key number and timecode associated with that detection. (All you need for this is a simple RF receiver with a decryption code connected to a simple chipset that includes a non-volatile memory Flash card.) You also supply this device with a GPS chipset and record location data (as accurately as is available) every 20 seconds.

4. At the end of the survey, you then collect the RF receivers from the survey participants and collate the data to extrapolate the 'ratings' for all transmitter-equipped sites.

There are a few additional points...like for example, how you record the 'qualitative data' and how you use the GPS data to corroborate and validate the 'proximity detection' data that the RF receivers obtained...but in essence...that's it! That's what TOP is all about...providing accurate, trustable, recordings of every occasion that a survey respondent has a 'realistic exposure' to the sites being surveyed.


© 2003 Geoff Mills [MFA, BA, GradDipBusAdmin, DipTeach]


[Back to TOP Home] [TOP Information - PDF file] [Return to AUS3 Home]

Hit Counter