As Nielsen incorporates Amazon’s own data, networks demand answers

Nielsen plans to incorporate Amazon Prime’s streaming data into its audience measurement starting with this season’s “Thursday Night Football” games in a move that has rival networks crying foul.

TV network executives and their trade group the Video Advertising Bureau are alleging bias from two referees in the industry—Nielsen and the Media Rating Council—which audit the accuracy of media measures. They’re questioning the unusual speed of the audit, which is set to be considered by the MRC in an Aug. 30 meeting. The proposed change also appears to have caught some agency executives by surprise.

What’s at stake could be the future of who owns the rights to televise the biggest-ticket sports leagues, tilting the playing field further to well-heeled digital platforms, should Amazon benefit from a substantial ratings boost by trading on its own data when other media companies can’t, said VAB CEO Sean Cunningham.

“Having DNA-level factors that are inherently biased baked into the system is putting [Amazon] at an advantage,” Cunningham said. “That’s not fair to the other players, especially since there hasn’t been any constructive engagement from the co-authors of this.” He said Nielsen has “gone completely opaque as to how these numbers were cooked up in the first place” and failed to explain the change to “some of their largest customers, who are some of the longest holders of these [broadcast] rights.”

Behind the change

But George Ivie, the MRC’s CEO, said the speed of his group’s effort was exactly about trying to shed light on Nielsen’s proposed methodology to incorporate Amazon first-party data. He disputed any bias in moving fast, saying the group initiated an effort to look into “Thursday Night Football” measurement early this year, after seeing the significant disconnect between Nielsen and Amazon numbers last year.

At that point, Ivie said the two parties were already well into discussions around incorporating Amazon first-party data into the ratings. Nielsen was planning to put the new data into the market by September, Ivie said, and the MRC pushed to have it audited first, which now has happened. The results of the audit, and whether MRC members will approve the new methodology next week, remains to be seen, he said.

Amazon since last year has cast doubt on Nielsen’s household panel-based ratings for “TNF,” publishing its own numbers alongside Nielsen’s each week last year. Amazon’s data averaged 18% higher for overall viewership across all 15 games.

The result of mashing up Nielsen and Amazon data looks a lot more like Amazon’s numbers, at least when it comes to the number of households watching, based on historical impact data Nielsen provided to clients in June. A copy of that report obtained by Ad Age shows the new methodology calculated household audiences that averaged 18% higher than Nielsen panel data and 19% higher than Nielsen’s “big data” from set-top boxes and smart TVs for the five Amazon “TNF” games in December.

On a viewership basis, however, it was more like splitting the difference between Nielsen and Amazon numbers. Measurement of viewership for all people ages two and up was 9% higher using the new methodology than using Nielsen’s panel and big data combined, a Nielsen spokeswoman said.

Look back: Last season’s NFL ratings fell 2%

The impact data was made available to clients on June 15 amid this year’s lengthy upfront negotiations. Nielsen communications that went out at the time did not specify that the company expected to have audits and possibly accreditation for the new methodology in place for the current season as currency.

One agency executive expressed uncertainty about whether the new data figured into negotiations with Amazon. Another reported being unaware of the impact of the data Nielsen had made available in June. “Our investment folks were not under the expectation that Amazon methodology would change during upfront negotiation,” the executive said.

Bottom line, advertisers could end up paying more than they bargained for when making this year’s deals for “TNF” based on projections from last year’s numbers, or at least shouldn’t count on much in the way of make goods for audience under-delivery this year, should the Nielsen-Amazon hybrid numbers become the new currency.

Next week’s MRC meeting to consider the change follows what was described by some participants as a contentious meeting last week that provided a first look at Nielsen’s complex methodology for reconciling and blending the data sets, in which several questions were raised about the statistical validity of the process.

Amazon: NFL advertisers will be able to target in-game ads

MRC’s role started early

“We became aware that Nielsen and Amazon were going through this process of defending their measurement to each other and going through a kind of reconciliation,” Ivie said. The MRC had its auditors sit in on those discussions, and the results were inconclusive initially, he said. “They were trying to collaborate, but they really didn’t find a smoking gun of the source of the differences, so that concerned us,” Ivie said.

Ultimately, the two sides agreed on a process to incorporate Amazon data into the national service, Ivie said, “and we said, based on what we know about this, we need to audit this and audit it hard.” That includes what he called a “two-sided audit” looking at all factors of Amazon’s origination of the data and Nielsen’s processing and calculations.

Amazon servers are not directly integrated with Nielsen’s computer systems; rather, Amazon is providing summaries of its data, Ivie said, and part of the process is evaluating the accuracy of those summaries.

The speed was driven largely by Nielsen and Amazon’s timeline to get the new data into the market by next month, Ivie said, and the MRC concluded it was important enough that it should be audited before anyone trades on it.

Watch: NFL promotes itself as if its players were actors

Networks say they don’t get similar service

Countering allegations by networks that Amazon is getting special hurry-up treatment that others don’t, Nielsen pointed to its client communication earlier this year outlining the collaboration with Amazon and offering to incorporate data from other streamers into its measurement.

But the VAB’s Cunningham said networks that have tried to get Nielsen to make similar moves with them haven’t been able to get information about technical specifications, how Nielsen will evaluate data quality or who will pay for audits or timelines. That includes at least one media company that began asking Nielsen about integrating its server data as early as February, when the company convened an industry summit about sports programming measurement.

Nielsen disputed that it’s slow walking or refusing first-party data integration with other streamers. “We would not turn away business,” a spokeswoman said. “That seems like a very silly accusation.”

Cunningham said he believes Nielsen has “its thumb on the scale” in favor of Amazon as part of a broader effort to curry favor and lock in business with big streaming platforms seen as the future of connected TV and sports programming. That’s a possibly misguided strategy, given that the bulk of its revenue today comes from the big network players, and digital platforms appear to pay far less than they do, he said.

Look back: CTV ad fraud spiked 69% in 2022, DoubleVerify finds

As evidence of favoritism, he provided Nielsen national data comparisons showing co-viewing estimates of viewers per household as much higher for Amazon’s “Thursday Night Football” than any other network’s co-viewing numbers last year across all day parts, and well ahead of what “Thursday Night Football” got on Fox and the NFL Network in 2021. Cunningham also pointed to out-of-home viewing data for “Thursday Night Football” on Amazon that were well ahead of those of any other network last year across all day parts and of “TNF” broadcasts the year before. That’s despite questions about how many bars and restaurants had access to Amazon Prime, though Amazon sought to address that through a deal that sent its broadcasts to 300,000 establishments via DirectTV.

Nielsen and Amazon executives stood by the numbers and pointed out that the data came before the incorporation of Amazon first-party data into Nielsen’s calculations proposed for this year.

“Nielsen is committed to evolving our methodologies to provide the most accurate measurement of what the audience is watching,” according to a company statement. “We are making modifications for live streaming measurement to more accurately reflect the growing impact of streaming and first party data. We will continue to leverage newer data sets as they become available in order to drive our measurement capabilities forward, with the ultimate goal of counting and reporting the viewing habits of everyone, everywhere.”