The art of polling may be based on science, but there is also a subjective element that can't be eliminated. The best polls are the ones that have the least amoung of subjectivity, i.e., not trying to "rig" a poll by fashioning questions designed to get the desired response.

On Friday, the polling industry organization that oversees polling companies cried foul on the Ne3w York Times and CBS for conducting a poll whose methodology had “little grounding in theory."

Politico:

Last Sunday, the two news organizations unveiled their “Battleground Tracker,” an online survey updated each month. The poll — conducted by Internet pollster YouGov — interviewed more than 100,000 people nationwide, the news organizations said, allowing them to project results for each Senate race in the country.

The results were featured on CBS’s “Face the Nation,” and the New York Times “Upshot” data-driven vertical published multiple entries about the survey. The first edition of the “Battleground Tracker” showed Republicans leading in 51 Senate races, which prompted The Upshot to project that Republicans had a 60 percent chance of winning control of the Senate.

The American Association for Public Opinion Research’s statement on Friday criticized CBS and the Times for using a survey method that has “little grounding in theory” and for a lack of transparency.

“[M]any of the details required to honestly assess the methodology remain undisclosed,” according to the AAPOR statement, issued under the organization’s letterhead and signed by president Michael Link. “This may be an isolated incident with the Times / CBS News providing more information on this effort in the coming weeks. If not, it is a disappointing precedent being set by two of our leading media institutions.”

CBS News elections director Anthony Salvanto disputed that charge, saying in a statement that CBS disclosed the methodology for the study “in great detail.”

“Battleground Tracker is a pioneering project that delivers a comprehensive look at the electorate for viewers and readers in a manner that has never been seen in midterms before,” said Salvanto. “As always, the methodology is available in great detail along with our findings at cbsnews.com. The Battleground Tracker model represents another example of the rich history of industry-wide innovations from CBS News.”

The two news organizations issued a joint statement defending the survey.

AAPOR President Michael Link had some harsh words for the Times, who he accused of abandoning their old polling standards without replacing it with another set of standards:

“This means no standards are currently in place,” Link wrote. “It is unclear why the decision was made to pull the existing standards before the new ones are developed, vetted and published. Yes, all responsible institutions need to review their standards periodically, making appropriate changes as technologies and methodologies change. However, standards need to be in place at all times precisely to avoid the 'we know it when we see it (or worse yet, ‘prefer it’)' approach, which often gives expediency and flash far greater weight than confidence and veracity.”

Remember all those "victories" for Ron Paul in 2012 on-line polls following the GOP debates? What the AAPOR is saying is that no scientific standards have been developed to use online polls in the way that CBS and the NYT used them. And it's questionable whether either media organization vetted the new standards adequately.

This may sound like an esoteric argument, but the implications are huge. Pollsters are  constantly trying to find ways to improve accuracy. The inclusion of cell phones, for instance, has markedly improved the accuracy of polls given the demographic that uses cell phones the most skews younger. Pollsters are also looking to improve minority participation in polls, as well as seeking to overcome the problem of non-participation (the number of people who actually fully respoind to pollserts questions compared to the number of calls made has been declining).

This argument isn't over yet. We'll see if the media organizations can convince AAPOR that they're actually on to something.