| In the prior thread, there was discussion about how 7 Days disclosed all its polling and lack of participation, which is technically accurate, but missing a bit: it's fairly easy to find a 7 Days article that references the poll results without talking about poor participation, such as:
http://www.7dvt.com/2010legislative-survey-results. On that piece there is no mention of the poor results. You have to go to the piece which does "analysis" of the findings for that. It indicates a low response rate but doesn't acknowledge that that low response rate is relatively poor:
Of the 400 surveys sent out, only 30 came back with legitimate answers - a response rate of 7.5 percent. That's better than direct mail - for which a 2 percent return rate is considered successful - and not too much worse than the turnout for a Burlington election, which was 23 percent on Town Meeting Day. Honestly, we hoped for better.
Just to clarify here: direct mail has an entirely different goal than surveys. Direct mail isn't intended to assess opinion. It's intended to sell things.
Surveys which attempt to assess opinion generally use sampling, so having a small sample size is acceptable, but it depends on the size of the sample group and how much stock you want to take in the results. A sample of about 100 people out of 400 would give you a fairly solid confidence interval. A sample of less than 1/3rd of that gives you very little confidence at all in the results.
Honestly, the comparison of the results to direct mail response results isn't just poor judgment: it represents a complete and total lack of understanding of what statistics are and how they work. |
That piece also makes an odd inference. When referencing the "most ethically challenged" item, it doesn't just report on the results, but actually tries to connect them to a specific event:
During last month's Mardi Gras parade in Burlington, Sen. Peter Shumlin climbed aboard an anti-Vermont Yankee float sponsored by the Vermont Public Interest Research Group. No doubt he saw it as an opportunity in the wake of the Senate vote against the nuke plant. Turns out he may have used VPIRG polling information to his political advantage, too. Smart, affable and possessing sales skills some would describe as "slick," the senator from Putney never misses an opportunity to advance his ambitious agenda...
This isn't part of the survey. It's not part of anything involving the survey at all. It's just 7D waxing about Shumlin. That's fine, but it's not analysis of the results. It's just opinion under the guise of information.
Which brings me to the last point about this survey's "analysis:" when you send out surveys like this to individuals with political ambition and goals, it's very easy for a small group of people to game the survey to serve their own agendas.
And to be fair: I expect this sort of nonsense from political organizations. The Heritage Foundation routinely uses real data to create an "analysis" of the information designed and targeted to suit their political agenda. It's slimy, but I expect it from that sort of organization.
Media, on the other hand, has an ethical obligation to see to it that those doing "analysis" of their information actually have the basic competencies to do the job and to do so in a fashion which doesn't serve a political agenda. By performing this sort of "analysis" under the guise of a legitimate news organization, they are basically doing the work of the Dubie campaign by allowing them to take fake facts and present them as real facts.
There's a lot I like about 7 Days, but the way they handled this? I don't know if it's malice or incompetence at play here, but if it's neither of those, it's just reckless disregard for the truth. |
