When looking at two polls, there are some mathematically solid methods that help you distinguish the more reliable one.
Who did they ask? The most reliable polls ask likely voters. The next most reliable ask registered voters. The least reliable ask adults. (I guess the very least reliable ask unregistered teenage zombies, but I've seen very few polls who will admit that is their sample set.)
When did they ask? If everything else is equal between two polls, the most recent gives the best information.
Almost all polls at the state level ask likely voters, and as for freshness of a poll, in this crazy season a poll one week old is next to no good at all.
The next most important question that has a true mathematical foundation is How many people did they survey? A larger survey gives you a smaller margin of error.
But for me, when I decide on what polls to pay attention to for my prognostications, I pay attention to something else instead of sample size. When the election is truly close at hand, I ask How large is the Undecided vote? Technically, it shouldn't matter but experience has shown me that polls with large None of the Above content a day or two before the election are probably not doing a very good job. In the Michigan polling, a local company named Mitchell Research steadily shows more undecided voters than anyone else, so I have discounted their results regardless of sample size.
Enter We Ask America, a polling company that is nationwide but relatively new. I don't remember them when I was gathering polling data in the 2008 election and their online archives only go back to 2010. Regardless, they are busy little bees and almost produce as many polls as the two big dogs on the block nowadays, PPP and Rasmussen.
The thing is, We Ask America always has awful amounts of Undecided, even close to election day. For example, their last poll before the South Carolina contest had 15% undecided at a time when other polls were showing 4% or 5%.
And so we have the two contests tomorrow in Michigan and Arizona. A week ago, We Ask America found 13% undecided in Arizona and 20% in Michigan, suspiciously high numbers compared to most other companies. But miracle of miracles, it all turned around this weekend. In both states, the number of undecided in their polls was -1%. In other words, because of rounding error, the sum of the four candidates was 101%.
For both of these polls to be so unnaturally tidy is just as suspect as ones where they have twice as much undecided as everyone else. These kinds of miracles do not come from improved polling techniques. They come from upper management saying, "Bring down the undecided numbers or it's your ass!"
I'd like to see We Ask America change their name to the more accurate We Make Shit Up.
p.s. I'll have my predictions and Nate Silver's up by tomorrow morning. PPP is promising one more poll released and I love to get the freshest data possible.
