Today, I want to discuss the sentiment/behavior divide.
If you have been reading this site any length of time, you know of my disdain for sentiment surveys. The poster child for this has been the Black Friday Retail shopping surveys, which I have been lambasting for 15 years.
There are many other forms of bad surveys, and front and center are the Political polls. They are a specialized form of problematic sentiment surveys, and if 2016 didn’t convince you of this, 2020 should.
This is not hindsight bias. On October 2nd, I explained why skepticism of polls was warranted:
“What are the odds that the person responding to a pollster:
-Is part of a representative pool that is similar to the electorate?
-Will or won’t change their mind?
-Is responding honestly to the questions?
-Will actually cast a vote at all?The variables are many, and they are not neatly captured by polling.” 1
That said, I wish I was even more skeptical of the polls than I have been.
Wishful thinking creeps into everything we see and do. In a play on the traditional stock/bond portfolio, I noted what the new 60/40 was: “Assume anything you read from either side is about 60% data analysis, 40% wishful thinking.”
I noted it was a reminder to be cautious about any predictions I might even think about making. Man, was that ever true.
The key is the difference between measuring sentiment versus predicting actual behavior. The question of “Who Will You Vote For?” in polls is overly simplistic. That might capture sentiment at any given moment, but it misses many bigger issues.
Pollsters have tried a variety of methods to get around the sentiment/behavior divide. They try to ask other questions, in order to circumvent this issue:
-How likely are you to vote?
-Who are your friends and family voting for?
-Who do you think will win?
But the bottom line is simply this: When you ask someone a question about a future behavior, the answer you get is their current belief about what they expect to do at some defined point beyond today. Pollsters have yet to figure out how to do this for national polls in a large regionally diverse nation like America.
I hope I get wrong he following prediction: We will completely forget about this lesson in 2024.
Previously:
Prediction Markets, Polls and the President’s Covid19 (October 2, 2020)
______
1. “In my mind, the biggest variable being which side gets out the vote more effectively. Not all elections are as portent and motivating as this one is. But in most elections, the side that does the better job of getting out the vote — especially in those swing states — has the best chance of winning. This is not well captured by either polling or betting markets.”
