I have been a consistent critic of survey “data” and polling, along with traditional measures of sentiment. There are many reasons for this: Half of Americans do not vote, so when they respond to polls they are making up answers. Even if they say they are going to vote, there is little reason to believe them. I don’t know who still has a landline, or who answers an unknown phone call on a cell phone, but I question if these folk represent broader America.
People responding to these polls are at best guessing — who will be on the ticket, who they will support, and most unknown of all, whether they actually will get out of the house and go to their local polling place to cast a vote on election day.
~~~
In a car filled with interesting people on the way up to Camp Kotok in Grand Lake Stream, this curious question came up on the the general question of polling/survey questions, and what people know, don’t realize they don’t know, but answer related questions anyway.
Tom Morgan of The Leading Edge raised this issue in response to a discussion of how under-utilized the phrase “I don’t know” is — especially but not exclusively in finance.
He asked us all:
“What do people actually know relative to what they believe they know?“
Tom shared a fascinating analysis that looked at how people conceptualize other groups, whether by economic strata, behavior, race, religion, etc. Taylor Orth, Director of Survey Data Journalism at YouGov,.looked at what various people believed when it came to the size of different subgroups of Americans.
There were two enormous takeaways from this.
The first: How wrong people are in general. Two separate YouGov polls “Asked respondents to guess the percentage (ranging from 0% to 100%) of American adults who are members of 43 different groups, including racial and religious groups, as well as other less frequently studied groups, such as pet owners and those who are left-handed.”
Americans vastly overestimate the size of minority groups, including sexual minorities, the proportion of gays and lesbians (estimate: 30%, true: 3%), bisexuals (estimate: 29%, religious minorities, racial and ethnic minorities, etc.
And, people tend to underestimate majority groups.
Look at the chart above; you will see the average answer ranges from very to laughably wrong. None of this is complex or hard-to-find information; it’s all readily available to anyone who wants to search for it.
We mostly did pretty well answering Tom’s Q&A on what “actual and estimated” numbers were across these groups. But it was a car full of economists, fund managers, and philosophers, all of whom spend an inordinate amount of time looking at data, especially BLS NFP/Labor numbers. We collectively did much better than those surveyed, guessing the actual data and how far off the survey respondents were. If anything, we underestimated how wrong people were in answering these questions.
~~~
The second aspect of this discussion is even more fascinating to me than merely being data ignorant: Why don’t people say I DON’T KNOW when they, in fact, do not know?
As an example, we discussed whether COVID-19 escaped from a Lab or was transmitted in a Wet Market. I argued that there is but one acceptable answer to this question: “As someone who is neither a virologist nor an intelligence operative, I do not have the tools needed to render an expert judgment about the origins of Covid.”
This is an answer we rarely hear.
Constance Hunter (formerly of KPMG & AIG) added, “We pillory people for rightfully saying “I don’t know” when we should applaud the honesty and integrity of that answer.”
Dave Nadig floated a theory why: “Social media has made it mandatory for everyone to have an opinion about everything.”
We should all ask ourselves, Why?
Previously:
Learning to say “I Don’t Know” (September 9, 2016)
What Do You Believe? Why? (June 29, 2023)
Source:
From millionaires to Muslims, small subgroups of the population seem much larger to many Americans
by Taylor Orth
YouGov, March 15, 2022
