Do pollsters build in an “embarrassment factor” into their predictions? If not, they should.
The Conservatives in this country always seem to do better than the polls suggest, as people are a bit too embarrassed to admit in public that they will support them. When it comes to the privacy of the voting booth things are rather different. I suspect this is a factor in why they also seem to have been taken by surprise by the turnout for both Brexit and Trump.
I’ve been discussing this with some friends this week – did the pollsters actually get Brexit wrong?
They definitely got the last General Election wrong, and they have some form in that regard, but my recollection of the Referendum is that about a week or so beforehand the polls swung pretty firmly in the direction of “Leave”, and then moved back to being broadly within the margin of error.
I was in the States with work in the run in: virtually every conversation on that trip eventually found it’s way to Brexit, and I recall telling anyone who would listen that the polls suggested a strong chance we’d be out, much to the disbelief of my American hosts.
The above is a BBC page compiling polling data (phone and online) in the lead up to the referendum (click on “Show all polling data”).
It appears to confirm that on 22 June, the day before the vote, of four major polls available, 2 suggested a win for “Leave”, one suggested a win for “Remain”, and the other was deadlocked. If you go back a week earlier, you can see regular 5-10 point leads for “Leave”.
I’m just wondering whether my memory of this period is confused (it wouldn’t be the first time), or if people are mixing up the Referendum polling and the General Election polling – I’ve heard an awful lot of people this week saying that the polls didn’t forecast a Leave vote, but I vividly recall assuming Leave would win on the basis of the poll numbers, together with the paucity of the Remain camp’s arguments.
Obviously, either way the pundits got this one massively, massively wrong, so just a point of interest really.
There was a lot of talk about ‘shy Tories’ not admitting how they would vote after the last election, but that’s not what happened at all. The Cinservatives polled higher than predicted, but within a reasonable margin. It was people who said they were gounfpg to vote Labour and then didn’t who skewed the polls and don’t really know why that was.
Re: being too embarrassed to admit in public that you’ll be voting Tory (or whoever)
I’ve long thought that there should be a category in polls called “I’m not telling you”.
As in, are you voting for X or Y? Well, if neither, I’ll have to put you down as a “Don’t Know”.
A great chunk of those classified as “Don’t Know”s, I believe, do know. They just don’t want to tell.
Pollsters are out of the loop because from where I’m sitting reason seems to have gone out of the window. Maybe they are working on the basis of old models and they haven’t allowed for a world where people are more likely to believe a tweet/social media post over an argument that cannot be summarised in 140 words.
It seems the only reliable polls are exit polls as was the case in June 2015.
Seconded. Voters seem to be looking for vague, meaningless slogans rather than sober, considered realism, with no consideration for the consequences of this dumb approach.
There was also a good explanation on fivethirtyeight of why elections can be so unpredictable (small historical sample size), and referendums even more unpredictable (no historical data at all). They never had trump at less than one in ten, and up to one in four yesterday morning. But it seems like the basis for all the polls was flawed because this situation was so different – who do you ask and how do you extrapolate.
That said, i am in a state of shock, tending toward fear.
Yes, I think it’s an indication of how things have changed that the polls seem to be all over the place at the moment. It suggests that all their standard assumptions no longer apply and that something fundamental is going on with voting patterns and loyalties.
As I recall the pollsters were seeing the Referendum as close, but I think the overall summary was still for a Remain vote. The polls underestimated the support for Brexit, for Trump, for Corbyn. All votes (in different ways and to different degrees) that were versions of a vote for ‘fuck the establishment’.
See above; the BBC has all the pre-Referendum polls available, and it was extremely tight – I think the overall result was well within the margin of error, and the polls nearest to referendum day suggested a “Leave” win.
Regarding Corbyn, did they underestimate his support?
YouGov had him with a big lead a month ahead of the first leadership vote. They didn’t predict that he’d win from the start of the contest, but it’s quite possible (indeed, likely) that his support grew over time. The above link has him on 53% of the vote one month out – he ended on 59.5%, but it’s quite possible he grew his share in the run up, particularly given all the media attention he was getting.
From where I’m sat, the pollsters totally cocked up our last General Election result, and were even further off-beam last night (although I suppose they’d observe that Clinton won the popular vote). Something is clearly going wrong, but I’m not sure the issue is completely endemic just yet.
I think it goes like this with opinion polls …. If you’re going to vote for Trump/Brexit/Donald Duck, you know the reaction of a lot of people is going to be ‘How could you be so stupid?’ So you don’t answer, or say you’re undecided, then you vote for that candidate. This often gives them 4-5% more than expected, and can swing a result when it’s tight.
I seem to recall that after the last UK general election proved so disastrous for Labour, despite the polls predicting a tight result, some group of pundits determined that the poll results had all been skewed, because people more likely to have right-leaning views (self-employed tradespeople, small business owners, managerial types, the working poor) tend not to want to respond to pollsters at all, citing responding to polls (on any subject, not just on politics) as being a waste of their valuable time.
I’m sure, given the Brexit poll results and these election results, none of the polling organisations have yet found ways of allowing for this in their projections.
It’s the effect on voters that polls have that is a worry. When the polls say ‘Hillary’ or ‘remain’ then how many people who would have voted that way think that it’s a done deal and therefore don’t bother to vote.
Personally I feel that voting should be compulsory….but that’s another argument!
Still think the polling industry is bollox though!
You’d only get the “done deal” supposition if the polls are consistently pointing one way.
If it’s a close-run thing or the polls are swinging towards your chosen candidate’s opponent, you’ll make sure that you get out and vote.
I was at a business seminar last week – the guy presenting started up a company that collects data from cinema loyalty schemes and shares that info with the studios. He seemed to say that now the studios have access to exactly what their audiences want, how old they are, when they like to watch films and where the audience lives and does for a job – and a whole new world opens up for them. This data information is a goldmine for large organisations looking to engage on social media etc etc.
While I was listening, I was reflecting on the fact that I almost always lie when I am asked for personal info by a non-essential organisation like Trip Advisor or shops. Sometimes I am lady in her late 60s who enjoys trampolining holidays. Other times I am 18, working at a vegan abbatoir, slaughtering carrots. It’s an opportunity for harmless mischief.
Is this why the pollsters are getting things so wrong? Do they think they know more than they actually do?
Studio lawyer here. You’re underestimating our ability to tell good data from bad (the stuff we’re interested in is the behavioural data you don’t even know you’re leaving in your wake – personal data of the kind you describe is too much regulatory hassle and far too untrustworthy).
That said, studios have less access to quality data than you’d think, for various reasons, and they’re also not great at sifting through it all, at least compared to the tech companies, who have this stuff in their DNA. Which is why we still make so many movies no one wants to see. It is improving though – Netflix is forcing everyone to up their game.
Do pollsters build in an “embarrassment factor” into their predictions? If not, they should.
The Conservatives in this country always seem to do better than the polls suggest, as people are a bit too embarrassed to admit in public that they will support them. When it comes to the privacy of the voting booth things are rather different. I suspect this is a factor in why they also seem to have been taken by surprise by the turnout for both Brexit and Trump.
I’ve been discussing this with some friends this week – did the pollsters actually get Brexit wrong?
They definitely got the last General Election wrong, and they have some form in that regard, but my recollection of the Referendum is that about a week or so beforehand the polls swung pretty firmly in the direction of “Leave”, and then moved back to being broadly within the margin of error.
I was in the States with work in the run in: virtually every conversation on that trip eventually found it’s way to Brexit, and I recall telling anyone who would listen that the polls suggested a strong chance we’d be out, much to the disbelief of my American hosts.
http://www.bbc.co.uk/news/uk-politics-eu-referendum-36271589
The above is a BBC page compiling polling data (phone and online) in the lead up to the referendum (click on “Show all polling data”).
It appears to confirm that on 22 June, the day before the vote, of four major polls available, 2 suggested a win for “Leave”, one suggested a win for “Remain”, and the other was deadlocked. If you go back a week earlier, you can see regular 5-10 point leads for “Leave”.
I’m just wondering whether my memory of this period is confused (it wouldn’t be the first time), or if people are mixing up the Referendum polling and the General Election polling – I’ve heard an awful lot of people this week saying that the polls didn’t forecast a Leave vote, but I vividly recall assuming Leave would win on the basis of the poll numbers, together with the paucity of the Remain camp’s arguments.
Obviously, either way the pundits got this one massively, massively wrong, so just a point of interest really.
There was a lot of talk about ‘shy Tories’ not admitting how they would vote after the last election, but that’s not what happened at all. The Cinservatives polled higher than predicted, but within a reasonable margin. It was people who said they were gounfpg to vote Labour and then didn’t who skewed the polls and don’t really know why that was.
Re: being too embarrassed to admit in public that you’ll be voting Tory (or whoever)
I’ve long thought that there should be a category in polls called “I’m not telling you”.
As in, are you voting for X or Y? Well, if neither, I’ll have to put you down as a “Don’t Know”.
A great chunk of those classified as “Don’t Know”s, I believe, do know. They just don’t want to tell.
I’m just pleased they don’t predict when the village bus is due.
I’d be unnecessarily freezing me knackers off on a regular basis if they did.
Pollsters are out of the loop because from where I’m sitting reason seems to have gone out of the window. Maybe they are working on the basis of old models and they haven’t allowed for a world where people are more likely to believe a tweet/social media post over an argument that cannot be summarised in 140 words.
It seems the only reliable polls are exit polls as was the case in June 2015.
Seconded. Voters seem to be looking for vague, meaningless slogans rather than sober, considered realism, with no consideration for the consequences of this dumb approach.
To be fair, fivethirtyeight.com had Trump’s chances at 30% or so.
There was also a good explanation on fivethirtyeight of why elections can be so unpredictable (small historical sample size), and referendums even more unpredictable (no historical data at all). They never had trump at less than one in ten, and up to one in four yesterday morning. But it seems like the basis for all the polls was flawed because this situation was so different – who do you ask and how do you extrapolate.
That said, i am in a state of shock, tending toward fear.
Yes, I think it’s an indication of how things have changed that the polls seem to be all over the place at the moment. It suggests that all their standard assumptions no longer apply and that something fundamental is going on with voting patterns and loyalties.
As I recall the pollsters were seeing the Referendum as close, but I think the overall summary was still for a Remain vote. The polls underestimated the support for Brexit, for Trump, for Corbyn. All votes (in different ways and to different degrees) that were versions of a vote for ‘fuck the establishment’.
See above; the BBC has all the pre-Referendum polls available, and it was extremely tight – I think the overall result was well within the margin of error, and the polls nearest to referendum day suggested a “Leave” win.
Regarding Corbyn, did they underestimate his support?
https://yougov.co.uk/news/2015/08/10/corbyn-pull-ahead/
YouGov had him with a big lead a month ahead of the first leadership vote. They didn’t predict that he’d win from the start of the contest, but it’s quite possible (indeed, likely) that his support grew over time. The above link has him on 53% of the vote one month out – he ended on 59.5%, but it’s quite possible he grew his share in the run up, particularly given all the media attention he was getting.
From where I’m sat, the pollsters totally cocked up our last General Election result, and were even further off-beam last night (although I suppose they’d observe that Clinton won the popular vote). Something is clearly going wrong, but I’m not sure the issue is completely endemic just yet.
But then, what do I know?
I think it goes like this with opinion polls …. If you’re going to vote for Trump/Brexit/Donald Duck, you know the reaction of a lot of people is going to be ‘How could you be so stupid?’ So you don’t answer, or say you’re undecided, then you vote for that candidate. This often gives them 4-5% more than expected, and can swing a result when it’s tight.
For all the talk about shy Trump voters, there did seem to be an awful lot of proudly noisy ones out there.
I seem to recall that after the last UK general election proved so disastrous for Labour, despite the polls predicting a tight result, some group of pundits determined that the poll results had all been skewed, because people more likely to have right-leaning views (self-employed tradespeople, small business owners, managerial types, the working poor) tend not to want to respond to pollsters at all, citing responding to polls (on any subject, not just on politics) as being a waste of their valuable time.
I’m sure, given the Brexit poll results and these election results, none of the polling organisations have yet found ways of allowing for this in their projections.
It’s the effect on voters that polls have that is a worry. When the polls say ‘Hillary’ or ‘remain’ then how many people who would have voted that way think that it’s a done deal and therefore don’t bother to vote.
Personally I feel that voting should be compulsory….but that’s another argument!
Still think the polling industry is bollox though!
You’d only get the “done deal” supposition if the polls are consistently pointing one way.
If it’s a close-run thing or the polls are swinging towards your chosen candidate’s opponent, you’ll make sure that you get out and vote.
I was at a business seminar last week – the guy presenting started up a company that collects data from cinema loyalty schemes and shares that info with the studios. He seemed to say that now the studios have access to exactly what their audiences want, how old they are, when they like to watch films and where the audience lives and does for a job – and a whole new world opens up for them. This data information is a goldmine for large organisations looking to engage on social media etc etc.
While I was listening, I was reflecting on the fact that I almost always lie when I am asked for personal info by a non-essential organisation like Trip Advisor or shops. Sometimes I am lady in her late 60s who enjoys trampolining holidays. Other times I am 18, working at a vegan abbatoir, slaughtering carrots. It’s an opportunity for harmless mischief.
Is this why the pollsters are getting things so wrong? Do they think they know more than they actually do?
Studio lawyer here. You’re underestimating our ability to tell good data from bad (the stuff we’re interested in is the behavioural data you don’t even know you’re leaving in your wake – personal data of the kind you describe is too much regulatory hassle and far too untrustworthy).
That said, studios have less access to quality data than you’d think, for various reasons, and they’re also not great at sifting through it all, at least compared to the tech companies, who have this stuff in their DNA. Which is why we still make so many movies no one wants to see. It is improving though – Netflix is forcing everyone to up their game.
“the stuff we’re interested in is the behavioural data you don’t even know you’re leaving in your wake…”
Do you mind expanding on that?
Shhhh…. I’ve already said too much; the walls have ears.