Explaining the misinterpretation of public opinion polls
Many critical thinkers find the misinterpretation of poll findings abhorrent. This misinterpretation has been occurring for many years. The problem is perpetrated by some pollsters, some political commentators, and some journalists who are the gatekeepers of sharing information with the public. One problem that bedevils them, which is the focus of this column, is the misinterpretation of polls which show a statistical dead heat. There are other problems that will be discussed in a future column. Public opinion polling, particularly pre-election polls, is well established and globally accepted. Predictive polls are mostly correct when the scientific method is followed.
The interpretation of two recent pre-election polls highlights the problem. The PNP commissioned June 2023 Don Anderson Poll reported that the PNP had 35 per cent support, and the JLP 30 per cent with a margin of error of +3 or -3 per cent. Polls should not be interpreted without considering the margin of error, that is the extent to which the polls may be wrong. The PNP with +3 per cent could be 8 per cent ahead, the lead of 5 per cent added to the +3 per cent margin of error; or the PNP could be 2 per cent ahead, take the -3 per cent from the 5 per cent lead. It is best to stick with the slim 2 per cent lead for the PNP since part of its small 5 per cent lead falls within the margin of error. The PNP, therefore, would have a negligible lead of 2 per cent.
The August to September 2023 Blue Dot-Nationwide Poll reported that the JLP had 31 per cent support and the PNP 25 per cent with a margin of error of +2.75 or -2.75 per cent, which I have rounded off to 3 per cent. The JLP could be 9 percent ahead, the +3 per cent margin of error added to the 6 per cent lead, or 3 per cent ahead, the -3 percent margin of error out of the 6 per cent lead. I will stay with the slim 3 per cent lead for the same reason given above for the PNP.
The JLP and the PNP are in a statistical dead heat which means they are tied in the party standings. The Blue Dot-Nationwide Poll (3 per cent JLP lead) and the Don Anderson Poll (2 per cent PNP lead) are saying the same thing. A one percentage point difference in popularity is meaningless. Large sections of the public that argued that the PNP was leading in June are now saying that the JLP is leading. The question to be asked is why some journalists, some political commentators and some pollsters are interpreting the polls showing a statistical dead heat inaccurately, thereby misleading the public. Two easy explanations that readily come to mind are innumeracy because of poor Caribbean Secondary Education Council Mathematics passes over many years, and the lack of integrity in the country. We should not ignore these reasons but there are other potent interconnected reasons for the misinterpretation problem. I explain some of these reasons.
The postmodern idea pathogen
The idea pathogen of postmodernism is embedded in the minds of some gatekeepers. Postmodernism’s main argument is that there are no universal truths except the assumption that there are no universal truths. Therefore, gatekeepers pick a winner when some or all the difference between the parties fall within the margin of error. They look at the same poll numbers and arrive at “their truth”. How they feel about the issue is more important than what is true. They feel it is their right to interpret the scientific evidence as they wish. Despite its popularity, postmodernism is anathema to the scientific method, and has obliterated critical thinking among certain sections of the academy and so is a danger to human progress. Some postmodern “scholars” will be upset but I stand by the scientific method that determines what is true through objective measurement.
Intolerance for ambiguity
People do not like ambiguity when they have to interpret things. Intolerable uncertainty makes people psychologically uncomfortable, so they want a clear cut leader or winner declared in the polls. People demand that pollsters “call it”. Any pollster who cannot call it is not worth her or his salt. The parties are in a political race and they both cannot share the gold medal. There cannot be two leaders in the same poll, or two winning parties in the same electoral race. This situation pressures some pollsters to engage in misinterpretation. I have seen pollsters initially explain statistical dead heat results and subsequently declare a leader saying that the momentum is with party A without conducting a new poll. There is no way these pollsters could detect party A moving ahead of party B without going back into the field.
Winners make excellent headlines
Some media houses and journalists are caught up in the public’s demand to know the leader or winner. This demand also shapes how some journalists interpret the polls reinforced by the time-honoured code that sensationalism sells. These headlines attract readers and listeners. Increasing readership and listenership creates a larger market and increases earnings from advertisements in a situation where some media houses in Jamaica are struggling financially. There is also the influence of unconscious partisan bias (some will say deliberate bias), which results in some of these gate keepers structuring their interpretations of the polls in support of their political party without realizing this. We all suffer from cognitive traps that negatively influence our decision making.
Comments versus evidence
Traditional media prides itself on taking a balanced approach to issues by allowing multiple viewpoints from political commentators with limited expertise. Common sense is elevated over scientific evidence in these discussions and political scientists are seldomly invited to be a part of the discussion. What is worse is that I have never seen a statistician invited to be a part of the discussion. A related problem is that some pollsters are insufficiently assertive. Although these pollsters state the importance of the margin of error, they allow some political commentators and some journalists to take the discussion south of the evidence. It was in similar discussions of the past that the late pollster Professor Carl Stone was deemed arrogant and aggressive because he did not allow people without the expertise and skills to dominate the narrative about his poll results.
Declining trust in society
Some people become suspicious and distrustful when an insignificant lead is explained away within the margin of error because of declining trust among Jamaicans. Trust is the psychological glue that holds people, leaders and institutions together in society. Moreover, with many competing and varied sources of political information in the digital age, these distrustful citizens latch on to these contradictory sources of information. These Jamaicans refuse to believe in a science that is unable to declare a leader, or winner, in the same way they refuse to believe in a science that cannot make a vaccine which provides 100 per cent immunity.
The interpretation problem creates public confusion and undermines the credibility of polls. Therefore, the media should invite a statistician and a political scientist to participate on discussion panels to help with interpretation, guests should be mindful of their biases, pollsters should be conversant with the science of voting behaviour and how the Westminster and the first pass-the-post systems work to contextualize the interpretation of pre-election polls. Finally, some political commentators and journalists should brush up on their basic statistics, and the latter should push evidence rather than perspectives to enlighten the public.
Christopher A.D. Charles, Ph.D. is a Professor of Political and Social Psychology in the Department of Government at the University of the West Indies, Mona. He builds statistical models for election forecasting and is a Council Member of the Political Forecasting Group of the American Political Science Association.
