A beginner's guide to public opinion research

There is a difference between public opinion and public opinion research. First of all, the opinions of the public can be found almost everywhere, at the coffee shop, in the workplace, in letters to the editor, op-ed pieces and elsewhere. Then there are those who claim to speak on the public behalf, advocacy groups, elected officials, charities, churches, unions and so on. From radio talk shows to Internet discussion boards, there is plenty of public opinion out there on any subject you can think of. And the diversity of opinion in public discourse on these topics is astounding.

However, if you want to understand the opinion of the general public on a subject, it cannot be assumed or generated from an analysis of all these expressions of public opinion. Public opinion research must be conducted. 

Although it has been disparaged as more of an art (or as pure chicanery) rather than a science, public opinion research is the application of scientific methods of inquiry to understand the views of the public. When done properly and with intellectual rigour, research can provide an accurate understanding of the public psyche on almost any issue or set of issues (within the limits of budget and time, of course). When done haphazardly, the results vary from being a nuisance to outright nonsense. As one of my mentors once said, “Bad data is worse than no data.”

We certainly know that public opinion research is used to guide almost every major decision in the public and private sector. It is used to develop new products and policies. It is used to evaluate the commercials we see and the programs that we use. It even provides content for news stories, columnists and pundits. But what is it? How can the views of such a relatively small number of Canadians responding to a telephone survey, say a thousand or so, represent a diverse nation of 32 million souls? Why is any credibility given to the views of one hundred people, selected at random in groups of ten or so, that are expressed in two-hour focus group sessions held in five different major urban centres across Canada?

With regard to the first example, the telephone survey, the views of 1,000 people can be representative of the population if they are chosen according to a sound methodology to ensure random selection and representation across all demographic groups. The point of this kind of research is to “quantify” public opinion by using a questionnaire that is administered consistently to participants. For example, questions on voting intention are used to demonstrate, usually in percentage terms, the support for the various political parties nationally, regionally and by major demographic grouping. The data that comes from quantitative research is expressed in numeric terms (“37% say they would prefer the second option” or “social priorities obtain a mean value of 7.2 versus economic priorities at 6.7”), which enables the use of a full rage of statistical analysis on the data. The only limitation to the analysis (outside of time and budget) is the amount of data collected and the number of people participating in the survey (the sample).

The accuracy of a survey is determined by the number of people in the sample. The fewer the number of participants; the less accurate the results. In our sample of 1,000 participants, for example, the overall results will be accurate to within +/- 3.2 percent. This is the margin of error for this sample size. If we want to look at what female participants say nationally in this survey, the sub-sample will contain approximately 500 participants and will have an accuracy of +/-4.5 percent, which is still fairly good. However, if we want to look at what female participants in British Columbia have to say in this survey, data from the 65 or so participants would have an accuracy of +/-12.4 percent, which limits the utility of this data.

On the other hand, focus groups take an almost opposite approach to investigating public opinion. While quantitative research methods heavily rely on numbers, focus groups, executive interviews and other qualitative research methods reject the use of numbers or any kind of quantification. In the industry guidelines, research practitioners are strongly urged not to use percentages, fractions or numerical expressions of any kind in the reporting of these results. This is not only due to the fact that this kind of research is not statistically representative, but also because the focus of qualitative research is on the discussion with participants and how they agree or disagree to the points raised. Rather than trying to determine the amount of interest in a party, product or program, qualitative research can investigate the reasons why there is an interest (or lack thereof), receive feedback in the participants’ own words and test possible options.

In both types of public opinion research, participants are solicited for their views. This is an important distinction from the other kinds of public opinion discussed at the start of this article. Those who write letters to the editor or call a radio talk show are prompted to express their own views, usually in the negative. The motivations are diverse, but they are “selecting themselves” to contribute their opinions. This self-selection is an important part of public discourse, but it is something that is avoided in good public opinion research (which is why it does you no good to call up a research company to volunteer for focus groups).

As a general practice, a healthy scepticism is required when trying to interpret the results of any public opinion research reported in the media and elsewhere. A review of the questions used, the dates when the data was collected, and the sample size (with regard to quantitative research) will tell you a great deal about the way the data should be interpreted. Unfair questions put to an unrepresentative slice of the population, the staple of media website “polls”, will not give any useful insights and are mischievous at best. Trying to extrapolate a complex regional analysis from a meagre national sample is another common error.

We can be better consumers of public opinion research. In this regard, we should remember John Diefenbaker’s criticism from the opposition benches “They use polls the way a drunk uses a lamppost – for support rather than for illumination.”

- 30 -

Originally published in The Hill Times

The challenge of Electoral Reform

No one said that democracy was supposed to be simple. Or easy.

As every candidate currently on the campaign trail can tell you, being a front-line participant in the election process has numerous and unexpected challenges. Local campaign volunteers also try to cope with the almost unscaleable mountain of tasks that need to be accomplished within the short campaign period traditional to Canadian elections. Even Elections Canada, one of the most respected and efficient electoral institutions in the world, has major challenges in organizing the staff, managing voters lists and coordinating the operation of hundreds of polling stations in the 308 ridings across our country.

The relationship between electors and the elected is never clearer than at election time. Just as aspiring candidates and parties are asking for support, voters want to know what will be accomplished by supporting one candidate over another.

The decision to vote for one candidate or another is not as clear-cut as some may want to believe. There are the merits of the various candidates. The candidate’s party and party leader play an important role in considering a decision, especially when it comes to the policies they promote and their perceived ability to deliver on what they promise. Other factors, such as the competitiveness of the preferred party or the attributes of the other parties, play a role. In all, determining how to vote is a complex and nuanced process for most individuals.

Elections also bring forward a discussion on the possibilities for electoral reform.  It is very difficult, if not impossible, to design an electoral system that can reflect the complexity of a voter’s choice.

Our current system, called Single Member Plurality in the textbooks and commonly referred to as “First-Past-the Post”, has evolved along with our institutions of democratic self-government.

There are those who feel that both our institutions and the way that we select our representatives to these institutions need to change in a fundamental way.

If we were to ask Canadians what they would like to see in a reformed electoral system, one that leads to the best possible representation in Parliament, they would quickly identify a number of priorities that should be included in a process of reform.

The first is that they want to have a dialogue with their representatives, and specifically with those in power, that extends beyond Election Day. They want to be sure that those who are elected are accountable to those who elected them – the voters.

Voters also want the opportunity to be heard. Whether it is the opportunity to discuss and issue or request assistance from a local representative, or to have the concerns of the community voiced in the national forum, voters must have a way to contribute their input on how decisions are made.

Electoral reform also needs to look at the issue of campaign financing. While many voters feel that the ability to raise money from a broad base of donors is part of the democratic process, there is a concern that money, especially large donations from small groups, has an impact on the impartiality of the recipients.

Access to money is a factor in determining the outcomes of elections. It is critical in the success of candidates in the riding nomination process and it certainly key to the ability to run for the party leadership.

It is important to note that the ability of money to determine the outcome of a local election increases with the size of the riding, both in terms of geography and the overall number of voters. In a relatively compact riding with a smaller number of electors, voters are more likely to know the candidates, either personally or by their reputations (good, bad or indifferent). Money, especially in terms of advertising, paid staff and technology, has a lesser impact on the ability to sway voters than the personal actions of, and interactions with, the local candidates. Local volunteers and party activists are also more influential in smaller ridings.

However, as a riding increases in size, money becomes a more important factor and the “connectedness” of the local candidate becomes less important. It becomes more important to find a “star” candidate, who may or may not have a local connection. In a large riding, it becomes critical to have the funds to purchase advertising, conduct telephone canvassing, or undertake other campaign activities. Local volunteers are still needed, but the chief strategic roles in the local campaign become filled by paid staff.

Just as there is a concern about the undue influence of money in the electoral process, there is an ongoing concern about the independence of Members of Parliament to represent the needs and concerns of their constituents.

Although the evils of party discipline and caucus solidarity are overstated, there is a concern that the independence of MPs is compromised under a political regime that emphasizes party loyalty and conformity. The increasing sophistication of political tactics and public advocacy (as well as an omnivorous media) places different demands and stresses on the MP as they balance the demands of both constituency and party.

Any process of electoral reform needs to address all of these above-mentioned issues at the same time. To be a success, any change to our electoral system needs to preserve the accountability of elected officials to their voters, allow local voices to be heard, restrict the influence of money on electoral outcomes, and preserve the independence of elected members.

Some people offer Proportional Representation (PR) as an option for electoral reform. Unfortunately, PR has the affect of eliminating both the local accountability of elected officials to the voters and the ability of local voices to be heard. Because PR deals with larger regions than the single-member constituencies we are more familiar with, it increases the influence of money over electoral outcomes. Finally, because legislators are chosen purely on the basis of party affiliation, the independence of these officials is non-existent. On these four factors alone, PR fails the challenge of positive electoral reform.

Regardless of the model that is offered, PR may be proportional but it is not representation.

- 30 -

Originally published in The Telegraph-Journal, April 23, 2011.

Take the media bias challenge

Even though politics is a serious business, Canadian politics has shown a capacity to be endlessly diverting. I might say “entertaining”, but when I want entertainment, I find that this is best left to the professionals.

The political debate leading up to this election call is a good example of the thought provoking nature of our system. This is not to say that our politicians have stirred Canadians with the compelling logic of their cases or have illuminated the deep recesses of their philosophies. On the contrary, we are warned by our politicians that the upcoming electoral contest will be nasty and brutish (but not short, to complete the Hobbesian paraphrase). Further, the politicians who are telling us to be wary of demonization, fear-mongering and intimidation are the same ones who are labelling their opponents as gangsters, corrupt and sleazy. These statements are often made within minutes, if not seconds, of each other. That is not interesting. What is interesting is that these individuals make these contradictory statements with passionate sincerity and, even more interesting, that their followers embrace them wholesale.

We know that politicians are capable of self-deception. The ability to believe in victory in the face of certain defeat or to promote a policy in the face of continued hostility is an essential job requirement for those who offer themselves for office. But why do we, the voters, let them get away with it? We are reasonable and intelligent people, why do we embrace these contradictory viewpoints without challenge?

For many years, people who studied political behaviour held to the belief that the way we saw the world shaped our partisan leanings. Our family lives, childhood relationships, the signal events of growing up, brushes with the law and injustice, our exposure to the broader world, and so on – all of these were thought to create the basis of our political thinking and the way that we relate (or don’t relate) to the political process and voting behaviour.

But, as political behaviourists study this issue, they are coming across evidence that our partisan leanings and attachments may have more of an impact on how we see the world than the reverse. This is most true when it comes to assessing the veracity of political statements and perceptions of media bias.

We know that, as human beings, we have a predisposition to hear those things that we want to hear and are more likely to remember those things that we find agreeable and repress those things we find disagreeable. Psychologists call this phenomenon “selective screening.” If a person identifies with a particular party, they are more willing to accept statements by party figures as true and are more critical of the statements made by their party’s opponents. This is true even if the statements are neutral and concern matters of broad agreement.

For example, if I were to pose a question on a public opinion survey asking participants to agree or disagree with the statement that “The Internet has changed the way Canadians communicate”, I would expect a substantial amount of agreement. However, if I were to preface the statement by saying that a party leader, let’s say former Reform Party Leader Preston Manning, made that statement, the survey results would show that supporters of the Mr. Manning would show higher levels of agreement with that statement and adherents of other leaders would be more likely to disagree. If the survey question attributed the statement to former Liberal Party Leader Jean Chrétien, those with Liberal leanings would be more in agreement and non-Liberals would show higher levels of disagreement, even though nothing about the statement has changed.

Rather than assessing the truthfulness of statement in its own right, we view the statement through the lens of our own political standpoint. Perhaps this is best summed up by former American President Harry S. Truman when he said of his opponents, “I never give them hell. I just tell the truth and they think it is hell.”

This behaviour not only affects the way we consume political information, but also lies at the heart of our perceptions of media bias.

Interestingly, if a news story, columnist or editorial confirms a view held by the reader, our most common response is to state that there is no bias at all. We believe that the story accurately describes the situation or that the columnist (finally!) understands the situation. This rule applies equally to coverage that could be construed as positive toward the party or viewpoint supported by the reader as well as negative coverage of opposing parties or views.

Typically, “media bias” is only seen when the content is critical of the party or political leader that the reader supports or is positive toward the parties and personalities opposed by the reader. This is not to say that all reporting is absolutely objective, but it is important to note that most charges of media bias depend on whose ox is being gored.  Frankly, claims of bias in the media would carry more weight if those making these claims could identify those times when the bias was in their favour as it was to their detriment. But then again, as noted in the previous paragraph, there is no perception of bias when reporters “get the story right.”

Let’s face it. The reason why we like political satire shows like This Hour Has 22 Minutes or Royal Canadian Air Farce is that they pick on “the other guys” three-quarters of the time. They get it totally wrong when it comes to so-and-so, but they sure have everyone else pegged right.

So, how do we resolve this self-perpetuating conundrum? With the election upon us, each of us has an opportunity to see how we react to the statements and actions of the politicians and political parties as presented in the media. Do you show a preference for news items that confirm your current views and perceptions? Or do you seek out items that may challenge your perceptions and give you different political points of view? You may think that you belong to the second group. I hope you do.

- 30 -

Originally published in The Hill Times 2004

The hardest job in Government

Regardless of your views on politics, you have to feel some sympathy for Finance Minister Blaine Higgs. He has the unenviable job of trying to keep the lid on provincial spending while respecting the political commitments that he and his colleagues in the Alward Government made in last September’s election. Even if the Provincial Budget was balanced, it would be difficult to find the $300 million or so in new spending that they promised without dramatic cuts in programs and services (or an equally large increase in taxes).

At the same time that Minister Higgs is trying to find the savings within Government, he will be facing pressure from his ministerial colleagues who are seized with the priorities within their own departments. Given the principle of Cabinet solidarity, which is earned rather than commanded, it may be that Higgs finds himself facing a united front of “spenders” as he tries to make the difficult choices that go into making a Budget. It can be a lonely experience to be the voice of fiscal rectitude against the administrative and political demands of your colleagues.

But the situation that Higgs faces is not unique. In the early 1990s, Saskatchewan was not the newly-crowned “Have” province of Confederation. The ruinous policies of Grant Devine and his Progressive Conservatives had pushed Saskatchewan to the very brink, a situation much worse than the one that New Brunswick finds itself in today. Government revenues were sinking, population was in decline and outside observers were claiming that Saskatchewan would be the first Canadian province to go into receivership.

Janice MacKinnon, the Minister of Finance in the newly elected NDP Government, faced the same challenge in those years that Higgs does now. Despite their public pronouncements, her cabinet colleagues were largely in denial about the state of the provincial finances. Even if the Province was running an unsustainable deficit, they said, perhaps there was a way to “free up a few million dollars” for a needed service or worthy new initiative.

It was down to Premier Roy Romanow to provide some much needed backbone for Cabinet. In addition to his unqualified support for his Finance Minister, he reminded his Cabinet colleagues of the strong fiscal management and balanced budgets of previous NDP and CCF Governments. According to one account, Romanow quoted Tommy Douglas saying “Deficit financing only helps the bankers.”

Romanow was able to provide the political support that allowed MacKinnon to administer the strong medicine that the times required. The measures were not popular but they were effective. Within three years, the Provincial budget was balanced.

To keep it balanced, an additional four years of fiscal restraint were required – seven years in all. These years were not easy, especially for those on the left of the Party and the public sector unions, but they restored the finances of the Government. For those who use a political metric to judge success, the NDP were re-elected twice more under Romanow and won once more under Romanow’s successor, Lorne Calvert.

With the election of the Chrétien Liberals federally in 1993, Finance Minister Paul Martin found himself in the same position as MacKinnon, facing the spending demands of his colleagues while trying to balance the Budget in a time of fiscal stress. Again, the unwavering support of the First Minister was critical to securing the support of Cabinet for the difficult choices that were required.

As Chrétien himself recounts, there was one Cabinet session on the Budget where Ministers kept bringing forward ideas for additional spending rather than focussing on the task at hand. With obvious frustration, Chrétien quipped that the next Minister to recommend more spending would see an automatic ten percent reduction in their departmental budget.

As in Saskatchewan, the federal budget was balanced within three years. Further, the fiscal rectitude of the Liberals was a key to their electoral success in the following three elections.

There are two lessons to be learned from these examples. The first is that Blaine Higgs cannot do what needs to be done without the strong political support of David Alward. The hardest job in Government, balancing the budget in difficult times, becomes the most futile if the First Minister does not rally the Cabinet behind the Minister of Finance.

The second lesson is that achieving a balanced budget within an expeditious time frame is more likely to lead to re-election than to defeat. Aside from Romanow and Chrétien, there are many other examples, in Canada and elsewhere, that prove this point.

- 30 -

Originally published in the Telegraph-Journal

A question of representation

When it comes to reducing government spending, some cuts are easier to make than others. Cuts that affect those outside of government, such as the municipal sector, not-for-profit organizations and students for example, can be made with impunity without affecting operations within departments.

There may even be political benefits to making these types of cuts, especially when they are aimed at critics of government policy or advocates for social change. As citizens, we should be especially wary of any spending reductions that affect the ability of citizens to hold their government to account or represent their views to those in power.

In this regard, the announced intention to reduce the number of representatives in the Legislative Assembly is largely a self-serving initiative on the part of the Executive. It is a cynical bid to exploit public antipathy to “politicians” while reducing the ability of the Legislative Assembly to hold the Premier and Cabinet to account. Further, this action would undermine the representation ordinary New Brunswickers currently have and would reduce the quality of governance that we receive.

If there is a general principle about public governance in Western democracies, it is with regard to benefits of separating the executive from the legislative and judicial branches of government. The executive, which is responsible for the delivery of services, the enforcement of laws, and the raising and spending revenue, is represented by the Premier and Cabinet. The legislative branch, which consists of the MLAs, is responsible for monitoring and approving government spending, debating legislation, representing local and sectoral interests, and keeping Cabinet accountable to the electorate. Even members of the government caucus, who are not members of the Executive but are necessary for its existence, play a key role in keeping Ministers and Premiers in line.

It is in the interest of the Premier and the Cabinet to diminish the ability of the Legislative Assembly to hold them to account. By cutting the number of MLAs, they reduce the level of oversight on government activities in a number of ways. Fewer MLAs means that there are fewer people to ask questions, to inquire into government activities, or to represent local interests in a province that is geographically and demographically diverse.

Ridings that are larger in size and in the number of voters also increases the constituency workload of the remaining MLAs. As one person, who may have the assistance of part-time secretary and some research support in Fredericton, an MLA is expected to represent the views of just over 13,000 New Brunswickers in their riding. Even in a relatively compact urban riding in Saint John or Moncton, this can be a daunting task. In rural ridings, where the issues can vary from community to community, MLAs are constantly on the road.

Aside from representing riding concerns, MLAs are expected to help constituents who are having difficulty accessing government services or need help complying with the regulatory and reporting requirements that come when dealing with the provincial government. In those rural areas covered by Local Service Districts, approximately two-thirds of New Brunswick’s land area, the MLA is the only representative of government that people can turn to for help.

Although municipal officials in northern communities are the most vocal in expressing concern about this loss of representation, everyone will lose out if this plan goes ahead. There will be fewer seats to contest, which means that the influence of money in determining the election outcome is increased. Fewer ridings will make it more challenging to recruit women and other under-represented groups as candidates.

Riding boundaries are meant to represent communities that share common interests or attributes. Significantly reducing the number of ridings will have a profound impact on the way these boundaries are drawn. In Saint John, for example, boundaries would need to be redrawn to create four riding where there were once six. In Miramichi, four ridings which are already quite large would become two even larger ridings.

It is also fair to ask how linguistic rights would be affected by this change. Currently, one riding is allowed to have a smaller-than-average number of electors in order to respect the language difference between this riding and ones that surround it. The riding in question is Tantramar, which gives the Anglophone communities centered on Sackville a representative in the Legislature. Reducing the number of MLAs would mean that this riding would be absorbed into the largely Francophone ridings in the Southeast.

This rash pledge to reduce the number of MLAs will have a number of unexpected and unpleasant consequences. That said, future Governments would likely enjoy the reduced oversight and scrutiny that would come with a smaller Legislative Assembly.

Frankly, if the Premier and Cabinet wanted to demonstrate that belt-tightening should start at the top, then they should have looked to their own offices and salaries than on the Legislature that holds them accountable.

- 30 -

Orginially published in the Telegraph Journal, December 3, 2011

A Liberal-NDP Merger? This dog won't hunt

As Premier and, later, as Leader of the newly formed NDP, Tommy Douglas was often presented with various schemes that had superficial appeal but would not bear up under scrutiny. His simple phrase in these situations was straightforward “This dog won’t hunt.”

As someone who fought his hardest political battles against Liberals, Douglas would have the same reply to those who think that electoral cooperation (or merger) between the federal Liberals and the NDP is desirable or possible. Frankly, anyone calling out to “unite the left” knows little about either the Liberal Party or the NDP. Differences in philosophy, tradition and party structure makes merger, or even electoral co-operation, nearly impossible.

First of all, the Liberal Party is not a party of the left - it is a capitalist, centrist party. That Liberal Governments are responsible for the social safety net and strongly promotes human rights may lead many people to think that it is a “leftist” party, but this is also the party that embraced balanced budgets and debt repayment. In the words of Pierre Eliot Trudeau, Liberals are the party of the “Radical Centre.”

The New Democrats, founded by members of the Cooperative Commonwealth Federation (CCF) and the Canadian Labour Congress, is a socialist party (or social democratic party, if you find this a less pejorative term). If there is to be a merger, or even electoral cooperation, how can these two parties reconcile this fundamental difference on this essential economic question?

There is also a deep chasm between the New Democrats and the Liberals on the question of national unity as it pertains to Québec. The Liberals still adhere to a “one size fits all” approach to our federation, where all provinces are treated uniformly in the Constitution, and claim that there is no “right” for any province to withdraw. The New Democrats, at least in their upper echelons, are friendly to the idea of “asymmetrical federalism” and have endorsed Québec’s “right to self-determination.” 

In the NDP, unions play a structural role in the decision-making processes of the party. In addition to contributing significant amounts to the NDP war chest, unions and union locals affiliated with the NDP send their own delegates to party conventions, including leadership races. By being able to vote in the delegate selection process for riding associations and affiliated unions, the views of these members are over-represented in the councils of the NDP. This practice of “double voting” would be as unlikely for Liberals to accept as it would be for the NDP to surrender. (In the UK, Tony Blair brought an end to a similar practice in the Labour Party to make it a more acceptable option for British voters.)

While a merger may not be in the cards, what about “electoral cooperation?” This would see one party in a riding “stand down” in favour of the other, presumably to increase the chances of victory.

One significant barrier to this concept is the perception of Canadian voters. It is a political convention in Canada that, to be seen as a serious national party, a political party has to have a candidate in every riding in the country – 308 in all. It is not a formal rule but the consequences for a party that does not run a full slate of candidates is quite clear.

Further, given that the public financing of political parties is based on the number of votes that a party receives ($1.75 per vote per year), a decision to “stand down” in a riding in favour of another party affects the Party’s bottom line as much as their reputation as a serious national party.

Having lost its rural base, the NDP is now competing for voters in the same urban areas as the Liberal Party. The fact that Liberals and NDP candidates are each others’ strongest competition in many ridings, in Toronto and across the country, means that there is little to gain from electoral cooperation.

Even cooperation in ridings where the difference between an NDP or Liberal victory over a Conservative candidate is within the margin of voters for the other party, the possibility of this actually occurring is slim. This presumes that NDP voters in a riding would move en bloc to a Liberal candidate carrying the banner of electoral cooperation and vice versa. Partisan attachments may not be as strong now as in the past but we shouldn’t underestimate this factor among core supporters of either party. A core Liberal or New Democrat would be more likely to stay at home than to vote for another party – even if their party leadership encouraged them to do so.

The idea of a merger or some form of electoral cooperation between the Liberals and the NDP is based more on novelty, opportunism and distaste for the current minority government than sound political reasoning. In the hunt for a progressive government, just remember what Tommy said about this type of dog.

- 30 -

Originally published in the Toronto Star, republished in the Telegraph Journal