ELECTION SPECIAL: WHAT CAN WE LEARN FROM THE POLLS?

28736470374_2390793780_o

Like it or loathe it, in less than five days, America will have a new president.

The climax of one of the most divisive and hard fought elections in US history has been made all the more dramatic by the tightening of the electoral polls in the last two weeks. Before the final vote is cast, these polls serve as a litmus test for voter sentiment and campaign momentum and are relied upon to forecast results.

Here, there’s a striking parallel with client engagement surveying. In the context of wealth management, the most effective client feedback processes don’t just collect satisfaction metrics, but can be leveraged to build a full picture of the relative strengths and weaknesses of the service, experience and proposition. They can ultimately be used to understand the drivers of different types of customer behavior.

In analyzing the latest electoral polls and some of our recent client engagement research, we’ve noticed that well executed insight programs excel at the same three things: a representative audience, a clear understanding of how to read results and a comprehensive question set.

  1. Amplify the voice of the real audience

The first commandment of effective surveying – be it in the context of electoral polling or client feedback programs – is know thy audience.

In early October, Reuters faced criticism after their poll forecasted a 6-point lead for Hillary Clinton, a margin that Zerohedge claims was achieved because of a misrepresentative poll sample. While 44% of the Reuters sample consisted of self-identified Democrats and 33% identified as Republican, in reality the proportion of registered party members are just 33% and 29% respectively. With a sample skewed towards Democrats, Zerohedge argues that Clinton’s lead would be over-emphasized.

Knowing who has taken part is paramount to providing the right lens on the data that is collected. This is where online electoral flash polls fall down. They are typically hosted on firms’ websites and have little real control over who has answered the poll and how often, meaning the insights gathered are rendered useless. Dig deeper, and there’s also evidence to suggest that respondents and net-bots can be mobilized – at short notice and in disproportionate numbers – to take part in such flash polls to demonstrate support for a particular candidate or a cause.

These examples provide a cautionary tale when thinking about client surveying in the context of wealth management. While wealth management firms have the assurance that they are going out to their own customers for feedback, big swings in client satisfaction results year-on-year could be attributed to the type of client that answers the survey. One example would be shifts in the distribution of asset levels in the sample which could impact on the results achieved. When doing annualized programs, ensuring that the audience is similar and representative of your client base is paramount.

  1. Beware the margin of error

When reviewing survey results, a 5% difference may or may not be significant, depending on the size of the audience you reach out to. The more survey participants, the smaller the margin of error, so achieving as large as sample size as possible is desirable. If this isn’t viable, then understanding the margin for error is crucial to interpreting results.

At a given point in time, Hillary Clinton posted 43% of the popular vote compared to 41% for Donald Trump, with a margin of error of 3%.  This would of course illustrate a statistical tie, since the difference is within the margin of error. This issue has become more and more pertinent as the electoral polls have tightened in the run up to the election.

For wealth managers surveying clients rather than national audience, samples are inevitably going to be smaller, leading to a greater margin for error. Let’s say you fielded two surveys in consecutive months with a sample size of 100 and the overall satisfaction score moved from 43% to 50%.  With a 7 percentage point difference you may be tempted to high five your colleagues, but the margin of error on a sample size of 100 is over 8%, meaning that the dial would not have moved.

This issue becomes even more important when firms attempt to extend their survey programs down to the branch or even RM level.  At these levels, it’s often challenging to gather sample, sometimes well below 50, where the margin for error is much higher.  While there aren’t a lot of ways to get around the issue, the manner in which the results are interpreted are of primary importance.

  1. Harness the feedback

With the advent of technology, polling methodologies have developed greatly in their sophistication to suit pollsters’ objectives. At their most basic, they can be short and binary offering simple insights into the likelihood of victory for each candidate; at their finest, they’re carefully worded with a comprehensive set of questions that deliver maximum insight into how they can improve further.

For campaign teams, it’s not enough to know who has the best chance of entering the White House without knowing why. As such, the very best polls don’t just accurately predict the election outcome; they also indicate development areas for candidates to focus on as they march down the campaign trail.

Similarly, in client satisfaction campaigns, well-designed surveys do not simply ask a customer’s propensity to recommend (Net Promoter Score); they inquire about the full experience to ascertain why NPS could be high or low, in the process identifying actionable insights and areas of improvement for relationship mangers.

Our research shows that feedback is an important part of the wealth management relationship, but many institutions are failing to adopt this comprehensive insight process. Almost two-thirds of HNWIs currently provide feedback to their wealth management firms. But in just 43% of cases this was an online survey where a comprehensive and scalable program can be implemented. Typically, feedback is controlled by the relationship manager rather than as part of an institution-wide program with clear, comparable and benchmarked questions.

The real winners?

In the context of the presidential election, the results will come down to the wire. FiveThirtyEight, which aggregates polling and other data to produce its forecasts, rates Clinton’s chance of winning the Election at 66.2% compared to Trump’s 33.8%. The BBC, which uses the median average of the five most recent national US polls, has Clinton and Trump neck and neck at 45% each. While Fox News’ poll, based on its own landline and phone interviews, has the Democrat nominee at 44% and Republican at 41%.

The differences highlight how poll methodology and sample can have a profound impact on our ability to forecast outcomes. For wealth managers, client surveying can offer a chance to better understand the perceptions and actions of customers. But, this can only be achieved when the process is considered carefully and comprehensively.

News from the world of wealth:

DBS Bank to acquire wealth management and retail banking business of ANZ in five Asian markets (International Business Times)

OCBC cites compliance burden as reason to expand private bank (The Straits Times)

Half of banking customers globally using Fintech services (Private Banker International)

Tax crackdown shows nobody dares to defend the wealthy (Financial Times)

What’s an advisor’s “fiduciary responsibility” really worth? (Wealthmanagement.com)

Events:

Events

David Lo, Scorpio Partnership

Author: David Lo, Associate Partner

Background: David has 15 years of market research experience and joined Scorpio Partnership from JD Power and Associates where he was responsible for the financial services practice.

Education: David completed a degree in business at Central Michigan University and received a MBA from Oakland University.

And at the weekends: In his spare time, David spends his time thinking of new and easier ways to explain to his family what exactly he does for a living.

Photo from girardatlarge, used under creative commons license

Leave a Comment