In part one of this series, we looked at the background of the Globe and Mail discount brokerage rankings as well as how they’re structured and who they’re targeted towards. In this next part, we take a detailed look at what the discount brokerage rankings are actually measuring and some interesting observations we made about the Canadian discount brokerage industry over time. Lastly, we provide some important tips to keep in mind when using rankings as part of your product research.
What Do the Discount Brokerage Rankings Measure?
When looking at any ranking or rating, one of the most important questions to be clear on is what the ranking or rating is actually measuring. In our review of the J.D. Power Investor Satisfaction Survey, we saw that “investor satisfaction” was being measured by six components: interaction, trading charges and fees, account information, account offerings, information resources and problem resolution. By comparison, the Globe and Mail discount brokerage rankings are measuring what Rob Carrick thinks is the “best discount brokerage” for “mainstream” investors.
As we saw in part one, when looking across the last eleven rankings, it appears that the categories that go into defining “the best” discount brokerage are not static. The most stable characteristics of what it means to be “the best” seem to cluster around costs, trading and tools. According to Carrick, the categories that he chooses vary in large part because they are based on a combination of data from reader surveys and his perceptions of what mainstream investors are curious about or would find worthwhile.
Strengths of the Discount Brokerage Rankings
A strength of this approach is that the discount brokerage rankings are somewhat reflective of the mood or sentiment of mainstream Canadian investors. If investors are curious about certain features, such as commission free ETFs or user experience of a discount brokerage, the rankings have incorporated these kinds of innovations into their structure. Having looked at a decade of results, it is fair to say that the rankings reflect the pulse of what mainstream investors are exposed to from the discount brokerage industry and hence curious about.
Limitations of the Discount Brokerage Rankings
While Rob Carrick’s opinion is certainly informed by monitoring Canadian discount brokerages for over 14 years, his opinion may not necessarily be shared by other investors, something that readers should keep in mind when doing their research. The degree to which his opinion can be generalized rests on how accurately the needs of “mainstream investors”, a term that is loosely defined, are captured in the questions he uses to survey discount brokerages and in the process he uses to evaluate their products and services.
A second limitation of the rankings is historical comparability. Because the criteria have changed as often as they have, it is difficult to compare historical performance of Canadian discount brokerages in a meaningful way. It may be possible to compare results on costs, trading and tools because of their relative stability as categories however the total scores from year to year are largely incomparable.