How the Canadian Discount Brokerage Rankings are Structured
The reviews each year generally consist of two parts. The first part is the numerical “score” which is composed of scores on categories considered to be relevant to mainstream investors. The second component of the rankings is a commentary by Rob Carrick on his experiences with each brokerage.
Over the last 11 rankings, three “core” categories have consistently been measured: costs, trading and tools. These three components have collectively made up roughly 65% of the total score. The remainder of the score (~35%) has been made up of various categories such as customer satisfaction, account information, website usability and most recently innovation. The non-core categories have changed 7 times in the past 11 years. For example, in 2008 “Account Information” made up 10% of the total score however in 2012 it now makes up 25% (which also happens to be the same weighting that “cost” has).
The consequence of changing categories is that there are some limitations on comparing results from one year to the next. While the majority of the score (~65% or so) is comparable from one year to the next, a substantial portion of the score (~35%) changes fairly often. If discount brokerages are very similar, and therefore likely to have similar scores, this level of variability becomes an issue because the difference a single percentage point makes in one year could be as a result of the ranking criteria changing not necessarily a change/improvement in the way a company is performing.
Alongside the “score” component is the more subjective commentary provided by Rob Carrick. The comments that make up this portion of the ranking description reflect Rob Carrick’s experiences while doing his research, what he thinks are salient points for readers as well his distinct commentary style. The comments have evolved over time from a “pros and cons” style to more recently a high-level observation of each discount brokerage and what they might be planning for the future.