Executive Summary
-
Rating of the past performance of securities is considered crucial by investors when they make investment decisions.
-
Several rating methods are used in the financial literature and by the investing community to rate the performance of securities.
-
Performance is considered to be in some way predictable, and prediction is based on past performance.
-
This article empirically assesses the rating of exchange-traded funds (ETFs) and prediction of their performance.
-
The methods examined are the Morningstar rating process, the excess return, the Sharpe ratio, and the Treynor ratio.
-
The empirical results reveal a high consistency among the rating methods and a sufficient level of predictability of ETF performance.
-
ETF performance is persistent over the short term.
Introduction
Exchange-traded funds, or ETFs, are a relatively new investment product, but they are very important for both institutional and retail investors. ETFs are hybrids of ordinary corporate stocks and open-ended mutual funds which invest in baskets of shares that closely replicate the performance and risk levels of specific broad sector and international indexes. As such, ETFs offer investors a considerable level of risk diversification with just a single transaction. The risk of investing in ETFs can be moderated by choosing non-equity investments such as corporate bonds or treasury bonds, both of which are less risky choices than the most common equity-linked ETFs. Also, fixed-income ETFs, which usually carry low risk, are available for investors along with commodity and real estate ETFs.
ETFs are cheap investment tools because their administrative costs are low. This is reflected in low expense ratios due to their passive investment character, which requires managers simply to follow the tracking indexes and not to develop complicated and high-cost investment strategies. Nevertheless, it should be borne in mind that extremely frequent trading can offset the benefits of low expense ratios. The level of ETF expense ratios varies. In particular, ETFs that track broadly diversified indexes have the lowest expenses, followed by those that track sector indexes and others which invest in international indexes. Beyond managerial costs, ETFs pay commission to brokerage companies.
ETFs provide significant trading flexibility since they offer continuous pricing and the ability to trade throughout the day, unlike most mutual funds, which are traded at the end of the day. Furthermore, ETFs offer opportunities for the implementation of both passive and active trading strategies. The most common investment strategy in ETFs is the passive buy-and-hold strategy, the return of which depends exclusively on market performance. Also, ETFs allow active intraday trading and enable investors to buy and sell, in essence, all of the securities that make up an entire market with a single trade. They therefore provide the flexibility to get into or out of a position at any time throughout the day.
Another significant element of ETFs is the potential for high tax efficiency that they offer, since they tend to generate fewer capital gains than traditional mutual funds. The tax efficiency of ETFs arises from their discrete “in kind” creation/redemption process. ETFs are created in block-sized units of 25,000, 50,000, or 100,000 shares by large investors and institutions. The creator of an ETF purchases and deposits with a trustee a portfolio of stocks that approximates the composition of a specific index. In return for this deposit, the creator receives a fixed number of ETF shares, all of which are then usually traded on a secondary exchange market. The redemption of ETFs follows the reverse direction. Buying and selling of ETF shares usually takes place among shareholders and, as a result, there is no need for the ETF to sell its assets to meet redemptions. This advantageous feature of ETFs restricts the realization of taxable capital gains.
The trading price of ETFs usually deviates from their corresponding net asset value, providing arbitrage opportunities for big investors. If the value of the underlying portfolio of stocks is greater than the ETF price, the institutional investor will redeem the low-priced units of ETF by receiving the high-priced securities. In contrast, if the value of the underlying stocks is lower than the ETF price, the investor will exchange the low-priced securities for a newly created unit of the ETF.
Finally, ETFs are characterized by large liquidity, which contributes to easy and rapid trading near their fair market value and to the narrowness of bid/ask spreads and volatility. The liquidity of an ETF is not related to its daily trading volume but rather to the liquidity of the stocks contained in the index. The high liquidity of ETFs is achieved due to the ability of market-makers, which are usually large brokerage houses, to create and redeem shares of ETFs perpetually in response to market demand.
Because of their success, ETFs have begun to attract significant interest in the finance literature. An issue that so far has not been thoroughly examined is the rating of ETF performance and the ability of ratings to predict future performance. Nevertheless, several companies provide ranking services. The most popular is Morningstar, Inc., which rates ETFs on a scale of one to five stars according to past performance. Here we provide an introduction to ETF performance rating by investigating whether ratings are indicative of future returns. We do so using a sample of 50 Barclays iShares.
Performance Rating
Morningstar
We first rate the performance of ETFs by using the Morningstar star rating. We calculate the “Morningstar” return, which is adjusted for expenses such as management fees, 12b-1 fees (annual marketing or distribution fees charged by some mutual funds), custodian fees, and other costs that are deducted from the assets of ETFs. Then we divide average excess return by either the average excess return or the average risk-free rate. The risk-free rate is used when the average excess return is negative or lower than the average risk-free rate. Morningstar return is expressed by the following formula:
Morningstar return = (Expense and load-adjusted return of ETF − Treasury bill) ÷ max[(Average sample return − Treasury bill), Treasury bill]
The risk-free return is used in the dominator of the equation in cases where the average excess return of ETFs is negative or very low.
We then calculate “Morningstar” risk by summing up all the negative average daily excess returns of each ETF and dividing by the number of days in the assessing time period. Morningstar risk is represented by the following equation:
Morningstar risk = Average underperformance of ETF ÷ Average underperformance of sample
Finally, the ETF’s star rating is calculated by subtracting its Morningstar risk from its Morningstar return. Afterwards, we classify ETFs in five classes, each of which includes 10 ETFs.
Morningstar, Inc., adjusts the returns of funds for expenses such as management fees, 12b-1 fees, custodian fees, and other costs that are deducted from the assets of funds. Return is also adjusted for front-end and deferred loads. However, here we do not need to adjust for expenses and loads because we start out by calculating returns with expense-free net asset values, meaning that we can then treat ETFs as no-load funds and removing the need to adjust for loads.
Excess Return, Sharpe Ratio, and Treynor Ratio
The second performance measure we consider is the average daily excess return of ETFs, which is simply calculated by subtracting a fund’s risk-free performance from its return. The third performance measure is the Sharpe ratio. Sharpe ratio is calculated by dividing the average daily excess return of ETFs by the standard deviation of daily excess returns. The last performance measure is the Treynor ratio. This is computed by dividing the average daily excess return of ETFs by their systematic risk. Systematic risk is estimated by the single index market model, where the daily excess return of each ETF is regressed on the excess return of its benchmark.
Performance Prediction
We examine predictability following regression analysis, represented by the next equation:
Pi = δ0 + δ4D4i +δ3D3i + δ2D2i + δ1D1i + u
where Pi is the out-of-sample performance of ETFs. Performance is, successively, the Morningstar return, the excess return, the Sharpe ratio, and the Treynor ratio. The control factors of the model are four variables symbolized as D4, D3, D2, and D1, representing the ETFs that receive four, three, two, and one stars, respectively. The class of top-performing ETFs that are assigned five stars is represented by the δ0 coefficient. This class is the reference group, and hence deltas account for the difference between the top-performing ETFs and other classes.
To estimate the model represented by this equation, we first compute all the performance measures of ETFs in a specific year between 2001 and 2006 and rank them in five classes in descending order. Then, we calculate each of the four types of performance for the subsequent period (2002–07, 2003–07, 2004–07, 2005–07, 2006–07, and 2007). The predictive ability of the model is confirmed when, first, δ estimates are negative and statistically significant and, second, when deltas become more negative as we move from δ4 to δ1.
It has been shown in the literature that there is a positive correlation between fund flows and persistence of performance (e.g. Wermers, 2003). Given that investors tend to put more money in mutual funds or ETFs that receive high grades from Morningstar or other agencies, we assume that this new money pushes up prices and returns and therefore that there should be a meaningful relationship between ratings and future performance.
Performance Persistence
We examine persistence by applying simple regression analysis—specifically, cross-sectional regression of ETFs’ performance in a given year on their performance in the previous year. The beta coefficient of the model is the indicator of persistence. Positive and significant betas imply persistence, and evidence of persistence strengthens as beta approaches unity. Significant negative beta values reflect inversions of ETF performance, while insignificant betas imply unsystematic variation of performance.
This study is presented in the next section.
Case Study
Barclays iShares
Here we will empirically examine the rating and predictability of ETF performance using a sample of 50 Barclays Global Investors iShares during the period 2001–07. Of this sample, 27 ETFs track domestic broad market or sector indexes (20 and 7 ETFs, respectively), while the other 23 ETFs track the country indexes of Morgan Stanley or other international indexes (21 and 2 ETFs, respectively).
The average estimates of the four performance measures are as follows. The average Morningstar performance is negative and equal to –0.496. The average excess return and Sharpe and Treynor ratios are 0.017, 0.013, and 0.017, respectively.
We evaluate the consistency among the ratings by applying a simple cross-sectional model. Specifically, we regress the rating of ETFs according to method i to the rating of ETFs according to method j. More specifically, we regress the rankings of ETFs (i.e. rankings 1, 2, 3, 4, and 5) and not the actual estimates derived by the Morningstar rating method on the rankings derived by excess return. We repeat the regression for all the pairs of methods used to evaluate the performance of ETFs. The measure of consistency is the beta of the model. Positive beta estimates indicate consistency among ratings. Negative or statistically insignificant betas indicate inconsistency among the ratings. Alternatively, we assess consistency by calculating the correlation coefficients among the rankings obtained using the four methods.
The results, presented in Table 1, reveal high consistency among the performance measures. All betas are positive and significant and approach unity, ranging from 0.910 for the regression between Morningstar and excess return ratings to 0.990 for the pairing of excess return and Treynor ratio. This means that the best performing ETFs receive five stars almost consistently regardless of the rating method. This is also the case for ETFs in the other four classes. Table 2 presents the correlation coefficients among the rankings given by the four methods. Correlation coefficients are all greater than 0.900 and approximate unity, confirming the high consistency among the ranking results. Overall, the results reveal that there is no best method for the rating of ETF performance, and therefore investors (could) consult various alternative methods to make their investment choices based on the available information.
Table 1. Consistency in performance rating
| Estimated model | Alpha | t-test | Beta | t-test | R2 | F-stat |
| Morningstar return = α0 + β (Excess return) + u | 0.270 | 1.360 | 0.910* | 15.206 | 0.828 | 231.23* |
| Morningstar return = α0 + β (Sharpe ratio) + u | 0.210 | 1.193 | 0.930* | 17.530 | 0.865 | 307.29* |
| Morningstar return = α0 + β (Treynor ratio) + u | 0.240 | 1.279 | 0.920* | 16.263 | 0.846 | 264.50* |
| Excess return = α0 + β (Sharpe ratio) + u | 0.090 | 0.773 | 0.970* | 27.644 | 0.941 | 764.18* |
| Excess return = α0 + β (Treynor ratio) + u | 0.030 | 0.444 | 0.990* | 48.621 | 0.980 | 2364.06* |
| Sharpe ratio = α0 + β (Treynor ratio) + u | 0.120 | 0.895 | 0.960* | 23.753 | 0.922 | 564.24* |
* Indicates statistical significance at the 1% level.
Table 2. Correlation coefficients between performance ratings
| Morningstar | Excess | Sharpe | Treynor | |
| Morningstar | 1.000 | 0.910 | 0.930 | 0.920 |
| Excess | 0.910 | 1.000 | 0.970 | 0.990 |
| Sharpe | 0.930 | 0.970 | 1.000 | 0.960 |
| Treynor | 0.920 | 0.990 | 0.960 | 1.000 |
The regression results for performance prediction are reported in Table 3. To begin with, the average δ0 estimates are positive for each performance measure. Second, the average δ4, δ3, δ2, and δ1 estimates are all negative. Informationally, the majority of individual δ0 in the individual regressions performed for each year of the period are positive and statistically significant, while the majority of δ4 to δ1 estimates are negative. Considering the significance of the δ4 to δ1 estimates, the results of individual regressions indicate that there is no significant difference between the ETFs included in classes 5 and 4, while there is a definite difference between the top-performing ETFs and medium and low-performing ETFs.
Table 3. Regression results in predicting performance
| Variables | δ0 | δ4 (4-star) | δ3 (3-star) | δ2 (2-star) | δ1 (1-star) | R2 | F-stat |
| Morningstar return | 0.525 | −0.147 | −0.846 | −0.913 | −0.438 | 0.200 | 1.510 |
| Excess return | 0.060 | −0.016 | −0.032 | −0.036 | −0.028 | 0.196 | 2.874 |
| Sharpe ratio | 0.044 | −0.006 | −0.011 | −0.022 | −0.011 | 0.142 | 2.041 |
| Treynor ratio | 0.061 | −0.018 | −0.033 | −0.038 | −0.029 | 0.201 | 2.957 |
The results are interpreted as follows: First, the positive δ0 estimates indicate that the top-rated ETFs display a constant behavior through time. In other words, an ETF that performs well now is likely to perform well in the future. Second, there is no significant difference in the performance of the top-rated and second-rank ETFs. Third, there is evidence that the performance of the medium- and low-rated ETFs is sufficiently predictable, the performance of both these groups being inferior to that of the highly rated ETFs.
The sufficient predictability of ETF returns on the basis of rating in a specific year or period revealed by the results indicates that institutional and retail investors should take into consideration the published ratings of ETFs when they assess their investment choices. However, investors should always bear in mind that returns are not guaranteed and markets can move both up and down. Therefore, ratings are useful but should not be the only criterion in choosing among the bulk of ETFs. Other features, such as risk and expenses, should also be taken into consideration by investors.
Regression results for performance persistence are presented in Table 4. Regarding Morningstar, beta estimates provide evidence for short-term persistence in ETF performance during the periods 2001–02 and 2003–04 but a reversal for the period 2005–06. Beta estimates for the first two periods are positive and significant, while the beta for the third mentioned period is significantly negative. Considering excess return and Treynor ratio, the results indicate short-term persistence in the periods 2001–02, 2003–04, 2004–05, and 2006–07. The excess return results indicate a reversal of performance during the period 2002–03. With respect to Sharpe ratio, the results reveal performance persistence for all the sub-periods except 2005–06, when performance reversed.
Table 4. Regression results in performance persistence
| Period | Alpha | t-test | Beta | t-test | R2 | F-stat |
| Dependent variable: Morningstar return | ||||||
| 2001–02 | –9.305* | –7.359 | 0.744* | 3.447 | 0.198 | 11.883* |
| 2002–03 | 0.009 | 0.056 | 0.001 | 0.060 | 0.000 | 0.011 |
| 2003–04 | 0.000 | 0.000 | 0.853** | 2.141 | 0.217 | 13.318* |
| 2004–05 | –0.044 | –0.115 | 0.065 | 0.207 | 0.226 | 6.699* |
| 2005–06 | –0.009 | –0.051 | –0.145** | –2.447 | 0.176 | 3.127** |
| 2006–07 | –0.487 | –0.723 | 0.077 | 0.107 | 0.154 | 4.187** |
| Dependent variable: Excess return | ||||||
| 2001–02 | –0.058* | –3.288 | 0.296* | 3.302 | 0.185 | 10.904* |
| 2002–03 | 0.103* | 7.968 | –0.247*** | –1.956 | 0.168 | 4.636** |
| 2003–04 | –0.004 | –0.257 | 0.538* | 4.748 | 0.320 | 22.542* |
| 2004–05 | –0.009 | –1.048 | 0.530* | 2.927 | 0.242 | 15.319* |
| 2005–06 | 0.059* | 4.616 | –0.044 | –0.190 | 0.291 | 6.006* |
| 2006–07 | –0.017 | –1.221 | 0.588* | 2.940 | 0.153 | 8.645* |
| Dependent variable: Sharpe ratio | ||||||
| 2001–02 | –0.034* | –6.139 | 0.266* | 2.936 | 0.078 | 4.596** |
| 2002–03 | 0.124* | 13.624 | 0.377** | 2.022 | 0.168 | 4.089** |
| 2003–04 | 0.007 | 0.299 | 0.547** | 2.392 | 0.248 | 15.851* |
| 2004–05 | 0.005 | 0.524 | 0.280*** | 2.010 | 0.077 | 4.039*** |
| 2005–06 | 0.068* | 3.317 | –0.557* | –4.344 | 0.367 | 6.095* |
| 2006–07 | –0.006 | –0.501 | 0.298*** | 1.867 | 0.068 | 3.486*** |
| Dependent variable: Treynor ratio | ||||||
| 2001–02 | –0.059* | –8.282 | 0.302* | 3.367 | 0.191 | 11.334* |
| 2002–03 | 0.108* | 4.990 | –0.197 | –0.698 | 0.141 | 3.783** |
| 2003–04 | –0.007 | –0.520 | 0.565* | 5.222 | 0.362 | 27.274* |
| 2004–05 | –0.009 | –1.045 | 0.529* | 2.929 | 0.243 | 15.406* |
| 2005–06 | 0.060* | 4.526 | –0.031 | –0.136 | 0.300 | 6.275* |
| 2006–07 | –0.017 | –1.229 | 0.584* | 2.972 | 0.155 | 8.831* |
* Statistically significant at the 1% level. **Statistically significant at the 5% level. ***Statistically significant at the 10% level.
Overall, the beta estimates provide sound evidence for persistence patterns in ETF performance at the short-term level. These findings boost the results obtained for the predictability of ETF performance. In other words, persistence may be explained by the performance of either the top or the bottom-rated ETFs. Combining the predictability and persistence of performance, investors may find profitable opportunities by investing in ETFs.
Making It Happen
-
ETFs provide investors with a large range of investment choices covering a variety of domestic, regional, international, and global markets. In addition, ETFs are invested in stocks, bonds, commodities, currencies, and fixed-income products.
-
The assets of ETFs have shown continuous worldwide growth after their introduction on Amex in 1993.
-
ETFs are preferable for both retail and institutional investors due to their trading convenience, low cost, tax efficiency, risk diversification, and portfolio transparency.
-
Information on ETF profiles, management, trading processes, return, risk, holdings, and characteristics can be found from a range of sources.
-
Investors should consider both the rating of an ETF’s past performance and the past performance itself. However, they should bear in mind that past performance does not guarantee future returns.
-
Investors should select an ETF after assessing their own investment profile and evaluating both returns and risks.
Conclusion
We have investigated the ability of ETF performance ratings to predict the future performance of these funds. We ranked ETFs using the overall Morningstar star rating methodology along with three alternative performance measures: the excess return, the Sharpe ratio, and the Treynor ratio.
First, the results reveal a high level of consistency among the four types of performance measure. In other words, all assign similar ratings to ETFs without significant deviations among them. Going further, regression analysis showed that the performance of ETFs is sufficiently predictable. More specifically, the results show that the highly graded ETFs perform well through time, while low-rated ETFs deliver consistently poor performance. In addition, it was found that there is no significant difference between ETFs assigned five and four stars, respectively.
Considering the predictive ability of each performance measure, the results show that the Treynor ratio produces the most significant results—specifically, it has better predictive ability than the other performance measures. The Morningstar rating has less predictive ability than Treynor ratio and excess return while it is essentially equivalent to Sharpe ratio. Finally, the results provide strong evidence for persistence in the performance of ETFs, at least in the short term.
www.qfinance.com