Man vs. Machine: Comparing Discretionary and Systematic Hedge Fund Performance (2017)
Campbell R. Harvey, Sandy Rattray, Andrew Sinclair, Otto Van Hemert
The Journal of Portfolio Management 43(4), URL/SSRN
This week’s AGNOSTIC Paper examines the ongoing Man vs. Machine question in asset management at the example of hedge funds. The paper is therefore a predecessor to AgPa #21 that examines the same question for AI-powered mutual funds. The authors mention that there are still myths around systematic investing and many investors seem to have some kind of algorithm aversion. This is in-line with my own experiences, so I believe the paper fills an important gap for better education. In addition to that, the authors provide a practical framework to evaluate the performance and risks of hedge funds which I believe goes beyond the question of Man vs. Machine.
Everything that follows is only my summary of the original paper. So unless indicated otherwise, all tables and charts belong to the authors of the paper and I am just quoting them. The authors deserve full credit for creating this material, so please always cite the original source.
Setup and Idea
The idea and motivation of this paper is straight-forward: Who performed better? Systematic or discretionary hedge funds? Before we go into details, let me first clarify some semantics. Systematic refers to investment processes in which humans do not decide about every single investment, but set up a model or system to execute trades more or less automatically. Discretionary, in contrast, are human portfolio managers who develop and implement investment ideas. Some people criticize those labels because systematic portfolio managers also have discretion over their systems. Similarly, discretionary portfolio managers usually also follow some implicit systematic in their decisions.
Another motivation of the paper is to tackle algorithm aversion within the asset management industry. The authors provide various examples of concerns like “systematic funds are hard to understand”, “systematic funds are less transparent […]”, or the classic “systematic funds are bound to perform worse […] because they only use data from the past”.
Despite the long time (7 years!) since the paper was published and the continuing quantification of finance, I believe those concerns are still on top of many investors’ agenda.1It is always surprising to me that some people do not trust a transparent quantitative model, but have no trouble to trust whatever human being. But maybe I am the strange person in this respect. In my view, the best weapons against concerns are evidence and education. So although the results are unfortunately quite old now, I think it still makes sense to look at them.
Data and Methodology
In my view, the authors develop two interesting methodologies to examine hedge funds. First, they use natural language processing to classify hedge funds into systematic or discretionary based on their fund description. For this purpose, the authors obtain a sample of pre-classsified hedge funds from Hedge Fund Research (HFR) and identify the words “algorithm”, “approx.”, “computer”, “model”, “statistical”, and “system” to be representative for systematic funds.2In an appendix, the authors provide additional details about their methodology. Compared to today’s opportunities with Chat-GPT and the likes, this of course appears quite simplistic. Back in 2017, however, this was quite innovative and I bet it is still more sophisticated than what many institutional investors do nowadays.
The second part of the methodology is the factor model to evaluate performance and risks. Since the authors evaluate hedge fund performance after-the-fact, they design a quite practical model. In particular, they focus on “[…] factors that (1) were well known by 1996, when our sample period starts, and (2) are easy to implement”.
In my opinion, this is very sensible. Of course, you leave some information off the table when you only include the major asset classes and factors (URL). On the other hand, the authors rightly argue that including newer factors introduces some look-ahead bias and is no fair benchmark.3We cannot criticize the ancient Egyptians that they haven’t used modern cranes to build the pyramids faster because there were no such cranes back then… Even more important, the factors you benchmark against should be investable. What is the point of looking at a benchmark that you cannot buy when your investment disappoints? Needless to say, this is also a problem of many newer and advanced factors.
Overall, I therefore regard the 8-factor-model of the authors as a practical, but still rigor framework to evaluate hedge funds. The following exhibit describes and summarizes the factors over the sample period.
So much to methodology, now to data. The authors collect a sample of 6,955 equity and 2,182 macro hedge funds from the HFR database. Importantly, the exclude backfilled returns, include terminated funds, and start the sample period in 1996 to ensure the best-possible data quality. They further exclude funds that do not provide monthly or net-of-fee returns. Finally, they only consider funds with more than $100m of assets under management (AuM) as of 2014. For the previous years, they include funds when their AuM make up the same fraction of the total sample AuM as $100m make up in 2014.
Important Results and Takeaways
One further important comment before we finally get to the insights. The following results are based on equal weighted indices of all hedge funds within a certain category. So we look at the average performance of systematic and discretionary hedge funds. Those are very hypothetical numbers. It is very unlikely that real-world investors actually invest in an equal weighted portfolio of thousands of hedge funds. Since there is no easily-accessible passive benchmark (URL) for hedge funds, investors cannot expect to earn the average return of systematic or discretionary funds going forward. Their ultimate results will inevitably depend on their manager selection skill (URL).
Macro hedge funds: systematic beat discretionary
The exhibit below summarizes the performance of the systematic (left) and discretionary (right) macro funds. The authors report factor loadings and alphas (Panel A), a factor-based return attribution, appraisal ratios (alpha divided by residual standard deviation), and cumulative performance over time (both unadjusted and risk-adjusted).
What does the data tell us? Panel A shows that systematic macro funds generate a substantial and mildly significant alpha of 4.85% per year against the most comprehensive factor model. Their discretionary peers only achieve an insignificant alpha of 1.57% per year. More alpha is better than less, so this point goes to systematic. Interestingly, the relevant risk factors of the two groups are quite related, but not totally similar. Systematic funds load significantly on the bond market, momentum (URL), and the volatility factor (the funds tend to do well when volatility rises). The discretionary funds, in contrast, also have significant loadings on the equity market and FX carry.
The R2 of the regressions further show that the risk factors explain more of discretionary macro funds’ returns (25%) than they do for systematic funds (16%). What does this mean? In very simple (and statistically incorrect) terms, discretionary macro hedge funds tend to have more exposure to the well-known factors than their systematic peers. All else equal, this is also a point for systematic. Hedge fund investors shouldn’t want to have exposure to simple factors. You can easily invest in stocks, bonds, credit, and the other factors yourself. You don’t need a well-paid hedge fund manager for that.
The return attribution in Panel B also shows this with specific numbers. The systematic funds deliver an average annual performance of 5.01% and only 0.15%-points of that come from the known factors. For discretionary funds, the factors explain 1.28%-points of the average annual performance of 2.86%. Once again, a hedge fund investor (and manager) should strive for as much alpha as possible. After all, that is the ultimate reason for hedge funds’ existence. Deliver returns from sources that we cannot easily access ourselves.
Overall, discretionary macro hedge funds lose against their systematic peers both unadjusted and risk-adjusted in this sample period. In addition to that, systematic funds also had higher appraisal ratios (0.44 vs. 0.35) which suggests that they squeezed out more alpha per unit of risk. Needless to say, this is exactly what we want to have – more alpha at less risk.
Equity hedge funds: a draw between systematic and discretionary
The exhibit above shows the same performance data now for systematic and discretionary equity hedge funds. Before going into the comparison, however, there is a striking observation for both groups. The factor loadings on the equity market are with 0.42 for systematic and 0.62 for discretionary funds substantial and significant. Why is this striking? Well, the name hedge fund originally comes from the idea that those funds are hedged to market movements and still make money. At least in aggregate and for this sample period, this value proposition doesn’t hold up.
Is this a new insight? Absolutely not. The scope of (equity) hedge funds broadened considerably over the last years and many managers who run unregulated equity funds without any attempt to be market-neutral often call themselves hedge fund. In fact, Cliff Asness (a hedge fund manager) already mentioned in 2001 that hedge funds do not hedge as much as their name suggests (URL).
Now back to systematic vs. discretionary. In terms of unadjusted returns, the discretionary equity hedge funds clearly win against their systematic peers. Average annual returns are with 4.09% vs. 2.88% considerably higher and the blue lines in Panel C show the cumulative difference over time.
Adjusted for factor exposures, however, the difference almost disappears (gray lines). Annualized alphas are with 1.11% per year for systematic and 1.22% for discretionary funds almost identical. The authors also mention that this small difference is not statistically significant. As for macro funds, the appraisal ratio is again better for systematic funds (0.35 vs. 0.25). Once again, this indicates that systematic managers achieve more alpha per unit of risk and therefore apparently construct more efficient portfolios.
Based on this data, I believe the Man vs. Machine battle for equity hedge funds is a fair draw. Discretionary funds achieved higher unadjusted returns, but risk-adjusted alphas are de-facto similar. On the other end, systematic funds once again delivered better appraisal ratios.
While hedge fund investors (in theory) should focus on alpha beyond easy-to-get factors, most real-world investors probably still care about unadjusted returns. Rightly so! It is nice that the systematic funds achieved the same risk-adjusted performance as the discretionary, but in the end, their investors made less real-world money and underperformed by about 1.2%-points per year. From my own experience, I can tell you that most asset owners ultimately prefer money in their bank account over beautiful regression intercepts. So even though the systematic funds are comparable in terms of alpha, I suspect that the discretionary had the easier run in reality.
Systematic and discretionary funds are quite similar
The authors next examine systematic and discretionary hedge funds from another angle. The following exhibit shows a correlation matrix of both unadjusted and risk-adjusted returns for all four fund categories.
In statistical terms, the differences between systematic and discretionary managers are quite small for both macro and equity hedge funds. The correlation of unadjusted returns is 0.72 for macro funds and even 0.89 for equity funds. The coefficients for risk-adjusted returns are also quite high at 0.77 and 0.63 respectively.
What does this mean? Well, it suggests that at least in aggregate and for this sample period, the returns of systematic and discretionary managers are quite related. Stated differently, discretionary and systematic funds are probably more alike than many people initially think. This result is in line with other studies which show that many discretionary investors implicitly also bet on factor premiums.4A good example for this is Buffett’s Alpha by Frazzini et al. (2018) in AgPa #43, or Superstar Investors by Brooks et al. (2019). The implementations (discretionary vs. systematic) are different, but the underlying philosophies (profitable factors) are often quite similar.
There are also important takeaways for investors. We have seen that both systematic and discretionary funds delivered decent performance. And while there is considerable correlation among them, they are not perfectly correlated. So there is some scope for diversification. The idea is simple. It is better to have two different investments that deliver 5% per year than having just a single one.
This argument gets even stronger when we look across hedge fund categories. Quite unsurprisingly, the correlation between macro and equity funds is considerably lower than the correlation within each category. The unadjusted returns of systematic macro funds are for example even uncorrelated to both types of equity funds. So if you can pick decent macro and equity hedge funds, the combination of the two may be even better than any single one of them alone. As we will see in the following section, however, this is a big if.
Hedge fund investing is more difficult than averages suggest
In the final part of the paper, the authors mention and examine an important limitation of their analysis. All results so far are based on a hypothetical aggregate index for each hedge fund category. In my view, this is a perfectly sensible approach to compare the overall performance of systematic and discretionary managers. In practice, however, such an index is most likely not investable and investors are left with the challenge of picking hedge funds in each category.5There are of course more diversified products like fund of funds, but they almost certainly do not cover the full HFR universe. In addition to that, not all hedge funds are listed on HFR anyway. And the list of practical difficulties goes on and on… To do that, investors must go beyond averages.
The exhibit above provides some insights about the distribution of hedge fund returns. I will not comment on every single number, but I believe the results are quite striking. The interquartile-ranges for unadjusted annualized returns (Panel A) range from 5.54% to 7.65%. In addition to that, the 25%-quantiles are mostly between 0% and 1%. What does this tell us? Well, if you did a poor selection and ended up with bottom-quartile hedge funds, your performance was quite dismal. On the contrary, there are high rewards for selecting top-quartile funds. So there is quite substantial return dispersion in the hedge fund industry. This is again no new insight and the pattern is very similar or even stronger for Sharpe ratios, alphas, and appraisal ratios.
I repeat myself, but the table highlights a very important feature of hedge fund investments. Unlike for equities or bonds, there is no simple passive benchmark for hedge funds. You simply cannot invest in the average hedge fund performance and are always left with the challenge of picking individual funds. This is generally not a problem, but you probably should have some selection skill to not end up in the bottom-quartile of the distribution.
Conclusions and Further Ideas
In my view, there are three important takeaways from the study. First, there is no obvious winner of the Man vs. Machine battle in this hedge fund sample. Having said that, the skepticism against systematic funds seems unjustified. Systematic macro hedge funds delivered better unadjusted and risk-adjusted returns than their discretionary peers. For equity hedge funds, systematic managers underperformed, but generated about the same alpha when properly adjusted for risk-factors.
Second, it seems to me that systematic funds are somewhat better in controlling risks and construct more efficient portfolios (higher appraisal ratios). On the other hand, discretionary equity funds come with considerably better unadjusted returns which are probably the ultimate reality for most investors.
Third, the data suggests that systematic and discretionary funds are more alike than many people may initially think. In my view, this is no real surprise as the philosophy of systematic and discretionary portfolio managers is often similar, just implemented differently.
I like the paper for its (back then) novel approach to classify hedge funds into systematic and discretionary and for the rigor yet applied framework to evaluate returns. It would be interesting to do a follow-up with more recent data to see if the patterns persisted over the last 7 years.
Nonetheless, I want you to keep in mind that the results are mostly hypothetical as they are based on broad and probably non-investable averages of hedge funds. The authors are well-aware of this limitation and for the purpose of showing aggregate performance, their approach is perfectly fine. In reality, however, there is the inevitable challenge of manager selection and the portfolio of any single real-world hedge fund investor looks probably quite different than the results here.
Also note that the author team had relations to the Man Group at the time of publication. Among other things, the Man Group is known for offering systematic hedge fund strategies. I have no doubts about the intellectual honesty of the authors. The paper is peer-reviewed and published in the Journal of Portfolio Management, so I am very confident that there is no issue here. Nonetheless, I believe it is always important to check where a paper comes from.
As you know from my previous work on Man vs. Machine in asset management (see AgPa #21 and here), I generally agree with the authors. I also believe systematic and discretionary portfolio managers are more alike than many investors (or even the managers themselves) make them. Finally, the evidence in this paper once again reinforces my view on systematic vs. discretionary investing. None of the two is inherently better than the other and both can lead to success.
- AgPa #82: Equity Risk Premiums and Interest Rates (2/2)
- AgPa #81: Equity Risk Premiums and Interest Rates (1/2)
- AgPa #80: Forget Factors and Keep it Simple?
- AgPa #79: The Momentum OGs – 30 Years Later
This content is for educational and informational purposes only and no substitute for professional or financial advice. The use of any information on this website is solely on your own risk and I do not take responsibility or liability for any damages that may occur. The views expressed on this website are solely my own and do not necessarily reflect the views of any organisation I am associated with. Income- or benefit-generating links are marked with a star (*). All content that is not my intellectual property is marked as such. If you own the intellectual property displayed on this website and do not agree with my use of it, please send me an e-mail and I will remedy the situation immediately. Please also read the Disclaimer.
Endnotes
| 1 | It is always surprising to me that some people do not trust a transparent quantitative model, but have no trouble to trust whatever human being. But maybe I am the strange person in this respect. |
|---|---|
| 2 | In an appendix, the authors provide additional details about their methodology. |
| 3 | We cannot criticize the ancient Egyptians that they haven’t used modern cranes to build the pyramids faster because there were no such cranes back then… |
| 4 | A good example for this is Buffett’s Alpha by Frazzini et al. (2018) in AgPa #43, or Superstar Investors by Brooks et al. (2019). |
| 5 | There are of course more diversified products like fund of funds, but they almost certainly do not cover the full HFR universe. In addition to that, not all hedge funds are listed on HFR anyway. And the list of practical difficulties goes on and on… |