Quant log | 2 min read

The M6 Competition: Battle of the Forecasting Models

Antoine Ledoux, Quantitative Research Analyst
December 7, 2022

These days new Machine Learning techniques seem to be coming out every other month claiming to be the next big thing and it is hard to distinguish the good from the bad. So, what better way to compare these models than to have them compete against one another to see which one is indeed the best in time series forecasting (our bread and butter).

This is how the M competitions came about. Brought to life in 1982 by Spyros Makridakis (who gave his name to the event, the Makridakis Competitions, now known as M Competitions), the goal of these competitions is to compare all-time series forecasting methods known to date and see which one performs best. The main conclusion from the first M Competitions (hosted in 1982, 1993 and 2000) was that “statistically sophisticated or complex methods do not necessarily provide more accurate forecasts than simpler ones”.

However, this changed somewhat in the 4th competition (M4) when a data scientist from Uber Technologies surprised the jury with a hybrid approach utilizing both Statistical and Machine Learning methods, beating the benchmark by a whopping 10%. For the first time, the best approaches were hybrid ones: classical statistical models like EWMA or ARIMA were used to create features that were then fed into a more complex Machine Learning model. In M5, the top performers were pure Machine Learning models, beating all statistical benchmarks and their combinations. The latest competition (M6) is still on-going and relevant to our work at Graham, as it will compare forecasting models on price data for 50 S&P500 stocks and 50 international ETFs. May the best forecasting model win!

DISCLOSURE

This presentation includes statements that may constitute forward-looking statements. These statements may be identified by words such as “expects,” “looks forward to,” “anticipates,” “intends,” “plans,” “believes,” “seeks,” “estimates,” “will,” “project” or words of similar meaning. In addition, our representatives may from time to time make oral forward-looking statements. Such statements are based on the current expectations and certain assumptions of Graham Capital Management’s (“GCM”) management, and are, therefore, subject to certain risks and uncertainties. A variety of factors, many of which are beyond GCM’s control, affect the operations, performance, business strategy and results of the accounts that it manages and could cause the actual results, performance or achievements of such accounts to be materially different from any future results, performance or achievements that may be expressed or implied by such forward-looking statements or anticipated on the basis of historical trends.

This document is not a private offering memorandum and does not constitute an offer to sell, nor is it a solicitation of an offer to buy, any security. The views expressed herein are exclusively those of the authors and do not necessarily represent the views of Graham Capital Management. The information contained herein is not intended to provide accounting, legal, or tax advice and should not be relied on for investment decision making.

Tables, charts and commentary contained in this document have been prepared on a best efforts basis by Graham using sources it believes to be reliable although it does not guarantee the accuracy of the information on account of possible errors or omissions in the constituent data or calculations. No part of this document may be divulged to any other person, distributed, resold and/or reproduced without the prior written permission of GCM.

Subscribe to our Resource Center

By clicking the Button you confirming that you’re agree with our following Terms and Conditions

Subscribe to our Resource Center

Welcome to Graham Capital Management

By entering this website you are agreeing to its Terms of Use.

The risk of loss in trading commodities can be substantial. You should therefore carefully consider whether such trading is suitable for you in light of your financial condition. The high degree of leverage that is often obtainable in commodity trading can work against you as well as for you. The use of leverage can lead to large losses as well as gains.

In some cases, managed commodity accounts are subject to substantial charges for management and advisory fees. It may be necessary for those accounts that are subject to these charges to make substantial trading profits to avoid depletion or exhaustion of their assets. The disclosure document contains a complete description of the principal risk factors and each fee to be charged to your account by the commodity trading advisor (“CTA”). The regulations of the commodity futures trading commission (“CFTC”) require that prospective clients of a CTA receive a disclosure document when they are solicited to enter into an agreement whereby the CTA will direct or guide the client’s commodity interest trading and that certain risk factors be highlighted. This brief statement cannot disclose all of the risks and other significant aspects of the commodity markets. Therefore, eligible investors should carefully study the disclosure document to determine whether such trading is appropriate for you in light of your financial condition. Eligible investors are encouraged to access the disclosure document by contacting Graham, which will be provided  at no additional cost. The CFTC has not passed upon the merits of participating in this trading program nor on the adequacy or accuracy of the disclosure document. Other disclosure statements are required to be provided to you before a commodity account may be opened for you.  By accepting the terms of this statement and entering the site you are confirming your understanding of this statement.

Any “benefit plan investor” (e.g. An IRA or other ERISA investor) investor is hereby deemed to represent to Graham Capital Management (“Graham”) that an independent fiduciary (as defined in the ERISA fiduciary rule) is exercising its independent judgment with regard to such investor’s investment in any Graham managed fund and is aware of and acknowledges and agrees that Graham and its affiliates are relying on the exception set forth in clause (c)(1) of the ERISA fiduciary rule (i.e., the “transactions with independent fiduciaries with financial expertise” exception) with respect to any communications made to the investor or the investor’s fiduciary that are considered recommendations concerning any transaction and such benefit plan investor.

Although the site may include investment-related information, nothing on the site is a recommendation that you purchase, sell or hold any security or other investment, or that you pursue any investment style or strategy.

We do not give any advice or make any representations through the site as to whether any investment is suitable to you or will be profitable.

Nothing on the site is intended to be, and you should not consider anything on the site to be, investment, accounting, tax or legal advice. If you would like investment, accounting, tax or legal advice, you should consult with your own financial advisors, accountants or attorneys regarding your individual circumstances and needs.

The past performance of any investment, investment strategy or investment style is not indicative of future performance.

If you have been provided with a password to access the site you are solely responsible for maintaining the confidentiality and security of your password. You may not disclose your password to any third party. You accept full responsibility for any use of your password. You must notify Graham immediately of any actual or suspected loss, theft or unauthorized use of your password.

We are not obligated to inquire as to the authority or propriety of any use of or action taken under your password. We will not be responsible for any loss to you that arises from such use or action or from your failure to comply with these provisions.

The site, including all content, is provided as is and as available.

We disclaim all representations and warranties, express or implied, of any kind with respect to the site and the content including warranties of merchantability, fitness for a particular purpose and non-infringement of intellectual property and proprietary rights.

Without limiting our general disclaimer, we do not warrant the availability, accuracy, completeness, timeliness, functionality, reliability, sequencing or speed of delivery of the site or the content.