Friday, February 02, 2018

FX Order Flow as a Predictor

Order flow is signed trade size, and it has long been known to be predictive of future price changes. (See Lyons, 2001, or Chan, 2017.) The problem, however, is that it is often quite difficult or expensive to obtain such data, whether historical or live. This is especially true for foreign exchange transactions which occur over-the-counter. Recognizing the profit potential of such data, most FX market operators guard them as their crown jewels, never to be revealed to customers. But recently FXCM, a FX broker, has kindly provided me with their proprietary data, and I have made use of that to test a simple trading strategy using order flow on EURUSD.

First, let us examine some general characteristics of the data. It captures all trades transacted on FXCM occurring in 2017, time stamped in milliseconds, and with their trade prices and signed trade sizes. The sign of a trade is positive if it is the result of a buy market order, and negative if it is the result of a sell. If we take the absolute value of these trade sizes and sum them over hourly intervals, we obtain the usual hourly volumes (click to enlarge) aggregated over the 1 year data set:

It is not surprising that the highest volume occurs between 16:00-17:00 London time, as 16:00 is when the benchmark rate (the "fix") is determined. The secondary peak at 9:00-10:00 is of course the start of the business day in London.

Next, I compute the daily total order flow of EURUSD (with the end of day at New York's midnight), and I establish a histogram of the last 20 days' daily order flow. I then determine the average next-day return of each daily order flow quintile. (I.e. I bin a next-day return based on which quintile the prior day's order flow fell into, and then take the average of the returns in each bin.) The result is satisfying:

We can see that the average next-day returns are almost monotonically increasing with the previous day's order flow. The spread between the top and bottom quintiles is about 12 bps, which annualizes to about 30%. This doesn't mean we will generate 30% annualized returns, since we won't be able to arbitrage between today's return (if the order flow is in the top or bottom quintile) with some previous day's return when its order flow was in the opposite extreme. Nevertheless, it is encouraging. Also, this is an illustration that even though order flow must be computed on a tick-by-tick basis (I am not a fan of the bulk volume classification technique), it can be used in low-frequency trading strategies.

(One may be tempted to also regress future returns against past order flows, but the result is statistically insignificant. Apparently only the top and bottom quintiles of order flow are predictive. This situation is actually quite common in finance, which is why linear regression isn't used more often in trading strategies.)

Finally, one more sanity check before backtesting. I want to see if the buy trades (trades resulting from buy market orders) are filled above the bid price, and the sell trades are filled below the ask price. Here is the plot for one day (times are in New York):

We can see that by and large, the relationship between trade and quote prices is satisfied. We can't really expect that this relationship holds 100%, due to rare occasions that the quote has moved in the sub-millisecond after the trade occurred and the change is reported as synchronous with the trade, or when there is a delay in the reporting of either a trade or a quote change.

So now we are ready to construct a simple trading strategy that uses order flow as a predictor. We can simply buy EURUSD at the end of day when the daily flow is in the top quintile among its last 20 days' values, and hold for one day, and short it when it is in the bottom quintile. Since our daily flow was measured at midnight New York time, we also define the end of day at that time. (Similar results are obtained if we use London or Zurich's midnight, which suggests we can stagger our positions.) In my backtest, I have subtracted 0.20 bps commissions (based on Interactive Brokers), and I assume I buy at the ask and sell at the bid using market orders. The equity curve is shown below:

The CAGR is 13.7%, with a Sharpe ratio of 1.6. Not bad for a single factor model!

Acknowledgement:  I thank Zachary David for his review and comments on an earlier draft of this post, and of course FXCM for providing their data for this research.


Industry update

1) Qcaid is a cloud-based platform that provides traders with backtesting, execution, and simulation facilities. They also provide servers and data feed.

2) How Cadre Uses Machine Learning to Target Real Estate Markets.

3) Check out Quantopian's new tutorial on getting started in quantitative finance.

4) A new Matlab-based backtest and live trading platform for download here.

5) A nice resource page for open source algorithmic trading tools at QuantNews.

My Upcoming Workshops

February 24 and March 3: Algorithmic Options Strategies

This online course focuses on backtesting intraday and portfolio option strategies. No pesky options pricing theories will be discussed, as the emphasis is on arbitrage trading.

June 4-8: London workshops

These intense 8-16 hours workshops cover Algorithmic Options Strategies, Quantitative Momentum Strategies, and Intraday Trading and Market Microstructure. Typical class size is under 10. They may qualify for CFA Institute continuing education credits. (Bonus: nice view of the Thames, and lots of free food.)


David Bryant said...

Thanks for this Ernie, thought provoking stuff. I have always ignored volume in my systems, may have to re-visit. I wonder if the order flow system would transfer to intra-day movements? Say hourly orderflow?


Ernie Chan said...

Thanks, David. Order flow should translate to intraday return prediction, but my study of hourly flow on EURUSD didn't work.


Anonymous said...

Interesting article, though I don't think you can draw too many conclusions on a 1 year backtest. Does the strategy hold on other FX rates?



Ernie Chan said...

Hi Daniel,
Agreed -unfortunately there is only 1 year of data available. This also doesn't work as well for other rates.

TC said...

The average Joe can't obtain this data from an OTC broker.
Would it work using tick data of CME FX futures (classifiy-ing buys or sells by either a big data job of matching up timestamped actual trades with the just prior bid/ask ... or determining buys abd sells by the inferior, but more tractable, "bulk classification")

Ernie Chan said...

Yes, if you have true futures tick feed (not like the feed provided by IB which is sampled at 250ms), you can apply the tick rule to estimate order flow. If you are paying for the expensive direct MDP feed from CME, they will tell you the order flow explicitly (via the aggressor tag on each trade).

I have determined that results from bulk volume classification are inferior to a tick-by-tick computation, hence I have disavowed it.


Eduardo Gonzatti said...
This comment has been removed by the author.
Eduardo Gonzatti said...

Thank you for the article, Dr Chan!

A quick question for you, regarding the logic as a whole, you summed all the trades into net values of volume, considering that sell trades were negative and buy trades were positive, sorted then into quintiles over a moving window of 20 trading days and then realized that there was almost a linear relationship between the top quintiles (buy skewed) and positive returns on the next trading day, and vice versa for the lowest quintile. Right?


Edit for typos

Ernie Chan said...

Hi Eduardo,
You summarized correctly!

Eduardo Gonzatti said...

Tks Dr! Definitely worthy of further testing.

Unknown said...

Thanks. It is insightful. I think only Oanda supply order flow information but it I don think it can be downloaded. Maybe retail traders have to trade manually.

Ernie Chan said...

Good to know about Oanda - thanks.


Ernie Chan said...

By the way, FXCM told me that they can offer anyone free historical order flow data for 6 months in 2017. Just email


M said...

thanks for an interesting post. this seems as a strong indicator that could be turned into a high Sharpe Ratio strategy even on daily data. I think I have seen papers on predictive power of order flow
I have a suspicion though, that the proprietary order flow would correlate strongly with the buying pressure on the order book, ratio of the sums of limit sizes on the ask and bid books. That would mean it can be estimated from publicly available data.

Ernie Chan said...

Hi M,
Yes, order book imbalance has also been shown to be predictive of future price change.
See Cartea 2015 (the HFT book on my Recommended Books list on right sidebar).

Brad Schrage said...

Hi Ernie,

I enjoyed your class you taught for MSPA. Interesting post. Quandl also apparently offers data from CLS which may be an even better indicator for volume since it takes into account large institutional trades. I don't subscribe, but the service does seem to offer volume hourly.


Ernie Chan said...

Hi Brad,
You are right about Quandl. The cost, however, is beyond the reach of most retail investors.

R said...

Inform yourselves about the moral problem of trading :

M said...

Hi Ernie,
By buying pressure I meant the sum of volume over all order book levels, or or restricted to 10 best or similar. In my experience almost as strong an indicator as order imbalance, with a bit different dynamics.

Ernie Chan said...

Hi M,
Presumably you are only summing all the buy orders for a buying pressure?