TL;DR – There really is a low volatility anomaly in the SPY data; low volatility today predicts low volatility tomorrow and risk-adjusted returns are higher investing daily in the lower vol half of predicted market days. Same data, new analysis, better graphs and you’ll see it too.
First up, a mea culpa. I misused my shallow understanding of stats::lag() in the last post and ended up future-peeking badly. Actually, not so much future-peeking as correlating my signal to returns from two days earlier, and telling you that the returns were from two days later. Needless to say, that made the graphs look really good, and they really shouldn’t have looked that good. This was an amateurish mistake on my part. I claim to be an amateur quant, but I intend to avoid amateurism as much as I can. You can read the fix and the comments in the code in that post (Part 9) or at the Github.
In that previous post, I presented a plot that was really quite remarkable as far as showing the amazing low volatility anomaly in index returns:
The annotations are not in the original. As one insightful reader said in an email, these results are “a bit unbelievable”. And I agree! I think I was so excited to get the graphics right that I really jumped the gun and pushed out the post without properly vetting the results and verifying them in other ways. So I got the lag wrong (directionally wrong: I lagged returns +2 days instead of the volatility signal +2 days), and thus correlated volatilities with the Open-Open returns from two days past instead of two days hence. So instead of showing what the volatilities predicted, I graphed what the volatilities had measured! Not terribly useful in this context.
So let’s see the new graphs, with a proper lag.
And… boom! Our wonderful low volatility anomaly has disappeared completely. That’s more like it should be; if any sort of anomaly had been as strong as I put forward in the first version it would have been traded into oblivion long ago. You can run the code and see that the low vol anomaly for 2-day lagged Op-Op returns has also disappeared (more or less) for the Rogers-Satchell, Parkinson, and Garman-Klass Yang-Zhang measures of volatility.
Now I’m still claiming the low vol anomaly for the SPY exists… What we need to do is risk-adjust these daily returns. So first, let’s take another look at returns in volatility bins, and take the annualized standard deviation of those returns. The first difference in the next graph set is the binning: we’re using 2 bins, 10 bins and 100 bins instead of 10,30,100 as we did above. So the 2 bin graphs (at top) represent the data lower than the median vol value, and the data higher than the median. The second difference is, instead of mean daily returns (a very small number) we’re looking at the sum of SPY lagged returns over each bin. So for two bins, that’s about 1690 daily returns summed in each bin (we’re using log returns SO THAT we can sum them). On the left: sum of daily returns by vol bin. On the right: standard deviation (annualized by sqrt(252)) by vol bin.
Nice! The summed return bins on the left are very similar to the daily return bins above, but the really amazing (and I think reasonable and proper) result is the gorgeously gradual increase in standard deviation as the vol bins progress from low (left) to high vol (right). Lots of studies have shown the vol regimes are fairly persistent, that low volatility in the market today predicts low volatility in the market tomorrow (this ref, that ref, other ref). But it’s quite nice to see this relationship in action, to see that the bins of lagged returns have volatilities that track the volatilities from the previous day. It’s a little confusing perhaps, because they’re two different volatilities (bins derive from C2C SPY volatility measured on signal day, and the data inside the bins are the annualized std dev for the Open-Open returns with a 2-day lag from the signal), but it is a really nice relationship to see.
So the only thing left to do is show the data as a risked return. I’m not sure using a Sharpe ratio or other formal measure is proper here, since we’re binning data and completely divorcing it from any kind of time series relationship. So I’ve simply divided the summed returns per bin on the left column by the ann std dev of those binned returns on the right column to get a relative risked return measure. The graph below shows the risked returns in a new third column, so that you can see the data all together.
Woot! There we have it, the risked returns column shows that the low vol half of the returns shows a higher risk-adjusted return than the higher vol half of the returns. In other words, our low volatility anomaly is back! Not for real returns, but definitely for risk-adjusted returns. It’s easiest to see in the top row (two bins, low vol half wins), but squinting you can eyeball the same result in the lower two rows (10 bins and 100 bins).
Let’s triple check this by presenting the same graph, but using the other three vol measures, with labels RS for Rogers-Satchell, Park for Parkinson, and GKYZ for Garman-Klass Yang-Zhang (spoiler: all three check out, though it’s a squeaker for GKYZ):
So I think it’s fair to say that there is no anomaly in real returns, but a fairly strong anomaly in 3 of the four volatility measures (it’s a tossup for GKYZ). And there’s an extremely strong volatility begets volatility relationship across all four vol measures. As one further test, we can take an average volatility across all four measures, and run one more analysis on that average value:
This plot with the average volatility of all four measures is very similar in pattern and magnitude to the individual four measures we have been using. In particular, the low volatility anomaly for risk-adjusted returns is consistent across four of the five volatility metrics shown, and a dead heat for the fifth.
Side note: An interesting (to me) aspect of these plots is the remarkable variability between vol measures in the 10-bin data sets. I had suspected that there might be some regularity to them, such as the first three bins (lowest vol bins) might on average have higher summed returns than the next two, or at least that there would be some pattern between vol measures. But I can’t see anything like regularity at all. Each vol measure has a unique and likely random pattern of highs and lows among the categories of the 10 (and 100) bin analyses. Apparently small differences in vol measures can lead to slightly different sets of returns which may exhibit quite large differences in summed returns overall. I may investigate this further to support/reject this hypothesis, but it’s not critical for me at this time.
Another aspect of this to consider is whether the anomaly holds up across different lookback periods. So far we’ve only considered the 20day lookback period, which although a commonly used metric is hardly the only one around. In our next set of charts we’ll explore all the daily lookback periods from 2-30 days in length to examine the persistency of this anomaly across this important variable. Here is a plot of two bar charts, representing the risk-adjusted summed returns for C2C volatility across this range of lookback periods, the upper bar chart (blue bars) represents the low volatility half of the data while the lower (orange) bars represent the higher vol half of the data.
So, our initial analysis of the 20-day lookback was clearly not cherry picked to show great results, many of these low vol bars show a greater contrast with their high vol counterpart. Across the lookback period, the low vol anomaly is clear. Only one blue bar (29-day) sneaks below a value of 3.0 on the relative risk-adjusted returns scale, it is still higher than its equivalent 29-day orange, high volatility bar.
Here are the other four comparison charts:
No big surprises. All the vol measures behaved similarly, the low volatility anomaly exists across all of them, and is remarkably strong… roughly twice (or better) the risk-adjusted returns by this simple measure for the low vol half of returns versus the high vol half. Interestingly, the 20-day lookback period in the Garman-Klass Yang-Zhang volatility that we said was a “tie” (ie, no clear low vol anomaly) was itself an anomaly among the GK-YZ dataset: the 20-day was the single lowest low vol risk-adjusted return for that vol measure, and overall the GK-YZ vol measure performed well for documenting the low vol anomaly across all lookback periods, with only the C2C measure having a lower (4.23:2.13) ratio of low vol to high vol average risk-adjusted returns.
Since the beginning of this project I’ve had an intuitive sense that there is a real low-vol anomaly in this index, that a low-vol approach for a quantitative investment strategy makes sense. Now, with this analysis, I feel more strongly than before that there is a statistical underpinning for this approach. I expect to return with my next posts to striving for the elusive secret sauce that will juice my strategy returns above that of buy and hold SPY on a real return basis, and not only on a risk-adjusted basis.
There are two more things to note here: avoiding future-peeking, and pursuing a further analysis like this one.
The first involves a minor future-peeking bugaboo in this type of analysis. If you were to take this statistical analysis, and say “I’ll use this to make a backtest of the low vol anomaly strategy, using the median volatility value as my threshold, only investing on days when the signal flashes low vol”, you risk a minor future-peeking error. This would happen because we used this analysis of the July 2006-Dec 2019 SPY data to find the median volatility signal… using that median value as threshold would be improper for a backtest over this same period because you could NOT have known that median value back in 2006 when starting the strategy. Instead, one should run a similar analysis of earlier SPY data to find a median value (or pick a threshold on some other basis), and use that threshold. Never look ahead to find a parameter to carry back into the past; this creates a “Biff Tanner” logical error that only works in fiction.
The second item of note is that I should really do just that, take a deeper dive into the past to examine the low vol anomaly and its history. SPY only goes back so far, but the S&P 500 goes back farther. I’ll try to put that on my todo list for the not-too-distant future. Thanks for reading. I also hope to upgrade my WordPress approach so that I can use live plotly graphs here, so you can see the data underlying the graphs instead of just having static PNGs to look at.
I’ll clean up the R codes I used to generate all these graphs and post them soon here and on the Github, but meanwhile I’ll get this overdue post off my plate. Thanks for reading, and special thanks to Ted, Philipp and Smoosh for comments, ideas, code, and valuable constructive criticism.
Author update 12-Sept-2023: minor update to the AVE4 risked returns (black and white) figure, and here is the updated code (also available at the Github).
### rv_vs_rtns_0.0.2.R by babbage9010 and friends
# update code from v1 to produce plots for blog post:
# https://babbage9010.wordpress.com/2023/09/11/there-is-a-low-vol-anomaly-in-spy/
# Not intuitive, but it works, like this:
# 1) to reproduce the B&W plots (1 lookback period)
# a) set "switchy" (Line 118) to FALSE
# b) choose vol signal as x_dat (L 48-52)
# c) use numcols (L 73) to choose 2 or 3 column plot
#
# 2) to reproduce the color, lookback spectrum plots
# a) set "switchy" (L 118) to TRUE
# b) choose vol signal as x_dat (L 178-183)
# c) play with maximum lookback if you want (L 163)
# d) to exactly duplicate Y scales, see notes (L 126-137)
# and set them manually, e.g., (0.0,8.4)
#
### released under MIT License
# Step 1: Load libraries and data
library(quantmod)
library(PerformanceAnalytics)
start_date <- as.Date("2006-07-01") #SPY goes back to Jan 1993
end_date <- as.Date("2019-12-31")
getSymbols("SPY", src = "yahoo", from = start_date, to = end_date, auto.assign = FALSE) -> gspc_data
pricesAd <- na.omit( Ad(gspc_data) )
pricesOp <- na.omit( Op(gspc_data) )
pricesCl <- na.omit( Cl(gspc_data) )
pricesHi <- na.omit( Hi(gspc_data) )
pricesLo <- na.omit( Lo(gspc_data) )
# choose one of those here
trade_prices <- pricesOp
signal_prices <- pricesAd
bench_prices <- pricesAd
#plot it
roc1 <- ROC(signal_prices, n = 1, type = "continuous")
lookbk <- 20
rv <- runSD(roc1, n = lookbk) * sqrt(252)
rs <- volatility(gspc_data, n=lookbk, calc="rogers.satchell")
gkyz <- volatility(gspc_data, n=lookbk, calc="gk.yz")
park <- volatility(gspc_data, n=lookbk, calc="parkinson")
avgvol <- (rv+rs+gkyz+park) / 4
#choose one of these to uncomment
x_dat <- rv; x_dat_label = "C2C"
#x_dat <- rs; x_dat_label = "RS"
#x_dat <- gkyz; x_dat_label = "GKYZ"
#x_dat <- park; x_dat_label = "Park"
#x_dat <- avgvol; x_dat_label = "AVE4"
vollabel = paste(x_dat_label," ",lookbk, "d vol",sep="")
#y - SPY open lagged returns
roc_trade1 <- ROC(trade_prices, n = 1, type = "continuous")
returns_spy_open <- roc_trade1
returns_spy_open <- stats::lag(returns_spy_open, -2) #lag is -2 to properly align with vol signal
# see v1 code for details
y_dat <- returns_spy_open #lagged, as above
#rid of NAs to even up the data (tip: this avoids NA-related errors)
dat <- as.xts(y_dat)
dat <- cbind(dat,x_dat)
dat <- na.omit(dat)
datcore <- coredata(dat)
# Set for six graphs, or nine
# to recreate the black&white plots in the blog post
numrows <- 3
numcols <- 2 #either 2 or 3 columns please
# Set up 2x2 graphical window
par(mfrow = c(numrows, numcols))
#qnums <- c(10,30,100) #number of quantiles (buckets) (eg 10 for deciles)
qnums <- c(2,10,100) #number of quantiles (buckets) (eg 10 for deciles)
for(q in 1:3){
qnum <- qnums[q]
xlabel = paste(vollabel," with ",qnum," vol buckets",sep="")
decs <- unname(quantile(datcore[,2], probs = seq(1/qnum, 1-1/qnum, by = 1/qnum)))
decs[qnum] <- max(decs) + 1
decsmin <- min(decs) - 1
#loop through volatility buckets to get summed and risked returns
sums <- c()
annvols <- c()
riskdRtns <- c()
for(i in 1:qnum){
# datx = data segment from x_dat[,1] (returns) to summarize
lowbound <- ifelse(i == 1, decsmin, decs[i-1])
hibound <- decs[i]
datx <- ifelse( datcore[,2] >= lowbound & datcore[,2] < hibound, datcore[,1], NA)
datx <- na.omit(datx)
sums[i] <- sum(datx)
annvols[i] <- sd(datx) * sqrt(252)
riskdRtns[i] <- sums[i] / annvols[i]
#print( paste("decile",i,"mean:",means[i],"vol range:",lowval,"-",hival) )
}
barplot(sums,xlab=xlabel,ylab="Sum of SPY daily returns (log)",main="Sum of daily SPY log returns per volatility bucket",sub="low vol on left, high vol on right")
if(numrows >= 3){
#barplot(wins,xlab=xlabel,ylab="SPY mean daily return",main="Daily win % for SPY returns per vol bucket",sub="low vol on left, high vol on right")
#abline(h=c(0.54),col="red")
barplot(annvols,xlab=xlabel,ylab="SPY annualized standard deviation",main="Std Dev (ann) for SPY returns per vol bucket",sub="low vol on left, high vol on right")
if(numcols >= 3){
barplot(riskdRtns,xlab=xlabel,ylab="SPY risked returns (relative)",main="Relative risked returns per volatility bucket",sub="low vol on left, high vol on right")
}
}
}
#
# HERE BE THE COLOR BAR CHARTS
#
# Use this switch to turn on(T)/off(F) this section
# for some (R Studio?) reason it also seems to turn off the earlier plots
# so you only get one or the other plots based on this switch
#
switchy <- TRUE #FALSE OR TRUE
if(switchy){
# function generates the bar graph from loop below
barr = function(wha="net",whichdec,signalname,value=""){
x <- lbaks
if(wha=="net") { y <- decnets[whichdec,]
titl <- paste(signalname,": net returns per quantile by lookback (Y scale varies)")
ystuff <- list(title = "net return for Vol bucket", range = c(min(decnets[whichdec,]),max(decnets[whichdec,]))) #this automates the range max
#ystuff <- list(title = "net return for Vol bucket", range = c(0.0,0.6)) #this lets you specify the range
}
if(wha=="vol") { y <- decvols[whichdec,]
titl <- paste(signalname,": avg volatility per quantile by lookback (Y scale varies)")
ystuff <- list(title = "volatility for Vol bucket", range = c(min(decvols[whichdec,]),max(decvols[whichdec,])))
#ystuff <- list(title = "volatility for Vol bucket", range = c(0.0,0.25)) #specified
}
if(wha=="rat") { y <- decrats[whichdec,]
titl <- paste(signalname,": Return/Volatility per quantile by lookback (Y scale varies)")
ystuff <- list(title = "reward/risk ratio for Vol bucket", range = c(min(decrats[whichdec,]),max(decrats[whichdec,])))
#ystuff <- list(title = "reward/risk ratio for Vol bucket", range = c(0.0,8.4)) #specified
}
text= y
data = data.frame(x=factor(x,levels=x),text,y)
fig <- plot_ly(
data, name = paste("Q",whichdec,value), type = "bar",
x = ~x, textposition = "outside", y= ~y, text =~text)
fig <- fig %>%
layout(title = titl,
xaxis = list(title = "Volatility lookback period (days)"),
#not working: yaxis = list(title = paste("SPY lagged O-O net return for Vol Decile ",whichdec)),
yaxis = ystuff,
autosize = TRUE,
showlegend = TRUE)
return(fig)
}
# Now set dec (quantiles) and lookbacks range
# dec is the number of quantiles to use
# (Y scale hard to read when dec > 10)
dec <- 2
#lookbacks plot ok as high as 100 or 200
lbaks <- c(2:30) #lookbacks
decnets <- c() #matrix of decile Net Return data
decvols <- c() #matrix of decile returns volatility data annualized
decrats <- c() #matrix of decile Ratios of returns/volatility
n <- 0
for(lb in lbaks){
qnum <- dec #how many quantiles this loop
n <- n+1
#use roc1 as above (non-lagged)
rv <- runSD(roc1, n = lb) * sqrt(252)
rs <- volatility(gspc_data, n=lb, calc="rogers.satchell")
gkyz <- volatility(gspc_data, n=lb, calc="gk.yz")
park <- volatility(gspc_data, n=lb, calc="parkinson")
avg4 <- (rv+rs+gkyz+park) / 4
#choose your vol measure here! using #
x_dat2 <- rv; x_dat_label <- "Close2Close"
#x_dat2 <- rs; x_dat_label <- "Rogers-Satchell"
#x_dat2 <- park; x_dat_label <- "Parkinson"
#x_dat2 <- gkyz; x_dat_label <- "GK-YZ"
#x_dat2 <- avg4; x_dat_label <- "Average4"
vollabel = paste(x_dat_label," ",lb, "d vol",sep="")
xlabel = paste(vollabel," with ",qnum," vol buckets",sep="")
datp <- as.xts(y_dat) #y_dat is lagged, as above
datp <- cbind(datp,x_dat2)
datp <- na.omit(datp)
datcore2 <- coredata(datp)
decs2 <- unname(quantile(datcore2[,2], probs = seq(1/qnum, 1-1/qnum, by = 1/qnum)))
decs2[qnum] <- max(decs2) + 1
decsmin <- min(decs2) - 1
#loop through volatility buckets to get returns data
netgain2 <- c()
annvols <- c()
rvratios <- c()
for(i in 1:qnum){
# datx = data segment from x_dat[,1] (returns) to summarize
lowbound <- ifelse(i == 1, decsmin, decs2[i-1])
hibound <- decs2[i]
datx <- ifelse( datcore2[,2] >= lowbound & datcore2[,2] < hibound, datcore2[,1], NA)
datx <- na.omit(datx)
netgain2[i] <- sum(datx)
annvols[i] <- sd(datx) * sqrt(252)
rvratios[i] <- netgain2[i] / annvols[i]
}
decnets <- cbind(decnets,netgain2)
decvols <- cbind(decvols,annvols)
decrats <- cbind(decrats,rvratios)
}
figs1 <- list()
figs2 <- list()
figs3 <- list()
for(x in 1:dec){
figs1[x] <- barr("net",x, x_dat_label, paste(":",round(mean(decnets[x,]),2)))
figs2[x] <- barr("vol",x, x_dat_label, paste(":",round(mean(decvols[x,]),2)))
figs3[x] <- barr("rat",x, x_dat_label, paste(":",round(mean(decrats[x,]),2)))
print(paste("Dec:",x,"Sum:",sum(decnets[x,]),"Avg:",mean(decnets[x,])))
#print(paste("Dec:",x,"Vol:",sum(decvols[x,]),"S/V:",sum(decnets[x,])/sum(decvols[x,])))
print(paste("Dec:",x,"Vol:",sum(decvols[x,]),"S/V:",mean(decrats[x,])))
}
figgy <- subplot(figs1, nrows = length(figs1), shareX = TRUE)
ziggy <- subplot(figs2, nrows = length(figs2), shareX = TRUE)
biggy <- subplot(figs3, nrows = length(figs3), shareX = TRUE)
print(figgy) #figgy is the summed (net) returns per quantile
print(ziggy) #ziggy is the std dev annualized
print(biggy) #biggy is the risk-adj returns used in the blog post
} #end of switchy statement
par(mfrow = c(1, 1))
babbage9010 
Leave a comment