Volatility art and rhythm

Some days ago a very rythmed video was published by Artmeis CM on You Tube on two decades of volatilities and S&P values.

Volatility has been an issue since the studies of Markowitz in the late 50s who, from an operational research perspective, brought the idea of optimizing the risk-return pair, i.e. maximize the return for a given level of risk or minimize the risk for a desired level of return. And "risk" was mapped to the notion of standard deviation or volatility. Now, volatility represents a measure of dispersion around the trend that you are following, and is synonym of "uncertainty" first - today, we know that risk can be much more than just volatility. And therefore, a low volatility does not mean necessarily "good news": a rapidly declining market can show a low volatility if there are strong beliefs about the declining trend.

Since the 70s, volatility then became a key input parameter to the valuation of options, i.e. insurance contracts on the prices of stocks, commodities, currencies, indices,.... Those options need additional inputs/parameters such as the maturity of the contract and the price level where the contract can be exercised. But overall, insurance has a value only because there is uncertainty. And the more you feel risky and you are risk averse about that uncertainty, the higher will be the price you are ready to pay for your insurance contract. On the other hand, the award-winning Black-Scholes-Merton model that was used for option pricing since 1973, was assuming a constant volatility over the life of the contract. And all option traders, statisticians, econometricians knew that, depending on the time window, the measure was changing.

In the 80s, one of them, Robert Engle, appeared working on Autoregressive Conditional Heteroskedasticity models of the volatility, at some point also with Clive Granger in 1987 (Econometrica), and both won the Nobel Prize in Economics in 2003 (Engle and Granger lectures). This would give rise to the ARCH, GARCH, EGARCH, HGARCH, NGARCH, QGARCH, TGARCH....of this World in the years after. More recently, Robert Engle has been the instigator of V-Lab, an online platform producing regularly and automatically estimates for a wide range of time-series. Some nice resources are also available online with the FT.

Finally, a key differentiation must be made between "statistical, historical or realized volatility", whatever the methodology to assess, estimate and predict volatility could be, and "implied volatility". The CBOE has since then created the VIX, an index of implied volatilities from S&P options and others, assumed to represent the expected volatility over the next 30 days. Now, historical volatility and implied volatilities are very different in their approach and meaning. Since the implied volatility is a parameter that has been calibrated based on options of varying maturity and moneyness (the distance between the current price and the strike), it is a measure that depends not only on what the market perceives from the uncertainty on underlying's returns but also of how much the market wants insurance. Therefore the implied volatility is before everything else a measure of how expensive is the insurance today. In times of crisis, you might have a big discrepancy between the realized and the implied volatilities if the general market fear impacts a lot option prices but much less the current offer and demand of the underlying itself (see for example a blog on the matter).

But the true evidence that volatility remains a complex notions that should not be seen as static or independent of risk perceptions (in the case of the implied one), is that the implied volatility measure changes with the maturity and the moneyness of the option. This means that, when crossing both, we  obtain a surface of implied volatilities (see example picture from Bloomberg):

Now, philosophically, it seems a little bit peculiar to show up a volatility surface by "inverting" an option-pricing formula that...assumes a constant volatility. Thus the impetu of econometricians to come up with a model for the evolution of volatility...