“Let me warn you…” of the limitations of VecViz’s Analytics.

“Let me warn you, Icarus, to take the middle way, in case the moisture weighs down your wings, if you fly too low, or if you go too high, the sun scorches them. Travel between the extremes.” Ovid, Metamorpheses

Similar to Daedelus’ warning to Icarus regarding his wings, VecViz advises you use the Vector Model and related metrics (option fair values, the V-Score, etc.) with moderation. We advise against ascribing any significant degree of certainty to it on any one position or grouping of positions on any one day or grouping of days. Keep in mind that out of sample back testing results are on average across tickers, over time. Such testing is not truly representative of results that would have been obtained trading in live market conditions.

Hubris is a key risk when using forward probability estimation models such as the Vector Model.

LTCM and CDO credit ratings are prime examples of this dynamic. The logical rigor and mathematical elegance of these forward probability oriented endeavors, and the credentials of the people and institutions behind them were indisputable.

Yet both LTCM and the CDO boom ended in tears, arguably because they were pressed too far in terms of leverage and market influence. Liability Driven Investing (LDI) by pension funds in the UK is a somewhat more recent example of a probability estimation oriented endeavor gaining wide acceptance and adoption but ultimately causing quite a large amount of pain, again because they were too levered and too large a part of the market.

The primary shortcomings of the VecViz and its analytics we are aware of at this time include:

  • VecViz, which created and operates the Vector Model, has very limited resources. At present, on a day to day basis VecViz is primarily one person.
  • VecViz’s Vector Model machine learning based price probability percentile forecasts are based entirely on the daily closing price history of the ticker being evaluated and the price histories of the tickers the Vector Model was trained on. There is so much more information in the world that can be brought to bear on the issue of estimating the probability distribution for the future value of any given ticker’s prices that it would be foolish to view VecViz’s Vector Model as anything other than just one of many perspectives on the subject.
  • V-Scores are based on an even smaller training data set than the Vector Model price probabilities
  • The VecEvent narrative elements that VecViz can link to the Vector Sets for a particular ticker may reflect the subjective biases of the person who enters them. At present they are generated solely by VecViz.
  • The VecEvents that VecViz supplies as it launches the Vector Model should not be relied upon as being accurate or insightful. VecViz is not an SEC registered investment advisor and no VecEvent it presents should be interpreted as a recommendation to buy or sell any security.
  • Shortcomings of the Vector Model related back testing results:
    1. Similar to much of the academic literature on return forecasting, VecViz’s back testing results assume positions are entered at closing prices, which are not available to execute upon by the time the Vector Model’s calculations are completed.
    2. They have not been audited or even closely reviewed by any external party.
    3. The summary statistics of their performance represent an over many tickers and time periods.
    4. The Vector Model and Sigma’s price probability forecasts apply to the forecast period end dates specified (or the prior market close if the date specified happens on a weekend or holiday).
      1. Values in excess of the price percentiles on an interim / intraday basis are not captured in the breakage rates presented in the back testing results suggest.
    5. “Sigma” is presented as a representation of Quantitative Finance based VaR, and is used as a benchmark for evaluating Vector Model performance. Sigma is a fair representation of Quantitative Finance VaR in the sense that it is parametric, using formulas promulgated in the industry literature.
    6. The “Return on VaR Based Capital” (ROVBC) metric can be easily misinterpreted. ROVBC for the Vector Model reflects the ticker’s return scaled up or down based on how much lower or higher Vector Model VaR is relative to Sigma VaR, subject to a cap. No transaction costs are contemplated in the ROVBC for either model.
    7. Likewise, the “Return on OaR (Opportunity at Risk) Based Capital” (ROOBC) metric can also be easily misinterpreted. ROOBC for the Vector Model reflects the negative of the ticker’s return scaled up or down based on how much lower or higher Vector Model VaR is relative to Sigma VaR, subject to a cap. No transaction costs are contemplated in the ROOBC for either model.
    8. Many of the multi-day periods for which out of sample back testing results are presented overlap, in some cases to a considerable extent. Thus, the extent of the back testing we have done is in some respects overstated.
  • Vector Model based option fair value uses standard strike % distances and terms to expiration. The user therefore must interpolate for the specific strikes and expiration dates they are interested in. VecViz does not advise whether the user should do that interpolation on a linear or exponential basis.
    1. Note that the fair value estimates are calculated using Model Date closing price and are for informational purposes only.

Leave a Comment

Your email address will not be published. Required fields are marked *