Black boxes: Why stochastic modelling demands heightened regulation

Experts don't often argue for increased regulation of financial services. However, Bruce Moss believes the world of stochastic modelling needs greater oversight

The world of stochastic models is largely opaque.

They are seen as the “black boxes” of the investment analysis tool-kit and users are often asked to take their projections on trust.

However, they are widely used for many purposes and, in particular, are highly effective as a means of illustrating the trade-off between investment risk and reward.

Generally, users are provided with so little information that they should probably question why they believe the output from stochastic models and why they are basing important financial decisions on it.

Unique advantages

The reason for using stochastic models and their forecasts is that they are able to do things which are particularly valuable and helpful:

  • They illustrate risk and the extra return likely to be associated with different levels of risk;
  • They can be used to construct efficient portfolios expected to offer the best return for the level of risk taken; and
  • They can identify the extent and nature of a guarantee (i.e. how valuable it is) and how much it is costing in terms of foregone return.

In short, a good stochastic model provides a robust mechanism for linking investment risk and the associated returns in a way which is not dependent on a specific period of historic investment data. The problem with using a specific period of past data is, of course, that it is “no guide to the future”. The forecasts from a good stochastic model are not dependent on historical data for any specific period but, rather, are produced by building models for each asset class and economic variable based on all observable historic data.

The return for a specific asset class is based upon its observed dependence on the movements in other asset classes and economic variables over many years, as well as many different economic cycles.

Pension freedom

This ability to help people understand the benefits and costs associated with guarantees has particular importance with the forthcoming changes to pensions from 6 April. Stochastic forecasts are uniquely able to explain the risks (and potential rewards) involved in not buying an annuity.

They can show the income, depending on how retirement savings are invested, which is likely to be sustainable for life.

“Safely nets” can be created by modelling different investment strategies, so that retirees’ essential income needs can be met while leaving the potential for higher income if things go well.

Stochastic forecasts are also extremely flexible and can allow retirees to develop plans for meeting all their retirement goals. Most importantly, retirees can gain an understanding of the chances of achieving these goals and have realistic expectations for their retirement lifestyle.

Flexi-access drawdown involves uncertainty over the amount of income to be received, how it may vary and whether it will be sustainable for life.

Given that these risks are durational, the only effective way in which the risk can be assessed, compared to the risk-free option of an annuity, is to use a stochastic asset model to forecast the amounts of income year-by-year and to compare the results with an annuity.

A stochastic asset model produces thousands of simulations of plausible future investment market scenarios. Some examples of stochastic forecasts of the outcomes from flexi-access drawdown compared to an annuity are shown on Figure 1 (below).

Stochastic models: The need for greater transparency chart 1

The green line shows the after-tax income from an annuity. The forecast results can be displayed in different ways. This example shows shading to represent the probability of the outcome – the darkest blue area being the most likely outcome for flexi-access drawdown.

The diagram shows that there is an appreciable chance that flexi-access drawdown will give a worst outcome than an annuity, particularly before an annuity is purchased at age 75.

As can be seen from Figure 1, stochastic forecasting can be a powerful tool to help consumers understand the potential journey they may be embarking on with flexi-access drawdown.

The ‘seven veils’ of stochastic mystery

The benefits of stochastic models are clear but the models themselves, sadly, less so! This is a major problem as the “stochastic black box” has the potential to introduce serious product bias and lead the user to draw the wrong conclusions.

The issue is generally not so much with the model but what it is being used for. To use an analogy, you could have a very good model for predicting and managing traffic flows in New York that would be useless in London (good model, wrong application). The same is true of stochastic asset models.

The term “stochastic model” is a generic term and stochastic models can differ widely in terms of how they are built and what they are suitable for. There are two basic types of stochastic asset model:

  • Mean, variance co-variance (MVC) models, and
  • Economic scenario generators (ESGs).

MVC models are very simple and are primarily used for making short term tactical asset allocation decisions. They use a single set of assumptions about the expected return for each asset class, its volatility (or variance) and how asset classes move relative to one another (correlation or co-variance). The set of assumptions can either be based on current market conditions or on views about long-term “normal” conditions.

What an MVC model cannot do is tell you anything about how markets might progress from current conditions into the future. MVC models provide a “snapshot” at a single point in time.

ESGs, on the other hand, model the economy and investment markets from today’s current conditions into the future. An ESG models the “investment journey” and is, thus, the appropriate type of model for forecasting retirement income outcomes from drawdown compared to an annuity.

Stochastic models: The need for greater transparency chart 2

The charts on Figure 2 (above) show how misleading using MVC models for retirement income forecasting can be. The diagrams show the downside (worst case) as the red bars. The expected (or average) outcome is shown as the blue bars. To simplify the diagrams, the upside (best case) has not been shown.

Depending on the single set of assumptions used (a) based on current market conditions or (b) long-term “normal” conditions, very different conclusions on the relative attractiveness of annuities and drawdown would be reached.

Essentially, an MVC model is not a suitable model for modelling retirement income – it was not designed for that purpose. ESGs, however, give a more balanced picture. The differences between models tend to be less extreme as ESGs start with current market conditions and forecast potential future outcomes year by year.

Forecasts from different ESGs may therefore diverge with the duration of the forecast, but overall differences are much less dramatic than with MVC models.

Regulation needed

The main conclusion to be drawn is that stochastic models have some unique and valuable benefits, but that considerable due diligence needs to be given to the type of model used, particularly for forecasting retirement income.

Unless this is done, there is a considerable danger of serious product bias and the wrong conclusions being drawn depending on the type of stochastic model and the assumptions used.

The Financial Conduct Authority (FCA) has made it clear that, as with all tools used, the responsibility for carrying out the due diligence on a stochastic model’s suitability resides with the financial adviser. While this is logical, it is (as things stand) an impossible task for advisers to undertake and FCA needs to come to their aid.

Regulation should ensure that there is greater transparency on the workings of models and assumptions.

‘Oh dear”, I hear the reader say, “haven’t we got enough regulation? It is seldom helpful and does not ease the burden of our day-to-day work”.

Well, on this occasion, it is different and regulation could be beneficial to both consumers and advisers and not onerous. All that is required is for the supplier of the stochastic model to describe the workings of the model; for what uses it is suitable (for instance, MVC models are suitable for constructing portfolios but not for forecasting a retirement income journey); and whenever the model is updated, supply details of the impact on the output from the model.

Currently, there is no obligation on the suppliers of stochastic models to provide any information about the model or guidance on how it should be used. Clearly, this is wrong and must change. Stochastic models are too valuable and too widely used to be left unregulated.

Bruce Moss is strategy director at eValue