Risk Library
   Documents by Author
     Committees at the Bank for International...
       An Internal Model-Based Approach to Mark...
         II. Lessons learnt from the testing exer...


An Internal Model-Based Approach to Market Risk Capital Requirements

II. Lessons learnt from the testing exercise

1. In order to help determine which model parameters should be standardised or constrained for the purpose of measuring market risk capital requirements, the Committee carried out some preliminary testing in the second half of 1994. One object of the test was to establish how great a difference there would be between different models' measures of value-at-risk when an absolute minimum number of parameters was specified. Another was to check whether the value-at-risk measures would produce, in the Committee's view, reasonable value-at-risk estimates relative to the size of the portfolio. For this purpose, a task force set up by the Committee compiled a test portfolio of approximately 350 positions. The portfolio was evaluated by fifteen banks in the major G-10 countries who measured the value-at-risk produced by their own models for the portfolio, using a ten-day holding period and a 99% confidence interval, as of the same date. In doing so, they were asked to produce a total value-at-risk figure, as well as individual values-at-risk for foreign exchange, interest rate and equity risk categories and also to test four different variants of the portfolio, one balanced and one unbalanced, each with and without options positions.

2. Although the raw results provided by the banks were quite disparate, further investigation was able to pinpoint the main factors contributing to the observed differences. Among the several factors which led to the dispersion, the easiest ones to interpret related to ambiguities which occurred in inputting the portfolio and the fact that banks were using methods of varied sophistication for measuring options risks. After accounting for these factors, slightly over half of the individual responses fell into a sufficiently close range but significant overall dispersion remained.

3. The exercise identified several important differences in model practice that appeared to be responsible for differences in the test results. Although the Committee realises the inherent limitations of a single testing exercise, it believes that the main systematic differences in model output are related to the following factors:

  • when inviting banks to conduct the testing, the Committee's task force did not set any constraints on the historical time horizon over which price volatility is observed. Some of the participating banks use very short periods, as short as a few months, while others use periods of several years;

  • another cause of dispersion in the overall value-at-risk measures was differences in the methods of aggregating different measures of risks, both within and across risk factor categories (e.g. exchange rates, interest rates). For example, some banks aggregated their value-at-risk numbers for different risk factor categories using a simple sum method, others used a "square root of the sum of the squares" method, whereas others used historical correlations;

  • treatment of options risk varies across banks, as many are still researching and implementing more advanced approaches;

  • yet another difference in the measure of interest rate risk was caused by the number and definition of interest rate risk factors used by different banks. For example, the number of time buckets used varied widely and banks had different ways of measuring the risk of changes in the yield curve and spreads between yield curves;

  • so far as the basic methodology for calculating value-at-risk is concerned, the task force found no systematic difference between the results of banks using the historical simulation approach and the variance/co-variance approach.

4. In summary, the preliminary testing exercise was extremely useful in providing further insight into the issues which arose from the use of internal models. The Committee has been guided by these insights in its choice of quantitative and qualitative standards set out in the remainder of this paper.

Contact us * Risk Library * Documents by Author * Committees at the Bank for International Settlement (BIS) * An Internal Model-Based Approach to Market Risk Capital Requirements