This document presents the framework developed by the Basle Committee on Banking Supervision ("the Committee") for incorporating backtesting into the internal models approach to market risk capital requirements. It represents an elaboration of Part B.4 (j) of the Amendment to the Capital Accord, which is being released simultaneously.
Many banks that have adopted an internal model-based approach to market risk measurement routinely compare daily profits and losses with model-generated risk measures to gauge the quality and accuracy of their risk measurement systems. This process, known as "backtesting", has been found useful by many institutions as they have developed and introduced their risk measurement models.
As a technique for evaluating the quality of a firm's risk measurement model, backtesting continues to evolve. New approaches to backtesting are still being developed and discussed within the broader risk management community. At present, different banks perform different types of backtesting comparisons, and the standards of interpretation also differ somewhat across banks. Active efforts to improve and refine the methods currently in use are underway, with the goal of distinguishing more sharply between accurate and inaccurate risk models.
The essence of all backtesting efforts is the comparison of actual trading results with model-generated risk measures. If this comparison is close enough, the backtest raises no issues regarding the quality of the risk measurement model. In some cases, however, the comparison uncovers sufficient differences that problems almost certainly must exist, either with the model or with the assumptions of the backtest. In between these two cases is a grey area where the test results are, on their own, inconclusive.
The Basle Committee believes that backtesting offers the best opportunity for incorporating suitable incentives into the internal models approach in a manner that is consistent and that will cover a variety of circumstances. Indeed, many of the public comments on the April 1995 internal models proposal stressed the need to maintain strong incentives for the continual improvement of banks' internal risk measurement models. In considering how to incorporate backtesting more closely into the internal models approach to market risk capital requirements, the Committee has sought to reflect both the fact that the industry has not yet settled on a single backtesting methodology and concerns over the imperfect nature of the signal generated by backtesting.
The Committee believes that the framework outlined in this document strikes an appropriate balance between recognition of the potential limitations of backtesting and the need to put in place appropriate incentives. At the same time, the Committee recognises that the techniques for risk measurement and backtesting are still evolving, and the Committee is committed to incorporating important new developments in these areas into its framework.
The remainder of this document describes the backtesting framework that is to accompany the internal models capital requirement. The aim of this framework is the promotion of more rigorous approaches to backtesting and the supervisory interpretation of backtesting results. The next section deals with the nature of the backtests themselves, while the section that follows concerns the supervisory interpretation of the results and sets out the agreed standards of the Committee in this regard.