Sensitivity analysis examines how changes in the assumptions of an economic model affect its predictions. By definition, an economic model is a simplified mathematical representation of a complex interaction of economic variables, and as such is built upon certain assumptions. These assumptions, which include the structural specification of the model and the values of its parameters, are made to best approximate the phenomenon the model attempts to capture. At the same time, however, a model’s assumptions are typically subject to uncertainty and error. For example, variables’ current values often are not known with precision, and their future values may change. A properly designed sensitivity analysis can be a powerful modeling tool that contributes to an understanding of the relationships between the assumptions of an economic model and its results. Moreover, such an analysis can help validate the model’s predictions even given uncertainty about its assumptions. An incorrectly designed sensitivity analysis, however, can be used to support a flawed model and can lead to wrong conclusions. Accordingly, it is essential to design this analysis carefully.
The first step in designing a sensitivity analysis involves selecting the assumptions that will be tested. For example, a model of financial risk management may rely on assumptions about the extent to which returns on different assets are correlated. Sensitivity analysis can be used to test how different values of such “cross-correlations” affect risk. Because economic systems are complex and frequently consist of many interrelated variables, it may be appropriate to test the effects of changing more than one variable at a time. In these circumstances, understanding the relationships among variables is important for designing a sensitivity analysis because, if variables are correlated, a change to one variable is likely to be accompanied by changes to one or more other variables. If, for instance, larger values of one variable are likely to be associated with larger values of another variable, then it is more realistic to consider simultaneous changes that affect both variables in the same direction. In the financial risk model, for example, higher levels of market risk may be associated with higher cross-correlations between certain asset classes. Sensitivity analyses of market risk may require increasing the extent of covariant risk between assets to capture these effects.
Sensitivity analyses can assess changes not only in the parameters of the model, but also in the specification of the model. Specification changes may include the addition of constraints, changes in functional form, etc. As with the selection of parameters, introducing combined changes to the structure of the model may be justified by a systematic relationship between structural characteristics. In the example above, widespread securitization of mortgage assets during the past decade may require modifying risk management models to account separately for housing-related events that may affect portfolio risk.
The second step in the design of a sensitivity analysis entails selecting the alternative values or specifications. Simple proximity or other arbitrary criteria alone cannot guide the analysis. For example, examining the model assuming parameter values that are close to the “base case” assumptions may not be informative if these values are unlikely or if other likely values are excluded from the analysis. Selecting the appropriate set of alternative values requires examining the distribution of alternatives, including both the range of possible values and the probability of these alternatives. For example, risk management practitioners often test their model results using variability observed over the previous year. In a more uncertain environment, however, it may be useful to model variation over longer periods of time, perhaps over three or even five years. While it may be impractical to simulate the model for all possible alternative parameter values, the analysis should consider at least the alternatives that are likely to occur. Without sufficient knowledge of the underlying distribution of parameter values, the model’s conclusions may need to be qualified or limited in scope, and they may even be too imprecise to be useful.
Flaws in the available data often motivate sensitivity analyses. But uncertainty regarding a variable’s value is different than uncertainty regarding the range or distribution of that value. For example, you might not observe the exact value of a particular cross-correlation between assets, but you may be relatively certain that it falls in the range of 0.3 and 0.7. Because the conclusions that can be derived from a sensitivity analysis depend on knowledge of the underlying distribution of possible values, sensitivity analyses can be informative even when data are imprecise or insufficient. Nonetheless, if no information about the distribution of likely alternative values is available, sensitivity analysis may offer little, if any, information about the robustness of the model.
When correctly designed, a sensitivity analysis is a valuable modeling tool because it may provide information on the robustness of a model’s predictions. That information can help validate an economic model in the presence of uncertainty. A sensitivity analysis also can contribute to the specification of a model by assessing the individual contribution of a variable and the need to include it or not. Moreover, a sensitivity analysis can help interpret the results of a model. For example, a sensitivity analysis may help identify thresholds for certain variables that trigger outcomes of interest. In the case of a risk management model, cross-correlations above a particular value may be associated with significantly higher losses that require higher reserves or a different investment approach. Also, to the extent that the outcomes of a sensitivity analysis yield the probabilities of different results, users of the model can assess the upside and downside risks associated with alternative scenarios.