Mark Transtrum > Differential
Geometry and Sloppy Models > Model Reduction
Model Reduction:


Iterating these four steps removes parameter combinations one at a time until the model is sufficiently simple. What constitutes "sufficiently simple" will vary based on context. The final model will then correspond to a hypercorner of the original model manifold.
The manifold boundary approximation method naturally overcomes the three challenges to model reduction listed above. By repeatedly evaluating limiting approximations in the model, the irrelevant parameters are removed and the remaining parameters naturally group into the physically important combinations. These combinations are often nonlinear combinations of the original bare parameters. MBAM naturally connects microscopic physics with emergent, macroscopic physics in a semiautomatic way. This is perhaps best illustrated by an example.
One of the first models to be identified as sloppy was a systems biology model describing signaling in a developing rat cell. This particular model had 48 parameters and 29 differential equations. However, because the model was made of many inhomogeneous components interacting in highly nonlinear ways, it was not obvious which parameter combinations were important nor could any of the traditional model reduction methods be used to simplify it. MBAM, however, was able to identify a series of limiting approximations that could be applied to the model. The result was a new effective model that had only 12 parameters and 6 differential equations.

The figure shows the network diagram for the original model and the new, effective diagram for the simplified model. Notice how most of the superfluous components have been removed. What remains is a dramatic illustration of the negative feedback loop (from Erk to P90/RSK to Ras) that is the mechanism responsible for the system behavior. Qualitatively, this negative feedback loop is how the biologists had understood the behavior of the system. MBAM naturally recovered the same result and produced a quantitative, mathematical model to describe this interpretation in the process. What remains after the simplifying process are precisely the 12 parameters that are necessary for the model to be predictive. This can be seen by noting that there are no small eigenvalues in the FIM for the new model. All the sloppy parameters (combinations) have been removed and all of the stiff parameter (combinations) have been kept!
The nice thing about the approximate model is that it is not just a "black box". Instead, it explains the system's collective behavior in terms of the components of the original model. For example, consider the effective interaction between C3G and Erk in the new model. The strength of this interaction is controlled by a parameter (labeled \(\phi\) in the diagram). This parameter has a definition in terms of the original model parameters that emerges through the sequence of limiting approximations: $$ \phi = \frac{ (\mathrm{k_{Rap1ToBRaf}}) (\mathrm{K_{mdBRaf}}) (\mathrm{k_{pBRaf}}) (\mathrm{K_{mdMek}}) }{ (\mathrm{k_{dBRaf}}) (\mathrm{K_{mRap1ToBRaf}}) (\mathrm{k_{dMek}}) }.$$ This parameter is the "renormalized" parameter of the effective model. Notice that it is a nonlinear combination of 7 other parameters. Each of the seven parameters can influence the behavior of the model but only through their effect on \(\phi\).
Since this model has 12 parameters, there are 11 other expressions (of varying complexity) similar to the one for \(\phi\) above. Each one of these parameters controls an important feature in the model's behavior. In fact, if the simplified model were fit to data, all of the parameters could be inferred with small error bars (notice the eigenvalues of the simplified model in the figure). The new model is not sloppy!
The manifold boundary approximation method makes very few assumptions about the model. We have seen that it performs spectacularly on our systems biology model. To what other models can it be applied in what contexts?
At its heart, MBAM is a tool for identifying limiting approximations. There are a wide variety of model reduction techniques that have grown up searching for these types of approximations in different contexts. For example, in dynamical systems, singular perturbation theory explores the behavior of differential equations in the limit that one of the time scales is much faster than the others. If there is a parameter in the model that explicitly controls the time scale, then the limit that that parameter becomes zero will correspond to a boundary of the model manifold. Therefore, singular perturbation theory can be recovered as a special case of the MBAM procedure.
Many other approximation techniques can similarly be subsumed as special cases of the MBAM procedure. Continuum limits and thermodynamic limits are two other obvious examples of a limiting approximation. However, other approximation methods that are not obviously limiting approximations can similarly be reproduced by MBAM. For example, the renormalization group, a powerful tool in statistical physics, as well as balanced truncation, the cornerstone of dynamical systems order reduction in control theory, both can be recovered as special cases of MBAM.
MBAM provides a new way to think about model reduction that is very general. By unifying and generalizing many timetested model reduction techniques, it is a step toward a unified theory of model reduction.