coefficients of the independent variables combinations in the fitted
model.
While each parameter “per se” can have a great uncertainty, this
does not mean that any parameter can be varied independently of
the others and the resulting model can still fit the data. Fisher
Information Matrix (FIM) is a sort of covariance matrix [2] report-
ing the mutual between parameters dependence: this covariance
structure stems from the fact the data place constraints on combi-
nations of parameters.
The eigenvalue distribution of the FIM thus corresponds to an
estimate of the relative strength of the constraints exerted by the
data on the “free variation” (uncertainty) of the model parameters.
Top eigenvalues in Fig.1 correspond to the parameters combina-
tions relevant for data fitting (and consequently for their interpre-
tation), low-level eigenvalues correspond to irrelevant aspects. In
the case reported in Fig.1 only the top three eigenvalues are
sufficiently discriminated by the measurement error to be of practi-
cal use; this suggests the presence of very general (and robust)
ordering principles going together with many irrelevant details
(minor eigenvalues).
Machine learning experts are aware of this phenomenon since
long time: it is the so-called overfitting [3] effect. The basic issue is
that each set of experimental data is affected by noise, being the
main noise sources experimental errors and (still more dangerous)
the selection bias, i.e., the fact that any data set is relative to aFig. 1The figure reports the trend of FIM eigenvalues for different system parameter combinations. Few top
eigenvalues correspondent to the “stiff” part of the model go together with a plethora of largely irrelevant
(sloppy) parameter combinations
58 Alessandro Giuliani