# DFG project on System Identification with Regularized FIR Models

**System Identification with Regularized Finite
Impulse Response Models**

High quality models are the basis for most sophisticated
methods in e.g., prediction, simulation, optimization, control,
fault detection and diagnosis. System identification is
concerned with the estimation of dynamic models from finite,
noisy measurement data. Typically, system identification is
applied to problems where the dominant part of information
stems from the measurement data and no or only little prior
knowledge about the process is available. The case where some
prior knowledge beyond very simple structural assumptions shall
be integrated in the estimation procedure is called gray-box
modeling.

The idea proposed in [1] corresponds to a gray-box modeling
approach for the estimation of FIR model structures. Usually,
the estimation of finite impulse response (FIR) models requires
a high number of parameters which leads to a high variance
error [2]. To overcome this problem regularization can be used.
By introducing an additional penalty term in the estimation
procedure, prior knowledge can be included. The new cost
function to be minimized is given by

The first term is the difference between the measured
outputs y and model outputs ŷ which is minimized to estimate
the optimal parameters __θ̂__ of the FIR model without prior
knowledge [3]. The second term is the regularization term which
introduces a relationship between the parameters of the FIR
model themselves. This relationship reduces the effective
number of parameters and is defined by the matrix __R__. Due
to the fact, that the parameters of a FIR model correspond to
the impulse response of the modeled system, prior knowledge on
the dynamic behavior can be incorporated in the matrix
__R__. In Fig. 1 the relationship of the parameters of an
impulse response is shown for a first order system.

*Figure 1: Properties of an
impulse response of a first order system which can be used for
gray-box modeling*

To accomplish a good tradeoff between this prior knowledge and
the information included in the measurement data, the
hyperparameter λ is used to adjust the strength of the
regularization. The effect of different values of the
regularization strength λ is shown in Fig. 2. Thereby, the
approach works even if the prior does not correspond exactly to
the process.

*Figure 2: Effect of
different values of the regularization strength λ*

For a regularization strength of zero the prior has no
influence on the estimated parameters, which is called
black-box modeling. In contrast, for a high regularization
strength only the prior is considered. This tradeoff between
using prior and measured data is shown in Fig. 3.

*Figure 3: Tradeoff between
black-box and white-box modeling*

A big advantage of introducing prior knowledge in form of
regularization is the possibility to minimize the cost function
J analytically, which yields a computationally efficient
implementation. To choose an appropriate matrix __R__
and the regularization strength λ a higher-order hyperparameter
optimization can be utilized. By this, the parameterization
which is strongly depended on the modeled system and the
measurement data is be performed automatically [4]. The
procedure of the nonlinear hyperparameter tuning is shown in
Fig. 4. The corresponding model parameters are calculated for
different hyperparameter combinations. Subsequently, the leave
one out error or marginal likelihood maximization can be used,
to determine the optimal hyperparameters.

*Figure 4: Procedure of the
hyperparameter tuning*

By using this approach, the estimated models are inherently
stable and their model parameters allow physical
interpretability regarding characteristics. Additionally, the
identification is insensitive to the system order and a
possible deadtime.

**Contact:**

Prof. Dr.-Ing. Oliver Nelles

Universität Siegen

Institut für Mechanik und Regelungstechnik - Mechatronik

Paul-Bonatz-Str. 9-11

D 57068 Siegen

www.mb.uni-siegen.de/mrt

[1] Gianluigi Pillonetto and Giuseppe De
Nicolao. A new kernel-based approach for linear system
identification. Automatica, 46(1):81-93, 2010.

[2] Tobias Münker, Timm Peter, and Oliver
Nelles. Gray-box identification with regularized FIR models.
at-Automatisierungstechnik, 66(9):704-713, September 2018.

[3] Oliver Nelles. Nonlinear System
Identification. Springer, Berlin, Germany, 2001.

[4] Gianluigi Pillonetto, Francesco Dinuzzo,
Tianshi Chen, Giuseppe De Nicolao, and Lennart Ljung. Kernel
methods in system identification, machine learning and function
estimation: A survey. Automatica, 50(3):657-682, 2014.