DFG project on System Identification with Regularized FIR Models
System Identification with Regularized Finite
Impulse Response Models
High quality models are the basis for most sophisticated
methods in e.g., prediction, simulation, optimization, control,
fault detection and diagnosis. System identification is
concerned with the estimation of dynamic models from finite,
noisy measurement data. Typically, system identification is
applied to problems where the dominant part of information
stems from the measurement data and no or only little prior
knowledge about the process is available. The case where some
prior knowledge beyond very simple structural assumptions shall
be integrated in the estimation procedure is called gray-box
modeling.
The idea proposed in [1] corresponds to a gray-box modeling
approach for the estimation of FIR model structures. Usually,
the estimation of finite impulse response (FIR) models requires
a high number of parameters which leads to a high variance
error [2]. To overcome this problem regularization can be used.
By introducing an additional penalty term in the estimation
procedure, prior knowledge can be included. The new cost
function to be minimized is given by
The first term is the difference between the measured outputs y and model outputs ŷ which is minimized to estimate the optimal parameters θ̂ of the FIR model without prior knowledge [3]. The second term is the regularization term which introduces a relationship between the parameters of the FIR model themselves. This relationship reduces the effective number of parameters and is defined by the matrix R. Due to the fact, that the parameters of a FIR model correspond to the impulse response of the modeled system, prior knowledge on the dynamic behavior can be incorporated in the matrix R. In Fig. 1 the relationship of the parameters of an impulse response is shown for a first order system.
Figure 1: Properties of an impulse response of a first order system which can be used for gray-box modeling
To accomplish a good tradeoff between this prior knowledge and
the information included in the measurement data, the
hyperparameter λ is used to adjust the strength of the
regularization. The effect of different values of the
regularization strength λ is shown in Fig. 2. Thereby, the
approach works even if the prior does not correspond exactly to
the process.
Figure 2: Effect of different values of the regularization strength λ
For a regularization strength of zero the prior has no
influence on the estimated parameters, which is called
black-box modeling. In contrast, for a high regularization
strength only the prior is considered. This tradeoff between
using prior and measured data is shown in Fig. 3.
Figure 3: Tradeoff between black-box and white-box modeling
A big advantage of introducing prior knowledge in form of
regularization is the possibility to minimize the cost function
J analytically, which yields a computationally efficient
implementation. To choose an appropriate matrix R
and the regularization strength λ a higher-order hyperparameter
optimization can be utilized. By this, the parameterization
which is strongly depended on the modeled system and the
measurement data is be performed automatically [4]. The
procedure of the nonlinear hyperparameter tuning is shown in
Fig. 4. The corresponding model parameters are calculated for
different hyperparameter combinations. Subsequently, the leave
one out error or marginal likelihood maximization can be used,
to determine the optimal hyperparameters.
Figure 4: Procedure of the hyperparameter tuning
By using this approach, the estimated models are inherently
stable and their model parameters allow physical
interpretability regarding characteristics. Additionally, the
identification is insensitive to the system order and a
possible deadtime.
Contact:
Prof. Dr.-Ing. Oliver Nelles
Universität Siegen
Institut für Mechanik und Regelungstechnik - Mechatronik
Paul-Bonatz-Str. 9-11
D 57068 Siegen
www.mb.uni-siegen.de/mrt
[1] Gianluigi Pillonetto and Giuseppe De
Nicolao. A new kernel-based approach for linear system
identification. Automatica, 46(1):81-93, 2010.
[2] Tobias Münker, Timm Peter, and Oliver
Nelles. Gray-box identification with regularized FIR models.
at-Automatisierungstechnik, 66(9):704-713, September 2018.
[3] Oliver Nelles. Nonlinear System
Identification. Springer, Berlin, Germany, 2001.
[4] Gianluigi Pillonetto, Francesco Dinuzzo,
Tianshi Chen, Giuseppe De Nicolao, and Lennart Ljung. Kernel
methods in system identification, machine learning and function
estimation: A survey. Automatica, 50(3):657-682, 2014.