DFG project on System Identification with Regularized FIR Models

System Identification with Regularized Finite Impulse Response Models

High quality models are the basis for most sophisticated methods in e.g., prediction, simulation, optimization, control, fault detection and diagnosis. System identification is concerned with the estimation of dynamic models from finite, noisy measurement data. Typically, system identification is applied to problems where the dominant part of information stems from the measurement data and no or only little prior knowledge about the process is available. The case where some prior knowledge beyond very simple structural assumptions shall be integrated in the estimation procedure is called gray-box modeling.

The idea proposed in [1] corresponds to a gray-box modeling approach for the estimation of FIR model structures. Usually, the estimation of finite impulse response (FIR) models requires a high number of parameters which leads to a high variance error [2]. To overcome this problem regularization can be used. By introducing an additional penalty term in the estimation procedure, prior knowledge can be included. The new cost function to be minimized is given by

The first term is the difference between the measured outputs y and model outputs ŷ which is minimized to estimate the optimal parameters θ̂ of the FIR model without prior knowledge [3]. The second term is the regularization term which introduces a relationship between the parameters of the FIR model themselves. This relationship reduces the effective number of parameters and is defined by the matrix R. Due to the fact, that the parameters of a FIR model correspond to the impulse response of the modeled system, prior knowledge on the dynamic behavior can be incorporated in the matrix R. In Fig. 1 the relationship of the parameters of an impulse response is shown for a first order system. 

 

Figure 1: Properties of an impulse response of a first order system which can be used for gray-box modeling

To accomplish a good tradeoff between this prior knowledge and the information included in the measurement data, the hyperparameter λ is used to adjust the strength of the regularization. The effect of different values of the regularization strength λ is shown in Fig. 2. Thereby, the approach works even if the prior does not correspond exactly to the process.

 

 Figure 2: Effect of different values of the regularization strength λ

For a regularization strength of zero the prior has no influence on the estimated parameters, which is called black-box modeling. In contrast, for a high regularization strength only the prior is considered. This tradeoff between using prior and measured data is shown in Fig. 3.

 

 

Figure 3: Tradeoff between black-box and white-box modeling

A big advantage of introducing prior knowledge in form of regularization is the possibility to minimize the cost function J analytically, which yields a computationally efficient implementation. To choose an appropriate matrix R  and the regularization strength λ a higher-order hyperparameter optimization can be utilized. By this, the parameterization which is strongly depended on the modeled system and the measurement data is be performed automatically [4]. The procedure of the nonlinear hyperparameter tuning is shown in Fig. 4. The corresponding model parameters are calculated for different hyperparameter combinations. Subsequently, the leave one out error or marginal likelihood maximization can be used, to determine the optimal hyperparameters.
 

 

Figure 4: Procedure of the hyperparameter tuning

By using this approach, the estimated models are inherently stable and their model parameters allow physical interpretability regarding characteristics. Additionally, the identification is insensitive to the system order and a possible deadtime.

Contact:

Prof. Dr.-Ing. Oliver Nelles
Universität Siegen
Institut für Mechanik und Regelungstechnik - Mechatronik
Paul-Bonatz-Str. 9-11
D 57068 Siegen
www.mb.uni-siegen.de/mrt

---

[1]    Gianluigi Pillonetto and Giuseppe De Nicolao. A new kernel-based approach for linear system identification. Automatica, 46(1):81-93, 2010.
[2]    Tobias Münker, Timm Peter, and Oliver Nelles. Gray-box identification with regularized FIR models. at-Automatisierungstechnik, 66(9):704-713, September 2018.
[3]    Oliver Nelles. Nonlinear System Identification. Springer, Berlin, Germany, 2001.
[4]    Gianluigi Pillonetto, Francesco Dinuzzo, Tianshi Chen, Giuseppe De Nicolao, and Lennart Ljung. Kernel methods in system identification, machine learning and function estimation: A survey. Automatica, 50(3):657-682, 2014.