Dynamic Representations

Disadvantages of Traditional Nonlinear Dynamic Models

• The one-step-ahead prediction error (series-parallel configuration) is optimized (usually).
• Tuning in simulation (parallel configuration) is possible → Complex nonlinear problem.
• Simulation can even become unstable → Unreliable.
• Online adaptation is therefore very risky.

NARX

• Stability issues due to external output feedback.
• Very sensitive to the correct order and dead time.
• Interpolation → Unwanted effects.
• Optimization of equation error (in series-parallel). Drawbacks:
  • emphasis on high frequencies
  • one common denominator A(z) → identical dynamics for all inputs
     

NFIR
 

• No stability issues because there is no feedback.
• Optimization of simulation error → All NARX issues have been resolved.
• New drawback: Requires huge orders and dimensions.

NOBF
 

• No stability issues because only local feedback is used.
• Optimization of simulation error → All NARX issues have been resolved.
• New drawback: Selection of filter pole(s)?
   

What's Wrong With NARX? Interpolation of 2 ARX Models
 

What's Wrong With NARX? Extrapolation
 

• Extrapolation behavior along the y(k‒i) axis determines the dynamics
  • slope ≈ 0 → static
  • |slope| < 1 → stable dynamics
  • |slope| > 1 → unstable dynamics
• A major advantage for all linearly extrapolating model architectures, such as local linear models. 
• Extremely dangerous for polynomials.
• Most other model architectures extrapolate constantly.

What's Wrong With NARX? MISO Models
 

• Each input → output: Individual dynamics.
• One common denominator A(z).
• A multivariate ARX model requires a high order, which is the sum of the individual orders.
• Approximate zero/pole cancellations.
• Static inputs: Bi(z) must cancel out A(z).
• Popular solution in the linear case → Subspace identification in state space (FIR-based approach)

But what about NOE? MISO models 

• OE: Linear case, each input:
  • individual transfer function
  • individual feedbacks
  • individual states
  • number of state variables = number of inputs × order 

• NOE: Nonlinear case, each input:
  • one feedback independent of inputs
  • one set of states
  • number of state variables = order 

Regularized FIR Estimation – The Linear Case

Recent Progress in FIR Identification

• Use of regularization for the estimation of linear FIR models (proposed by Pillonetto et al., 2011)
   

Modified Loss Function for FIR Estimation
 

• Regularized least squares problem
• Choice of the regularization matrix
• “Tuned correlated” (TC) kernel, which penalizes an exponentially decaying version of the first-order derivative of the system
• Interpretation as an RKHS or a Gaussian process model
• Nonlinear optimization of the hyperparameters

Regularized FIR
 

The loss function

• P specifies the parameter covariances
• P should decrease exponentially with α (α is the pole)
  • 
• P should also decrease exponentially with the distances k–l.
  • 

This is the TC kernel:

3 priors—Gaussian process, 3 realizations each, order m = 20

with: 

a) white, bk independent, non-decreasing
b) white, bk independent, exponentially decreasing
c) correlated, black-dependent, exponentially decreasing

Example: Identification of a Damped Second-Order System
 

Advantages of Regularized FIR (Blue Curve)

• The high variance of the FIR is suppressed
• Compared to ARX, the stability of the impulse response is guaranteed a priori

Extension to the Nonlinear Case – Concept

Idea

• Use regularized FIR models as local models
• Validity functions z can depend on delayed inputs and outputs in either of the following ways

Advantages

• Complete model trained for simulation error
• Stability of the identified model is guaranteed
• Model order estimation is not required

Disadvantage

• Hyperparameter search for each local model

Additional considerations

• Consideration of the LMN offset in the models
• Interpretation of model error as a combination of nonlinearity and noise error

Example: Saturation-Type Wiener System

Estimated Results Using LOLIMOT and Different Local Model Types

Next Chapter: 6. Classification Back to Overview