A necessary and sufficient stabilization condition for discrete time-varying stochastic system s with multiplicative noise

Rong GAO ,Xiaohua LIU ,Huanshui ZHANG

1.School of Control Science and Engineering,Shandong University,Jinan Shandong 250061,China;

2.School of Mathematics and Statistics Science,Ludong University,Yantai Shandong 264025,China

1 Introduction

Linear stochastic system s with multiplicative noise in the state and control channels are quite important in practice.Many system s can be modeled by stochastic differential equations or difference equations with multiplicative noise such as system s with hum an operator,mechanical system s,aerospace system s and network system s[1–3],etc.Hence over the past decades,considerable attention has been devoted to the study of different issues related to the linear stochastic system s with multiplicative noise.The mean-square stabilization problem,as one of a fundamental stochastic control problem,has been studied by many researchers in[3–12].For instance,based on linear quadratic method,different results for mean-square stabilization were given in[3–8].Ghaoui[9]gave the stabilization conditions using linear matrix inequality(LM I).In[10]Zhang et al.studied the mean-square stabilization based on generalized Lyapunov equation.In[11,12]the uncertain network system w as modeled by stochastic multiplicative noise model and the necessary and sufficient conditions for mean-square stabilization of the network control system were given.In recent years,stabilizing control of time-varying stochastic system s has been a hot area of research,see for example[13,14]and references therein.It should be noted that for time-varying stochastic system s a necessary and sufficient stabilization condition remains to be investigated.

In order to solve stabilization problem for time varying stochastic system s,receding horizon control(RHC)was adopted.The essence of the RHC is to solve an optimization problem on the finite horizon at the current time and implement only the first solution as a current control.This procedure then repeats at the next time.Due to its many advantages,such as good tracking performance,effective control for time-varying system s and I/O constraint hand ling capability,a great deal of attention has been devoted to RHC.RHC was first pioneered for dealing with the stabilization of time-varying system s by Kwon et al.[15].Since then,it has been widely investigated as a successful feedback strategy for deterministic system s,especially for time-varying deterministic system s[15–19].

In this paper different from the previous literatures on stochastic RHC [20, 21], which focus on constraind linear time-invariant stochastic systems, we consider unconstrained linear time-varying stochastic systems. The main contribution of this paper is as follows: By defining the optimal cost value as a Lyapunov function, the necessary and sufficient RHC stabilization condition for time-varying stochastic system with multiplicative noise is presented for the first time. In the previous literature,even for deterministic systems the RHC stabilization condition is only sufficient as far as we know.

The remainder of this paper is organized as follow s:Section 2 gives the formulation of the problem for stochastic system s with multiplicative noise.The corresponding RHC law and the necessary and sufficient condition for the asymptotic mean-square stability of the closed system are obtained in Section 3.In Section 4 RHC for stochastic system s with multiple multiplicative noises is studied.Three examples are provided in Section 5 to validate the performance of the proposed receding horizon control.Finally,conclusions are given in Section 6.

NotationRnstands for then-dimensional Euclidean space.The subscript“′”represents the matrix transpose;a symmetric matrixM＞0(≥0)means that it is strictly positive definite(positive semi-definite);{Ω,F,P,{Fk}k≥0}denotes a complete probability space on which a scalar white noise ωkis defined such that{Fk}k≥0is the natural filtration generated by ωk,i.e.,Fk=σ{ω0...ωk}.

2 Problem statement

Consider the following linear discrete time-varying stochastic system

For simplicity,letthen system(1)becomes

wherexk∈Rnis the state;uk∈Rmis the control input;andBkare matrices of appropriate dimensions;and ωkis a scalar random white noise with zero mean and variance σ.

Rem ark 1It should be noted that the results to be presented in this paper are applicable to more general stochastic system s with multiple multiplicative noises with no substantial difference

where the variance of the noise is given by

3 Asymptotic mean-square stability of discrete time-varying stochastic system s via RHC

In this section,we shall present our main result on asymptotic mean-square stability for the discrete time varying stochastic system s(1).

3.1 Receding horizon control

In order to solve the problem formulated in Section 2,we define the following cost function:

where E is the mathematical expectation over the noise{ωk,...,ωk+N}.Qk+i≥0,Rk+i＞0,Ψk+N+1＞0 andNis a finite positive integer.xk+i|kanduk+i|kdenote the state and control sequence with initial timekin the finite horizon optimal control.

We apply Pontryagin’s maximum principle[22,23]to system(1)with cost function(4).Then the optimal controller is given by

wherePj(k+N)satisfies the following generalized Ric-cati equation:

with the terminal value

The receding horizon control at timekis given as

whereHkis as in(8)withj=k.

3.2 Asymptotic mean-square stability

In this section,firstly we present a matrix inequality on the terminal weighting matrix Ψk,under which the optimal costJ＊(xk,k)is nonincreasing with the increasing ofk.Then a necessary and sufficient condition for mean-square stabilization is obtained.

Lemma1Assume there exists Ψk＞0 in(4)satisfying the following matrix inequality for someHk,

The optimal costJ＊(xk,k)satisfies the following relation:

ProofLetJ＊(xk,k)be the optimal cost resulting from the optimal control(5)with the initial statexk.

whereare optimal controls minimizing the cost function(4)with initial state,respectively.are optimal state trajectories generated when the system is controlled by

Let us rep lace the controlin(12)with

whereHk+N+1is the control gain to be selected.Then,it follows from(12),(13)that

Further,note that the termin(14)can be rewritten as

where(2)has been used in the above equalities.

Thus it follow s from(14)that

where the facthas been used.By using the property of conditional expectation,we have

Therefore,

According to(10),we obtain

By using Lemma1,the main result of the paper is presented in the following Theorem 1.

Theorem 1GivenQk＞0 andRk＞0,then system(1)with the receding horizon control(9)is asymptotically mean square stable if and only if there exists Ψk＞0 andHksatisfying(10).

Proof(Sufficiency)According to Lemma 1,there exists Ψksatisfying(10),then we obtainSinceJ＊(xk,k)is nonincreasing andJ＊(xk,k)＞0,thus

exists,and

By virtue of(16),(17)and(10),we obtain

Com bined with(20),it yields

Note thatRk＞0,thus we get

Combining(23)and(22),one has

Note thatQk＞0,then

(Necessity)System(1)is mean-square stabilizable.There existuk=Hkxksuch that the closed-loop system is mean-square stability.Then according to stochastic Lyapunov stability theorem[24],for each sequence of positive definite matrices Θ(k)＞0,the following matrix difference equation

has positive definite solutions Π(k).let Θk=Qk+we obtain

which im p ly(10)holds.

Rem ark 2If w e letsystem(1)reduces to a deterministic system.The corresponding stabilizability condition(10)becomes

Com pared with the result in[16]and[17],where only sufficient stabilization condition has been considered,we give the necessary and sufficient RHC stabilization condition in this paper.Further,we have generalized the result from deterministic system to stochastic system s with multiplicative noise in this paper.

因?yàn)槌贿@么看。他與靜秋相戀兩年，除了吻過(guò)她的額頭，兩個(gè)人從沒(méi)有越雷池一步。他認(rèn)為靜秋對(duì)他越來(lái)越冷淡全因了康芳。——康芳有一個(gè)需要照顧的丈夫，康芳只有一個(gè)女兒，自私的她希望女兒和女婿能夠守在她的身邊。

In the following the time-invariant stochastic system is considered.If we letin(1),then system(1)reduced to a time-invariant stochastic system

Consider the cost function as:

The result analogous to Theorem 1 is expressed as follow s:

Corollary 1GivenQ＞0 andR＞0 in(28),then system(27)with the receding horizon control is asymptotically mean square stable if and only if there exists Ψ＞0 andHsatisfying

Rem ark 3Note that inequality(29)is equivalent to the following LMI,which is easier to be tested.

where

4 RHC for stochastic system s with multiple multiplicative noises

In this section,w e shall generalize the results in the previous section to stochastic system s with multiple multiplicative noises.Consider the more general system(3)and cost function(4).We first generalize the backwards recursion in(6)–(8)as follows:Forj=k,...,k+N,

with the terminal valueThe receding horizon control at timekis

Theorem 2GivenQk＞0 andRk＞0,then system(3)with the receding horizon control(31)is asymptotically mean square stable if and only if there exists Ψk＞0 andHksatisfying

ProofAs the proof is similar to that for Theorem 1,we have omitted it.□

5 Simulation

In this section,three examples are presented to illustrate the proposed method.

Example 1Consider a discrete time-varying stochastic system with multiplicative noise whose model parameters are given by

The weighting matricesQk,Rk,and Ψkin(10)are chosen to be 5,1,and 10,respectively.Hkin(10)is chosen to be?1.The horizon lengthNis chosen to be 3.According to Theorem 1 the stabilization condition(10)is satisfied.The corresponding receding horizon control gain curve is drawn in Fig.1 according to(9).

Fig.1 Control gain curve.

with the controller the state trajectory of the closed-loop system is drawn in Fig.2.From Fig.2,we can see that the proposed RHC stabilizes the discrete-time multiplicative noise system with differentQk,Rk,and Ψkof the cost.

Fig.3 represents the optimal receding horizon costsJ＊against time.It can be seen that the optimal receding horizon costsJ＊decrease monotonically with time and converge to zero.This cost monotonicity implies that the proposed RHC stabilizes the time-varying stochastic system.

Exam p le 2Consider the network control system depicted in[25].Suppose there is only data packet dropout in the network control system,then the overall network control system can be described as

w herexk∈Rnis the state,uk∈Rpis the control input.{γk}k≥0is modeled as a i.i.d Bernoulli process with probability distribution P(γk=0)=pand P(γk=1)=1?p,w herep∈(0,1)is named as the packet dropout rate.It can be seen that(32)is the special case of(1)with

Suppose system(32)withG=1.2,L=0.4,x0=0.1,p=0.5,and the cost function withQk=1,Rk=1,N=5,Ψk=10.By applying Theorem 1,it is easy to verify that condition(10)is satisfied.The state trajectoryof the closed-loop system with the controller is drawn in Fig.4.It can be seen that the proposed RHC stabilizes the network control system from Fig.4.

Fig.2 State trajectory E(x(k)′x(k))due to the proposed RHC.

Fig.3 Optimal receding horizon cost.

Fig.4 State trajectory E(x(k)′x(k))of network control system due to the proposed RHC.

Example 3Consider the two dimensional stochastic system with multiplicative noise whose parameters are given by

The weighting matrixesQandRin(29)are chosen to beI2andI1.The weighting matrixes Ψ and feedback gainHin(29)are decided by solving the linear matrix inequality(30)using MATLAB LM I toolbox which are given as

According to(9)the receding horizon controller is given as

The state trajectory of the closed loop system with the controller(33)is drawn in Fig.5.It is shown that the proposed RHC stabilizes the stochastic system with multiplicative noise.

Fig.5 State trajectory E(x(k)′x(k))due to the proposed RHC.

6 Conclusions

The paper has proposed a receding horizon control approach for stabilization of discrete time-varying stochastic system s.Explicit stabilization controller has been obtained by solving a generalized Riccati equation.By applying the tools of stochastic stability,a necessary and sufficient condition on the terminal weighting matrix has been proposed to guarantee the asymptotic mean-square stability of the closed-loop system.Some desirable extensions would be to time-varying stochastic system s with state or control delay.

References

[1]X.R.Mao.Stochastic Differential Equations and Applications.2nd ed.Chichester,U.K.:Horwood Publication,2007.

[2]L.Li,H.Zhang.Linear quadratic regultion for discrete-time systems with state delays and multiplicative noise.Control Theory and Technology,2015,13(4):348–359.

[3]J.L.willems,J.C.willems.Feedback stabilizability for stochastic system s with state and control dependent noise.Automatica,1976,12(3):277–283.

[4]U. Haussmann. Stability of linear systems with control dependent noise. SIAM Journal on Control, 1973, 11(2): 382 – 394.

[5]T.Morozan.Stabilization of some stochastic discrete-timecontrol system s.Stochastic Analysis and Applications,1983,1(1):89–116.

[6]E.Yaz.Stabilization of discrete-time system s with stochastic parameters.System s&Control Letters,1985,5(5):321–326.

[7]E.Yaz.Certainty equivalent control of stochastic system s:stability property.IEEE Transactions on Automatic Control,1986,3(12):178–180.

[8]M.A.Ram i,X.Zhou.Linear matrix inequalities,Riccati equations,and indefinite stochastic linear quadratic controls.IEEE Transactions on Automatic Control,2000,45(6):1131–1143.

[9]L.E.Ghaoui.State-feedback control of systems with multiplicative noise via linear matrix inequalities.System s&Control Letters,1995,24(3):223–228.

[10]W.Zhang,H.Zhang,B.Chen.Generalized Lyapunov equation app roach to state-dependent stochastic stabilization/detectability criterion.IEEE Transactions on Automatic Control,2008,53(7):1630–1642.

[11]N.Elia.Remote stabilization over fading channels.System s&Control Letters,2005,54(3):237–249.

[12]K.Y.You,L.H.Xie.Minimum data rate for mean square stabilization of discrete LTI system s over lossy channels.IEEE Transactions on Automatic Control,2010,55(10):2373–2378.

[13]S.Aberkane,V.Dragan.Robust stability and robust stabilization of a class of discrete-time time-varying linear stochastic system s.SAIM Journal on Control and Optimization,2015,53(1):30–57.

[14]L.Sheng,W.Zhang,M.Gao.Mixed H2/H∞control of time-varying stochastic discrete-time system s under uniform detectability.IET Control Theory and Application,2014,8(17):1866–1874.

[15]W.H.Kwon,A.E.Pearson.A modified quadratic cost problem and feedback stabilization of a linear system.IEEE Transactions on Automatic Control,1977,22(5):838–842.

[16]J.W.Lee,W.H.Kwon,J.H.Choi.On stability of constrained receding horizon control with finite terminal weighting matrix.Automatica,1998,34(12):1607–1612.

[17]W.H.Kwon,K.B.Kim.On stabilizing receding horizon controls for linear continuous time-invariant system s.IEEE Transactions on Automatic Control,2000,45(7):1329–1334.

[18]G.D.Nicolao,L.Magni,R.Scattolini.Stabilizing receding horizon control of nonlinear time-varying system s.IEEE Transactions on Automatic Control,1998,43(7):1030–1036.

[19]M.A.Mohammadkhani,F.Bayat,A.A.Jalali.Design of explicit model predictive control for constrained linear system s with disturbances.International Journal of Control,Automation,and System s,2014,12(2):294–301.

[20]J.A.Primbs,C.H.Sung.Stochastic receding horizon control of constrained linear system s with state and control multiplicative noise.IEEE Transactions on Automatic Control,2009,54(2):221–230.

[21]D.Bernardini,A.Bemporad.Stabilizing model predictive control of stochastic constrained linear system s.IEEE Transactions on Automatic Control,2012,57(6):1468–1480.

[22]H.Zhang,H.Wang,L.Li.Adapted and casual maxim um principle and analytical solution to optimal control for stochastic multiplicative noise systems with multiplt input-delays.Proceedings of the 51st IEEE Annual Conference on Decision and Control,Hawaii:IEEE,2012:2122–2177.

[23]L.Chen,Z.Wu.Maxim um principle for the stochastic optimal control problem with delay and application.Automatica,2010,46(6):1074–1080.

[24]S.Niwa,M.Hayase,I.Sugiura.Stability of linear time-varying system s with state dependent noise.IEEE Transactions on Automatic Control,1976,21(5):775–776.

[25]C.Tan,L.Li,H.Zhang.Stabilization of networked control system s with both network-induced delay and packet dropout.Automatica,2015,59:194–199.