The existence of quasi-stationary distributions for continuous-time Markov processes*

ZHANG Hanjun, ZHU Yixia

(1.School of Mathematics and Computational Science, Xiangtan University, Xiangtan 411105, China；2.School of Mathematics and Statistics, Hunan University of Finance and Economics, Changsha 410205, China)

Abstract：We consider Markov processes on the positive integers for which the origin is an absorbing state. Quasi-stationary distributions (QSDs) describes as the limiting behavior of an absorbing process when the process is conditioned to survive. Under the assumption that the expectation of the absorption time at the origin, iT0, of the process starting from state i goes to infinity as i→∞, we show that the existence of a QSD is equivalent to i(eθT0)<∞ for some positive θ and i. Finally, an application to a birth-death process is discussed to demonstrate the usefulness of our results.

Key words：Markov process; quasi-stationary distribution; birth and death process; renewal process

0 引言

We consider a continuous-time Markov process on a denumerable state spaceE={0}∪Cwhere 0 is the only absorbing state andC={1,2,…} is the set of transient states. We will denote byiits distribution started atiandμits distribution whenX0is distributed according to a probability measureμ. A quasi-stationary distribution (QSD) for the process is a probability measureμonCwith the property that, starting withμthe conditional distribution, given that absorption has not occurred by the timet, is stillμ. That is,

μj=μ(Xt=j|T0>t),t>0 ,

(1)

whereT0:=inf{t≥0:Xt=0} is the absorption time at 0.

The existence, uniqueness and domains of attraction of QSDs of Markov processes have been investigated by many papers[2-8]. The existence of QSDs for various Markov processes has been extensively discussed (see e.g.[8] and references therein). Intimately related with the notion of QSD is the so-called Yaglom limit:

If the Yaglom limit exists, then it is a QSD[9]. The existence of the Yaglom limit was established for branching processes[10]in a pioneer work, and for birth and death processes[5,11].In the case of a finite Markov chain, there is only one QSD and it is the Yaglom limit[12]. For a wide class of Markov processes, the existence of a QSD is equivalent toi(eθT0)<∞ for some positiveθandiunder the assumption that the absorption time at the origin,T0, of the process starting from stateigoes to infinity in probability asi→∞[3].The method is based on the study of the renewal process. More recently, assuming that the Markov process comes back quickly from infinity, the process admits a unique quasi-stationary distribution, and the distribution of the process converges exponentially fast in total variation norm to its quasi-stationary distribution[8]. In the special case of birth-death processes, a process evolving inCcan have no QSD, exactly one QSD or a continuum of QSDs[5].There is a unique quasi-stationary distribution that attracts all initial distributions supported inC, if and only if ∞ is an entrance boundary for the minimal birth-death process[11].

In this paper we study QSDs by means of the expectation of the absorption timeT0. The purpose of this paper is to obtain conditions for the existence of a QSD for a general Markov process. Under the assumption that the expectation ofT0, of the process starting from stateigoes to infinity asi→∞, a necessary and sufficient condition for the existence of a QSD is thati(eθT0)<∞ for someθ>0 and somei∈C.Our approach, inspired by [3], is based on a renewal dynamical process. For a general Markov process, we extend and improve Theorem 1.1 of [3].

This paper is organized as follows. In the next section, we give some definitions and notation and provide our main result concerning the existence of the QSDs. In Section 2, we prove the main result. This part may be regarded as an improvement of Ferrari 's works[3]. Some known results are reformulated so that we can apply them for subsequent developments. Finally, in Section 3, We will apply the result to birth and death processes.

1 Preliminaries and main result

LetQ=(qij,i,j∈E) be a transition rate matrix andP(t)=(Pij(t),i,j∈E) be the transition function. We will assume that there is sure killing at 0, soi(T0<∞)=1 for alli∈C.Letμbe a QSD. Starting fromμ,T0is exponentially distributed[4]. That is,

μ(T0>t)=e-θμtfor someθμ>0.

(2)

This result is useful in the study of QSDs. We integrate both sides of (2), obtainingμT0=1/θμ<∞.This implies thatiT0=1/θμ<∞ for somei∈C(and hence for alli∈C). We obtain easily thati(T0<∞)=1 for alli∈C. It should be remarked that ?i∈C,iT0<∞ is a necessary condition for the existence of QSD. Of course, our main interest is always to determine the QSDs. So we always assume thatiT0<∞ for alli∈C.

For an irreducible and surely absorbed Markov process, the exponential rate of the transition probabilities, denoted byθKand called the Kingman’s parameter[13], is well-defined and does not depend oni,j∈C,

For most of the results in this paper we shall use the basic hypothesis below.

{Xt}t≥0corresponding toQis an honest process.

We show a key lemma which is Theorem 1.1 of [3].

Lemma 1[3]Assume (H). Assume further that

(2)

Then a necessary and sufficient condition for the existence of a QSD is that

i(eθT0)<∞

(3)

for someθ>0 and somei∈C(and hence for alli).

The main result of this paper is the following.

Theorem 1Assume (H). Assume further that

(5)

Then a necessary and sufficient condition for the existence of a QSD is (4) holds.

ti(T0≥t)≤iT0

2 The existence of a QSD

Let Φ be the transformation Φ:μ→νwhereνis a unique solution of the following equation:

LetFμbe the distribution of the process starting withμand setmk(Fμ) as thekth moment ofFμ, and assume thatmk(Fμ)<∞.

Lemma 2[15]A subsetB?P0(C) is relative compact in the weak topology if and only if there exists a compact functionhand a constantKsuch that

(6)

ΦnμT0→θ.

Define

and

Φ:Mθ→Mθ.

ProofIn Lemma 4, We have proved thatMθis compact, and the convexity ofMθis trivial. Following the proof of Proposition 4.2 of [3], we can easily obtain the continuity of Φ. Then the transformation Φ has a fixed point follows from an application of the Schauder-Tychonov fixed point theorem([16], Theorem V.10.5).

Ifμis a fixed point inMθ, then Φμ=μ, which means thatμis an invariant distribution ofQμ, soμis a QSD. This completes the proof.

Proof of Theorem 1First, let us prove the condition (4) is necessary. If there exists a QSDμ, then

μ(T0>s+t)=μ(T0>s)μ(T0>t).

SoFμis exponential. Sincei(T0<∞)=1 for alli, alsoμ(T0<∞)=1 and consequently the exponentially distributedT0must haveμT0<∞. Thenθ<1/μT0, we haveμ(eθT0)<∞. Since the irreducibility ofQimplies thatμcharges every point and

i(eθT0)<(1/μi)μ(eθT0),

3 Applications

LetYtbe a birth-death process withq-matrixQgiven

qi,i+1=bi,i≥0;qi,i-1=di,i≥1;

whereb0=d0=0 anddi>0,bi>0 fori≥1.

We next turn to recall that the series

where {πn,n≥1} is the potential coefficients. Define the potential coefficients

π={πn,n∈C} byπ1=1 and, forn≥2,

Since we assume thatQis regular, hence[17]

It is well known that

see for example, [18], formula 7.10. Then for eachi∈C,iT0<∞ if and only ifis equivalent to the birth-death processYtonEis absorbed with probability 1[19].

Lemma 5[1]Suppose that a birth-death process onEis absorbed with probability 1[19](which is equivalent toA=∞).If we define

Proposition 2(The existence of a QSD for a birth-death process) For a birth-death processYtthat satisfiesA=∞, the following statements are equivalent:

(2)θK>0;

(3)δ<∞;

(4) There exist a QSD.

Proof(1) ～ (2). Statements (1) and (2) are equivalent by virtue of Theorem 3.3.2 of [20].

(2)～ (3). From Lemma 5, we haveθK>0 if and only ifδ<∞.

(2)～ (4). Statements (2) and (4) are equivalent by virtue of Theorem 3.2 of [5].

Then there exists a QSD if and only ifθK>0. This is consistent with the conclusion of Theorem 3.2 of [5].