Pdf of Continuous Exponential Random Variable

Exponential Random Variable

The exponential random variable is defined by the density function [see Fig.1-2b](1.4-5)P(x) = {a exp(–ax), if x≥0,0, if x>0,where a is any positive real number.

From: Markov Processes , 1992

Random Variables, Distributions, and Density Functions

Scott L. Miller , Donald Childers , in Probability and Random Processes, 2004

3.4.2 Exponential Random Variable

The exponential random variable has a probability density function and cumulative distribution function given (for any b > 0) by

(3.19a) $f_X(x)=\frac{1}{b}exp\left(-\frac{x}{b}\right)u(x),$

(3.19b) $f_X(x)=\left[1-exp\left(-\frac{x}{b}\right)\right]u(x).$

A plot of the PDF and the CDF of an exponential random variable is shown in Figure 3.9. The parameter b is related to the width of the PDF and the PDF has a peak value of 1/b which occurs at x = 0. The PDF and CDF are nonzero over the semi-infinite interval (0, ∞), which may be either open or closed on the left endpoint.

Figure 3.9. Probability density function (a) and cumulative distribution function (b) of an exponential random variable, b = 2.

Exponential random variables are commonly encountered in the study of queueing systems. The time between arrivals of customers at a bank, for example, is commonly modeled as an exponential random variable, as is the duration of voice conversations in a telephone network.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780121726515500038

The Exponential Distribution and the Poisson Process

Sheldon M. Ross , in Introduction to Probability Models (Twelfth Edition), 2019

5.2.1 Definition

A continuous random variable X is said to have an exponential distribution with parameter λ, $\lambda >0$ , if its probability density function is given by

$f(x)=\{\begin{array}{cc}\lambda e^{-\lambda x},\hfill & x⩾0\hfill \\ 0,\hfill & x<0\hfill \end{array}$

or, equivalently, if its cdf is given by

$F(x)={\int }_{-\infty }^{x}f(y)dy=\{\begin{array}{cc}1-e^{-\lambda x},\hfill & x⩾0\hfill \\ 0,\hfill & x<0\hfill \end{array}$

The mean of the exponential distribution, $E[X]$ , is given by

$\begin{array}{cc}\hfill E[X]=& {\int }_{-\infty }^{\infty }xf(x)dx\hfill \\ \hfill =& {\int }_0^{\infty }\lambda x{e}^{-\lambda x}dx\hfill \end{array}$

Integrating by parts $(u=x,dv=\lambda e^{-\lambda x}dx)$ yields

${E[X]=-x{e}^{-\lambda x}|}_0^{\infty }+{\int }_0^{\infty }{e}^{-\lambda x}dx=\frac{1}{\lambda }$

The moment generating function $\varphi (t)$ of the exponential distribution is given by

(5.1) $\begin{array}{cc}\hfill \varphi (t)=& E[e^{tX}]\hfill \\ \hfill =& {\int }_0^{\infty }{e}^{tx}\lambda e^{-\lambda x}dx\hfill \\ \hfill =& \frac{\lambda }{\lambda -t}\text{for}t<\lambda \hfill \end{array}$

All the moments of X can now be obtained by differentiating Eq. (5.1). For example,

$\begin{array}{cc}\hfill & {E[X^2]=\frac{d^2}{d{t}^2}\varphi (t)|}_{t=0}\hfill \\ \hfill & {=\frac{2\lambda }{{(\lambda -t)}^3}|}_{t=0}\hfill \\ \hfill & =\frac{2}{{\lambda }^2}\hfill \end{array}$

Consequently,

$\begin{array}{cc}\hfill \text{Var}(X)=& E[X^2]-{(E[X])}^2\hfill \\ \hfill =& \frac{2}{{\lambda }^2}-\frac{1}{{\lambda }^2}\hfill \\ \hfill =& \frac{1}{{\lambda }^2}\hfill \end{array}$

Example 5.1 Exponential Random Variables and Expected Discounted Returns

Suppose that you are receiving rewards at randomly changing rates continuously throughout time. Let $R(x)$ denote the random rate at which you are receiving rewards at time x. For a value $\alpha ⩾0$ , called the discount rate, the quantity

$R={\int }_0^{\infty }{e}^{-\alpha x}R(x)dx$

represents the total discounted reward. (In certain applications, α is a continuously compounded interest rate, and R is the present value of the infinite flow of rewards.) Whereas

$E[R]=E[{\int }_0^{\infty }{e}^{-\alpha x}R(x)dx]={\int }_0^{\infty }{e}^{-\alpha x}E[R(x)]dx$

is the expected total discounted reward, we will show that it is also equal to the expected total reward earned up to an exponentially distributed random time with rate α.

Let T be an exponential random variable with rate α that is independent of all the random variables $R(x)$ . We want to argue that

${\int }_0^{\infty }{e}^{-\alpha x}E[R(x)]dx=E[{\int }_0^{T}R(x)dx]$

To show this define for each $x⩾0$ a random variable $I(x)$ by

$I(x)=\{\begin{array}{cc}1,\hfill & \text{if}x⩽T\hfill \\ 0,\hfill & \text{if}x>T\hfill \end{array}$

and note that

${\int }_0^{T}R(x)dx={\int }_0^{\infty }R(x)I(x)dx$

Thus,

$\begin{array}{cc}\hfill E[{\int }_0^{T}R(x)dx]=& E[{\int }_0^{\infty }R(x)I(x)dx]\hfill \\ \hfill =& {\int }_0^{\infty }E[R(x)I(x)]dx\hfill \\ \hfill =& {\int }_0^{\infty }E[R(x)]E[I(x)]dx\text{by}\text{independence}\hfill \\ \hfill =& {\int }_0^{\infty }E[R(x)]P\{T⩾x\}dx\hfill \\ \hfill =& {\int }_0^{\infty }{e}^{-\alpha x}E[R(x)]dx\hfill \end{array}$

Therefore, the expected total discounted reward is equal to the expected total (undiscounted) reward earned by a random time that is exponentially distributed with a rate equal to the discount factor.   ■

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B978012814346900010X

Operations on a Single Random Variable

Scott L. Miller , Donald Childers , in Probability and Random Processes (Second Edition), 2012

4.7 Characteristic Functions

In this section, we introduce the concept of a characteristic function. The characteristic function of a random variable is closely related to the Fourier transform of the PDF of that random variable. Thus, the characteristic function provides a sort of "frequency domain" representation of a random variable, although in this context there is no connection between our frequency variable ω and any physical frequency. In studies of deterministic signals, it was found that the use of Fourier transforms greatly simplified many problems, especially those involving convolutions. We will see in future chapters the need for performing convolution operations on PDFs of random variables, and hence frequency domain tools will become quite useful. Furthermore, we will find that characteristic functions have many other uses. For example, the characteristic function is quite useful for finding moments of a random variable. In addition to the characteristic function, two other related functions, namely, the moment-generating function (analogous to the Laplace transform) and the probability-generating function (analogous to the z-transform), will also be studied in the following sections.

Definition 4.7: The characteristic function of a random variable, X, is given by

(4.36) ${\Phi }_X\left(\omega \right)=E\left[{\text{e}}^{j\omega X}\right]={\int }_{-\infty }^{\infty }{\text{e}}^{j\omega x}{f}_X\left(x\right)\text{d}x.$

Note the similarity between this integral and the Fourier transform. In most of the electrical engineering literature, the Fourier transform of the function f_X (x) would be Φ(−ω). Given this relationship between the PDF and the characteristic function, it should be clear that one can get the PDF of a random variable from its characteristic function through an inverse Fourier transform operation:

(4.37) $f_X\left(x\right)=\frac{1}{2\pi }{\int }_{-\infty }^{\infty }{\text{e}}^{-j\omega x}{\Phi }_X\left(\omega \right)\text{d}\omega .$

The characteristic functions associated with various random variables can be easily found using tables of commonly used Fourier transforms, but one must be careful since the Fourier integral used in Equation (4.36) may be different from the definition used to generate common tables of Fourier transforms. In addition, various properties of Fourier transforms can also be used to help calculate characteristic functions as shown in the following example.

Example 4.18

An exponential random variable has a PDF given by f_X (x) = exp(−x)u(x). Its characteristic function is found to be

${\Phi }_X\left(\omega \right)={\int }_{-\infty }^{\infty }{\text{e}}^{j\omega x}{f}_X\left(x\right)\text{d}x={\int }_0^{\infty }{\text{e}}^{j\omega x}{\text{e}}^{-x}\text{d}x=-{\frac{{\text{e}}^{-\left(1-j\omega \right)x}}{1-j\omega }|}_0^{\infty }=\frac{1}{1-j\omega }.$

This result assumes that ω is a real quantity. Now suppose another random variable Y has a PDF given by f_Y (y) = a exp(−ay)u(y). Note that f_Y (y) = af_X (ay), thus using the scaling property of Fourier transforms, the characteristic function associated with the random variable Y is given by

${\Phi }_Y\left(\omega \right)=a\frac{1}{\left|a\right|}{\Phi }_X\left(\frac{\omega }{a}\right)=\frac{1}{1-j\omega /a}=\frac{a}{a-j\omega },$

assuming a is a positive constant (which it must be for Y to have a valid PDF). Finally, suppose that Z has a PDF given by f_Z (z) = a exp(−a(z − b))u(z − b). Since f_Z (z) = f_Y (z − b), the shifting property of Fourier transforms can be used to help find the characteristic function associated with the random variable Z:

${\Phi }_Z\left(\omega \right)={\Phi }_Y\left(\omega \right){\text{e}}^{-j\omega b}=\frac{a{\text{e}}^{-j\omega b}}{a-j\omega }.$

The next example demonstrates that the characteristic function can also be computed for discrete random variables. In Section 4.8, the probability-generating function will be introduced which is preferred by some when dealing with discrete random variables.

Example 4.19

A binomial random variable has a PDF which can be expressed as

$f_X\left(x\right)=\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right)p^{\kappa }{\left(1-p\right)}^{n-k}\delta \left(x-k\right).$

Its characteristic function is computed as follows:

$\begin{array}{c}{\Phi }_X\left(\omega \right)={\int }_{-\infty }^{\infty }{\text{e}}^{j\omega x}\left(\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right)p^k{\left(1-p\right)}^{n-k}\delta \left(x-k\right)\right)\text{d}x=\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right)p^k{\left(1-p\right)}^{n-k}{\int }_{-\infty }^{\infty }\delta \left(x-k\right)e^{j\omega x}\text{d}x\\ =\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right)p^k{\left(1-p\right)}^{n-k}{\text{e}}^{j\omega k}=\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right){\left(p{\text{e}}^{j\omega }\right)}^k{\left(1-p\right)}^{n-k}={\left(1-p+p{\text{e}}^{j\omega }\right)}^n.\end{array}$

Since the Gaussian random variable plays such an important role in so many studies, we derive its characteristic function in Example 4.20. We recommend that the student commit the result of this example to memory. The techniques used to arrive at this result are also important and should be carefully understood.

Example 4.20

For a standard normal random variable, the characteristic function can be found as follows:

${\Phi }_X\left(\omega \right)={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }}{\text{e}}^{-\frac{x^2}{2}}{\text{e}}^{J\omega x}\text{d}x={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }}\text{exp}\left(-\frac{\left(x^2-2j\omega x\right)}{2}\right)\text{d}x.$

To evaluate this integral, we complete the square in the exponent.

${\Phi }_X\left(\omega \right)=\text{exp}\left(-\frac{{\omega }^2}{2}\right){\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }}\text{exp}\left(-\frac{\left(x^2-2j\omega x-{\omega }^2\right)}{2}\right)\text{d}x=\text{exp}\left(-\frac{{\omega }^2}{2}\right){\int }_{-\infty }^{\infty }\frac{1}{2\pi }\text{exp}\left(-\frac{{\left(x-j\omega \right)}^2}{2}\right)\text{d}x.$

The integrand in the above expression looks like the properly normalized PDF of a Gaussian random variable, and since the integral is over all values of x, the integral must be unity. However, close examination of the integrand reveals that the "mean" of this Gaussian integrand is complex. It is left to the student to rigorously verify that this integral still evaluates to unity even though the integrand is not truly a Gaussian PDF (since it is a complex function and hence not a PDF at all). The resulting characteristic function is then

${\Phi }_X\left(\omega \right)=\text{exp}\left(-\frac{{\omega }^2}{2}\right).$

For a Gaussian random variable whose mean is not zero or whose standard deviation is not unity (or both), the shifting and scaling properties of Fourier transforms can be used to show that

$f_X\left(x\right)=\frac{1}{\sqrt{2\pi {\sigma }^2}}{\text{e}}^{-\frac{{\left(x-\mu \right)}^2}{2{\sigma }^2}}\leftrightarrow {\Phi }_X\left(\omega \right)=\text{exp}\left(j\mu \omega -\frac{{\omega }^2{\sigma }^2}{2}\right).$

Theorem 4.3: For any random variable whose characteristic function is differentiable at ω= 0,

(4.38) $E\left[X\right]=-j{\frac{\text{d}}{\text{d}\omega }{\Phi }_X\left(\omega \right)|}_{\omega =0}.$

Proof: The proof follows directly from the fact that the expectation and differentiation operations are both linear and consequently the order of these operations can be exchanged.

$\frac{\text{d}}{\text{d}\omega }{\Phi }_X\left(\omega \right)=\frac{\text{d}}{\text{d}\omega }\left(E\left[{\text{e}}^{j\omega X}\right]\right)=E\left[\frac{\text{d}}{\text{d}\omega }\left({\text{e}}^{j\omega X}\right)\right]=E\left[jX{\text{e}}^{j\omega X}\right]=jE\left[X{\text{e}}^{j\omega X}\right].$

Multiplying both sides by −j and evaluating at ω = 0 produces the desired result.

Theorem 4.3 demonstrates a very powerful use of the characteristic function. Once the characteristic function of a random variable has been found, it is generally a very straightforward thing to produce the mean of the random variable. Furthermore, by taking the kth derivative of the characteristic function and evaluating at ω = 0, an expression proportional to the kth moment of the random variable is produced. In particular,

(4.39) $E\left[X^k\right]={\left(-j\right)}^k{\frac{{\text{d}}^k}{\text{d}{\omega }^k}{\Phi }_X\left(\omega \right)|}_{\omega =0}.$

Hence, the characteristic function represents a convenient tool to easily determine the moments of a random variable.

Example 4.21

Consider the exponential random variable of Example 4.18 where f_Y (y) = a exp(−ay)u(y). The characteristic function was found to be

${\Phi }_Y\left(\omega \right)=\frac{a}{a-j\omega }.$

The derivative of the characteristic function is

$\frac{\text{d}}{\text{d}\omega }{\Phi }_Y\left(\omega \right)=\frac{ja}{{\left(a-j\omega \right)}^2},$

and thus the first moment of Y is

$E\left[Y\right]=-j{\frac{\text{d}}{\text{d}\omega }{\Phi }_Y\left(\omega \right)|}_{\omega =0}={\frac{a}{{\left(a-j\omega \right)}^2}|}_{\omega =0}=\frac{1}{a}.$

For this example, it is not difficult to show that the kth derivative of the characteristic function is

$\frac{{\text{d}}^k}{\text{d}{\omega }^k}{\Phi }_Y\left(\omega \right)=\frac{j^{k}k!a}{{\left(a-j\omega \right)}^{k+1}},$

and from this, the kth moment of the random variable is found to be

$E\left[Y^k\right]={\left(-j\right)}^k{\frac{{\text{d}}^k}{\text{d}{\omega }^k}{\Phi }_Y\left(\omega \right)|}_{\omega =0}={\frac{k!a}{{\left(a-j\omega \right)}^{k+1}}|}_{\omega =0}=\frac{k!}{a^k}.$

For random variables that have a more complicated characteristic function, evaluating the kth derivative in general may not be an easy task. However, Equation (4.39) only calls for the kth derivative evaluated at a single point (ω = 0), which can be extracted from the Taylor series expansion of the characteristic function. To see this, note that from Taylor's theorem, the characteristic function can be expanded in a power series as

(4.40) ${\Phi }_X\left(\omega \right)=\sum _{k=0}^{\infty }\frac{1}{k!}\left({\frac{{\text{d}}^k}{\text{d}{\omega }^k}{\Phi }_X\left(\omega \right)|}_{\omega =0}\right){\omega }^k.$

If one can obtain a power series expansion of the characteristic function, then the required derivatives are proportional to the coefficients of the power series. Specifically, suppose an expansion of the form

is obtained. Then the derivatives of the characteristic function are given by

(4.42) ${\frac{{\text{d}}^k}{\text{d}{\omega }^k}{\Phi }_X\left(\omega \right)|}_{\omega =0}=k!{\phi }_k.$

The moments of the random variable are then given by

This procedure is illustrated using a Gaussian random variable in the next example.

Example 4.22

Consider a Gaussian random variable with a mean of μ = 0 and variance σ². Using the result of Example 4.20, the characteristic function is Φ_X(ω) = exp(−ω²σ²/2). Using the well-known Taylor series expansion of the exponential function, the characteristic function is expressed as

${\Phi }_X\left(\omega \right)=\sum _{n=0}^{\infty }\frac{{\left(-{\omega }^2{\sigma }^2/2\right)}^n}{n!}=\sum _{n=0}^{\infty }\frac{{(-1)}^n{\sigma }^{2n}}{2^{n}n!}{\omega }^{2n}.$

The coefficients of the general power series as expressed in Equation (4.41) are given by

${\phi }_k=\{\begin{array}{cc}\frac{j^k{\left(\sigma /\sqrt{2}\right)}^k}{\left(k/2\right)!},& k\text{even},\\ 0,& k\text{odd}.\end{array}$

Hence, the moments of the zero-mean Gaussian random variable are

$E\left[X^k\right]=\{\begin{array}{cc}\frac{k!}{\left(k/2\right)!}{\left(\frac{\sigma }{\sqrt{2}}\right)}^k,& k\text{even},\\ 0,& k\text{odd}.\end{array}$

As expected, E[X ⁰] = 1, E[X] = 0 (since it was specified that μ = 0), and E[X ²] = σ² (since in the case of zero-mean variables, the second moment and variance are one and the same). Now, we also see that E[X ³] = 0 (as are all odd moments), E[X ⁴] = 3σ⁴, E[X ⁶] = 15σ⁶, and so on. We can also conclude from this that for Gaussian random variables, the coefficient of skewness is c _s = 0 while the coefficient of kurtosis is c _k = 3.

In many cases of interest, the characteristic function has an exponential form. The Gaussian random variable is a typical example. In such cases, it is convenient to deal with the natural logarithm of the characteristic function.

Definition 4.8: In general, we can write a series expansion of ln[Φ _X (ω)] as

(4.44) $\text{ln}\left[{\Phi }_X\left(\omega \right)\right]=\sum _{n=1}^{\infty }{\lambda }_n\frac{{\left(j\omega \right)}^n}{n!},$

where the coefficients, λ _n , are called the cumulants and are given as

(4.45) ${\lambda }_n=\frac{{\text{d}}^n}{\text{d}{\left(j\omega \right)}^n}{\left\{ln\left[{\Phi }_X\left(\omega \right)\right]\right\}|}_{\omega =\text{C}},n=1,2,3,\dots$

The cumulants are related to the moments of the random variable. By taking the derivatives specified in Equation (4.45) we obtain

(4.48) ${\lambda }_3=E\left[X^3\right]-3{\mu }_{X}E\left[X^2\right]+2{\mu }_X^3=E\left[{\left(X-{\mu }_X\right)}^3\right].$

Thus, λ₁ is the mean, λ₂ is the second central moment (or the variance), and λ₃ is the third central moment. However, higher-order cumulants are not as simply related to the central moments.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123869814500072

Operations on a Single Random Variable

Scott L. Miller , Donald Childers , in Probability and Random Processes, 2004

4.7 Characteristic Functions

In this section we introduce the concept of a characteristic function. The characteristic function of a random variable is closely related to the Fourier transform of the PDF of that random variable. Thus, the characteristic function provides a sort of "frequency domain" representation of a random variable, although in this context there is no connection between our frequency variable ω and any physical frequency. In studies of deterministic signals, it was found that the use of Fourier transforms greatly simplified many problems, especially those involving convolutions. We will see in future chapters the need for performing convolution operations on PDFs of random variables and hence frequency domain tools will become quite useful. Furthermore, we will find that characteristic functions have many other uses. For example, the characteristic function is quite useful for finding moments of a random variable. In addition to the characteristic function, two other related functions, namely, the moment-generating function (analogous to the Laplace transform) and the probability-generating function (analogous to the z-transform), will also be studied in the following sections.

DEFINITION 4.7: The characteristic function of a random variable, X, is given by

(4.35) ${\Phi }_X\left(\omega \right)=E\left[e^{j\omega X}\right]={\int }_{-\infty }^{\infty }{e}^{j\omega x}{f}_X\left(x\right)dx.$

Note the similarity between this integral and the Fourier transform. In most of the electrical engineering literature, the Fourier transform of the function fX(x) would be Φ(– ω). Given this relationship between the PDF and the characteristic function, it should be clear that one can get the PDF of a random variable from its characteristic function through an inverse Fourier transform operation:

(4.36) $f_X\left(x\right)=\frac{1}{2\pi }{\int }_{-\infty }^{\infty }{e}^{-j\omega x}{\Phi }_X\left(\omega \right)d\omega .$

The characteristic functions associated with various random variables can be easily found using tables of commonly used Fourier transforms, but one must be careful since the Fourier integral used in Equation 4.35 may be different from the definition used to generate common tables of Fourier transforms. In addition, various properties of Fourier transforms can also be used to help calculate characteristic functions as shown in the following example.

EXAMPLE 4.18: An exponential random variable has a PDF given by fx(x) = exp(–x)u(x). Its characteristic function is found to be

${\Phi }_X\left(\omega \right)={\int }_{-\infty }^{\infty }{e}^{j\omega x}{f}_X\left(x\right)dx={\int }_0^{\infty }{e}^{j\omega x}{e}^{-x}dx=-\frac{e^{-\left(1-j\omega \right)x}}{1-j\omega }|\begin{array}{c}\infty \\ 0\end{array}=\frac{1}{1-j\omega }.$

This result assumes that ω is a real quantity. Now suppose another random variable Y has a PDF given by f_Y (y) = a exp(-ay)u(y). Note that f_Y (y) = af_X (ay), thus using the scaling property of Fourier transforms, the characteristic function associated with the random variable Y is given by

${\Phi }_Y\left(\omega \right)=a\frac{1}{\left|a\right|}{\Phi }_X\left(\frac{\omega }{a}\right)=\frac{1}{1-j\omega /a}=\frac{a}{a-j{\omega }^{\prime }}$

assuming a is a positive constant (which it must be for Y to have a valid PDF). Finally, suppose that Z has a PDF given by f_Z (z) = a exp(-a(z - b))u(z - b). Since f_Z (z)=f_Y (z-b), the shifting property of Fourier transforms can be used to help find the characteristic function associated with the random variable Z:

${\Phi }_Z\left(\omega \right)={\Phi }_Y\left(\omega \right)e^{-j\omega b}=\frac{a{e}^{-j\omega b}}{a-j\omega }.$

The next example demonstrates that the characteristic function can also be computed for discrete random variables. In Section 4.8, the probability-generating function will be introduced and is preferred by some when dealing with discrete random variables.

EXAMPLE 4.19: A binomial random variable has a PDF that can be expressed as

$f_X\left(x\right)=\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right)p^k{\left(1-p\right)}^{n-k}\delta \left(x-k\right).$

Its characteristic function is computed as follows:

$\begin{array}{l}{\Phi }_X\left(\omega \right)={\int }_{-\infty }^{\infty }{e}^{j\omega x}\left(\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right)p^k{\left(1-p\right)}^{n-k}\delta \left(x-k\right)\right)dx\\ =\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right)p^k{\left(1-p\right)}^{n-k}{\int }_{-\infty }^{\infty }\delta \left(x-k\right)e^{j\omega x}dx=\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right)p^k{\left(1-p\right)}^{n-k}{e}^{j\omega k}\\ =\sum _{k=0}^n\left(\begin{array}{c}n\\ k\end{array}\right){\left(p{e}^{j\omega }\right)}^k{\left(1-p\right)}^{n-k}={\left(1-p+p{e}^{j\omega }\right)}^n.\end{array}$

Since the Gaussian random variable plays such an important role in so many studies, we derive its characteristic function in Example 4.20. We recommend that the student commit the result of this example to memory. The techniques used to arrive at this result are also important and should be carefully studied and understood.

EXAMPLE 4.20: For a standard normal random variable, the characteristic function can be found as follows:

${\Phi }_X\left(\omega \right)={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }}{e}^{-\frac{x^2}{2}}{e}^{j\omega x}dx={\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }}exp\left(-\frac{\left(x^2-2j\omega x\right)}{2}\right)dx.$

To evaluate this integral, we complete the square in the exponent.

$\begin{array}{l}{\Phi }_X\left(\omega \right)=\text{exp}\left(-\frac{{\omega }^{\text{2}}}{2}\right){\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }}\text{exp}\left(-\frac{\left(x^2-2j\omega x-{\omega }^2\right)}{2}\right)dx\\ =exp\left(-\frac{{\omega }^2}{2}\right){\int }_{-\infty }^{\infty }\frac{1}{\sqrt{2\pi }}\text{exp}\left(-\frac{{\left(x-j\omega \right)}^2}{2}\right)dx\end{array}$

The integrand in this expression looks like the properly normalized PDF of a Gaussian random variable, and since the integral is over all values of x, the integral must be unity. However, close examination of the integrand reveals that the "mean" of this Gaussian integrand is complex. It is left to the student to rigorously verify that this integral still evaluates to unity even though the integrand is not truly a Gaussian PDF (since it is a complex function and hence not a PDF at all). The resulting characteristic function is then

${\Phi }_X\left(\omega \right)=exp\left(-\frac{{\omega }^2}{2}\right).$

For a Gaussian random variable whose mean is not zero or whose standard deviation is not unity (or both), the shifting and scaling properties of Fourier transforms can be used to show that

$f_X\left(x\right)=\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{{\left(x-\mu \right)}^2}{2{\sigma }^2}}\leftrightarrow {\Phi }_X\left(\omega \right)=exp\left(j\mu \omega -\frac{{\omega }^2{\sigma }^2}{2}\right).$

THEOREM 4.3: For any random variable whose characteristic function is differentiable at ω = 0,

(4.37) $E\left[X\right]=-j\frac{d}{d\omega }{\Phi }_X\left(\omega \right)|\begin{array}{c}\\ \omega =0\end{array}.$

PROOF: The proof follows directly from the fact that the expectation and differentiation operations are both linear and consequently the order of these operations can be exchanged.

$\frac{d}{d\omega }{\Phi }_X\left(\omega \right)=\frac{d}{d\omega }E\left[e^{j\omega X}\right]=E\left[\frac{d}{d\omega }{e}^{j\omega X}\right]=E\left[jX{e}^{j\omega X}\right]=jE\left[X{e}^{j\omega X}\right].$

Multiplying both sides by –j and evaluating at ω = 0 produces the desired result.▪

Theorem 4.3 demonstrates a very powerful use of the characteristic function. Once the characteristic function of a random variable has been found, it is generally a very straightforward thing to produce the mean of the random variable. Furthermore, by taking the k^th derivative of the characteristic function and evaluating at ω = 0, an expression proportional to the k^th moment of the random variable is produced. In particular,

(4.38) $E\left[X^k\right]={\left(-j\right)}^k\frac{d^k}{d{\omega }^k}{\Phi }_X\left(\omega \right)|\begin{array}{c}\\ \omega =0\end{array}.$

Hence, the characteristic function represents a convenient tool to easily determine the moments of a random variable.

EXAMPLE 4.21: Consider the exponential random variable of Example 4.18 where f_Y (y)=aexp(-ay)u(y). The characteristic function was found to be

${\Phi }_Y\left(\omega \right)=\frac{a}{a-j\omega }.$

The derivative of the characteristic function is

$\frac{d}{d\omega }\Phi \gamma \left(\omega \right)=\frac{ja}{{\left(a-j\omega \right)}^2},$

and thus the first moment of Y is

$E\left[Y\right]=-j\frac{d}{d\omega }{\Phi }_Y\left(\omega \right)|\begin{array}{c}\\ \omega =0\end{array}=\frac{a}{{\left(a-j\omega \right)}^2}|\begin{array}{c}\\ \omega =0\end{array}=\frac{1}{a}.$

For this example, it is not difficult to show that the kth derivative of the characteristic function is

$\frac{d^k}{d{\omega }^k}{\Phi }_Y\left(\omega \right)=\frac{j^{k}k!a}{{\left(a-j\omega \right)}^{k+1}},$

and from this, the kth moment of the random variable is found to be

$E\left[Y^k\right]={\left(-j\right)}^k\frac{d^k}{d{\omega }^k}{\Phi }_Y\left(\omega \right)|\begin{array}{c}\\ \omega =0\end{array}=\frac{k!a}{{\left(a-j\omega \right)}^{k+1}}|\begin{array}{c}\\ \omega =0\end{array}=\frac{k!}{a^k}.$

For random variables that have a more complicated characteristic function, evaluating the k^th derivative in general may not be an easy task. However, Equation 4.38 only calls for the kth derivative evaluated at a single point (ω = 0), which can be extracted from the Taylor series expansion of the characteristic function. To see this, note that from Taylor's Theorem, the characteristic function can be expanded in a power series as

(4.39) ${\Phi }_X\left(\omega \right)=\sum _{k=0}^{\infty }\frac{1}{k!}\left(\frac{d^k}{d{\omega }^k}{\Phi }_X\left(\omega \right)|\begin{array}{c}\\ \omega =0\end{array}\right){\omega }^k.$

If one can obtain a power series expansion of the characteristic function, then the required derivatives are proportional to the coefficients of the power series. Specifically, suppose an expansion of the form

(4.40) ${\Phi }_X\left(\omega \right)=\sum _{k=0}^{\infty }{\phi }_k{\omega }^k$

is obtained. Then the derivatives of the characteristic function are given by

(4.41) $\frac{d^k}{d{\omega }^k}{\Phi }_X\left(\omega \right)|\begin{array}{c}\\ \omega =0\end{array}=k!{\phi }_k.$

The moments of the random variable are then given by

(4.42) $E\left[X^k\right]={\left(-j\right)}^{k}k!{\phi }_k.$

This procedure is illustrated using a Gaussian random variable in the next example.

EXAMPLE 4.22: Consider a Gaussian random variable with a mean of μ = 0 and variance σ ². Using the result of Example 4.20, the characteristic function is ΦX(ω)=exp(-ω²σ²/2). Using the well-known Taylor series expansion of the exponential function, the characteristic function is expressed as

${\Phi }_X\left(\omega \right)=\sum _{n=0}^{\infty }\frac{{\left(-{\omega }^2{\sigma }^2/2\right)}^n}{n!}=\sum _{n=0}^{\infty }\frac{{\left(-1\right)}^n{\sigma }^{2n}}{2^{n}n!}{\omega }^{2n}.$

The coefficients of the general power series as expressed in Equation 4.40 are given by

${\phi }_k=\{\begin{array}{cc}\frac{j^k{\left(\sigma /\sqrt{2}\right)}^k}{\left(k/2\right)!}& k\text{even}\\ 0& k\text{odd}\end{array}.$

Hence the moments of the zero-mean Gaussian random variable are

$E\left[X^k\right]=\{\begin{array}{cc}\frac{k!}{\left(k/2\right)!}{\left(\frac{\sigma }{\sqrt{2}}\right)}^k& k\text{even}\\ 0& k\text{odd}\end{array}.$

As expected, E[X ⁰] = 1, E[X] = 0 (since it was specified that μ = 0), and E[X²]= σ ² (since in the case of zero-mean variables, the second moment and variance are one and the same). Now, we also see that E[X ³]= 0 (i.e., all odd moments are equal to zero), E[X ⁴]=3σ⁴, E[X⁶] =15σ⁶, and so on. We can also conclude from this that for Gaussian random variables, the coefficient of skewness is c_s = 0, while the coefficient of kurtosis is c_k = 3.

In many cases of interest, the characteristic function has an exponential form. The Gaussian random variable is a typical example. In such cases, it is convenient to deal with the natural logarithm of the characteristic function.

DEFINITION 4.8: In general, we can write a series expansion of ln[Φx(ω)] as

(4.43) $\text{ln}\left[{\Phi }_{\text{X}}\left(\omega \right)\right]=\sum _{n=1}^{\infty }{\lambda }_n\frac{{\left(j\omega \right)}^n}{n!}.$

where the coefficients, λ_n, are called the cumulants and are given as

(4.44) ${\lambda }_n=\frac{d^n}{d{\left(j\omega \right)}^n}\{ln\left[{\Phi }_X\left(\omega \right)\right]\}|\begin{array}{c}\\ \omega =0^{\prime }\end{array}n=1,2,3,.\dots$

The cumulants are related to the moments of the random variable. By taking the derivatives specified in Equation 4.44 we obtain

(4.45) ${\lambda }_1={\mu }_X,$

(4.46) ${\lambda }_2=E\left[X^2\right]-{\mu }_X^2={\sigma }_X^2,$

(4.47) ${\lambda }_3=E\left[X^3\right]-3{\mu }_{X}E\left[X^2\right]+2{\mu }_X^3=E\left[{\left(X-{\mu }_X\right)}^3\right].$

Thus, λ₁ is the mean, λ₂ is the second central moment (or the variance), and λ3 is the third central moment. However, higher order cumulants are not as simply related to the central moments.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B978012172651550004X

Random Variables, Distributions, and Density Functions

Scott L. Miller , Donald Childers , in Probability and Random Processes (Second Edition), 2012

3.4.7 Rayleigh Random Variable

A Rayleigh random variable, like the exponential random variable, has a one-sided PDF. The functional form of the PDF and CDF is given (for any σ > 0) by

(3.28a) $f_X\left(x\right)=\frac{x}{{\sigma }^2}exp\left(\frac{x^2}{2{\sigma }^2}\right)u\left(x\right),$

(3.28b) $F_X\left(x\right)=\left(1-exp\left(\frac{x^2}{2{\sigma }^2}\right)\right)u\left(x\right).$

Plots of these functions are shown in Figure 3.11. The Rayleigh distribution is described by a single parameter, σ ², which is related to the width of the Rayleigh PDF. In this case, the parameter σ ² is not to be interpreted as the variance of the Rayleigh random variable. It will be shown later that the Rayleigh distribution arises when studying the magnitude of a complex number whose real and imaginary parts both follow a zero-mean Gaussian distribution. The Rayleigh distribution arises often in the study of noncoherent communication systems and also in the study of wireless communication channels, where the phenomenon known as fading is often modeled using Rayleigh random variables.

Figure 3.11. (a) Probability density function and (b) cumulative distribution function of a Rayleigh random variable, σ ² = ½.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123869814500060

Basic Concepts in Probability

Oliver C. Ibe , in Markov Processes for Stochastic Modeling (Second Edition), 2013

1.8.6 The Exponential Distribution

A continuous random variable X is defined to be an exponential random variable (or X has an exponential distribution) if for some parameter λ>0 its PDF is given by

$f_X(x)=\{\begin{array}{ll}\lambda {\mathrm{e}}^{-\lambda x}\hfill & x\ge 0\hfill \\ 0\hfill & x<0\hfill \end{array}$

The CDF, mean, and variance of X, and the s-transform of its PDF are given by

$\begin{array}{ll}\hfill F_X(x)& =P[X\le x]=1-{\mathrm{e}}^{-\lambda x}\hfill \\ \hfill E\left[X\right]& =\frac{1}{\lambda }\hfill \\ \hfill E\left[X^2\right]& =\frac{2}{{\lambda }^2}\hfill \\ \hfill {\sigma }_X^2& =E\left[X^2\right]-{(E[X])}^2=\frac{1}{{\lambda }^2}\hfill \\ \hfill M_X(s)& =\frac{\lambda }{s+\lambda }\hfill \end{array}$

Like the geometric distribution, the exponential distribution possesses the forgetfulness property. Thus, if we consider the occurrence of an event governed by the exponential distribution as an arrival, then given that no arrival has occurred up to time t, the time until the next arrival is exponentially distributed with mean 1/λ. In particular, it can be shown, as in Ibe (2005), that

$f_{X|X>t}(x|X>t)=\lambda {\mathrm{e}}^{-\lambda (x-t)}x>t$

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780124077959000013

Functions of Random Variables

Oliver C. Ibe , in Fundamentals of Applied Probability and Random Processes (Second Edition), 2014

6.4.5 The Spare Parts Problem

From Figure 6.2 we can see that finding the sum of independent random variables is equivalent to finding the lifetime of a system that achieves continuous operation by permitting instantaneous replacement of a component with a spare part at the time of the component's failure. One interesting issue is to find the probability that the life of the system exceeds a given value. For the case where only one spare part is available, we are basically dealing with the sum of two random variables. For the case where we have n  −   1 spare parts, we are dealing with the sum of n random variables. For the case of n  =   2, we have that if the lifetime of the primary component is X and the lifetime of the spare component is Y, where X and Y are independent, then the lifetime of the component W and its PDF are given by

$\begin{array}{c}\mathit{W}=\mathit{X}+\mathit{Y}\\ {\mathit{f}}_{\mathit{W}}\left(\mathit{w}\right)={\displaystyle {\int }_0^{\mathit{w}}{\mathit{f}}_{\mathit{X}}\left(\mathit{w}-\mathit{y}\right){\mathit{f}}_{\mathit{Y}}\left(\mathit{y}\right)\mathit{dy}}\end{array}$

Thus, if we define the reliability function of the system by R _W (w), the probability that the lifetime of the system exceeds the value w ₀ is given by

$\mathit{P}\left[\mathit{W}>{\mathit{w}}_0\right]={\displaystyle {\int }_{{\mathit{w}}_0}^{\mathit{\infty }}{\mathit{f}}_{\mathit{W}}\left(\mathit{w}\right)}\mathit{dw}=1-{\mathit{F}}_{\mathit{W}}\left({\mathit{w}}_0\right)={\mathit{R}}_{\mathit{W}}\left({\mathit{w}}_0\right)$

If it is desired that P[W  > w ₀]   ≥ φ, where 0   ≤ φ  ≤   1, then we could be required to find the parameters of X and Y that are necessary to achieve this goal. For example, if X and Y are independent and identically distributed exponential random variables with mean 1/ λ, we can find the smallest mean value of the random variables that can achieve this goal.

For the case of n  −   1 spare parts, the lifetime of the system U is given by

$\mathit{U}={\mathit{X}}_1+{\mathit{X}}_2+\cdots +{\mathit{X}}_{\mathit{n}}$

where X _k is the life time of the kth component. If we assume that the X _k are independent, the PDF of U is given by the following n-fold convolution integral:

$\begin{array}{c}{\mathit{f}}_{\mathit{U}}\left(\mathit{u}\right)={\displaystyle {\int }_0^{\mathit{u}}{\displaystyle {\int }_0^{{\mathit{x}}_1}\cdots {\displaystyle {\int }_0^{{\mathit{x}}_{\mathit{n}-1}}{\mathit{f}}_{{\mathit{X}}_1}\left(\mathit{u}-{\mathit{x}}_1\right){\mathit{f}}_{{\mathit{X}}_2}\left({\mathit{x}}_1-{\mathit{x}}_2\right)\cdots {\mathit{f}}_{{\mathit{X}}_{\mathit{n}-1}}\left({\mathit{x}}_{\mathit{n}-1}-{\mathit{x}}_{\mathit{n}}\right)}}}\\ \times {\mathit{f}}_{{\mathit{X}}_{\mathit{n}}}\left({\mathit{x}}_{\mathit{n}}\right)\mathit{d}{\mathit{x}}_{\mathit{n}}\mathit{d}{\mathit{x}}_{\mathit{n}-1}\cdots \mathit{d}{\mathit{x}}_1\end{array}$

For the special case when the X _k are identically distributed exponential random variables with mean 1/λ, U becomes an nth order Erlang random variable with the PDF, CDF, and reliability function given by

(6.12) $\begin{array}{c}{\mathit{f}}_{\mathit{U}}\left(\mathit{u}\right)=\left\{\begin{array}{l}\frac{{\mathit{\lambda }}^{\mathit{n}}{\mathit{u}}^{\mathit{n}-1}{\mathit{e}}^{-\mathit{\lambda u}}}{\left(\mathit{n}-1\right)!}\mathit{n}=1,2,\dots ;\mathit{u}\ge 0\\ 0\mathit{u}<0\end{array}\right.\\ {\mathit{F}}_{\mathit{U}}\left(\mathit{u}\right)=1-{\displaystyle \sum _{\mathit{k}=0}^{\mathit{n}-1}\frac{{\left(\mathit{\lambda u}\right)}^{\mathit{k}}{\mathit{e}}^{-\mathit{\lambda u}}}{\mathit{k}!}}\\ {\mathit{R}}_{\mathit{U}}\left(\mathit{u}\right)=1-{\mathit{F}}_{\mathit{U}}(\mathit{u})={\displaystyle \sum _{\mathit{k}=0}^{\mathit{n}-1}\frac{{\left(\mathit{\lambda u}\right)}^{\mathit{k}}{\mathit{e}}^{-\mathit{\lambda u}}}{\mathit{k}!}}\end{array}$

Example 6.12

A system consists of one component whose lifetime is exponentially distributed with a mean of 50 hours. When the component fails, it is immediately replaced by a spare component whose lifetime is independent and identically distributed as that of the original component without the system suffering a downtime.

a.

What is the probability that the system has not failed after 100 hours of operation?

b.

If the mean lifetime of the component and its spare is increased by 10%, how does that affect the probability that the system exceeds a lifetime of 100 hours?

Solution:

Let X be a random variable that denotes the lifetime of the component and let U be the random variable that denotes the lifetime of the system. Then, U is an Erlang-2 random variable whose PDF, CDF, and reliability function are given by

$\begin{array}{c}{\mathit{f}}_{\mathit{U}}\left(\mathit{u}\right)=\left\{\begin{array}{c}{\mathit{\lambda }}^2\mathit{u}{\mathit{e}}^{-\mathit{\lambda u}}\mathit{u}\ge 0\\ 0\mathit{u}<0\end{array}\right.\\ {\mathit{F}}_{\mathit{U}}\left(\mathit{u}\right)=1-{\displaystyle \sum _{\mathit{k}=0}^1\frac{{\left(\mathit{\lambda u}\right)}^{\mathit{k}}{\mathit{e}}^{-\mathit{\lambda u}}}{\mathit{k}!}}=1-{\mathit{e}}^{-\mathit{\lambda u}}\{1+\mathit{\lambda u}\}\\ {\mathit{R}}_{\mathit{U}}\left(\mathit{u}\right)=1-{\mathit{F}}_{\mathit{U}}(\mathit{u})={\displaystyle \sum _{\mathit{k}=0}^1\frac{{\left(\mathit{\lambda u}\right)}^{\mathit{k}}{\mathit{e}}^{-\mathit{\lambda u}}}{\mathit{k}!}}={\mathit{e}}^{-\mathit{\lambda u}}\{1+\mathit{\lambda u}\}\end{array}$

a.

Since 1/λ  =   50, we have that

$\mathit{P}\left[\mathit{U}>100\right]={\mathit{R}}_{\mathit{U}}\left(100\right)={\mathit{e}}^{-100/50}\left\{1+\frac{100}{50}\right\}=3{\mathit{e}}^{-2}=0.4060$

b.

When we increase the mean lifetime of the component by 10%, we obtain 1/λ  =   50(1   +   0.1)   =   55. Thus, with the new λu  =   100/55, the corresponding value of R _U (100) is

${\mathit{R}}_{\mathit{U}}\left(100\right)={\mathit{e}}^{-100/55}\left\{1+\frac{100}{55}\right\}=2.8182{\mathit{e}}^{-1.82}=0.4574$

That is, the probability that the system lifetime exceeds 100 hours increases by approximately 13%.

Example 6.13

The time to failure of a component of a system is exponentially distributed with a mean of 100 hours. If the component fails, it is immediately replaced by an identical spare component whose time to failure is independent of that of the previous one and the system experiences no downtime in the process of component replacement. What is the smallest number of spare parts that must be used to guarantee continuous operation of the system for at least 300 hours with a probability of at least 0.95?

Solution:

Let X be the random variable that denotes the lifetime of a component, and let the number of spare parts be n  −   1. Let U be the random variable that denotes the lifetime of the system. Then U  = X ₁  + X ₂  +   ⋯   + X _n , which is an Erlang-n random variable whose reliability function is given by

${\mathit{R}}_{\mathit{U}}\left(\mathit{u}\right)=1-{\mathit{F}}_{\mathit{U}}\left(\mathit{u}\right)={\displaystyle \sum _{\mathit{k}=0}^{\mathit{n}-1}\frac{{\left(\mathit{\lambda u}\right)}^{\mathit{k}}{\mathit{e}}^{-\mathit{\lambda u}}}{\mathit{k}!}}={\mathit{e}}^{-\mathit{\lambda u}}\left\{1+\mathit{\lambda u}+\frac{{\left(\mathit{\lambda u}\right)}^2}{2!}+\cdots +\frac{{\left(\mathit{\lambda u}\right)}^{\mathit{n}-1}}{\left(\mathit{n}-1\right)!}\right\}\ge 0.95$

Since 1/λ  =   100, we have that

${\mathit{R}}_{\mathit{U}}\left(300\right)={\mathit{e}}^{-3}\left\{1+3+\frac{{\left(3\right)}^2}{2!}+\frac{{\left(3\right)}^3}{3!}+\frac{{\left(3\right)}^4}{4!}+\cdots +\frac{{\left(3\right)}^{\mathit{n}-1}}{\left(\mathit{n}-1\right)!}\right\}\ge 0.95$

The following table shows the values of R _U (300) for different values of n:

n  -   1	R U (300)
0	0.0498
1	0.1991
2	0.4232
3	0.6472
4	0.8153
5	0.9161
6	0.9665

Thus, we see that with n  −   1   =   5 we cannot provide the required probability of operation, while with n  −   1   =   6 we can. This means that we need 6 spare components to achieve the goal.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128008522000067

Simulation Techniques

Scott L. Miller , Donald Childers , in Probability and Random Processes (Second Edition), 2012

12.1.3 Generation of Random Numbers from a Specified Distribution

Quite often, we are interested in generating random variables that obey some distribution other than a uniform distribution. In this case, it is generally a fairly simple task to transform a uniform random number generator into one that follows some other distribution. Consider forming a monotonic increasing transformation g() on a random variable X to form a new random variable Y. From the results of Chapter 4, the PDFs of the random variables involved are related by

Given an arbitrary PDF, f_X (x), the transformation Y = g(X) will produce a uniform random variable Y if dg/dx = f_X (x) or equivalently g(x) = F_X (x). Viewing this result in reverse, if X is uniformly distributed over (0, 1) and we want to create a new random variable, Y with a specified distribution, F_Y (y), the transformation Y = Fy ⁻¹ (X) will do the job.

Example 12.3

Suppose we want to transform a uniform random variable into an exponential random variable with a PDF of the form

$f_Y\left(y\right)=a\exp \left(-ay\right)u\left(y\right).$

The corresponding CDF is

$f_Y\left(y\right)=\left[1-\exp \left(-ay\right)\right]u\left(y\right).$

Therefore, to transform a uniform random variable into an exponential random variable, we can use the transformation

$Y=F_Y^{-1}\left(X\right)=-\frac{\ln \left(1-X\right)}{a}.$

Note that if X is uniformly distributed over (0, 1), then 1 − X will be uniformly distributed as well so that the slightly simpler transformation

$Y=-\frac{\ln \left(X\right)}{a}$

will also work.

This approach for generation of random variables works well provided that the CDF of the desired distribution is invertible. One notable exception where this approach will be difficult is the Gaussian random variable. Suppose, for example, we wanted to transform a uniform random variable, X, into a standard normal random variable, Y. The CDF in this case is the complement of a Q-function, F_Y (y) = 1 − Q(y). The inverse of this function would then provide the appropriate transformation, y = Q ⁻¹ (1 − x), or as with the previous example, we could simplify this to y = Q^{− 1} (x). The problem here lies with the inverse Q-function which can not be expressed in a closed form. One could devise efficient numerical routines to compute the inverse Q-function, but fortunately there is an easier approach.

An efficient method to generate Gaussian random variables from uniform random variables is based on the following 2 × 2 transformation. Let X ₁ and X ₂ be two independent uniform random variables (over the interval (0, 1)). Then if two new random variables, Y₁ and Y₂ are created according to

12.5a $Y_1=\sqrt{-2\text{ln}\left(X_1\right)}\cos \left(2\pi X_2\right),$

12.5b $Y_2=\sqrt{-2\text{ln}\left(X_1\right)}\sin \left(2\pi X_2\right),$

then Y ₁ and Y ₂ will be independent standard normal random variables (see Example 5.24). This famous result is known as the Box−Muller transformation and is commonly used to generate Gaussian random variables. If a pair of Gaussian random variables is not needed, one of the two can be discarded. This method is particularly convenient for generating complex Gaussian random variables since it naturally generates pairs of independent Gaussian random variables. Note that if Gaussian random variables are needed with different means or variances, this can easily be accomplished through an appropriate linear transformation. That is, if Y ∼ N(0, 1), then Z = σY + μ will produce Z ∼ N(μ, σ ²).

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123869814500151

Simulation

Sheldon M. Ross , in Introduction to Probability Models (Tenth Edition), 2010

Method 1. Sampling a Poisson Process

To simulate the first T time units of a nonhomogeneous Poisson process with intensity function λ(t), let λ be such that

$\lambda \left(t\right)\le \lambda \text{for}\text{all}t\le T$

Now, as shown in Chapter 5, such a nonhomogeneous Poisson process can be generated by a random selection of the event times of a Poisson process having rate λ. That is, if an event of a Poisson process with rate λ that occurs at time t is counted (independently of what has transpired previously) with probability λ(t)/λ then the process of counted events is a nonhomogeneous Poisson process with intensity function λ(t),0 ≤ t ≤ T. Hence, by simulating a Poisson process and then randomly counting its events, we can generate the desired nonhomogeneous Poisson process. We thus have the following procedure:

Generate independent random variables X ₁, U ₁, X ₂, U ₂, … where the X_i are exponential with rate λ and the U_i are random numbers, stopping at

$N=min\left\{n:\sum _{i=1}^n{\text{x}}_i>T\right\}$

Now let, for j = 1,…,N − 1,

$I_j=\{\begin{array}{l}1,\text{if}{U}_j\le \lambda \left(\sum {\text{x}}_i\right)/\lambda \\ 0,\text{otherwise}\end{array}$

and set

$J=\{j:I_j=1\}$

Thus, the counting process having events at the set of times $\left\{{\sum }_{i=1}^j{\text{x}}_i:j\in J\right\}$ constitutes the desired process.

The foregoing procedure, referred to as the thinning algorithm (because it "thins" the homogeneous Poisson points) will clearly be most efficient, in the sense of having the fewest number of rejected event times, when λ(t) is near λ throughout the interval. Thus, an obvious improvement is to break up the interval into subintervals and then use the procedure over each subinterval. That is, determine appropriate values k, 0 < t ₁ < t ₂ < … < t_k < T, λ₁, … λ _k+l, such that

(11.9) $\lambda \left(s\right)\le {\lambda }_i\text{when}{t}_{i-1}\le s<t_i,i=\mathrm{1,\dots },k+1\left(\text{where}{t}_0,t_{k+1}=T\right)$

Now simulate the nonhomogeneous Poisson process over the interval (t_i − 1, t_i ) by generating exponential random variables with rate λ _i and accepting the generated event occurring at time s, s ∈ (t_i − 1, t_i ), with probability λ(s)/λ _i . Because of the memoryless property of the exponential and the fact that the rate of an exponential can be changed upon multiplication by a constant, it follows that there is no loss of efficiency in going from one subinterval to the next. In other words, if we are at t ∈ [t_i − 1, t_i ) and generate X, an exponential with rate λ _i , which is such that t + X > t_i then we can use λ _i [X − (t_i − t)]/λ _i+1 as the next exponential with rate λ _{i + 1}. Thus, we have the following algorithm for generating the first t time units of a nonhomogeneous Poisson process with intensity function λ(s) when the relations (11.9) are satisfied. In the algorithm, t will represent the present time and I the present interval (that is, I = i when t_i − 1 ≤ t < t_i ).

Step 1:

t = 0, I = 1.

Step 2:

Generate an exponential random variable X having rate λ _I .

Step 3:

If t λ X < t_i , reset t = t + X, generate a random number U, and accept the event time t if U ≤ λ (t)/λ _I . Return to step 2.

Step 4:

(Step reached if t + X ≥ t_I ). Stop if I = k + 1. Otherwise, reset X = (X = t_I + t)λ _I /λ _{I + 1}. Also reset t = t_I and I = I + 1, and go to step 3.

Suppose now that over some subinterval (t_i − 1, t_i ) it follows that λ _i > 0 where

(11.10) ${\lambda }_i\equiv \text{infimum}\{\lambda (s):t_{i-1}\le s<t_i\}$

In such a situation, we should not use the thinning algorithm directly but rather should first simulate a Poisson process with rate λi over the desired interval and then simulate a nonhomogeneous Poisson process with the intensity function $\lambda \left(s\right)=\lambda \left(s\right)-{\underset{\_}{\lambda }}_i$ when s ∈ (t_i − 1, t_i ). (The final exponential generated for the Poisson process, which carries one beyond the desired boundary, need not be wasted but can be suitably transformed so as to be reusable.) The superposition (or, merging) of the two processes yields the desired process over the interval. The reason for doing it this way is that it saves the need to generate uniform random variables for a Poisson distributed number, with mean ${\underset{\_}{\lambda }}_i\left(t_i-t_{i-1}\right)$ of the event times. For instance, consider the case where

$\lambda \left(s\right)=10+s,0<s<1$

Using the thinning method with λ = 11 would generate an expected number of 11 events each of which would require a random number to determine whether or not to accept it. On the other hand, to generate a Poisson process with rate 10 and then merge it with a generated nonhomogeneous Poisson process with rate λ(s) = s, 0 < s < 1, would yield an equally distributed number of event times but with the expected number needing to be checked to determine acceptance being equal to 1.

Another way to make the simulation of nonhomogeneous Poisson processes more efficient is to make use of superpositions. For instance, consider the process where

$\lambda \left(t\right)=\{\begin{array}{l}exp\left\{t^2\right\},0<t<1.5\\ exp\left\{2.25\right\},1.5<t<2.5\\ exp\left\{{(4-t)}^2\right\},2.5<t<4\end{array}$

A plot of this intensity function is given in Figure 11.3. One way of simulating this process up to time 4 is to first generate a Poisson process with rate 1 over this interval; then generate a Poisson process with rate e − 1 over this interval, accept all events in (1, 3), and only accept an event at time t that is not contained in (1, 3) with probability [λ(t) − 1]/(e − 1); then generate a Poisson process with rate e ^2.25 − e over the interval (1, 3), accepting all event times between 1.5 and 2.5 and any event time t outside this interval with probability [λ(t) − e]/(e ^2.25 − e). The superposition of these processes is the desired nonhomogeneous Poisson process. In other words, what we have done is to break up λ(t) into the following nonnegative parts:

Figure 11.3.

$\lambda \left(t\right)={\lambda }_1\left(t\right)+{\lambda }_2\left(t\right)+{\lambda }_3\left(t\right),0<t<4$

where

$\begin{array}{l}{\lambda }_1\left(t\right)\equiv 1,\\ {\lambda }_2\left(t\right)=\{\begin{array}{ll}\lambda \left(t\right)-1,& 0<t<1\\ e-1,& 1<t<3\\ \lambda \left(t\right)-1,& 3<t<4\end{array}\\ {\lambda }_3\left(t\right)=\{\begin{array}{ll}\lambda \left(t\right)-e,& 1<t<1.5\\ e^{2.25}-e,& 1.5<t<2.5\\ \lambda \left(t\right)-e,& 2.5<t<3\\ 0,& 3<t<4\end{array}\end{array}$

and where the thinning algorithm (with a single interval in each case) was used to simulate the constituent nonhomogeneous processes.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123756862000017

Sampling Distributions

Kandethody M. Ramachandran , Chris P. Tsokos , in Mathematical Statistics with Applications in R (Second Edition), 2015

4A A Method to Obtain Random Samples from Different Distributions

Most of the statistical software packages contain a random number generator that produces approximations to random numbers from the uniform distribution U [0, 1]. To simulate the observation of any other continuous random variables, we can start with uniform random numbers and associate these to the distribution we want to simulate. For example, suppose we wish to simulate an observation from the exponential distribution

$\mathit{F}\left(\mathit{x}\right)=1-{\mathrm{e}}^{-0.5\mathit{x}},0<\mathit{x}<\infty .$

First produce the value of y from the uniform distribution. Then solve for x from the equation

$\mathit{y}=\mathit{F}\left(\mathit{x}\right)=1-{\mathrm{e}}^{-0.5\mathit{x}}.$

So x  =   [−   ln (1   − y )]/0.5 is the corresponding value of the exponential random variable. For instance, if y  =   0.67, then x  =   [−   ln (1   − y)]/0.5   =   2.2173. If we wish to simulate a sample from the distribution F from the different values of y obtained from the uniform distribution, the procedure is repeated for each new observation x.

(a)

Simulate 10 observations of a random variable having exponential distribution with mean and standard deviation both equal to 2.

(b)

Select 1500 random samples of size n  =   10 measurements from a population with an exponential distribution with mean and standard deviation both equal to 2. Calculate sample mean for each of these 1500 samples and draw a relative frequency histogram. Based on Theorems 4.1.1 and 4.4.1, what can you conclude?

It should be noted that in general, if Y  ~ U (0, 1) random variable, then we can show that $\mathit{X}=-\frac{1\mathrm{n}\mathit{Y}}{\mathit{\lambda }}$ inwill give an exponential random variable with parameter λ. Uniform random variables could also be used to generate random variables from other distributions. For example, let U_i s be iid U[0, 1] random variables. Then,

$\mathit{X}=-2{\displaystyle \sum _{\mathit{i}=1}^{\mathit{v}}ln\left({\mathit{U}}_{\mathit{i}}\right)\sim {\mathrm{\chi }}_{2\mathit{v}}^2,}$

and

$\mathit{Y}=-\mathit{\beta }{\displaystyle \sum _{\mathit{i}=1}^{\mathit{\alpha }}ln\left({\mathit{U}}_{\mathit{i}}\right)\sim \mathrm{Gamma}\left(\mathit{\alpha },\mathit{\beta }\right).}$

Of course, these transformations are useful only when v and α are integers. More efficient methods based on Monte Carlo simulations, such as MCMC methods, are discussed in Chapter 13.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780124171138000047

morenoantionne.blogspot.com

Source: https://www.sciencedirect.com/topics/mathematics/exponential-random-variable

Share this post