Two versions of the exponential cumulant generating function

In this note I want to record an interesting issue I have noticed with regard to how the form of the large deviation entropy function can `flip’ depending on the way the underlying cumulant generating function is formulated.

The cumulant generating function of a random variable is the natural log of its moment generating function. In the case of an exponential random variable with mean $\mu$ and probability density function

$f(x) = \frac{1}{\mu} e^{-x/\mu}$

the moment generating function is the same as the Laplace transform of the random variable. Thus, one way to compute the cumulant generating function of an exponential random variable is as follows:

$\lambda(k) = \ln \langle e^{kx} \rangle$

$= \ln \bigg(\int_0^\infty e^{kx} \frac{1}{\mu} e^{-x/\mu} dx \bigg)$

$= \ln \bigg(\frac{1}{\mu} \int_0^{\infty} e^{-(1-\mu k)x/\mu} dx \bigg)$

$= \ln \bigg(\frac{1}{\mu} \bigg[ -\frac{\mu}{1-\mu k}e^{-(1-\mu k)x/\mu} \bigg]_0^{\infty} \bigg)$

$= \ln \bigg( \frac{1}{1-\mu k}\bigg)$

$= -\ln(1 - \mu k) \qquad \qquad \qquad (1)$

Note that it is clear from (1) that there is a singularity at $k = \frac{1}{\mu}$ and that the natural logarithm is defined only if the Laplace transform parameter $k$ is restricted to the region $k < \frac{1}{\mu}$ . Since $\lambda(k)$ is everywhere differentiable in this region, it follows from the Gärtner-Ellis Theorem of large deviation theory that the probability density function of values of the random variable in the tails of the exponential distribution can be approximated using the large deviation principle, with entropy function

$I(s) = \sup_k (ks - \lambda(k)) \qquad \qquad \qquad (2)$

All well and good. However, what I want to point out here is that there is a mathematically equivalent way to formulate the cumulant generating function, as follows:

$\lambda(k) = -\ln \langle e^{-kx} \rangle$

$= -\ln \bigg(\int_0^\infty e^{-kx} \frac{1}{\mu} e^{-x/\mu} dx \bigg)$

$= -\ln \bigg(\frac{1}{\mu} \int_0^{\infty} e^{-(1+\mu k)x/\mu} dx \bigg)$

$= -\ln \bigg( \frac{1}{1+\mu k}\bigg)$

$= \ln(1+\mu k) \qquad \qquad \qquad (3)$

It is less obvious from (3) that there are singularities, etc., but they are still there. It’s just that everything is negated when compared to the first formulation. There is now a singularity at $k = -\frac{1}{\mu}$ , and the logarithm in (3) is now only defined for values of $k$ in the region $k > -\frac{1}{\mu}$ . What interested me is that it is not only the admissible region of values of $k$ that gets reversed, but also the form of the entropy function in (2) above. Since in the second formulation of the cumulant generating function we have negated both the outer function and the parameter $k$ , the signs of the terms on the right-hand side of (2) have also become reversed, so the entropy function as a whole now gets flipped around. It should therefore read as follows in this case:

$I(s) = \sup_k (\lambda(k) - ks)$

Two versions of the exponential cumulant generating function

Published by Dr Christian P. H. Salas

Leave a comment Cancel reply