Moment Generating Functions

This section develops and applies some of the properties of the moment-generating function. It turns out, despite its unlikely appearance, to be a very useful tool that can dramatically simplify certain calculations.

The moment-generating function (mgf) of a random variable $X$ is 

$M(t) =E(e^{t X })$

 if the expectation is defined. In the discrete case,

$M(t) =\sum_x e^{tx} p(x)$

and in the continuous case,

$M(t) =\int_{-\infty}^{\infty}e^{tx} f (x) dx$

The expectation, and hence the moment-generating function, may or may not exist for any particular value of $t$. In the continuous case, the existence of the expectation depends on how rapidly the tails of the density decrease; for example, because the tails of the Cauchy density die down at the rate $x^{−2}$, the expectation does not exist for any $t$ and the moment-generating function is undefined. The tails of the normal density die down at the rate $e^{−x^2}$ , so the integral converges for all $t$.


The $r $th moment of a random variable is $E(X^r )$ if the expectation exists. We have already encountered the first and second moments  that is, $E(X)$ and $E(X^2)$. Central moments rather than ordinary moments are often used: The $r$ th central moment is $E{[X − E(X)]^r }$. The variance is the second central moment and is a measure of dispersion about the mean. The third central moment, called the skewness, is used as a measure of the asymmetry of a density or a frequency function about its mean; if a density is symmetric about its mean, the skewness is zero. As its name implies, the moment-generating function has something to do with moments. To see this, consider the continuous case:

$M(t)=\int_{-\infty}^{\infty} e^{tx} f(x) dx$

The derivative of $M'(t)$ is

$M'(t)=\frac{\mathrm{d} }{\mathrm{d} t}\int_{-\infty}^{\infty} e^{tx} f(x) dx$

It can be shown that differentiation and integration can be interchanged, so that

$M'(t)=\int_{-\infty}^{\infty} x e^{tx} f(x) dx$

and $M'(0)$

$M'(0)=\int_{-\infty}^{\infty} x  f(x) dx=E(X)$

Differentiating $r$ times, we find

M^{(r )}(0) = E(X^r )$

It can further be argued that if the moment-generating function exists in an interval containing zero, then so do all the moments. We thus have the following property.


To find the moments of a random variable from the definition of expectation, we must sum a series or carry out an integration. The utility of Property B is that, if the mgf can be found, the process of integration or summation, which may be difficult, can be replaced by the process of differentiation, which is mechanical. We now illustrate these concepts using some familiar distributions




Comments

Post a Comment

Popular posts from this blog

Foundations of Machine Learning CST 312 KTU CS Elective Notes

  Introduction About Me Syllabus What is Machine Learning ( video) Learn the seven steps in Machine Learning ( video) Overview Of Machine Learning Linear Algebra in Machine Learning Module-1 Linear Algebra 1.Geometry of Linear Equations (video) 2.Elimination with Matrices (video) 3.Solving System of equations using Gauss Elimination Method 4.Row Echelon form and Reduced Row Echelon Form 5.Solving system of equations Python code 6. Practice problems Gauss Elimination ( contact) 7.Finding Inverse using Gauss Jordan Elimination (video) 8.Finding Inverse using Gauss Jordan Elimination-Python code 9.Vector spaces and sub spaces 10.Linear Independence 11.Linear Independence, Basis and Dimension (video) 12.Generating set basis and span 13.Rank of a Matrix 14.Linear Mapping and Matrix Representation of Linear Mapping 15.Basis and Change of basis 16. Transformation Matrix in new Basis 17.Image and Kernel 18.Example Problems ( contact) Module 2: Linear Algebra 1.Vector Norms 2.Inner Products 3.
Read more

Random Variables- Discrete and Continuous

Image
Discrete Random Variables A random variable is essentially a random number. As motivation for a definition, let us consider an example. A coin is thrown three times, and the sequence of heads and tails is observed; thus, $\Omega= \{hhh, hht, htt, hth, ttt, tth, thh, tht\}$ Examples of random variables defined on $\Omega$ are (1) the total number of heads, (2) the total number of tails, and (3) the number of heads minus the number of tails. Each of these is a real-valued function defined on $\Omega$; that is, each is a rule that assigns a real number to every point $\omega \in \Omega$ . Since the outcome in $\Omega$ is random, the corresponding number is random as well. In general, a random variable is a function from $\Omega$ to the real numbers. Because the outcome of the experiment with sample space $\Omega$ is random, the number produced by the function is random as well. It is conventional to denote random variables by italic uppercase letters from the end of the alphabet. Fo
Read more

Joint Distributions

Image
Joint Distributions concerned with joint probability structure of two or more random variables defined on the same sample space.Joint distributions arise naturally in many applications. • The joint probability distribution of the x, y, and z components of wind velocity can be experimentally measured in studies of atmospheric turbulence. • The joint distribution of the values of various physiological variables in a population of patients is often of interest in medical studies. • A model for the joint distribution of age and length in a population of fish can be used to estimate the age distribution from the length distribution. The age distribution is relevant to the setting of reasonable harvesting policies. The joint behavior of two random variables, $X$ and $Y$, is determined by the cumulative distribution function $F(x, y) = P(X ≤ x, Y ≤ y)$ regardless of whether $X$ and $Y$ are continuous or discrete. The cdf gives the probability that the point $(X, Y )$ belongs to a semi-inf
Read more