Probability
All knowledge degenerates into probability.

Top
Site Menu

Continuous Random Variable Distribution Families

Continuous distributions are used to describe random variables that have outcomes that fall anywhere within a given interval or intervals such as the lifetime of a certain insect, heights university students, or the amount of time kindergarteners spend watching TV in a given day. There are many families of continuous distributions; a few of the most common are discussed on this page.


The Uniform Distribution

The probability density function of a uniform random variable looks like a horizontal line segment over the support. This indicates that for any interval of a given length within the support, the probability that a value in that interval occurs is the same.

Keith likes to drink a mixture of Sprite and fruit juice each morning. He likes to change it up each day, so he has a computer output a randomly generated value between 0 and 1. He lets this be the proportion of the mixture that is Sprite. If the random variable $X$ describes the proportion of the mixture that is Sprite, what type of probability distribution does $X$ follow? What are its parameters?


This is a uniform distribution, because when the computer generates a random value between 0 and 1, every possible outcome is equally likely. The parameters of a uniform distribution are the lower bound ($a$) and the upper bound ($b$). In this case, $a=0$ and $b=1$. So, $X\sim Uniform[0,1]$.


$f(x) = \frac{1}{b-a}$, for $a \leq x \leq b$

$E(X) = \frac{b+a}{2}$, $Var(X)=\frac{(b-a)^2}{12}$


The Exponential Distribution

This distribution is related to the Poisson distribution. The Poisson distribution counted how many times an event would occur over a specified time or distance when the average number of occurrences for that time or distance is known. The Exponential distribution measures the amount of time or distance until the next occurrence of such an event, when we know the average time or distance it takes to see another occurrence.

The parameter, λ, is the rate parameter. It is calculated by finding how many times the event is observed (on average) in certain blocks of time or distance, such as "about 2 energy spikes per 30 seconds" ($\lambda = \frac{1}{15}$), or "an average of 4 breakages per 70 feet of cable" ($\lambda = \frac{2}{35}$). Whatever units the random variable (say, $X$) is measuring (e.g. seconds until the next spike, or feet of cable until the next breakage) will be found on the bottom of the rate parameter.

This distribution is the only "memoryless" continous distribution. This means that, if you have already waited 20 seconds without seeing the event in question, and you want to know the probability that it will take 10 more seconds for the event to be observed, it is the same as the probability that it would have taken 10 seconds from the beginning to see the event occur. It doesn't matter how long you have already waited without seeing the event, whether 0 seconds or 100; the probability that the event will happen in exactly 10 more seconds will always be the same.

On a certain rural highway, an average of 40 cars stop at Rex's gas station every 24 hours. That means, after a car has stopped at his gas station, he waits an average of 36 minutes until the next car stops there. As one car drives away, Rex's optimistic daughter wonders what the probability is that they could see another car stop there in just 10 minutes. Let $X$ denote the time (in minutes) that passes until the next car arrives at Rex's gas station. What type of probability distribution does $X$ follow? What are its parameters?


This is an exponential distribution, since we are measuring the amount of time or distance until the next occurrence of a relatively rare event. The exponential distribution has just one parameter: the rate (λ). $X$ is measured in minutes, so minutes will be on the bottom of the fraction that generates our rate parameter. At Rex's gas station, the average rate is 1 car per 36 minutes. Then $\lambda = \frac{1}{36}$. So, $X\sim exponential(\frac{1}{36})$.


$f(x)=\lambda e^{-\lambda x}$, for $0 \leq x$

$E(X) = \frac{1}{\lambda}$, $Var(X)=\frac{1}{\lambda^2}$

The Gamma Distribution

The Gamma Distribution is the maximum entropy probability distribution for a random variable $X$ that has a positive expected value of $\frac{\alpha}{\lambda}$. We often use this distribution when nothing is known about a distribution except that it describes a measurement of waiting time, distance, decay, power loss, mass accumulation, or another similar process. We use the "maximum entropy" distribution when we know very little about the distribution because it minimizes the amount of prior information built into the distribution, and because many physical systems tend to move toward maximal entropy configurations over time.

In a Gamma[α,λ] distribution, α is the shape parameter, and λ is the rate parameter. Whatever units the random variable (say, $X$) is measuring (e.g. years until the next millimeter of cancer cells accumulates, or loss of power in Joules within the next minute interval) will be found on the bottom of the rate parameter ($\lambda=\frac{1\hspace{3 pt}millimeter}{6\hspace{3 pt}years}$ or $\lambda=\frac{2\hspace{3 pt}minutes}{2235\hspace{3 pt}Joules}$).

The Exponential Distribution (see above) and the Chi-Square Distribution (see below) are special cases of the Gamma Distribution. An exponential(λ) distribution is a Gamma[1,λ] distribution, and a Chi-Square($\nu$) distribution is a Gamma[$\frac{\nu}{2},\frac{1}{2}$] distribution.

In Istanbul, the average rainfall in February is 4.2 inches. Across most months, the rain falls at a rate where it takes about 0.12 months to accumulate another inch of rainfall. A researcher wants to know what the probability of getting 4.5 inches of rain is next February. She believes that she can use a Gamma distribution to make a reasonable estimate of the probability. What would be the appropriate parameters for that Gamma distribution?


We're measuring how many inches we get in a month, so inches will be on the bottom of the fraction that creates the rate parameter. We were given that the rain falls at a rate of $\frac{0.12\hspace{3 pt}months}{1\hspace{3 pt}inch}$, so $\lambda = 0.12$.

A Gamma distribution has an expected value of $\frac{\alpha}{\lambda}$, where α is the shape parameter and λ is the rate parameter. In this case, we know the expected value is 4.2. Then $\frac{\alpha}{0.12} = 4.2$, so $\alpha = 0.504$. Then the researcher can estimate the probability by using a Gamma[$0.504,0.12$] distribution.


$f(x)=\frac{\lambda^{\alpha}}{\Gamma(\alpha)}x^{\alpha-1}e^{-\lambda x}$, for $0 \leq x$

Where, for a positive integer $n$,
$\Gamma(n) = (n-1)!$,
or, more generally, for all complex numbers $z$ with a positive real part,
$\Gamma(z) = \int_0^{\infty}x^{z-1}e^{-x}dx$

$E(X) = \frac{\alpha}{\lambda}$, $Var(X)=\frac{\alpha}{\lambda^2}$

The Chi-Square Distribution

The Chi-Square Distribution is dependent on the parameter $\nu$, which is referred to as the "degrees of freedom". It is the distribution of a sum of the squares of $\nu$ independent standard normal random variables. It is most commonly used in Chi-Square Goodness of Fit Tests -- which determine how well categorical data fit an expected set of proportions -- and in Chi-Square Tests of Independence, which determine the probability that data classified across two categorical variables could have come from a population where the two categorical variables are independent of one another.

The parameter $\nu$, or the degrees of freedom, is the minimum number of categories that need data given before the rest can be found mathematically, if we are given the total. This is why we call it the "degrees of freedom", because it denotes how many categories have the "freedom" to take on any value before the rest of the categories' values are fixed numbers.

For instance, if we separate 100 Starburst candies into piles of red, pink, yellow, and orange, and then count how many candies are in each pile, we only need to count three of the piles. Once we have counted 30 red candies, 22 pink candies, and 20 orange candies, we know that there must be 28 yellow candies in order for there to be 100 candies. Then, in this case, $\nu = 3$.

To find out if males or females are more likely to drop out of college before obtaining a Bachelor's degree, the president of a certain college looks at the records for the 24,669 undergraduates and records their gender and their year. In total, there are 13,579 males and 11,090 females. The totals for college year are 7,454 freshmen, 6,779 sophomores, 5,684 juniors, and 4,752 seniors. The president plans on using a Chi-Square Test of Independence to determine if gender and class year are independent on each other. What would be the parameter(s) of this distribution?


We'll be able to use the Chi-Square Distribution once we know how many degrees of freedom we have. In this case, once we know how many freshman males there are, we'll know how many freshman females there are (since we know the total number of freshmen). Similarly, once we know how many sophomore males there are, we'll know how many sophomore females there are. The same with junior males.

Once we know how many male freshmen, male sophomores, and male juniors there are, we'll know how many male seniors there are (and subsequently how many female seniors there are), because we know the total number of males. Then, we only need to know how many male freshmen, male sophomores, and male juniors there are to know how many students fit in every category.

So, out of the 8 categories a student could fall under, we only need to know 3 of them before we can just use mathematics and the totals to fill in the rest. In other words, just three categories are free to have any number of students in them...after those three have a number of students in them, the other five categories are fixed values. Thus, there are 3 degrees of freedom. So, the president should use a $Chi\text{-}Square(3)$ distribution for her test.


$f(x)= \sum^{\nu}_{i=1} Z^2_i$, for $0 \leq x$,
where $Z_1, Z_2, \ldots Z_{\nu}$ are independent, standard normal random variables.

$E(X) = \nu$, $Var(X)=\nu$

The t Distribution

"Student's t-distribution" is a family of probability distributions that estimate the mean of a normally distributed population with a small-sized sample, where the population standard deviation is unknown.

The t-distribution is dependent on the parameter $\nu$, which is referred to as the "degrees of freedom". $\nu$ is the minimum number of subjects in the sample that we need to get measurements for before the rest can be found mathematically, if we already know the entire sample's average or sum. This is why we call it the "degrees of freedom", because it denotes how many subjects have the "freedom" to take on any value before the rest of the subjects' values are fixed numbers. Here, for any sample of size $n$ (meaning there are $n$ subjects in the sample), $\nu = n-1$.

For example, suppose we obtain a random sample of 5 values from a normally distributed population, and the sample average is 20. Then, as we start recording the individual values, we get 23 for the first subject, 15 for the second, 18 for the third, 25 for the fourth...and we don't have to measure the fifth subject. The fifth subject's value is fixed, and can only be one value. Since we know the average is 20, we can find the fifth subject's value (denoted by $x$) with the following equation: $20=\frac{23+15+18+25+x}{5}$. Solving for $x$, we get $x=19$. Only the first 4 (or $n-1$) values were free variables. After four subjects have been measured, the fifth subject's value is automatically known.

On average, female rats weigh about 400 grams, with a standard deviation of 30 grams. The weights of rats also tend to be normally distributed. A young lab assistant has been doing tests on a small group of 7 female rats to see if ingesting a certain chemical increases the amount of muscle mass the rats develop. After two weeks of testing, he wants to know how the weights of these rats compare with the average rat. He weighs the 7 rats, and wants to use this information to estimate the true average weight of all female rats after taking the chemical for two weeks. He also thinks that the standard deviation of the rats' weights may differ from the original distribution. He needs to find a probability distribution to help him make that estimate. What distribution should he use, and with what parameters?




In this case, the lab assistant should use a t-distribution with $\nu=6$. He is trying to estimate the mean of a population that is most likely normally distributed, with a very small sample size ($n=7$), and where the standard deviation of the population is unknown. These are the criteria for using a t-distribution.


$f(x) = \frac{\Gamma(\frac{\nu+1}{2})}{\sqrt{\nu\pi}\Gamma(\frac{\nu}{2})} (1+\frac{t^2}{\nu})^{-\frac{\nu+1}{2}}$, for $-\infty < x < \infty$

$E(X) = 0$
$Var(X) = \frac{\nu}{\nu-2}$ for $ \nu > 2, \infty$ for $1 < \nu \leq 2$, otherwise undefined.