Sums of independent discrete random variables
Master sums of independent discrete random variables in Year 11 VCE Specialist Mathematics. When independent variables are added their means always add, and — because they are independent — their variances add too, so \(n\) independent copies of \(X\) give \(E(S)=n\mu\) and \(\mathrm{Var}(S)=n\sigma^2\). It sits in the Data analysis, probability and statistics area of study of the VCE Mathematics Study Design (VCAA), within the Simulation, sampling and sampling distributions topic of Unit 2.
You will learn to find the mean and variance of a sum, build the distribution of \(X_1+X_2\) by convolution, and see why doubling one value (\(2X\), variance \(4\sigma^2\)) differs from adding two copies (\(X_1+X_2\), variance \(2\sigma^2\)) — key ideas behind sampling distributions.
Theory
When independent random variables are added, their means always add and — provided the variables are independent — their variances add too. In Year 11 Specialist Mathematics, for \(n\) independent copies each distributed as \(X\) (mean \(\mu\), variance \(\sigma^2\)), \(E(X_1+\cdots+X_n)=n\mu\) and \(\text{Var}(X_1+\cdots+X_n)=n\sigma^2\). This explains the key contrast between doubling one value, \(2X\), and adding two independent copies, \(X_1+X_2\): both have mean \(2\mu\), but their spreads differ.
A discrete random variable \(X\) takes listed values with given probabilities. Its mean (expected value) is \(\mu=E(X)=\sum x\,P(X=x)\), and its variance is \(\sigma^2=\text{Var}(X)=E(X^2)-\mu^2\); the standard deviation is \(\sigma=\sqrt{\sigma^2}\), a measure of spread in the same units as \(X\).
Random variables are independent when the outcome of one has no effect on the others — for example, separate spins of a spinner or separate rolls of a die. We write \(X_1,X_2,\ldots,X_n\) for \(n\) independent copies of \(X\): each has the same distribution as \(X\), and they do not influence one another.
The sum \(S=X_1+X_2+\cdots+X_n\) is itself a random variable. Its mean is always the sum of the means, \(E(S)=n\mu\), whether or not the variables are independent. Its variance is \(\text{Var}(S)=n\sigma^2\) — but this addition of variances requires independence.
To find the whole distribution of a sum (not just its mean and variance), list every combination of outcomes that produces each possible total and add the probabilities. For independent variables the probability of a pair \((a,b)\) is \(P(a)\times P(b)\); collecting these by total is called a convolution.
Doubling a single observation, \(2X\), is not the same as adding two independent copies, \(X_1+X_2\). Scaling multiplies the variance by the square of the constant, so \(\text{Var}(2X)=4\sigma^2\), while the independent sum gives \(\text{Var}(X_1+X_2)=2\sigma^2\). Both share the mean \(2\mu\), yet \(2X\) is more spread out.
| \(s\) | \(2\) | \(3\) | \(4\) | \(5\) | \(6\) |
|---|---|---|---|---|---|
| pairs \((a,b)\) | \(1\) | \(2\) | \(3\) | \(2\) | \(1\) |
| \(P(X_1+X_2=s)\) | \(\dfrac{1}{9}\) | \(\dfrac{2}{9}\) | \(\dfrac{3}{9}\) | \(\dfrac{2}{9}\) | \(\dfrac{1}{9}\) |
| value \(v\) | \(2\) | \(3\) | \(4\) | \(5\) | \(6\) |
|---|---|---|---|---|---|
| \(P(2X=v)\) | \(\dfrac{1}{3}\) | \(0\) | \(\dfrac{1}{3}\) | \(0\) | \(\dfrac{1}{3}\) |
| \(P(X_1+X_2=v)\) | \(\dfrac{1}{9}\) | \(\dfrac{2}{9}\) | \(\dfrac{3}{9}\) | \(\dfrac{2}{9}\) | \(\dfrac{1}{9}\) |
For \(n\) independent copies \(X_1,\ldots,X_n\) of a variable \(X\) with mean \(\mu\) and variance \(\sigma^2\), the sum \(S=X_1+\cdots+X_n\) has:
The mean of a sum is always the sum of the means. Adding the variances, however, requires independence:
Contrast this with scaling a single observation by a constant \(k\), which multiplies the variance by \(k^2\):
To build the full distribution of an independent sum, add the products of the outcome probabilities over every pair that gives each total (a convolution):
Working with a sum of independent variables
- Find \(\mu\) and \(\sigma^2\) for one copy using \(\mu=\sum x\,P(X=x)\) and \(\sigma^2=E(X^2)-\mu^2\).
- Mean of the sum: multiply by the number of copies, \(E(S)=n\mu\) — this needs no independence.
- Variance of the sum: if the copies are independent, add the variances, \(\text{Var}(S)=n\sigma^2\); then \(\text{SD}(S)=\sqrt{n}\,\sigma\).
- For a full distribution (e.g. \(X_1+X_2\)), list every independent outcome pair, multiply the two probabilities, and collect them by total.
- To compare \(2X\) with \(X_1+X_2\): use \(\text{Var}(2X)=4\sigma^2\) but \(\text{Var}(X_1+X_2)=2\sigma^2\); the means match at \(2\mu\), the spreads do not.
For the spinner \(X\in\{1,2,3\}\) with \(\mu=2\) and \(\sigma^2=\tfrac{2}{3}\), the distribution of \(X_1+X_2\) is:
| \(s\) | \(2\) | \(3\) | \(4\) | \(5\) | \(6\) |
|---|---|---|---|---|---|
| pairs \((a,b)\) | \(1\) | \(2\) | \(3\) | \(2\) | \(1\) |
| \(P(X_1+X_2=s)\) | \(\dfrac{1}{9}\) | \(\dfrac{2}{9}\) | \(\dfrac{3}{9}\) | \(\dfrac{2}{9}\) | \(\dfrac{1}{9}\) |
Mean of one spin, \(\mu=E(X)\):
| \(\mu\) | \(=\) | \(\dfrac{1+2+3}{3}\) |
| \(=\) | \(\dfrac{6}{3}=2\) |
Variance of one spin, \(\sigma^2=E(X^2)-\mu^2\):
| \(E(X^2)\) | \(=\) | \(\dfrac{1+4+9}{3}=\dfrac{14}{3}\) |
| \(\sigma^2\) | \(=\) | \(\dfrac{14}{3}-2^2=\dfrac{14}{3}-\dfrac{12}{3}\) |
| \(=\) | \(\dfrac{2}{3}\) |
Sum of five independent copies: means add, and (independent) variances add:
| \(E(S)\) | \(=\) | \(5\mu=5\times 2=10\) |
| \(\text{Var}(S)\) | \(=\) | \(5\sigma^2=5\times\dfrac{2}{3}=\dfrac{10}{3}\) |
\(\mu=2\), \(\sigma^2=\dfrac{2}{3}\); for the total, \(E(S)=10\) and \(\text{Var}(S)=\dfrac{10}{3}\).
There are \(4\times 4=16\) equally likely ordered pairs, each with probability \(\tfrac{1}{16}\). List the pairs that total \(5\):
| \(\text{pairs giving } 5\) | \(:\) | \((1,4),(2,3),(3,2),(4,1)\) |
| \(P(T=5)\) | \(=\) | \(4\times\dfrac{1}{16}\) |
| \(=\) | \(\dfrac{4}{16}=\dfrac{1}{4}\) |
Counting the pairs for every total gives the full distribution of \(T\) — a symmetric triangular shape peaking at \(5\):
| \(t\) | \(2\) | \(3\) | \(4\) | \(5\) | \(6\) | \(7\) | \(8\) |
|---|---|---|---|---|---|---|---|
| \(P(T=t)\) | \(\dfrac{1}{16}\) | \(\dfrac{2}{16}\) | \(\dfrac{3}{16}\) | \(\dfrac{4}{16}\) | \(\dfrac{3}{16}\) | \(\dfrac{2}{16}\) | \(\dfrac{1}{16}\) |
\(P(T=5)=\dfrac{1}{4}\).
Doubling scales the variance by \(2^2\):
| \(\text{Var}(2X)\) | \(=\) | \(2^2\times\dfrac{5}{4}\) |
| \(=\) | \(4\times\dfrac{5}{4}=5\) |
Adding two independent copies adds the variances:
| \(\text{Var}(X_1+X_2)\) | \(=\) | \(2\times\dfrac{5}{4}\) |
| \(=\) | \(\dfrac{5}{2}=2.5\) |
Both have mean \(2\mu=5\); compare the spreads:
| \(\text{Var}(2X)=5\) | \(>\) | \(\text{Var}(X_1+X_2)=2.5\) |
\(\text{Var}(2X)=5\) and \(\text{Var}(X_1+X_2)=2.5\); with the same mean \(5\), \(2X\) is more spread out.
A total of \(1\) comes from the pairs \((0,1)\) and \((1,0)\):
| \(P(X_1+X_2=1)\) | \(=\) | \(2\times\dfrac{1}{2}\cdot\dfrac{1}{2}\) |
| \(=\) | \(\dfrac{1}{2}\) |
One toss has \(\mu=\tfrac12\) and \(\sigma^2=\tfrac14\); apply the sum rules:
| \(E(X_1+X_2)\) | \(=\) | \(2\mu=2\times\dfrac{1}{2}=1\) |
| \(\text{Var}(X_1+X_2)\) | \(=\) | \(2\sigma^2=2\times\dfrac{1}{4}=\dfrac{1}{2}\) |
\(P(X_1+X_2=1)=\dfrac{1}{2}\), \(E(X_1+X_2)=1\) and \(\text{Var}(X_1+X_2)=\dfrac{1}{2}\).
Mean, \(E(X)=\sum x\,P(X=x)\):
| \(E(X)\) | \(=\) | \(1(0.2)+2(0.5)+3(0.3)\) |
| \(=\) | \(0.2+1.0+0.9=2.1\) |
Variance, \(E(X^2)-\mu^2\):
| \(E(X^2)\) | \(=\) | \(1(0.2)+4(0.5)+9(0.3)=4.9\) |
| \(\text{Var}(X)\) | \(=\) | \(4.9-2.1^2=4.9-4.41\) |
| \(=\) | \(0.49\) |
Now the contrast — scale by \(2\) versus add two independent copies:
| \(\text{Var}(2X)\) | \(=\) | \(4\times 0.49=1.96\) |
| \(\text{Var}(X_1+X_2)\) | \(=\) | \(2\times 0.49=0.98\) |
\(E(X)=2.1\), \(\text{Var}(X)=0.49\); \(\text{Var}(2X)=1.96\) while \(\text{Var}(X_1+X_2)=0.98\), so \(2X\) is more spread out.
Common pitfalls
Frequently asked questions
What are the mean and variance of a sum of independent random variables?
For \(n\) independent copies of \(X\) with mean \(\mu\) and variance \(\sigma^2\), the sum \(S=X_1+\cdots+X_n\) has \(E(S)=n\mu\) and \(\text{Var}(S)=n\sigma^2\), so \(\text{SD}(S)=\sqrt{n}\,\sigma\). The means always add; the variances add because the variables are independent.
What is the difference between \(2X\) and \(X_1+X_2\)?
\(2X\) doubles a single observation, while \(X_1+X_2\) adds two independent copies. Both have mean \(2\mu\), but \(\text{Var}(2X)=4\sigma^2\) is twice \(\text{Var}(X_1+X_2)=2\sigma^2\), so \(2X\) is more spread out and the two have different distributions.
Why does \(2X\) have a larger variance than \(X_1+X_2\)?
Scaling by a constant \(k\) multiplies the variance by \(k^2\), so \(\text{Var}(2X)=2^2\sigma^2=4\sigma^2\). Adding two independent copies only adds the two variances, giving \(2\sigma^2\). Doubling one value amplifies its variation more than combining two separate values does.
How do I build the distribution of \(X_1+X_2\)?
List every pair of outcomes \((a,b)\), multiply their probabilities (they are independent, so \(P(a)P(b)\)), and add up the probabilities for each total \(s=a+b\). This is a convolution: \(P(X_1+X_2=s)=\sum_{a+b=s}P(X_1=a)P(X_2=b)\).
Do the variances always add when I add random variables?
Only when the variables are independent. The means always add, \(E(X_1+X_2)=E(X_1)+E(X_2)\), but \(\text{Var}(X_1+X_2)=\text{Var}(X_1)+\text{Var}(X_2)\) requires independence. If the variables influence one another, this rule does not hold.
Does adding a uniform variable to itself give a uniform result?
No. If \(X\) is uniform on \(1,2,3\), then \(X_1+X_2\) is triangular over \(2\) to \(6\): the middle total \(4\) has three contributing pairs (probability \(\tfrac{3}{9}\)) while the extremes \(2\) and \(6\) have only one each (\(\tfrac{1}{9}\)).