Samples, Surveys and Reports
Theory
A survey is only as good as who was asked and how they were asked. This Year 9 maths page shows how to classify data, choose a census or a sample, name and judge sampling methods, spot bias in samples and questions, use a sample to estimate a population count, mean or median, and read survey reports and graphs in the media with a critical eye. Curriculum: VC2M9ST01, VC2M9ST02 / AC9M9ST01, AC9M9ST02.
A variable is anything recorded about each member of a group. A numerical variable has number values: discrete if counted (number of texts sent) and continuous if measured (time, mass). A categorical variable has labels: nominal if the categories have no order (eye colour) and ordinal if they can be ranked (very, somewhat, not at all). Postcodes and jersey numbers are labels, so they are categorical.
The population is the whole group being studied. A census collects data from every member. A sample collects data from part of the population, chosen to represent all of it. The sample size is the number in the sample.
A representative sample is like the whole population. A biased sample leans one way, usually because it leaves out a whole sub-group, such as asking about gym membership outside a gym. A self-selected sample, such as an online poll, is made up of people who chose to answer, so people with strong views are over-represented. A huge number of replies does not fix bias.
Different samples from the same population give slightly different results, even with the same method. Larger random samples vary less, so their estimates are more reliable.
A report of a survey should say how the data were collected: who was asked, how they were chosen, how many replied, and who paid for it. A graph can be drawn to support a point of view, for example with a vertical axis that does not start at zero or with pictures whose area grows faster than the numbers.
| Sampling method | How the sample is chosen | Watch out for |
|---|---|---|
| simple random | every member has the same chance (names drawn, random numbers) | needs a full list of the population |
| systematic | every \(k\)th member of a list, from a random start | a pattern in the list |
| stratified | a random sample from each group, in proportion to its size | group sizes must be known |
| self-selected | people choose to take part (phone-in, online poll) | usually biased: only keen people reply |
| Fault in a question | Why it spoils the data |
|---|---|
| leading wording, such as “Don’t you agree…” | it pushes the reader towards one answer |
| a vague word, such as “often” | each person reads it differently |
| overlapping options, such as \(1\)–\(10\) and \(10\)–\(20\) | one answer fits in two boxes |
| missing options | some honest answers cannot be given |
How to judge a survey or report
- Name the population the conclusion is about.
- Describe the sample: who was asked, how they were chosen (which method), and how many.
- Ask who was left out. If they differ on the question, the sample is biased. Say which way the result leans: too high or too low.
- Check the questions: neutral, precise, and a box for every honest answer.
- Check the size: a tiny sample gives an unreliable estimate, but a bigger biased sample is still biased.
- Estimate for the population using the sample proportion, mean or median, and call it an estimate.
(i) What is the population?
(ii) What percentage of the households were surveyed?
(iii) Estimate the number of households in the shire with solar panels.
(i) The population is all \(12\,500\) households in the shire.
| \(\text{(ii)} \ \dfrac{250}{12\,500} \times 100\%\) | \(=\) | \(\mathbf{2\%}\) |
| \(\text{(iii)} \ \dfrac{60}{250} \times 12\,500\) | \(=\) | \(\mathbf{3000}\) |
This is an estimate: another random sample of \(250\) would give a slightly different answer.
| population | \(:\) | every student at the school |
| sample | \(:\) | esports club members only |
Club members game far more than most students, and students who never game cannot be chosen. The sample is biased, and the estimate will be too high.
A better method is a stratified sample: choose students at random from each year group, in proportion to the size of each year.
(i) Find the sample mean and median.
(ii) Which better estimates a typical household’s bill?
(iii) Estimate the total weekly bill for a town of \(2000\) households.
| \(\text{mean}\) | \(=\) | \(\dfrac{583}{9} \approx \$64.78\) |
| \(\text{median}\) | \(=\) | \(\text{5th score} = \$52\) |
(ii) Eight of the nine bills are below the mean, because the \(\$186\) bill drags it up. The median, \(\$52\), is the better estimate of a typical bill.
(iii) A total is the mean times the count, so use the mean here:
| \(\text{total}\) | \(\approx\) | \(\dfrac{583}{9} \times 2000\) |
| \(\) | \(\approx\) | \(\mathbf{\$129\,556}\) |
(i) Explain why the choice of teenagers makes the claim unreliable.
(ii) Explain why the sample size makes it unreliable.
(i) Teenagers at a launch party chose to come because they already like the company’s games. The sample is biased towards the new game, so the result is likely too high for teenagers in general.
| \(\text{(ii)} \ 80\% \text{ of } 20\) | \(=\) | \(16 \text{ teenagers}\) |
| \(\text{one teenager}\) | \(=\) | \(\dfrac{1}{20} = 5\%\) |
(ii) With only \(20\) people, one answer changes the result by \(5\%\). A different sample of \(20\) could easily give a very different percentage, so the claim cannot be trusted for all teenagers.
Common pitfalls
Frequently asked questions
What is the difference between a census and a sample?
A census collects data from every member of the population. A sample collects data from only part of it, chosen to represent the whole.
What makes a sample biased?
A sample is biased when some part of the population has little or no chance of being chosen, or when people choose themselves, so the results lean one way.
What are the four main sampling methods?
Simple random (everyone has an equal chance), systematic (every kth member of a list), stratified (a random share from each group) and self-selected (people choose to take part, which is usually biased).
How do you estimate a population number from a sample?
Find the fraction of the sample with the feature and multiply by the population size. If \(60\) of \(250\) households have solar panels, then about \(\dfrac{60}{250} \times 12\,500 = 3000\) of \(12\,500\) households do.
How can a graph be misleading?
Its vertical axis may not start at zero, which exaggerates small differences, or pictures may grow in both height and width, so the area grows much faster than the numbers.