Confidence

In this exercise, we will determine the global life expectancy. We make the following assumptions:

  • we are interested in the life expectancy per country, i.e. we ignore the difference in population sizes completely.

  • we only have resources to check the life expectancy of 10 countries

  • the life expectancy follows a Normal Distribution

  • we do not have access to historical data

Exercise 1: Expert estimate

Let’s start with an estimate by ourselves: Without looking anything up, write down what you think the life expectancy is.

Compare the results. Calculate the mean as a consensus estimate. Discuss briefly why you estimated which way.

This estimate is better than nothing, but not that much better.

Exercise 2: Representative sample

Because we may only use data for 10 countries, we need to come up with a list of 10 countries. This is your sample. For an unbiased estimate, the sample should be representative.

Discuss the following alternatives:

  • randomly pick 10 countries with a fixed random seed.

  • pick 2 countries from each continent

  • sort alphabetically, then pick every 20th country

  • create 10 slices for the latitude, pick one country from each slice

  • find the 10 most frequent languages, then pick one country where that language is spoken

What other ways of generating a representative sample can you think of?

Exercise 3: Key concepts

Clarify the concepts:

  • population

  • sample

  • representative sample

  • population mean

  • sample mean

Exercise 4: Sample mean

Calculate the mean life expectancy \(\overline{x}\) of the 10 countries from the table in countries2015.csv. (or get a more recent dataset from gapminder.org/data)

In Polars, you can use the following expression on a pl.DataFrame to select multiple rows by the index:

countries = ["name1", "name2", ...]
sample = df.filter(pl.col("name").is_in(countries))

Calculate the mean of the sample.

Exercise 5: Standard Error

Calculate the standard error of the sample mean using the equation:

\[stderr = \frac{\sigma}{\sqrt{n}}\]

where \(\sigma\) is the standard deviation of the population.

Hint

Compare the standard error against the standard deviation of the population.Is it larger or smaller?

If you increase the sample size n, does the standard error go up or down?

Exercise 6: Critical value

We want to calculate a confidence interval of the sample mean, so that:

\[\overline{x} \pm c * stderr\]

Use the following procedure:

  1. set a confidence threshold (e.g. 90%, 95% or 99%)

  2. calculate the critical value (or z-score) of a standard normal distribution that gives half that probability.

The critical value says, how many standard deviations you need to move away from the mean so that the are under the normal distribution is your given probability.

For a 90% confidence interval, you need to find the critical value for a probability of 0.95. In Python, you can use:

import statistics

# standard normal distribution
norm = norm = statistics.NormalDist(0.0, 1.0)
norm.inv_cdf(0.95)

Multiply the critical value with the standard error. This is your margin. Adding and substracting the margin to/from your sample mean gives you the upper and lower boundary of your confidence interval.

Use the values to formulate a sentence like:

the avearage age of cats is between 1 and 99 years with 77% confidence.

Exercise 7: Population mean

Finally, calculate the mean life expectancy \(\mu\) of the population (the entire dataset). Which of your estimates are inside the confidence interval?

Exercise 8: Multiple samples

If you want to verify the process, create 1000 samples of the same size using:

df.sample(10)

Calculate the mean life expectancy of each sample. You should end up with 1000 sample means.

Sort the means, then slice off the middle 90% with [50:-50]. The min and max of the remaining ones should have a similar width as your confidence interval.

Compare the histogram of the sample means against the histogram of actual life expectancies. It should be quite different.