### Teaching-L3-Statistical-Reports

```How are we doing?
Sort the types of error into sampling and non-sampling
errors, then match the situations to the types of error.

Understanding sampling error and non-sampling error

Choosing the correct rule of thumb for margin of error

Knowing what to do when proportion is < 30% or > 70%

Deciding whether a study is an experiment or an observation

Understanding sampling methods

Polls and surveys

What else?
 Understanding
sampling error
sampling error and non-
ESA Study guide Level 3 Statistics (and ESA L3
Statistics Learning Workbook)
“Non-sampling errors result from how information is collected
from a sample… Examples are:
Non-participation
False answers
Unavailability
Lack of opinion…”
No mention of major sources of non-sampling
error such as incorrect sampling frame, biased
sampling method, survey method, etc.
 AME
Level 3 Statistics Workbook (revised
for 2014)
 Nulake
… but that leaves us with textbooks
and exercise books with either no
information or unclear information.
Where to from here?
 Sampling
error arises due to the
variability that occurs by chance because
a random sample, rather than an entire
population, is surveyed.
 Non-sampling
error is all error that is not
sampling error.
Hidden agenda
Sampling frame
which doesn’t match
target population
Biased
sampling
method
Transfer of
findings
Unclear or
leading
questions
Misinterpretation
of analysis
Wrong analysis
methods
Coding
errors
Survey format
effects
Behavioural
issues
Interviewer
effects
Non-response
 Choosing
the correct rule of thumb for
margin of error
“If the results are within the same survey or poll, then
Margin of Error of Difference = 2×Margin of Error of the Poll”
“Dependent Probabilities
… two events from within the same question then these two
probabilities are dependent on each other.
Independent Probabilities
If we have two polls or two independent questions in the same
survey that are independent of each other…
… MOE = 1.5(average MOE).”
Questions can be asked of different groups within
the same survey or poll, so difference is not
always 2×MOE
 The definition of dependent probabilities makes
it seem as though comparing different questions
for the same group would be independent.

“A confidence interval for the population mean is  ±

…

…A confidence interval for the population proportion is
p±
1−

”
 ESA
Study Guide Level 3 Statistics
 ESA Study Guide Level 3 Learning
Workbook
 Sigma (not in the book at all)
 AME Level 3 Statistics Workbook
 D&D Practice External Assessments
 others?

Focus on one group or two groups
• One group, one answer
• One group, difference
• Two groups, difference

Improve our resources
The 3 rules of thumb are derived from 4 margin of
error formulas for confidence intervals.
(1−)


1 group, 1 question  =  ×

1 group, difference, answers to 1 question
=×

1 group, difference, between 2 questions
=×

1 +2 − 1 −2 2

1 +2 , 1 +2 −(1 −2 )2

2 groups, difference
1 (1 − 1 ) 2 (1 − 2 )
E=z×
+

…comparing answers to the same question
(eg National or Labour) for one group is
different from comparing the answers to
two different questions for one group.

1 group, difference between 2 questions
=×

1 +2 , 1 +2 −(1 −2 )2

1 +2 − (1 − 2 )2 and
1 +2 − (1 − 2 )2
both ≈ 1 for p1 , p2 between 0.3 and 0.7
so rule of thumb for MOE simplifies to  = 2 ×
1

1
p1
k=0 . 8
0.8
k=0 . 9
k=1
0.6
k=1 . 1
0.4
k=1 . 2
0.2
p2
0.2
0.4
0.6
0.8
1
 It
is enough for students to know that there
are formulas that simplify to the rules of
thumb for percentages between 30% and
70%.
 Students
should not be memorising
complicated formulas to use below 30% or
over 70%.
 Students
should not be memorising one full
formula and substituting it into the other
rules of thumb.
 Knowing
what to do when proportion is
< 30% or > 70%
Explain why it would be inappropriate to
use the reported margin of error to
construct a confidence interval for the
percentage of respondents from the
November 2009 survey who never talk to
their friends on a landline.
(the reported percentage was 9%)

Confidence interval using the rule of thumb
 Interpret
CI:
With at least 95% confidence,
we can infer that the percentage of adult New
Zealanders with health insurance who will have
had their blood pressure checked during the
previous 12 months is somewhere between
87.4% and 92.6%.

that they can be more confident that the true
population proportion (or percentage point difference)
is within the rule of thumb based interval if the
proportion(s) is/are below 30% or over 70%.

“At least 95% confidence” conveys this understanding.

the actual level of confidence for the rule of thumb
confidence interval is higher than 95%.

the true 95% confidence interval would be narrower
than the rule of thumb confidence interval.

“margin of error” describes half of a 95% CI so we
can’t call the result of our rule of thumb calculation a
MOE unless 0.3<p<0.7.
1.
2.
3.
4.
Match the context to the correct rule of
thumb.
Calculate the confidence interval.
Match your confidence interval to the
correct confidence interval (harder than
you think!)
Match your context and interval to the
correct interpretation.
 Deciding
whether a study is an
experiment or an observation
 The
word “experiment” is often used
loosely to mean an observational study.
A
true experiment must have an
intervention. The experimenter must
change something.
 The
 An
intervention can be very small.
experiment where the groups are not
randomly allocated is still an experiment,
just not a well-designed one.
 In
the real world, sampling is often very
complex
 Students should be familiar with random
sampling (simple, systematic, stratified,
cluster)
 Students should be aware of nonprobability methods (convenience, quota)
Weighting is used with stratified sampling.
Pasifika make up 7% of NZ population so in a
random sample of 1000, we would expect only
70 Pasifika people. By taking an extra sample
of 200 more Pasifika we reduce the MOE for
comparisons involving Pasifika. When using
our data for all the NZ population we weight
the 270 Pasifika as if they were only 70 people
(multiply by 70/270), so that they are
represented in proportion to the population.
Surveys tend to get more nonresponse
from men than women, so weighting is
used to get a picture of the population in
proportion. If we take a sample of 1000
people and get 400 men and 600 women,
we weight the men’s responses (×0.5/0.4)
and the women’s (×0.5/0.6), so that they
are in proportion for the population. We
only weight when combining to get a
picture of the whole population.
A
poll is a type of survey.
 All polls are surveys but not all surveys are polls.
poll
survey
Few questions
(how many is few?)
May have many
questions
Multi-choice
questions
May ask open
questions
```