### - Unlocking the Power of Data

```Bootstraps and Scrambles:
Letting Data Speak for
Themselves
Robin H. Lock
Burry Professor of Statistics
St. Lawrence University
[email protected]/* <![CDATA[ */!function(t,e,r,n,c,a,p){try{t=document.currentScript||function(){for(t=document.getElementsByTagName('script'),e=t.length;e--;)if(t[e].getAttribute('data-cfhash'))return t[e]}();if(t&&(c=t.previousSibling)){p=t.parentNode;if(a=c.getAttribute('data-cfemail')){for(e='',r='0x'+a.substr(0,2)|0,n=2;a.length-n;n+=2)e+='%'+('0'+('0x'+a.substr(n,2)^r).toString(16)).slice(-2);p.replaceChild(document.createTextNode(decodeURIComponent(e)),c)}p.removeChild(t)}}catch(u){}}()/* ]]> */
Science Today
SUNY Oswego, March 31, 2010
Bootstrap CI’s &
Randomization Tests
(1) What are they?
(2) Why are they being used more?
(3) Can these methods be used to
introduce students to key ideas of
statistical inference?
Example #1: Perch Weights
Suppose that we have collected a sample of
56 perch from a lake in Finland.
Estimate and find 95% confidence bounds for
the mean weight of perch in the lake.
From the sample:
n=56
X=382.2 gms
s=347.6 gms
Classical CI for a Mean (μ)
“Assume” population is normal, then
X 
s
*
~ t n 1
 X  t n 1
s
n
n
For perch sample:
347.6
382.2  2.004
56
382 .2  2.004  46.5
382 .2  93.1
(289.1, 475.3)
Possible Pitfalls
What if the underlying population is NOT normal?
Dot Plot
Perch
200
400
600
800
1000
Weight
What if the sample size is small?
What is you have a different sample statistic?
What if the Central Limit Theorem doesn’t apply?
(or you’ve never heard of it!)
Bootstrap
Basic idea: Simulate the sampling distribution
of any statistic (like the mean) by repeatedly
sampling from the original data.
Bootstrap distribution of perch means:
• Sample 56 values (with replacement)
from the original sample.
• Compute the mean for bootstrap sample
• Repeat MANY times.
Original Sample (56 fish)
Bootstrap “population”
Sample and compute means
from this “population”
Bootstrap Distribution of 1000 Perch Means
Dot Plot
Measures from Sample of Perch
250
300
350
400
xbar
450
500
550
CI from Bootstrap Distribution
Method #1: Use bootstrap std. dev.
X  z * Sboot
For 1000 bootstrap perch means: Sboot=45.8
382.2  1.96 45.8  382.2  89.8  (292.4,472.0)
CI from Bootstrap Distribution
Method #2: Use bootstrap quantiles
Dot Plot
Measures from Sample of Perch
2.5%
250
300
2.5%
350
400
450
xbar
299.6
95% CI for μ
476.1
500
550
Butler & Baumeister (1998)
Example #2: Friendly Observers
Experiment: Subjects were tested for performance
on a video game
Conditions:
Group A: An observer shares prize
Group B: Neutral observer
Response: (categorical)
Beat/Fail to Beat score threshold
Hypothesis: Players with an interested observer
(Group A) will tend to perform less ably.
A Statistical Experiment
Divide
Record
at
therandom
24
data
subjects
(Beat
into or
two
Nogroups
Beat)
Group A: Share
Group B: Neutral
Friendly Observer Results
Group A
(share prize)
Group B
(prize alone)
Beat Threshold
3
8
11
Failed to Beat
Threshold
9
4
13
12
12
Is this difference “statistically significant”?
Friendly Observer - Simulation
11 Black (Beat) and 13 Red (Fail to Beat)
2. Shuffle the cards and deal 12 at random to
form Group A.
3. Count the number of Black (Beat) cards in
Group A.
Automate this
4. Repeat many times to see how often a
random assignment gives a count as small as
the experimental count (3) to Group A.
Friendly Observer – Fathom
Computer Simulation
48/1000
Automate: Friendly Observers Applet
Allan Rossman & Beth Chance
http://www.rossmanchance.com/applets/
Observer’s Applet
Fisher’s Exact test
12 12  12 12  12 12  12 12 
           
0 11  1 10   2  9   3  8 

P( A Beat < 3)  24  24  24  24
 
 
 
 
 
 
 
 
11 
11 
11 
11 
 0.000005  0.00032  0.0058  .04363
 P(A Beat  3)  0.0498
Example #3: Lake Ontario Trout
X = fish age (yrs.)
Y = % dry mass of eggs
n = 21 fish
r = -0.45
Is there a significant
negative association
between age and %
dry mass of eggs?
Ho:ρ=0 vs. Ha: ρ<0
Scrambled
FishEggs FishEggs
Age
PctDM
1
7
37.15
37.35
2
8
36.15
38.05
3
8
37.45
37.1
4
9
38.95
35.7
5
9
36.15
37.9
6
9
37.05
36.45
7
9
36.15
35.1
8
10
38.35
36.75
9
10
38.95
37.15
10
11
37.7
36.5
11
11
37.4
35.1
12
12
38.05
37.7
13
12
36.45
37.1
14
13
37.9
37.4
15
13
37.55
36.5
16
13
36.35
17
14
37.55
36.75
18
15
37.45
37.05
19
17
37.35
36.15
20
18
38.35
35.7
<new>
Randomization Test for
Correlation
•Randomize the PctDM values to be
assigned to any of the ages (ρ=0).
•Compute the correlation for the
randomized sample.
•Repeat MANY times.
•See how often the randomization
correlations exceed the originally
observed r=-0.45.
Randomization Distribution of
Sample Correlations when Ho:ρ=0
Dot Plot
Measures from Scrambled FishEggs
26/1000
-0.6
-0.4
r=-0.45
-0.2
0.0
r
0.2
0.4
0.6
Confidence Interval for Correlation?
Construct a bootstrap distribution of correlations
for samples of n=20 fish drawn with replacement
from the original sample.
Bootstrap Distribution of
Sample Correlations
Dot Plot
Measures from Sample of FishEggs
-0.8
r=-0.74
-0.6
-0.4
-0.2
r
0.0
r=-0.08
0.2
Bootstrap/Randomization Methods
• Require few (often no) assumptions/conditions
on the underlying population distribution.
• Avoid needing a theoretical derivation of
sampling distribution.
• Can be applied readily to lots of different
statistics.
• Are more intuitively aligned with the logic of
statistical inference.
Can these methods really be used to introduce
students to the core ideas of statistical inference?
Coming in 2012…
Statistics: Unlocking the Power of Data
by Lock, Lock, Lock, Lock and Lock
```