Report

Research supported by TLRI Randomisation: part 2 Randomisation S1 Study design and inference Experiments & the Randomisation Test: • The difference between two means Watch out for: • • The ‘chance is acting alone’ explanation How we assess the plausibility of the ‘chance alone’ explanation – (test for ‘chance alone’) Randomisation S2 What does ‘chance alone’ look like? iNZightVIT Randomisation Chapter 1 The Walking Babies Experiment Does a special exercise programme lower walking age? Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn” Science, Vol. 176 (1972), pp314-315 10 male infants (& parents) were randomly assigned to one of two treatment groups. First walked without support: Treatment Exercise Control 9 13.25 Age (months) 9.5 9.75 10 11.5 12 13.5 11 11.5 Randomisation S4 The Walking Babies Experiment 9 10 11 12 Age (months) 13 14 Randomisation S5 Is chance alone likely to generate differences as big as our difference? Tail proportion: roughly ____% Re-randomisation distribution of differences (under chance alone) The Walking Babies Experiment Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the exercise has no effect) • We can rule out ‘chance is acting alone’ as a plausible explanation for the difference between the two groups. • We have evidence against ‘chance is acting alone’ • We have evidence that chance is not acting alone Randomisation S7 The Walking Babies Experiment Possible explanation: If chance not acting thendifference what else One possibleis explanation for alone, the observed is also acting togroups: help produce the observed between these two Chance is acting alone (the exercise has no effect) difference? • We can rule out ‘chance is acting alone’ as a plausible Remember: explanation for the difference between the two groups. Random assignment to 2 groups & each group receives • different We havetreatment. evidence against ‘chance is acting alone’ • We have evidence that chance is not acting alone Randomisation S8 The Walking Babies Experiment Conclusion: Because the male infants (& parents) were randomly assigned to the groups, we may claim that the exercise was effective in lowering the walking age. Because these subjects in this experiment were volunteers (not randomly selected), then we would need to consider carefully as to which wider group(s) this conclusion may apply. Randomisation S9 Two types of Inference There are two types of inference 1. Sample-to-population eg x = 172cm so the population mean is about 172cm. 2. Experiment-to-causation eg The treatment was effective Randomisation S10 The Walking Babies Experiment Does a special exercise programme lower walking age? Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn” Science, Vol. 176 (1972), pp314-315 10 male infants (& parents) randomly assigned to either the Exercise group or the Control exercise group. First walked without support: Treatment Exercise Control exercise 9 11 Age (months) 9.5 9.75 10 10 15 11.75 13 10.5 Randomisation S11 The Walking Babies Experiment 9 10 11 12 13 14 15 Age (months) Randomisation S12 The Walking Babies Experiment Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the exercise has no effect) Randomisation S13 The Walking Babies Experiment Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the exercise has no effect) Is the ‘Chance alone’ explanation simply not plausible? • Would our observed difference be unlikely when chance is acting alone? • How do we determine whether a difference is unlikely when chance is acting alone? • See what’s likely and what’s unlikely when chance is acting alone Randomisation S14 Our observed difference = 1.4 months Is chance alone likely to generate differences as big as our difference? Re-randomisation distribution of differences (under chance alone) Tail proportion: roughly ___/1000 (____%) Do the actual exercises lower the walking age? Our tail proportion of ____/1000 = ___% means: • ___ times out-of-a-1000 times we get a difference of 1.4 months or more, when chance is acting alone. • Under chance alone, it’s not unusual to get a difference bigger than or equal to our observed difference of 1.4 months. • A difference of 1.4 months or greater is not unusual when chance is acting alone, . . . therefore chance could be acting alone. Randomisation S16 The Walking Babies Experiment Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the treatment has no effect) • We can NOT rule out ‘chance is acting alone’ as a plausible explanation for the observed difference between the two groups. • We have no evidence against ‘chance-is-acting-alone’ • Chance COULD be acting alone . . . BUT something else, as well as chance, COULD ALSO be acting. Randomisation S17 The Walking Babies Experiment Possible explanation: One possibleis explanation for alone, the observed If chance not acting thendifference what else between thesebe two groups:to help produce the could also acting Chance is acting alone (the treatment has no effect) observed difference? • We can NOT rule out ‘chance is acting alone’ as a Remember: plausible explanation for the observed difference Random assignment to 2 groups & each group receives between the two groups. different treatment. • We have no evidence against ‘chance-is-acting-alone’ • Chance could be acting alone . . . BUT something else, as well as chance, COULD ALSO be acting. Randomisation S18 The Walking Babies Experiment Conclusion: Because the male infants (& parents) were randomly assigned to the groups, we conclude that the observed difference is the result of • EITHER chance acting alone • OR an exercise effect together with chance acting – we do NOT have ENOUGH INFORMATION to MAKE A CALL as to which one. Randomisation S19 Did she brush her teeth (2)? 1. Formulate statement to test. 1. She has brushed her teeth. 2. Data (information at hand). 2. The toothbrush is wet. 3. Consider 1. and the data: 3. The-toothbrush-is-wet would If 1. is true, then what are the be likely if she had brushed chances of getting data like her teeth. that in 2.? 4. Review the statement in 1. in 4. Therefore, she could have light of 3. together with the brushed her teeth. data in 2. We have no evidence that she has not brushed her teeth. I do not know Randomisation S20 Is the actual exercise effective? 1. Formulate statement to test. 1. Chance is acting alone. 2. Data (information at hand). 2. Observed diff = 1.4 months 3. Consider 1. and the data: 3. A difference of 1.4 months or If 1. is true, then what are the greater is chances of getting data like that in 2.? 4. Review the statement in 1. in light of 3. together with the data in 2. Randomisation S21 Is the actual exercise effective? 1. Formulate statement to test. 1. Chance is acting alone. 2. Data (information at hand). 2. Observed diff = 1.4 months 3. Consider 1. and the data: 3. A difference of 1.4 months or If 1. is true, then what are the greater is not unusual when chances of getting data like chance is acting alone. that in 2.? 4. Therefore, chance could 4. Review the statement in 1. in be acting alone OR light of 3. together with the something else could be acting along with data in 2. chance. we do NOT have ENOUGH INFORMATION to MAKE A CALL as to which one.Randomisation S22 Guidelines for assessing ‘Chance alone ’ When the tail proportion is small (less than 10%): • the observed difference would be unlikely when chance is acting alone . . . therefore, it’s a fairly safe bet chance is not acting alone. • we have evidence against ‘chance-is-acting-alone’ • we have evidence that chance is not acting alone Chapter 1 Guidelines for assessing ‘Chance alone’ When the tail proportion is large (10% or more ) then: • the observed difference is not unusual when chance is acting alone, therefore chance could be acting alone • we have NO evidence against ‘chance is acting alone’ • EITHER chance could be acting alone OR something else as well as ‘chance’ COULD also be acting. -- we do NOT have ENOUGH INFORMATION to MAKE A CALL as to which one. Randomisation S24 Questions… Chapter 1