Report

9 – Judgment Most people do not understand chance and risk, often with dire consequences. These errors reflect misconceptions about probability. These misconceptions are revealed by people’s response to the following questions: The Banker Question Linda is 31, single, outspoken, and highly educated. Which one of the following statements is more likely to be true? She is a banker. She is a banker and a democrat. Banker. chance of just one chance of both conjunction fallacy chance of both chance of just one Correct Answer (Tversky & Kahneman, 1983) Hospital Question At Big City Hospital, about 50 babies are born daily. At Small Town Hospital, about 10 babies are born daily. At each hospital, someone counts the number of girls and boys born each day. If at least 60% of babies born that day are girls, a pink sticker is placed on the calendar. In a typical year, which hospital will have more pink stickers? ● Big City Hospital ● Small Town Hospital ● About the same Modal Answer About the same Correct Answer Small Town Hospital Explanation smaller sample less accurate survey Example # of births Probability of at least 60% girls 10 38% 50 10% (Kahneman & Tversky, 1972) Coin Question A fair coin turns up Heads 5 times in a row. Which is more likely to happen on the next toss? ● Heads ● Tails ● Neither – chance is 50-50 Correct Answer Neither. A coin has no memory! Common Error Tails gambler’s fallacy event is less likely if it just occurred More gambler’s fallacy When I was a kid, my best friend was 1 of 3 kids in the family. All 3 kids were boys. Then their mom got pregnant. The mom said, “Since I have 3 boys, I’ll probably have a girl. ” (gambler’s fallacy) Truth = 50% (Though she had a girl.) blank Demo: Imagine rolling a fair die 25 times. Write the outcomes below (1, 2, 3, 4, 5, or 6). ___ ___ ___ ___ ___ ___ 4 ___ ___ 4 ___ ___ 6 ___ ___ 3 ___ ___ 3 ___ ___ 3 ___ ___ 2 ___ ___ ___ Step 2 Write check by each outcome that matches previous outcome On average, there should be 4 repetitions Typical subject < 4 repeitions ___ ___ ___ ___ Why? Gambler’s Fallacy More Gambler’s Fallacy When teachers write multiple-choice tests, repeats are too rare. More gambler’s fallacy May 2012 My insurance company phone rep tells me that I need more hurricane coverage. I point out that hurricane hasn’t hit Florida since 2005. She says that’s why I need more coverage – the chances are getting higher! By her logic, Iowa is really in danger. Note: Cost of insurance coverage has climbed every year since 2005. Weather Facts No hurricane has hit Tampa since they started keeping records in 1851 en.wikipedia.org/wiki/List_of_Florida_hurricanes Data show Florida hurricanes come in clusters. Example. 4 in 2004 and 3 in 2005 That is, chance of hurricane is LESS if it’s been a long time since last one. Hot Hand Fact: In basketball, players often have streaks. Example. A 50% shooter might hit 4 in a row or she might miss 4 in a row. Widespread Belief: Chance of hit after a hit > Chance of hit after a miss False! Basketball statistics Chance of hit after a hit = Chance of hit after a miss hot hand fallacy = success is more likely after a success than after a failure This fallacy occurs commonly in games of skill and sometimes in games of luck. (Gilovich, Vallone, & Tversky, 1985) Data from 9 players from Philadelphia 76ers Data show that chance of hit is NOT greater after a hit (Gilovich, Vallone & Tversky, 1985) blank Poetry Lover Question Tom is a randomly selected U.S. citizen. He is asked if he likes poetry. He answers “yes.” Which is more likely to be his profession? ● Trucker ● Latin Professor Correct Answer Trucker Explanation Even if only 0.3% of truckers like poetry, they exceed 1000. Yet fewer than 1000 people are Latin Professors. Thus, answer depends partly on base rate (percent of population with trait X) Wrong answer is example of base rate neglect Disease Question On average, 1 in 191 people has Disease X. There is a blood test for X. Of people who have X, 100% test positive. (True Positive rate = 100%) Of people who don’t have X, 10% test positive. (False Positive rate = 10%) During a routine physical, Homer tests positive for X. Intuitively, what is the chance that Homer has X? ● less than 10% ● about 50% ● higher than 80% Correct Answer 5% Explanation base rate TP FP Disease X 1 in 191 100% 10% 5% Disease Y 1 in 11 100% 10% 50% Disease X 191 Chance that Homer has X 1 has X 1 TP 190 do not 19 FPs 1 / 20 = 5% Disease Y 11 1 has X 1 TP 10 do not 1 FP 1 / 2 = 50% Examples blood test base rate TP FP Disease Q 1 in 91 100% 10% Disease R 21 in 91 100% 10% Disease S 1 in 11 100% 0% 1. Quinn tests positive for Q, and Rick tests positive for R. Which event is less likely: Quinn has Q or Rich has R (smaller base rate) 2. Reece has R. Find the chance that he will test positive for R. 100%. True positive rate = chance that person with R will test positive. 3. Stan tests positive for S. What is chance that Stan has S? 100%. The test is perfect. Study Harvard Med students and faculty given Disease Question (with slightly different data). Results Answer Percent of Subjects 2% (correct) 18% 95% 45% Troubling Implication Patient tests positive for X Doctor tells patient his chance of having X is 95% but truth is 2% Patient chooses risky surgery and dies. (Casscells, Schoenberger, & Graboys, 1978, NEJM) Base rate neglect in academia … What is the chance that a manuscript will be accepted? Academics rely heavily on their judgment of manuscript’s quality. They tend to ignore the journal’s acceptance rate (i.e., base rate). Journal acceptance rates vary dramatically. For APA journals, acceptance rate is between 9% and 69% (2009 data) For all psych journals, range is even greater. blank Conditional Probabilities Disease question requires understanding of conditional probabilities. conditional probability = Chance that A is true if B is true In Disease X question, Chance that Homer has disease X if he tested positive) = Chance that Homer will test positive if he has disease X) = 100% Conditional probabilities are hard to understand. (e.g., Cosmides & Tooby, 1996; Gigerenzer, 2002) 5% Example In 1995, celebrity O. J. Simpson was put on trial for murdering his ex-wife. Simpson had previously battered her. His attorney: “Only 0.04% of male batterers later murder the battered woman.” Translation: If Mr. X once battered Ms. Y, the chance that X will murder Y = 0.04% Relevant data: If Mr. X once battered Ms. Y, the chance that X is murderer if Y was murdered = 93% Neither statistic was presented to jury. (I. J. Good, Nature, 1995, 1996; Strogatz, New York Times, 2010) People better understand risk if they are presented with frequency data. Study Doctors given two versions of the same question Conditional Probability Version The probability that a woman has breast cancer is 1%. If she has breast cancer, the probability of a positive mammogram is 80%. If a woman does not have breast cancer, the probability of a positive result is 10%. Take, for example, a woman who has a positive result. What is the probability that she actually has breast cancer? Natural Frequency Version 10 out of every 1000 women have breast cancer. Of the 10 women with breast cancer, 8 will have a positive result on mammography. Of the 990 women who do not have breast cancer, 99 will still have a positive result. Take, for example, a sample of women who have positive mammograms. How many of these women actually have breast cancer? Results (Hoffrage & Gigerenzer, 1998) Hungry Sailors 10 starving men in a lifeboat agree to sacrifice one man so that the others will live. They put 9 green marbles and 1 red marble in a bag. One at a time, each man chooses a marble. After each marble is chosen, the marble is NOT placed back in the bag. The first player to choose red is the loser. Is it better to choose first or tenth, or does it not matter? Answer: It does not matter Hungry Sailors Version 2 Same as above, except each man replaces his marble before next man chooses. Is it better to choose first or tenth, or does it not matter? Answer: Tenth The whale ship Essex was sunk by a whale on November 20, 1820. Twenty men escaped in three small craft and wandered the Pacific. After being unable to catch fish, they ultimately resorted to cannibalism by common consent. Three months later there were two boats and five men left. Rescuers found bug-eyed stick figures hunkered over a pile of human ribs, with finger bones stashed in their pockets. (Mark Shone, 2000) A similar scenario took place in Edgar Allen Poe novel The Narrative of Arthur Gordon Pym of Nantucket blank Two Girls Question A woman moves into the neighborhood. When you meet her, you find out that her name is Beth and that she has two kids. Later, you see one of Beth’s kids playing in her yard, and the kid is a girl. What is the chance that Beth’s other child is also a girl? Answer 1/3 Explanation Random sample of 400 women with exactly 2 kids. first born second born ~ N_ B B 100 B G 100 G B 100 G G 100 1/3 of these have 2 girls The Birthday Question A group of 41 people is randomly selected from the general population. What is the chance that at least two of them will have the same birthday? 10% Answer 30% 50% 70% 90% Explanation The question is: does any person share a birthday with any other person. 41 people 820 different pairs. (e.g., Voracek et al., 2008) 90% # of people 23 32 41 50 366 # of possible matches 253 496 820 1225 66795 Probability of a match 50% 75% 90% 97% 100% n n(n-1)/2 -- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - # of people P(match) = 1 – P(no match) 2 1 – (364/365) person 2 = .003 3 1 – (364/365)(363/365) person 2 person 3 = .008 Calculation assumes 1) all birthdates are equally common (not quite true) 2) no Feb 29 Correcting for these assumptions has no discernible effect on answer. Monty Hall Dilemma A game show contestant is shown 3 curtains. One curtain has a prize behind it. (Stagehands randomly choose prize location.) The emcee knows which curtain has the prize behind it. Player chooses a curtain. Emcee then opens a non-chosen losing curtain. Player is then allowed to stay with original choice or switch. Should player switch or stay, or does it not matter? Modal Response It doesn’t matter. Correct Answer Switch 2/3 chance of winning 10-Curtain Version of Monty Hall Dilemma 1 of 10 curtains has a prize behind it. (Stagehands randomly choose prize location.) The emcee knows which curtain has the prize behind it. Player chooses curtain. Emcee then opens 8 non-chosen losing curtains. Player is then allowed to stay or switch. Should player switch or stay, or does it not matter? Chance of winning if player switches = 9/10 The End Bankers Banker-Democrats Banker. chance of just one chance of both conjunction fallacy chance of both chance of just one Correct Answer (Tversky & Kahneman, 1983) Disease Question On average, 1 in 91 people has Disease X. There is a blood test for X. Of people who have X, 100% test positive. (TP = True Positive rate = 100%) Of people who don’t have X, 10% test positive. (FP = False Positive rate = 10%) During a routine physical, Homer tests positive for X. What is the chance that Homer has X? 10% Correct Answer 10% Modal Answer 90% 30% 50% 70% base rate neglect (continued) 90% Explanation base rate TP FP Chance that Homer has X Disease X 1 in 91 100% 10% 10% Disease Y 21 in 91 100% 10% 75% 91 1 has X 1 TP 90 do not 9 FPs Disease X 1 / 10 = 10% Disease Y 91 21 have X 21 TPs 70 do not 7 FPs 21 / 28 = 75% Real Medical Study Success rate for two kidney stone treatments Treatment A All Patients Treatment B 78% (273/350) 83% (289/350) Small Stones 93% (81/87) 87% (234/270) Large Stones 73% (192/263) 69% (55/80) How can this be? Patients were NOT randomly assigned. Instead, doctors usually chose A for large stones, which are harder to treat. Simpson’s Paradox trend reverses when samples combined (Simpson, 1951) More Simpson’s Paradox - UC Berkeley gender bias UC Berkeley sued by because of these 1973 admission data Men Women Applicants % admitted 8442 44% 4321 35% Then data were partitioned by department. Department Men Women Applicants % admitted Applicants % admitted A 825 62% 108 82% B 560 63% 25 68% C 325 37% 593 34% D 417 33% 375 35% E 191 28% 393 24% F 272 6% 341 7% Question Scientists compute p in order to test the null hypothesis (H0). What does p equal? P(data given H0 is true) correct OR P(H0 is true given data) common belief (if true, p would be useful) The Disease X question is formally answered by using: Bayes’ Theorem where P(A | B) = Probability of A given that B has happened and P(B | A) = Probability of B given that A has happened (Bayes, 1763) Solution to Disease X question: Let A = disease X is present P(A) P(B | A) P(B | not A) and let B = positive test = 1/91, which means that P(not A) = 90/91 = P(positive test given presence of X) = TP = 100% = 1 = P (positive test given absence of X ) = FP = 10% = 0.10 P(X | positive test) = P(B | A) = (1)(1/91) / [(1)(1/91) + (0.1)(90/91)] = 0.1 Based on studies described thus far today, few people understand chance. However… others argue that these studies are misleading because: The questions require algorithms, but people use “fast-and-frugal” heuristics. The questions were contrived to exploit fallibility of heuristics. The heuristics work well in everyday life. (e.g., Gigerenzer, 2002) (e.g., Gigerenzer, 2002) Two versions of mammogram data for women over 50 Relative Risk version If woman has mammogram, her chance of cancer death in next 10 years drops 25% Frequency version If 1000 women skip mammogram, 4 will die of breast cancer in next decade. If all 1000 have mammogram, the number who die will decrease from 4 to 3. Frequency Data are easier to understand and more informative. (e.g., Gigerenzer & Edwards, 2003) Hungry Sailors Six men in a lifeboat are starving, so they agree that one must be sacrificed. They put 5 green marbles and 1 red marble in a bag. One at a time, each man chooses a marble. The first player to choose red is the loser. After each marble is chosen, the marble is NOT placed back in the bag. Is it better to choose first or sixth, or does it not matter? Going first has obvious disadvantage: You’re the only player who must choose! But going first has an advantage: Only 1 in 6 is red. For others, 1 in 5, 1 in 4, … Answer: It does not matter Optional P(Man A gets red) = 1/6 P(Man B gets red) = P (A doesn’t choose red AND B does) = (5/6)(1/5) = 1/6 Hungry Sailors Version 2 Six men in a lifeboat are starving, so they agree that one must be sacrificed. They put 5 green marbles and 1 red marble in a bag. One at a time, each man chooses a marble. The first player to choose red is the loser. After each marble is chosen, the marble is placed back in the bag. Is it better to choose first or sixth, or does it not matter? Answer: Sixth Optional P(Man A gets red) = 1/6 = 6/36 P(Man B gets red) = P(A gets green AND B gets red)= (5/6)(1/6) = 5/36