### SIU ASQ Spring Conf 2013 - Jim Akers

```<Function>
<Process Name>
Planning for a
Successful Design of
Experiment
Jim Akers
SIU ASQ Spring Conference 2013
Topics we will cover
This presentation will provide a process for planning a
successful design of experiment.
We will cover
• the basics of DOE
• creating a charter
• choosing an appropriate design
The participants will take away a DOE charter template
and a flowchart for simple design selection. The major
create an appropriate experiment.
Jim Akers
2
THE BASICS OF DOE
Jim Akers
3
Objectives in Using DOE
–
–
–
–
Improving performance characteristics
Understand relationships between process
variables
Reducing costs
Understand how to optimize processes
• Creates the ultimate process knowledge to
Better
Faster
Cheaper
Jim Akers
4
Data
18
32
31
21
23
16
26
27
19
20
30
16
23
25
22
29
24
38
22
19
28
27
33
17
21
21
17
14
12
14
17
17
13
21
45
41
33
26
21
15
8
19
11
25
24
17
18
28
26
34
Fuel Economy of 50 automobiles (in mpg)
Plot a histogram and calculate the average and standard deviation
Fuel Economy
16
X  22.88
S  7.7266
14
Number of Cars
12
10
8
6
4
2
0
0 to <6
6 to <12
12 to <18
18 to <24
24 to <30
30 to <36
36 to <42
42 to <48
48 to <54
54 to <=60
mgp
Jim Akers
5
What Might Explain the Variation?
MANPOWER
METHOD
MACHINE
EFFECT
Effect
MEASUREMENT
MOTHERNATURE
MATERIAL
• DOE is about discovering and quantifying the
magnitude of cause and effect relationships.
• We need DOE because intuition can be misleading....
but we’ll get to that in a minute.
• Regression can be used to explain how we can
model data experimentally.
Jim Akers
6
Mileage Data with Vehicle Weight:
• Let’s take a look at the mileage data and see
if there’s a factor that might explain some of
the variation.
• Draw a scatter diagram for the following data:
X
Observation
1
2
3
4
5
6
7
8
9
10
Jim Akers
X - Weight (lbs)
3000
2800
2100
2900
2400
3300
2700
3500
2500
3200
Y=f(X)
Y
Y - Mileage(mpg)
18
21
32
17
31
14
21
12
23
14
7
Regression Analysis
• Plot the data on a scatter chart and
draw a best fit line
• Determine the equation for that line,
–
you now have a ‘model’ for the data
Y=f(X)
mpg
Scatter Chart (Weight vs mpg)
35
30
25
~21
20
15
10
5
0
1900
Jim Akers
y = -0.0152x + 63.507
R² = 0.9191
2400
2800
2900
Weight
We have now
experimented with
one factor, but that
does not explain all
of the variation.
3400
3900
8
Experimenting with a System
There are a few basic ways to understand a process
you are working on. We will talk about two of them.
• Classical 1FAT experiments
–
One factor at a time (1FAT) focuses on one variable at
two or three levels and attempts to hold everything else
constant (which is impossible to do in a complicated
process).
• DOE
–
When properly constructed, it can focus on a wide
range of key input factors and will determine the
optimum levels of each of the factors.
Jim Akers
9
1FAT Example
• Let’s consider how two known (based on
years of experience) factors affect gas
mileage, tire size (T) and fuel type (F).
T(1,2)
F(1,2)
Jim Akers
Y=f(X)
Y
Fuel Type
Tire size
F1
T1
F2
T2
10
1FAT Design
Step 1:
Select two levels of tire size and two kinds
of fuels.
Step 2:
Holding fuel type constant (and everything
else), test the car at both tire sizes.
Jim Akers
Fuel Type
Tire size
Mpg
F1
T1
20
F1
T2
30
11
1FAT Design
Since we want to maximize mpg the more
desirable response happened with T2
Step 3: Holding tire size at T2, test the car at
both fuel types.
Jim Akers
Fuel Type
Tire size
Mpg
F1
T2
30
F2
T2
40
12
1FAT Design
• Looks like the ideal setting is F2 and T2 at
40mpg.
• This is a common experimental method.
What about the possible interaction effect
of tire size and fuel type? F2T1
Jim Akers
Fuel Type
Tire size
Mpg
F1
T2
30
F2
T2
40
13
1FAT Design
• Suppose that the untested combination F2T1 would
produce the results below.
• There is a different slope so there appears to be an
interaction. A more appropriate design would be to test
all four combinations.
–
That is called a full factorial
70
60
F2
mpg
50
40
30
F1
20
10
0
T1
T2
Tire Size
Jim Akers
14
What About Other Factors – and Noise?
• We need a way to
–
–
–
investigate the relationship(s) between variables
distinguish the effects of variables from each other (and maybe tell if
they interact with each other)
quantify the effects...
...So we can predict, control, and optimize processes.
Jim Akers
15
DOE to the Rescue!!
Y=f(X)
DOE uses purposeful changes of the inputs (factors) in order to
observe corresponding changes to the outputs (response).
Remember the IPO’s we did – they are real important here.
Run
X1
X2
X3
X4
1
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
2
3
4
5
6
7
8
Jim Akers
Y1
Y2
Y3
Y-bar
SY
16
The Basics
• To ‘design’ an experiment, means to pick the
points that you’ll use to understand the
design space.
X1
A
B
(-,+)
Design Space
Y
(+,+)
High (+)
Factor B Settings
In tabular form, it
would look like:
Low (-)
(-,-)
Low (-)
Jim Akers
X2
(+,-)
Factor A Settings
Run
A
B
AB
1
2
3
4
+
+
+
+
+
+
High (+)
17
Full vs. Fractional Factorial
• A full factorial is an experimental design which
contains all levels of all factors. No possible
treatments are omitted.
–
–
The preferred (ultimate) design
Best for modeling (Response Surface Methods)
• A fractional factorial is a balanced experimental
design which contains fewer than all
combinations of all levels of all factors.
–
–
–
Jim Akers
The preferred design when a full factorial cannot
be performed due to lack of resources
Okay for some modeling
Good for screening
18
2k  r runs
2 Level Designs
k  number of factors
• Full factorial
–
–
–
–
2 level
3 factors
8 runs
Balanced (orthogonal)
23  8 runs
• Fractional factorial
–
–
–
231  4 runs
Jim Akers
–
2 level
3 factors
4 runs - half fraction
Balanced (orthogonal)
19
3k  r runs
3 Level Designs
k  number of factors
• Full factorial
–
–
–
–
–
Jim Akers
3 level
3 factors
27 runs
Balanced (orthogonal)
33  27 runs
Used when it is expected the response is non-linear
20
Measuring An “Effect”
Response - Y
Average Y when
A was set ‘high’
Average Y when
A was set ‘low’
Low
High
Factor A
• The difference in the average Y when A was ‘high’ from
the average Y when A was ‘low’ is the ‘factor effect’
• The differences are calculated for every factor in the
experiment
Jim Akers
21
Looking For Interactions
B = High
B = Low
Strong
Slight
Low
-Y
Response
B = Low
B = High
Low
Factor A
High
High
Factor A
Jim Akers
Response - Y
Response - Y
When the effect of one factor changes due to the effect
of another factor, the two factors are said to ‘interact.’
None
B = Low
more than two factors can
interact at the same time,
but it is rare outside of
chemical reactions.
B = High
Low
Factor A
High
22
Reasons Why a Model Might Not Confirm:
•
•
•
•
•
•
Too much variation in the response
Measurement error
Poor experimental discipline
Aliases (confounded) effects
Something changed
- And: -
There may not be a true
cause-and-effect relationship. 
Jim Akers
23
Proof – Storks do bring babies!
• A plot of the population of Oldenburg, Germany
at the end of each year against the number of
storks observed in that year, 1930-1936.
Source: “Statistics for Experimenters” by Box, Hunter, and Hunter. (1978)
Jim Akers
24
Response Surface Method
- 2D Contour Plot
• Useful to see how factors effect the response and to
determine what other settings provide the same
response
Jim Akers
25
Response Surface Method
- 3D Response Surface Plot
• Helpful in reaching the optimal result
Jim Akers
26
CREATING A CHARTER
Jim Akers
27
Planning - DOE Steps
• Set objectives - create a Charter
–
Comparative
• Determine what factor is significant
–
Screening
• Determine what factors will be studied
–
Model – Response Surface Method
• Determine interactions and optimize
•
•
•
•
Select factors (process variables from C&E) and levels you will test
Select an experimental design
Execute the experiment
CONFIRM the model!! Verify the data is consistent with the
experimental assumptions
• Analyze and interpret the results
• Use/present the results
Jim Akers
28
Planning - Charter
• I. STATEMENT OF THE PROBLEM:
__________________________________________
__________________________________________
(During this step you should estimate your current
level of quality by way of Cpk, dpm, or total loss. This
estimate will then be compared with improvements
found after Step XII.)
• II. OBJECTIVE OF THE EXPERIMENT:
__________________________________________
__________________________________________
• III. START DATE: ________ END DATE: ________
Jim Akers
29
Planning - Charter
• IV. SELECT MEASUREABLE QUALITY
CHARACTERISTICS
(also known as responses, dependent variables, or
output variables). These characteristics should be
related to customer needs and expectations.
Jim Akers
30
Planning - Charter
• V. COMPLETE A LITERATURE REVIEW,
PROCESS FLOW DIAGRAM, AND CAUSE &
EFFECT DIAGRAM. FROM THE CAUSE &
EFFECT DIAGRAM SELECT FACTORS (also
known as parameters or input variables) which are
anticipated to have an effect on the response. Write
Standard Operating Procedures for all variables that
are to be held constant.
Jim Akers
31
Planning - Charter
• VI. DETERMINE THE NUMBER OF RESOURCES
TO BE USED IN THE EXPERIMENT. Consider the
desired number, the cost per resource, time per
experimental trial, the maximum allowable number of
resources.
• VII. WHICH DESIGN TYPES AND ANALYSIS
STRATEGIES ARE APPROPRIATE? Discuss
• VIII. SELECT THE BEST DESIGN TYPE AND
ANALYSIS STRATEGY TO SUIT YOUR NEEDS.
Jim Akers
32
Planning - Charter
• IX. CAN ALL THE RUNS BE RANDOMIZED?
____________________________________________
________________________________________
WHICH FACTORS ARE MOST DIFFICULT TO
RANDOMIZE?________________________________
________________________________________
• X. CONDUCT THE EXPERIMENT AND RECORD THE
DATA. (Monitor both of these events for accuracy.)
• XI. ANALYZE THE DATA, DRAW CONCLUSIONS,
MAKE PREDICTIONS, AND PERFORM
CONFIRMATION TESTS.
Jim Akers
33
Planning - Charter
• XII. ASSESS RESULTS, MAKE DECISIONS, AND
(Evaluate your new state of quality and compare with
the quality level prior to the improvement effort.
Estimate your return on investment. If necessary,
conduct more experimentation.)
MS Word Template available at:
Jim Akers
34
CHOOSING AN APPROPRIATE
DESIGN
Jim Akers
35
Choosing an Appropriate Design
Jim Akers
Source: Understanding Industrial Designed Experiments – ISBN 1-880156-03-2
36
2-Level Design Summary
Jim Akers
Source: Understanding Industrial Designed Experiments – ISBN 1-880156-03-2
37
<Function>
<Process Name>
Thank you
Jim Akers
```