Rev.
1/12/01
Section 8 – Statistical Details:
Design Selection
This section of the Design-Ease® software manual provides details on design selection.
You should complete the tutorials before wading into these details.
We will presume that you possess a working knowledge of the “standard” approach to
design of experiments (DOE). If you need background on the subject, read Design and
Analysis of Experiments by Douglas Montgomery. You can buy this textbook from
Stat-Ease or direct from the publisher - John Wiley and Sons, Inc., New York.
If you need a quick education on the important statistics, consider attending one or more
of Stat-Ease’s computer-intensive workshops. Call us to get details on course content
and schedule. You will find contact information at the end of the Introduction.
Design-Ease offers six factorial design types:
• Two-level factorial (2-15 factors)
• Irregular fractions (4-9 factors)
• General factorial (1-12 factors)
• D-optimal design (an option to the full general factorial)(2-14 factors)
• Plackett-Burman (11, 19, 23, 27 or 31 factors)
• Taguchi OA (Orthogonal Arrays for up to 63 factors)
Standard Two-Level Factorials
The 2-Level Factorial selection offers standard two-level full factorial and fractional
factorial designs. You can investigate from 2 to 15 factors in 4, 8, 16, 32, 64, 128 or
256 runs. This collection of designs provides an effective means for screening through
many factors to find the critical few.
Full two-level factorial designs may be run for up to eight factors. These designs permit
estimation of all main effects and all interaction effects (except those confounded with
blocking). Design-Ease offers an option to completely replicate these designs up to 100
times. (Fractional factorials can be replicated also, but it would not make sense to do
so.)
Design-Ease offers 50 fractional factorial designs, ranging from a 1/2 replicate for 3 to 9
factors to a 1/2048 replicate of 15 factors in 16 experiments. The program gives
detailed information on the alias structure, which you can inspect to be sure that you get
clean estimates of desired effects.
Design-Ease 6 User’s Guide Statistical Details – Design Selection • 8-1
You will find the resolution of each fractional factorial by looking at the colors on the 2-
Level Factorial design display. They’re set up like a stoplight.
• Red: Resolution III design. Stop and think. One or more main effects will
be aliased with at least one two-factor interaction. Resolution III designs
can be misleading when two factor interactions significantly affect the
response.
• Yellow: Resolution IV design. Proceed with cauti/n. One or more two-
factor interactions will be aliased with at least one other two factor
interaction. The main effects will be clear of two-factor interactions, so
Resolution IV designs can be a good choice for a screening design.
• Green: Resolution V (or higher) designs. Go ahead. Assuming that no
three-factor (and higher) interactions occur, all the main effects and two-
factor interactions can be estimated. Resolution V designs work very well
for screening. They’re more efficient than full factorials.
The software picks the highest resolution design possible. When several designs of
equivalent resolution are available, Design-Ease defaults to a choice with minimum
aberration in the lengths of the words listed in the defining contrast.
Refer to part one of the Two-Level Factorial Tutorials for detailed illustrations of
several choices from this design option.
Blocking
Design-Ease provides various options for blocking standard two-level factorials,
depending on how many runs you choose to perform, and the number of factors. For
example, in the full factorial experiments with 16 runs, you may choose to carry out the
experiment in 1, 2, 4 or 8 blocks. Keep the default selection of 1 for blocks if you want
no blocking. A selection of 2 for blocks might be particularly helpful if, for some
reason, you must do half the runs on one day and the other half on the next day. In this
case, any day-to-day variation will be removed by blocking.
When you choose to block your design, one or more effects will no longer be estimable.
Design-Ease will tell you which effects, if any, will be “lost to blocks.” The software
picks the highest resolution design possible. When several designs of equivalent
resolution are available, Design-Ease defaults to a choice with minimum aberration in
the lengths of the words listed in the blocking generator(s).
Center Points
A useful extension of two-level factorial and fractional factorial designs incorporates
center points into the factorial structure. If you have at least one numeric factor, you
can choose to add center points to your design. Data from the center points provides:
• Estimates of pure error
• Estimates of curvature.
8-2 • Statistical Details – Design Selection Design-Ease 6 User’s Guide
Rev. 1/12/01
If there is curvature of the response surface in the region of the design, the center point
will be either higher or lower than predicted by the factorial design points. Curvature of
the surface may indicate that the design is in the region of an optimum.
Design-Ease automatically accounts for the presence of center points, constructing the
estimate of pure error, as well as the test for curvature. For factorial designs, Design-
Expert uses the average of the center point values, rather than the polynomial model, to
predict the center point response. This excludes curvature from the Lack-of-Fit test and
the residuals, thus providing more information about the fit of the model. (The software
does a separate test for the curvature.) One benefit of this procedure is that the
assumptions concerning normality and constant variance can be checked even in the
presence of curvature. This allows you to identify problems in the data analysis that
might be otherwise obscured by curvature inflating the residuals.
The effects of curvature and blocking do not appear in the predictive model. However,
for purposes of computing residuals, these effects are including in the predicted values.
If you choose blocking in addition to center points, the number of center points entered
during design selection will be multiplied by the number of blocks. For example, if you
select a design with two blocks and three center points, six center points will be created,
three in each block.
If you include categorical factors in your design, true center points cannot be
constructed. For designs with both categorical and numeric factors, Design-Ease
generates pseudo center points at the centers of the numeric factors for every
combination of the categorical factors. Asking for center points in combination with
categorical factors can produce large (but balanced) designs. For obvious reasons,
Design-Ease will not allow center points if the design contains only categorical factors.
Irregular Fractions
Design-Ease offers a series of Resolution V designs with an irregular fraction of powers
of two. The choices for factors and runs are: 4 in 12, 5 in 24, 6 in 48, 7 in 48, 8 in 48
and 9 in 96. The 4, 5 and 6-factor options are three-quarter replicates of full factorials.
A smaller irregular fraction is provided for the 7 through 9 factor designs. The
construction of these designs comes from Addelman, “Irregular Fractions of the 2n
Factorial Experiments,” Technometrics, 3, 479-496, and also John, “Three-Quarter
Replicates of 24 and 25 Designs,” Biometrics, 17, 319-321. With the exception of the
five-factor option, the irregular fractions allow you to get by with fewer runs than the
Resolution V designs on the standard two-level factorial design builder. For example,
you can study four factors in only 12 runs and detect all the two-factor interactions. A
standard (regular) design of similar quality would require 16 runs.
General Factorial Designs
The General Factorial option allows you to design for 1 to 12 categorical factors with
varying numbers levels of up to 20 each (maximum runs:32766). For example, you can
set up an experiment with three suppliers of four alternative materials to be processed in
two different machines (3 by 4 by 2). If the factor(s) can be varied in a continuous
Design-Ease 6 User’s Guide Statistical Details – Design Selection • 8-3
manner, rather than categorical, consider doing a response surface design instead of a
general factorial (you will need a program such as Design-Expert® to do this). You may
replicate the entire design any number of times, limited only by the memory in your
computer. Experiments can be run completely randomized or blocked. In the
randomized block design each replicate is placed in a different block.
Refer to part one of the General Factorial Tutorials for a detailed illustration of this
design option and its offshoots: split plot and nested.
D-Optimal Factorial Designs
The D-Optimal Factorial is designed for use with categorical factors as an alternative to
the General Factorial design option. The D-optimal criteria selects a set of design
points that minimize the variance associated with the estimates of the coefficients in the
model you specify. For details on optimality criteria see Myers and Montgomery,
Response Surface Methodology. D-Optimal design offers the run-saving equivalent of
fractional factorials. Based on the model that you specify, the algorithm chooses a
desirable subset of runs generated from the full factorial array.
To use this design, first choose how many categorical factors you have. On the next
screens, specify the names and number of levels for each factor. After completing
those, you will reach a screen where you must specify the model that you want to be
able to approximate. The default model is a two-factor interaction model (2FI).
Depending on the number of levels that each factor has even a 2FI model may contain
too many runs. In that case you will need to use your subject matter knowledge to
decide if any of the two-factor interactions are unlikely to occur. If this is true, then
these interactions can be individually eliminated from the model and the number of runs
can be further reduced.
Create the design and carefully look it over to make sure that all the runs generated are
possible to perform. Some runs will be replicated to be able to estimate pure error and
lack of fit.
Plackett-Burman Designs
Plackett-Burman designs use a set of orthogonal contrasts with -1 and +1 coefficients.
Some experimenters use Plackett-Burman designs to check if large effects exist in a
process. But these designs are better suited for ruggedness testing, where you hope to
see little or no effect on the response due to any of the factors. In this latter case, a
process or product that withstands the testing without any significant change would be
declared to be robust. Design-Ease permits Plackett-Burman designs with 12, 20, 24, 28
or 32 experiments. The number of factors cannot exceed the number of runs minus one.
For example, a design with 12 runs will enable you to estimate all main effects for up to
11 factors. This is called a “saturated design.” If you have less than the maximum
allowable factors, the remainder become “dummy factors,” which will be used to
estimate error.
Design-Ease provides alias patterns for all the Plackett-Burman designs. In the 12, 20,
and 24-run designs each main effect gets partially confounded with several two-factor
8-4 • Statistical Details – Design Selection Design-Ease 6 User’s Guide
Rev. 1/12/01
interactions. If any interactions are active in your system, their effect will be attributed
to several main effects, thus creating potential confusion as to which factors are really
important. In unsaturated designs, you may even see dummy factors generating
apparently significant effects. This is not a good sign! Consider augmenting your
design via a foldover, as shown in the last part of the Factorial Tutorials section.
Taguchi Designs
Design-Ease provides a variety of Taguchi designs with differing numbers of factors
and levels. See the Taguchi Design Tutorial for an example. If you are committed to
the Taguchi approach, consider a standard two-level fractional factorial design, which
offers better alias structures.
To use the Taguchi designs, first pick the design you need from the pull-down list.
Then click on Continue and check out the alias structure for this design. It is likely to
be very complex. On the next screen, enter your response names. Then click on
Continue and the design layout will be created.
Design-Ease software sets up saturated Taguchi designs. Use Taguchi’s linear graphs
(available in Taguchi textbooks) to determine which columns you want to use and
which to eliminate. You can enter the names of the factors and the levels by right-
clicking on each factor column header and selecting Edit Info.
Order of Experiments
Design-Ease constructs designs in the standard order used in the referenced textbooks
on design of experiments. To perform the experiments, a random order, called the
“Run” is assigned by the program. Blocking affects this run order. The order will be
randomized within each “Block.” You can change the assigned run numbers, as well as
modify other aspects of the design, in the design layout.
Factor Coding
Design descriptions and analyses are done best with coded factors. Coding sets up
factor levels that can be orthogonal or nearly so. Also, coding reduces the range of each
factor to a common scale, and places the origin of the coordinate system in the center of
the design space. It is easier to work with changes from low to high level for the factors
rather than their actual values, especially when computing interactions. For example,
one factor may vary from 100 to 200 while another varies from 0.1 to 0.5. This large
discrepancy in the range of values makes interpretation of factor coefficients difficult.
It is also important to remember that a regression coefficient tell us how the response
changes relative to the intercept. The origin of the coordinate system (and therefore the
intercept) is in the center of the design space in coded units, but can be far from our data
in the actual units.
Design-Ease applies the standard coding of –1 as the low level of a factor, +1 as the
high level, and 0 as the middle level.
Design-Ease 6 User’s Guide Statistical Details – Design Selection • 8-5
The factor coding is defined by what you enter as the “Low” and “High” value during
design building. These values can be changed by right-clicking on a factor column
header and selecting Edit Info. (Note: Do not attempt to change the low and/or high
values by editing the numbers in the design layout. This will change the values on your
screen, but not the low and high values used to define the coded values. It will result in
non-orthogonal designs and ill-defined regression coefficients. Always use the Edit Info
to change low and high values.)
Design-Ease provides coding for categorical factors at as many as 20 levels. The
general structure is illustrated by the following screen shot for a three-level factor.
Coding for a Three-Level Categorical Factor
Note the warning about editing the contrasts. This can be done only after clicking the
“Make contrasts editable” option. The dialog box also presents a synopsis of how to
interpret the model coefficients for the coded model. Other examples of coding for
general factorials are shown below for four levels and five levels.
Examples of Coding for Four-Level and Five-Level Categorical Factors
To determine coding for greater numbers of levels, you can extrapolate from these
examples, or actually set up such designs on Design-Ease and view the coding via a
right-click on the factor column heading (choose Edit Info).
8-6 • Statistical Details – Design Selection Design-Ease 6 User’s Guide
Rev. 1/12/01
Design Editing
The design layout can be used to change virtually any aspect of your design. You can
change levels; add, delete or duplicate runs; or modify blocking assignments. Factor
names can be edited. You can also name blocks. The following section gives you the
highlights of these and other design editor features, but to really get a feel for what can
be done, there’s no substitute for experimentation. Build your own design and check it
out. Try right mouse clicks on column headings and explore the options.
Order of Experiments
The design layout defaults to run order. You can switch to standard order by right
clicking on either of the column headings or using the View menu. The standard order
labels can be changed and sorted to identification (“ID”) order by right clicking on the
column headings. The ID for replicate runs will be identical, whereas both standard and
run orders must be unique.
Point Type
The various parts of your design, for example, factorial versus center points, can be
identified by doing a right click on the Block column.
Actual vs. Coded Values
The design layout works in coded or actual values, as you wish. Changes made in one
scale are automatically applied to the other scales. All analyses are done in the coded
scale for response surface problems. You can switch between coded and actual values
by using the Options menu.
Botched Design
You can use the design layout to correct badly missed factor levels from a “botched”
design. This feature should not be used to adjust for normal variation of the factors
around their design levels. Make changes only when significant deviations from the
desired factor levels occur. Ideally, you will be able to assign a special cause to the
deviation. In most cases you will be interested in predicting the response as a function
of the designed factor levels, so adjustment will not be desirable. Box, Hunter, and
Hunter provide an interesting case study of a botched factorial in Appendix 14C, page
503, of their book Statistics for Experimenters.
Delete Factors
In most cases, when you run a factorial screening design, some factors will not be
significant. You can delete the insignificant factors by doing a right click on the
unwanted columns and select Delete. Exercise this option with care in fractional
factorial designs, because the factor you delete may be involved in aliased interactions.
Design-Ease 6 User’s Guide Statistical Details – Design Selection • 8-7
Response Editing
Use the design layout to enter responses after completing your experiments. To make
entry simple, sort the experiment in run order, the random order assigned when Design-
Ease sets up the experiment. Right clicking on the response column headings brings up
added options for formatting, changing names, units, etc.
Power Calculations
The Design Evaluation feature in Design-Expert now calculates “power”, which tells
you the probability of finding an effect of a given size (either ½, 1 or 2 standard
deviations) for any specific design. (For other details on design evaluation refer back to
the last part of the section on Advanced Design Features.) In general, power increases
as you add more runs. The sweet spot for power is from .8 to .95 probability. At lower
than .8 power you risk missing effects of interest. On the other hand, it’s wasteful to
invest in more runs than needed for power of .95.
If you are not sure what power can do for you, refer to the short primer on this topic in
the “Handbook for Experiments,” which we send to all registered software users. For
the statistical details, see “Sizing Fixed Effects for Computing Power in Experimental
Designs,” Fall Technical Conference, 2000, by Oehlert and Whitcomb. The following
discussion provides some guidance on how to generate data on power, and how the
program does the calculations, which depends on the chosen design.
The power calculations for two-level factorial designs focus on the effects - the
difference in response means at the high (coded +1) versus low (coded –1). The model
term equals one-half the effect. Power for unreplicated factorials can only be estimated
if you designate at least one term for error. For example, if you set up a full two-level
factorial on three factors and do a design evaluation on the default model, you will see
the following error message for power.
Error Message When Analyzing Unreplicated Factorial for Default Model
In this case you must go back to the Model screen and designate individual terms as
error, or select a lesser model as shown below.
8-8 • Statistical Details – Design Selection Design-Ease 6 User’s Guide
Rev. 1/12/01
Selecting a Different Model with Term(s) Left for Error
Now when you press the Results button you get the following results for power at the
5% alpha level. (Note: go to Edit Preferences if you want to change the significance
threshold from 5% to either 1% or 10%.)
Power Calculated for Two-Level Factorial (Three Factors, ABC as Error)
This design, because it’s so small, is not very powerful. For example, the report shows
only a 17.6% probability (for power) of detecting a two standard deviation change in
any given term. As shown below, the power increases dramatically if you replicate the
design (accomplished by entering 2 in the associated field when re-building the design).
Power Increased by Replication (ABC designated as an error term)
Design-Ease 6 User’s Guide Statistical Details – Design Selection • 8-9
If you try to reproduce this output, remember to designate ABC as error. It’s not
necessary to do this, due to the pure error obtained from replication, but it makes
comparison of the power results fairer.
For general factorial designs, power is defined as the probability of finding a difference
separating the two most extreme terms in a group of terms. For example, see the output
below, where factor A has three levels. Factors B and C are at two levels.
Power Calculation for General Factorial Design
The evaluation was done for the two-factor interaction (2FI) model, with the ABC term
designated for error estimation.
8-10 • Statistical Details – Design Selection Design-Ease 6 User’s Guide