ISO/TR 29901:2007
(Main)Selected illustrations of full factorial experiments with four factors
Selected illustrations of full factorial experiments with four factors
ISO/TR 29901:2007 describes the steps necessary to specify, to use and to analyse full factorial designs with four factors through illustration, with five distinct applications of this methodology. Depending on the application, a different number of factors other than four may be considered in the experiment.
Illustrations choisies de plans d'expérience factoriels complets à quatre facteurs
General Information
Standards Content (Sample)
TECHNICAL ISO/TR
REPORT 29901
First edition
2007-11-01
Selected illustrations of full factorial
experiments with four factors
Illustrations choisies de plans d'expérience factoriels complets à quatre
facteurs
Reference number
ISO/TR 29901:2007(E)
©
ISO 2007
---------------------- Page: 1 ----------------------
ISO/TR 29901:2007(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.
COPYRIGHT PROTECTED DOCUMENT
© ISO 2007
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail copyright@iso.org
Web www.iso.org
Published in Switzerland
ii © ISO 2007 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/TR 29901:2007(E)
Contents Page
Foreword. iv
Introduction . v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions. 2
4 Symbols and abbreviated terms . 4
5 Generic description of full factorial designs . 4
5.1 Overview of the structure of the four factor examples in Annexes A through E. 4
5.2 Overall objective(s) of the experiment. 4
5.3 Response variable(s). 5
5.4 Factors affecting the response(s). 5
5.5 “Full” factorial design . 5
5.6 Analyse the results — Numerical summaries and graphical displays. 6
5.7 Present the results. 7
5.8 Perform confirmation runs . 7
6 Description of Annexes A through E. 7
6.1 Comparing and contrasting the examples. 7
6.2 Experiment summaries . 7
Annex A (informative) Solder bar experiment . 8
Annex B (informative) Direct mail marketing campaign. 17
Annex C (informative) Button tactility experiment. 25
Annex D (informative) Optimizing a customer PVC formulation. 34
Annex E (informative) Genetic algorithms for DNA sequencing experiment. 44
Bibliography . 52
© ISO 2007 – All rights reserved iii
---------------------- Page: 3 ----------------------
ISO/TR 29901:2007(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies
(ISO member bodies). The work of preparing International Standards is normally carried out through ISO
technical committees. Each member body interested in a subject for which a technical committee has been
established has the right to be represented on that committee. International organizations, governmental and
non-governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with the
International Electrotechnical Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of technical committees is to prepare International Standards. Draft International Standards
adopted by the technical committees are circulated to the member bodies for voting. Publication as an
International Standard requires approval by at least 75 % of the member bodies casting a vote.
In exceptional circumstances, when a technical committee has collected data of a different kind from that
which is normally published as an International Standard (“state of the art”, for example), it may decide by a
simple majority vote of its participating members to publish a Technical Report. A Technical Report is entirely
informative in nature and does not have to be reviewed until the data it provides are considered to be no
longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO shall not be held responsible for identifying any or all such patent rights.
ISO/TR 29901 was prepared by Technical Committee ISO/TC 69, Applications of statistical methods.
iv © ISO 2007 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/TR 29901:2007(E)
Introduction
The Six Sigma and international statistical standards communities share a philosophy of continuous
improvement and many analytical tools. The Six Sigma community tends to adopt a pragmatic approach
driven by time and resource constraints. The statistical standards community arrives at rigorous documents
through long-term international consensus. The disparities in time pressures, mathematical rigor and statistical
software usage have inhibited exchanges, synergy and mutual appreciation between the two groups.
4
The present document takes one specific statistical tool (full factorial designs with four factors, 2 designs) and
develops the topic somewhat generically (in the spirit of International Standards) but then illustrates it through
the use of five detailed and distinct applications. The generic description focuses on the commonalities across
4
2 designs. These commonalities hold more generally for arbitrary numbers of factors, but a value of four was
chosen for this Technical Report. The annexes containing the five illustrations follow the basic framework but
also identify the nuances and peculiarities in the specific applications. Each example offers at least one
“wrinkle” to the problem, which is generally the case for real Six Sigma applications. It is thus hoped that
practitioners can identify with at least one of the five examples, if only to remind them of the basic material on
factorial designs that was encountered during their Six Sigma training. Each of the five examples is developed
and analysed using statistical software of current vintage. The explanations throughout are devoid of
mathematical detail — such material can be readily obtained from the many design and analysis of
experiments textbooks available (such as those given in the Bibliography).
© ISO 2007 – All rights reserved v
---------------------- Page: 5 ----------------------
TECHNICAL REPORT ISO/TR 29901:2007(E)
Selected illustrations of full factorial experiments
with four factors
1 Scope
4
This Technical Report describes the steps necessary to specify, to use and to analyse 2 full factorial designs
through illustration, with five distinct applications of this methodology.
Depending on the application, a number of factors other than four may be considered in the experiment.
NOTE 1 Each of these five illustrations is similar in that sufficient resources were available to implement the design.
Other commonalities among the five examples are noted (e.g. study objective, two levels for factors, response variable(s),
factors effecting the response). The individual illustrations have some salient features that are distinct such as
presence/absence of repetitions, centre points, interactions, or different types of response variables. Each illustration
takes place in a different environment such as marketing, software, manufacturing, telecommunications and chemical
processing.
NOTE 2 For the purposes of this Technical Report, the selection of four factors with two levels (aside from centre
points) was made in advance. Furthermore, the detailed use of response surface designs as a follow-up or augmentation
of the existing designs was excluded from this Technical Report, although their use is noted in some of the illustrations.
Likewise, Taguchi designs and blocking designs were not included.
NOTE 3 Full factorial experiments are often employed by individuals (so-called “black belts” or “green belts”)
associated with Six Sigma methods. Six Sigma methods are concerned with problem solving and continuous improvement.
A full factorial experiment with four factors is one of many tools available to Six Sigma practitioners, but hitherto has not
been addressed in detail in ISO International Standards.
2 Normative references
The following referenced documents are indispensable for the application of this document. For dated
references, only the edition cited applies. For undated references, the latest edition of the referenced
document (including any amendments) applies.
ISO 3534-1:2006, Statistics — Vocabulary and symbols — Part 1: General statistical terms and terms used in
probability
ISO 3534-2, Statistics — Vocabulary and symbols — Part 2: Applied statistics
ISO 3534-3:1999, Statistics —Vocabulary and symbols — Part 3: Design of experiments
© ISO 2007 – All rights reserved 1
---------------------- Page: 6 ----------------------
ISO/TR 29901:2007(E)
3 Terms and definitions
For the purposes of this document, the terms and definitions in ISO 3534-1, ISO 3534-2, ISO 3534-3 and the
following apply.
3.1
analysis of variance
ANOVA
technique which subdivides the total variation of a response variable into meaningful components associated
with specific sources of variation
NOTE Adapted from ISO 3534-3:1999, definition 3.4.
3.2
binomial distribution
discrete distribution having the probability mass function
n!
x nx−
PX()==x p(1−p)
xn!( −x)!
where x = 0, 1, …, n and with indexing parameters n = 1, 2, …, and 0 < p <1.
NOTE Adapted from ISO 3534-1:2006, definition 2.46.
3.3
block
collection of experimental units more homogeneous than the full set of experimental units
NOTE Adapted from ISO 3534-3:1999, definition 1.11.
3.4
centre point
vector of factor level settings of the form (a , a , ., a ), where all a equal 0, as notation for the coded levels of
1 2 k i
the factors
NOTE Adapted from ISO 3534-3:1999, definition 1.36.
3.5
design matrix
matrix with rows representing individual treatments (possibly transformed according to the assumed model)
which can be extended by deduced levels of other functions of factor levels (interactions, quadratic terms,
etc.) but are dependent upon the assumed model
NOTE Adapted from ISO 3534-3:1999, definition 2.7.1.
3.6
factor
predictor variable that is varied with the intent of assessing its effect on the response variable
NOTE Adapted from ISO 3534-3:1999, definition 1.5.
3.7
full factorial experiment
experiment consisting of all possible treatments formed from two or more factors, each being studied at two or
more levels
NOTE Adapted from ISO 3534-3:1999, definition 2.1.
2 © ISO 2007 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/TR 29901:2007(E)
3.8
interaction
effect for which the apparent influence of one factor on the response variable depends upon one or more
other factors
NOTE Adapted from ISO 3534-3:1999, definition 1.17.
3.9
level
potential setting, value or assignment of a factor
NOTE Adapted from ISO 3534-3:1999, definition 1.6.
3.10
normal distribution
continuous distribution having the probability density function
2
()x−µ
−
1 2
2σ
fx()= e
σ 2π
where −∞ < x < ∞ and with parameters −∞ < µ < ∞ and σ > 0
NOTE Adapted from ISO 3534-1:2006, definition 2.50.
3.11
predictor variable
variable that can contribute to the explanation of the outcome of an experiment
NOTE Adapted from ISO 3534-3:1999, definition 1.3.
3.12
randomization
process used to assign treatments to experimental units so that each experimental unit has an equal chance
of being assigned a particular treatment
NOTE Adapted from ISO 3534-3:1999, definition 1.29.
3.13
replication
performance of an experiment more than once for a given set of predictor variables
NOTE Adapted from ISO 3534-3:1999, definition 1.27.
3.14
split-plot design
design in which a group of experimental units (plot) to which the same level assigned to the principal factor is
subdivided (split) so as to study one or more additional principal factors within each level of that factor
NOTE Adapted from ISO 3534-3:1999, definition 2.3.6.
© ISO 2007 – All rights reserved 3
---------------------- Page: 8 ----------------------
ISO/TR 29901:2007(E)
4 Symbols and abbreviated terms
The symbols and abbreviated terms used in this Technical Report are as follows:
y Response variable
A, B, C, D Factors
AB, AC, AD, BC, BD, CD 2-way interactions
ABC, ACD, BCD 3-way interactions
ABCD 4-way interactions
+1/−1 High and low settings
4
2 Four factors each with two levels
σ Standard deviation
5 Generic description of full factorial designs
5.1 Overview of the structure of the four factor examples in Annexes A through E
This Technical Report provides general guidelines on the design, conduct and analyses of two-level full
factorial designs and illustrates the steps with five distinct applications given in Annexes A through E. Each of
these five examples follows the basic structure given in Table 1.
The steps given in Table 1 apply to design and analysis of experiments in general, although this Technical
4
Report focuses on 2 full factorial designs. Each of the seven steps is explained in general below. Specific
explanations of the substance of these steps is provided in the examples in Annexes A through E.
Table 1 — Basic steps in experimental design
1 State the overall objective(s) of the experiment
2 Describe the response variable(s)
3 List the factors that might affect the response(s)
4 Select a “full” factorial design
5 Analyse the results – Numerical summaries and graphical displays
6 Present the results
7 Perform a confirmation run
5.2 Overall objective(s) of the experiment
Experiments are conducted for a variety of reasons. The primary motivation for the experiment should be
clearly stated and agreed to by all parties involved in the design, conduct, analysis and implications of the
experimental effort. There may be secondary objectives which could be addressed with the full factorial
experiment.
The ultimate outcome of the experiment could be to take immediate action on factor levels or to obtain a
predictive model, both of which dictate some elements of the analyses.
4 © ISO 2007 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/TR 29901:2007(E)
5.3 Response variable(s)
Associated with the objective of an experiment is a measurable outcome or performance measure. A
response of interest could involve maximization (larger is better), minimization (smaller is better) or meet a
target value (be close to a specified value). The response variable (denoted here by the variable y) should be
intimately (if not directly) related to the objective of the experiment. For some situations, there may be multiple
characteristics of interest to be considered, although there typically is a primary response variable associated
with the experiment. In other cases, multiple responses must be considered; however, for purposes of this
document, a single response is considered in each example.
5.4 Factors affecting the response(s)
The response variable likely depends in some unknown way on a variety of conditions that occur or could be
set in the course of generating a response variable outcome. These conditions are presumed to relate to
controllable factors that may be continuous (temperature, concentration) or discrete (two assembly lines A
4
or B, two vendors, two packaging styles, and so forth). For 2 experiments, we simplify the experimental
design process by selecting two levels for each factor to be varied in the experiment. For discrete factors with
only two possible settings, the levels of this factor are just these two settings. For continuous factors, there is
discretion in choosing the two specific values. In some cases, the two settings could be the historical value
and a proposed value. In other cases, the two settings could be a nominal adjustment from the historical
setting. In any event, the settings should be sufficiently far removed to have an opportunity to reveal an impact
subject to the inherent uncertainty, while not being so disparate that the settings are unreasonable from a
practical, safety or sensibility standpoint. The setting of levels of continuous factors benefits from the domain
expert collaborating on the experiment.
There may be additional factors that could impact that response variable but may be deemed less important
than the chosen four factors or are too difficult or expensive to control. Finally, it is the case that the factors
are to be set independently from each other. It could, however, be discovered that the factors interact (the
setting of one factor impacts how a second factor affects the response).
5.5 “Full” factorial design
This classical design consists of 16 runs obtained by considering all combinations of four factors with two
possible levels. Table 2 provides the basic layout in a standard order for ease of understanding. Each row of
the table represents one set of experimental conditions that when run will produce a value of the response
variable y. The four factors are designated as A, B, C and D. For an individual factor, the level “−1” is the “low”
setting, or one of the two levels if the factor is categorical. The level “+1” is the “high” setting, or the other level
of the categorical factor. The column “y” is a placeholder for the response value once a run has occurred.
In addition to providing the explicit 16 experimental runs to be conducted, Table 2 provides an example of a
randomized run order. The experiment should not be run in the standard order (for example, an increasing
trend in the response could be confused with the effect of factor D on the response). The final two columns
provide abbreviated names for the 16 runs. The first name coincides with the “+” or “−” levels of the factors
given in the order for ABCD. The second naming convention includes a lower case letter for each factor at its
high level. By convention, the four settings at the low level are designated as “(1)” since otherwise the
abbreviation would be null.
© ISO 2007 – All rights reserved 5
---------------------- Page: 10 ----------------------
ISO/TR 29901:2007(E)
4
Table 2 — Layout of a generic 2 full factorial design
Row Name Alternate
A B C D y Run order
number of run name of run
1 −1 −1 −1 −1 y 6 − − − − (1)
1
2 +1 −1 −1 −1 y 14 + − − − a
2
3 −1 +1 −1 −1 y 4 − + − − b
3
4 +1 +1 −1 −1 y 11 + + − − ab
4
5 −1 −1 +1 −1 y 9 − − + − c
5
6 +1 −1 +1 −1 y 2 + − + − ac
6
7 −1 +1 +1 −1 y 3 − + + − bc
7
8 +1 +1 +1 −1 y 1 + + + − abc
8
9 −1 −1 −1 +1 y 8 − − − + d
9
10 +1 −1 −1 +1 y 13 + − − + ad
10
11 −1 +1 −1 +1 y 7 − + − + bd
11
12 +1 +1 −1 +1 y 10 + + − + abd
12
13 −1 −1 +1 +1 y 15 − − + + cd
13
14 +1 −1 +1 +1 y 16 + − + + acd
14
15 −1 +1 +1 +1 y 5 − + + + bcd
15
16 +1 +1 +1 +1 y 12 + + + + abcd
16
In the situation with all four factors being continuous (quantitative), some experimenters elect to run additional
“centre” points. The levels of the centre point are at the midpoint of the two levels of each factor. The inclusion
of centre points facilitates statistical testing of effects, using the variability of the responses at the centre points
as a guideline. It also provides the means to test for non-linearity (curvature) of the response.
In the situation where a particular factor is difficult or expensive to change from one level to the other, the
experiment could be conducted in “blocks”, one block for each level of the difficult factor.
One final situation concerns replication of the runs in the design. Multiple replications at each experimental
setting accommodate estimates of the overall inherent variability which can then be pooled for statistical
testing purposes.
5.6 Analyse the results — Numerical summaries and graphical displays
At the completion of the conduct of the experiment, the y values would be replaced by the actual observed
i
responses. Many existing statistical software packages exist to facilitate the generation of output to aid in the
understanding of the results of the experiment. Of immediate concern is the determination of the impact of the
four factors individually on the response variable. Thus, the main effects for A, B, C and D are to be estimated
as well as the following interaction terms:
⎯ two-way: AB, AC, AD, BC, BD, CD;
⎯ three-way: ABC, ABD, ACD, BCD;
⎯ four-way: ABCD.
6 © ISO 2007 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/TR 29901:2007(E)
In addition to the actual estimates, it is useful to arrange the estimated effects from largest to smallest in the
form of a Pareto Chart. The effects can also be presented on a normal or half-normal plot to identify the
stronger effects. Depending on the inclusion of centre points or replication, the experimental error can be
estimated directly and in turn the standard error of the effects can be determined. In the absence of these
experimental runs, the experimental error can be estimated indirectly by presuming that three-way and four-
way interactions are negligible. Effects plots, interaction plots and residual plots are also generally considered
in assessing the results of an experiment. These are illustrated in Annexes A through E.
5.7 Present the results
Frequently, the purpose of the experiment is to develop a predictive model so as to explore alternate settings
of the factors. Contour plots are available from most software packages. Moreover, with a predictive model,
tentative consideration of optimal settings can be identified. The predictive model will consist of a function
(typically linear) of the estimated effects and possibly interactions. Alternate mathematical models may be
envisaged depending on examination of the residuals (collection of predictive values minus observed values).
5.8 Perform confirmation runs
4
Follow-up experiments may prove useful to demonstrate that the lessons learned from the 2 full factorial
experiments are verified in subsequent runs. A natural follow-on experiment could be to identify a promising
direction in which to adjust the factors for improved performance in the response variable.
6 Description of Annexes A through E
6.1 Comparing and contrasting the examples
4
Five distinct examples of 2 full factorial designs are illustrated in Annexes A to E. Each of these examples
follows the same general template as given in Table 1 and follows a version of the standard design given in
Table 2.
6.2 Experiment summaries
Table 3 summarizes the five examples detailed in the annexes and indicates aspects of the analyses which
were unique to each experiment.
Table 3 — Experiment summaries found by annex
Annex Experiment Problem-specific aspects
Mixture of discrete and continuous factors; repetitions;
A Solder bars
important 2-way interaction
Proportion response variable, standard errors based on
B Direct mail campaign
binomial distribution
C Button tactility Centre points; curvature in response
D PVC formulation optimization Split-plot; centre points; contour plots
Randomized order not necessary; set up for interim analyses;
E Genetic algorithms
replication; dispersion effect
© ISO 2007 – All rights reserved 7
---------------------- Page: 12 ----------------------
ISO/TR 29901:2007(E)
Annex A
(informative)
1)
Solder bar experiment
A.1 General
The material solder can be produced in a number of forms. One of the most common is a bar of solid solder.
The bars are sold by mass, namely 0,5 kg, 1,0 kg and 2,0 kg.
A.2 Overall objective for the experiment
The process had recently been relocated to a new site and subsequently, on the underside of the bars there
were a large number of “rosettes” (small pits on the surface of the solder bar). This led to numerous customer
complaints, customers assuming that the solder was of an inferior standard. The current level of rosettes on
bars was assessed to be about 30 per bar.
The operational staff had tried over several months various “one-factor-at-a-time” experiments, none of which
produced a large enough improvement for the producer to say to their customers that they had solved the
problem. It was decided to design and then run an experiment to understand the factors responsible for the
defect and then how to manage the process so as to minimize the number of rosettes.
A.3 Description of the process
Solder bars are produced by pouring molten solder into moulds pre-selected according to the target mass of
the bars to be produced.
Ingots of solder are firstly melted in an electric furnace at about 290 °C. An operator then produces the bars
by filling a “kettle” with the molten solder and then by pouring the solder from the kettle into moulds. A kettle is
similar to the kitchen utensil of the same name but with extra spouts, thus enabling several bars to be poured
simultaneously.
The bars are cast upside down so the bottom of each bar is uppermost and open to the atmosphere. The
moulds are arranged in banks on a table with water-cooling integrated into it that can be turned on or off.
Before casting, the moulds can be “smoked”. This is a layer of carbon applied with an oxyacetylene torch.
After a short period of time during which the bars solidify, they are de-moulded, stacked and then packed.
A.4 Response variable
A.4.1 Choice of variable
The response chosen for the experiment was the average number of rosettes counted on the surface of the
bars. The objective was to minimize the average number of rosettes.
1) This example has been kindly donated by Cookson Electronics (Fry's Metals Glasgow).
8 © ISO 2007 – All rights reserved
---------------------- Page: 13 ----------------------
ISO/TR 29901:2007(E)
A.4.2 Measurement of the response variable
Approximately 160 bars were produced in each experimental “run”. The number of rosettes was counted
across all of the solder bars and the arithmetic mean of the number of rosettes per bar calculated for each
experimental run. This became the output for that experimental run.
A.4.3 Relationship of the response variable to the objective of the experiment
The mean number of rosettes per bar is directly linked to the objective of the experiment.
A.5 Factors affecting the response
A.5.1 Description of each factor (continuous/discrete) to be varied
The factors chosen to be varied within the experiment were determined using the knowledge of the production
operators and the technical staff. These were:
A: the casting temperature of the molten solder;
B: water-cooling, applied or not;
C: the pouring rate of the molten solder into the moulds; and
D: whether the moulds were smoked or not.
The experiment was to be run using these four factors each at two levels.
Fact
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.