Marco Cavazzuti

Optimization Methods:
From Theory to Design
Scientific and Technological
Aspects in Mechanics

123
Marco Cavazzuti
Dipartimento di Ingegneria ‘‘Enzo Ferrari’’
Università degli Studi di Modena e
Reggio Emilia
Modena
Italy

ISBN 978-3-642-31186-4 ISBN 978-3-642-31187-1 (eBook)

DOI 10.1007/978-3-642-31187-1
Springer Heidelberg New York Dordrecht London

Library of Congress Control Number: 2012942258

! Springer-Verlag Berlin Heidelberg 2013

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or
information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed. Exempted from this legal reservation are brief
excerpts in connection with reviews or scholarly analysis or material supplied specifically for the
purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the
work. Duplication of this publication or parts thereof is permitted only under the provisions of
the Copyright Law of the Publisher’s location, in its current version, and permission for use must always
be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright
Clearance Center. Violations are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt
from the relevant protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media ([Link])

Foreword

There are many books that describe the theory of optimization, there are many
books and scientific journals that contain practical examples of products designed
using optimization techniques, but there are no books that deal with theory having
the application in mind.
This book, written after several years of doctoral studies, is a novelty as it
provide an unbiased overview of ‘‘design optimization’’ technologies with the
necessary theoretical background but also with a pragmatic evaluation of the pros
and cons of the techniques presented.
I’ve been thinking about writing a book like this for years but when I had the
opportunity to read the Ph.D. thesis written by Dr. Cavazzuti I thought that it
would have been far better to encourage the publication of his work: the good
mixture of curiosity, mathematical rigor, and engineering pragmatism was there.
The book will be an invaluable reading for engineering students who could
learn the basis of optimization as it would be for researchers who might get
inspiration. Needless to say that practitioners in industry might benefit as well: in
one book the state of the art of this fascinating and transversal discipline is
summarized.

University of Trieste, Italy, August 2012 Prof. Carlo Poloni

vii
Preface

Over the past few years while studying for my doctorate, many times when
explaining what my research consisted of, the reaction to my saying that I was
‘‘studying the topic of optimization’’, was always the same: ‘‘Optimization of
what?’’. Moreover, it was always accompanied by a puzzled look on the part of the
interlocutor. The first time I was rather surprised by such a question and look then,
as time passed by, I become accustomed to them. In fact, I found it rather amusing
to repeat the same old phrase to different people, irrespective of their age, edu-
cation, social background or culture, and to be able to foresee their reaction and
their answer. On my part, I tried to answer using the simplest words I could find,
avoiding any technicality in order to be understood if possible: ‘‘Well—I replied—
everything and nothing: I am studying the theory of optimization. It is a general
approach, rather mathematical, that you can apply to any problem you like.
In particular I am applying it to some test cases, mainly in the fields of thermo-
dynamics and fluid dynamics’’. However, with an even more puzzled look they
seemed to say: ‘‘Are you kidding me?’’. To my chagrin, I realized I had not been
able to communicate to my listeners any understanding of what I meant. Neither I
had any idea on how to explain things in a simpler way. It seemed optimization
could not constitute a research topic in itself, being necessarily associated to
something more practical. Worse still, it was as if in ‘‘optimization’’ no ‘‘theory’’
was needed since just some common sense was enough, thus, there was nothing to
study! I had the overall impression that most people think that optimizing some-
thing is a sort of handicraft job in which one would take an object, whatever it is,
and with a long build-and-test approach, almost randomly, trying again and again,
so would hopefully manage to improve its working. At other times it seemed to me
that ‘‘optimization’’ and ‘‘design’’ were thought of as incompatible, with the field
of interest of optimization limited to some sort of management issue for industrial
processes.
For my part, I never thought of it in this way when I started my doctorate, these
questions and ideas not even coming to mind when optimization was proposed as

ix
x Preface

research. Probably I was more oriented towards the idea of studying the theory,
perhaps making a contribution to the scientific community in terms of some novel,
and hopefully significant optimization algorithm. But how sound was my reaction
original? Nevertheless, was my reaction the best thing to do? After all, in the world
of optimization theory there are plenty of good algorithms, based on very bright
ideas. Was adding one more to the list what was really needed?
As my research progressed I began to understand what an extremely powerful
instrument optimization was. Despite this, it still had to break out and spread
within the technological and scientific worlds, for it was still not properly
understood. Perhaps the people I had spoken to over the last few years were right,
for even though they may have had a limited turn of mind over the issue, was my
mind any less limited despite my research over the topic? I was still focused on the
mathematical aspects (‘‘theory’’) while they were focused on the practical aspects
(let us call them ‘‘design’’). The fact was that theory and design were too far away
from each other and still had to meet. This was what was missing and what was
worth dealing with in my research: the creation of a link between the theory of
optimization and its practical outworking in design. It had to be shown that such a
link was possible and that optimization could be used in real-life problems.
Optimization can be a very powerful instrument in the hand of the designer and
it is a highly interdisciplinary topic which can be applied to almost any kind of
problem; despite this is still struggling to take off. The aims of this research work
are to show that using optimization techniques for design purpose is indeed viable,
and to try to give some general directions to a hypothetical end user, on how to
adopt an optimization process. The latter is needed mostly because each optimi-
zation algorithm has its own singularities, being perhaps more suitable for
addressing one specific problem rather than another. The work is divided into two
parts. The first, focuses on the theory of optimization and, in places, can become
rather complicated to understand in terms of mathematics. Despite the fact that
these are things which can be found in several books on optimization theory, I
believe that a theoretical overview is essential if we are willing to understand what
we are talking about when we deal with optimization. The second part addresses
some practical applications I investigated over these years. In this part, I essen-
tially try to explain step-by-step the way in which a number of optimization
techniques were applied to some test cases. At the end, some conclusions are
drawn on the methodology to follow in addressing different optimization
problems.
Finally, of course, I come to the acknowledgments. Since I would like to thank
too many people to be able to name them individually, I decided not to explicitly
mention anybody. However, I would like to thank my family, my supervisors and
the colleagues who shared the doctorate adventure with me at the Department of
Mechanical and Civil Engineering of the University of Modena and Reggio Emilia
and during my short stay at the School of Engineering and Design at Brunel
University. A special thanks must be given to all those hundreds of people that,
Preface xi

with puzzled look and without knowing it, helped me day-by-day to better
understand the meaning and the usefulness of optimization. Equal thanks too are
due to the many friends that, with or without that puzzled look, in many different
ways, walked with me along the path of life, and still do!

Fiorano Modenese, Italy, October 2008 Marco Cavazzuti

Summary

Optimization Methods: From Theory to Design Scientific

and Technological Aspects in Mechanics
Many words are spent on optimization nowadays, since it is a powerful instrument
to be applied in design. However, there is the feeling that it is not always well
understood and the focus still remains on creating new algorithms more than on
understanding the way these can be applied to real-life problems.
This book is about optimization techniques and is subdivided into two parts.
In the first part a wide overview on optimization theory is presented. This is
needed for the fact that having knowledge on how the algorithms work is
important in order to understand the way they should be applied, since it is not
always straightforward to setup an optimization problem correctly. Moreover, a
better knowledge of the theory allows the designer to understand which are the
pros and cons of the algorithms, so that he will be able to choose the better ones
depending on the problem at hand.
The optimization theory is introduced and the main ideas in optimization theory
are discussed. Optimization is presented as being composed of five topics, namely:
design of experiments, response surface modelling, deterministic optimization,
stochastic optimization, and robust engineering design. Each chapter, after
presenting the main techniques for each part, draws application-oriented conclu-
sions including didactic examples.
In the second part, some applications are presented to guide the reader through
the process of setting up a few optimization exercises, analyzing critically the
choices which are made step-by-step, and showing how the different topics that
constitute the optimization theory can be used jointly in an optimization process.
The applications which are presented are mainly in the field of thermodynamics
and fluid dynamics due to the author’s background. In particular, we deal with
applications related to heat and mass transfer in natural and in forced convection,
and to Stirling engines. Notwithstanding this, it must be reminded that
optimization is an inherently interdisciplinary and multidisciplinary topic and
the discussion which is made is still valid for other kind of applications.
Summarizing, the idea of the book is to guide the reader towards applications in

xiii
xiv Summary

the optimization field because looking at the literature and at industry there is a
clear feeling that a link is missing and optimization risks to remain a nice theory
but with not many chances for application after all, while instead it would be a
very powerful instrument in industrial design.
This is probably enhanced by the fact that the literature in the field is clearly
divided into various sub-fields of interest (e.g. gradient-based optimization or
stochastic optimization) that are treated as worlds apart and no book or paper has
been found trying to put the things together and give a wider overview over the
topic. This is limiting optimization application to often ineffective one-shot
applications of an algorithm.
It could be argued that the book also discusses many techniques that are not
properly optimization methods in themselves, such as design of experiments and
response surface modelling. However, in the author’s opinion, it is important to
include also these methods since in practice they are very helpful in the
optimization of real-life industrial application. A practical and effective approach
in solving an optimization problem should be an integrated process involving
techniques from different subfields. Every technique has its particular features to
be exploited knowledgeably, and no technique can be self-sufficient.
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 First Steps in Optimization . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Terminology and Aim in Optimization . . . . . . . . . . . . . . . . . 1
1.3 Different Facets in Optimization. . . . . . . . . . . . . . . . . . . . . . 6
1.3.1 Design of Experiments and Response
Surface Modelling . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.2 Optimization Algorithms . . . . . . . . . . . . . . . . . . . . . 7
1.3.3 Robust Design Analysis . . . . . . . . . . . . . . . . . . . . . 8
1.4 Layout of the Book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Part I Optimization Theory

2 Design of Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.1 Introduction to DOE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Terminology in DOE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 DOE Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.1 Randomized Complete Block Design . . . . . . . . . . . . 15
2.3.2 Latin Square . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.3 Full Factorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3.4 Fractional Factorial . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.5 Central Composite . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.6 Box-Behnken . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.7 Plackett-Burman . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.8 Taguchi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.3.9 Random. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.10 Halton, Faure, and Sobol Sequences . . . . . . . . . . . . . 32
2.3.11 Latin Hypercube. . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.3.12 Optimal Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

xv
xvi Contents

3 Response Surface Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.1 Introduction to RSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.2 RSM Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2.1 Least Squares Method . . . . . . . . . . . . . . . . . . . . . . . 44
3.2.2 Optimal RSM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2.3 Shepard and K-Nearest . . . . . . . . . . . . . . . . . . . . . . 50
3.2.4 Kriging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.2.5 Gaussian Processes . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.2.6 Radial Basis Functions . . . . . . . . . . . . . . . . . . . . . . 61
3.2.7 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

4 Deterministic Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.1 Introduction to Deterministic Optimization . . . . . . . . . . . . . . 77
4.2 Introduction to Unconstrained Optimization . . . . . . . . . . . . . . 78
4.2.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.2.2 Line-Search Approach. . . . . . . . . . . . . . . . . . . . . . . 80
4.2.3 Trust Region Approach . . . . . . . . . . . . . . . . . . . . . . 81
4.3 Methods for Unconstrained Optimization. . . . . . . . . . . . . . . . 82
4.3.1 Simplex Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
4.3.2 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.3.3 Quasi-Newton Methods . . . . . . . . . . . . . . . . . . . . . . 85
4.3.4 Conjugate Direction Methods. . . . . . . . . . . . . . . . . . 87
4.3.5 Levenberg–Marquardt Methods . . . . . . . . . . . . . . . . 89
4.4 Introduction to Constrained Optimization . . . . . . . . . . . . . . . 90
4.4.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.4.2 Minimality Conditions . . . . . . . . . . . . . . . . . . . . . . 92
4.5 Methods for Constrained Optimization . . . . . . . . . . . . . . . . . 93
4.5.1 Elimination Methods. . . . . . . . . . . . . . . . . . . . . . . . 93
4.5.2 Lagrangian Methods . . . . . . . . . . . . . . . . . . . . . . . . 94
4.5.3 Active Set Methods . . . . . . . . . . . . . . . . . . . . . . . . 95
4.5.4 Penalty and Barrier Function Methods . . . . . . . . . . . 96
4.5.5 Sequential Quadratic Programming . . . . . . . . . . . . . . 97
4.5.6 Mixed Integer Programming . . . . . . . . . . . . . . . . . . 97
4.5.7 NLPQLP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

5 Stochastic Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

5.1 Introduction to Stochastic Optimization. . . . . . . . . . . . . . . . . 103
5.1.1 Multi-Objective Optimization. . . . . . . . . . . . . . . . . . 105
5.2 Methods for Stochastic Optimization . . . . . . . . . . . . . . . . . . 107
5.2.1 Simulated Annealing. . . . . . . . . . . . . . . . . . . . . . . . 107
5.2.2 Particle Swarm Optimization . . . . . . . . . . . . . . . . . . 110
5.2.3 Game Theory Optimization . . . . . . . . . . . . . . . . . . . 113
Contents xvii

5.2.4 Evolutionary Algorithms . . . . . . . . . . . . . . . . . . . . . 116

5.2.5 Genetic Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . 121
5.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

6 Robust Design Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

6.1 Introduction to RDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
6.1.1 MORDO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
6.1.2 RA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.2 Methods for RA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
6.2.1 Monte Carlo Simulation . . . . . . . . . . . . . . . . . . . . . 135
6.2.2 First Order Reliability Method . . . . . . . . . . . . . . . . . 136
6.2.3 Second Order Reliability Method . . . . . . . . . . . . . . . 137
6.2.4 Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . 137
6.2.5 Transformed Importance and Axis
Orthogonal Sampling . . . . . . . . . .............. 139
6.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . .............. 141

Part II Applications

7 General Guidelines: How to Proceed

in an Optimization Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.2 Optimization Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.2.1 Design of Experiments . . . . . . . . . . . . . . . . . . . . . . 149
7.2.2 Response Surface Modelling . . . . . . . . . . . . . . . . . . 149
7.2.3 Stochastic Optimization. . . . . . . . . . . . . . . . . . . . . . 150
7.2.4 Deterministic Optimization . . . . . . . . . . . . . . . . . . . 151
7.2.5 Robust Design Analysis . . . . . . . . . . . . . . . . . . . . . 152

8 A Forced Convection Application: Surface Optimization

for Enhanced Heat Transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.2 The Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
8.3 Methodological Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
8.3.1 Experiments Versus Simulations. . . . . . . . . . . . . . . . 160
8.3.2 Objectives of the Optimization. . . . . . . . . . . . . . . . . 160
8.3.3 Input Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
8.3.4 Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
8.3.5 The Chosen Optimization Process . . . . . . . . . . . . . . 165
8.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
8.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
xviii Contents

9 A Natural Convection Application: Optimization

of Rib Roughened Chimneys . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
9.2 The Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
9.3 Methodological Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
9.3.1 Experiments Versus Simulations. . . . . . . . . . . . . . . . 179
9.3.2 Objectives of the Optimization. . . . . . . . . . . . . . . . . 179
9.3.3 Input Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
9.3.4 Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
9.3.5 The Chosen Optimization Process . . . . . . . . . . . . . . 183
9.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
9.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

10 An Analytical Application: Optimization of a Stirling Engine

Based on the Schmidt Analysis and on the Adiabatic Analysis . . . 195
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
10.1.1 The Stirling Thermodynamic Cycle . . . . . . . . . . . . . 196
10.1.2 The Schmidt Analysis . . . . . . . . . . . . . . . . . . . . . . . 197
10.1.3 The Adiabatic Analysis . . . . . . . . . . . . . . . . . . . . . . 200
10.2 The Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
10.3 Methodological Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
10.3.1 Experiments Versus Simulations. . . . . . . . . . . . . . . . 206
10.3.2 Objectives of the Optimization. . . . . . . . . . . . . . . . . 206
10.3.3 Input Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
10.3.4 Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
10.3.5 The Chosen Optimization Process . . . . . . . . . . . . . . 210
10.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
10.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

11 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
11.1 What Would be the Best Thing to do? . . . . . . . . . . . . . . . . . 225
11.2 Design of Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
11.3 Response Surface Modelling . . . . . . . . . . . . . . . . . . . . . . . . 228
11.4 Stochastic Optimization. . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
11.5 Deterministic Optimization . . . . . . . . . . . . . . . . . . . . . . . . . 229
11.6 Robust Design Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
11.7 Final Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

Apendix A: Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Chapter 1
Introduction

If you optimize everything you will always be unhappy.

Donald Ervin Knuth

1.1 First Steps in Optimization

Optimization is a very powerful and versatile instrument which could potentially be

applied to any engineering discipline, although it still remains rather unknown both in
the technological and in the scientific fields. It is true that the topic is not particularly
simple in itself and that the newcomer at first will probably get lost among the many
existing techniques, together with their tweaks, and will be disoriented among the
discrepancies between different sources of information over the same issue.
Moreover, although many books have been written on optimization, they always
focus on a limited view of the topic, between them the terminology is not always
clear and uniform, and they usually remain highly theoretical and lack in addressing
practical examples and other aspects an end user, which may not be so theoretically
skilled, would probably like to know.
Including everything in a text dedicated to optimization, from a deep and full
theoretical treatment to a wide discussion on how to apply the theory into practice,
probably would be asking too much. In this book, a wide and general theoretical
view is given in the first part; some applications are then discussed. The objective is
to give clues to the newcomers on how to move their steps when entering the world
of optimization, bearing in mind that different methods have different characteristics
suitable for different classes of problems, and different users may have different goals
which could affect the “optimal” approach to an optimization problem. For example,
in technological applications reaching a solution quickly is commonly crucial, while
in the scientific field precision is more important.

M. Cavazzuti, Optimization Methods: From Theory to Design, 1

DOI: 10.1007/978-3-642-31187-1_1, © Springer-Verlag Berlin Heidelberg 2013
2 1 Introduction

1.2 Terminology and Aim in Optimization

In order to clarify the meaning and the aim of optimization from a technical point
of view, and the way some terms are used throughout the text, a few definitions are
needed. This is even more necessary since the terminology used in this field is not
fully standardized, or maybe, at times is a bit messed up because it is not always fully
understood. Starting from a general definition of optimization, the english Oxford
dictionary [1] says that optimization is
the action or process of making the best of something; (also) the action or process of rendering
optimal; the state or condition of being optimal.

WordReference online dictionary [2] adds that

in an optimization problem we seek values of the variables that lead to an optimal value of
the function that is to be optimized.

First of all we have to identify the object of the optimization, giving an identity to
the “something” cited in the first definition: we will refer to it as the problem to be
optimized, or optimization problem.
According to the second definition, we need to address the variables influencing
the optimization problem. Therefore, some sort of parameterization is required. We
seek a set of input parameters which are able to fully characterize the problem from
the design point of view.
The set of input parameters can be taken as the set of input variables, or vari-
ables, of the problem. However, it must be kept in mind that the complexity of an
optimization problem grows exponentially with the number of variables. Thus, the
number of variables has to be kept as low as possible and a preliminary study to asses
which are the most important ones could be valuable. In this case the set of the input
variables can be a subset of the input parameters. A variable is considered important
if its variations can affect significantly the performance measure of the problem.
If we look at the n variables of a problem as a n-dimensional Euclidean geometrical
space, a set of input variables can be represented as a dot in the space. We call the dot
sample and the n-dimensional space the samples belong to design space or domain
of the optimization problem.
Once the problem and its input variables are defined, a way of evaluating the
performance of the problem for a given sample is needed. What it is sought is,
essentially, a link between the input variables and a performance measure. The link
can be either experimental or numerical and we will refer to it as the experiment or
simulation.
From the experiment, or from the post-processing of the numerical simulation,
information about the problem can be collected: we will call this output informa-
tion output parameters. Obviously, the output parameters are functions, through the
experiment or the simulation, of the input variables.
The performance measure is called objective function, or simply objective and
the range of its possible values is the solution space. In the most simple case the
objective to be optimized can be one of the output parameters. Otherwise it can be
 Part I
Optimization Theory
Chapter 2
Design of Experiments

All life is an experiment.

The more experiments you make the better.
Ralph Waldo Emerson,
Journals

2.1 Introduction to DOE

Within the theory of optimization, an experiment is a series of tests in which the

input variables are changed according to a given rule in order to identify the reasons
for the changes in the output response. According to Montgomery [4]
“Experiments are performed in almost any field of enquiry and are used to study the perfor-
mance of processes and systems. […] The process is a combination of machines, methods,
people and other resources that transforms some input into an output that has one or more
observable responses. Some of the process variables are controllable, whereas other vari-
ables are uncontrollable, although they may be controllable for the purpose of a test. The
objectives of the experiment include: determining which variables are most influential on the
response, determining where to set the influential controllable variables so that the response
is almost always near the desired optimal value, so that the variability in the response is
small, so that the effect of uncontrollable variables are minimized.”

Thus, the purpose of experiments is essentially optimization and RDA. DOE, or

experimental design, is the name given to the techniques used for guiding the choice
of the experiments to be performed in an efficient way.
Usually, data subject to experimental error (noise) are involved, and the results
can be significantly affected by noise. Thus, it is better to analyze the data with appro-
priate statistical methods. The basic principles of statistical methods in experimental
design are replication, randomization, and blocking. Replication is the repetition
of the experiment in order to obtain a more precise result (sample mean value)
and to estimate the experimental error (sample standard deviation). Randomization

M. Cavazzuti, Optimization Methods: From Theory to Design, 13

DOI: 10.1007/978-3-642-31187-1_2, © Springer-Verlag Berlin Heidelberg 2013
14 2 Design of Experiments

refers to the random order in which the runs of the experiment are to be performed.
In this way, the conditions in one run neither depend on the conditions of the previous
run nor predict the conditions in the subsequent runs. Blocking aims at isolating a
known systematic bias effect and prevent it from obscuring the main effects [5]. This
is achieved by arranging the experiments in groups that are similar to one another.
In this way, the sources of variability are reduced and the precision is improved.
Attention to the statistical issue is generally unnecessary when using numerical
simulations in place of experiments, unless it is intended as a way of assessing the
influence the noise factors will have in operation, as it is done in MORDO analysis.
Due to the close link between statistics and DOE, it is quite common to find in
literature terms like statistical experimental design, or statistical DOE. However,
since the aim of this chapter is to present some DOE techniques as a mean for
collecting data to be used in RSM, we will not enter too deeply in the statistics
which lies underneath the topic, since this would require a huge amount of work to
be discussed.
Statistical experimental design, together with the basic ideas underlying DOE,
was born in the 1920s from the work of Sir Ronald Aylmer Fisher [6]. Fisher was the
statistician who created the foundations for modern statistical science. The second era
for statistical experimental design began in 1951 with the work of Box and Wilson [7]
who applied the idea to industrial experiments and developed the RSM. The work
of Genichi Taguchi in the 1980s [8], despite having been very controversial, had a
significant impact in making statistical experimental design popular and stressed the
importance it can have in terms of quality improvement.

2.2 Terminology in DOE

In order to perform a DOE it is necessary to define the problem and choose the
variables, which are called factors or parameters by the experimental designer.
A design space, or region of interest, must be defined, that is, a range of variability
must be set for each variable. The number of values the variables can assume in
DOE is restricted and generally small. Therefore, we can deal either with qualitative
discrete variables, or quantitative discrete variables. Quantitative continuous vari-
ables are discretized within their range. At first there is no knowledge on the solution
space, and it may happen that the region of interest excludes the optimum design. If
this is compatible with design requirements, the region of interest can be adjusted
later on, as soon as the wrongness of the choice is perceived. The DOE technique
and the number of levels are to be selected according to the number of experiments
which can be afforded. By the term levels we mean the number of different values a
variable can assume according to its discretization. The number of levels usually is
the same for all variables, however some DOE techniques allow the differentiation
of the number of levels for each variable. In experimental design, the objective func-
tion and the set of the experiments to be performed are called response variable and
sample space respectively.
Chapter 3
Response Surface Modelling

E ancora che la natura cominci dalla ragione e termini

nella sperienzia, a noi bisogna seguitare il contrario,
cioè cominciando dalla sperienzia, e con quella
investigare la ragione.
Although nature commences with reason and ends in experience,
it is necessary for us to do the opposite, that is to commence with
experience, and from this to proceed to investigate the reason.
Leonardo da Vinci

3.1 Introduction to RSM

Response surface modelling, or response surface methodology, is strictly related

to DOE. The main idea is to use the results of a DOE run in order to create an
approximation of the response variable over the design space. The approximation is
called response surface or meta-model and can be built for any output parameter. The
reason for building a response surface is that, although it is just an approximation,
it can be used to estimate the set of input parameters yielding an optimal response.
The response surface is an analytical function, thus an optimization based on such
a model is very fast and does not require additional experiments or simulations to
be performed. Therefore, the use of meta-models can be very advantageous, and
can be applied even when little is known about the problem, although it must be
kept in mind that if the design space exploration (made with the DOE or the RSM
model adopted) is poor, and the response variable is particularly irregular, the result
of the meta-model-assisted optimization can be far from the truth because of the bad
estimation of the model coefficients or the choice of an unsuitable model.

M. Cavazzuti, Optimization Methods: From Theory to Design, 43

DOI: 10.1007/978-3-642-31187-1_3, © Springer-Verlag Berlin Heidelberg 2013
44 3 Response Surface Modelling

Recalling Eq. 1.1, the objective function, or response variable, y is an unknown

function of the input parameters x. The response surface ŷ is an approximation of
this function
y = f (x) = fˆ (x) + ϵ (x) =⇒ ŷ = fˆ (x) (3.1)

where ϵ (x) is the error in the estimated response.

The outcome of a DOE made of N experiments consists in N (xi , yi ) couples in
which to a point xi in the design space is associated the result of the experiment yi .
The response surface is said to be interpolating if for each DOE sample point yi =
fˆ (xi ) holds, or approximating if ϵ (xi ) ̸= 0.
To help visualize the shape of a response surface contour plots are often used. In
contour plots, lines of constant response are drawn in the plane made by two of the
parameters.

3.2 RSM Techniques

RSM was firstly introduced by Box and Wilson in 1951 [7] who suggested the use
a first-degree polynomial model for approximating a response variable. Since then,
many RSM techniques have been developed. Some of the most common ones are
presented in this section.

3.2.1 Least Squares Method

Least squares method (LSM) is used to solve overdetermined systems and it can be
interpreted as a method for data fitting. The method was developed by Gauss around
1795 and published several years later [24]. It consists of adjusting the coefficients
of a model function (the response surface) so that it best fits a data set (the results of
a DOE run).
The model function is a function fˆ (x, β), where β = [β1 , . . . , βm ]T is the vector
of the m coefficients to be tuned and x = [x1 , . . . , xk ]T is the vector of the k input
parameters. The data set consists in (xi , yi ) pairs, i = 1, . . . , N , where xi is the
vector of the input parameters of the ith experiment, whose response variable is yi .
What is meant by saying best fit can be defined in different ways, the LSM looks
for the choice of the β j , j = 1, . . . , m coefficients giving the minimum sum S of
squared residuals at the points in the data set

N
!
S= ϵi2 . (3.2)
i=1

The residuals are the difference between the experimental responses and the value
predicted by the model function at the locations xi in the design space
Chapter 4
Deterministic Optimization

Minima maxima sunt.

The smallest things are most important.

4.1 Introduction to Deterministic Optimization

Deterministic optimization, or mathematical programming, is the classical branch of

optimization algorithms in mathematics. It embodies algorithms which rely heavily
on linear algebra since they are commonly based on the computation of the gradient,
and in some cases also of the Hessian, of the response variables. Obviously, deter-
ministic optimization has both advantages and drawbacks. A remarkable advantage
is that the convergence to a solution is much faster when compared to the use of
stochastic optimization algorithms. With “faster” we mean that it requires a lower
number of evaluations of the response variable, or function evaluations, to reach the
solution. A function evaluation involves an experiment or a simulation to be per-
formed, therefore the number of estimates required by an optimization algorithm
to reach a solution is a measure of the time required by the optimization process
itself. Being based on a rigorous mathematical formulation not involving stochastic
elements, the results of a deterministic optimization process are unequivocable and
replicable. However, this could be true also for a stochastic optimization in that the
randomization process is pseudo-random and is usually driven by a random seed gen-
erator. On the other hand, deterministic optimization algorithms look for stationary
points in the response variable, thus, the optimal solution eventually found could be
a local optimum and not the global optimum. Moreover, deterministic algorithms are
intrinsically single objective.

M. Cavazzuti, Optimization Methods: From Theory to Design, 77

DOI: 10.1007/978-3-642-31187-1_4, © Springer-Verlag Berlin Heidelberg 2013
78 4 Deterministic Optimization

Whether the method is deterministic or stochastic, some elements are needed

to set up an optimization process. In particular a feasible sample, or a group of
feasible samples, to start from, and a stopping criterion must be chosen. The different
evolution we have from the initial samples to the solution of the optimization process
depends on the algorithm. By feasible sample we mean an assignment to each input
variable so that all the constraints of the problem are satisfied. Finding a feasible
sample for highly constrained problems could be a challenging task. Suitable methods
exist for accomplishing this task which are known as constraint satisfaction problem
(CSP) algorithms. A CSP is solved by heuristic methods and can be considered a
sort of optimization problem itself. However, highly constrained problems are not
commonly found in practice and we will not enter into the details of CSP.
In this chapter we will discuss the two main aspects of deterministic optimization,
namely: unconstrained optimization and constrained optimization. Conclusions will
be drawn in the end, to drive the choice of the most suitable algorithm depending on
the problem at hand. For a deeper and exhaustive insight on the topic we address the
reader to the work of Fletcher [40] from which most of the theoretical part of this
chapter takes cue.

4.2 Introduction to Unconstrained Optimization

4.2.1 Terminology

An unconstrained optimization! (n)algorithm

" generally starts from a point x(1) and gen-
erates a sequence of points x in the design space converging to the solution x∗ .
We call line the set of points

x (α) = x′ + αs, ∀α ∈ R (4.1)

where x′ is a point in the design space and s a direction. We assume that the response
variable y = f (x) is sufficiently smooth (class C 1 or C 2 , whether we need to
compute gradients or Hessians). By function of class C m we mean a function which
is continuous, derivable, and with continuous derivatives up to the order m. The
Hessian matrix is the square matrix of second-order partial derivatives of a function,
thus, to be determined unambiguously in each point of the domain, it requires the
function to be of class C 2 .
By the chain rule the derivatives (slope and curvature) of the response variable
along any line, assuming ∥s∥ = 1, are

d f (x) #k dx (α) ∂ f (x )
i i
=
dα i=1 dα ∂xi
k
# ∂ f (xi )
= si = sT ∇ f (x) = ∇ f (x)T s = g (x)T s (4.2)
i=1 ∂xi
Chapter 5
Stochastic Optimization

For rarely are sons similar to their fathers:

most are worse, and a few are better than their fathers.
Homer,
The Odyssey

5.1 Introduction to Stochastic Optimization

Stochastic optimization includes the optimization methods in which randomness

is present in the search procedure. This is quite a general definition for stochastic
optimization, since randomness can be included in many different ways. Stochastic
optimization algorithms could be classified into different families, to cite a few:
• Simulated Annealing (SA) [59]: aims at emulating the annealing heat treatment
process of steel.
• Particle Swarm Optimization (PSO) [60,61]: aims at emulating the social behav-
iour of birds flocking.
• Game Theory-based optimization (GT) [62]: aims at emulating the evolution of a
game in which different players try to fulfil their own objectives. They are based
on the game theory from Nash [63].
• Evolutionary Algorithms (EA) [64, 65]: aim at emulating the evolution of species
by natural selection according to Darwin’s theory [3]. It is the most important
category of stochastic optimization together with genetic algorithms.
• Genetic Algorithms (GA) [66]: alike EAs, aim at emulating the evolution of
species. For this reason at times they are considered a subcategory of EAs.
However, EAs and GAs, in practice, have different approaches to the emulation
of evolution and can also be considered two different categories of stochastic
optimization algorithms.

M. Cavazzuti, Optimization Methods: From Theory to Design, 103

DOI: 10.1007/978-3-642-31187-1_5, © Springer-Verlag Berlin Heidelberg 2013
104 5 Stochastic Optimization

These fanciful descriptions may at first struck the reader for their strangeness. The
source of inspiration of many randomized search methods comes from the obser-
vation of nature. Concepts from biology, physics, geology, or some other field of
investigation, are borrowed and implemented in a simplified model of some natural
phenomena. Most of these methods are population-based algorithms, in which a set
of initial samples evolves (or moves) up to convergence. The rules of the evolution,
which always include some randomness factor, depend on the natural model embod-
ied. Population-based algorithms are also known as Swarm Intelligence (SI) when
they mimic the collective behaviour of self-organized natural systems. Commonly,
the collective behaviour which is mimicked is taken from the animal kingdom: herd-
ing, flocking, shoaling and schooling, swarming, hunting, foraging, feeding. On the
wake of this, we may find many optimization algorithms such as: ant colony opti-
mization, honey bee colony optimization, glowworm swarm optimization, but also
river formation dynamics, stochastic diffusion search, gravitational search algorithm,
charged system search, and so on. A leading role, at least from a chronological point of
view, in stochastic optimization has to be acknowledged to evolutionary and genetic
algorithms which opened the door to the other nature-mimicking methods, and still
are among the most well-known and applied ones. The main strength of SI, and of
stochastic optimization in general, is the ability of the algorithms to overcome local
minima and explore the design space thanks to the role of randomness, and to the
level of interaction among the individuals in the swarm and between the individuals
and their environment. The tricky part in these algorithms is the balancing between
the need of exploring the design space for improving the algorithm robustness, and
the need of converging to a solution within a reasonable amount of time. The tuning
is achieved by setting some control parameters. It must be noted that the choice of
the control parameters can have a remarkable influence on the global behaviour of
the algorithm, and this is often forgotten when claiming the good features of an algo-
rithm over another. We must consider that the parallelism between the natural world
and stochastic optimization algorithms in general is limited to just a few aspects of
reality and even though the algorithms are somewhat inspired by nature, the numeri-
cal model is often a rather freely-adapted simplification of the natural world created
for the purpose of solving an optimization problem through some evolution-based
scheme. Moreover the behaviour of the algorithm also depends by the environment in
which the algorithm is applied (that is, the optimization problem at hand) for which
a detailed parallelism with the complexity of the natural world is often unfitting.
In this chapter we will introduce the reader to how these curious ideas for devel-
oping stochastic optimization algorithms have been effectively put into practice for
optimization purposes. Stochastic optimization methods are the most innovative and
advanced approaches to optimization. Compared to deterministic optimization meth-
ods, they have both advantages and drawbacks:
• they are less mathematically complicated,
• contain randomness in the search procedure,
• have a much slower convergence towards the optimum solution,
Chapter 6
Robust Design Analysis

I forgot perfection a long time ago;

I just hope something’s going to work
somehow, sometime for someone, somewhere.
Michael W. Collins

6.1 Introduction to RDA

In Chap. 5 the term robustness was referred to the ability of a stochastic optimization
method to investigate the design space reaching the global minimum design point
without getting stuck in local minima. In Chap. 4 the term reliability was referred to
the fact that a certain optimization method was unlikely to diverge, failing to find a
solution.
These concepts are different in Robust Design Analysis (RDA). RDA can be con-
sidered a step further in optimization whose aim is not just to find an optimum
solution, but also to evaluate the ability of the solution not to deteriorate its per-
formance as noise (also referred to as uncertainty) is added to the input variables.
This is an important issue since an optimum design is not a desirable solution if its
performance changes abruptly as it is displaced slightly in the design space. From
this perspective robustness, reliability, and quality are almost synonyms and refer to
this ability.
A design is said to be robust if it is capable of coping well with variations in its
operating environment with minimal damage, alteration, or loss of functionality. In
statistics, reliability is the consistency, not necessarily the accuracy, of a set of mea-
surements and is inversely related to the random error. Reliability is often reported
in terms of probability.
Quality is a widely discussed topic in industry nowadays, yet at times its meaning
is vague. Different definitions have been given for quality, to cite a few: fitness

M. Cavazzuti, Optimization Methods: From Theory to Design, 131

DOI: 10.1007/978-3-642-31187-1_6, © Springer-Verlag Berlin Heidelberg 2013
132 6 Robust Design Analysis

for use [89], conformance to requirements [90], the result of care [91], degree to
which a set of inherent characteristics fulfils requirements [92], number of defects
per million opportunities [93]. Quality assurance procedures are now regulated by
ISO standards. ISO 9000:2000 bases quality assurance mainly on the checking of
the finished product. In its more recent evolution, ISO 9000:2005, the standard has
moved to a fully integrated approach ensuring the quality from checking the whole
industrial process.
In practice, using RDA, we wish to evaluate in which way small changes in the
design parameters and operating conditions are reflected in the objective function.
The noise stands for
• errors which could be made during the manufacturing of an object (tolerance),
• the deterioration of an object with use which causes the design point and perfor-
mance to change (wear),
• the fact that an object does not operate according to the requirements it was
designed for (operating conditions),
• everything else that may occur and it is not possible to keep under control (external
factors).
Robust design is the management of the uncertainties [94], and uncertainties are
potential deficiencies due to lack of knowledge [95]. The reason for performing RDA
is that traditional optimization techniques tend to over-optimize, finding solutions
that perform well at the design point but have poor off-design characteristics.
From a mathematical point of view an objective function subject to uncertainties
is in the form
f (x, y) : X × Y → R (6.1)

where X is the design space and Y the space of the noise variables, x ∈ X is a
design point and y ∈ Y is a noise vector. Two different approaches to RDA are
possible, namely: Multi-Objective Robust Design Optimization (MORDO) [14], and
Reliability Analysis (RA) [27].

6.1.1 MORDO

The basic idea of MORDO is to transform a generic l objectives, k variables opti-

mization problem

minimize f (x) , x ∈ Rk
(6.2)
subject to c (x) = 0

where f : Rk → Rl , and c (x) are the equality constraint functions, into a 2 l

objectives optimization problem
 Part II
Applications
Chapter 7
General Guidelines: How to Proceed
in an Optimization Exercise

Est modus in rebus.

There is a measure in everything.
Horace,
The first book of the Satires

7.1 Introduction

In the second part of the book we discuss a few optimization applications. In each
chapter, a case is presented, and the methodological aspects are focused through
which we coped with the problem. The results are briefly presented and conclusions
on the methods adopted are drawn. For more information on the scientific aspects and
the results obtained we cross-refer to the papers the author has published in journals
or conference proceedings.
In this chapter, a general discussion is made over the optimization methods seen
in the first part, and a methodology on how to proceed in an optimization problem is
given. The methodology comes from the author’s experience and is not necessarily
the only possible approach to optimization nor the best. However, it is a general
approach taking into consideration the many facets of optimization theory and we
believe that an engineering problem following this guidelines is well-posed.

7.2 Optimization Methods

The range of the possible choices, putting together all the elements seen in the first
part of the book, is extremely wide. Formally, citing only the methods which have
been discussed, we could choose any, or any combination of:

M. Cavazzuti, Optimization Methods: From Theory to Design, 147

DOI: 10.1007/978-3-642-31187-1_7, © Springer-Verlag Berlin Heidelberg 2013
148 7 General Guidelines: How to Proceed in an Optimization Exercise

• Design of Experiments (Chap. 2):

Randomized Complete Block Design, Latin Square, Graeco-Latin Square, Hyper-
Graeco-Latin Square, Full Factorial, Fractional Factorial, Central Composite Cir-
cumscribed, Central Composite Faced, Central Composite Inscribed, Central
Composite Scaled, Box-Behnken, Plackett-Burman, Taguchi, Random, Halton,
Faure, Sobol, Latin Hypercube, Optimal Design.
• Response Surface Modelling (Chap. 3):
Least Squares Method, Optimal Response Surface Modelling, Shepard, Kriging
Nearest, Mollifier Shepard, Kriging, Gaussian Process, Radial Basis Functions,
Neural Networks.
• Stochastic Optimization (Chap. 5):
Simulated Annealing, Particle Swarm Optimization, Game Theory Optimization,
Evolutionary Algorithms, Genetic Algorithms.
• Deterministic Optimization (Chap. 4):
Spendley Simplex, Nelder and Mead Simplex, Newton, Steepest Descent, DFP
Quasi-Newton, BFGS Quasi-Newton, Broyden Quasi-Newton, Conjugate Gradi-
ents, Direction Set, Levenberg-Marquardt, Penalty Functions, Barrier Functions,
Sequential Quadratic Programming, Mixed Integer Programming, NLPQLP.
• Robust Design Analysis (Chap. 6):
Monte Carlo Multi-Objective Robust Design Optimization, Latin Hypercube
Multi-Objective Robust Design Optimization, Monte Carlo Sampling Reliability
Analysis, Latin Hypercube Sampling Reliability Analysis, First Order Reliability
Method, Second Order Reliability Method, Importance Sampling Monte Carlo,
Simple Importance Latin Hypercube Sampling, Transformed Importance Latin
Hypercube Sampling, Axis Orthogonal Importance Sampling Monte Carlo, Axis
Orthogonal Latin Hypercube Sampling.
For instance, we could start from a design of experiments followed by a response
surface modelling (DOE+RSM) in order to investigate the design space and refine
the search to a smaller design space to be used with a multi-objective optimization
algorithm, or to create a metamodel to speed up the optimization process. A MORDO
could also have been performed, or a RA added at the end of the optimization process.
There is no optimum choice, although having a certain knowledge on the methods
helps in making reasonable choices. The various elements in the list can be used
on their own, or a selection of methods can be used in cascade. An hypothetical
“complete” optimization process, taking into consideration at least one element in
each category, would be desirable, however it is often too expensive and troublesome
to be performed. The categories can be thought of as different modules which can
be used on their own, or arranged together, to build an optimization process. In case
the elements are arranged together, it is not necessary that each of them is present in
the process, and some can also be missing. Figure 7.1 summarizes the elements of
an optimization process.
Chapter 8
A Forced Convection Application: Surface
Optimization for Enhanced Heat Transfer

Test everything,
retain what is good.
St. Paul of Tarsus,
First letter to the Thessalonians

8.1 Introduction

Compact heat exchangers are an interesting topic for a wide range of industrial
applications. In particular, compact heat exchangers are sought which are able to
transfer a large amount of heat in a limited volume or with a reduced weight, also
inducing a limited pressure drop on the heat-carrier fluids. In the automotive field, for
instance, the air side of radiators is often made of straight channels through which air
flows. Using wavy channels in place of straight channels would improve the amount
of heat dissipated by the heat exchanger within the same volume. In turn, the size of
the heat exchanger could be reduced at equal heat transfer rate.
Several papers are found in the literature in regard to compact heat exchang-
ers [100], corrugated wall channels [101], and periodic sinusoidal wavy channels
[102–105]. Some papers in which optimization techniques are employed for the
optimization of wavy channels are also available in the literature [106–109].
In this chapter, we discuss the way in which optimization techniques were applied
in order to find optimum shapes for a periodic wavy channel. The results of the
analysis were published by the author in [110].

M. Cavazzuti, Optimization Methods: From Theory to Design, 153

DOI: 10.1007/978-3-642-31187-1_8, © Springer-Verlag Berlin Heidelberg 2013
154 8 A Forced Convection Application: Surface Optimization for Enhanced Heat Transfer

8.2 The Case

Compact heat exchangers are liquid-to-liquid or liquid-to-air exchangers, usually

made of metal plates or fins through which two non-mixing fluids flow. In automotive
radiators the coolant fluid flows into straight pipes, and these are connected to a series
of thin metal plates stacked together acting as fins. The gaps between the plates are
the passages through which the cooling air flows. The heat transfer bottleneck in
automotive radiators is on the outer side, due to the low thermal conductivity of air.
For this reason, if the maximization of the heat transfer is sought, it is better to focus
on the optimization of the air side. Heat transfer on the air side could be enhanced
in different ways, such as:
• placing pins or ribs into the fluid stream so that air mixing and turbulence are
promoted,
• increasing the thickness of the plates of the heat exchangers so that uniform tem-
perature condition at the channel walls is drawed nigh, and the fin efficiency is
maximized,
• modifying the shape of the channel walls, and consequently of the passages be-
tween them, so that air mixing and flow impingement is favoured. In this case the
maximization of the wall-to-air heat transfer coefficient is sought.
The latter option is considered here. The solutions of this kind aim at enhancing the
overall heat transfer rate of the device mainly by disturbing the flow within the air
passages, by
• promoting turbulence and air mixing in the heat exchanger passages,
• promoting the flow impingement against the channel walls, to break the boundary
layer,
• increasing the overall surface of the heat exchanger.
In the simplest case the plates are plain surfaces as exemplified in Fig. 8.1a. A more
efficient situation could be achieved by shaping the passages of the heat exchanger in
a different way. One of the easiest choices is to generate sinusoidal channels like the
ones in Fig. 8.1b. However, an infinity of different shapes are possible, and sinusoidal
channels are not necessarily the best solution.
The idea behind the optimization experiment performed here is that it is possible
to create new designs for corrugated plates in a much more flexible way with the aim
of finding a better compromise between the heat transfer rate and head losses on the
air side. The basic assumptions for the exercise are:
• constant and uniform wall temperature at the plates:
since the heat transfer coefficient at the water side and the thermal conductivity of
the metal plates are relatively high, the hypothesis of constant and uniform wall
temperature Tw at the plates is not far from reality and is thus acceptable.
• periodic streamwise flow and heat transfer conditions:
the periodic shape of the channel allows us to focus the analysis on a single period,
or module. In fact, since the channel height to length ratio is very small, the flow
Chapter 9
A Natural Convection Application: Optimization
of Rib Roughened Chimneys

La perfection est réalisée,

pas quand il n’y a rien à davantage ajouter,
mais quand il n’y a plus rien à emporter.
Perfection is achieved,
not when there is nothing more to add,
but when there is nothing left to take away.
Antoine de Saint-Exupéry

9.1 Introduction

Natural convection heat transfer from vertical channels is important in several prac-
tical applications. For instance, two-dimensional channels with ribs, or other types
of protuberances, represent a configuration frequently encountered in the thermal
control of electronic equipment, where free convection cooling is often preferred to
forced convection cooling because of its inherent reliability.
Several papers were published on the topic, including different chimney and rib
configurations, involving both experimental [114–119] and numerical [119–123]
works, and applying either uniform wall temperature (UWT) or uniform heat flux
(UHF) boundary conditions at the channel walls.
In this chapter we discuss the way in which optimization techniques were applied
in order to find the optimum shape for ribs in a natural convection vertical chan-
nel with five evenly spaced ribs on a heated wall. The results of the analysis were
presented by the author in [124].

M. Cavazzuti, Optimization Methods: From Theory to Design, 175

DOI: 10.1007/978-3-642-31187-1_9, © Springer-Verlag Berlin Heidelberg 2013
176 9 A Natural Convection Application: Optimization of Rib Roughened Chimneys

9.2 The Case

Natural convection in vertical chimneys with a heated ribbed wall is encountered

in a number of technological application. It could exemplify the problem of elec-
tronic equipment cooling in cases where, for some reason, forced convection is not
exploitable. The aims in natural convection chimneys are either or both
• to transfer the larger amount of heat possible,
• to enhance the mass flow rate across the channel.
We note that, a larger mass flow rate implies higher fluid velocities in the chim-
ney which are likely to enhance convection, yielding a larger average heat transfer
coefficient, and ultimately a larger heat dissipation by the chimney.
In case the surface of the chimney is ribbed, it is of interest to investigate which is
the configuration of the ribs, if any, able to satisfy the given objectives. According to
Tanda [114] the presence of horizontal ribs affects natural convection heat transfer
in vertical chimneys owing to different circumstances:
• the blockage effect associated with the presence of protrusions could provoke a
weaker induced flow rate, potentially reducing the heat transfer rate,
• the roughness could induce disturbances in the overlying laminar boundary layer,
thus causing premature transition to turbulence,
• when thermally active the roughness elements add an extra heat transfer surface
area.
The idea behind the optimization experiment performed is to check whether the
presence of ribs on a heated surface in a natural convection vertical channel does
really improve the heat transfer, and, in case, to determine which shape, size, spacing,
and number of ribs is better to adopt for heat transfer enhancement.
The basic assumptions for the exercise are:
• the fluid is air with constant properties (Pr = 0.744), and the Boussinesq approx-
imation is enforced,
• the chimney is placed in vertical position: the fluid enters the chimney from the
bottom of the chimney (inlet) and leaves the chimney from the top (outlet). Constant
and uniform pressure and temperature are fixed at the inflow section. The inlet
temperature is 300 K,
• the chimney is a bidimensional channel with a heated ribbed wall facing a smooth
adiabatic wall, the ribs are attached to the heated wall and are perpendicular to the
fluid flow,
• on the heated wall a constant and uniform temperature (UWT) condition is hypoth-
esized. Preliminary checks were made by varying the wall temperature from 310 K
(!T = 10 K) to 345 K (!T = 45 K), where !T is the heated wall to ambient
temperature difference. During the optimization process !T was fixed at 45 K,
• the height of the chimney, H , was also varied during preliminary simulations.
In the final optimization process H was fixed at 175 mm, the distance between the
Chapter 10
An Analytical Application: Optimization
of a Stirling Engine Based on the Schmidt
Analysis and on the Adiabatic Analysis

Alles soll so einfach wie möglich

gemacht werden, aber nicht einfacher.
Everything should be made
as simple as possible, but not simpler.
Albert Einstein

10.1 Introduction

Stirling engines are external combustion engines converting thermal energy into
mechanical energy by alternately compressing and expanding a fixed quantity of air
or other gas (called the working or operating fluid) at different temperatures [126].
Stirling engines were invented by Robert and James Stirling in 1818. Despite their
high efficiency and quiet operation they have not imposed themselves over the Diesel
and Otto engines. In recent years interest in Stirling engines has grown, since they are
good candidates to become the core component of micro Combined Heat and Power
(CHP) units. In this chapter, we discuss an optimization experiment performed on
Stirling engines. In particular, optimization algorithms are applied to the Schmidt
and to the adiabatic analyses. These are two simple and rather idealized analytical
models of the Stirling machine. Before discussing the optimization issue we briefly
recall the basic elements of the Stirling cycle, and the Schmidt and the adiabatic
analyses.

M. Cavazzuti, Optimization Methods: From Theory to Design, 195

DOI: 10.1007/978-3-642-31187-1_10, © Springer-Verlag Berlin Heidelberg 2013
196 10 An Analytical Application: Optimization of a Stirling Engine

Fig. 10.1 The ideal Stirling cycle

10.1.1 The Stirling Thermodynamic Cycle

Stirling engines are based on the Stirling regenerative thermodynamic cycle which
is composed of four thermodynamic transformations:
• an isothermal expansion at high temperature,
• an isochoric regenerative heat removal,
• an isothermal compression at low temperature,
• an isochoric regenerative heat addition.
Since the operating fluid is expanded at high temperature and compressed at low
temperature a net conversion of heat into work is attained. The theoretical efficiency
of the cycle in case of complete reversibility equals that of the ideal Carnot cycle,
as stated by the Reitlinger theorem [127]. An ideal Stirling cycle between the tem-
peratures Tl and Th (Tl < Th ), and between the volumes Vl and Vh (Vl < Vh ) is
represented in Fig. 10.1 and is described by the following equations
!
W1,2 = 12 pdV = M RTh ln VVh > 0 Q 1,2 = W1,2 = M RTh ln VVh > 0
l l
W2,3 =0 Q 2,3 = Mcv (Tl − Th ) < 0
W3,4 = M RTl ln VVl < 0 Q 3,4 = W3,4 = M RTl ln VVl < 0
h h
W4,1 =0 Q 4,1 = Mcv (Th − Tl ) = −Q 2,3 > 0
Wnet = W1,2 − W3,4 = M R (Th − Tl ) ln VVh > 0 η= W Q
net = 1 − Tl = η
T car not
l 1,2 h
(10.1)
where Wm,n and Q m,n respectively are the amount of work and the heat exchanged
by the system during the transformation from the status m to the status n, p is the
pressure, and M the mass of the operating fluid in the system, R is the specific
gas constant, cv is the specific heat at constant volume of the gas, Wnet is the net
work output, and η the thermodynamic efficiency of the cycle. Q 2,3 and Q 4,1 are
exchanged regeneratively, thus they are not included into the efficiency equation.
Chapter 11
Conclusions

Do you know
what would be the best thing to do?
Fyodor Dostoyevsky,
The Brothers Karamazov

11.1 What Would be the Best Thing to do?

In conclusion, what would be the best thing to do for solving an optimization prob-
lem? In the spirit of the no free lunch theorem [87, 88], there is no optimum choice
which could be applied indistinctly to every problem. However, in engineering appli-
cations, some theoretical knowledge and some practical experience, make it possible
to find a way out. The only hardware we need is a simulation model or an apparatus
for laboratory experiments for collecting data. Then we have to choose a proper
optimization process to be applied. The optimization process suggested by the theo-
retical knowledge and the practical experience probably will not be the best possible
choice, and we will never know whether it is. Anyhow, it can be a good trade-off
between the accuracy of an optimum solution, and the effort we have to afford to
obtain it.
We can think at the choice of the optimization process as an optimization problem
itself in which the objectives are the effort required by the process (for instance in
terms of time, cost, hardware, people), to be minimized, and the accuracy of the
optimum solution which is found, to be maximized. The design space, however, in this
case has infinite size and the variables are the alternative optimization methods which
could be applied, the way in which they can be assembled to give an optimization
process, and all the parameters governing the process and defining the design space
of the original optimization problem. The Pareto frontier is given by the set of the

M. Cavazzuti, Optimization Methods: From Theory to Design, 225

DOI: 10.1007/978-3-642-31187-1_11, © Springer-Verlag Berlin Heidelberg 2013
226 11 Conclusions

most efficient processes which could have been chosen, and we are supposed to find
a good Pareto approximation in just one shot, without running an optimization.
Out of the metaphor, although this seems an impossible task, it is true that there
is no obvious choice, but it is also true that this task is not as impossible as it seems.
In this final chapter we want to give some directions on how to choose an optimization
process to be applied to an optimization problem. We do this by recollecting what has
been said throughout the text. These directions also depend on the author’s feeling
and experience on the field, and do not want to be a rigid scheme to be applied to
any optimization problem.
It must be kept in mind that the outcome of an optimization does not depend only
on those factors defining and tuning the optimization algorithms which are applied,
but it largely depends on many other aspects, which were thoroughly discussed in
the second part of the thesis, like:
• the experimental apparatus or the simulator, and the assumptions made during
their set up,
• the parameterization of the problem,
• the objectives of the optimization,
• the constraints of the optimization.
Although these may seem secondary issues, since they are not directly related to the
way the optimization algorithms work, they have a definite impact on the results. For
this reason, it is necessary to be extremely careful while considering those aspects
during the setting up of the optimization. Each choice has its advantages and its draw-
backs, and affects in some way the whole process. It is still the delicate equilibrium
between accuracy and efforts which come into play, for instance:
• an optimization process based on a simple simulation process yields a huge amount
of inaccurate results very quickly; a complex simulation process yields a few
accurate results with a lot of effort,
• an optimization process based on a simple parameterization involving a small
number of variables limits the degrees of freedom of the problem, but will converge
quickly; a larger parameterization allows the exploration of a more complex design
space, and thus could find better solutions, however it will require a much larger
effort,
• objectives and constraints are somewhat related to each other. In fact, the output
parameters, if they are of any interest, can be either optimized or constrained.
Each output variable which is optimized participates to the definition of the
Pareto frontier, makes the problem more general, and increases the complexity
of the optimization. Each constraint is reducing the degrees of freedom (it is like
restricting an hypothetical solution space to a sub volume or to a section), making
the optimization problem somewhat easier to solve, although less general.
Defining the constraints and the objectives, demands special care. It was shown
in Chap. 9, how the nondimensionalization of the objective functions, when the
definition of the nondimensional forms involves the input variables, might force
References

1. Oxford english dictionary. Oxford: Oxford University Press, 2008.

2. Wordreference online language dictionaries. [Link]
3. Darwin, C. (1859). On the origin of species by means of natural selection or the
preservation of favoured races in the struggle for life. London: John Murray.
4. Montgomery, D. C. (2000). Design and analysis of experiments (5th ed.). New York: Wiley.
5. NIST/SEMATECH (2006). NIST/SEMATECH e-handbook of statistical methods. http:
//[Link]/div898/handbook/.
6. Fisher, R. A. (1925). Statistical methods for research workers. Edinburgh: Oliver and Boyd.
7. Box, G. E. P., & Wilson, K. B. (1951). Experimental attainment of optimum conditions.
Journal of the Royal Statistical Society, 13, 1–45.
8. Taguchi, G., & Wu, Y. (1980). Introduction to off-line quality control. Nagoya: Central
Japan Quality Control Association.
9. Box, G. E. P., Hunter, W. G., & Hunter, S. J. (1978). Statistics for experimenters. New
York: Wiley.
10. Tartaglia, N. (1562). Quesiti et inventioni diverse. Vinegia: Curtio Troiano dee Nauò.
11. Box, G. E. P., & Behnken, D. (1960). Some new three level designs for the study of
quantitative variables. Technometrics, 2, 455–475.
12. Plackett, R. L., & Burman, J. P. (1946). The design of optimum multifactorial experiments.
Biometrika, 33(4), 305–325.
13. Berni, R. (2002). Disegno sperimentale e metodi di Taguchi nel controllo di qualità off-line.
Università di Trieste.
14. modeFRONTIERTM 3.1 user manual.
15. van der Corput, J. G. (1935). Verteilungsfunktionen. Proceedings of the Koninklijke
Nederlandse Akademie van Wetenschappen, 38, 813–821.
16. Quasi-monte carlo simulation. Pontificia Universidade Catòlica do Rio de Janeiro.
[Link]
17. Halton, J. H. (1960). On the efficiency of certain quasi-random sequences of points in
evaluating multi-dimensional integrals. Numerische Matematik, 2(1), 84–90.
18. Faure, H. (1982). Discrepance de suites associees a un systeme de numeration (en
dimension s). Acta Aritmetica, 41, 337–351.
19. Faure, H. (1992). Good permutations for extreme discrepancy. Journal of Number Theory,
42, 47–56.
20. Sobol’ I. M. (1967). On the distribution of points in a cube and the approximate evaluation
of integrals. USSR Computational Mathematics and Mathematical Physics, 7(4), 86–112.
21. Olsson, A., Sandberg, G., & Dahlblom, O. (2003). On latin hypercube sampling for
structural reliability analysis. Structural Safety, 25(1), 47–68.

M. Cavazzuti, Optimization Methods: From Theory to Design, 251

DOI: 10.1007/978-3-642-31187-1, ! Springer-Verlag Berlin Heidelberg 2013
252 References

22. Hardin, R. H., & Sloane, N. J. A. (1993). A new approach to the construction of optimal
designs. Technical report, AT&T Bell Laboratories.
23. Kappele, W. D. (1998). Using I-optimal designs for narrower confidence limits. In
Proceedings of the IASI Conference, Orlando, FL, February 1998.
24. Gauss, J. C. F. (1825). Combinationis observationum erroribus minimis obnoxiae.
Gottingen: University of Gottingen.
25. Edwards, L. A. (1984). An introduction to linear regression and correlation (2nd ed.). San
Francisco: Freeman.
26. Bates, D. M., & Watts D. G. (1988). Nonlinear regression and its applications. New York:
Wiley.
27. Optimus revision 5.0 users manual.
28. Krige, D. G. (1951). A statistical approach to some basic mine valuation problems on the
witwatersrand. Journal of the Chemical, Metallurgical and Mining Society of South Africa,
52(6), 119–139.
29. Hengl, T. (2007). A practical guide to geostatistical mapping of environmental variables.
Technical report, European Commission Joint Research Centre Institute for Environment
and Sustainability.
30. Gstat manual.
31. Mackay, D. J. C. (1997). Introduction to Gaussian processes. Technical report, Cambridge
University, Cavendish Laboratory.
32. Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning.
Cambridge: MIT Press.
33. Bayes, T. (1763). An essay towards solving a problem in the doctrine of chances by the late
Rev. Mr. Bayes, F. R. S. communicated by Mr. Price, in a letter to John Canton, A. M. F.
R. S. Philosophical Transactions, Giving Some Accounts of the Present Undertakings,
Studies and Labours of the Ingenious in Many Considerable Parts of the World, 53,
370–418.
34. Baxter, B. J. C. (1992). The interpolation theory of radial basis functions. PhD thesis,
Trinity College, Cambridge University.
35. Applied Research Associates New Zealand. [Link]
theory/[Link].
36. Fausett, L. (1993). Fundamentals of neural networks. Architecture, algorithms, and
applications. Englewood Cliffs: Prentice Hall.
37. Freeman, J. A., & Skapura, D. M. (1991). Neural networks. Algorithms, applications, and
programming techniques. Reading: Addison-Wesley.
38. Veelenturf, L. P. J. (1995). Analysis and applications of artificial neural networks.
Englewood Cliffs: Prentice Hall.
39. Rojas, R. (1996). Neural networks. Berlin: Springer.
40. Fletcher, R. (1987). Practical methods of optimization (2nd ed.). Chichester: Wiley.
41. Goldstein, A. A. (1965). On steepest descent. SIAM Journal on Control and Optimization,
3, 147–151.
42. Wolfe, P. (1968). Convergence conditions for ascent methods. SIAM Review, 11, 226–235.
43. Powell, M. J. D. (1976). Some global convergence properties of a variable metric algorithm
for minimization without exact line searches. In SIAM-AMS Proceedings, Philadelphia.
44. Spendley, W., Hext, G. R., & Himsworth, F. R. (1962). Sequential application of simplex
design in optimization and evolutionary operation. Technometrics, 4, 441–461.
45. Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. Computer
Journal, 7(4), 308–313.
46. Davidon, W. C. (1959). Variable metric method for minimization. Technical report, AEC
Research and Development Report ANL-5990.
47. Fletcher, R., & Powell, M. J. D. (1963). A rapidly convergent descent method for
minimization. Computer Journal, 6, 163–168.
References 253

48. Broyden, C. G. (1970). The convergence of a class of double rank minimization algorithms,
parts I and II. Journal of the Institute of Mathematics and its Applications, 6, 222–231.
49. Fletcher, R. (1970). A new approach to variable metric algorithms. Computer Journal, 13,
317–322.
50. Goldfarb, D. (1970). A family of variable metric methods derived by variational means.
Mathematics of Computation, 24, 23–26.
51. Shanno, D. F. (1970). Conditioning of quasi-Newton methods for function minimization.
Mathematics of Computation, 24, 647–656.
52. Polak, E. (1971). Computational methods in optimization: A unified approach. New York:
Academic Press.
53. Courant, R. (1943). Variational methods for the solution of the problems of equilibrium and
vibration. Bulletin of the American Mathematical Society, 49, 1–23.
54. Carroll, C. W. (1961). The created response surface technique for optimizing nonlinear
restrained systems. Operations Research, 9, 169–184.
55. Frisch, K. R. (1951). The logarithmic potential method of convex programming. Oslo: Oslo
University Institute of Economics Memorandum, May 1951.
56. Neumaier, A., & Shcherbina, O. (2004). Safe bounds in linear mixed-integer programming.
Mathematical Programming, 99, 283–296.
57. Schittkowski, K. (2001). NLPQLP: A new Fortran implementation of a sequential quadratic
programming algorithm for parallel computing. Technical report, University of Bayreuth.
58. Schittkowski, K. (1985–1986). NLPQL: A Fortran subroutine solving constrained nonlinear
programming problems. Annals of Operations Research, 5, 485–500.
59. Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing.
Science, 220(4598), 671–680.
60. Kennedy, J., & Eberhart, R. C. (1995). Particle swarm optimization. In IEEE International
Conference on Neural Networks, Perth, November/December 1995.
61. Mostaghim, S., Branke, J., & Schmeck, H. (2006). Multi-objective particle swarm
optimization on computer grids. In Proceedings of the 9th annual conference on genetic and
evolutionary optimization, London.
62. Rao, S. S. (1987). Game theory approach for multiobjective structural optimization.
Computers and Structures, 25(1), 119–127.
63. Nash, J. F. (1951). Non-cooperative games. Annals of Mathematics, 54, 286–295.
64. Rechenberg, I. (1973). Evolutionsstrategie: Optimierung technischer systeme nach
prinzipien der biologischen evolution. Stuttgart: Fromman-Holzboog.
65. Schwefel, H. P. (1981). Numerical optimization for computer models. Chichester: Wiley.
66. Holland, J. H. (1975). Adaptation in natural and artificial systems: An introductory analysis
with applications to biology, control, and artificial intelligence. Ann Arbor: University of
Michigan.
67. Pareto, V. (1906). Manuale d’economia politica con una introduzione alla scienza sociale.
Milano: Società Editrice Libraria.
68. Reyes-Sierra, M., & Coello Coello, C. A. (2006). Multi-objective particle swarm
optimizers: A survey of the state-of-the-art. International Journal of Computational
Intelligence Research, 2(3), 287–308.
69. Ahn, C. W. (2006). Advances in evolutionary algorithms. Theory, design and practice.
Berlin: Springer.
70. Rothlauf, F. (2006). Representations for genetic and evolutionary algorithms (2nd ed.).
Berlin: Springer.
71. Metropolis, N. C., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953).
Equations of state calculations by fast computing machines. Journal of Chemical Physics,
21(6), 1087–1092.
72. Millonas, M. M. (1994). Swarms, phase transitions, and collective intelligence. In
C. G. Langton (Ed.), Artificial life III. Reading: Addison-Wesley.
254 References

73. Clarich, A., Rigoni, E., & Poloni, C. (2003). A new algorithm based on game theory for
robust and fast multi-objective optimisation. Technical report, ESTECO.
74. Fraser, A. S. (1957). Simulation of genetic systems by automatic digital computers.
Australian Journal of Biological Sciences, 10, 484–499.
75. Bäck, T., Fogel, D. B., & Michalewicz, Z. (2000). Evolutionary computation 1. Basic
algorithms and operators. Bristol: Institute of Physics Publishing.
76. Bäck, T., Fogel, D. B., & Michalewicz, Z. (2000). Evolutionary computation 2. Advanced
algorithms and operators. Bristol: Institute of Physics Publishing.
77. Karaboğa, D., & Ökdem, S. (2004). A simple and global optimization algorithm for
engineering problems: differential evolution algorithm. Turkish Journal of Electric and
Computer Sciences, 12(1), 53–60.
78. Parsopoulos, K. E., Tasoulis, D. K., Pavlidis, N. G., Plagianakos, V. P., & Vrahatis, M. N.
(2004). Vector evaluated differential evolution for multiobjective optimization. In Proceed-
ings of the 2004 Congress on Evolutionary Computation.
79. Shokhirev, N. V. Optimization. [Link]
80. Schwefel, H. P. (1977). Numerische optimierung von computer-modellen mittels der
evolutionsstrategie. Basel: Birkhäuser.
81. Beyer, H. -G., & Deb, K. (1999). On the analysis of self-adaptive evolutionary algorithms.
Technical report, University of Dortmund, May 1999.
82. Runarrson, T. P., & Yao, X. (2002). Continuous selection and self-adaptive evolution
strategies. In Proceedings of the 2002 Congress on Evolutionary Computation.
83. Giannakoglou, K. C., & Karakasis, M. K. (2006). Hierarchical and distributed metamodel-
assisted evolutionary algorithms. In J. Périaux & H. Deconinck (Eds.), Introduction to
optimization and multidisciplinary design, Lecture Series 2006-03. Brussels: von Karman
Institute for Fluid Dynamics.
84. Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning.
Reading: Addison-Wesley.
85. Mitchell, M. (1998). An introduction to genetic algorithms. Cambridge: MIT Press.
86. Fogel, D. B. (2006). Evolutionary computation: Toward a new philosophy of machine
intelligence (3rd ed.). Piscataway: IEEE Press.
87. Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE
Transactions on Evolutionary Computation, 1(1), 67–82.
88. Wolpert, D. H., & Macready, W. G. (2005). Coevolutionary free lunches. IEEE Trans-
actions on Evolutionary Computation, 9(6), 721–735.
89. Juran, J. M., Gryna, F. M. J., & Bingham, R. S. (1974). Quality control handbook. New
York: McGraw-Hill.
90. Crosby, P. B. (1979). Quality is free. New York: McGraw-Hill.
91. Jones, D. R. (1989). Exploring quality: What Robert Pirsig’s ‘‘zen and the art of motorcycle
maintenance’’ can teach us about technical communications. IEEE Transactions on
Professional Communication, 32(3), 154–158.
92. ISO 9000 (2005). Quality management systems: Fundamentals and vocabulary. Geneva:
International Organization for Standardization.
93. Pyzdek, T. (2003). The six sigma handbook. New York: McGraw-Hill.
94. Pediroda, V., & Poloni C. (2006). Robust design, approximation methods and self
organizing map techniques for MDO problems. In J. Périaux & H. Deconinck (Eds.),
Introduction to optimization and multidisciplinary design, Lecture Series 2006-03. Brussels:
von Karman Institute for Fluid Dynamics.
95. AIAA (1998). Guide for verification and validation of computational fluid dynamic
simulation. AIAA guide G-077-1998.
96. Stocki, R., Kolanek, K., Jendo, S., & Kleiber, M. (2005). Introduction to reliability-based
design. Warsaw: Institute of Fundamental Technological Research, Polish Academy of
Sciences, Division of Computational Mechanics.
References 255

97. Adhikari, S., & Langley, R. S. (2002). Reduction of random variables in structural
reliability analysis. Technical report, Cambridge University.
98. Cizelj, L., Mavko, B., & Riesch-Oppermann, H. (1994). Application of first and second
order reliability methods in the safety assessment of cracked steam generator tubing.
Nuclear Engineering and Design, 147, 359–368.
99. Schuëller, G. I., Pradlwarter, H. J., & Koutsourelakis, P. S. (2003). A comparative study of
reliability estimation procedures for high dimensions. In Proceedings of the 16th ASCE
Engineering Mechanics Conference, University of Washington, Seattle, July 2003.
100. Shah, R. K., & London, A. L. (1978). Laminar flow forced convection in ducts: A source
book for compact heat exchanger analytical data (Advances in Heat Transfer, Suppll. 1).
New York: Academic Press.
101. Goldstein, L., & Sparrow, E. M. (1977). Heat and mass transfer characteristics for flow in a
corrugated wall channel. ASME Journal of Heat Transfer, 99, 187–195.
102. Nishimura, T., Murakami, S., Arakawa, S., & Kawamura, Y. (1990). Flow observations and
mass transfer characteristics in symmetrical wavy-walled channels at moderate Reynolds
numbers for steady flow. International Journal of Heat and Mass Transfer, 33(5), 835–845.
103. Wang, G., & Vanka, S. P. (1995). Convective heat transfer in periodic wavy passages.
International Journal of Heat and Mass Transfer, 38(17), 3219–3230.
104. Ničeno, B., & Nobile, E. (2001). Numerical analysis of fluid flow and heat transfer in
periodic wavy channels. International Journal of Heat and Fluid Flow, 22(2), 156–167.
105. Stalio, E., & Piller, M. (2007). Direct numerical simulation of heat transfer in converging-
diverging wavy channels. ASME Journal of Heat Transfer, 129, 769–777.
106. Hilbert, R., Janiga, G., Baron, R., & Thévenin, D. (2006). Multi-objective shape
optimization of a heat exchanger using parallel genetic algorithms. International Journal
of Heat and Mass Transfer, 49(15–16), 2567–2577.
107. Foli, K., Okabe, T., Olhofer, M., Jin, Y., & Sendhoff, B. (2006). Optimization of micro heat
exchanger: CFD, analytical approach and multi-objective evolutionary algorithms.
International Journal of Heat and Mass Transfer, 49(5–6), 1090–1099.
108. Kim, H. -M., & Kim, K. -Y. (2004). Design optimization of rib-roughened channel to
enhance turbulent heat transfer. International Journal of Heat and Mass Transfer, 47(23),
5159–5168.
109. Nobile, E., Pinto, F., & Rizzetto, G. (2006). Geometrical parameterization and multi-
objective shape optimization of convective periodic channels. Numerical Heat Transfer
Part B: Fundamentals, 50(5), 425–453.
110. Cavazzuti, M., & Corticelli, M. A. (2008). Optimization of heat exchanger enhanced
surfaces through multi-objective genetic algorithms. Numerical Heat Transfer, Part A:
Applications, 54(6), 603–624.
111. Nishimura, T., Ohori, Y., Kawamura, Y. (1984). Flow characteristics in a channel with
symmetric wavy wall for steady flow. Journal of Chemical Engineering of Japan, 17(5),
466–471.
112. Bézier, P. E. (1977). Essai de définition numérique des courbes et des surfaces
expérimentales. PhD thesis, Université Pierre et Marie Curie, Paris.
113. Piegl, L., & Tiller, W. (1997). The NURBS book (2nd ed.). Berlin: Springer.
114. Tanda, G. (1997). Natural convection heat transfer in vertical channels with and without
transverse square ribs. International Journal of Heat and Mass Transfer, 40(9), 2173–2185.
115. Acharya, S., & Mehrotra, A. (1993). Natural convection heat transfer in smooth and ribbed
vertical channels. International Journal of Heat and Mass Transfer, 36(1), 236–241.
116. Bhavnani, S. H., & Bergles, A. E. (1990). Effect of surface geometry and orientation on
laminar natural convection heat transfer from a vertical flat plate with transverse roughness
elements. International Journal of Heat and Mass Transfer, 33(5), 965–981.
117. Aydin, M. (1997). Dependence of the natural convection over a vertical flat plate in the
presence of the ribs. International Communications in Heat and Mass Transfer, 24(4),
521–531.
256 References

118. Polidori, G., & Padet, J. (2003). Transient free convection flow on a vertical surface with an
array of large-scale roughness elements. Experimental Thermal and Fluid Science, 27(3),
251–260.
119. Onbasioglu, S. U., & Onbas!ioğlu, H. (2004). On enhancement of heat transfer with ribs.
Applied Thermal Engineering, 24(1), 43–57.
120. Kelkar, K. M., & Choudhury, D. (1993). Numerical prediction of periodically fully
developed natural convection in a vertical channel with surface mounted heat generating
blocks. International Journal of Heat and Mass Transfer, 36(5), 1133–1145.
121. Desrayaud, G., & Fichera, A., (2002). Laminar natural convection in a vertical isothermal
channel with symmetric surface-mounted rectangular ribs. International Journal of Heat
and Fluid Flow, 23(4), 519–529.
122. ElAlami, M., Najam, M., Semma, E., Oubarra, A., & Penot, F. (2004). Chimney effect in a
‘‘T’’ form cavity with heated isothermal blocks: The blocks height effect. Energy
Conversion and Management, 45(20), 3181–3191.
123. Bakkas, M., Amahmid, A., & Hasnaoui, M. (2006). Steady natural convection in a
horizontal channel containing heated rectangular blocks periodically mounted on its lower
wall. Energy Conversion and Management, 47(5), 509–528.
124. Cavazzuti, M., & Corticelli, M. A. (2008). Optimization of a bouyancy chimney with a
heated ribbed wall. Heat and Mass Transfer, 44(4), 421–435.
125. Cavazzuti, M., Pinto, F., Corticelli, M. A., & Nobile, E. (2007). Radiation heat transfer
effect on natural convection in asymmetrically heated vertical channels. In Proceedings of
the XXV Congresso Nazionale UIT sulla Trasmissione del Calore, Trieste, June 18–20 2007.
126. Walker, G. (1973). Stirling-cycle machines. Oxford: Oxford University Press.
127. Reitlinger, J. (1873). Ueber kreisprocesse mit zwei isothermischen curven. Zeitschrift des
Österreicische Ingenieure Vereines, 245–252.
128. Schmidt, G. (1871). Theorie der lehmannschen calorischen maschine. Zeit Der Vereines
deutscher Ing, 15, 97–112.
129. Urieli, I., & Berchowitz, D. M. (1984). Stirling cycle engine analysis. Bristol: Adam Hilger.
130. Naso, V. (1991). La macchina di Stirling. Milano: Editoriale ESA.
131. Euler, L. (1768). Institutionum calculi integralis volumen primum in quo methodus
integrandi a primis principiis usque ad integrationem aequationum differentialium primi
gradus pertractatur. Petropoli: Impenfis Academiae Imperialis Scientiarum.
132. Runge, C. (1895). Ueber die numerische auflösung von differentialgleichungen.
Mathematische Annalen, 46, 167–178.
Index

A Bipolar sigmoid function, 66

Activation function, 66 Blend cross-over, 119
Active set method, 91 Blocking, 14
Actual reduction, 82 Blue, see best linear unbiased
Adiabatic analysis, 200 estimator, 51
Adjusted regression parameter, 48 Box-Behnken, 25
Aim, 42, 75 Bracket, 81
Aliasing, 22 Bracketing, 80
Allele, 107 Branch and bound method, 91, 97
Anisotropic kriging, 59 Broyden family, 87
Anisotropy, 59
Approximating, 44
Approximation, 71 C
Architecture, 66 Central composite, 23
Archive, 112 circumscribed, 24
Axis orthogonal importance latin hypercube faced, 24
sampling, 139 inscribed, 24
Axis orthogonal importance sampling scaled, 24
Monte Carlo, 136 Child, 107
Chimney, 176
Cholesky decomposition, 35
B Chromosome, 107
B-spline, 157 Coefficient of variation, 135
Bézier curve, 157 Cognitive learning factor, 111
Backpropagated error, 70 Cold dead volume ratio, 198
Backpropagation algorithm, 61 Combination factor, 117
Balanced, 21 Compact heat exchanger, 153
Barrier function, 91 Compression space, 197
Barrier function method, 96 Conditional probability, 60
Basis functions, 59 Confounding, 22
Bayesian method, 50 Conjugate direction methods, 87
Bernstein basis polynomials, 158 Conjugate gradient method, 87
Best fit, 44 Constraint, 3
Best linear unbiased estimator, 51 Constraint satisfaction problem, 78
BFGS formula, 87 Continuous selection, see steady-state
Bias, 67 evolution, 118
Binary step function, 66 Contour plot, 44

M. Cavazzuti, Optimization Methods: From Theory to Design, 257

DOI: 10.1007/978-3-642-31187-1, ! Springer-Verlag Berlin Heidelberg 2013
258 Index

C (cont.) Euler method, 204

Control factor, see control factor, 15 Evolutionary algorithm, 103
Control points, 157 Evolutionary algorithms, 116
Control variables, 27 Exact penalty functions, 91
Cooler, 197 Expanded design matrix, 39
Cooperative game, 114 Expansion space, 197
Correlation reduction, 34 Expected value, see mean value, 30
Cost function, 67 Experiment, 2, 13
Courant penalty function, 96 Experimental design, see design
Covariance function, 51 of experiments, 13
Covariance matrix, 34, 38 External factors, 132
Craziness, 111
Cross-over, 123
Cross-over constant, 117 F
Cross-over operator, 116 Factor, 14
Crossed array, 27 Failure area, 133
Cumulative distribution, see distribution Failure probability, 10, 133
curvature, 34 Faure sequence, 33
Curvature, 78 Feasible point, 90
Feasible region, 90
Feedforward, 67
D First order necessary condition, 79
Data set, 44 First order reliability method, 136
Degree of freedom, 19 Fitness function, 105
Delta rule, 67 Follower, 114
Derandomized evolution strategy, 116 Fractional factorial, 21
Derandomized evolutionary one-half, 21
strategy, 120 one-quarter, 21
Design factor, see primary factor, 15 Friction factor, 156
Design of experiments, 6, 13, 149 Full factorial, 17
Design point, 134 adjustable, 19
Design resolution, 22 two-levels, 17
Design space, 2 Function evaluation, 77
Deterministic optimization, 77, 151 Fuzzy recombination, 119
DFP formula, 87
Differential evolution, 116
Direct elimination method, 93 G
Direct numerical simulation, 156 Game theory, 103
Direction, 78 Gauss-Newton algorithm, 47
Direction set method, 87 Gene, 7, 107
Directional cross-over, 124 General linearly constrained
Displacer, 197 optimization, 91
Distribution, 34, 121 Generalized elimination method, 94
normal Gaussian, 34 Generation, 107
Disturbance factors, see Generational evolution, 124
nuisance factor, 15 Generational selection, see generational
DNA, 107 evolution, 119
Generator, 22
Genetic algorithm, 103, 121
E Genotype, 107
Effort, 75 Global intermediate, 119
Elimination, 90 Golden section method, 81
Elitism operator, 124 Graeco-latin square, 16
Emissivity, 190 Guide, 111
Index 259

H Kriging variance, 52
Halton sequence, 33 Kuhn-Tucker conditions, 92
Heater, 197
Hessian matrix, 78
Hidden layer, 67 L
Hierarchical and distributed metamodel-assis- Lag, 53
ted evolutionary algorithms, 120 Lagrange multipliers method, 90
Hierarchical competitive game, 114 Lagrange–Newton method, 97
Hierarchy, 120 Lagrangian function, 92
Homogeneous covariance function, 62 Lagrangian matrix, 95
Hot dead volume ratio, 198 Lagrangian method, see lagrange
Hyper-graeco-latin square, 16 multipliers method, 94
Hyperbolic tangent sigmoid function, 66 Laminar flow, 157
Larger-the-better, 29
Latin hypercube, 33
I Latin hypercube sampling, 136, 138
Identity function, 66 Latin square, 16
Importance latin hypercube sampling, 138 Leader, 11, 114
Importance sampling, 137 Learning rate, 70
Importance sampling Monte Carlo, 138 Least squares, 44
Individual, 107 Levels, 14
Inertia factor, 111 Levenberg–Marquardt methods, 89
Initial value problem, 204 Levenberg–Marquardt trajectory, 90
Inner array, 27 Limit state function, 133
Input layer, 67 Line, 78
Input parameters, 2 Line-search, 79
Input variable, 2 Linear least squares, 45
Integrated prediction variance, 38 Linear programming, 91
Interaction effect, 19 Load effect, 133
Internal energy, 108 Logistic sigmoid function, 66
Interpolating, 44
Interpolation, 71
Involute, 182 M
Main interaction, 18
Marginal probability, 60
J Mass flow rate, 178
Joint probability, 61 Mathematical programming, 7
Mean value, 13, 29, 34
Merit function, 98
K Meta-model, 43
K-nearest, 50 Metamodel, 121, 150
Khayyam triangle, see Micro combined heat and power unit, 195
Tartaglia triangle, 19 Mixed integer programming, 91, 97
Kriging, 50 Mixing number, 118
disjunctive, 52 Model function, 44
indicator, 52 Mollifier Shepard, 50
IRF-k, 51 Moment matrix, 39
lognormal, 52 Monte Carlo simulation, 135
multiple-indicator, 52 Multi-disciplinary optimization, 160
ordinary, 51 Multi-layer, 68
simple, 51 Multi-membered
universal, 51 evolution strategy, 116
Kriging error, see kriging variance, 52 Multi-objective genetic algorithm, 124
Kriging nearest, 50 Multi-objective optimization, 105
260 Index

M (cont.) convex, 8
Multi-objective robust design optimization, 9, deterministic, 7
132 discrete, 8
Mutant individual, 116 evolutionary, 7
Mutation constant, 117 genetic, 7
Mutation operator, 116 global, 8
gradient-based, 7
local, 8
N multi-objective, 3, 8
Nash equilibrium, 113 multivariate, 8
Neural networks, 66 single objective, 3, 8
Neuron, 66 stochastic, 7
Newton’s method, 85 unconstrained, 7
NLPQLP, 98 Order of convergence, 79
No free lunch theorem, 130 Orthogonal, 18
Noise, 71, 131 Outer array, 27
Noise, see noise factors, 13 Output layer, 67
Noise factors, 9 Output parameters, 2
Noise variables, 27
Nominal-the-best, 30
Non uniform rational b-spline, 157 P
Non-smooth optimization, 91 Parameter, 14, 75
Nondimensional analysis, 157 Parent, 107
Nonlinear least squares, 46 Pareto dominance, 105
Nonlinear programming, 91 Pareto frontier, 105
Nonstationary covariance function, 62 Pareto optimality, 105
Normal regression parameter, 47 Partial sill, 55
Normalized average, see integrated prediction Particle swarm optimization, 103, 110
variance, 38 Pascal triangle, see Tartaglia triangle, 19
Nugget, 51, 55 Penalty function, 91
Nuisance factor, 15 Penalty function method, 96
Number of experiments, 41 Phenotype, 107
Number of levels, 41 Plackett-Burman, 26
Number of parameters, 41 Player, 113
Nusselt number, 156 Plenum, 178
Population, 107
Power piston, 197
O Practical range, 55
Objective, see objective function, 2 Predicted reduction, 82
Objective function, 2 Prediction variance, 39
Offspring, 107 Predictive capability of the model, 48
One-point cross-over, 123 Pressure swing ratio, 199
Operating conditions, 132 Primal active set method, 95
Operating fluid, 195 Primary factor, 15
Optimal design, 36 Prior probability, see marginal probability, 60
a-optimal, 40 Problem, see optimization problem, 2
d-optimal, 40 Pseudo-random numbers generator, 32
e-optimal, 40
g-optimal, 40
i-optimal, 38 Q
Optimal RSM, 49 Quadratic programming, 91
Optimization problem, 2 Quality, 131
Optimization, 2, 3 Quasi-Newton condition, 86
constrained, 7 Quasi-Newton methods, 85
Index 261

R Sampling map, 33
Radial basis function Scaling factor, 117
Gaussian, 62 Schmidt analysis, 197
inverse multiquadric, 63 Second order necessary condition, 79
multiquadric, 63 Second order reliability method, 137
polyharmonic splines, 63 Sectioning, 80
Radiation heat transfer, 190 Selection, 122
Random, 32 Self-adaptive evolution, 116
Random search, 109 Semivariance, 53
Random seed generator, 77 Semivariogram, 51
Randomization, 13 Semivariogram cloud, 53
Randomized complete block design, 15 Semivariogram model, 53
Range, 55 Bessel, 55
Rank one formula, 86 circular, 55
Rayleigh number, 177 exponential, 55
Recirculation, 187 Gaussian, 55
Recurrent, 67 linear, 55
Reduced dead volume, 198 pentaspherical, 55
Reduced gradient vector, 94 spherical, 53
Reduced Hessian matrix, 94 Sequential competitive game, see hierarchical
Regenerator dead volume ratio, 198 competitive game, 113
Regenerator mean effective temperature, 198 Sequential quadratic programming, 91
Region of interest, 14 Set of active constraints, 90
Regression parameter, 47 Shepard, 50
Regularity, 71 Shift vector, 46
Reinforcement learning, 66 Signal-to-noise ratio, 29
Reliability, 131 Sill, 55
Reliability analysis, 9, 132 Simple importance latin hypercube
Reliability index, 10, 134 sampling, 139
Replication, 13 Simplex method for linear optimization, 91
Resistance effect, 133 Simplex method for nonlinear optimization, 82
Response surface, 20, 43 Simulated annealing, 103, 107
Response surface methodology, see response Simulated binary cross-over, 119
surface modelling, 43 Simulation, 2
Response surface modelling, 6, 43, 149 Simultaneous competitive game, 113
Response variable, 14 Single-layer, 68
Restricted step, 79 Sinusoidal wavy channel, 153
Rib, 176 Slope, 78
Regenerator, 197 Smaller-the-better, 29
Robust design analysis, 131, 152, 8 Sobol sequence, 33
Robust engineering design, see robust design Social learning factor, 111
analysis, 8 Solution space, 2
Robust parameter design problem, 27 Space filling, 30
Robustness, 105, 131, 8 Spatial auto-correlation effect, 53
Rotatability, 25 Standard deviation, 13, 29, 34
Roulette-wheel selection, 122 Standard normal space, 134
Runge–Kutta methods, 204 Star points, 23
Stationary covariance function, 61
Statistical design of experiments, see statistical
S experimental design, 14
Safe area, 133 Statistical experimental design, 14
Sample, 2 Steady-state evolution, 124
Sample size, 15 Steady-state selection, see steady-state
Sample space, 14 evolution, 118
 262 Index

S (cont.) U
Steepest descent method, 85 Uncertainty, see noise, 131
Stirling cycle, 196 Uniform cross-over, 123
Stirling engine, 195 Uniform heat flux condition, 175
Stochastic optimization, 103, 150 Uniform wall temperature condition, 175
Strength of the mutation, 119 Unimodal normally distributed cross-over, 119
Supervised learning, 66 Unsupervised learning, 66
Swarm intelligence, 104

V
T Van der Corput sequence, 32
Taguchi, 27 Variable, see input variable, 2
Tartaglia triangle, 19 Variance, 38
Temperature ratio, 198 Volume ratio, 198
Tolerance, 132
Tournament selection, 122
Training algorithm, 66 W
Transformed importance latin hypercube Wavy channel, 153
sampling, 139 Wear, 132
Transitional flow, 156 Wetted area, 177
Travelling salesman problem, 109 Wolfe–Powell conditions, 80
Treatment factor, see primary factor, 15 Word, see generator, 22
Trial individual, 116 Words, 22
Trust region, 79 Working fluid, see operating fluid, 195
Turbulence, 111 Working space, 195
Turbulence model, 156
Two-points cross-over, 123

View publication stats