See discussions, stats, and author profiles for this publication at: [Link]

net/publication/301455159

Measuring synergy using entropy

Conference Paper · November 2015

DOI: 10.3390/ecea-2-C001

CITATION READS

1 1,688

2 authors:

Klaus Jaffe Gerardo L. Febres

Simon Bolívar University [Link]/view/kjaffe Simón Bolívar University, Baruta, Venezuela
284 PUBLICATIONS 4,999 CITATIONS 18 PUBLICATIONS 155 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Self-Organized Free Intelligent Academia (SOFIA) View project

Evolutionary sociodynamics View project

All content following this page was uploaded by Klaus Jaffe on 26 May 2016.

The user has requested enhancement of the downloaded file.

 OPEN ACCESS

[Link]/conference/ecea-2
Conference Proceedings Paper – Entropy

Measuring synergy using entropy

Klaus Jaffe * and Gerardo Febres

Universidad Simon Bolivar, Caracas, Venezuela, E-Mail: kjaffe@[Link]

Published: 13 November 2015

Abstract: Synergy is phenomenon found everywhere in nature at all levels of organization

in physics, chemistry, biology, social sciences, and the arts. The English Wikipedia defines
Synergy as "the creation of a whole that is greater than the simple sum of its parts. The
term synergy comes from the Attic Greek word συνεργία synergia from synergos,
συνεργός, meaning working together". Synergy is an important concept that is difficult to
define precisely and even more difficult to quantify. Entropy on the other hand has been
used as a way to measure order or complexity. Here we explore the changes in entropy
induced by diverse process described as synergistic. The paper uses seven examples
reported in the literature to quantify synergy in different settings. The result shows that
synergistic processes are associated to large decreases in entropy and slightly less large
increases in work output of the system where they occur. Systems which suffer changes in
entropy that are not associated to synergistic processes show a different pattern These
results open the door to use entropy measures to identify and classify synergistic processes.

Keywords: complexity; entropy; social; evolution; synergy

1. Introduction

No unambiguous definition of synergy exists, but most non-additive phenomena might qualify as
synergy. Wikipedia defines it as the creation of a whole that is greater than the simple sum of its parts.
That is, 1+1 > 2. This phenomena seems to be quite common everywhere (Corning 1983, Jaffe 2015)
but a precise way of tackling the study of this phenomena has jet to be developed. For example, the
transformation of oxygen and hydrogen atoms to for a molecule of water has often been presented as a
synergistic process. Certainly, the properties of water cannot be deduced from those of oxygen and
hydrogen atoms: they differ radically. It seems a clear cut case of 1+1 <> 2. The advent of quantum
chemistry has allowed us to understand this phenomena in great detail, making the reference to
synergy unnecessary. Thus, the presence of synergy in natural phenomena might just reveal our
ignorance about them.
In thermodynamic, chemical reactions have properties that relate to its complexity. Quantitative
measurements of Helmholtz free energy and Gibbs free energy or enthalpy have shown to be very
useful in understanding chemical reactions. A similar approach should help us understanding synergy
in complex systems. This can be done my measuring changes in the complexity of the system triggered
by a given synergistic process. One way of estimating complexity is by measuring or estimating the
entropy of the system.
The great advantage of science over other heuristic protocols is the use of empirical methods (Jaffe
2010). Here we want to explore examples where an empirical and quantitative approach to the study of
synergy is possible. In all of them, the concept of Shannon's entropy is fundamental in understanding
the emergence of synergy. In information theory, Shannon entropy is the expected value (average) of
the information contained in each message or structure that contains information. The entropy of the
message is its amount of uncertainty; it increases when the message is closer to random, and decreases
when it is less than random. Named after Boltzmann's H-theorem, Shannon defined the entropy Η of a
discrete random variable x as:
H(x) = ∑i P(xi) logb P(xi)
where b is the base of the logarithm used, and P a probability function of states.
H is often related to the complexity of a system. The lower H the more complex structure the
system possesses. The assessment of complexity depends on how it is regarded. If complexity C is
regarded as the effort needed to make a description of the system, then it can be measured as
Shannon's (1951) information entropy H. Since a description's entropy depends on the suitability of the
language or metric used for that specific description, Kolmogorov (1965) stated that complexity equals
the information needed assuming the language or metric used is the best suited for the description. Not
being able to ensure we are using the best description, we have to be content with estimating
Kolmogorov's complexity KC by using Shannon's information entropy. Thus KC ≈ H.
Complexity can also be regarded as the effort needed to describe the internal activity the system has
to perform in order to keep itself structured and under control, and at least temporarily achieve the
objective of limiting the natural and inexorable tendency of its entropy to increase. A way to asses this
estimation of complexity is to contrast the Kolmogorov's complexity KC of the system with the
'distance' the system places itself from total disorder, which is 1-KC.

2. Methods

As H can be calculated or estimated quantitatively, here we relate measures of H to seven processes

where synergy is known to be acting and where the relevant data for calculating H is available. We
compare H in two states, before (H1) and after (H2 ) the synergistic process using a ratio Rho (P), and
estimate the rate of change from state 1 to state 2. This unit free scalar was called the Rho of H and
calculated as Ρ(H) = H2/H1. A mirror scalar called P(N) was the unit free quantity transformed to a
scalar related to negentropy so that P(N) = 1/P(H).
Estimates of work output or efficiency of a parameter that relates to the quantity or quality of the
output were calculated for each example. These values were used to calculate the ratio Rho by dividing
the values associated with the lowest entropy in the final state after the synergistic process occurred by
those associated with the highest entropy before the synergistic process started. We called this ratio P(W).
A summary of the data is presented in Table 6.
3. Results

3.1. Social complexity and minimal cost of maintenance of society (Data taken from Jaffe & Hebling-
Beraldo 1993)

Living in society must have benefits for the inhabitants if societies are to be sustainable. That is,
social synergy allows social organisms to be more efficient in resource use than non-social ones. Data
measuring the efficiency of energy consumption in ant societies of different social complexity and size
are available. The study used here finds that ant species with higher polymorphisms (i.e. larger number
of morphological distinct worker castes) have a lower per capita energy requirement for workers.
Specifically, among ants in the family Formicinae species with 1 worker caste consume on average
about 1.1 more energy than ant species with 2 worker castes. Among Myrmicinae the relationship is
1.7 times, and among Attini (leaf-cutter ants) with Atta species having >2 castes, the proportion is at
least 2. This gradation corresponds well with the difference in social complexity of monomorphic and
polymorphic ant societies among these three groups of ants. If P(H) is calculated just using the number
of castes (1 and 2), a great underestimation of real complexity of these ant societies, we have that
Shannon entropy of the society decreases at least by half. That is, P(N) = 2 ; whereas the Rho P(W) for
work output, as estimated for Myrmicinae's increase in energy efficiency with increased number of
castes was 1.7.

3.2. Size of social aggregates and social synergy (Data taken from Jaffe 2010)

Larger cities are more efficient in their use of energy than smaller ones. The exponent by which per
capita consumption of electricity diminishes according to the number of inhabitants a city hosts varies
from -0.47 in Brazil of -0.18 in Denmark. If we accept that heterogeneity in quality of services in
Brazil is much higher than in Denmark, and that smaller rural towns have less services than larger
metropolis in Brazil, we can assume the proportionality exponents is related to the difference in
complexity of large and small cities. That is, in Brazil cities are 2.6 times more heterogeneous
regarding complexity of services than those in Denmark. Among ant societies, these difference is at
least 4.6 times. Here we assume that this difference is a proxy of 1/H.

3.3. Science, Economics and Social Synergy (Data taken from Jaffe et al. 2013a,b and Jaffe, Rios,
Florez 2013a).

The complexity of an economy and of a scientific community can be calculated with great accuracy.
But both have to be calculated differently. Economic Complexity (Hausmann 2011) Table 1 shows the
relationship between different ways of calculating complexity of the knowledge of a society with
Economic Complexity (ECI) as calculated by Hausmann et al (2011) and Economic Growth as
calculated by the World Bank as Gross Domestic Product per capita (GDPc). The knowledge of a
society can be calculated as ECI, as academic publications per capita and as a scientific complexity
index (SCI) calculated in the sane way as ECI but using academic publications as output rather than
exports of products as in ECI (Jaffe et al 2013a). Another index that showed to be related to the
sophistication of a scientific community is the Revealed Comparative Advantages or RCA (Laursen
1998). Specifically RCS in Neuroscience (Jaffe et al 2013b).

Table 1. Correlation coefficients between scientific publications / capita; Science

Complexity Index; Economic Complexity Index and Revealed Comparative Advantages of
Neurociences with the Economic Complexity Index and Gross National Product per capita
of nations in 2010

Spearman
ECI 2008 GDPc 2010
correlations
Publications /
0.79 0.9
capita
SCI 2010 -0.06 0.1

ECI 2008 - 0.75

RCA 2010
0.68 0.72
Neuroscience
Table 1 shows that the index for knowledge complexity that best correlate with economic growth is
Publications/capita. The log of this value as reported in the mentioned paper ranged between -2.5 and
0.6 (P(N) = 3.1), whereas the log of GDPc ranged between 2.4 and 5.0 (P(W) = 2.6). That is the ratio
was >2 in both cases. We can thus relate P(W) estimated as the ratio of the extremes in GDP, and P(H)
as the ratio of the extremes of scientific complexity or complexity of knowledge managed by the
society.

3.4. Division of labor and the invisible hand (Jaffe 2015).

In simulations of economic markets exploited by societies of agents using Sociodynamica, the

amount of resources a society without division of labor could accumulate was 8 units, compared to 63
units accumulated by societies with division of labor into 3 discrete categories (miners, farmers and
traders). Thus, a P(N)= 3 achieved economic gains of 7.8 times which makes P (W) = log(63)/log(8) =
2.96. This result was used as a clear demonstration that division of labor allows for synergistic
interactions of agents which allow economic markets to work efficiently, triggering economic growth.
This example is considered as a classical example of the working of synergy.

3.5. Brain complexity and social synergy (Jaffe & Perez 1989).

Brain complexity might be thought is an adaptive feature to allow for complex cognitive task in
their bearers. If more complex societies require more complex tasks from their participating
individuals, we would expect a relationship between bran complexity and complexity of the society the
individuals form. Even a synergistic relationship might exist. Data from ant societies however reveal a
different result (Table 2)
 Table 2. Various indicators of brain complexity for workers of different ant species
showing a variety of social complexity

Ant species Asymmetry between Total surface Social complexity Estimated

internal and external of folding of estimated with average
calyxes the calyx polymorphism colony size

Odontomachus
2 80 1.1 500
bauri
Trachymyrmex
7 88 1.2 1000
urichi
Acromyrmex
13 100 1.5 5000
rugosus
Acromyrmex
23 128 2 100000
octospinosus

Atta laevigata 32 90 3 500000

In some cases, as the overall surface of a brain structure called the Calyx, This relationship is
concave with a maximum value in species with an intermediate level of social complexity such as
Acromyrmex octospinosus. In the most socially complex society studied, Atta laevigata, individuals
showed a lower index of overall surface of the calyx. This relationship might suggest that socially very
complex societies require cognitive less developed individuals.
The asymmetry between the two calyxes, however, was related positively and continually with
social complexity, This parameters is related to the complexity of the brain rather than to its total
capacity. We choose this last index to make P(N) estimates. The rate P(N) between the minimum and
maximum value of this index was 16 or in entropy scalars log(16) = 4
Estimates of work output P(W) were two. The degree of worker caste polymorphism, and the
estimated size of the average colony in a log scale. Each of these two different estimates of social
complexity are related to the complexity of the information content the society has to manage. The
ratio P (W) between maximum and minimum vales for each of them was 2.7 and 3 respectively

3.6. Synergies and language (Febres, Jaffe, Gershenson 2014 and Febres & Jaffe 2015).

Languages can be regarded as systems formed by large number of symbols and rules. For natural
languages, even though there are other options, words can play the role of symbols. As a message
written in a natural language is more organized, the entropy of the message reduces making it prone
for a better understanding and therefore more effective as an information transmission process. We
measured the entropy based on the word frequency of speeches written in English and Spanish.
Assuming that those writers who won the literature Nobel Prize are masters of the use of natural
languages, we compared the entropy computed for their speeches with the entropy computed for the
speeches of non-Nobel writers. We also considered the readability index RES (Flesch 1951) of English
texts as possible reflex of synergy. For Spanish texts we evaluated the readability using the perspicuity
index IPSZ (Szigriszt-Pazos 1993). The result of this figures is shown in Table 3.
 Table 3. Entropy H, Spanish perspicuity index IPSZ and English Readability index RES
for groups of speeches written by Nobel laureates and non- Nobel writers.

Group
Language H IPSZ RES
Literature Nobel Laureates
Spanish (19 speeches) 0.8302 63.4
English (37 speeches) 0.7880 56.9

Non-Nobel
Spanish (117 speeches) 0.8518 60.6
English (101 speeches) 0.8381 61.0

There is a clear difference between the entropy of the Nobel laureate speeches and the non-Nobel
speeches. This difference suggests that the more effective structure reached by master writers can be
recognized by the ratio of entropies. This behavior is consistent for Spanish and English. In English,
the P(N) is about 1.06 and in Spanish about 1.02
On the other hand, the change of readability index between Nobel writers and non-Nobel does not
follow the same direction in Spanish and English; while in Spanish the presumably better use of the
written language performed by Nobel laureates could be associated with a higher IPSZ, P(W) = 1.05.
The same change does not show for RES index in English which showed a P(W) based on RES of 0.93

3.7. Music and entropy (Febres & Jaffé 2016).

Why we are delighted by music remains a mystery and an old subject for discussion. Perhaps it is
due to the sound patterns we find in it, that we like music. If so, then the degree of understanding -
predictability- of those patterns should be a source of pleasure and therefore the organization of the
information perceived as music should be related to the degree of success of any musical piece.
Obviously there are factors as personal experience and culture which have an important influence over
the process of interpreting a sequence of sounds. We may or may not like some specific type of music;
but this only supports our belief that pleasure, especially in the case of music, comes after certain
degree of predictability of the sounds. This is not to oversimplify the phenomenon of pleasure and its
causes; in fact sometimes pleasure comes after the surprise of hearing a different path of sounds and
contrasting it with the path we expected before actually listening the piece. But in any case, we think
the popularity of a music piece results from the way the composer organized the sound patterns. For a
large number of symbols, -sounds is this case- more understandable patterns are those which the
sounds are arranged to lower the entropy of the whole set of symbols. For the music most people like,
there is a relatively low entropy and therefore low information structure that we can recall and also
build a clearer map of expectations for the upcoming sounds. We have selected some of the most
recognized music pieces from the Baroque and Classical periods. We compare the entropy of these
pieces with the averages of the entropy for many music pieces in the same periods. Results are
presented in Table 4, showing that the value of P(N) in this case is 1.04
 Table 4. Comparison of entropy for MIDI music of Baroque and Classical periods and
some of the most recognized pieces in the same period.

average average
Composer Period Piece H H H
Vivaldi Baroque Four [Link] 0.6025
Bach Baroque TOCC&[Link] 0.5372 0.5431
Handel Baroque Hallelujah 0.4897 0.5346

Beethoven Classic Symph.9_4 0.4658

0.5262
Mozart Classic Eine Kleine Nachtmusik K525 0.5865

6 composers Baroque 55 pieces 0.5714

0.56335
5 composers Classic 45 pieces 0.5553

H changed according to the estimated number of instruments used for recordings. When comparing
the minimum (a single instrument) with the maximum (orchestra) we find a tendency for the larger
number of instruments to cause an entropy reduction, suggesting an increase of structure for the
musical pieces interpreted by a larger number of instruments. We think this might be the result of the
proper use of the additional degrees of freedom introduced by the addition of instruments. We shall
highlight the fact that adding instruments increases the difficulty of keeping harmonies and
synchronization in polyphonic music, approaching a point in which the risk of losing structure and
beauty of the sound, is an important factor in establishing the limits of the size of the orchestra.
Considering the two cases here analyzed we estimate a P(N) ranging from 1.02 to 1.20 (average 1.1)

Table 5. Comparison of entropy for two pieces of MIDI music interpreted by different
number of instruments.

Composer Period Piece Instrument H

[Link] Romantic [Link] about 10 0.5530
FlyOfBumblebee.Piano1 piano solo 0.7154
FlyOfBumblebee.Piano2 piano solo 0.5972
FlyOfBumblebee.Piano3 piano solo 0.6724
RACHMANINOV 20th Century PianoConcerto.Nro2 about 10 0.5166
PianoConcerto.Nro2 piano solo 0.5291
3.8. Summary of Results:

Table 6. Impact of synergy estimated for several type of systems.

W P(W) P(N)
Example Entropy measure Measure of work Ratio P(W)/P(N)
Ratio W
output Negentropy
Social Complexity in Efficiency in energy
1a 1.70 2.00 0.85
Myrmicinae ants consumption
Social Complexity in Efficiency in energy
1b 2.00 2.20 0.91
Attini ants consumption
Social complexity in Exponent of energy
2 2.50 2.60 0.96
aggregates efficiency function
Scientific Economic
3 2.60 3.10 0.84
development development
Economic
4 Division of labor 2.96 3.00 0.99
efficiency

5a Brain Complexity Polymorphysm 2.70 4.00 0.68

5b Brain Complexity Log Colony size 3.00 4.00 0.75

6a Spanish text Readability 0.93 1.06 0.88

6b English text Readability 1.05 1.02 1.03

7a Entropy in music Popularity 2.00 1.04 1.92

Number of
7b Entropy in music 2.00 1.10 1.82
insytruments

4. Conclusions

In some of the cases examined here, a synergistic increase in work was highly correlated to a
decrease in information entropy of the system. This increase in negentropy was correlated to the
efficiency of work output of the system which also increased strongly, but slightly less than that of
negentropy. This is consistent with our understanding of the second law of thermodynamics,
suggesting an increasing loss of information in the process of transforming it to work.
In some examples, the phenomena showing synergy was not present. That is, for languages such as
English, Spanish and music, the impact of information content on the output was a reduction of
entropy making the system's structure more orderly, creating the conditions that achieve higher
aesthetic efficiencies. This process, however seems to be very different from that observed in known
synergistic phenomena.
References

Corning, P.A., 1983. The Synergism Hypothesis: A Theory of Progressive Evolution. McGraw-Hill,
New York.
Febres G and Jaffe K. Music viewed by its Entropy content: A novel window for comparative analysis.
Submitted
Febres G and Jaffe K. Quantifying structure differences in literature using symbolic diversity and
entropy criteria. Journal of Quantitative Linguistics, 2015 (in press).
Febres G, Jaffe K, Gershenson C. Complexity measures of natural and artificial languages.
Complexity 20: 25-48; doi:10.1002/cplx.21529 2014
Flesch, R. How to test readability. New York: Harpe & Brothers. 1951.
Hausmann R, Hidalgo CA, Bustos S, Coscia M, Chung S, Jimenez J, Simoes A, Yıldırım MA . The
Atlas of Economic Complexity: mapping paths to prosperity 2011.
Jaffe K and Hebling-Beraldo MJ. Oxygen consumption and the evolution of order: Negentropy criteria
applied to the evolution of ants. Experientia 49, 587-592, 1993.
Jaffe K, Perez E. A comparative study of brain morphology in ants. Brain Behav. Evol., 33, 25-33,
1989.
Jaffe K, Rios, A, Florez A. Statistics shows that economic prosperity needs both high scientific
productivity and complex technological knowledge, but in different ways. Interciencia 38: 150-
156, 2013a.
Jaffe K. Extended Inclusive Fitness Theory bridges Economics and Biology through a common
understanding of Social Synergy. ArXiv 2015.
Jaffe K. Quantifying social synergy in insect and human societies. Behavioural Ecology and
Sociobiology 64(11): 1721-1724, 2010.
Jaffe K. Visualizing the Invisible Hand of Markets: Simulating complex dynamic economic
interactions. Intelligent Systems in Accounting, Finance and Management 22(2) 115-132. 2015.
Jaffe K. What is Science, An interdisciplinary view. 2010.
Jaffe, K, Caicedo M, Manzanares M, Rios A, Florez A, Montoreano C, Davila V. Productivity in
physical and chemical science predicts the future economic growth of developing countries better
than other indices for knowledge. PloS One 0066239 2013b
Kolmogorov A, Three Approaches to the Quantitative Definition of Information, Problems Inform.
Transmission, vol. 1, pp. 1-7, 1965.
Laursen K Revealed Comparative Advantage and the Alternatives as Measures of International
Specialization. DRUID Working Papers 98-30, DRUID, Copenhagen Business School,
Department of Industrial Economics and Strategy/Aalborg University. (1998)
Laursen K Revealed Comparative Advantage and the Alternatives as Measures of International
Specialization. DRUID Working Papers 98-30, DRUID, Copenhagen Business School,
Department of Industrial Economics and Strategy/Aalborg University. 1998
Lopez-Ruiz R, Mancini H. L. and Calbet X., "A statistical measure of complexity," Physics Letters A,
vol. 209, no. 5-6, pp. 321-326, 25 12 1995.
Shannon C E, A Mathematical Theory of Communication, The Bell System Technical Journal, vol. 27,
pp. 379–423., 623–656, July, October 1948.
Szigriszt-Pazos, F. Sistemas Predictivos de legibilidad del Mensaje escrito; Fórmula de perspicuidad.
Madrid, Spain: Universidad Complutense de Madrid. 1993.

© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article
distributed under the terms and conditions of the Creative Commons Attribution license
([Link]

View publication stats