MICHELE COSCIA

IT UNIVERSITY OF COPENHAGEN
arXiv:2101.00863v1 [[Link]] 4 Jan 2021

T H E AT L A S F O R T H E
ASPIRING NETWORK
SCIENTIST
Copyright © 2021 Michele Coscia

michele coscia is employed by the it university of copenhagen, rued langgaards vej 7, 2300
copenhagen, denmark

[Link]

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in com-
pliance with the License. You may obtain a copy of the License at [Link]
LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the
License is distributed on an “as is” basis, without warranties or conditions of any kind, either
express or implied. See the License for the specific language governing permissions and limitations under
the License.

First printing, January, 2021

 Contents

1 Introduction 9

I Basics 21

2 Probability Theory 22

3 Basic Graphs 40

4 Extended Graphs 48

5 Matrices 68

II Simple Properties 89

6 Degree 90

7 Paths & Walks 110

8 Random Walks 121

9 Density 132
 4 the atlas for the aspiring network scientist

III Centrality 142

10 Shortest Paths 143

11 Node Ranking 159

12 Node Roles 176

IV Synthetic Graph Models 188

13 Random Graphs 189

14 Understanding Network Properties 199

15 Generating Realistic Data 210

16 Evaluating Statistical Significance 226

V Spreading Processes 236

17 Epidemics 237

18 Complex Contagion 250

19 Catastrophic Failures 265

VI Link prediction 277

20 For Simple Graphs 278

 CONTENTS 5

21 For Multilayer Graphs 295

22 Designing an Experiment 306

VII The Hairball 317

23 Bipartite Projections 318

24 Network Backboning 331

25 Network Sampling 346

VIII Mesoscale 363

26 Homophily 364

27 Quantitative Assortativity 375

28 Core-Periphery 384

29 Hierarchies 396

30 High-Order Dynamics 407

IX Communities 416

31 Graph Partitions 417

32 Community Evaluation 437

 6 the atlas for the aspiring network scientist

33 Hierarchical Community Discovery 457

34 Overlapping Coverage 467

35 Bipartite Community Discovery 484

36 Multilayer Community Discovery 495

X Graph Mining 510

37 Graph Embeddings 511

38 Graph Summarization 524

39 Frequent Subgraph Mining 535

XI Network Distances 549

40 Node Vector Distance 550

41 Topological Distances 566

XII Visualization 580

42 Node Visual Attributes 581

43 Edge Visual Attributes 596

44 Network Layouts 604

 CONTENTS 7

XIII Useful Resources 623

45 Network Science Applications 624

46 Data & Tools 635

47 Glossary 652

48 Most Common Abbreviations 661

Bibliography 665
1
Introduction

Network science is the search for the understanding of the form

of a complex system via the inspection of the relations between its
interacting components.

1.1 Everything is a Network

Understanding the structure and function of complex networks is

among the most beneficial intellectual endeavors because, at some
level, every aspect of reality seems to be made by interconnected
parts. Chemical compounds are atoms linked by bonds. Electronic
machines are made by components connected by wires. Art history
is a carousel of artistic pieces referencing each other. Therefore, if
you understand networks, you understand a little something about
everything. This book aims at providing you such an understanding.
Why is reality so keen in making itself understood in terms of
relations? It is certainly possible that this quality of reality is an
illusion. Our only way to experience the world is via our brain, and
our brain’s reasoning is inextricably polluted by the way it works.
The brain itself is a connected system, at all of its levels. At the
physical level, it is an intertwined web of neurons and synapses. At
its logical functioning level, it is a machine to make inferences – as 1
Karl Friston. The free-energy principle:
shown, e.g., by Friston1 –, which is another way to say connecting a unified brain theory? Nature reviews
stimuli to each other. neuroscience, 11(2):127–138, 2010
And yet, perhaps, the reason why the brain works this way is
because it could not do otherwise to fulfill its purpose. After all,
nervous systems have inherited their objectives from genes. Nervous
systems and genes alike process environmental stimuli to lengthen
survival, they just do so at different speeds and scales – the brain
provides an immediate action-reaction scheme to diagnose imminent
threats, while the gene operates across generations and plays the
long game, as the discovery of DNA and subsequent mechanical
explanations crowned the glorious intellectual achievements of
10 the atlas for the aspiring network scientist

2
AR Wallace. On the tendency of
Wallace2 , Darwin3 , and Huxley4 . And, again, genes are nothing more varieties to depart indefinitely from the
than interacting proteins. It’s interactions all the way down5 . In other original type. J. Linn. Soc. Lond. Zool., 3:
53–62, 1858
words: networks. You really should pay attention to them. 3
Charles Darwin. On the origin of
So we have reached the part of the introduction of any self- species. 1859
respecting network science book when we need to address the ques- 4
Thomas Henry Huxley. Evidence as to
tion: how did we get here? How did we discover that networks were Man’s Place in Nature. London, Williams
and Norgate, 1863
a thing, how to think about them, how to hone our tools to tame 5
Before you start protesting about
the complexity of reality, the same complexity you hopefully now preposterous examples, I will ask you
appreciate from the impressionistic picture I just painted? This is to be patient: you’ll discover in due
time that the brain, ecosystems, and
the time for the creation myth of network science. Which is problem- biological protein interactions are
atic, because creation myths are always a lie, as they try to identify classical examples of complex systems
studied via network analysis.
a discontinuity point in the continuous process of the expansion of
knowledge by accumulation.
But we need to start from somewhere, so what the hell.

1.2 The Network Science Creation Myth

That reality is made of “things” reacting to each other is an elemen-

tary concept. One could trace attempts to reconstruct the rules of
such interactions arbitrarily back in time, to the earliest philosophers,
or to even before that. However, not to stray too much, we should
probably say that the game changer in understanding reality was the
emergence of systematic investigation: the scientific method. As such,
Galilei is probably a good placeholder for the founding father figure
as any.
One of Galilei’s chief contributions was the primordial theory
of relativity. Basically, Galilei was saying that the laws of motion
work the same regardless whether you are standing still or part of an 6
Galileo Galilei. Dialogo sopra i due
object moving at a constant velocity6 . If you throw a ball in a train massimi sistemi del mondo, 1632
carriage, the equation describing its motion is the same, whether the
train is standing still or moving with no acceleration. This principle is
what gives value to all of what, e.g., Newton said about motion: the
“force is mass plus acceleration” concept, F = ma, is a valuable thing
to know only if it always applies, otherwise it’s just wasted ink on a
page.
So Galilei underpins what we nowadays do without a second
thought: describing reality as a series of equations. But what is
an equation? The equal sign in the middle of it should provide a
hint: it is a way to express a relation. When we say F = ma we are
providing an abstract generalization of a potentially infinite series
of relationships; which range from you pushing your shopping cart
around, to me getting punched in the face if I keep rambling about
the nature of reality for too long.
Galileian formulas, in other words, provide a general way to
 introduction 11

quantify relations. In a sense, they unveil that interactions are the

fundamental essence underlying the universe: that which never 7
This is even more fundamental than
interacts does not exist7 . You might point at universal constants such physical existence, because it allows for
as Plank lengths or the gravitational constant as an example of a non- the existence of ideas. At some level,
the Pythagorean theorem “doesn’t
relational quantity that exists. However, such constants only have
exist”: there is no physical “thing”
meaning when plugged into an equation, also know as an interaction. embodying it. But it does interact with
The next fundamental step bringing us closer to networks was our minds, as all ideas do, and thus it
exists.
made by Euler, inspired by Leibniz8 . The fundamental realization is 8
Maximilian Schich. Cultural analysis
that you can split geometry in two complementary parts: the part situs. 2019
that is all about quantifying relationships – i.e. measuring how long
something is –, and the part about qualifying relationships – “the
two right triangles in which you can split a square touch” is true
regardless of how long the square’s edges are. Thus F = ma tells you
about the quantity of all possible relationships between two objects
interacting physically, but you could explode it, and see all possible
Fs as having the qualitative property of having interacted.
Most famously, this insight is what Euler used when solving the 9
Leonhard Euler. Solutio problematis ad
famous Königsberg Bridge Problem9 . The problem asks whether geometriam situs pertinentis. Commen-
it is possible to cross all bridges of Königsberg (now Kaliningrad) tarii academiae scientiarum Petropolitanae,
pages 128–140, 1741
without crossing the same bridge twice. Following Leibniz’s intuition,
Euler realized that the problem didn’t require reasoning about any
quantities: it didn’t depend on the length of the bridges nor the size
of the islands. It was exclusively a problem about the qualitative
relationships between islands and bridges: whether you could use
the latter to reach the former.
The rest, as they say, is history. Reducing islands as nodes and
bridges as edges created the new symbolic language of graphs,
which are the mathematical model we use to understand interacting
complex systems: networks.

1.3 Network Science is a Data Science

If graphs have a three century long history, why does 99% of network
science happened from 1999 on? The reason is probably because,
up until the revolutionary invention of the computer, we only really
had a general intuition about the pervasiveness of networks, without
anything tangible to act upon.
Modern network science is a gift from sociology. Before sociology,
graphs were seen as exact and deterministic mathematical objects,
worthy of exploration through the manipulation of abstract symbols.
Sociologists saw the value in using these mathematical objects – sym-
bols – to investigate a statistical and stochastic reality. This was the
first – fundamental and necessary – explosion in possibilities for a
true network science. There were two problems, though: the trivial
12 the atlas for the aspiring network scientist

one was that we could only collect and manipulate data manually.
More importantly, we still didn’t have a unified language to repre-
sent all of reality as a symbol. The representations we had before
computers were ad-hoc, made only for a specific problem.
The value of the computer is not that it can perform lots of opera-
tions quickly, although that certainly helps. One can prove theorems
and lemmas without computers. Rather, the revolution of the com-
puter is in its parallel development of the usage of symbols to repre-
sent reality. Computers seem to be able to allow you to manipulate
anything: with spreadsheets you can tame problems in logistics, with
XML you can map semantic concepts, with media players you can 10
It has been legendarily said that VLC,
appreciate art and videos10 . And yet, inside computers you just have one of the most popular multimedia
a mass of zeroes and ones. software, can open anything – even a
can of tuna.
Thus, the power of the computer is its ability of seeing everything
– anything – as a symbol. Once everything is a symbol, you can
analyze it mathematically and understand its relations with other 11
One of the facts that never fails to
symbols11 . In this sense, the true revolution of the computer was not blow my mind is the realization that
pioneered by Babbage and von Neumann12 , with their mechanical a piece of software is, after all, just a
very cleverly composed number. Thus
inventions; but rather by Lovelace13 and Wittgenstein14 , with their
you can sum Adobe Photoshop to
logical inventions concerning the manipulation of symbols and their Google Chrome, although the result
interactions. won’t probably make much sense. That
is also why there exist such a thing
Thus, the second half of the XX century was the moment when we as an “illegal number” ([Link]
started connecting symbols together in the same place: the memory [Link]/wiki/Illegal_number)

banks of computers. Ironically, computers facilitated the emergence

12
John von Neumann. First draft of a
report on the edvac. 1945
of even more networks that were latently waiting to express their 13
Ada Lovelace. Notes on menabrea’s
potential. For instance, the invention of the Internet via ARPANET “sketch of the analytical engine in-
and of e-mail codified explicitly the relationships between centers of vented by charles babbage”, 1842
14
Ludwig Wittgenstein. Tractatus
command and of knowledge creation that existed across the world.
logico-philosophicus, 1921
But it was arguably Berners-Lee15 – following in the footsteps of 15
Timothy J Berners-Lee. Information
Bush16 – that truly understood how to piece symbols together in management: A proposal. Technical
report, 1989
computers. 16
Vannevar Bush et al. As we may think.
Once hypertexts birthed the Web, it was just a formality to get The atlantic monthly, 176(1):101–108, 1945
the papers published before modern network science could start.
We finally had both the tools and the data to really understand how
universal networks were, and developing the language we could
use to talk about them. Thus, in a sense, the XX century planted the
seed for the XXI: the great awakening of the world to the pervasive
presence of networks in everything we do. Such awakening could
have not happened – and will not prosper – without data and the
tools to manipulate data.
That is why I say that network science is a data science. We did a
lot of discrete mathematics and graph theory before 1999. But, while
you need to understand those subjects to be a network analyst, you
need a lot more, or else we would have had network science in 1742.
 introduction 13

1.4 What Else is Out There?

My take in connecting network science and data science is by no

means original nor unique. There are many other takes out there,
17
Jacob Levy Moreno, Helen Hall
some similar to mine, some rather different. I do not advocate this as
Jennings, and Ernest Stagg Whitin.
the only book you should read to understand network science and Group method and group psychotherapy.
to be a network analyst. In fact, I advocate the opposite: read diverse Number 5. Beacon House, 1932
18
Stanley Wasserman and Katherine
takes and use diverse structures to think about networks! I see this
Faust. Social network analysis: Methods
book as complementary to those out there. and applications, volume 8. Cambridge
What are these alternatives? As I already mentioned, one should university press, 1994
19
Duncan J Watts and Steven H Stro-
be aware that the first scientists connecting graphs and real world
gatz. Collective dynamics of ‘small-
data came before Berners-Lee – and, arguably, before Bush. It was world’networks. nature, 393(6684):440,
sociologists like Jennings and Moreno17 who invented the idea of 1998
20
Albert-László Barabási and Réka
mapping human interpersonal relationships via “sociograms”. Thus,
Albert. Emergence of scaling in random
one should consider social network analysis as the foundational networks. science, 286(5439):509–512,
approach to network science. In that, Wasserman’s and Faust’s book 1999
21
Guido Caldarelli and Michele Catan-
is a must read18 .
zaro. Networks: A very short introduction,
Then, it must be noted that post-1999 network science arguably volume 335. Oxford University Press,
took off because the innovations from sociology were picked up 2012
22
Guido Caldarelli. Scale-free networks:
by physicists19 , 20 , who inspired other fields with their quest for complex webs in nature and technology.
universal laws. Thus, it is no wonder that there exists a plethora Oxford University Press, 2007
of books21 , 22 , 23 and review articles24 taking the physics angle on Vito Latora, Vincenzo Nicosia, and
23

Giovanni Russo. Complex networks:

network science. Among these, a few certainly stand out. Barabási’s principles, methods and applications.
book25 towers in accessibility and clarity, while Newman’s work is Cambridge University Press, 2017
probably the most complete and in-depth26 . Physicists also have a 24
Claudio Castellano, Santo Fortunato,
and Vittorio Loreto. Statistical physics
good track record in publishing books for the wider audience27 , 28 , of social dynamics. Reviews of modern
not necessarily with a scientific background in mind. physics, 81(2):591, 2009
Given the reliance on computational tools and the need of process- 25
Albert-László Barabási et al. Network
science. Cambridge university press, 2016
ing large amounts of data, the computer science angle should not be
Mark Newman. Networks. Oxford
26

ignored29 . A great favorite of mine combines the computer science university press, 2018b
methodology with applications in economics30 . If you need yet an- 27
Albert-László Barabási. Linked: The
other proof of the breadth of approaches in network science, consider new science of networks, 2003
28
Duncan J Watts. Six degrees: The
that another major book on the topic was authored by a chemist31 . science of a connected age. WW Norton &
The natural question now is: if there are already so many network Company, 2004
science books and they are all great, what is the need of this one 29
Filippo Menczer, Santo Fortunato,
and Clayton A Davis. A First Course in
you’re reading? Is it just for updating with the newest developments Network Science. Cambridge University
in the field? Not really. I hope you noticed that, when presenting the Press, 2020
other network science books, I never introduced them as books writ- 30
David Easley and Jon Kleinberg.
Networks, crowds, and markets: Reasoning
ten by a network scientist. This was not by accident. My impression about a highly connected world. Cambridge
is that these books are aimed at introducing people from a variety of University Press, 2010
disciplines into the skills and tools of network science, rather than 31
Ernesto Estrada. The structure of
complex networks: theory and applications.
examining network science from within. Oxford University Press, 2012
There are now PhD programs for network scientists32 , but they 32
[Link]
are only a handful years old, meaning that there are only a few [Link]/phd
14 the atlas for the aspiring network scientist

graduates coming out of them and they do not have yet the time or
the experience to write a network science book. Worse still, I believe
there are even fewer master and bachelor programs in network
science, if any. This means that every book you can find on network
science really is “something adapted to network science”, with that
something being sociology, physics, computer science, archaeology, or
other.
This book has the – probably overambitious – aim of being really
the first network science book. In other words, the difference between
this and the other books is that this book considers “network science”
not as something one attaches to another discipline, but rather it
is a discipline in itself. People can – and should! – be trained from
scratch in it.
I believe my background is as close as it could be to the right mix
that network analysis requires. I am a digital humanist, a field pio- 33
Roberto Busa. Index thomisticus sancti
neered by Busa33 which focuses on the digital processing of content thomae aquinatis operum omnium
produced by humans. This is to say: the mathematical manipula- indices et concordantiae in quibus
verborum omnium et singulorum
tion and analysis of symbols representing different facets of reality
formae et lemmata cum suis frequentiis
– which are not necessarily mathematical – and their connections. If et contextibus variis modis referuntur.
this sounds familiar, it is because this is the exact characterization of 1974

network science as the key or representing and understanding reality

that I adopted in this introduction.
By the time I started a PhD, there was no one offering it in net-
work science – or digital humanities – so, formally, I am a computer
scientist as well. However, from day one, I immersed myself in net-
work science literature in all its facets – physics, computer science, so-
ciology – armed with my digital humanities toolbox. I designed new
network science algorithms and at the same time studied Dante’s In- 34
Amedeo Cappelli, Michele Coscia,
ferno as a complex network34 ; I scouted for laws in complex systems Fosca Giannotti, Dino Pedreschi, and
whilst fighting Mexican drug traffic35 . Everything I did was in an Salvo Rinzivillo. The social network of
dante’s inferno. Leonardo, 44(3):246–247,
attempt to be an all-round network scientist. And this is the same hat
2011
I’m wearing as the author of this book. 35
Michele Coscia and Viridiana Rios.
If I do so, it is because I am intimately convinced that network Knowing where and how criminal
organizations operate using web
science is truly a special field. This is not only because, as I opened content. In Proceedings of the 21st ACM
this introduction, relations are what I consider being the fundamental international conference on Information and
knowledge management, pages 1412–1421,
fabric of reality. That consideration is only the beginning: it caused
2012
network science to have its complex and multifaceted origin story –
combining all the fields that I’ve been mentioning so far. By birthing
out of many different scientific – and non-scientific! – disciplines,
network science is truly a method to grasp emergence.
Reality is too complex to be properly understood by compart-
mentalized fields. We need compartmentalized fields for exactly this
reason. Maybe you could arrive at describing societies via an expla-
nation of quantum interactions of elementary particles, climbing the
 introduction 15

36
[Link]
long and perilous ladder of fields in descending order of purity36 .
But this is not a given: some phenomena might not be reducible to 37
Philip W Anderson. More is different.
the underlying laws37 . Maybe we do need to toss away a good chunk Science, 177(4047):393–396, 1972
of physics, add a bunch of new tools, to understand this new field
called “chemistry”, because the change in scale causes the emergence
of new phenomena.
We already have a theory for this compartmentalization of the
scientific investigation. This is what Hayek called “division of knowl- 38
Friedrich August Hayek. The use
edge”38 , which is a much more powerful concept than Smith’s classi- of knowledge in society. The American
cal division of labor39 . If I specialize as a chemist and hone my skills economic review, 35(4):519–530, 1945
and tools to that specific task, I can be immensely more productive,
39
Adam Smith. The Wealth of Nations.
1776
because I am outsourcing all other knowledge discovery endeavors to
other specialists. This is how societies grow their pool of knowledge
efficiently. However, the result is that, now, no individual can really
fully grasp a well-rounded picture of reality. The collective society
can, but not its individual components. It is all deformed by the lens
of their specialization.
Network science is the field that gives us an understanding of
“emergence”. If we understand emergence we will know how the
different fields – physics, chemistry, biology, ... – relate and transform
into each other, which is necessary to reconstruct a picture of reality.
Connecting those fields means finding the relations between the
symbols they use – again, that same language returns. This under-
standing can be universal: as mentioned before, it is about quality
relationship. To recall the Frinston example from the very beginning
of this introduction: his theory about how brains work is purely
based on axioms about how information is aggregated in each node
given its neighbors. This is independent of what the nodes actually
are, as long as the conditions hold. You can use his network theory of
intelligence to describe not only how individual brains learn, but how
collectives made of brains learn.
It should now be clear why I consider network science important,
and a truly network science book necessary: it is our best shot at
building a collective understanding of all human knowledge, and
such attempt needs to be approached with the proper humility of
those who are not expert in anything else but gluing together the
pieces created by the real experts.
That said, I don’t want to oversell the importance of network sci-
ence. If what I said is really true, it means that we can represent any
– or at least most – aspects of reality as mathematical symbols and
we can manipulate them with the mathematical tools of computer
and network science. Which means that a complete understanding
of them is necessarily out of reach. Not just because, as Poincaré
would put it, “the head of the scientist, which is only a corner of the
16 the atlas for the aspiring network scientist

40
Henri Poincaré. The foundations of
universe, could never contain the universe entire”40 . Rather, because science. 1913
Gödel taught us that there is a strong bound of what is tractable 41
Kurt Gödel. Über formal unentschei-
in a formal system41 . At some point, even when you represent the dbare sätze der principia mathematica
entirety of reality as interconnected mathematical symbols, you will und verwandter systeme i. Monatshefte
für mathematik und physik, 38(1):173–198,
need to jump out and look at the loops from the outside42 . No book
1931
can really give you a scientific road map on how to do so. 42
Douglas R Hofstadter. Gödel, Escher,
Bach. Harvester press Hassocks, Sussex,
1979
1.5 What is in This Book?

This is all fine and dandy but, at the end of the day, what does this
book contain?
At a general level, it contains the widest possible span of all that
is related to network science that I know. It is the result of twelve
years of experience that I poured on the field. Virtually any concept
that I used or that I simply came to know in these twelve years is
represented in at least a sentence in this book.
As you might expect, this is a lot to include and would not fit a
book, not even a 650+ pages like this one. By necessity, many – if
not all – of the topics included in this book are treated relatively
superficially. I would not say that this book would provide you what
you need to know to be a network scientist. But it would point you 43
Connecting the symbols of network
to what you need to know43 . To borrow from Rumsfeld44 : the book science, maybe?
provides little to no known knowns, but it will provide you with all the 44
[Link]
known unknowns in network science – so that your unknown unknowns Transcripts/[Link]?
TranscriptID=2636
are aligned with those of everyone else. After internalizing this book,
you will know what you don’t know; you will be handed all the tools
you need to ask meaningful questions about network science in 2021.
You can go to the other books or to any other article, and find the
answers.
That is why I decided to call this book an “Atlas”. It is the map
you need to set foot among networks and start exploring. An atlas
doesn’t do the exploration for you, but you can’t explore without an
atlas. This is the book I wished I had twelve years ago.
At a more specific level, the book is divided in thirteen parts.

Part I is about setting the stage for network analysis. It starts with
a quick recap of the main concepts we borrow daily in network
analysis from probability theory. Then it teaches you what a graph
is and how many features to the simple mathematical model were
added over the years, to empower our symbols to tame more and
more complex real world entities. Finally, it pivots perspectives to
show an alternative way of manipulating networks, via matrices
and linear algebra.

Part II is a carousel of all the simplest analyses you can operate

 introduction 17

on a graph. These are either local node properties – how many

connections a node has –, or global network ones – how many
connections on average I have to cross to go from one node to
another. We see that some of these are easy to calculate on the
graph structure, while others are naturally solved by linear algebra
operations. Shifting perspectives is something you need to get
used to, if you want to make it as a network scientist.

Part III uses some of the tools presented in the previous part to build
slightly more advanced analyses. Specifically, it focuses on the
question: which nodes are playing which role in the network?
And: can we say that a node is more important than another? If
you want to answer these questions, you need to relate the entire
network structure to a node, i.e. to use fully what Part II trained
you to do.

Part IV teaches you the main approaches for the creation of synthetic
network data. It explores the main reasons why we want to do
it. Sometimes, it is because we need to test an algorithm and we
need a benchmark. Alternatively, we can use these models to
reproduce the properties of real world networks we investigated in
the previous parts, to see whether we understand the mechanisms
that make them emerge.

Part V starts considering not just network structures, but events on

networks. That is, your nodes and your edges represent real world
entities that actually do something, rather than simply connecting
to each other. Specifically, Part V deals with things spreading
through a network: when a node is affected by something, it
has a chance to pass it to its neighbors via its connections. This
something might be a disease in epidemiological models, or a
behavior in sociological ones.

Part VI evolves the idea of dynamic events on networks. In Part

V, we assumed that the network structure was unchanging: it
was a timeless snapshot of reality and all nodes and connections
are eternally the same. In Part VI, we acknowledge that things
change: if two nodes are not connected today, they might connect
tomorrow. So we investigate which techniques allow us to make a
good guess about which connections we will observe in the future,
on the basis of the ones we are observing now.

Part VII performs another leap: up until now, we mostly inhabited

an ideal world. We assumed we could model phenomena and
cleanly gather insights. Here, we have our first impact with the
real world. How do networks look like when you gather them via
18 the atlas for the aspiring network scientist

experiments and/or observations? Often, they don’t look at all like

the ones from your models. The only expectation that reality meets
is its inability to meet expectations. This part trains you in the art
of cleaning real world data to obtain something passable that can
be fed to your neat theories and analyses.

Part VIII opens the Pandora’s Box of the level of analysis that is
the most interesting and probably the one with which you will
struggle most of the time: the mesoscale. The mesoscale is what
lies between local node properties and global network statistics.
This includes – but is not limited to – questions such as: does my
network have a hierarchical structure? Is there a densely connected
core surrounded by a sparsely connected periphery? Do nodes
consider other nodes’ properties in their decision to connect to
them?

Part IX continues the exploration of the mesoscale. It needs to be

split off Part VIII because there is one mesoscale analysis that
has dominated all other subfields in network science: community
discovery. Community discovery is the network equivalent of what
clustering is for statistics: the task of dividing nodes in a network
into groups. Nodes in the same groups are densely connected to
each other, more so than with nodes in different groups. Or so
people would lead you to believe. The fact that there are literally
thousands of papers proposing different algorithms to tackle this
problem should hint you to the fact that things might not be as
simple.

Part X takes a steep turn into the realm of computer science. It deals
with graph mining: a collection of techniques that allow you to
discover patterns in your graph structure, even if you are not sure
about what these patterns might look like or hint at. It is what we
would call “bottom-up” discovery.

Part XI comprises a relatively new branch of network science that

deals with the problem of estimating network distances. Dif-
ferently from the simple analyses you will find in Part II, these
involve weighted distances between pairs of groups of nodes,
rather than simple distances between pairs of nodes.

Part XII includes a few tips and tricks for an aspect of network sci-
ence that is rarely covered in other books: how to browse/explore
your network data and how to communicate your results. Specifi-
cally, I will show you some best practices in visualizing networks.
I am a visual thinker and, sometimes, patterns and ideas about
those patterns emerge much more clearly when you see them,
 introduction 19

rather than scouting through summary statistics. Moreover, net-

work science papers thrive on visual communication and a good
looking network has an amazing chance of ending up in the cover
of the journal you’re publishing on. It is a mystery to me why you
would not spend some time in making sure that your network
figures are at least of passable quality. Moreover, even if we are all
primed to think dots and lines when it comes to visualize a graph,
you should be aware of the situations in which there are different
ways to show your network.

Part XIII is a final collection of miscellanea that you should need

to know to venture out in the real world of network analysis. It
contains a quick discussion of the applications of network science I
find most interesting, and a repository of tools you might need to
kickstart your career.

1.6 Acknowledgements

You’d be a complete fool if you trusted me to get this amount of

knowledge correct by myself. Any and all scientific endeavors are
only as good as the attention they receive by their peers, both in
terms of building on top of those results, but also in terms of catch-
ing and correcting mistakes. This is in line with what I’ve been
saying in this introduction: network science is too vast and hard for
me to grasp, so I need to rely on the help of the experts from all of
its subfields. It takes a village – or an extensive social network – to
write a textbook. Here I want to thank all those who helped me in
this journey.
The person who stood up tall above everybody was Aaron Clauset,
who gets my most sincere thanks. Aaron is the only one who re-
viewed almost the whole thing, all the 650 friggin’ pages of it. All
while rocking a few months old baby daughter. Aaron is a superhero
and should have everyone’s deep respect.
Another one who went beyond the call of duty was Andres
Gomez-Lievano. Andres and I shared a desk for years and I cher-
ish those as the most fun I had at work. Andres didn’t stop at the
chapters I asked him to review, but deeply commented on the philos-
ophy and framing of this book. I can see in his comments the spark
of the years we spent together.
My other kind reviewers were, in alphabetical order: Alexey
Medvedev, Andrea Tagarelli, Charlie Brummitt, Ciro Cattuto, Clara
Vandeweerdt, Fred Morstatter, Giulio Rossetti, Gourab Ghoshal,
Isabel Meirelles, Laura Alessandretti, Luca Rossi, Mariano Beguerisse,
Marta Sales-Pardo, Matté Hartog, Petter Holme, Renaud Lambiotte,
20 the atlas for the aspiring network scientist

Roberta Sinatra, Yong-Yeol Ahn, and Yu-Ru Lin. All these people
donated hours of their time with no real tangible reward, just to
make sure my book graduated from “incomprehensible mess” to
“almost passable and not misleading”. Thank you.
With their work, some reviewers expressed their intent to support 45
[Link]
charitable organizations. Speciphically, they mentioned TechWomen45
– to support the careers of women in STEM fields –, and Evidence 46
[Link]
Action46 – to expand our de-worming efforts and reaping the sur-
prisingly high societal payoff. You should also consider donating to
them.
If there’s any value in this book, it comes from the hard work of
these people. All the mistakes that remain here are exclusively due to
my ineptitude in properly implementing my reviewers’ valuable com-
ments. I expect there must be many of such mistakes, ranging from
trivial typos and clumsily written sentences, to more fundamental
issues of misrepresentation. If you find some, feel free to drop me an
email to mcos@[Link].
If, for some reason, you only have access to a printed version of
this book – or you found the PDF somewhere on the Internet, know 47
[Link]
that there is a companion website47 with data for the exercises, their
solutions, and – hopefully in the future – interactive visualizations.
Part I

Basics
2
Probability Theory

Before even mentioning networks, I need to lay the groundwork for

a basic understanding of a few concepts necessary to make you a
good network analyst. These concepts are part of probability theory.
Probability theory is the branch of mathematics that allows you to
work with uncertain events. It gives you the tools to make inferences
in cases of uncertainty.
Probability theory is grounded in mathematical axioms. However,
there are different ways to interpret what we really mean with the
term “probability”. With a very broad brush, we can divide the main
interpretations into two camps: the frequentist and the Bayesian.
There are more subtleties to this, but since these are the two main
approaches we will see in this book, there is no reason to make this
picture more complex than it needs to be.
To understand the difference, let’s suppose you have Mr. Frequent
and Mr. Bayes experimenting with coin tosses. They toss a coin
ten times and six out of ten times it turns heads up. Now they ask
themselves the question: what is the probability that, if we toss the
coin, it will turn heads up again?
Mr. Frequent reasons as follows: “An event’s probability is the
relative frequency after many trials. We had six heads after ten tosses,
thus my best guess about the probability it’ll come out as heads is
60%”. Note that Mr. Frequent doesn’t really believe that ten tosses
gave him a perfect understanding of that coin’s odds of landing
on heads. Mr. Frequent knows that he will get the answer wrong a
certain number of times, that is what confidence intervals are for, but
for the sake of this example we need not to go there.
“Hold on a second,” Mr. Bayes says, “Before we tossed it, I exam-
ined the coin with my Coin ExaminerTM and it said it was a fair coin.
Of course my Coin ExaminerTM might have malfunctioned, but that
rarely happens. We haven’t performed enough experiments to say
it did, but I admit that the data shows it might have. So I think the
probability we’ll get heads again is 51%”. Just like Mr. Frequent, also
 probability theory 23

Mr. Bayes is uncertain, and he has a different procedure to estimate

such uncertainty – in this case dubbed “credible intervals” – which
again we leave out for simplicity.
Herein lies the difference between a frequentist and a Bayesian.
For a frequentist only the outcome of the physical experiment mat-
ters. If you toss the coin an infinite number of times, eventually
you’ll find out what the true probability of it landing on heads is.
For a Bayesian it’s all about degrees of beliefs. The Bayesian has a
set of opinions about how the world works, which they call “priors”.
Performing enough new experiments can change these priors, us-
ing a standard set of procedures to integrate new data. However a
Bayesian will never take a new surprising event at face value if it is
wildly off its priors, because those priors were carefully obtained
knowledge coherent with how the world worked thus far.

Frequentist Bayesian
Prior expectation
Figure 2.1: Schematics of the
mental processes used by a fre-
Experiment outcome quentist and a Bayesian when
presented with the results of an
Updating priors
experiment.
Future expectation

Figure 2.1 shows the difference between the mental processes

between a frequentist and a Bayesian.
The default mode for this book is taking a frequentist approach.
However, here and there, Bayesian interpretations are going to pop
up, thus you have to know why we’re doing things that way.
As with many other chapters, my coverage of the subject is the
bare minimum I can get away with. If you want to dive deep into
probability theory, there are good books on the subject you should 1
Willliam Feller. An introduction to
check out1 , 2 . I will attempt, where possible, to give forward refer- probability theory and its applications,
ences to later topics in this book, to let you know why understanding volume 2. John Wiley & Sons, 1968

probability theory is important for a network scientist.

2
Rick Durrett. Probability: theory and
examples, volume 49. Duxbury Press,
1996

2.1 Notation

Probability theory is useful because it gives us the instruments of

talking about uncertain processes. For instance, a process could be
tossing a die. The first important thing is to understand the differ-
ence between outcome and event. An outcome is a single possible result
of the experiment. A die landing on 2 is an outcome. An event is a
set of possible outcomes on which we’re focusing. In our convention,
we use X to refer to outcomes. X is a random variable and it can take
24 the atlas for the aspiring network scientist

many values and forms, and we don’t know which of them it will be
before actually running the process. As for events, they are the focus
of all questions in probability theory: you can sum up probability
theory as the set of instruments that allow you to ask and answer
questions about events (sets of X) such as: “What is the probability
that X, the outcome of the process, is this and/or this but not that
and/or that?”
Mathematically one writes such a question as P( X ∈ S), where S
is a set of the values that X takes in our question. X is an outcome,
X ∈ S is an event. For instance, if we were asking about the event
“will the die land on an even number?”, S = {2, 4, 6}. So, P( X ∈ S)
asks what’s the probability of the “die lands on an even number”
event – or for X to take either of the 2, 4, 6 values. Note that elements
in S are all possible alternatives: if we write P( X ∈ {2, 4, 6}), we’re
asking about the probability of landing on 2 or 4 or 6. If you want to
have the probability of two events happening simultaneously, you
have to explicitly specify it with set notation: P( X ∈ {{2, 4, 6} ∩ {1}})
asks the probability of landing on an even side and on 1 at the same
time.
We also need to consider special questions. For instance, there
is the case in which no event happens: P( X ∈ ∅) (here ∅ refers to
the empty set, a set containing no elements). The converse is also
important: the probability of any event happening. In the case of
the die, there are a total of six possible outcomes. Notation-wise, we
define the set of all possible outcomes as Ω = {1, 2, 3, 4, 5, 6}. So
this is represented as P( X ∈ {1, 2, 3, 4, 5, 6}), or P( X ∈ Ω). Figure
2.2 shows how the mathematical notation corresponds to our visual
intuition.

Figure 2.2: A visual shorthand

for understanding the mathe-
or or
matical notation of probabilities
(left) and the possible outcomes
of the “tossing a die” event.

or or

or

or or

To be more concise, we can skip the explicit reference to the vari-

able X. For instance, we can codify the outcome “the die lands on
3” with the symbol 3. In this way, we can write P(3) to refer to the
probability of the die landing on 3, P({2, 4, 6}) for the probability of
 probability theory 25

landing on an even number, P({2, 4, 6}) ∩ P(1) for landing on an even

number and on 1, P(∅) for the probability of nothing happening, and
P(Ω) for the probability of anything possible happening.

2.2 Axioms

When building probability theory we need to establish a set of ax-

ioms: unprovable and – hopefully – self-evident statements that allow
you to derive all other statements of the theory. Probability theory
rests on three of such axioms.
First, the probability of an event is a non-negative number. Or:
talking about a “negative probability” doesn’t make any sense. Worst
case scenario, an event A is impossible, therefore P( A) = 0 – for
instance, this is the “nothing happens” case from the previous section
when A = ∅. If A is possible, P( A) > 0. In a borderless coin toss,
there are only two possible outcomes: heads (H) or tails (T). The coin
cannot land on the non-existing rim. Thus, the probability of landing
on the rim is zero. It cannot be negative.
Second, certain events occur with probability equal to one. That
is, if A is an absolutely certain event, P( A) = 1. Using the notation
from the previous section: P(Ω) = 1, with Ω = { H, T } for a coin toss.
Note that there isn’t anything magical about the number 1, we could
have said that the maximum probability is equal to 42, π, or “meh”.
It’s just a convenient convention to define your units.
Third, the probability of happening for mutually exclusive events
is the sum of their probabilities, or P({ H, T }) = P( H ) + P( T ). A coin 3
Get those Schrödinger coins out of my
cannot land on heads and tails at the same time3 , thus the probability classroom!
that it lands on heads or tails is the sum of the probability of landing
on heads and the probability of landing on tails.
As a corollary, you can also multiply probabilities. If A and B are
independent events, then P( A) P( B) – their multiplication – tells you
the probability of both events happening. Independent events are
events that have no relation to each other, such as you getting a pro-
motion and the appearance of a new spot on the sun. For dependent
events, you need to take into account this dependence before apply-
ing the multiplication. In a fair die, P(1) = P(2) = 1/6, but we know
that you can’t get a 1 if you are getting a 2, so P(1) P(2) is actually
zero, not 1/36. How to perform this check leads us to the world of
conditional probabilities.

2.3 Conditional Probability

Events do not usually happen in isolation. Things that have hap-

pened in the past might influence what will happen in the future.
26 the atlas for the aspiring network scientist

There is a certain probability that the coin will land on heads: P( H ).

But if I know something happened to the coin before the toss –
maybe I put some weights in it, event W – then the probability of
heads will change. To handle this scenario, we introduce the concept
of “conditional probability”. In our scenario, the notation is P( H |W ).
P( H |W ) is the probability of the coin landing on heads – H – given
that event W happened.
This view of probability is particularly in line with the Bayesian
interpretation, as what you call “prior” is really a synthesis of every-
thing that happened in the past. That is not to say that a frequentist
cannot understand conditional probabilities: they can, they just take
the usual approach of simply observing what happen before/after
something and be done with it.

Ws
Figure 2.3: The baseline prob-
P(H|W) > P(H) ability of H is 0.5. When you
add feet to the coin (W) the
coin is more likely to land
P(H|W) < P(H) on the opposite side. Thus,
P( H |W ) 6= P( H ) and the two
events are not independent –
P(H) = 0.5
P(H|W) = P(H) unless you add feet on both
sides as in the bottom example.

Conditional probabilities enable you to make a nice set of infer-

ences. Figure 2.3 shows the most basic ones. If you measure P( H |W ),
you can figure out what event W did to the coin. If P( H |W ) > P( H ),
it means that adding the weight to the coin made it more likely to
land on heads. P( H |W ) < P( H ) means the opposite: your coin is
loaded towards tails. The P( H |W ) = P( H ) case is equally interesting:
it means that you added the weight uniformly and the odds of the 4
Note that here I’m talking about
coin to land on either side didn’t change. statistical independence, which is not
This is a big deal: if you have two events and this equation, then the same as causal independence. Two
events could be statistically dependent
you can conclude that the events are independent – the occurrence of without being causally dependent. For
one has no effect on the occurrence of the other4 . This should be your instance, the number of US computer
science doctorates is statistically
starting point when testing a hypothesis: the null assumption is that
dependent with the total revenue of
there is no relation between an outcome (landing on heads) and an arcades ([Link]
intervention (adding a weight). “Unless,” Mr. Bayes says, “You have a spurious-correlations). This is what
the mantra “correlation does not imply
strong prior for that to be the case.” causation” means: correlation is mere
Reasoning with conditional probabilities is trickier than you might statistical dependence, causation is
causal dependence, and you shouldn’t
expect. The source of the problem is that, typically, P( H |W ) 6=
confuse one with the other. You should
P(W | H ), and often dramatically so. Suppose we’re tossing a coin to check [Pearl and Mackenzie, 2018] to
settle a dispute. However, I brought the coin and you think I might delve deeper into this.
 probability theory 27

be cheating. You know that, if I loaded the coin, the probability of it

landing on heads is P( H |W ) = 0.9. However, you can’t see nor feel
the weights: the only thing you can do is tossing it and – presto! – it
lands on heads. Did I cheat?
Naively you might rush and say yes, there’s a 90% chance I
cheated. But that’d be wrong, because the coin already had a 50%
chance of landing on heads without any cheating. Thus P( H |W ) 6=
P(W | H ), and what you really want to estimate is the probability I
cheated given that the coin landed on heads: P(W | H ). How to do so,
using what you know about coins (P( H )) and what you know about
my integrity (P(W )), is the specialty of Bayes’ Theorem.

2.4 Bayes’ Theorem

Bayes’ Theorem is an almost magical formula that allows you to

estimate the probability of an event based on your priors. Keep-
ing the example of cheating on a coin toss, we want to estimate the
probability I cheated and rigged the coin so it lands on heads after
we tossed it and it indeed landed on heads – in mathematical nota-
tion: P(W | H ). To do so, you need to have priors. You need to know:
what’s the probability of heads for all coins in the world (whether
they are rigged or not, P( H )), what’s the probability I rigged the
coin (P(W )), and what is the probability of obtaining heads on a
rigged coin (P( H |W )). Without further ado, here’s one of the most
important formulas in human history:

P ( H |W ) P (W )
P (W | H ) = .
P( H )

Figure 2.4: The table on the left

X = shows the occurrence of all pos-
P( ) = 5/9 P( | ) = 4/5 P( ) P( | ) = 4/9 sible events: red circles (5), blue
borders (6), red circles with
blue borders (4) and neither (2).
X =
P( ) = 6/9 P( | ) = 4/6 P( ) P( | ) = 4/9

P( ) P( | )
P( ) P( | ) = P( ) P( | ) P( | ) =
P( )

Figure 2.4 shows a graphical proof of the theorem. When trying

to derive P(W | H ) P( H ), we realize that’s identical to P( H |W ) P(W ),
from which Bayes’ theorem follows.
28 the atlas for the aspiring network scientist

I already told you that I’m a pretty good coin rigger (P( H |W ) =
0.9). For the sake of the argument, let’s assume I’m a very honest
person: the probability I cheat is fairly low (P(W ) = 0.3).
Now, what’s the probability of landing on heads (P( H ))? P( H ) is
trickier than it appears, because we’re in a world where people might
cheat. Thus we can’t be naive and saying P( H ) = 0.5. P( H ) is 0.5
if rigging coins is impossible. It’s more correct to say P( H | − W ) =
0.5: a non rigged coin (if W didn’t happen, which we refer to as
−W) is fair and lands on heads 50% of the times. The real P( H ) is
P( H | − W ) P(−W ) + P( H |W ) P(W ). In other words: the probability
of the coin landing on heads is the non rigged heads probability if I
didn’t rig it (P( H | − W ) P(−W )) plus the rigged heads probability if I
rigged it (P( H |W ) P(W )).
The probability of not cheating P(−W ) is equal to 1 − P(W ). This
is because cheating and non cheating are mutually exclusive and
either of the two must happen. Thus we have Ω = {W, −W }. Since
P(Ω) = 1 and P(W ) = 0.3, the only way for P(W, −W ) to be equal to
1 is if P(−W ) = 0.7.
This leads us to: P( H ) = P( H | − W ) P(−W ) + P( H |W ) P(W ) =
0.5 × 0.7 + 0.9 × 0.3 = 0.62. Shocking.
The aim of Bayes’ theorem is to update your prior about me
cheating (P(W )) given that, suspiciously, the toss went in my favor
(P(W ) → P(W | H )). Plugging in the numbers in the formula:

0.9 × 0.3
P (W | H ) = = 0.43.
0.62
A couple of interesting things happened here. First, since the event
went in my favor, your prior about me possibly cheating got updated.
Specifically, the event became more likely: from 0.3 to 0.43. Second,
even if my success probability after cheating is very high, it is still
more likely that I didn’t cheat, because your prior about my lack of
integrity was low to begin with.
This second aspect is absolutely crucial and it’s easy to get it
wrong in everyday reasoning. The textbook example is the cancer
diagnosing machine. Let’s say that 0.1% of people develop a cancer,
and we have this fantastic diagnostic machine with an accuracy of
99.9%: the vast majority of people will be diagnosed correctly (posi-
tive result for people with cancer and negative for people without).
You test yourself and the test is positive. What’s your chance of hav-
ing cancer? 99.9% accuracy is pretty damning, but before working on
your last will, you apply Bayes’ Theorem:

0.999 × 0.001
P(C |+) = = 0.5.
0.999 × 0.001 + 0.001 × 0.999
The probability you have cancer is not 99.9%: it’s a coin toss! (Still
 probability theory 29

5
Of course, in the real world, if you
bad, but not that bad).5 took the test it means you thought you
The real world is a large and scary environment. Many different might have cancer. Thus you were not
drawn randomly from the population,
things can alter your priors and have different effects on differ- meaning that you have a higher prior
ent events. The way a Bayesian models the world is by means of a that you had cancer. Therefore, the test
Bayesian network: a special type of network connecting events that is more likely right than not. Bayes’
theorem doesn’t endorse carelessness
influence each other. Exploring a Bayesian network allows you to when receiving a bad news from a very
make your inferences by moving from event to event. I talk more accurate medical test.
about Bayesian networks in Section 4.6.

2.5 Stochasticity

Colloquially, a stochastic process is one or more random variables

that change their values over time. The quintessential stochastic pro-
cess is Brownian motion. Brown observed very light pollen particles
on water changing directions, following a stochastic path that seemed
governed purely by randomness. Interestingly, this problem was later 6
Albert Einstein. Über die von der
solved by Einstein in one of his first contributions to science6 , work- molekularkinetischen theorie der
ing off important prior work7 . He explained the seemingly random wärme geforderte bewegung von in
ruhenden flüssigkeiten suspendierten
changes of direction as the result of collision between the pollen and
teilchen. Annalen der physik, 4, 1905
water molecules jiggling in the liquid. 7
Louis Bachelier. Théorie de la spécu-
When you have a stochastic process, there is an almost infinite set lation. In Annales scientifiques de l’École
normale supérieure, volume 17, pages
of results. The pollen can follow potentially infinite different paths.
21–86, 1900
When you observe an actual grain, you obtain only one of those
paths. The observed path is called a realization of the process. Figure
2.5 shows three of such realizations, which should help you visualize
the intrinsic randomness of the change of direction.

Figure 2.5: Three realizations of

a Brownian stochastic motion
on a two dimensional plane.

(a) (b) (c)

Whenever you encounter the word “stochastic” in this book or in a
paper, we’re referring to a process governed by these dynamics. For
instance, a stochastic matrix is a matrix whose rows and/or columns
sum up to one. We call it stochastic, because such matrices are rou-
tinely used to describe stochastic processes. By having their rows to
sum to one, you can interpret each entry of the row as the probability
of its corresponding event. The row in which you are tells you the
current state of the process, the column tells you the next possible
state, and the cell value tells you the probability of transitioning to
each of the next possible states (column) given the current state (row).
30 the atlas for the aspiring network scientist

In other words, it is the probability of one possible realization of a

single step in a stochastic process. In network science, you normally
have stochastic adjacency matrices, which are the topic of Section 5.3.

Figure 2.6: A right stochastic

matrix.

8
Specifically, it is a right stochastic
matrix8 .
Figure 2.6 is a stochastic The rows tell you your current matrix: the rows sum to one, although
there’s a bit of rounding going on. In a
state and the columns tell you your next state. If you are in the first
left stochastic matrix, the columns sum
row, you have a 30% probability of remaining in that state (the value to one.
of the cell in the first row and first column is 0.3). You have a 20%
probability of transitioning to state two (first row, second column), 8%
probability of transitioning to state three, and so on.

2.6 Markov Processes

It should be clear now that, even if the next state is decided by a

random draw, a stochastic process isn’t necessarily uniformly ran-
dom. In Brownian motion, the next position is determined by your
previous position as well as a random kick. This observation is at the
basis of a fundamental distinction between three flavors of stochas-
tic processes, which are the most relevant for network science. The
three flavors are: Markov processes, non-Markov processes, and
higher-order Markov processes.
In a Markov process, the next state is exclusively dependent on
the current state and nothing else. No information from the past is
used: only the present state matters. That is why a Markov process
is usually called “memoryless”. The stochastic process I described
when discussing Figure 2.6 is a typical Markov process. The only
thing we needed to know to determine the next state was the current
state: in which row are we?
The classical Markov process in network science is the random
walk. A random walker simply chooses the next node it wants to
occupy, and its options are determined solely by the node it is cur-
rently occupying. Rather surprisingly, random walks are one of the
most powerful tools in network science and have been applied to
practically everything. I’m going to introduce them properly in Chap-
ter 8, but they will pop up throughout the book – for instance, in
 probability theory 31

community discovery (Part IX) and in network sampling (Chapter 25).

Figure 2.7 shows an example of a random walk. As you can see, we
start from the leftmost node. From that state, reaching the two right-
most ones is impossible because the nodes are not connected. Only
when you transition to another state, new states become available.

Figure 2.7: A random walk. The

green arrows show the state
transitions.

A bit more formally, let’s assume you indicate your state at time
t with Xt . You want to know the probability of this state to be a
specific one, let’s say x. x could be the id of the node you visit at the
t-th step of your random walk. If your process is a Markov process,
the only thing you need to know is the value of Xt−1 – i.e. the id of
the node you visited at t − 1. In other words, the probability of Xt = x
is P( Xt = x | Xt−1 = xt−1 ). Note how Xt−2 , Xt−3 , ..., X1 aren’t part of
this estimation. You don’t need to know them: all you care about is
X t −1 .
On the other hand, a non-Markov process is a process for which
knowing the current state doesn’t tell you anything about the next
possible transitions. For instance, a coin toss is a non-Markov process.
The fact that you toss the coin and it lands on heads tells you nothing
about the result of the next toss – under the absolute certainty that
the coin is fair. The probability of Xt = x is simply P( Xt = x ): there’s
no information you can gather from your previous state.
Finally, we have higher-order Markov processes. Higher-order
means that the Markov process now has a memory. A Markov pro-
cess of order 2 can remember one step further in the past. This means
that, now, P( Xt = x | Xt−1 = xt−1 , Xt−2 = xt−2 ): to know the probabil-
ity of Xt = x, you need to know the state value of Xt−2 as well as of
Xt−1 . More generally, P( Xt = x | Xt−1 = xt−1 , Xt−2 = xt−2 , ..., Xt−m =
xt−m ), with m ≤ t.
The classical network examples of a higher order Markov pro-
cess is the non-backtracking random walk (Figure 2.8). In a non-
backtracking random walk, once you move from node u to node
v, you are forbidden to move back from v to u. This means that,
once you are in v, you also have to remember that you came from u.
Higher order Markov processes are the bread and butter of higher
order network problems, which is the topic of Chapter 30.
32 the atlas for the aspiring network scientist

Figure 2.8: A non-backtracking

random walk. The green arrows
show the state transitions.

No backtracking!

2.7 Probability Distributions

When you perform an experiment, or observe a stochastic process,

you have many possible outcomes. Sometimes, you’re not interested
in the probability of a specific outcome. Sometimes, you want to
know the probability of all possible outcomes, to determine which
is more likely and what your expectations should be. This is the
task of a probability distribution. A probability distribution is a
function that, for each outcome in the set of all possible ones (called
the “sample space”), tells you the probability of that outcome to
occur. Figure 2.9 shows a vignette on how to interpret a plot showing
you a probability distribution.

Figure 2.9: A probability dis-

Probability of outcome i tribution, connecting every
possible outcome in the sample
Probability

space (x axis) to a probability (y

Probability Distribution
axis).

Outcome i

Sample Space

There are two possible cases in your sample space: either it con-
tains discrete finite outcomes, or it contains effectively infinite contin-
uous ones. The first case is, for instance, a coin toss. There are only
two possible outcomes: heads or tails. The second case is when, for
instance, you’re measuring something that can take any real value as
an outcome. In the first discrete case, we call the probability distri-
bution a “probability mass function”. In the second case, we call it a
“probability density function”.
There are a few common probability distributions you should be
familiar with – Figure 2.10 shows some stylized representations of
each of them –:
 probability theory 33

Figure 2.10: A stylized repre-

sentation of the most common
probability distributions you’ll
encounter as a network scientist.
Uniform Normal
Solid lines show continuous
probability distributions, while
dots show discrete ones – note
that some distributions can be
both (dotted lines).
Binomial Poisson Hypergeometric

Exponential Power Law Lognormal

• Uniform: in this probability distribution each event is equally

likely. This distribution can be both discrete or continuous. In
the discrete case, if you have n possible events, each occurs with
probability p = 1/n. You get a uniform distribution if you look at
the id of a ball extracted from a urn, where all balls in the urn are
identified by a distinct progressive number without gaps.

• Normal (or Gaussian): this is the typical distribution of indepen-

dent continuous random variables. The classical example is the
distribution of people’s heights: most people are of average height,
and larger and larger deviations from the average are increasingly
– and predictably – unlikely.

• Binomial: this is a discrete distribution, in which you make n

experiments, each with success probability p, and you calculate
the probability of having n0 successes. For instance, the probability
of extracting 1, 2, 3, ... white balls from an urn containing 50 white
and 50 black balls – each time putting the ball you extracted back
into the urn. You can approximate the binomial distribution with
a normal one, in fact one might call the binomial distribution the
discrete equivalent of the continuous normal distribution.

• Poisson: this is another discrete distribution, which is the number

of successes in a given time interval, assuming that each success
arrives independently from the previous ones. For instance, the
number of meteorites impacting on the moon each year will
distribute following a Poisson. If the event’s probability is high,
this might look similar to a binomial, but a less common event or
34 the atlas for the aspiring network scientist

a shorter observation interval usually “cut off” the left side of the
distribution. Interestingly, many examples commonly mentioned
for explaining a Poisson distribution (number of admittances in a
hospital in an hour, number of email written in an hour, and so on)
aren’t actually Poisson distributions, because people making those 9
Albert-Laszlo Barabasi. The origin
examples fail to account for the burstiness of human behavior9 . of bursts and heavy tails in human
dynamics. Nature, 435(7039):207, 2005
• Hypergeometric: this is yet another discrete probability function.
It is very similar to a binomial distribution. If the binomial de-
scribed the success odds in an extraction-with-replacement urn
game, the hypergeometric describes the more common case of
extraction-without-replacement: when you extract a ball from the
urn, you don’t put it back. It is mathematically less tractable, but
much more useful. This is used especially for the task of network
backboning (Chapter 24).
• Exponential: the exponential distribution is the continuous version
of the geometric distribution. The geometric distribution tells you
the probability that the first success of an experiment happens
at trial n. Each experiment is independent and the probability
of a success will determine how steep the distribution is. One
cool property of the exponential/geometric distribution is that is
doesn’t “age”: it doesn’t matter how many trials you did so far –
the likelihood of a success doesn’t change. The classical example
of an “ageless” process is atomic decay: the half life of carbon14 is
the same regardless for how long it had been decaying. To go from
2kg to 1kg takes the exact same amount of time as going from 1kg
to 500 grams.
• Power law: a power law can be both a discrete or a continuous
distribution. It describes the relationship between two quantities,
where a relative change in one corresponds to a proportional
relative change in the other (so the second variable changes as a
power of the first). An example of discrete power law is Zipf’s
10
Mark EJ Newman. Power laws,
law10 . We’ll see more than you want to know about power laws
pareto distributions and zipf’s law.
when talking about fitting degree distributions in Section 6.3. Contemporary physics, 46(5):323–351,
2005b
• Lognormal: a lognormal distribution is the distribution of a con-
tinuous random variable whose logarithm follows a normal dis-
tribution – meaning the logarithm of the random variable, not of
the distribution. This is the typical distribution resulting from the
multiplication of two independent random positive variables. If
you throw a dozen 20-sided dice and multiply the values of their
faces up, you’d get a lognormal distribution. It’s very tricky to tell
this distribution apart from a power law, as we’ll see.
Sometimes, rather than looking at the probability mass/density
function, it’s more useful to look at their cumulative versions. In
 probability theory 35

Figure 2.11: A stylized repre-

sentation of a few cumulative
distributions. Same legend as
Figure 2.10.
Uniform Normal Exponential Power Law

practice, you want to ask yourself what is the probability of – say –

having x or fewer successes. Each distribution changes in predictable
ways, as Figure 2.11 shows.
For instance, a uniform cumulative distribution is a straight line,
because each event adds the same value to the cumulative sum. A cu-
mulative normal distribution assumes the familiar “S” shape. In the
power law case, as we’ll see, we actually want to see the complement
of the cumulative distribution (1 - CDF). This is, interestingly, also a
power law.

2.8 Mutual Information

Mutual Information (MI) is a key concept in information theory. It

is a measure of how related two random variables are. You can see
it as a sort of special correlation. Formally, it is a measure of the
mutual dependence between the two variables. What that means
is that MI quantifies how much information you obtain about one
variable if you know the value of the other variable. This “amount of
information” is usually measured in bits.
To understand MI, we need to take a quick crash course on in-
formation theory, which starts with the definition of information
entropy. It is a lot to take in, but we will extensively use these con-
cepts when it comes to link prediction and community discovery in
Parts VI and IX, thus it is a worthwhile effort.
Consider Figure 2.12. The figure contains a representation of a
vector of six elements that can take three different values. The first
thing we want to know is how many bits of information we need to
encode its content. We can be smart and use the shortest codes for
the elements that appear most commonly, in this case the red square.
Every time we see a red square, we encode it with a zero. If we don’t
see a red square, we write a one, which means that we need to look
at a second bit to know whether we saw a blue or a green square. If
it was a blue square, we write a zero, if it was green we write another
one. With these rules, we can encode the original vector using nine
bits, i.e. we use 1.5 bits per element.
36 the atlas for the aspiring network scientist

X X
Figure 2.12: A simple example
=0 0 to understand information
9 bits entropy. From left to right: the
0
vector x has six elements tak-
6 values
0 ing three different values. We

= 10 10 =9/6 can encode each value with

a sequence of zeros and ones.
10 = 1.5 bits/value Doing so allows us to transmit
x’s six elements using nine bits
= 11 11 of information. This means that
the number of bits per value is
This is close – but not exactly – the definition of information 1.5.
entropy. In information entropy, the probability of an event to occur
is weighted by its logarithm. Consider flipping a coin. Once you
know the result, you obtain one bit of information. That is because
there are two possible events, equally likely with a probability p of
50%.
Generalizing to all possible cases, every time an event with prob-
ability p occurs, it gives you − log2 ( p) bits of information for... 11
The amount of information of an
reasons11 . So, the total information of an event is the amount event is a function that only depends
of information you get per occurrence times the probability of on the probability p of the event to
happen, e.g. i a = f ( p a ) for event a. If
occurrence: − p log2 ( p). Summed over all possible events i in x: we have two events, a and b, happening
Hx = − ∑ pi log2 ( pi ), which is information entropy – how many bits with probability p a and pb , the event
i c defined as a and b happening has
you need to encode the occurrence of all events. probability pc = p a pb . Now, each
Mutual information is defined for two variables. As I said, it is the event also gives you an amount of
information, namely i a and ib . When
amount of information you gain about one by knowing the other, or c happens, it means that both a and b
how much entropy knowing one saves you about the other. Consider happened, thus you got both pieces of
information, or ic = i a + ib . What we
Figure 2.13. It shows the relationship between two vectors, x and y.
just said can be rewritten as f ( pc ) =
Note how y has equally likely outcomes: each color appears three f ( p a ) + f ( pb ), given the equation at
times. However, if we observe a green square in x, we know with the beginning. Since pc = p a pb , then
we can also rewrite the equation as
100% confidence that the corresponding square in y is going to be f ( p a pb ) = f ( p a ) + f ( pb ). The only
purple. This means that, knowing x’s values gives us information function f that we can possibly plug
into this equation maintaining it true
about y’s value. Mathematically speaking, mutual information is the
is the logarithm. Since probabilities
amount of information entropy shared by the two vectors. are lower than 1, the logarithm would
It would take − log2 (1/3) ∼ 1.58 bits to encode y on its own (it is be lower than zero, which would be
nonsense – you cannot get negative
a random coin with three sides). However, knowing x’s values makes information. Thus we take the negated
you able to use the inference rules we see in Figure 2.13. Those logarithm: i a = − log( p a ).
rules are helpful: note how their confidence is almost always higher
than 33%, which is the probability you’d have to get y’s color right
without any further information. The rules will save you around 0.79
bits, which is x and y’s mutual information.
The exact formulations of mutual information is similar to the
formula of entropy:
 probability theory 37

X Y Figure 2.13: An illustration

of what mutual information
If then (66%) means for two vectors. Vector
y has equal occurrences for its
If then (33%) values (there is one third prob-
ability of any colored square).
If then (50%) However, if we know the value
of x we can usually infer the
If then (50%) corresponding y value with a
higher than chance confidence.
If then (100%)

!
pij
MIxy = ∑ ∑ pij log pi p j
,
j∈y i ∈ x

where pij is the joint probability of i and j. Even if I don’t give you
the full explanation, you can hopefully see what’s going on here. The
meat is comparing the joint probability of i and j happening with
what you would expect if i and j were completely independent. If
they are, then pij = pi p j , which means we take the logarithm of one,
which is zero and everything collapses into zero if that’s always the
case. Any time the happening of i and j is not independent, we add
something to the mutual information. That something is the number
of bits we save.

2.9 Summary

1. Probability theory gives you the tools to make inferences about un-
certain events. We often use a frequentist approach, the idea that
an event’s probability is approximated by the aggregate past tests
of that event. Another important approach is the Bayesian one,
which introduces the concept of priors: additional information that
you should use to adjust your inferences.

2. Probabilities are non-negative estimates. The set of all possible

outcomes has a probability sum of one. Summing two proba-
bilities tells you the probability of either of two independent
outcomes to happen.

3. The conditional probability P( A| B) tells you the probability of an

outcome A given that you know another outcome B happened,
and the two are not independent. Bayes’ Theorem allows you to
infer P( A| B) from P( B| A).
38 the atlas for the aspiring network scientist

4. When we track the change over time of one or more random

variables, we’re observing a stochastic process. Markov processes
are stochastic processes whose status exclusively depends on the
status of the system in the previous time step.

5. To describe the probability of all outcomes of an event you can

draw its probability distribution. A cumulative probability dis-
tribution, instead, tells you the probability of observing a set of
outcomes, up to a certain point.

6. Mutual information is a measure of how related two random

variables are. It tells you how many bits of information you gain
about the status of one variable by knowing the other.

2.10 Exercises

1. Suppose you’re tossing two coins at the same time. They’re loaded
in different ways, according to the table below. Calculate the
probability of getting all possible outcomes:

p1 ( H ) p2 ( H ) H-H H-T T-H T-T

0.5 0.5
0.6 0.7
0.4 0.8
0.1 0.2
0.3 0.4

2. 60% of the emails hitting my inbox is spam. You design a phe-

nomenal spam filter which is able to tell me, with 98% accuracy,
whether an email is spam or not: if an email is not spam, the sys-
tem has a 98% probability of saying so. The filter knows 60% of
emails are spam and so it will flag 60% of my emails. Suppose
that, at the end of the week, I look in my spam box and see 963
emails. Use Bayes’ Theorem to calculate how many of those 963
emails in my spam box I should suspect to be non-spam.

3. You’re given the string: “OCZ XJMMZXO VINRZM”. Each letter

follows a stochastic Markov process with the rules expressed by 12
[Link]
the table at [Link] Caesar_cipher

txt. Follow the process for three steps and reconstruct the correct
answer. (Note, this is a Caesar cipher12 with shift 7 applied three
times, because the Caesar cipher is a Markov process).

4. Draw the probability mass function and the cumulative distribu-

tion of the following outcome probabilities:
 probability theory 39

Outcome p
1 0.1
2 0.15
3 0.2
4 0.21
5 0.17
6 0.09
7 0.06
8 0.02
5. How many bits do we need to independently encode v1 and v2
from [Link]
How much would we save in encoding v1 if we knew v2 ?
3
Basic Graphs

3.1 Simple Graphs

Every story should start from the beginning and, in this case, in 1
John Adrian Bondy, Uppaluri Siva Ra-
the beginning was the graph1 , 2 , 3 , 4 . To explain and decompose the machandra Murty, et al. Graph theory
elements of a graph, I’m going to use the recurrent example of social with applications, volume 290. Citeseer,
1976
networks. The same graph can represent different networks: power 2
Douglas Brent West et al. Introduction
grids, protein interactions, financial transactions. Hopefully, you can to graph theory, volume 2. Prentice hall
effortlessly translate these examples into whatever domain you’re Upper Saddle River, 2001
going to work.
3
Reinhard Diestel. Graph theory.
Springer Publishing Company, Incorpo-
Let’s start by defining the fundamental elements of a social net- rated, 2018
work. In society, the fundamental starting point is you. The person. 4
Jonathan L Gross and Jay Yellen. Graph
Following Euler’s logic that I discussed in the introduction, we want theory and its applications. CRC press,
2005
to strip out the internal structure of the person to get to a node. It’s
like a point in geometry: it’s the fundamental concept, one that you
cannot divide up into any sub-parts. Each person in a social network
is a node – or vertex; in the book I’ll treat these two terms as syn-
onyms. We can also call nodes “actors” because they are the ones
interacting and making events happen – or “entities” because some-
times they are not actors: rather than making things happen, things
happen to them. “Actor” is a more specific term which is not an
exact synonym of “node”, but we’ll see the difference between the 5
The understatement of the century.
two once we complicate our network model just a bit5 , in Section 4.2.
To add some notation, we usually refer to a graph as G. V indi-
cates the set of G’s vertices. Since V is the set of nodes, to refer to the
number of nodes of a graph we use |V | – some books will use n, but
I’ll try to avoid it. Throughout the book, I’ll tend to use u and v to
indicate single nodes.
So far, so good. However, you cannot have a society with only one
individual. You need more than one. And, once you have at least two
people, you need interactions between them. Again, following Euler,
for now we forget about everything that happens in the internal struc-
ture of the communication: we only remember that an interaction is
 basic graphs 41

taking place. We will have plenty of time to make this model more
complicated. The most common terms used to talk about interactions
are “edge”, “link”, “connection” or “arc”. While some texts use them
with specific distinctions, for me they are going to be synonyms, and
my preferred term will always be “edge”. I think it’s clearer if you
always are explicit when you refer to special cases: sure, you can
decide that “arc” means “directed edge”, but the explicit formula “di-
rected edge” is always better than remembering an additional term,
because it contains all the information you need. (What the hell are
“directed edges”? Patience, everything will be clear)
Again, notation. E indicates the set of G’s edges and | E| is the
number of edges – some books will use m as a synonym for | E|.
Usually, when talking about a specific edge one will use the notation
(u, v), because edges are pairs of nodes – unless we complicate the
graph model. Now we have a way to refer to the simplest possible
graph model: G = (V, E), with E ⊆ V × V. A graph is a set of nodes
and a set of edges – i.e. node pairs – established among those nodes.

Figure 3.1: (a) A node. (b) An

edge. (c) A simple graph.

(a) (b)
(c)
We’re going to talk about how to visualize networks much later
in Part XII, but it’s better to introduce some visual elements now,
otherwise how are we supposed to have figures before then? Nodes
are usually represented as dots, or circles – Figure 3.1(a). Edges are
lines connecting the dots – Figure 3.1(b). When all you have is nodes
and edges, then you have a simple graph – Figure 3.1(c). Note that
these visual elements are basic and widely used, but they are by no
means the only way to visualize nodes and edges. In fact, when you
want to convey a message about a network of non-trivial size, they’re
usually not a great idea.
The first famous graph in history is Euler’s Königsberg graph,
which I show in Figure 3.2. In the graph, each node represents a land-
mass and each edge represents a bridge connecting two landmasses.
Since there were multiple bridges connecting the same landmasses,
we have multiple edges between the same two nodes. This seemingly
trivial fact is actually rather interesting.
“Simple graph” means literally simple: nothing more than nodes
and edges – no attributes, no possibility of having multiple connec-
tions between the same two nodes. If you add any special feature,
it’s not a simple graph any more. Under this light, we discover that
Euler’s first graph wasn’t simple after all. It allowed for parallel
42 the atlas for the aspiring network scientist

Figure 3.2: The famous Königs-

berg graph Euler used.

edges: multiple edges between the same two nodes. Euler’s first
graph was a multigraph. That’s so non-standard that we’re not even
going to talk about it in this chapter: you’ll have to wait for the next
one, specifically for Section 4.2.
In our simple graph we also assume there are no self loops, which
are edges connecting a node with itself. Our assumption is that we
aren’t psychopaths: everybody is friend with themselves, so we don’t
need to keep track of those connections.

Figure 3.3: (a) A simple graph.

(b) Its complement.

(a) (b)

When you have a simple graph G, you can derive a series of

special simple graphs related to G. For instance, you can derive the
complement of G. This is equivalent to remove all of the original
edges of G, and then connect all the unconnected pairs of nodes in G.
Figure 3.3 shows an example.
This operation basically views G as a set of edges. If you take this
perspective, you can define many operations on graphs as sets. Given
two graphs G 0 and G 00 , you can calculate their union, intersection,
and difference, which are the union, intersection, and difference
of their edge sets. The union of G 0 and G 00 is a graph G that has
the edges found in either G 0 or G 00 ; the intersection of G 0 and G 00
is a graph G that has the edges found in both G 0 and G 00 ; and the
difference of G 0 and G 00 is a graph G that has the edges found in G 0 6
Hassler Whitney. Congruent graphs
and the connectivity of graphs. American
but not in G 00 .
Journal of Mathematics, 54(1):150–168,
Another important special graph is the line graph6 , 7 . The line 1932
graph of G represents each of G’s edges as a node. Two nodes in the 7
József Krausz. Démonstration nouvelle
d’une théoreme de whitney sur les
line graph are connected to each other if the edges they represent
réseaux. Mat. Fiz. Lapok, 50(1):75–85,
are attached to the same node in G. Figure 3.4 shows an example of 1943
 basic graphs 43

3-5
3 Figure 3.4: (a) A graph. (b) Its
4-5 linegraph version.
2 4 3-4
5 2-4

1-4
1
1-2
(a)
(b)

line graph. We’ll see how you can use line graphs to represent high
order relationships in Chapter 30, to find overlapping communities in
Chapter 34, and to estimate similarities between networks in Chapter
41.

3.2 Directed Graphs

Simple graphs are awesome. They allow you to represent a surpris-

ing variety of different complex systems. But they are not the end all
be all of network theory. There are many phenomena out there that
cannot be simply reduced to a set of nodes interacting through a set
of edges. Sometimes you really need to complicate stuff. In this and
in the next section we’re going to see two ways to enhance the simple
graph models. They all work in the same way: by slightly modifying
the definition of an edge. We’re going to see even more fundamental
reworkings of the simple graph model in Chapter 4.
The first thing we will do is realizing that not all relations are
reciprocal. The fact that I consider you as my friend – and I do, my
dear reader – doesn’t necessarily mean that you also consider me as
your friend – wow, this book is getting very real very fast. We can
introduce this asymmetry in the graph model. So far we said that
(u, v) is an edge and we implicitly assumed that (u, v) is the same as 8
Frank Harary, Robert Zane Norman,
(v, u). Directed graphs8 are graphs for which (u, v) 6= (v, u). and Dorwin Cartwright. Structural
models: An introduction to the theory of
directed graphs. Wiley, 1965

Figure 3.5: (a) A directed edge.

(b) A directed graph.

(a)

(b)
In a message passing game, (u, v) – or u → v – means that node
u can pass a message to node v, but v cannot send it back to u. Di-
rected graphs introduce all sorts of intricacies when it comes to
finding paths in the network, a topic we’re going to dissect in Chap-
44 the atlas for the aspiring network scientist

ter 7. The use of the arrow is a pretty straightforward metaphor to

indicate the lack of reciprocity: relationships flow from the tail to the
head of the arrow, not the other way around. It comes as no surprise,
then, that we can use the arrow to indicate a directed edge, as we
do in Figure 3.5(a). If E contains directed edges, we have a directed
graph – Figure 3.5(b). Note that, in a directed graph (or digraph)
representation, an edge always has a direction. If two nodes have
a reciprocal relationship, convention dictates that we draw two di-
rected edges pointing in the two directions, to make such relationship
explicit.
In general, when you have a directed graph G, you can calculate
its reverse graph by flipping all edge directions.

3.3 Weighted Graphs

Another way to make edges more interesting is realizing that two

connections are not necessarily equally important in the network.
One of the two might be much stronger than another. We are all
familiar with the concepts of “best friend” and “Facebook friend”.
One is a much more tightly knit connection than the other. 9
Alain Barrat, Marc Barthelemy, Ro-
For this reason, we can add weights to the edges9 , 10 . A weight is mualdo Pastor-Satorras, and Alessandro
simply an additional quantitative information we add to the connec- Vespignani. The architecture of complex
weighted networks. Proceedings of the
tion. A possible notation could be (u, v, w): nodes u and v connect national academy of sciences, 101(11):
to each other with strength w. So our graph definition now changes 3747–3752, 2004a
to G = (V, E, W ), where W is our set of possible weights. W is prac-
10
Mark EJ Newman. Analysis of
weighted networks. Physical review E, 70
tically always included in the set of real numbers, and most of the (5):056131, 2004a
times in the set of real positive numbers – i.e. W ⊆ R+ . Now we have
a weighted graph.

Figure 3.6: A weighted graph.

3 5 The weight of the edge dictates
3 its label and thickness.
1 2

Graphically, we usually represent the weight of a connection

either by labeling the edge with its value, or simply by using visual
elements such as the line thickness. I do both things in Figure 3.6.
Edge weights can be interpreted in two opposite ways, depending
on what the network is representing. They can be considered the
proximity between the two nodes or their distance. This can and
will influence the results of many algorithms you’ll apply to your
graph, so this semantic distinction matters. For instance, if you’re
 basic graphs 45

looking for the shortest path (see Chapter 10) in a road network, your
edge weight could mean different things. It could be a distance if it
represents the length of the trait of road: longer traits will take more
time to cross. Or it can be a proximity: it could be the throughput of
the trait of road in number of cars per minute that can pass through
it – or the number of lanes. If the weight is a distance, the shortest
path should avoid high edge weights. If the weight is a proximity, it
should do its best to include them.
To sum up, “proximity” means that a high weight makes the
nodes closer together; e.g. they interact a lot, the edge has a high
capacity. “Distance” means that a high weight makes the nodes
further apart; e.g. it’s harder or costly to make the nodes interact.
Edge weights don’t have to be positive. Nobody says nodes should
be friends! Examples of negative edge weights can be resistances in
electric circuits or genes downregulating other genes. This observa-
tion is the beginning of a slippery slope towards signed networks,
which is a topic for another time (namely, for Section 4.2, if you want
to jump there).
The network in Figure 3.6 has nice integer weights. In this case,
the edge weights are akin to counts. For instance, in a phone call
network, it could be the number of times two people have called each
other. Unfortunately, not all weighted networks look as neat as the
example in Figure 3.6. In fact, most of the weighted networks you
might work with will have continuous edge weights. In that case,
many assumptions you can make for count weights won’t apply – for
instance when filtering connections, as we will see in Chapter 24.
By far, the most common case is the one of correlation networks.
In these networks, the nodes aren’t really interacting directly with
one another. Instead, we are connecting nodes because they are
similar to each other, for some definition of similarity. For instance, 11
Boris C Bernhardt, Zhang Chen,
we could connect brain areas via cortical thickness correlations11 , or Yong He, Alan C Evans, and Neda
currencies according to their exchange rate12 , or correlating the taxa Bernasconi. Graph-theoretical analysis
reveals disrupted small-world organi-
presence in different biological communities13 . zation of cortical thickness correlation
These cases have more or less the same structure. I provide an ex- networks in temporal lobe epilepsy.
Cerebral cortex, 21(9):2147–2157, 2011
ample in Figure 3.7. In this case, nodes are numerical vectors, which 12
Takayuki Mizuno, Hideki Takayasu,
could represent a set of attributes, for instance. We calculate a corre- and Misako Takayasu. Correlation
lation between the vectors, or some sort of attribute similarity – for networks among currencies. Physica A:
Statistical Mechanics and its Applications,
instance mutual information (Section 2.8). We then obtain continuous
364:336–342, 2006
weights, which typically span from −1 to 1. And, since every pair of 13
Jonathan Friedman and Eric J Alm.
nodes have a similarity (because any two vectors can be correlated, Inferring correlation networks from
genomic survey data. PLoS computational
minus extremely rare degenerate cases), every node is connected to
biology, 8(9):e1002687, 2012
every other node. So, when working with similarity networks, you
will have to filter your connections somehow, a process we call “net-
work backboning” which is far less trivial that it might sound. We
46 the atlas for the aspiring network scientist

will explore it in Chapter 24.

A A Figure 3.7: A typical workflow

0.77
B 0.04

0.36
B
for correlation networks: (left to
right) from nodes represented
C
C
as some sort of vectors, to a
graph with a similarity measure
as edge weigth.

3.4 Summary

1. The mathematical representation of a network is the graph: a

collection of nodes – the actors of the network –, and edges – the
connections among those actors. In a simple graph, no additional
feature can be added, and there is only one edge between a pair of
nodes.

2. If connections are not symmetric, meaning that if you consider

me your friend I don’t necessarily consider you mine, then we
have directed graphs. In directed graphs, edges have a direction so
relations flow one way, unless there is a reciprocal edge pointing
back.

3. In weighted graphs, connections can be more or less strong, indi-

cated by the weight of the edge, a numerical quantity. It doesn’t
have to be a discrete number, nor necessarily positive: for instance
in correlation networks you can have negative continuous weights.

4. Weights can have two meanings: proximity – the edge is the

strength of a friendship –, or distance – the edge is a cost to pay
to cross from one node to another. Different semantics imply that
some algorithms’ results should be interpreted differently.

3.5 Exercises

1. Calculate |V | and | E| for the graph in Figure 3.1(c).

2. Mr. A considers Ms. B a friend, but she doesn’t like him back. She
has a reciprocal friendship with both C and D, but only C con-
siders D a friend. D has also sent friend requests to E, F, G, and
H but, so far, only G replied. G also has a reciprocal relationship
with A. Draw the corresponding directed graph.

3. Draw the previous graph as undirected and weighted, with the

weight being 2 if the connection is reciprocal, 1 otherwise.
 basic graphs 47

4. Draw a correlation network for the vectors in [Link]

eu/exercises/3/4/[Link], by only drawing edges with positive
weights, ignoring self loops.
4
Extended Graphs

The world of simple graphs is... well... simple. The only thing com-
plicating it a bit so far was adding some information on the edges:
whether they are asymmetric – meaning (u, v) 6= (v, u) – and whether
they are strong or weak. Unfortunately, that’s not enough to deal
with everything reality can throw your way. In this chapter, we
present even more graph models, which go beyond the simple addi-
tion of edge information.

4.1 Bipartite Graphs

Figure 4.1: (a) A simple graph

representing a social network
with no additional constraints.
(b) A cop-thief bipartite net-
work: nodes can be either a cop
or a thief, and cops can only
(a) (b) catch (connect to) thieves.
So far we have talked about networks in which relations run be-
tween peers: nodes are all the same to us. But nodes might belong
to two distinct classes. And connections can only be established be-
tween members of different classes. Figure 4.1 provides an example.
In a social network without node attributes nor types, anybody can
be friend with anybody else and there isn’t much to distinguish two
nodes. But if we want to connect cops with the thieves they catch,
then we are establishing additional connecting rules. Thieves don’t
catch each other. And, hopefully, cops aren’t thieves. Another ex-
ample could be connecting workers to the buildings hosting their
offices.
Stripping down the model to a minimum, bipartite networks are
networks in which nodes must be part of either of two classes (V1
 extended graphs 49

Figure 4.2: (a) An example of

a bipartite network. (b) An
example of a tripartite network.

(b)
(a)

and V2 ) and edges can only be established between nodes of unlike 1

Armen S Asratian, Tristan MJ Denley,
type1 , 2 . Formally, we would say that G = (V1 , V2 , E), and that E can and Roland Häggkvist. Bipartite
only contain edges like (v1 , v2 ), with v1 ∈ V1 and v2 ∈ V2 . Figure graphs and their applications, volume 131.
Cambridge University Press, 1998
4.2(a) depicts an example. 2
Jean-Loup Guillaume and Matthieu
Bipartite networks are used for countless things, connecting: Latapy. Bipartite structure of all complex
countries to the products they export3 , hosts to guest in symbiotic networks. Information processing letters,
90:Issue–5, 2004
relationships4 , users to the social media items they tag5 , bank-firm 3
César A Hidalgo and Ricardo Haus-
relationships in financial networks6 , players-bands in jazz7 , listener- mann. The building blocks of economic
band in music consumption8 , plant-pollinators in ecosystems9 , and complexity. Proceedings of the national
academy of sciences, 106(26):10570–10575,
more. You get the idea. Bipartite networks pop up everywhere. 2009
However, by a curious twist of fate, the algorithms able to work di- 4
Brian D Muegge, Justin Kuczynski,
rectly on bipartite structures are less studied than their non-bipartite Dan Knights, Jose C Clemente, Antonio
González, Luigi Fontana, Bernard
counterparts. For instance, for every community discovery algorithm Henrissat, Rob Knight, and Jeffrey I
that works on bipartite networks you have a hundred working on Gordon. Diet drives convergence
in gut microbiome functions across
non-bipartite ones. The distinction is important, because the standard
mammalian phylogeny and within
assumptions of non-bipartite community discovery do not hold in humans. Science, 332(6032):970–974, 2011
bipartite networks, as we will see in Part IX. 5
Renaud Lambiotte and Marcel Aus-
loos. Collaborative tagging as a tripartite
Why would that be the case? Because practically everyone who
network. In International Conference on
works on bipartite networks projects them. Most of the times, you Computational Science, pages 1114–1117.
are interested only in one of the two node types. So you create a Springer, 2006
6
Luca Marotta, Salvatore Micciche,
unipartite version of the network connecting all nodes in V1 to each
Yoshi Fujiwara, Hiroshi Iyetomi,
other, using some criteria to make the V2 count. The trivial way is Hideaki Aoyama, Mauro Gallegati,
to connect all V1 nodes with at least a common V2 neighbor. This and Rosario N Mantegna. Bank-firm
credit network in japan: an analysis
is so widely done and so wrong that I like to call it the Mercator of a bipartite network. PloS one, 10(5):
bipartite projection, in honor of the most used and misunderstood e0123079, 2015
map projection of all times. We’ll see in Chapter 23 why that’s not
7
Pablo M Gleiser and Leon Danon.
Community structure in jazz. Advances
very smart, and the different ways to do a better job. in complex systems, 6(04):565–573, 2003
Why stopping at bipartite? Why not go full n-partite? For instance, 8
Renaud Lambiotte and Marcel Aus-
a paper I cited before actually builds a tri-partite network (Figure loos. Uncovering collective listening
habits and music genres in bipartite
4.2(b) depicts an example): users connect to the social media they networks. Physical Review E, 72(6):066107,
tag and with the tags they use. However, the gains you get from 2005
50 the atlas for the aspiring network scientist

a more precise data structure quickly become much lower than 9

Colin Campbell, Suann Yang, Réka
the added complexity of the model. Even tripartite networks are Albert, and Katriona Shea. A network
model for plant–pollinator community
a rarity in network science. A couple of examples are the recipe- assembly. Proceedings of the National
ingredient-compound structure of the flavor network10 , or the aid Academy of Sciences, 108(1):197–202, 2011
organization-country-issue structure11 .
10
Yong-Yeol Ahn, Sebastian E Ahnert,
James P Bagrow, and Albert-László
Barabási. Flavor network and the
principles of food pairing. Scientific
4.2 Multilayer Graphs reports, 1:196, 2011
11
Michele Coscia, Ricardo Hausmann,
and César A Hidalgo. The structure and
One-to-One dynamics of international development
assistance. Journal of Globalization and
Traditionally, network scientists try to focus on one thing at a time. If Development, 3(2):1–42, 2013a
they are interested in analyzing your friendship patterns, they will
choose one network that closely approximates your actual social 12
Mikko Kivelä, Alex Arenas, Marc
relations and they will study that. For instance, they will download Barthelemy, James P Gleeson, Yamir
a sample of the Facebook graph. Or they will analyze tweets and Moreno, and Mason A Porter. Multilayer
networks. Journal of complex networks, 2
retweets.
(3):203–271, 2014

Figure 4.3: (a) A simple graph.

(b) A multigraph, with multiple
edges between the same node
pairs. (c) A simple multilayer
network, a multigraph where
each edge has a type (repre-
sented by the edge’s color).
13
Manlio De Domenico, Albert Solé-
Ribalta, Emanuele Cozzo, Mikko Kivelä,
Yamir Moreno, Mason A Porter, Sergio
Gómez, and Alex Arenas. Mathematical
(a) (b) formulation of multilayer networks.
(c) Physical Review X, 3(4):041022, 2013
14
Stefano Boccaletti, Ginestra Bianconi,
However, in some cases, that is not enough to really grasp the Regino Criado, Charo I Del Genio, Jesús
phenomenon one wants to study. If you want to predict a new con- Gómez-Gardenes, Miguel Romance,
Irene Sendina-Nadal, Zhen Wang, and
nection on Facebook, something happening in another social media
Massimiliano Zanin. The structure
might have influenced it. Two people might have started working in and dynamics of multilayer networks.
the same company and thus first connected on Linkedin, and then Physics Reports, 544(1):1–122, 2014

became friends and connected on Facebook. Such scenario could not

15
Michele Berlingerio, Michele Coscia,
Fosca Giannotti, Anna Monreale, and
be captured by simply looking at one of the two networks. Network Dino Pedreschi. Multidimensional
scientists invented multilayer networks12 , 13 , 14 , 15 , 16 , 17 to answer this networks: foundations of structural
analysis. WWW, 16(5-6):567–593, 2013a
kind of questions. 16
Mark E Dickison, Matteo Magnani,
There are two ways to represent multilayer networks. The simpler and Luca Rossi. Multilayer social networks.
is to use a multigraph. Remember Euler’s parallel edges in the Cambridge University Press, 2016
Königsberg graph from Figure 3.2? That’s what makes a multigraph.
17
Matteo Magnani and Luca Rossi.
The ml-model for multi-layer social
Differently from a simple graph (Figure 4.3(a)), in which every pair networks. In ASONAM, pages 5–12.
of nodes is forced to have at most one edge connecting them, in a IEEE, 2011
 extended graphs 51

multigraph (Figure 4.3(b)) we allow an arbitrary number of possible

connections.
If that is all, there wouldn’t be much difference between multi-
graphs and weighted networks. If all parallel edges are the same, we
could have a single edge with a weight proportional to the number
of connections between the two nodes. However, in this case, we can
add a “type” to each connection, making them qualitatively different:
one edge type for Facebook, one for Twitter, one for Linkedin (Figure
4.3(c)), etc.
In practice, every edge type – or label – represents a different layer
of the network. A pair of nodes can establish a connection in any
layer, even at the same time. Each layer is a simple graph. In this
book – and generally in computer science – the most used notation
to indicate a multilayer network is G = (V, E, L). V and E are the
sets of nodes and edges, as usual. L is the set of layers – or labels.
An edge is now a triple (u, v, l ) ∈ E, with u, v ∈ V as nodes, and
l ∈ L as the layer. This might seem similar to the notation used for
weighted edges – which was (u, v, w). The key difference is that w is
a quantitative information, while l is a qualitative one: a class, a type.
We can make the two co-exist in weighted multigraphs, by specifying
an edge as (u, v, l, w).
The model that we introduce in Figure 4.3(c) is but the simplest
way to represent multilayer networks. This strategy rests on the
assumption that there is a one-to-one node mapping between the
layers of the network. In other words, the entities in each layer are
always the same: you are always you, whether you manage your
Facebook account or your Linkedin one. Such simplified multilayer
networks are sometimes called multiplex networks.
Studies have shown how layers in a multiplex network could be 18
Alessio Cardillo, Jesús Gómez-
complementary18 . This means that a single layer in the network Gardenes, Massimiliano Zanin, Miguel
might not show the typical statistical properties you would expect Romance, David Papo, Francisco
Del Pozo, and Stefano Boccaletti.
from a real world network – the types of things we’ll see in this book.
Emergence of network features from
However, once you stack enough layers one on top of the other, the multiplexity. Scientific reports, 3:1344,
resulting network does indeed conform to our structural expectations. 2013

In other words, multilayer networks have emerging properties.

Multiplex networks, don’t necessarily cover all application scenar-
ios: sometimes a node in one layer can map to multiple nodes – or
none! – in another. This is what we turn our attention to next.

Many-to-Many

To fix the insufficient power of multiplex networks to represent true

multilayer systems we need to extend the model. We introduce the
concept of “interlayer coupling”. In this scenario, the node is split
52 the atlas for the aspiring network scientist

into the different layers to which it belongs. In this case, your identity
includes multiple personas: you are the union of the “Facebook
you”, the “Linkedin you”, the “Twitter you”. Figure 4.4(a) shows
the visual representation of this model: each layer is a slice of the
network. There are two types of edges: the intra-layer connections –
the traditional type: we’re friends on Facebook, Linkedin, Twitter –,
and the inter-layer connections. The inter-layer edges run between
layers, and their function is to establish that the two nodes in the
different layers are really the same node: they are coupled to – or
dependent on – each other.

Figure 4.4: The extended mul-

tilayer model. Each slice rep-
resents a different layer of the
network. Dashed grey lines rep-
resent the inter-layer coupling
connections. (a) A multilayer
network with trivial one-to-one
coupling. (b) A multilayer net-
work with complex interlayer
(a) (b)
coupling.
Formally, our network is G = (V, E, L, C ). V is still the set of
nodes, but now we split the set of edges in two: E is the set of classi-
cal edges, the intra-layer one – connections between different people
on a platform –; and C is the set of coupling connections, the inter-
layer one, denoting dependencies between nodes in different layers.
Having a full set of coupling connections enables an additional
degree of freedom. We can now have nodes in one layer expressing
coupling with multiple nodes in other layers. In our social media
case, we are now allowing you to have multiple profiles in one plat-
form that still map on your single profile in another. For instance,
you can run as many different Twitter accounts as you want, and they
are still coupled with your Facebook account. To get a visual sense
on what this means, you can look at Figure 4.4(b).
This new freedom comes to a cost. While in the one-to-one map-
ping it is easy to identify a node among layers, because all identities
of a node are concentrated in a single point in a layer, in the many-to-
many coupling that is not true any more. So we introduce the term
“actor”, which is the entity behind all the multiple identities across
layers and within a layer. In practice, the actor is a connected compo-
nent (see Section 7.4), when only considering inter-layer couplings as
the possible edges. If my three Twitter profiles all refer to the same
person, with maybe two Flickr accounts and one Facebook profile,
all these identities belong to the same actor: me. Figure 4.5 should
 extended graphs 53

Figure 4.5: An actor in a many-

to-many coupled multilayer
Same Actor
network. The orange outline
surrounds nodes with coupling
edges connecting them.

clarify this definition.

Note that there can be many ways to establish inter-layer cou-
plings between the different nodes belonging to the same actor.
As far as I know, when analyzing networks people usually use a
“cliquey” approach: every node belonging to the same actor is con-
nected to every other node as, for instance, in Figure 4.6(a). This
effectively creates a clique of inter-layer coupling connections – for
more information about what a clique is, see Section 9.3.

Figure 4.6: Different coupling

flavors for your multilayer net-
works. Showing a network with
a single actor and a single node
per actor per layer (represented
(a) Clique (b) Chain (c) Star by the border-colored polygon).
I color the coupling edges in
However, this is usually too cumbersome to draw. So, for illustra- purple.
tion purposes, the convention is to use a “chainy” approach (Figure
4.6(b)): you sort your layers somehow, and you simply place a line
representing your coupling connections piercing through the lay-
ers. We don’t really have to stop there. One could imagine using a
“starry” approach: defining one layer as the center of the system, and
connecting all nodes belonging to that actor to the node in the cen- 19
Jacopo Iacovacci, Zhihao Wu, and
Ginestra Bianconi. Mesoscopic structures
tral layer. To see what I mean, look at Figure 4.6(c). Using different reveal the network between the layers of
coupling flavors can be useful for computational efficiency: when you multiplex data sets. Physical Review E, 92
start having dozens or even hundreds of layers, creating cliques of (4):042806, 2015
20
Gregorio D’Agostino and Antonio
layers can add a significant overhead. Scala. Networks of networks: the last
Such many-to-many layer couplings are often referred to in the frontier of complexity, volume 340.
literature as “networks of networks”, because each layer can be seen Springer, 2014
21
Dror Y Kenett, Matjaž Perc, and
as a distinct network, and the interlayer couplings are relationships Stefano Boccaletti. Networks of
between different networks19 , 20 , 21 . networks–an introduction. Chaos,
Solitons & Fractals, 80:1–6, 2015
54 the atlas for the aspiring network scientist

Aspects
Do you think we can’t make this even more complicated? Think
again. These aren’t called “complex networks” by accident. To fully
generalize multilayer networks, adding the many-to-many interlayer
coupling edges is not enough. To see why that’s the case, consider
the fact that, up to this point, I considered the layers in a multilayer
network as interchangeable. Sure, they represent different relation-
ships – Facebook friendship rather than Twitter following – but they
are fundamentally of the same type. That’s not necessarily the case:
the network can have multiple aspects.
For instance, consider time. We might not be Facebook friends
now, but that might change in the future. So we can have our mul-
tilayer network at time t and at time t + 1. These are two aspects of
the same network. All the layers are present in both aspects and the
edges inside them change. Another classical example is a scientific
community. People at a conference interact in different ways – by
attending each other talks, by chatting, or exchanging business cards
– and can do all of those things at different conferences. The type of
interaction is one aspect of the network, the conference in which it
happens is another.
I can’t hope to give you here an overview of how many new things
this introduces to graph theory. So I’m referring you to a specialized 22
Ginestra Bianconi. Multilayer Networks:
book on the subject22 . Structure and Function. Oxford University
Press, 2018

Signed Networks
Signed networks are a particular case of multilayer networks. Sup-
pose you want to buy a computer, and you go online to read some
reviews. Suppose that you do this often, so you can recognize the
reviewers from past reviews you read from them. This means that
you might realize you do not trust some of them and you trust others.
This information is embedded in the edges of a signed network: there
are positive and negative relationships.
Signed networks are not necessarily restricted to either a single
positive or a single negative relationship – e.g. “I trust this person”
or “I don’t trust this person”. For instance, in an online game, you
can have multiple positive relationships like being friend or trading
together; and multiple reasons to have a negative relationship, like
fighting each other, or putting a bounty on each other heads.
A key concept in signed networks is the one of structural balance.
Since this is mostly related to the link prediction problem, I expand
on this in Section 21.1.
Positive and negative relationships have different dynamics. For
instance, in a seminal study looking at interactions between players
 extended graphs 55

23
Michael Szell, Renaud Lambiotte,
in a massively multiplayer online game23 , the authors studied the dif- and Stefan Thurner. Multirelational
ferent degree distributions (Section 6.2) for each type of relationship. organization of large-scale social
networks in an online world. Proceedings
They uncovered that positive relationships have a marked exponen-
of the National Academy of Sciences, 107
tial cutoff, while negative relationships don’t. You’ll become more (31):13636–13641, 2010
accustomed to what a degree distribution is and all the lingo related
to it in Chapter 6. For now, the meaning of what I just said is: there is
a limit to the number of people you can be friends with, but there is
no limit to the number of people that can be mad at you.

4.3 Hypergraphs

Figure 4.7: (a) Classical Graph.

(b) Hypergraph.

(a) (b)
In the classical definition, an edge connects two nodes – the gray
lines in Figure 4.7(a). Your friendship relation involves you and your
friend. If you have a second friend, that is a different relationship.
There are some cases in which connections bind together multiple
people at the same time. For instance, consider team building: when
you do your final project with some of your classmates, the same
relationship connects you with all of them. When we allow the
same edge to connect more than two nodes we call it a hyperedge –
the gray area in Figure 4.7(b). A collection of hyperedges makes a
24
Vitaly Ivanovich Voloshin. Introduction
hypergraph24 , 25 .
to graph and hypergraph theory. Nova
Graphs with simplicial complexes26 , 27 are related to hypergraphs. Science Publishers Hauppauge, 2009
The difference between the two is that simplicial complexes have a 25
Alain Bretto. Hypergraph theory: An
strong emphasis on geometry. Simplicial complex analysis specializes introduction. Mathematical Engineering.
Cham: Springer, 2013
in systems with many-to-many interactions that are embedded in 26
Vsevolod Salnikov, Daniele Cassese,
real physical spaces. For instance, you can use simplicial complexes and Renaud Lambiotte. Simplicial
to study groups of people interacting at a conference, because social complexes and complex systems.
European Journal of Physics, 40(1):014001,
groups will form in the two dimensional floor of the conference 2018
building. 27
Jakob Jonsson. Simplicial complexes of
graphs, volume 3. Springer, 2008
To make them more manageable, we can put constraints to hy-
peredges. We could force them to always contain the same number
of nodes. In a soccer tournament, the hyperedge representing a
team can only have eleven members: not one more nor one less, be-
28
Shenglong Hu and Liqun Qi. Alge-
cause that’s the number of players in the team. In this case, we call
braic connectivity of an even uniform
the resulting structure a “uniform hypergraph”, and have all sorts hypergraph. Journal of Combinatorial
of interesting properties28 . In general, when simply talking about Optimization, 24(4):564–579, 2012
56 the atlas for the aspiring network scientist

hypergraphs we have no such constraint. 29

Source: I tried once.
It is difficult to work with hypergraphs29 . Specialized algorithms
to analyze them exist, but they become complicated very soon. In
the vast majority of cases, we will transform hyperedges into simpler
network forms and then apply the corresponding simpler algorithms.
There are two main strategies to simplify hypergraphs. The first is
to transform the hyperedge into the simple edges it stands for. If the
hyperedge connects three nodes, we can change it into a unipartite
network in which all three nodes are connected to each other. In the
project team example, the new edge simply represents the fact that
the two people are part of the same team. The advantage is a gain
in simplicity, the disadvantage is that we lose the ability to know the
full team composition by looking at its corresponding hyperedge: we
need to explore the newly created structures.

Figure 4.8: The two ways to con-

vert a hyperedge into simpler
forms. A hyperedge connecting
three nodes can become a tri-
angle (top right), or a bipartite
network (bottom right).

The second strategy is to turn the hypergraph into a bipartite

network. Each hyperedge is converted into a node of type 1, and the
hypergraph nodes are converted into nodes of type 2. If nodes are
connected by the same hyperedge, they all connect to the correspond-
ing node of type 1. In the project team example, the nodes of type
1 represent the teams, and the nodes of type 2 the students. This is
an advantageous representation: it is simpler than the hypergraph,
but it preserves some of its abilities, for instance being able to re-
construct teams by looking at the neighbors of the nodes of type 1.
However, the disadvantage with respect to the previous strategy is
that there are fewer algorithms working for bipartite networks than
with unipartite networks.
Figure 4.8 provides a simple example on how to perform these two
conversion strategies on a simple hyperedge connecting three nodes.
When it comes to notation, the network is still represented by the
classical node and edge sets: G = (V, E). However, the E set now
 extended graphs 57

is special: its elements are not forced to be tuples any more. They
can be triples, quartuplets, and so on. For instance, (u, v, z) is a legal
element that can be in E, with u, v, z ∈ V.

4.4 Dynamic Graphs

Most networks are not crystallized in time. Relationships evolve: they

are created, destroyed, modified over time by all parties involved.
Every time we use a network without temporal information on its
edges, we are looking at a particular slice of it, that may or may not
exist any longer.

Figure 4.9: An example of dy-

namic network. Each figure
represents the same network,
observed at different points in
time.
(a) t = 1 (b) t = 2 (c) t = 3 (d) t = 4
For many tasks, this is ok. For others, the temporal information is
a key element. Imagine that your network represents a road graph.
Nodes are intersections, and edges are stretches of the street con-
necting them. Roadworks might cut off a segment for a few days. If
your network model cannot take this into account, you would end up
telling drivers to use a road that is blocked, creating traffic jams and
a lot of discomfort. That is why you need dynamic – or temporal –
30
Soon-Hyung Yook, Hawoong Jeong,
A-L Barabási, and Yuhai Tu. Weighted
networks30 , 31 , 32 , 33 , 34 . evolving networks. Physical review letters,
Consider Figures 4.9(a) to (d) as an example. Here, we have a 86(25):5835, 2001
social network. People are connected only when they are actually
31
Alain Barrat, Marc Barthélemy, and
Alessandro Vespignani. Weighted
interacting with each other. We have four observations, taken at evolving networks: coupling topology
four different time intervals. Suppose that you want to infer if these and weight dynamics. Physical review
letters, 92(22):228701, 2004b
people are part of the same social group – or community. Do they? 32
Petter Holme and Jari Saramäki.
Looking at each single observation would lead us to say no. In each Temporal networks. Physics reports, 519
time step there are individual that have no relationships to the rest of (3):97–125, 2012
the group. Adding the observations together, though, would create
33
Vincenzo Nicosia, John Tang, Cecilia
Mascolo, Mirco Musolesi, Giovanni
a structure in which all nodes are connected to each other. Taking Russo, and Vito Latora. Graph metrics
into account the dynamic information allows us to make the correct for temporal networks. In Temporal
networks, pages 15–40. Springer, 2013
inference. Yes, these nodes form a tightly connected group. 34
Naoki Masuda and Renaud Lam-
In practice, we can consider a dynamic network as a network with biotte. A Guidance to Temporal Networks.
edge attributes. The attribute tells us when the edge is active – or World Scientific, 2016
inactive, if the connection is considered to be “on” by default, like
the road graph. Figure 4.10 shows a basic example of this, with edges
between three nodes.
More formally, our graph can be represented as G = ( G1 , G2 , ..., Gn ),
58 the atlas for the aspiring network scientist

Time
A
Figure 4.10: An example of dy-
namic edge information. Time
B flows from left to right. Each
A row represents a possible po-
tential edge between nodes A,
C
B, and C. The moments in time
B
in which each edge is active are
C represented by gray bars.

where each Gi is the i-th snapshot of the graph. In other words,

Gi = (Vi , Ei ), with Vi and Ei being the set of nodes and edges active
at time i.
How do we deal with this dynamic information when we want
to create a static view of the network? There are a four standard
techniques.

• Single Snapshot – Figure 4.11(a). This is the simplest technique. You

choose a moment in time and your graph is simply the collection
of nodes and edges active at that precise instant. This strategy
works well when the edges in your network are “on” by default.
It risks creating an empty network when edges are ephemeral
and/or there are long lulls in the connection patterns, for instance
in telecommunication networks at night.

• Disjoint Windows – Figure 4.11(b). Similar to single snapshot. Here

we allow longer periods of time to accumulate information. Dif-
ferently from the previous technique, no information is discarded:
when a window ends, the next one begins immediately. Works
well when it’s not important to maintain continuity.

• Sliding Windows – Figure 4.11(c). Similar to disjoint windows, with

the difference that we allow the observation periods to overlap.
That is, the next window starts before the previous one ended.
Works well when it is important to maintain continuity.

• Cumulative Windows – Figure 4.11(d). Similar to sliding windows,

but here we fix the beginning of each window at the beginning
of the observation period. Information can only accumulate: we
never discard edge information, no matter how long ago it was
firstly generated. Each window includes the information of all
previous windows. Works well when the effect of an edge never
expires, even after the edge has not been active for a long time.

Note how these different techniques generate radically different

“histories” for the network in Figure 4.11(a) to (d), even when the
 extended graphs 59

Time Time
A A

B B

A A

C C

B B

C C

A C A C A C A C A C A C

B B B B B B

(a) Single Snapshot. (b) Disjoint Windows.

Time Time
A A

B B

A A

C C

B B

C C

A C A C A C A C A C A C

B B B B B B

(c) Sliding Windows. (d) Cumulative Windows.

Figure 4.11: Different strategies
edge activation times are identical. for converting dynamic edges
into a graph view.
4.5 Attributes on Nodes

Earlier I defined what a bipartite network is: a network with two

node types and edges connecting exclusively nodes of unlike type.
You could consider the node type as a sort of binary attribute on the
node. Once you make the step of adding some metadata to the nodes,
why stopping at just two values? And why constraining how edges
can connect nodes depending on their attributes? Welcome to the
world of node attributes!
Here we do not have the requirement of only establishing edges
between nodes with unlike attribute values. Moreover the attributes
don’t have to be binary. They also don’t have to be qualitative at all
(as in Figure 4.12(a)): they can be quantitative, as in Figure 4.12(b).
For instance, the number of times a user logged into their social me-
dia profile. Finally, nodes can have an arbitrary number of attributes
attached to them, not just one.
Consider for instance a trade network. The nodes in this network
60 the atlas for the aspiring network scientist

B 3
Figure 4.12: (a) A network with
A 3
qualitative node attributes,
represented by node labels
A 4
A 4 and colors. (b) A network with
quantitative node attributes,
represented by node labels and
A 5 sizes.

B 4
B 3
A 4
B 2
(a) (b)

are the various countries. They connect together if one country ex-
ports goods to another. We can have multiple quantitative attributes
on each country. For instance, it can be its GDP per capita, its pop-
ulation, its total trade volume. On the other hand, we can also put
countries in different categories: in which world region are they lo-
cated? Are they democracies or not? Of which trade agreement are
they part of?
In this case, our graph changes form again: G = (V, E, A). We
can see each v ∈ V not as a simple entity, but as a vector of attribute
values: v = ( a1 , a2 , a3 , ...). In this representation, a1 is the value for v
of the first attribute in A. a1 can be a real, integer, or a category.
Node attributes are important because nodes might have tenden-
cies of connecting – or refusing to connect – to nodes with similar
attribute values. We’ll explore this topic in the forms of “homophily”
in Chapter 26 for qualitative attributes, and “assortativity” in Chapter
27 for quantitative attributes. This is different from bipartite networks
because in bipartite networks edges between nodes with different
attribute values are forbidden, while in these cases edges are simply
correlated with attribute values. Moreover, bipartite networks are only
defined for qualitative attributes, not quantitative.
To wrap up, no one forces you to use a single of these more com-
plex graph models at a time. You can merge them together to fit your
analytical needs. For instance, you can create this monster graph
type: Gn = (V1 , V2 , E, L, W, A): a bipartite graph with V1 and V2
nodes, each with attributes in A, which is weighted (W) multilayer
with | L| layers and – for good measure – is also a hypergraph, allow-
ing edges in E with more than two nodes. And, of course, you can
observe it at multiple time intervals (G1 , G2 , ...). Yikes.
 extended graphs 61

4.6 Network Types

Now that you know more about the various features of different net-
work models, we can start looking at different types of networks. I’m
going to use a taxonomy for this section. I find this way of organizing
networks useful to think about the objects I work with.

Simple Networks
The first important distinction between network types is between
simple and complex networks. A simple network is a network we
can fully describe analytically. Its topological features are exact and
trivial. You can have a simple formula that tells you everything you
need to know about it. In complex networks that is not possible, you
can only use formulas to approximate their salient characteristics.
The difference between a simple network and a complex network
is the same between a sphere and a human being. You can fully
describe the shape of a sphere with a few formulas: its surface is
4
4πr2 , its volume is πr3 . If you know r you know everything you
3
need to know about the sphere. Try to fully describe the shape of
a human being, internal organs included, starting from a single
number. Go on, I have time.

Figure 4.13: (a) An infinite lat-

tice without boundaries. (b) A
finite lattice with 25 nodes and
40 edges.

(a) (b)
What do simple networks look like? I think the easiest example
conceivable is a square lattice. This is a regular grid, in which each
node is connected to its four nearest neighbors. Such lattice can either
span indefinitely (Figure 4.13(a)), or it can have a boundary (Figure
4.13(b)). Their fundamental properties are more or less the same.
Knowing this connection rule that I just stated allows you to picture
any lattice ever. That is why this is a simple topology.
Regular lattices can come in many different shapes besides square,
for instance triangular (Figure 4.14(a)) or hexagonal (Figure 4.14(b)).
They also don’t necessarily have to be two dimensional as the exam-
ples I made so far: you can have 1D (Figure 4.14(c)) and 3D (Figure
4.14(d)) lattices – the latter might be a bit hard to see, but it is a cube
of with four nodes per side.
62 the atlas for the aspiring network scientist

Figure 4.14: Different lattice

types. (a) Triangular. (b) Hexag-
onal. (c) One dimensional. (d)
Three dimensional cube.

(a) (b) (c) (d)

Even if deceptively simple, lattices can be extremely useful and are

used as starting point for many advanced tasks. For instance, they
are at the basis of the small-world graph generator (Section 14.2) and
of our understanding of epidemic spread in society (Chapter 17).
Lattices are not the only simple network out there. There is a wide
collection of other network types. These are usually developed as
the simplest illustrative examples for explaining new problems or
algorithms. A few of my favorites (yes, I’m the kind of person who 35
Graham Brightwell and Peter Winkler.
has favorite graphs) are the lollipop graph35 (a set of n nodes all Maximum hitting time for random
connected to each other plus a path of m nodes shooting out of it, walks on graphs. Random Structures &
Algorithms, 1(3):263–276, 1990
Figure 4.15(a)), the wheel graph (which has a center connected to
a circle of m nodes, Figure 4.15(b)), and the windmill graph (a set
of n graphs with m nodes and all connections to each other, also all
connected to a central node, Figure 4.15(c)). Once you figure out
what rule determines each topology, you can generate an arbitrary set
of arbitrary size of graphs that all have the same properties.

Figure 4.15: Different simple

networks. (a) Lollipop graph.
(b) Wheel graph. (c) Windmill
graph.

(a) (b) (c)

Complex Networks
If simple networks were the only game in town, this book would not
exist. That is because, as I said, you can easily understand all their
properties from relatively simple math. That is not the case when the
network you’re analyzing is a complex network. Complex networks
model complex systems: systems that cannot be fully understood if
all you have is a perfect description of all their parts. The interactions
between the parts let global properties emerge that are not the simple
 extended graphs 63

sum of local properties. Thus, there isn’t a simple wiring rule and,
even knowing all the wiring, some properties can still take you by
surprise.
Personally, I like to divide complex networks into two further
categories: complex network with fundamental metadata and with-
out fundamental metadata. As we saw so far, there are a number of
metadata you can attach to your nodes and network. You can have
quantitative and qualitative node/edge attributes, layers, bipartite
networks, and so on. The difference between the two types is that,
if the metadata are fundamental, they change the way you interpret
some or all the metadata themselves.
For instance, social networks, infrastructure networks, biological
networks, and so on, model different systems and have different
metadata attached to their nodes and edges. It can be age/gender,
activation types, up- and down-regulation. However, at a funda-
mental level, the algorithms and the analyses you perform on them
are the same, regardless of what the networks represent. They have
nodes and edges and you treat them as such. You perform the Euler
operation: you forget about all that is unnecessary so you can apply
standardized analytic steps.
That is emphatically not true for networks with fundamental
metadata. In that case, you need to be aware of what the metadata
represent, because they change the way you perform the analysis and
you interpret the results. A few examples:

• Affiliation networks. These are networks that, for instance, connect

individuals to the groups they belong to. This is easily represented
as a bipartite network. One node type is the individual, the other
is the group. However, the semantics that one node type includes
the other – the group includes the individual – is fundamentally
different when you have node types at an equal level – for instance
a bipartite network connecting people to the products they buy.

• Interdependent networks. These are usually multilayer networks

modeling some sort of physical system. The nodes in one layer
are coupled with nodes in another because they depend on each
other. Differently from regular multilayer networks, the removal
of one node in one layer has immediate and non-trivial repercus-
sions on all the layers depending into it, often with catastrophic
consequences (see Section 19.4).

• Correlation networks. We saw a glimpse of these networks when we

looked at weighted graphs. Here we have constraints on the edge
weights, which can also be negative. The interpretation of such
edge weights is different from what you would have in regular
64 the atlas for the aspiring network scientist

weighted networks. For instance, edges with very low weights are
important here, because a strong negative correlation is interesting,
even if its value (−1) is lower than no correlation at all (0).

A special mention for this class of networks should go to Bayesian 36

Finn V Jensen et al. An introduction
networks36 , 37 , 38 . In a Bayesian network, each node is a variable and to Bayesian networks, volume 210. UCL
directed edges represent dependencies between variables. If knowing press London, 1996
something about the status of variable u gives you information about 37
Nir Friedman, Dan Geiger, and
Moises Goldszmidt. Bayesian network
the status of variable v, then you will connect u to v with a directed classifiers. Machine learning, 29(2-3):
(u, v) edge. 131–163, 1997
In the classical example, you might have three variables: the
38
Nir Friedman, Lise Getoor, Daphne
Koller, and Avi Pfeffer. Learning
probability of raining, the probability of having the sprinklers on, probabilistic relational models. In IJCAI,
and the probability that the grass is wet. Clearly, rain and sprinklers volume 99, pages 1300–1309, 1999
both might cause the grass to be wet, so the two variables point to
them. Rain also might influence the sprinklers, because the automatic
system to save water will not turn them on when it’s raining, since it
would be pointless. Obviously, the fact that the sprinklers are on will
have no effect on whether it will rain or not.

Rain
Figure 4.16: (a) A Bayesian
T F
network. (b) The conditional
0.2 0.8
probability tables for the node
states. The tables are referring
Raining Sprinkler
to, from top to bottom: Rain,
Rain T F
Sprinkler, Wet.
T 0.01 0.99
Sprinklers F 0.2 0.8

Wet
Wet Rain Sprinkler T F
T T 0.99 0.01
(a) T F 0.98 0.02
F T 0.97 0.03
F F 0.01 0.99
(b)

We can model this system with the simple Bayesian network

in Figure 4.16(a) and the corresponding conditional probability
tables in Table 4.16(b). Bayesian networks are usually the output
of a machine learning algorithm. The algorithm will learn the best
network that fits the observations. Then, you can use the network to
predict the most likely probability of the state of a variable given a
new observation of a subset of variables.
Simple examples like this might seem boring, but when you start
having hundreds of variables you can find interesting patterns by
 extended graphs 65

applying some of the techniques you will learn later on. For instance,
you might discover set of variables that are independent of each
other, even if, at first glance, it might be difficult to tell.
A not so distant relative of Bayesian networks are neural networks,
the bread and butter of machine learning these days. Notwithstand-
ing their amazing – and, sometimes, mysterious – power, neural
networks are actually much more similar to simple networks than to
complex ones. Differently from Bayesian networks, the wiring rules
of neural networks – of which I show some examples in Figure 4.17 –
are usually rather easy to understand.

Figure 4.17: Different neural

networks. The node color de-
termines the layer type: input
(red), hidden (blue), output
(green).

(a) Feedforward. (b) Recurrent. (c) Modular.

The way they work is that the weight on each node of the out-
put layer is the answer the model is giving. This weight is directly
dependent on a combination of the weights of the nodes in the last
hidden layer. The contribution of each hidden node is proportional to
the weight of the edge connecting it to the output node. Recursively,
the status of each node in the hidden layer is a combination of all
its incoming connections – combining the edge weight to the node
weight at the origin. The first hidden layer will be directly dependent
on the weights of the nodes in the input layer, which are, in turn,
determined by the data.
What the model does is simply finding the combination of edge
weights causing the output layer’s node weights to maximize the
desired quality function.

4.7 Summary

1. Bipartite networks are networks with two node types. Edges can
only connect two nodes of different types. You can generalize
them to be n-partite, and have n node types.

2. In multigraphs we allow to have multiple (parallel) edges be-

tween nodes. We can have labeled multigraphs when we attach
labels to nodes and edges. Labels on nodes can be qualitative or
quantitative attributes.
66 the atlas for the aspiring network scientist

3. If we only allow one edge with a given label between nodes we

have a multiplex or multilayer network: the edge label informs us
about the layer in which the edge appears.

4. Multilayer networks are networks in which different nodes can

connect in different ways. To track which node is “the same”
across layers we use inter-layer couplings. Couplings can connect
a node in a layer to multiple nodes in another, making a many-to-
many correspondence.

5. Signed networks are a special type of multilayer network with

two layers: one positive (e.g. friendship) and one negative (e.g.
enmity).

6. Hypergraphs are graphs whose (hyper)edges can connect more

than two nodes at the same time. You can consider hyperedges as
cliques, bipartite edges, or simplicial complexes.

7. Dynamic graphs are graphs containing temporal information on

nodes and edges. This information tells you when the node/edge
was present in the network. There are many ways to aggregate this
information to create snapshots of your evolving system.

8. Simple networks are networks whose topology can be fully de-

scribed with simple rules. For instance, in regular lattices you
place nodes uniformly in a space and you connect them with their
nearest neighbors.

4.8 Exercises

1. The network in [Link]

[Link] is bipartite. Identify the nodes in either type and find the
nodes, in either type, with the most neighbors.

2. The network in [Link]

[Link] is multilayer. The data has three columns: source and
target node, and edge type. The edge type is either the numerical
id of the layer, or “C” for an inter-layer coupling. Given that this is
a one-to-one multilayer network, determine whether this network
has a star, clique or chain coupling.

3. The network in [Link]

[Link] is a hypergraph, with a hyperedge per line. Transform it
in a unipartite network in which each hyperedge is split in edges
connecting all nodes in the hyperedge. Then transform it into a
bipartite network in which each hyperedge is a node of one type
and its nodes connect to it.
 extended graphs 67

4. The network in [Link]

[Link] is dynamic, the third and fourth columns of the edge
list tell you the first and last snapshot in which the edge was
continuously present. An edge can reappear if the edge was
present in two discontinuous time periods. Aggregate it using a
disjoint window of size 3.
5
Matrices

Graphs, with their fancy nodes and edges, are not the only way to
represent a network. One can do so also by using matrices. In fact,
ask some people and they will tell you that everything is a matrix.
What’s a number if not a zero-dimensional tensor? I mean, come on!
Unfortunately, I am not one of those people, so this chapter will
contain only the bare minimum for you to smile and nod while
talking to them.
The reason of having this chapter is because sometimes operations
are more natural to understand with the graph models, and some-
times they are just matrix operations. Which perspective is more
useful – graph vs matrix – often depends on the perspective used by
the researcher(s) discovering a given property of developing a given
tool. So in the book I’ll often switch back and forth between these
two representations, and this chapter is your map not to get lost once
I start rambling about “positive semi-definite matrices”, whatever the
hell that means.
We start with the simplest object: the adjacency matrix (Section
5.1). In Section 5.2 we will see what kinds of operations you can do
on them, and then in Section 5.3 some special matrix representations
for graphs, namely the stochastic, the incidence, and the graph
Laplacian matrices. Finally, Section 5.4 shows more advanced matrix
operations, specifically how to decompose complex matrices in
smaller, simpler, and more informative objects.

5.1 Adjacency Matrix

The adjacency matrix is a deceptively simple object. Suppose that you

have a group of friends and you want to keep a tally of who’s friend
with whom. You can make a table with one friend per row and one
friend per column. If two people say they know each other, you can
just put a cross in the corresponding cell, as my example shows in
Figure 5.1(a). Well, that’s it. That’s an adjacency matrix.
 matrices 69

Yes! Yes!
Do you 5
know each
other?
2 Figure 5.1: (a) A vignette of
7
1 2 3 4 5
9 1 how one would construct an ad-
1
2
x
1 5
3
6 jacency matrix. (b) An example
4 8
5x graph. (c) The adjacency matrix
4 3 of (b). Rows and columns are in
(a) (b) (c) the same order as the node ids
(so the first row/column refers
to node 1, the second to node 2,
The adjacency matrix is the basic representation of a graph as etc).
a matrix. Each row/column corresponds to a node. Each cell rep-
resents an edge, set to one if the edge exists, and zero otherwise.
If the graph is undirected, each edge sets two cells to one. If the
edge connects nodes u and v both the Auv and the Avu entries are
equal to one. Figure 5.1(b) shows a graph and Figure 5.1(c) shows
its adjacency matrix – in the graph view I labeled the nodes with the
order as they appear in the adjacency matrix: the first row/column
represents node 1, the second row/column is for node 2, and so on.
In Figures 5.1(b) and 5.1(c) we have the simplest graph possible:
the unweighted undirected graph. In this case, the adjacency matrix
carries a few properties. For instance, the graph has no self-loops
– edges connecting a node to itself. For this reason, the diagonal
of the adjacency matrix contains zeros. We like to keep it that way,
because we’ll use the diagonal for all sorts of interesting stuff in the
future – for instance later on when dealing with the graph Laplacian.
The adjacency matrix is also square, meaning that it has the same
number of rows and columns. Moreover, it is symmetric, meaning
that ∀u, v Auv = Avu . The diagonal divides the matrix into two
identical triangular halves.
You can calculate the complement of any graph by simply calcu-
lating 1 − A, with 1 being a matrix full of ones – although you might
want to fill its diagonal with zeros to avoid self loops, which are
usually ignored in complement graphs.
We can adapt the adjacency matrix to deal with all the compli-

8
Figure 5.2: (a) A non-symmetric
2 6 adjacency matrix. (b) The corre-
sponding directed graph.
7
3
5

4 1
9

(a) (b)
70 the atlas for the aspiring network scientist

6
Figure 5.3: (a) A non-binary
5 5 adjacency matrix. (b) The corre-
4 4
3 2 sponding weighted graph.
2
3
5
9 3 7 5 2
3
5 6 5
2

4 8
2
1

(a) (b)

cations we introduced in the graph model in Chapters 3 and 4. For

instance, we can represent a directed graph by breaking the symme-
try property we just enunciated. If Auv is allowed to be zero when
Avu is one, then it means that we just introduced directionality in
the matrix, as Figure 5.2 shows. Note that different authors/papers
might follow different conventions. Some will represent the u → v
edge as Auv and some as Avu . So make sure you identify the conven-
tion before you start working your way through the paper!
If we want edge weights to exploit the power of linear algebra also
on weighted graphs (Section 3.3), we can allow values different than
one for the cells representing edges. Now we can have an arbitrary
real value in the cells, representing the connection’s strength – see
Figure 5.3.

1 2 3 4 5 6 7 8 9 10
Figure 5.4: (a) A non-square
adjacency matrix. (b) The corre-
sponding bipartite graph.

4 5 3 6 2 1

(a) (b)
What else? We can make the adjacency matrix not square if we
need to represent a bipartite network. The different numbers of rows
and columns allow us to use one dimension to represent the nodes
in V1 and the other to represent the nodes in V2 . Figure 5.4 depicts
an example. The downside is that we lose the power of the diagonal
we had in the adjacency matrix – which doesn’t seem like a big deal
now, because at the moment I’m being all hush hush about what this
power really is.
Of course, it’s possible to have a square adjacency matrix for a
bipartite network if |V1 | = |V2 |. You can also “squarify” a bipartite
adjacency matrix by dividing it in four blocks. The blocks on the
main diagonal contain zeros, while the blocks in the other diagonal
contain the original adjacency matrix. Such a construct is a (|V1 | +
|V2 |) × (|V1 | + |V2 |) matrix, and they can be useful. Figure 5.5 shows
 matrices 71

|V1| |V2| Figure 5.5: A way to build a

(|V1 | + |V2 |) × (|V1 | + |V2 |)
|V1| 0 A square matrix starting from
A, a non-square bipartite adja-
cency matrix.

|V2| AT 0
an example.
Finally we can – and do – represent even multilayer networks
with matrices. Or, to be more precise, we use tensors to represent
them. I’m not going deep into technicalities, so I’m going to give
you a superficial view of tensors: just enough to have an intuition.
Technically speaking, a tensor is a generalized vector. A vector can be
seen as a monodimensional array: a list of values. A matrix could be
said to be a two-dimensional array. A tensor is a multidimensional
array: we can have as many dimensions as we want.

Figure 5.6: (a) A three dimen-

sional tensor. (b) The corre-
sponding multilayer graph.

(a) (b)

A one-to-one coupled multilayer network can be represented

with a three-dimensional vector. The first two dimensions – rows
and columns – are the nodes, and the third dimension is the layers.
Mathematically, the Auvl entry in the tensor tells you the relationship
between nodes u and v in layer l. Figure 5.6 provides an intuitive
example. Note that we are assuming that the nodes are sorted in the
same way across the third dimension, thus the inter-layer couplings
(see Section 4.2) are implicit. If we want a many-to-many coupling
we cannot have this tacit assumption and we have to introduce
the inter-layer coupling edges explicitly. But I think this is already
enough complexity, so I will stop here.
A few interesting properties of binary adjacency matrices. The
72 the atlas for the aspiring network scientist

sum of the rows – and of the columns in a symmetric matrix – is

equal to the node’s number of connections, which we call the degree
(and will be the topic of Chapter 6). If the graph is directed the
row/columns give you the in/out degree. The sum of the entries of
the matrix is 2 times the number of edges (undirected) or the number
of edges (directed).

5.2 Linear Algebra

To fully appreciate what looking at networks as matrices rather

than graphs can buy us, we need to dust off some linear algebra. A
big caveat here: this is a very superficial recap of only the crucial
concepts we need to continue. This is not supposed to be even an in-
troductory section to linear algebra itself. If your objective is to learn 1
Gilbert Strang. Introduction to linear
actual linear algebra, there is no substitute for dedicated textbooks1 , 2 algebra. Wellesley-Cambridge Press
and online courses3 . Wellesley, MA, 1993
2
Carl D Meyer. Matrix analysis and
Linear algebra – and matrix analysis specifically – can buy you applied linear algebra, volume 71. Siam,
a lot of analytic power and it is usually computationally efficient. 2000
3
Here I take a pragmatic approach, just showcasing what some basic [Link]
courses/mathematics/
operations on your adjacency matrices mean. Hopefully, you can 18-06-linear-algebra-spring-2010/
extrapolate to your use cases when needed.

Transpose
Let’s consider a matrix A, whose rows and columns are your nodes.
If we transpose A it means that all Auv becomes Avu and viceversa.
In this book, for convention, A T will be the transpose of A. Figure
5.7 shows the case of a squared non symmetric matrix transpose. In
practice, transposing is like placing a mirror on the diagonal.

Figure 5.7: A matrix and its

transpose. Note how the (u, v)
entries of A equal to one trans-
posed to the (v, u) entries of A T ,
for instance (1, 3) to (3, 1).

(a) A (b) A T
If your matrix is squared and symmetric – an undirected unipar-
tite graph – transposing has no effect: A T = A. For directed graphs,
A T and A will be different, because of the directionality. To be really
blunt, transposing A in a directed graph means to reverse all edge
directions. In a bipartite network with V1 and V2 node types, the
shape of A will change, from being a |V1 | × |V2 | matrix to a |V2 | × |V1 |
 matrices 73

matrix.

Matrix Multiplication
Formally, matrix multiplication is an operation that produces a
matrix C from two matrices A and B. You cannot multiply any two
matrices together, though: they have to have one dimension of equal
size. So, if A is an n × m matrix, it can only be multiplied by B if B is
either an m × x or an x × n matrix, with x being anything. Suppose
that B is m × x: the result of A multiplied to B will be a n × x matrix.
The common dimension “disappears”.
Understanding why this is the case is easy once you know what
matrix multiplication actually does. Each cuv entry of C is equal to
the sum of the products of all entries in the uth row of A and the vth
m
column of B. Formally: cuv = ∑ auk bkv .
k =1

1 2 3 Figure 5.8: An example of ma-

trix multiplication. Each cell is
4 5 6 the result of the combination
of the rows/columns of the
corresponding color, whose
1 2 9 12 15 element-wise products are
summed. So the red cell equals
3 4 19 26 33 to 9 because it is the sum of
1 × 1 (the product of the first el-
5 6 29 40 51 ements) plus 2 × 4 (the product
of the second elements).
7 8 39 54 69

Figure 5.8 shows a graphical representation of the algorithm to

multiply two matrices.
So, why would we do transposes and multiplications? I think in
this case a practical scenario would help. Suppose you have a bipar-
tite network and what you really want to know is not which node
of type V1 connects to nodes in type V2 , but how similar nodes in V1
are, because they connect to the same V2 nodes. This is practically
the subject of Chapter 23 but, to make it simple for this example, you
want the probability of going from a V1 node to another V1 node,
passing via V2 nodes.
If you divide the bipartite adjacency matrix by its row sum, you
get the probability to go from a V1 node to a V2 node – we’ll see more
about this operation in Section 5.3. However, we don’t know how
to go back: in that case we should normalize by column sum! That
74 the atlas for the aspiring network scientist

could be achieved by normalizing A T , A’s transpose, by its row sum.

Since A is a |V1 | × |V2 | matrix and A T is a |V2 | × |V1 | matrix, you can
multiply one by the other: AA T is in fact a |V1 | × |V1 | which is exactly
what you need.
But wait, what does AA T actually mean? What’s the result of such
an operation? Well, following Figure 5.8, the (v1 , v2 ) cell is the sum of
the probability of going from v1 to any V2 node times the probability
of arriving to v2 from any V2 node. Which is what we wanted!
Without linear algebra, you’d have to represent the adjacency by
sets of neighbors, and then calculate intersections and dividing vari-
ous scalars in isolation. But transposes and matrix multiplications are
such standard operations that many libraries will have implemented
them very efficiently, with the result of being blazingly fast to calcu-
late and extremely easy to incorporate in your code. Moreover, if you
use sparse matrix representations you can also be memory efficient,
meaning you can go big with your matrices!

Positive (Semi)Definite Matrices

Combining transpose and multiplications is the source of infinite 4

At least for me.
fun4 . Let’s take a look at another example. Suppose you have a vector
of length m and a n × m matrix. It follows from the properties of
matrix multiplication that you can always multiply them – because
a vector of length m is a m × 1 matrix. Hopefully, you know what
the result would be: a vector of length n – remember: the common
dimension “disappears”. Use Figure 5.8 as your guide: if the matrix
on the top would be only the first row vector (1, 4) (in red), then the
result of multiplying that vector to the matrix on the left is the vector
(9, 19, 29, 39). Multiplying a 2 × 1 vector to a 4 × 2 matrix resulted in a
4 × 1 vector.
For the very same reason, you can always multiply a vector with
the transpose of itself. If z is our vector, z T z is a legit operation:
you’re multiplying a 1 × m matrix with m × 1 matrix. And the result is
a 1 × 1 matrix: a scalar, a number. Figure 5.8 can be your guide again:
in red I highlight a 2 × 1 vector multiplying another 2 × 1 vector to
result in a 1 × 1 number. Multiplying (1, 4) to (1, 2) gives 9, because it
is 1 + 8, as the caption says.
If we put together what we discovered in the previous two para-
graphs: if A is a square matrix, then z T Az is a scalar.
Now, some matrices A are special. For these special matrices, it
doesn’t matter what you put in z, as long as it is a vector of real num-
bers: the result of z T Az is always going to be greater than zero. We
call these special matrices “positive definite”. Relaxing the concept
a bit, if z T Az ≥ 0 for any real number z, then A is positive semi-
 matrices 75

definite – “semi” because we allow the result to be zero sometimes.

Positive definite matrices are awesome, because they allow you
to do a bunch of cool stuff. For instance, consider the Euclidean
distance. If you have a m dimensional space, you can represent the
coordinates of two points with vectors of length m. Say that these
vectors are p and q.

p = (4, 5)
Figure 5.9: An example of
Euclidean distance in m = 2
dimensions. Note that we build
((p - q)T(p - q))1/2
the special p − q vector to have,
p2 - q 2 = 4
at its ith entry, the difference
between the ith entries of p and
q.
p1 - q 1 = 3

q = (1, 1)

(p - q) = (p1 - q1, p2 - q2)

The Euclidean distance is the length of the line segment connect-

ing the points, which is nothing more that the hypotenuse of a right
triangle (this is now unexpectedly a geometry book). Pythagoras
teaches us that the length of the hypotenuse is the square root of the 5
Really? You wanted a reference for
sum of the squares of the catheti lengths5 . The sum of the squares of this?
the catheti, as Figure 5.9 shows, is just the square of a special vector:
p − q. Squaring this vector means to calculate ( p − q) T ( p − q) – i.e., to
multiply it with itself.
Now consider the identity matrix I. I is a diagonal matrix whose
diagonal is filled with ones and the rest of the values are equal to
zero. When you multiply I to any z, the result is always z. So you
can sneak it into your Euclidean distance without changing much:
( p − q)T ( p − q) = ( p − q)T I ( p − q). Aha! But remember that, in the
Euclidean definition, we take the square root, which implies that
( p − q)T ( p − q) is always non-negative and, since ( p − q)T ( p − q) =
( p − q)T I ( p − q), then also ( p − q)T I ( p − q) ≥ 0. As a consequence I is,
at the very least, positive semi-definite (it’s actually positive definite).
Moreover, you can put any positive semi-definite matrix M in
your ( p − q) T M( p − q) and you’re going to calculate some distance!
(Admittedly, this excites me much more than it should). This is the
exact thing that you do, for instance, when calculating a Mahalanobis
distance – which is a smart Euclidean that takes into account the
correlation between the vectors, see Section 40.1. The Mahalanobis
distance is (( p − q) T cov( p, q)−1 ( p − q))1/2 , where cov( p, q)−1 is the
inverse of the covariance matrix between p and q. The covariance
76 the atlas for the aspiring network scientist

matrix is a matrix whose element in the i, j position is the covariance

between the i-th and j-th elements of p and q. Surprise surprise,
cov( p, q)−1 is positive semidefinite.
If you’re still reading, and wondering what the heck is going on,
all of this will be extensively used in Chapter 40, when we will try
to figure out how to compare two different activation states in a
network.

Eigenvalues and Eigenvectors

Consider Figure 5.10. Given a vector v, we can apply any arbitrary

matrix transformation A to it. We then obtain a new vector w = Av.
Any transformation A has special vectors: A scales these special vec-
tors without altering their directions. In practice, the transformation
A simply multiplies the elements of such vectors by the same scalar
λ: w = λv. We have a name for this: v is A’s eigenvector and λ is its
associated eigenvalue. Mathematically, we represent this relation as
Av = λv.

(x’, y’)
(λx, λy) Figure 5.10: A graphical depic-
tion of an eigenvector.
w=Av=λv
w'=A'v
(x, y)

The formula we just introduced is the one for right eigenvectors,

because the vector multiplies the matrix from the right. As you might
expect, there are also left eigenvectors, which multiply the matrix
from the left: vA = vλ. Right and left eigenvectors are different,
have different values, but their corresponding eigenvalues are the
same. From now on, when I mention eigenvectors, I refer to the right
eigenvectors. Right eigenvectors are the default, and when I refer to
left eigenvectors I will explicitly acknowledge it.
Our adjacency matrix A, which has |V | rows and |V | columns, has
|V | eigenvalues. Usually, we sort them in decreasing order.
A key term you need to keep in mind is “multiplicity”. The mul-
tiplicity of an eigenvalue is the number of eigenvectors to which it is
associated. If you have an n × n matrix, but only d < n distinct eigen-
values, some eigenvalues are associated to more than one eigenvector.
Thus their multiplicity is higher than one.
 matrices 77

5.3 Special Matrices

Stochastic

Adjacency matrices are nice, but I think most of the times you’ll see
them transformed in various ways to squeeze out all the possible
analytic juice. The simplest makeover we can give to the adjacency
matrix is to convert it into a stochastic matrix. This means that we
normalize it, dividing each entry by the sum of its corresponding row
– this means that each of its rows sums to one. If nodes u and v are
connected, and u has 5 connections, the Auv entry will be 1/5 = 0.2.
Figure 5.11 shows an example of this stochastic transformation.

5
2 Figure 5.11: (a) The original
7
9 1 graph. (b) The adjacency matrix
6 of (a). (c) The corresponding
8 stochastic version.
4 3

(a) (b) (c)

What’s the usefulness of the stochastic adjacency matrix? The first

direct use we can make of it is to calculate transition probabilities.
This is literally what it contains: each entry is the probability that
a random walker (see Chapter 8) on a given node (row) will cross
that edge. Since non-edges have value zero, it is impossible to follow
them. In Figure 5.11(a) we see that node 9 has degree equal to five.
This corresponds to having five entries set to one in Figure 5.11(b).
If we close our eyes and pick one of these at random, each one has a
probability of 0.2 to be picked. That is the value in Figure 5.11(c).
Suppose we picked node 6 and that we repeat the exercise. Picking
one of node 6’s neighbors at random has a probability of 0.17, and we
end up – for instance – on node 3. We might want to know what was
the likelihood of ending in node 3 starting from node 9 and doing
exactly two random jumps. The probability is not simply the product
of the two jumps – as we would do naively for independent events
(see Section 2.2) – because there is an alternative route. We could
have visited node 8 first and then moved to 3. We have to keep track
of all possible alternative paths, and this becomes really unwieldy
when we start considering longer random walks.
Luckily, we don’t have to do it. The stochastic matrix has the
power of telling us what we want. It’s literally its power. Say A is
our stochastic matrix. We just saw how A is just the probability of
transitioning from one node to another. In other words, it gives us
the probability of all transitions for random walks of length 1. Let’s
78 the atlas for the aspiring network scientist

now write this matrix as A1 , which is the same thing as A. Let’s say
this again: A1 is the probability of all transitions for random walks of
length 1. Could it be, then, that A2 is the probability of all transitions
for random walks of length 2? And that An is the probability of all
transitions for random walks of length n? Yes, they are!
From the matrix multiplication crash course I gave you in Section
5.2 you know why: A2 ’s uv entry is, as the formula I wrote there
shows, the sum of the multiplication of probabilities of all nodes k
that are connected to both u and v, and thus can be used in a path
of length 2. Multiplying Auk to Akv means asking the probability of
going from u to k and from k to v. Summing Auk1 Ak1 v to Auk2 Ak2 v
means asking the probability of passing through k1 or k2 . See Chapter
2 for a refresher on what multiplying and summing probabilities
mean.

Figure 5.12: Different powers of

the stochastic adjacency matrix
of the graph in Figure 5.11(b).

(a) A1 (b) A2 (c) A3

Let’s take a closer look at A2 and A3 in Figures 5.12(b) and 5.12(c).

First, they are stochastic matrices, and Section 2.5 taught you that
the rows of a stochastic matrix always sum to 1. So each entry in the
matrix is still a transition probability. This time, though, it’s not the
transition probability of the direct connection, but of a path of length
2 and 3, respectively.
Second, the diagonal is not zero any more. That is because, with
a random walk of length 2, there is a chance to select the same edge
twice, and therefore returning to the point of origin. So the A2vv
entry tells you the likelihood of starting from v and returning back
to v in two steps. That is because – as the matrix multiplication
section showed you mathematically – A2vv is the combination of the
probabilities of going from any of v’s neighbors to v, weighted by the
probability of having reached each of v’s neighbors from v itself.
Third, while A2 still has zero entries, A3 does not. This is because
some node pairs are farther than two edges away, so the probability
of a random walker to reach them in two hops is zero – check Figure
5.11(a) again if you don’t believe me! On the other hand, no pair of
nodes is farther than three hops away, and thus there is always a path
of length three between any node pair, no matter how unlikely.
Finally, stochastic matrices – either A or any of its powers – are
not symmetric any more, even if the “raw” adjacency matrix of an
 matrices 79

undirected graph is. This is because the likelihood of ending in u

from v isn’t necessarily the same as the other way around: if v has
better connected neighbors, the random walkers are more likely to
be led astray. For instance, the probability to go from node 4 to 5 in
three random steps is 0.04, while the other way around is 0.02. That
is because node 5 has an extra connection, which can lead the walker
to be unable to reach 4 in two additional hops.
This last property means that, if you transpose a stochastic ad-
jacency matrix, A T 6= A even for undirected graphs! Since you
normalized by row sum, the Auv entry can be different from the Avu :
the only thing you know is that they’re both non-zero.
There is one surprise hidden in the folds of An for a suitably large
n. This surprise is waiting for you in Section 8.1.
Note that there are two valid stochastic adjacency matrices for
a bipartite network. If you normalize by row sum, the stochastic
A tells you the probability of going from a V1 node to a V2 node.
Normalizing by column sum, which is equivalent of taking the
stochastic A T (i.e., transposing A beforehand), then the matrix tells
you the probability of going from a V2 node to a V1 node.
Coming back to eigenvalues and eigenvectors, the largest eigen-
value of a stochastic adjacency matrix is always equal to one – we
also call it the “leading” eigenvalue, or λ1 . This takes a special value:
any stochastic adjacency matrix you can come up with will always
have λ1 = 1. No eigenvalue will ever be greater.
As we saw in Figure 5.10, each eigenvalue has a corresponding
eigenvector. We call the eigenvector associated to the largest eigen-
value the “largest” or “leading” eigenvector, for convenience. The
point of looking at eigenvectors is that there is a relationship between
the v-th entry in the i-th eigenvector of an adjacency matrix and node
v’s relationship with the entire graph.
For instance, the multiplicity of the largest eigenvalue of the
stochastic adjacency matrix is important. It can happen that second
largest eigenvalue λ2 of could be equal to the first. And, actually, also
the third, fourth, fifth, ... could be equal to the first. This is related to
the first application of linear algebra to network analysis, which we
will fully appreciate when it will be time to discuss about connected
components (Section 7.4).

Incidence

In general, an incidence matrix is a matrix telling you what are the

relations between two classes of objects. For instance, you can have
an incidence matrix telling you for which company a person works.
Since the two classes might have a different number of members,
80 the atlas for the aspiring network scientist

incidence matrices are not necessarily square. In fact, you could

say that the adjacency matrix of a bipartite network is an incidence
matrix.
However, when performing network analysis, there is one type
of incidence matrix that is widely used, and thus “owns” this term.
The vast majority of times, if you read a paper talking about the
“incidence matrix”, you’ll see the same object: a matrix that has nodes
on the rows, edges on the columns, and it has an entry equal to one
if the node and the edge are connected to each other. Figure 5.13(b)
shows the incidence matrix of the graph at Figure 5.13(a).

3 6 Figure 5.13: (a) A graph with

2 5 nodes and edges labeled with
1 3 4 7 5 their ids. (b) The incidence
1 4
matrix of (a), with nodes on the
2
rows and edges on the columns.
(a) (b)
Incidence matrices have interesting properties, some more trivial
than others. For instance, you know that, in the incidence matrix of
a simple graph, each column sums to two because each edge only
connects two nodes. Only in the incidence matrix of a hypergraph a
column can sum to a number larger than 2. You can use the incidence
matrix to construct other special matrix representations. For instance,
you can construct the adjacency matrix of the line graph of G by
calculating B T B − 2I, assuming that B is the incidence matrix, and I is
a | E| × | E| identity matrix.
An incidence matrix can also be oriented. In an oriented incidence
matrix, the columns sum to zero. For every edge, one of the two non
zero entries – the nodes to which it is attached – is equal to 1 and the
other is equal to −1. It doesn’t really matter which of the two you
pick, as long as you make sure all columns sum to zero. If B is an
oriented incidence matrix, you can use it to construct the Laplacian as
BB T . The Laplacian is a super cool matrix and I’ll focus on it now.

Laplacian

The stochastic adjacency matrix is nice, but the real superstar when
it comes to matrix representations of networks is the Laplacian. To
know what that is, we need to introduce the concept of Degree ma-
trix D – which is a very simple animal. It is what we call a “diagonal”
matrix. A diagonal matrix is a matrix whose nonzero values are ex-
clusively on the main diagonal. The other off-diagonal entries in the
matrix are equal to zero. In D the diagonal entries are the degrees of
 matrices 81

5
2 Figure 5.14: The degree matrix
7
9 1 (a) of the sample graph (b).
6
8

4 3

(a) (b)

the corresponding nodes. Figure 5.14 shows an example of a degree

matrix.
The Laplacian version of the adjacency matrix – which we call L –
is the result of a simple operation: L = D − A. In practice, we take
the degree matrix D and we subtract A from it. L is a matrix that has
the node degree in the diagonal, −1 for each entry corresponding to
an edge in the network, and zero everywhere else. Figure 5.15 depicts
the operation. L has some obvious properties: since it has the degree
of the node on the diagonal and −1 for each of the node’s connection,
the sums of all rows and columns are equal to zero.

Figure 5.15: The operation pro-

ducing the Laplacian matrix L
(c), subtracting the adjacency
matrix A (b) from the degree
matrix D (a).
(a) D (b) A (c) L

The Laplacian of a connected undirected graph is part of the

positive semi-definite club, which will come in handy in Section 40.2.
Just like for A, we are also interested in the eigenvectors of L.
We need to make some adjustments, though. Instead of looking at
the largest eigenvalues, we focus on the smallest ones – meaning
that now we use λ1 to refer to the smallest eigenvalue. This takes a
special value like for A but, for L, λ1 = 0. Besides doing some of the
same things you can do with the eigenvector of the adjacency matrix,
the Laplacian has a few more tricks up its sleeve. For instance, one
can use it to solve the normalized cut problem, which is useful for
community discovery and we will discuss it in detail in Section 8.4.
Another connection between stochastic and Laplacian matrices
is on multiplicity. The multiplicity of the smallest eigenvalue of
the Laplacian plays the exact same role as the one of the largest
eigenvalue of the stochastic matrix – a role that you will appreciate in
Section 7.4 when we will study connected components.
We will see what makes the Laplacian so important in Chapter
8, when I’ll show you how many things you can do with its quasi-
82 the atlas for the aspiring network scientist

mystical properties.

5.4 Matrix Factorization

In some cases, you might want to express a matrix as the result of

the multiplication of other matrices. This can be useful because the
matrices you use to reconstruct your observed matrix might be made
of pieces you can more easily interpret. We call this decomposition
of a matrix “factorization”, because we divide the matrix into its
“factors”, its building blocks. There are countless ways to factorize
a matrix. Here we examine only the ones that you’re most likely to
encounter in network analysis. I divide them in two classes: the ones
operating on regular bi-dimensional matrices, and the ones which
work on scary and confusing multidimensional matrices (i.e. tensors).

Matrix Decomposition
One of the easiest ways to perform matrix factorization is what we
call “eigendecomposition”. The adjacency matrix A of an undirected
unweighted graph can always be decomposed as A = ΦΛΦ T . Rather
than being the left and right eyes of a really pissed frowny face, Φ is
the matrix we obtain piling all eigenvectors next to each other, and Λ
is a diagonal matrix with the eigenvalues on its main diagonal and
zeros everywhere else:
 
λ0 ... 0
Λ=
 .. 
0 . 0 

0 ... λn .
The eigendecomposition is useful to solve a set of linear difference
equations. We are mostly interested in it as the special case of the
more general Singular Value Decomposition (SVD) – which can be
applied to any matrix, even non-square ones. In SVD, we simply
replace Φ and Λ with generic matrices. In other words, we say that
we can reconstruct A with the following operation: A = Q1 ΣQ2T . Like
Λ, also Σ is a diagonal matrix. The difference is that Σ contains the
singular values of A, rather than its eigenvalues. While there is only
one valid Σ to solve this equation – that is why it is called “singular”
– there could be multiple Q1 and Q2 matrices that you could plug
in, as long as they’re both unitary matrices. A unitary matrix Q is a
matrix whose transpose is also its inverse: QQ−1 = I = QQ T , with
I being the identity matrix. SVD is especially useful for estimating
node distances on networks (Section 40.2).
Along with eigendecomposition, the two most common and useful
matrix decomposition tools are the Principal Component Analysis
 matrices 83

Day Temp (°C) Wind (km/h) Sunlight (%) Rain (mm) Snow (mm)
1 27 10 80 2 0
2 26 1.2 95 1 0
3 32 7.6 100 0 0
4 12 2.3 12 20 0
5 14 3.8 8 25 0
6 6 0.2 24 40 1
7 4 0.1 2 8 30
8 2 0.9 4 1 40
9 −1 1.1 4 0 80
Figure 5.16: A table recording
(PCA) and the Non-Negative Matrix Factorization (NMF). in a matrix the characteristics of
To understand PCA, suppose that your matrix is just a set of some days.
observations and variables. Each row of the matrix is an observation
and each column is a variable. PCA, like NMF, is used to summarize
this matrix of data. If two columns/variables are correlated it means
they contain redundant information. Thus, you’re after a way to
describe your data in such a way that each variable has no redundant
information.
Figure 5.16 shows an example: each row is a day and each col-
umn is some measurement taken in that day – the temperature,
wind speed, the millimeters of rain/snow that fell that day, etc. You
might expect that some of these variables might be correlated. For
instance, it is very difficult to have a single millimeter of snow if
the temperature is above a certain value. Rather than describing a
day by all variables, you want to describe it by its similarity with an
“archetypal” day: is this a snow day or a rain day?

Day PC1 PC2

Figure 5.17: The first two princi-
1 0.2 −0.05
pal components of the matrix in
2 −0.05 0.1
Figure 5.16.
3 −0.1 −0.1
4 2.6 −0.01
5 2.9 0
6 3.2 0.35
7 0.3 2.4
8 0.1 2.6
9 −0.05 2.8
This is the aim of PCA. Let’s repeat the previous paragraph math-
ematically: you want to transform your correlated vectors in a set
of uncorrelated, or orthogonal, vectors which we call “principal
components”. Each component is a vector that explains the largest
possible amount of variance in your data, under the condition of be-
ing orthogonal with all the other components. You can have as many
84 the atlas for the aspiring network scientist

components as you have variables, but usually you want much fewer
– for instance two, so you can plot the data. That is because the first
component explains the most variance in the system, the second a bit
less, and so on, until the last few components which are practically
random. Thus you want to stop collecting components after you’ve
taken the first n, setting n to your delight. In Figure 5.17, I collect the
first two – they’re there for illustrative purposes so don’t be shocked
if you realize they’re not really orthogonal.

9
8 Figure 5.18: A scatter plot with
7 the first principal component
2nd Comp

of the matrix in Figure 5.16

on the x axis and the second
component on the y axis.

6
2 4 5
3 1
1st Comp

PCA is extremely helpful when performing data clustering. Sup-

pose that we’re looking only at the first two principal components
of our matrix describing our days. We can make a two dimensional
scatter plot of the system, with one point per day. It might look like
Figure 5.18. This seems successful, because we can clearly see three
clusters: days dominated by the first component (in blue), days domi-
nated by the second component (in green), and days which have low
values in both (in red). When we look at the original data, we might
recognize that the first class of days had high rain precipitation, the
second high snow precipitation, and the third group was mostly
sunny days. In this sense, PCA aided us in finding our archetypal
days: the first component describes the archetypal rainy day, while
the second component describes the archetypal snowy day.
PCA has no restrictions in the way it builds the principal compo-
nents, besides the fact that all these components must be orthogonal
with each other. This means that you might end up with compo-
nents with negative values, as we do in Figure 5.17. This might not
be ideal. What does it mean for a day to be a “negative rainy day”?
PCA is interpretable, but sometimes the intepretation can be a bit...
confusing.
Non-Negative Matrix Factorization solves this problem. With-
out going into technical details, NMF is PCA with the additional
constraint that no component can have a negative entry – hence
the “Non-Negative” part in the name. At a practical level, if there
 matrices 85

were no negative entries in Figure 5.17, then the two components

in that figure could be results of NMF. This additional constraint
comes at the expense of some precision: PCA can fit the data better
because it does not restrict its output space. However, usually, NMF
components are more easy to interpret.
Given their links to data clustering, both PCA and NMF are exten-
sively used when looking for communities in your networks (Part
IX).

Tensor Decomposition

In Section 5.1, I mention tensors as an instrument to represent the

adjacency matrix of a multilayer graph. I treat them as a sort of
multidimensional matrix: the first two dimensions are the nodes
and the third dimension is the layer. You can think of a tensor as a
cuboid and a slice of it is the adjacency matrix of one layer. If you
find it difficult to picture this in your head, don’t worry: you’re not
alone. That is why many researchers put effort into finding ways to
decompose tensors in lower-dimensional representations that can
sum up their main properties. This process is generally known as
“tensor decomposition”.
Tensor decomposition is a general term encompassing many
techniques to express a tensor as a sequence of elementary operations
(addition, multiplication, etc) on other, simpler tensors. For instance,
you can represent a 3D tensor as a combination of three vectors, one 6
Tamara G Kolda and Brett W Bader.
per dimension. Or as a matrix and a vector. You want to do this to Tensor decompositions and applications.
SIAM review, 51(3):455–500, 2009
solve complex network analyses on multilayer networks by taking 7
Lieven De Lathauwer, Bart De Moor,
the full dimensionality into account at the same time, rather than and Joos Vandewalle. A multilinear sin-
performing the analysis on each layer separately and then merge the gular value decomposition. SIAM journal
on Matrix Analysis and Applications, 21
results somehow. Examples of applications of tensor decomposition
(4):1253–1278, 2000
range from node ranking (Chapter 11), to link prediction (Part VI), to 8
Elina Robeva and Anna Seigal. Sin-
community discovery (Part IX). gular vectors of orthogonally decom-
posable tensors. Linear and Multilinear
I am going to mention very briefly only two of these techniques:
Algebra, 65(12):2457–2471, 2017
tensor rank decomposition and Tucker decomposition. You should 9
Frank L Hitchcock. The expression
look elsewhere for a more complete treatment of the subject6 . There of a tensor or a polyadic as a sum of
products. Journal of Mathematics and
also exists a tensor SVD7 , 8 , but it is relatively similar to a special case
Physics, 6(1-4):164–189, 1927
of Tucker decomposition, so I will not cover it. 10
Richard A Harshman et al. Founda-
Tensor rank decomposition is the oldest of the two9 and has tions of the parafac procedure: Models
and conditions for an" explanatory"
historically been referred to as PARAFAC10 or CANDECOMP11 . Let’s
multimodal factor analysis. 1970
say you have your nice 3D tensor A representing your multilayer 11
J Douglas Carroll and Jih-Jie Chang.
network. This is a three dimensional matrix of dimensions |V | × |V | × Analysis of individual differences in
| L| – for simplicity here we assume that the network is not bipartite, multidimensional scaling via an n-
way generalization of “eckart-young”
thus two dimensions are the same, however the method also works decomposition. Psychometrika, 35(3):
if all three dimensions are different. Tensor rank decomposition tells 283–319, 1970
86 the atlas for the aspiring network scientist

you that there is a way to decompose A as the following combination:

A∼ ∑ λ k a k ◦ bk ◦ c k .
k
Here, A = a ◦ b ◦ c → Aijk = ai b j ck which means that ◦ represents
the outer product. a and b are vectors of length |V | and c is a vector
of length | L|. Finally, λk is just a scaling factor that tells us how
much to count the kth element of the sum. The convention is to call
λk ak ◦ bk ◦ ck a component, while the vectors are the factors.

c1 c2 ck
a1 a2 ak

|V| ~ + +…+
|L| b1 b2 bk
|V|

If you find difficult to understand what’s going on by just looking Figure 5.19: A schema of tensor
at the formula, take inspiration from Figure 5.19. What this operation rank decomposition.
does is to find the right set of one-dimensional vectors a, b and c such
that, once they are scaled by factors λ, they can best represent the
full tensor A. At that point, you are working in a lower dimensional
space and all the rest of linear algebra starts making sense again.
How many components does this sum have? Or, in other words,
how big should k be to approximate A? That depends on the rank
of the tensor. Unfortunately, calculating the rank of a tensor isn’t as
easy as calculating the rank of a matrix. The rank of a matrix is the
number of columns (or rows) that are lineraly independent from each
other. The definition is the same for a tensor but, in this case, there
12
Joseph B Kruskal. Three-way arrays:
is no straightforward algorithm to determine it12 . What happens is
rank and uniqueness of trilinear
that, to find the rank of a tensor, you would literally apply the rank decompositions, with application to
decomposition with different k values and find the one that works arithmetic complexity and statistics.
Linear algebra and its applications, 18(2):
the best. 95–138, 1977
Tucker decomposition13 takes a different approach. It decomposes 13
Ledyard R Tucker. Some mathematical
our tensor A into a smaller core tensor and a set of matrices. If we notes on three-mode factor analysis.
Psychometrika, 31(3):279–311, 1966
keep our simplified case of a 3D tensor representing an adjacency
matrix, mathematically speaking the Tucker factorization does:

A ∼ A × X × Y × Z.
Here, A is the core tensor, whose dimensions are smaller than A’s.
X, Y, and Z are matrices which have one dimension in common with
A and the other in common with A – so that the matrix multiplica-
tion of them with A reconstructs a tensor with A’s dimensions.
 matrices 87

Figure 5.20: A schema of Tucker

decomposition.

|V| ~
|L|
|V|

Again, for the visual thinkers, Figure 5.20 might come in handy.
In Tucker decomposition you have the freedom to choose the di-
mensions of the core tensor A. Smaller cores tend to be more inter-
pretable, because they defer most of the heavy lifting to X, Y, and Z.
However, they also tend to make the decomposition less precise in
reconstructing A.

5.5 Summary

1. You can represent a graph with an adjacency matrix. The matrix

has a row/column per node, and cells are equal to one if the two
nodes are connected, zero otherwise.

2. The stochastic adjacency matrix is a row-normalized (or column-

normalized depending on the convention) adjacency matrix,
whose rows (or columns) sum to one. It describes transition proba-
bilities from one node to another.

3. The degree matrix D is a matrix having the degree of the node

in the main diagonal and zero everywhere else. If you subtract
the adjacency matrix from the degree matrix you obtain the graph
Laplacian, which is widely used in many network applications.

4. Transposing means mirroring the matrix on its main diagonal. It

can be used to flip the direction of edges in a directed network,
or looking at two different modes of connections in a bipartite
network.

5. The Auv element of the matrix result of matrix multiplication is

the sum of the products of the uth row and the vth column of the
two multiplied matrices.

6. A matrix’s eigenvector is a vector that gets stretched by a factor

(the eigenvalue) but does not change direction when multiplied by
that matrix. Many eigenvectors of the stochastic adjacency and the
Lapacian are useful for different network tasks.
88 the atlas for the aspiring network scientist

7. In Principal Component Analysis, we deconstruct a matrix in its

“principal components”: uncorrelated vectors that express most of
the variation in the (correlated) values of the matrix. If no value
in these vectors can be negative, we call it “Non-Negative Matrix
Factorization”.

8. Tensor decomposition is an operation expressing a multidimen-

sional matrix as the result of the sum/product or other, simpler,
tensors.

5.6 Exercises

1. Calculate the adjacency matrix, the stochastic adjacency ma-

trix, and the graph Laplacian for the network in [Link]
[Link]/exercises/5/1/[Link].

2. Given the bipartite network in [Link]

exercises/5/2/[Link], calculate the stochastic adjacency matrix
of its projection. Project along the axis of size 248. (Note: don’t
ignore the weights)

3. Calculate the eigenvalues and the right and left eigenvectors of

the stochastic adjacency obtained in the previous question. Make
sure to sort the eigenvalues in descending order (and sort the
eigenvectors accordingly). Only take the real part of eigenvalues
and eigenvectors, ignoring the imaginary part.
 Part II

Simple Properties
6
Degree

So far we just described graph representations. We haven’t actually

done anything with them. Here, we start probing their properties, to
say things about their nodes, edges, and structures. In this chapter
we deal with the simplest possible statistics of a node: its degree.
This is such a basic concept that I actually already mentioned it
multiple times without defining it. It’s hard not to, when talking
about networks.
The intuition behind the degree is easy to understand in a social
network. What is the simplest way for you to know how well you’re
doing in a society? Well, you could look around you, see the friends
you have, and count them. This is the degree. I’ve got my decent
couple of hundreds Facebook friends, like almost everyone else.
But some people are superstars, and count the number of their
acquaintances in the thousands. How many people does Brad Pitt
know? Probably a couple orders of magnitude more than me. Those
are the kinds of differences you can quantify by calculating the
degree of all nodes in your network.

2 Figure 6.1: A network where

I labeled each node with its
2 3
degree.
1

The degree of a node is simply the number of edges incident 1

Reinhard Diestel. Graph theory. Springer
to it1 , as Figure 6.1 shows. Or the number of its connections. Or – Publishing Company, Incorporated,
given some assumption that I’ll break later on – the number of its 2018
neighbors. It is the first, most fundamental measure of structural im-
portance of a node. A node of high degree ought to be an important
node in the network. If it were to disappear, many nodes would lose
 degree 91

a connection.
The degree is a property of a node. Let’s call k v the degree of node
v. We can aggregate the degrees of all nodes in a network to get a
“global” information about its connectivity. The most common way to
do it is by calculating the average degree of a network. This would
be k̄ = ∑ k v /|V |, however it’s much simpler to remember that
v ∈V
k̄ = 2| E|/|V |. The average degree of a network is twice the number
of edges divided by the number of nodes. Why twice? Because each
edge increases by one the degree of the two nodes it connects.
In a social network, this is how many friends people have on
average. What would that number be in your opinion? If we have
a social network including two billion people, what’s the average
degree? It turns our that this number is usually ridiculously lower
than one would expect, because – as we’ll see in Section 9.1 – real 2
Anna D Broido and Aaron Clauset.
networks are sparse2 . Scale-free networks are rare. Nature
We call a node with zero degree, a person without friends, an communications, 10(1):1017, 2019
isolated node, or a singleton. A node with degree one is a “leaf”
node: this term comes from hierarchies, where nodes at the bottom
– the leaves of the tree – can only have one incoming connection
without outgoing ones. The sum of all degrees is 2| E|, which implies
that any graph can only have an even number of nodes with odd 3
Leonhard Euler. Solutio problematis ad
degree3 – otherwise the sum of degrees would be odd and thus it geometriam situs pertinentis. Commen-
cannot be two times something. tarii academiae scientiarum Petropolitanae,
pages 128–140, 1741

2
Figure 6.2: A graph and its
2 3
degree sequence.
1

2
[3, 2, 2, 2, 1]
(a) (b)
4
Michael Molloy and Bruce Reed.
The size of the giant component of a
A degree sequence is the list of degrees of all nodes in the net- random graph with a given degree
work4 , 5 . Typically, we sort the nodes in descending degree, so you sequence. Combinatorics, probability and
computing, 7(3):295–305, 1998
always start with the node with maximum degree and you go down 5
Béla Bollobás, Oliver Riordan, Joel
until you reach the node with the lowest degree. Figure 6.2 shows an Spencer, and Gábor Tusnády. The degree
example. sequence of a scale-free random graph
process. Random Structures & Algorithms,
Note that not all lists of integers are valid degree sequences. Some 18(3):279–290, 2001
lists cannot generate a valid graph. The easiest case to grasp is if
they contain an odd number of odd numbers. As we just saw, the
6
Gerard Sierksma and Han Hoogeveen.
degree sequence must sum to an even number (2| E|), thus a sequence
Seven criteria for integer sequences
summing to an odd number cannot describe a simple undirected being graphic. Journal of Graph theory, 15
graph6 . We call all valid sequences “graphic”. We’ll see that there are (2):223–231, 1991

other, more subtle, requirements for a graphic sequence.

92 the atlas for the aspiring network scientist

6.1 Degree Variants

Of course, the degree definition I just gave only makes sense in the
world of undirected, unweighted, unipartite, monolayer networks.
We had two whole chapters detailing when such a simple model
doesn’t work in complex real scenarios. We need to extend the
definition of degree to take into account all different graph models
we might have to deal with.

Directed
As we saw in Section 3.2, edges can have a direction, meaning that
the edge going from u to v doesn’t necessarily point back from v
to u. Such is life. In directed graphs you can obviously still keep
counting the degree as simply the number of connections of a node,
but there is a more helpful way to think about it. You might want
to distinguish the people who send a lot of connections – but don’t
necessarily see them reciprocated –, and those who are the target of a
lot of friends requests – whether they accept them or not.

1 0 Figure 6.3: (a) A network where

I labeled each node with its
in-degree. (b) A network where
I labeled each node with its
1 2 out-degree.

1 1
2 0

0 2

(a) (b)
So we split the concept in two parts, helpfully named in-degree 7
Frank Harary, Robert Zane Norman,
and out-degree7 , 8 . As one can expect, the in-degree is the number of and Dorwin Cartwright. Structural
models: An introduction to the theory of
incoming connections. If we represent a directed edge as an arrow, directed graphs. Wiley, 1965
the in-degree is the number of arrow heads attached to your node. 8
Jørgen Bang-Jensen and Gregory Z
See Figure 6.3(a) for a helpful representation. The out-degree is the Gutin. Digraphs: theory, algorithms and
applications. Springer Science & Business
number of outgoing connections, the number of arrow tails attached Media, 2008
to your node. I show the out-degree of the nodes in my example in
Figure 6.3(b).
A directed graph’s degree sequence is now a list of tuples. The
first element of the tuple tells you the indegree, while the second
 degree 93

element tells you the outdegree. Or you can have two sequences, but
you need to make sure that the nth positions of the two sequences re-
fer to the same node. If the two sequences are the same, meaning that
every node has the same in- and out-degree, we have a “balanced”
graph.

Weighted
Most of the time, people would not adapt the definition of degree
when dealing with weighted networks. Many network scientists like
how the standard definition works in weighted graphs, and keep it
that way. The degree is simply the number of connections a node has.

8 Figure 6.4: A weighted network

3 5 where I labeled each edge with
4 its weight and each node with
10 3 3 its weighted degree.
1 2
3
9
Alain Barrat, Marc Barthelemy, Ro-
mualdo Pastor-Satorras, and Alessandro
Other people don’t9 , so it’s better to mention it here and get over Vespignani. The architecture of complex
with it. In the case of weighted networks, one might be interested in weighted networks. Proceedings of the
the total weight incident into a node. We would call such quantity national academy of sciences, 101(11):
3747–3752, 2004a
the “weighted degree” or “node strength”10 . Node strengths are key 10
Tore Opsahl, Filip Agneessens, and
concepts in investigating propagating failures on networks (Section John Skvoretz. Node centrality in
19.3) and in some network backboning techniques (Section 24.5). weighted networks: Generalizing
degree and shortest paths. Social
Node strengths work exactly how you would expect them to: to networks, 32(3):245–251, 2010
get v’s weighted degree you sum the weights of all the edges incident
to v. Figure 6.4 shows an example. The advantage of this definition
is that it reduces to the classical degree definition if your network is
unweighted – that is to say that all its edge weights are equal to one.
By separating the unweighted count of connections (degree) from
the weighted sum of connections (weighted degree), we capture two
distinct notions of connectivity. One can have a node with enormous
strength but low degree – a core router on the internet with few high-
bandwidth connections – and a “peripheral” router on your street –
which has a large number of low-bandwidth connections.
The reasons to do so are many. For instance, if you’re looking a
road graph, each edge represents a trait of road. It might be weighted
with the number of cars passing through it per unit of time. Nodes,
in this case, are road intersections. A weighted degree will tell you
how many cars per unit of time want to clear that particular intersec-
tion. If the number is too high, you might be in trouble!
94 the atlas for the aspiring network scientist

Bipartite
The bipartite case doesn’t need too much treatment: the degree is
still the number of connections of a node. It doesn’t matter much
that for V1 nodes it is gained exclusively via connections to V2 nodes
and viceversa. However, there’s a little change when one uses a
matrix representation that it’s worthwhile to point out. Assuming
A as a binary adjacency matrix (not stochastic), in the regular case
the degree is the sum of the rows: the sum of first row tells you the
degree of the first node, and so on.

Figure 6.5: Calculating the

degree of a bipartite network
via its adjacency matrix A and
its transpose A T . The first V1
node has degree equal to two
(the sum of the first row is two).
(a) A (b) A T
The first V2 node has degree
equal to one, which you can
In a bipartite network that will only tell you the degree of the V1
calculate either by summing
nodes. You won’t know anything about the V2 nodes if you only look
the first column of A, or by
at row sums. You can fix the problem in two, equivalent, ways. You
summing the first row of A T .
can either looking at the column sums, or you can look at the row
sums of A T , the transpose of A. A T ’s rows are A’s columns and vice
versa, so the equivalence between these two approaches should be
self-evident – if it isn’t, try to play with Figure 6.5.
Just like directed graphs, also bipartite graphs have two degree
sequences, one for V1 nodes and the other for V2 nodes. They both
sum to the same value: | E|, implying that, in this case, you can have 11
Armen S Asratian, Tristan MJ Denley,
an odd number of odd degree nodes in each node type11 . and Roland Häggkvist. Bipartite
graphs and their applications, volume 131.
Cambridge University Press, 1998
Multigraph
When I introduced the degree I said that it can be the number of a
node’s connections or the number of its neighbors. These two were
assumed to be interchangeable, because each edge in a simple graph
will bring you to a distinct neighbor. Say that k u is u’s degree, and
Nu the set of its neighbors. In a simple graph, k u = | Nu | – assuming
there are no self-loops or, if there are, that Nu can contain u itself.
That is not the case in a multigraph. Since we allow parallel edges,
you can follow two distinct connections and end up in the same
neighbor. So we need to solve this ambiguity. The way I saw most
commonly accepted is to keep the degree (k u ) as the number of
connections of a node. The number of neighbors of a node (| Nu |) will
be just that: the number of neighbors. So, in a multigraph k u 6= | Nu |
 degree 95

or, to be more precise, k u ≥ | Nu |.

Multilayer
The multilayer case is possibly the most complex of them all. At first,
it doesn’t look too bad. The degree is still the number of connections
a node has. Then you realize that there are some connections you
shouldn’t count. For instance, no one – that I know of – counts the
interlayer coupling connections as part of the degree. It’s easy to
see why: these are not connections that lead you to a neighbor in a
proper sense. They lead you to... a different version of yourself.

Figure 6.6: Should we really say

that the degree of this isolated
node is ten just because there
are five layers in the network
and we couple them with each
other? Eight out of ten cats say
Even if we want to ignore this quirk, counting these connections “no”.
won’t really give you meaningful information. If you have a one-
to-one multilayer network in which all nodes are part of all layers,
12
Michele Berlingerio, Michele Coscia,
they are all going to have the same number of inter-layer couplings.
Fosca Giannotti, Anna Monreale, and
Sometimes, this number can be quite high. If you have five layers Dino Pedreschi. Multidimensional
and you connect all identities of the same actor across layers, you networks: foundations of structural
analysis. WWW, 16(5-6):567–593, 2013a
effectively have a clique (see Section 9.3) of inter-layer couplings. 13
Manlio De Domenico, Albert Solé-
If you count those as part of the degree, this actor would have a Ribalta, Emanuele Cozzo, Mikko Kivelä,
degree starting from ten – as I show in Figure 6.6 –, which would Yamir Moreno, Mason A Porter, Sergio
Gómez, and Alex Arenas. Mathematical
be unreasonable. You could have fewer inter-layer coupling using formulation of multilayer networks.
different coupling strategies, but that wouldn’t change the substance. Physical Review X, 3(4):041022, 2013
Since each layer is a network on its own, it is natural to want to 14
Federico Battiston, Vincenzo Nicosia,
and Vito Latora. Structural measures for
have a measure telling us the degree of a node in a particular layer. multiplex networks. Physical Review E,
So an actor can have many degrees: one per layer, and a general one, 89(3):032804, 2014
which we can define as the sum of each layer’s degree. However, 15
Manlio De Domenico, Albert Solé-
Ribalta, Elisa Omodei, Sergio Gómez,
things can get complicated with a many-to-many mapping. In that and Alex Arenas. Ranking in inter-
case, the actor can “own” more than one node in a layer. Each node connected multilayer networks reveals
has its own degree, but how much do they contribute to the actor’s versatile nodes. Nature communications, 6:
6868, 2015d
degree? The answer might vary, depending on what you’re interested 16
Manlio De Domenico, Albert Solé-
in calculating. Ribalta, Sergio Gómez, and Alex
Arenas. Navigability of interconnected
One can also combine layers to do all sorts of interesting stuff.
networks under random failures.
I’m going to give you some examples from a paper of mine12 , with Proceedings of the National Academy of
the caveat that the space of actual possibilities is much vaster than Sciences, 111(23):8351–8356, 2014
17
Federico Battiston, Vincenzo Nicosia,
this13 , 14 . What follows is also very related to the multilayer concept
and Vito Latora. Efficient exploration
of “versatility”: the ability of a node to be a relevant central node in of multiplex networks. New Journal of
different layers15 , 16 , 17 . Physics, 18(4):043035, 2016
96 the atlas for the aspiring network scientist

Figure 6.7: A multigraph rep-

resentation of a multilayer net-
work with one-to-one mapping,
where the edge color encodes
layer in which it appears.

Consider Figure 6.7. Let’s call u the bottom node, the one with two
orange edges, a blue and a purple one. We can see that its degree
is four (four edges), and its neighbor set is of size three: u has three
neighbors, | Nu | = 3.
Now, we can count the size of the neighbor set per layer too, or
Nu,l . In the orange layer u has two neighbors (| Nu,l | = 2), in the blue
and purple one it has only one (| Nu,l | = 1). There is a difference
between the neighbors in the orange layer and the ones in the other
layers. If u wants to communicate with them, it has to use the orange
layer: there is no alternative. On the other hand, if the blue layer
were to disappear, u could still use the purple one, and vice versa.
This observation is at the basis of the definition of the “exclusive
neighbor” set, or N XOR . Given a node u and a layer l, the Nu,l XOR

contains those neighbors of u that can be reached exclusively via l. If

there is an alternative path, those neighbors are not part of Nu,l XOR . So
XOR XOR
| Nu,l | = 2, if l is the orange layer, but | Nu,l | = 0 in the other two
cases. So the exclusive neighbor gives us a rather intuitive measure:
how many neighbors would u lose if layer l were to disappear?
We can use Nu,l and Nu,l XOR to establish some generalized degree

definitions, establishing the importance of l for u. For instance, the

Layer Relevance of l for u is the fraction of u’s neighbors that u can
reach through l, or | Nu,l |/| Nu |. In Figure 6.7 that’s 2/3 for the orange
layer, and 1/3 for both the blue and the purple layers. The exclusive
variant of Layer Relevance is the fraction of u’s neighbors that u can
reach through l and l alone: | Nu,l XOR | / | N |. In Figure 6.7 that’s still
u
2/3 for the orange layer, but it turns to zero for both the blue and the
purple layers.
We can also have a normalized version of Layer Relevance such
that it always sums to one for all nodes. In this version, for every
pair of connected nodes (u, v), each layer in which this connection
appears does not contribute one to the sum, but 1/| Lu,v |, where
| Lu,v | is the number of layers in which u and v are neighbors of each
other. In my example, the normalized Layer Relevance of the orange
dimension for u is still 2/3, but it turns to 1/6 for the blue and the
 degree 97

purple one, because they have to share fairly the remaining 1/3 of u’s
neighbors.

Hyper
As one might expect, allowing edges to connect an arbitrary number
of nodes – rather than just two – does unspeakable things to your
intuition of the degree. We can still keep our usual definition: the
degree in a hypergraph is the number of hyperedges to which a node 18
Paul Erdős and Miklós Simonovits. Su-
belongs – or: the number of its hyper-connections18 , 19 . However, if persaturated graphs and hypergraphs.
Combinatorica, 3(2):181–192, 1983
you take any step further, all hell breaks loose. The number of neigh- 19
Alain Bretto. Hypergraph theory: An
bors has no relationship whatsoever with the number of connections: introduction. Mathematical Engineering.
with a single hyperedge you can connect a node with the entirety of Cham: Springer, 2013
the network. Also the average degree is something tricky to calculate.
Forget about k̄ = 2| E|/|V |: if a single hyperedge can connect the
entire network, then | E| = 1, but k̄ = |V |.
Things are a bit less crazy for uniform hypergraphs – where we
force hyperedges to always have the same number of nodes. Which
might explain why they’re a much more popular thing to study,
rather than arbitrary hypergraphs.

6.2 Degree Distributions

The degree of a node only gives you information about that node.
The average degree of a network gives you information about the
whole structure, but it’s only a single bit of data. There are many
ways for a network to have the same average degree. It turns out that
looking at the whole degree distribution can shed light on surprising
properties of the network itself. Since degree distributions can be so
important, generating and looking at them is a second nature for a
network scientist. As a consequence, there are a lot of standardized
procedures you want to follow, to avoid confusing your reader by
breaking them.

# Nodes
1

2 2
Figure 6.8: The degree scatter
plot (left) of the graph on the
2
Degree right.
98 the atlas for the aspiring network scientist

Let’s break down all the components of a good degree distribution

plot. First, the basics. What’s a degree distribution? At its most
simple, it is just a degree scatter plot: the number of nodes with a
particular degree. The degree should be on the x axis and the number
of nodes on the y axis, just as I do in Figure 6.8. Commonly, one
would normalize the y axis by dividing its values by the number of
nodes in the network. At this point, the y axis is the probability of
a node to have a degree equal to k, not simply the node count. That
makes it easier to compare two networks with a different node count.

0.5 1
0.45
0.4 Figure 6.9: The degree distri-
0.35 0.1
0.3
bution of the protein-protein
p(k)

p(k)

0.25
0.2 0.01
interaction network. The distri-
0.15
0.1
butions are the same, but in (a)
0.05 0.001
0
we have a linear scale for the x
0 10 20 30 40 50 60 70 80 90 100 1 10 100
k k
and y axes, which is replaced in
(b) by a log-log scale.
(a) (b)
Figure 6.9(a) shows you the degree distribution of protein-protein
interaction for the Saccharomyces Cerevisiae, the beer bug. An inter-
esting pattern is that there are lots of nodes with few interactions,
and few nodes with many. As a consequence, we end up with all our
datapoints concentrated in the same part of the plot, and it’s difficult
to appreciate both the low- and the high-degree structure. These
degree patterns are more evident and easy to see when represented
on a log-log scale, as Figure 6.9(b) shows, which stretches out the
low-degree area while compressing the high-degree one.
So, we just discovered that this protein-protein interaction network
has something peculiar. The baseline assumption would be that
nodes connect at random. If that were the case, we would expect the
degree to distribute normally, in a nice bell-shape – see Chapter 13.
But Figure 6.9(b) is not what a normal distribution looks like. The
vast majority of nodes have a very low degree, and a few giant hubs
have a degree much larger than average. Is this common?
Yes it is. Most real world network would show such a broad 20
Holger Ebel, Lutz-Ingo Mielsch, and
distribution: email exchanges20 , synapses in the brain21 , internal cell Stefan Bornholdt. Scale-free topology of
interactions22 . Take a look at the degree distribution zoo in Figure e-mail networks. Physical review E, 66(3):
035103, 2002
6.10. To put it simply: in most networks we have many orders of 21
Victor M Eguiluz, Dante R Chialvo,
magnitude between the minimum and the maximum degree (x axis), Guillermo A Cecchi, Marwan Baliki,
and between the most and least popular degree value (y axis). This is and A Vania Apkarian. Scale-free brain
functional networks. Physical review
not what scientists initially expected. And when things are not as we letters, 94(1):018102, 2005
expected, we all get excited and start wonder why. 22
Reka Albert. Scale-free networks in
Before exploring these questions we need to finish our deep dive cell biology. Journal of cell science, 118(21):
4947–4957, 2005
into how to generate and visualize a proper degree distribution. The
 degree 99

100 100

Figure 6.10: The degree dis-

10-1 10-1
tributions of many real-world
10-2 10-2
p(k)

p(k)
networks: (a) coauthorship in
10-3 10-3
scientific publication [Leskovec
10-4 10-4 et al., 2007b]; (b) coappearance
1 10 100 1000 100 101 102 103 104
k k
of characters in the same comic
book [Alberich et al., 2002]; (c)
(a) (b)
100 100
interactions of trust between
10-1
PGP users [Boguñá et al., 2004];
10-1

10-2
(d) connections through the
p(k)

p(k)
10-2
10-3
Slashdot platform [Leskovec
10-3 et al., 2009].
10-4
-4
10
1 10 100 1000 100 101 102 103 104
k k

(c) (d)

disadvantages of the degree scatter plots is that they’re a bit messy.

The physicists in the audience would want a true functional form.
But one cannot do that if we have such a broad scatter, especially for
high degree values: the wide range of degree values carried only by a
node in the network generate what we call a “fat tail”, and we don’t In case you were wondering: yes, I
23

want that23 . am body shaming a distribution.

100
3 3
-1 10 10
10

102 102
p(k)

p(k)

p(k)

-2
10

1 1
10-3 10 10

10-4 10
0
10
0
0 1 2 3 0 1 2 3
1 10 100 1000 10 10 10 10 10 10 10 10
k k k

(a) Regular scatter. (b) Equal size binning. (c) Powerbinning.

Figure 6.11: The degree dis-
There are two ways to do it. The first is to perform a power- tribution of the PGP trust
binning of your x axis24 . Rather than drawing a point for each network.
distinct degree value you have in your network, you can lump to-
24
Staša Milojević. Power law distribu-
gether values into larger bins. Using equally-sized bins – with the
tions in information science: Making the
same increment for the entire space – doesn’t work very well: for low case for logarithmic binning. Journal
degree values you’re putting together very populated bins, while of the American Society for Information
Science and Technology, 61(12):2417–2425,
for high degree values usually the distribution is so dispersed that 2010
you aren’t actually grouping together anything. See Figure 6.11(b)
for an example. In Figure 6.11(b) we completely lost the head of the
distribution – the low degree values are all lumped together – and,
while less prominent, the fat tail is still there.
100 the atlas for the aspiring network scientist

That’s why you do power binning. You start with a small bin size,
usually equal to 1. Then each bin becomes progressively larger, by a
constant multiplicative factor. At first, the bins are still small. But, as
you progress, the bins start to be large enough to group a significant 25
An example of power binning,
portion of your space25 . A good power bin choice can make the plot starting with size 1 and increas-
clearer, as the one in Figure 6.11(c). In Figure 6.11(c) we saved the ing the bin size by 10% at each
step: [1, 2, 3, 5, 6, 8, 9, 11, 14, ...,
head of the distribution and further reduced the fat tail.
1410, 1552, 1709, 1881, 2070, 2278, ...]
# Nodes p(k>=x)
Figure 6.12: The degree scatter
plot (left) and its corresponding
complement of the cumulative
distribution (CCDF).

Degree x

One can do better than Figure 6.11(c), that’s why in network

papers you rarely see power-binned distributions. An issue of power-
binning is that it forces you to make a choice: to determine the bin
size function. Having a choice is a double-edged sword: it opens you
to the possibility of tricking yourself into seeing a pattern that is not
there.
The most common way to visualize degrees is by drawing cumula-
tive distributions (CDF), or – to be more aligned with the convention
you’ll see everywhere – the complement of a cumulative distribution
(CCDF). We can transform a degree histogram into a CCDF by chang-
ing the meaning of the y-axis. Rather than being the probability of
finding a node of degree equal to k, in a CCDF this is the probability
of finding a node of degree k or higher. This is not a scattergram any
more, but a function, which helps us when we need to fit it. Figure
6.12 shows an example, where we go from a degree histogram to its
equivalent CCDF.

1 1

Figure 6.13: The degree distri-

0.1 0.1
bution of the protein-protein
p(k>=x)
p(k)

0.01 0.01
interaction network. The distri-
butions are the same and are
0.001 0.001
both in log-log scale, but in (a)
1 10 100 1 10 100
k x
we have the degree histogram,
and in (b) we show the CCDF
(a) (b)
version (with the best fit in
We can see the relationship of our protein-protein network more
blue).
clearly in Figure 6.13. It appears that, in log-log space, the relation-
ship between degree and the number of nodes with a given degree
 degree 101

is fixed. This relationship can be approximated with a straight line

– at least asymptotically: in Figure 6.13(b) you can see that the head
doesn’t really fit. Is this a coincidence, or does it have meaning? To
answer this question we need to enter in the wonderful world of
power-law degree distributions and scale free networks.

6.3 Power Laws and Scale Free Networks

1
Figure 6.14: An example of
power law, showing how the
0.1
red line always goes down by
p(k>=x)

the same proportion as its left

0.01 movement, no matter if we look
a head, middle or tail.
0.001

1 10 100
x

Figure 6.15: The distribution

of crater sizes in the moon is
an example of quasi-power
law, with many tiny ones and
a huge one spanning the entire
picture. If the moon were an
ideal infinite plane and we
could zoom in indefinitely, the
picture we would get would be
In statistics, a power law is a functional relationship between equivalent to the original one,
two quantities, where a relative change in one quantity results in a i.e. the scale at which we’re
proportional relative change in the other quantity. The relation is observing the moon would not
independent of the initial size of those quantities: one quantity varies influence the observation result.
as a power of another. I show what I mean in Figure 6.14: each time
you move on the x-axis by a specific increment, you also always move
on the y-axis by a fixed function of that x increment, no matter where
you are in the distribution (head, tail, or middle).
You can find power laws in nature in many places: the frequencies
of words in written texts, the distribution of earthquake intensities,
etc... To grasp the concept you need a visual example, and my fa-
vorite is moon craters26 . You can see in Figure 6.15 there are a lot of
26
Mark EJ Newman. Power laws,
pareto distributions and zipf’s law.
tiny craters caused by small debris and a huge one. This is fractal Contemporary physics, 46(5):323–351,
self-similarity: if the moon were an infinite plane, you could zoom 2005b
102 the atlas for the aspiring network scientist

in and out the picture and the size distributions would be the same.
This is the scale invariance I’m talking about: no matter the zoom, the
picture looks the same – obviously in reality it doesn’t, because the
moon isn’t an infinite plane, and you cannot zoom in infinitely many
times (in fact, whether finite systems can actually generate power 27
Michael PH Stumpf and Mason A
laws is a controversial topic27 ). Porter. Critical truths about power laws.
Science, 335(6069):665–666, 2012
1
Figure 6.16: An example of
power law in a CCDF. The ver-
0.1
tical gray bar shows that the
p(k>=x)

point in the distribution is asso-

0.01 ciated with degree equal to two.
The horizontal gray bar shows
0.001 that this degree correspond
to a probability of around 0.5.
1 10 100
This means that half of the net-
x
work has a degree equal to or
This applies to networks too! There are many studies showing greater than two. Or, in other
how some networks possess this sort of self-similar structure at differ- words, that the other half of the
ent scales28 , 29 – i.e. they are fractals. This is not necessarily the same network has degree equal to
thing as looking at the degree distribution30 – although classifying one.
28
Chaoming Song, Shlomo Havlin, and
a network as “scale free” by looking at its degree distribution is a Hernan A Makse. Self-similarity of
common operation in the literature and it is also the stance I’ll adopt complex networks. Nature, 433(7024):
392–395, 2005
from now on in the book. 29
M Angeles Serrano, Dmitri Krioukov,
In Figure 6.16, I show the usual CCDF of the protein-protein and Marián Boguná. Self-similarity of
network: there we see that 50% of the nodes have a degree of 2 or complex networks and hidden metric
spaces. Physical review letters, 100(7):
more. This means that 50% of the nodes have degree equal to one. A
078701, 2008
formula you’ll see everywhere links the probability of a node having 30
Jin Seop Kim, Kwang-Il Goh, Byung-
degree k to k to the power of a constant α. Mathematically speaking, nam Kahng, and Doochul Kim. Frac-
tality and self-similarity in scale-free
the scale free network master equation is: networks. New Journal of Physics, 9(6):177,
2007
p (k ) ∼ k −α .

In this formula, we call α the scaling factor. Its value is important,

because it determines many properties of the distribution. In general,
if a real world network has a power law degree distribution, α tends
to be low (α ∼ 2, and for a majority α < 3, although you can find
networks with higher αs). This is rather unfortunate, because it
means that the degree distribution has a well defined mean, but
not a well defined variance. This implies that the average degree
has meaning, but it’s not very useful to do anything more than a
superficial description of the network.
This is all well and good, but what does it mean exactly to have
α = 2 or α = 3? How do two networks with these two different coeffi-
 degree 103

1
α=2 Figure 6.17: The CCDF degree
α=3
distributions of two random
0.1 networks with different α expo-
p(k>=x) nents.
0.01

0.001
1 10 100
x

cients look like? I provide an example of their degree distributions in

Figure 6.17, and I show two very simple random networks with such
degree distributions in Figure 6.18 – obviously, systems this small
are a very rough approximation. From Figure 6.17 you see that α
determines the slope of the degree distribution, with a steeper slope
for α = 3. This means that the hubs in α = 3 are “smaller”, they do
not have a ridiculously high degree.

Figure 6.18: An example of two

networks with scale free degree
distributions, with different α
exponents.

(a) α = 2
(b) α = 3

Figure 6.18 confirms this: in Figure 6.18(a) you see that, for α = 2,
you have only one obvious hub that is head and shoulders above the
rest, practically connected to the entire network. In Figure 6.18(b),
instead, you still have a clear winner catching your eye (in the top),
but it is much closer to the second best hub.
The average degree is heavily influenced by the outliers with
thousands of connections. For instance, in Figure 6.16 the average
degree is equal to three, meaning that around 70% of nodes are
below average. This is well illustrated by the stadium example: you
have a stadium with 79, 999 individuals sampled at random from
the US population. If you calculate their average net worth you’ll
obtain a value – it’s difficult to be precise, but let’s say it’s around
 104 the atlas for the aspiring network scientist

$100, 000. So their total net worth is ∼ 8 billion dollars. However,

the 80, 000th person entering the stadium is our outlier hub: Jeff 31
Bear with me, I know it has probably
Bezos. His net worth alone is 192 billion dollars31 . The new average doubled by the time you read this
is 200 billion divided by 80 thousand people: 2.5 million dollars. The paragraph.
average shifted dramatically: 2.5 million is very different from 100
thousand. This is because net worth distributes broadly and thus
has a crazy variance, which causes tremendous shifts in the average.
This makes it incorrect to apply to this kind of distribution traditional
statistics that are based on variance and standard deviation – such as
regression analysis, as we’ll see in the next section.

100 10
0
1
-1
10 -1
10
0.1
10-2
p(k>=x)

p(k>=x)

p(k>=x)
-2
-3
10
10 0.01
-4 -3
10 10
0.001
0 1 2 3 4 0 1 2 3 4
10 10 10 10 10 10 10 10 10 10 1 10 100
x x x
0 0 0
10 10 10
-1
10
-1 10
-1
10
-2
10-2
10
p(k>=x)

p(k>=x)

p(k>=x)
10-3 10
-2

10-3 -4
10
-3
-4 -5 10
10 10
-6
10
-5 10 10
-4

0 1 2 3 4 5 0 1 2 3 4 5 6 0 1 2 3 4
10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10
x x x

Early works have found power law degree distributions in many Figure 6.19: A showcase of
networks, prompting the belief that scale free networks are ubiqui- broad degree distributions from
tous. In fact, this seems true. Figure 6.19 shows the CCDFs of many the same networks used in
networks: protein interactions, PGP, Slashdot, DBpedia concept the examples in the previous
network, Gowalla, Internet autonomous system routers. section.
But we need to be aware of our tendency of seeing patterns when
they aren’t there – after all, as Feynman says, the easiest person you
can fool is yourself. So in the next section I’ll give you an arsenal to
defend yourself from your own eyes and brain.

6.4 Testing Power Laws

Often, people will just assume that any degree distribution is a

power law, calling “power laws” things that are not even deceptively
looking like power laws. I’ve seen distributions as the one in Figure
6.20(a) passing as power laws and that’s just... no. However, I don’t
 degree 105

0
10 100
PowerLaw

10
-1 10-1 Lognormal
Figure 6.20: (a) An example
10-2
of a CCDF that is most defi-
p(k>=x)

p(k>=x)
10-2 10-3
10-4
nitely NOT a power law, but
10
-3
??? 10-5 that a researcher with a lack of
-4 10-6 proper training might be fooled
10
1 10 100 1000 100 101 102 103 104 105 106
x x
into thinking it is. (b) Fitting a
power law (blue) and a lognor-
(a) (b)
mal (green) on data (red) can
yield extremely similar results.
want to pass as one perpetuating the myth that “everything that
looks like a straight line in a log-log space is a power law”. That is
equally wrong, even if more subtle and harder to catch.
Seeing the plot in Figure 6.20(b), you might be tempted to perform
a linear fit in the log-log space. This more or less looks like fitting the
logged values with a log( p( x )) = α log( x ) + β. Transforming this back
into the real values, the slope α becomes the scaling factor, and β is
the intercept, in other words: log( p( x )) = α log( x ) + β is equivalent to
p( x ) = 10β x α – assuming you logged to the power of ten.
A small aside: if you were to do this on the distributions from Fig-
ure 6.17, you would expect to recover α ∼ 2 and α ∼ 3, because I told
you I generated the degree distributions with those exponents. In-
stead, you will obtain α ∼ 1 and α ∼ 2, respectively. That is because,
in Figure 6.17, I showed you the CCDF of the degree distribution, not
the distribution itself. The CCDF of a power law is also a power law, 32
Heiko Bauke. Parameter estimation for
but with a different exponent32 . If you’re doing the fit on the CCDF, power-law distributions by maximum
you have to remember to add one to your α to recover the actual likelihood methods. The European
Physical Journal B, 58(2):167–173, 2007
exponent of the degree distribution.
Back to parameter estimation. If you perform a simple linear
regression, you’ll get an unbelievably high R2 associated to a super-
significant p value. Well, of course: you’re fitting a straight line over a
straight-ish line. Does that mean you’re looking at a power law? Not
really.
Just because something looks like a straight line in a log-log plot,
it doesn’t mean it’s a power law. You need a proper statistical test to
confirm your hypothesis. The reason is that other data generating
processes, such as the ones behind a lognormal distribution, can
generate plots that are almost indistinguishable from a power law.
33
Aaron Clauset, Cosma Rohilla Shalizi,
Figure 6.20(b) shows an example. You cannot really tell which of the
and Mark EJ Newman. Power-law
two functions fits the data better. distributions in empirical data. SIAM
What you need to do is to fit both functions and then estimate review, 51(4):661–703, 2009
34
Jeff Alstott and Dietmar Plenz Bull-
the likelihood of each model to explain the observed data33 . This
more. powerlaw: a python package for
can be done with, for instance, the powerlaw package34 , 35 – available analysis of heavy-tailed distributions.
for Python. However, be prepared for the fact that having a signifi- PloS one, 9(1), 2014
35
[Link]
cant difference between the power law and the lognormal model is powerlaw
106 the atlas for the aspiring network scientist

extremely hard.
In most practical scenarios, you’ll have to argue that your network
is a power law. How could you do it? Well, in complex networks
power law degree distributions can arise by many processes, but one
in particular has been observed time and time again: cumulative
advantage. Cumulative advantage in networks says that the more
connections a node has, the more likely it is that the new nodes will
connect to it. For instance, if you write a terrific paper which gathers
lots of citations this year, next year it will likely gain more citations 36
Derek de Solla Price. A general theory
than the less successful papers36 . of bibliometric and other cumulative
This is the same mechanism behind – for instance – Pareto distri- advantage processes. Journal of the
American society for Information science,
butions and the 80/20 rule. Pareto says that 80% of the effects are
27(5):292–306, 1976
generated by 20% of the causes37 . For instance, 20% of people control 37
Vilfredo Pareto. Manuale di economia
80% of the wealth. And, given that it takes money to make money, politica con una introduzione alla scienza
sociale, volume 13. Società editrice
they are likely to hold – or even grow – their share, given their abil- libraria, 1919
ity to unlock better opportunities. In fact, the Pareto distribution is
a power law. Similar to this is Zipf’s Law, the observation that the
second most common word in the English language occurs half of the
time as the most common, the third most common a third of the time, 38
Jean-Baptiste Estoup. Gammes
etc38 , 39 , 40 . In practice, the nth word occurs 1/n as frequently as the sténographiques: méthode et exercices
first, or f (n) = n−1 , which is a power law with α = 1. pour l’acquisition de la vitesse. Institut
sténographique, 1916
This is opposed to the data generating process of a lognormal 39
Felix Auerbach. Das gesetz der
distribution. To generate a lognormal distribution you simply have to bevölkerungskonzentration. Petermanns
multiply many random and independent variables, each of which is Geographische Mitteilungen, 59:74–76,
1913
positive. A lognormal distribution arises if you multiply the results 40
GK Zipf. The psycho-biology of
of many ten-dice rolls. You can see that there is no cumulative advan- language. 1935
tage here: scoring a six on one die doesn’t make a six more likely on
any other die – nor influences subsequent rolls.
So, to sum up, to test for a power law you have to do a few things.
First, make sure that your observations cannot be explained with
an exponential. Confusion between a power law and some other
distribution such as an exponential is easy, and so you should start
by assuming that your distribution is not a power law. Second, try to
see if you can statistically prefer a power law model over a lognormal. 41
Whether this holds true also for
In the likely event of you not being able to mathematically do so, networks is the starting point of
a surprisingly hot debate, see for
you should look at your data generating process. If you have the instance [Broido and Clauset, 2019] and
suspicion that it could be due to random fluctuations, then you might [Voitalov et al., 2018].
have a lognormal. Otherwise, if you can make a convincing argument
42
Gudlaugur Jóhannesson, Gunnlaugur
Björnsson, and Einar H Gudmundsson.
of non-random cumulative advantage, go for it. Afterglow light curves and broken
There are a few more technicalities. Pure power laws in nature are power laws: a statistical study. The
Astrophysical Journal Letters, 640(1):L5,
– as I mentioned earlier – rare41 . Your data might be affected by two 2006
impurities. Your power law could be shifted42 , or it could have an 43
Aaron Clauset, Cosma Rohilla Shalizi,
exponential cutoff43 . In a shifted power law, the function holds only and Mark EJ Newman. Power-law
distributions in empirical data. SIAM
on the tail. In an exponential cutoff the power law holds only on the review, 51(4):661–703, 2009
 degree 107

head.
Shifted power laws have an initial regime where the power law
doesn’t hold. Formally, the power law function needs a slowly grow-
ing function on top that will be overwhelmed by the power law for
large values of k – as I show in Figure 6.21(a). So we modify our
master equation as: p(k) ∼ f (k)k−α , with f (k) being an arbitrary
but slowly growing. Slowly growing means that, for low values of
k it will overwhelm the k−α term, but for high values of k, the latter
would be almost unaffected. In power law fitting, this means to find
the k min value of k such that, if k < k min we don’t observe a power
law, but for k > k min we do.

0 0
10 10

10
-1
p(k) ~ f(k)k-α -1
Figure 6.21: (a) An example of
10
10
-2 shifted power law. The area in
p(k>=x)

p(k>=x)

10-2
10-3 which the power law doesn’t
10
-4 10-3 p(k) ~ k-αe-λk hold is shaded in blue. (b) An
10-5 example of truncated power
10-4
100 101 102 103 104 105 1 10 100 1000 law: a power law with an ex-
x x
ponential cutoff. The area in
(a) (b)
which the power law doesn’t
Shifted power laws practically mean that “Getting the first k min hold is shaded in green.
connections is easy”. If you go and sign up for Facebook, you gener-
ally already have a few people you know there. Thus we expect to
find fewer nodes with degree 1, 2, or 3 than a pure power law would
predict. The main takeaway is that, in a shifted power law, we find
fewer nodes with low degrees than we expect in a power law.
Truncated power laws are typical of systems that are not big
enough to show a true scale free behavior. There simply aren’t
enough nodes for the hubs to connect to, or there’s a cost to new
connections that gets prohibitive beyond a certain point. This is
practically a power law excluding its tail, that’s why we call them
“truncated”. Mathematically speaking, this is equivalent to having an
exponential cutoff added to our master equation: p(k) ∼ kα e−λk . The
exponential function is dominated by the power law function for low
values of k, but it becomes dominant for high values of k. See Figure
6.21(b) for an example.
Truncated power laws practically mean that “Getting the last
connections is hard”: the biggest superstar on Twitter has a lot of
followers, but relatively speaking they are not that many more as the
second biggest superstar on Twitter. Thus its degree is not as big as
we would expect. The main takeaway is that, in a truncated power
law, the hubs have lower degrees than we expect in a power law.
At the end of the day, it doesn’t matter too much if your network
has an exponential, lognormal or power law degree distribution. On
108 the atlas for the aspiring network scientist

one thing the brotherhood of network scientists can agree: the vast
majority of networks have broad degree distributions, spanning mul-
tiple orders of magnitude. Most nodes have below-average degree
and hubs lie many standard deviations above the average. Even if
they are not power laws at all, that’s still pretty darn interesting.

6.5 Summary

1. The degree is the number of edges connected to a node and it’s

probably a node’s most important and basic feature. It tells us how
well connected and how structurally important the node is.

2. In more complex graph models (directed, weighted, bipartite,

multilayer), the degree measure becomes itself more complex. For
instance, in directed networks you have both in- and out-degree,
depending on the direction of the edge.

3. The degree distribution is a plot telling you how many nodes

have a specific degree value in the network, and it is one of the
network’s most important properties.

4. When plotting degree distributions, the standard choice is the

complement of the cumulative distribution, shown in a log-log
scale.

5. Many networks have a power law degree distribution, but rarely

this is a pure power law: it is often shifted or truncated. Fitting
a power law and finding the correct exponent is tricky and you
should not do it using a linear regression: you should use special-
ized tools.

6. Moreover, determining whether the degree follows a power law is

useful for modeling and theory, but it isn’t crucial empirically. The
interesting thing is that networks have broad and unequal degree
distributions. You can describe them with statistics that are easier
to get right than the tricky business of fitting power laws.

6.6 Exercises

1. Write the in- and out-degree sequence for the graph in Figure
6.3(a). Are there isolated nodes? Why? Why not?

2. Calculate the degree of the nodes for both node types in the
bipartite adjacency matrix from Figure 6.5(a). Find the isolated
node(s).
 degree 109

3. Write the degree sequence of the graph in Figure 6.7. First consid-
ering all layers at once, then separately for each layer.

4. Plot the degree distribution of the network at [Link]

eu/exercises/6/4/[Link]. Start from a plain degree distribu-
tion, then in log-log scale, finally plot the complement of the
cumulative distribution.

5. Estimate the power law exponent of the CCDF degree distribution

from the previous exercise. First by a linear regression on the log-
log plane, then by using the powerlaw package. Do they agree? Is
this a shifted power law? If so, what’s k min ? (Hint: powerlaw can
calculate this for you)

6. Find a way to fit the truncated power law of the network at http:
//[Link]/exercises/6/6/[Link]. Hint: use the
[Link].curve_fit to fit an arbitrary function and use the
functional form I provide in the text.
7
Paths & Walks

So far, we adopted a static vision of a network. We have a structure

and we ask simple questions about the structure as it is. Does it
have directed edges? What’s the degree of the nodes? Those are
interesting questions, but a network really shines when you use it for
what it is for: exploring its connections.
To understand what I mean, let’s come back to the social network
example. Nodes are people and edges connect friends. Suppose
you want to send a message to a person you are not connected to.
Maybe you want to sell them a new shampoo. How do you do it?
Well, you could tell the message to a friend, and instruct them to pass
the message on. This means having a packet of information travel
through the network, exploiting its edges.
There are many ways to cross a network using its edges, depend-
ing on which restrictions you want to put on your exploration. I’m
going to define a few technical terms (following graph theory litera- 1
Reinhard Diestel. Graph theory. Springer
ture1 ) but, if you find it simpler, you can call all of them “paths” plus Publishing Company, Incorporated,
a qualifier specifying what type of path it is. I’m going to present 2018

both conventions and you simply use the one you find most natural –
as everybody does in network analysis.

Figure 7.1: (a) An example

of a walk of length six in the
network, following the green ar-
rows. (b) An example of a path
of length three in the network.
(a) (b)
The most basic way to explore a graph is by performing a walk,
or “path with repeating nodes”. A walk is a sequence of nodes with
the property that each node in the sequence is adjacent to the node
next to it. In a walk you’re not imposing any rule in your exploration.
You can go back and forth between the same two nodes by using the
 paths & walks 111

same edge as many times as you want. The length of a walk is the
number of times you’re using the edges in your walk. If you use the
same edge n times, this will increase the walk’s length by n. Figure
7.1(a) shows an example of a walk of length 6 in a network.
In a walk the choice of the next edge to explore is yours. You can
have a slightly more constrained definition of a walk, where you
put rules to choose the next edge to traverse. For instance, in the
random walk you impose to make this choice completely at random.
We already saw a way to calculate node exploration probabilities via
a random walk using powers of the adjacency matrix in Section 5.1.
We’ll see that random walks are a phenomenally powerful way to
explore your network’s properties and are at the basis of countless
methods: Chapter 8 will be but a superficial introduction.
When you impose even more constraints on your walks, then
you can generate a path, or “simple path” (Figure 7.1(b)). This is a
walk that does not repeat nodes nor edges. Again, you can put more
qualifiers on your path to make it special. For instance, recalling
the seven bridges problem, an Eulerian path is a path that travels
through all edges of a connected graph – since it is a path, not only it
has to visit each edge, but it also has to do it exactly once. A cousin
of the Eulerian path is the Hamiltonian path, which instead wants to
visit each node – not edge – exactly once. More interestingly, you can
try to find the shortest path between two nodes. That will be the topic
of Chapter 10.
Similarly to a walk, also a path has a length. This is again defined
as the number of edges the path crosses. Since no edge can be used
twice in a path, this is also the number of distinct edges used.

7.1 Walks and Matrices

In Chapter 5 I showed you that taking powers of the stochastic matrix

is fun, because it tells us the probability of a random walker going
from nodes u to v. But looking at ye olde regular adjacency matrix
can be insightful. If A is binary and its diagonal is set to zero, then
An can tell us lots of interesting things.

2
Figure 7.2: (a) A graph. (b-c)
3 Different powers of its binary
4
5 adjacency matrix A.
6

(a) G (b) A1 (c) A2

112 the atlas for the aspiring network scientist

First, if Anuv = 0, then there are no walks of length n going from u

to v. In Figure 7.2(b) you see that A21,7 is zero, because node 7 is only
connected to node 6. Node 6 isn’t connected to node 1 so there’s no
way to go from 1 to 7 in two hops. If, instead, Anuv > 0, then the Anuv
is exactly the number of such walks! There are two ways to go from
node 1 to node 3 in two hops in Figure 7.2(b): one via node 2 and one
via node 4, since they’re both connected to node 3.
It doesn’t end here: you can set the diagonal of A to 1 by summing
to it the identity matrix I. Then, ( I + A)n tells you the number of
walks of length n or less. The reason is that the diagonal represents
self loops. If it is set to 1, the walker in u can choose to follow the self
loop to u an arbitrary number of times before reaching v.
Every time you calculate An , the resulting diagonal is interesting.
Specifically, Anuu is the number of loops or closed walks of length n –
walks starting and ending in the same node – to which u participates.
For n = 2 we have a special case: this value is equal to the degree –
or A2uu = k u . Check the diagonal of Figure 7.2(b) if you don’t believe
me! This means that you could consider Anuu as a sort of generalized
degree.

7.2 Cycles

You can make a walk and a path in any graph, no matter its topology.
There is a special path that you cannot always do, though. That is the
cycle. Picking up the social network example as before, now you’re
not happy just by reaching somebody with your message. You want
the message you originally sent to come back to you. Also, you don’t
want anybody to hear it twice. If you manage to do so, then you have
found a cycle in your social network.
A cycle is a path that begins and ends with the same node. Note
that I said “path”, so we don’t have any repeated nodes nor edges –

Figure 7.3: (a) An example of a

cycle in the network, following
the green arrows. (b) A tree.
(a) (b)
 paths & walks 113

except the origin, of course. Figure 7.3(a) shows an example of a cycle

in the network. The cycle’s length is the number of edges you use in
your cycle. Given its topological constraints, that is also the number
of its nodes.
Imposing cycles to be paths make them a non trivial object to have
in your network. We can easily see why there might be nodes that
participate in no cycles. If a node has degree equal to one, you can
start a path from it, but you can never go back to complete a cycle.
Doing so would force you to re-use the only connection they have.
Thus a cycle is impossible for such nodes.
In fact, we can go further. We can imagine a network structure
that has no cycles at all! I draw one such structure in Figure 7.3(b).
No matter how hard you squint, you’re never going to be able to
draw a cycle there. We have a special name for such structures: trees.
Trees are simple graphs with no cycles. In a tree you cannot get your
message back, unless somebody hears it twice. Given their lack of
cycles, some even call them acyclic graphs.

Figure 7.4: An example of a

directed walk, where we cannot
explore an edge if we do not
respect its direction.
Obey the law!

Directed networks add some spice to these concepts. In a directed

social network, people are only willing to pass messages to their
friends. If the friendship is not reciprocated, the person will not
pass the message back. In practice, a walk – and a path, and a cycle
– cannot use edges pointing to the direction opposite of the one they
want to go. Figure 7.4 shows a naughty walk trying to do exactly
that.
In the undirected case, cycles create only two types of networks:
those who have them (cyclic networks) and those who don’t (acyclic
networks). Instead, in the directed case, we can create a larger zoo
of different directed structures. Figure 7.5 showcases them. Figure
7.5(a) is the basic case: we have a cycle connecting the four nodes at
the bottom, thus it is a directed cyclic graph. There’s no such cycle
present in Figure 7.5(b), thus we call it a directed acyclic graph.
There could be a cycle in Figure 7.5(b), if we were to ignore edge
directions. That is why we create a category for those directed graphs
that would be acyclic if we were to ignore the edge directions. These
are directed trees, and Figure 7.5(c) provides an example.
If you have even a mild case of self-diagnosed OCD, you’ll proba-
bly be as irritated as I am about Figure 7.5(c). There’s a natural flow
114 the atlas for the aspiring network scientist

Figure 7.5: (a) A directed cyclic

graph. (b) A directed acyclic
graph. (c) A directed tree. (d)
An arborescence.

(a) (b) (c) (d)

to that directed tree, except for that little pesky edge at the bottom,
going into the opposite direction. To restore sanity to the network
world, we decided to create a final definition for directed graphs:
arborescences. This is French for “tree”, and in fact the two terms
are often used interchangeably. But, technically speaking, an arbores-
cence is a directed tree in which all nodes have in-degree of one,
except the root. In an arborescence, the root is a special node: the
only one with in-degree of zero. An arborescence must have one and
only one root. Figure 7.5(d) fixes Figure 7.5(c) to be compliant to the
definition of arborescence, and it is a work of art. So satisfying.

7.3 Reciprocity

Directed networks allow for a special type of path. In an undirected

network without parallel edges, each path using two edges will
necessarily bring you to a third node. You simply cannot use the
same edge to go back to your origin. This is actually possible in a
directed network. That is because the edge bringing you from u to v
is not the same edge that brings you back to u from v – by definition
of what a directed edge is.
In the social network case, this is about replying messages, or
considering as a friend somebody who also consider you as their
friend. So these are cycles of length two, or containing two nodes and
two edges.
In a social network, it is interesting to know the probability that, if
I consider you my friend, you also consider me your friend – which
hopefully is 100%, but it rarely is so. This is an important quantity
in network analysis, and we give it a name. We call it reciprocity,
because it is all about reciprocating connections.
To calculate reciprocity we count the number of connected pairs of
the network: pairs of nodes with at least one edge between them. In
Figure 7.6, we have five connected pairs. Then we count the number
of connected pairs that have both possible edges between them: the
 paths & walks 115

Figure 7.6: An example of a

directed network with some
reciprocal edges.

ones reciprocating the connection. In Figure 7.6, we have two of them.

Reciprocity is simply the second count over the first one. So, for the
example in Figure 7.6, we conclude that reciprocity is 2/5, or that the
probability of a connection to be reciprocated is 40%. Sad.

7.4 Connected Components

Walks and paths can help you uncover some interesting properties
in your network. Let’s pick up our game of message-passing. In this
scenario, we might end up in a situation where there is no way for
a message to reach some of the people in the social network. The
people you can reach with your message do not know anybody who
can communicate to your intended targets. In this scenario, it is
natural to divide people into groups that can talk to each other. These
are the network’s “components”.

Figure 7.7: A network with two

connected components, each
with five nodes. No matter how
long you try, you can never
find a path starting from u and
Reachable ending in v.
u v

Unreachable

I can translate what I just said in terms of walks and paths. If

two nodes cannot be connected by a walk, then they are on different
connected components. Connected components are subgraphs whose
nodes can be reached from one another by following the edges of the
network. The network in Figure 7.7 has two connected components:
2
One nice thing about graph theorists is
that they are less bad than average sci-
the nodes reachable with green-like walks, and the ones reachable by entists in naming things. For instance, if
blue-like walks.2 you have a network made by different
connected components and each of
A network with multiple connected components is usually bad
those components is a tree, then you
news. The whole point of a network is to connect nodes together so can call that a “forest”. ’cause it’s made
that they are in the same shared structure. However, when you have of trees. You get it? Anyone?
116 the atlas for the aspiring network scientist

multiple connected components, you effectively have two – or more –

separate networks which cannot talk to each other. That’s a bummer.
As you might expect, real world networks tend to have multiple 3
Svante Janson, Donald E Knuth,
components. Reality always comes in the way of a good story. How- Tomasz Łuczak, and Boris Pittel. The
ever, there is a silver lining. The vast majority of real networks host birth of the giant component. Random
Structures & Algorithms, 4(3):233–358,
most of their nodes in a single connected component. In practice, 1993
networks have what we call a “giant connected component” (GCC). 4
Sergey N Dorogovtsev, José Fer-
nando F Mendes, and Alexander N
One of the components of the network is usually ridiculously larger
Samukhin. Giant strongly connected
than all the others3 , 4 , as is the case in Figure 7.8. component of directed networks. Physi-
cal Review E, 64(2):025101, 2001

Figure 7.8: A network with a

giant connected component.
The node color codes the com-
ponent containing the node.

I mentioned earlier in this section that there is a relationship

between some matrix operations and random walks. If you recall
Section 5.1, raising the stochastic adjacency matrix to the power of n
tells you the probability of reaching a node with a random walk. So
you might expect that there are also some matrix operations related
to connected components.
Indeed, there are. We are interested in the eigenvalues of the
stochastic adjacency matrix – a more in-depth explanation of why
this is the case will come in Section 8.1. In Section 5.2 I said that
the largest eigenvalue of the stochastic adjacency is equal to one.
However, I also mentioned that the second eigenvalue λ2 could also
be equal to one – and so could λ3 and so on.
It turns out that the number of eigenvalues equal to one is the
number of components in the graph. The reason is that you can

3 Figure 7.9: The stochastic adja-

λ1
2 cency matrix of a disconnected
4
6 1 graph looks like two differ-
9 ent adjacency matrices pasted
5 on the diagonal. Thus, they
λ2 8
both have a (different) leading
7
eigenvalue equal to one.
 paths & walks 117

consider the adjacency matrix as two adjacency matrices pasted into

the same. They are disjoint matrices and each has as a maximum
eigenvalue of one. I show an example in Figure 7.9.
If you can use the leading eigenvalues to count the number of
connected components (Figure 7.9), the leading eigenvectors tell
you to which component the nodes belong. If the network has two
components, the nodes belonging to one will have a non-zero value
in the eigenvector, while the nodes which do not belong to that
component will have a zero – see Figure 7.10. As you might have
already deduced, if there is only one component then the leading
eigenvector contains the same non-zero value. Similar properties
hold for the Laplacian. The smallest eigenvectors of L play the very
same role as did the largest eigenvector of A: they are vectors telling
us to which component the node belongs.

a 0
a 0

λ1
Figure 7.10: If your graph has
a 0 two components, the eigen-
a 0
vectors associated with the
a 0
a 0 largest two eigenvalues of the
0 b stochastic adjacency matrix will
0
0
b
b λ2 tell you to which component
the node belongs, by having a
v1 v2
non-zero value.

Strong & Weak Components

So far, we saw that a measure of a network’s usefulness is connected-
ness. If there is no way to follow the edges of the network from node
u to node v, then u has no way to influence – or communicate to – v.
However, we dealt only with the case of undirected graphs. What if
our edges are not symmetric, but have a direction?
In that case we have two different scenarios. In the first scenario,
which we call strong, we want to ensure the same ability that the
undirected network endowed us: u must be able to contact v, and
vice versa. Figure 7.11(a) shows an example of such a component.
No matter where we choose to start our path, we can always go
back. Therefore, any u can reach any v, and vice versa. Since this
strong requirement is satisfied, we call these “strongly connected
components” (SCC).
It is not a surprise to reveal that strongly connected components
contain cycles. By definition, if you can find a pair of nodes that
you cannot join with a cycle – meaning starting from u and passing
through v makes it impossible to go back to u – then those nodes are
118 the atlas for the aspiring network scientist

1 Figure 7.11: (a) A strongly

connected component. (b) A
2 3 network with multiple strongly
connected components, coded
4 7 by different node colors.

5 6 8 9

(a) (b)

not part of the same strongly connected component. SCCs are impor-
tant: if you are playing a message-passing game where messages can
only go in one direction, you can always hear back from the players
in the same strongly connected component as you.
Popular algorithms to find strongly connected components in 5
Robert Tarjan. Depth-first search and
a graph are Tarjan’s5 , Nuutila’s6 , and others that exploit parallel linear graph algorithms. SIAM journal on
computation7 . computing, 1(2):146–160, 1972
The definition of SCC leaves the door open for some confusion.
6
Esko Nuutila and Eljas Soisalon-
Soininen. On finding the strongly
Even by visually inspecting a network that appears to be connected connected components in a directed
in a single component, you will find multiple different SCCs – as in graph. Inf. Process. Lett., 49(1):9–14, 1994
Figure 7.11(b). In the figure, there is no path that respects the edge
7
Sungpack Hong, Nicole C Rodia, and
Kunle Olukotun. On fast parallel detec-
directions and leads from node 1 to node 7 and back. The best one tion of strongly connected components
could do is 1 → 2 → 3 → 7 → 8 → 6 → 5 → 4. (scc) in small-world graphs. In Proceed-
ings of the International Conference on
However, it feels like this network should have one component,
High Performance Computing, Networking,
because we can see that there are no cuts, no isolated vertices. If Storage and Analysis, pages 1–11, 2013
we were to ignore edge directions, Figure 7.11(b) would really look
like a connected component in an undirected network. This feel-
ing of uneasiness led network scientists to create the concept of
“weakly connected components” (WCC). WCCs are exactly what I
just wrote: take a directed network, ignore edge directions, and look
for connected components in this undirected version of it. Under this
definition, Figure 7.11(b) has only one weakly connected component.

In & Out Components

Not all weakly connected components are created equal. In large
networks, one can find any sort of weird things. Suppose you are
working in an office. The core of the office works on documents
together, by passing them to each other multiple times and giving
them the core’s stamp of approval. This is by definition a strongly
connected component.
But you’re not part of the core of the office, you are in a weakly
connected component. Your job is simply to receive a document,
stamp it, and pass it to the next desk. Since you are in a WCC, you
 paths & walks 119

know you’re never going to see the same document twice. That
would imply that there is a cycle, and thus that you are in a strongly
connected component with someone. However, what you see in the
document can be radically different. The document might arrive
to you with or without the core’s stamp of approval. These two
scenarios are quite different.
If you are in the first scenario, it means your WCC is positioned
“before” the core. Documents pass through it and they are put in the
core. The flow of information originates from you or from some other
member of the weakly connected component, and it is poured into
the core. This is the scenario in Figure 7.12(a): you are one of the four
leftmost nodes. In this paragraph I highlighted the word in because
we decided to call these special WCCs in-components.

Figure 7.12: (a) An in-

component (in blue), composed
by the four leftmost nodes. (b)
A out-component (in green),
composed by the four rightmost
(a) (b)
nodes.
If you are in the second scenario, it means your WCC is positioned
“after” the core. The core does its magic on the documents, and
then outputs them into your weakly connected component. The
flow of information originates from the core and it is poured out to
your WCC. This is the scenario in Figure 7.12(b): you are one of the
four rightmost nodes. In this paragraph I highlighted the word out
because we decided to call these special WCCs out-components.

7.5 Summary

1. A walk is a sequence of nodes you can visit by following edges in

the network. Its length is the number of edges you use. A path is a
walk in which you never visit the same node or edge twice.

2. Cycles are paths which start and end in the same node. Acyclic
graphs are graphs without cycles. An undirected acyclic graph is
called a tree – a graph with |V | nodes and |V | − 1 edges. Otherwise,
you can have directed acyclic graphs which are not trees.

3. A directed acyclic graph with |V | nodes and |V | − 1 edges is a

directed tree. If all nodes in a directed tree have in-degree of one,
except one node with in-degree zero, then that directed tree is also
an arborescence.

4. Reciprocal edges in directed networks are edges between two

nodes pointing at each other. They allow cycles of length two. The
120 the atlas for the aspiring network scientist

number or reciprocated connections over the number of connected

pairs is the reciprocity of the directed network.

5. A connected component is a set of nodes that can all reach each

other by following walks on the edges. Real world networks
usually have one giant connected component which contains the
vast majority of nodes in the network.

6. You can count the number of connected components in a graph by

counting the number of eigenvalues equal to one of its stochastic
adjacency matrix. The non-zero entries in the corresponding eigen-
vectors tell you which nodes are in which connected component.

7. In directed networks you can have strong components: compo-

nents of nodes that can reach each other respecting the direction of
the edges. You can also have weak components, which ignore the
edge direction.

7.6 Exercises

1. Write the code to perform a random walk of arbitrary length on

the network in [Link]
txt.

2. Find all cycles in the network in [Link]

exercises/7/2/[Link]. Note: the network is directed.

3. What is the average reciprocity in the network used in the previ-

ous question? How many nodes have a reciprocity of zero?

4. How many weakly and strongly connected component does the

network used in the previous question have? Compare their sizes,
in number of nodes, with the entire network. Which nodes are in
these two components?
8
Random Walks

8.1 Stationary Distribution

Remember the stochastic adjacency matrix from Section 5.3? Figure

8.1 provides a refresher. Here we have the stochastic adjacency matrix
of a graph, A, raised to different powers: A1 (Figure 8.1(a)), A2
(Figure 8.1(b)), and A3 (Figure 8.1(c)).

Figure 8.1: Different powers of

the stochastic adjacency matrix
of the graph in Figure 5.11(b).

(a) A1 (b) A2 (c) A3

There’s one curious thing if we look at the columns of the A1 and
A3 matrices. In the first case, the minimum is zero and the maxi-
mum can be up to 0.5. But in A3 the minimum is higher than zero,
and the maximum is just 0.3. It seems that the values are somehow
converging. So what happens if we take A30 or – gasp! – A∞ ?

Figure 8.2: The result when

raising the stochastic A to the
power of 30.

Figure 8.2 shows the result. We see now that the columns are
constant vectors. These numbers have a specific meaning. When we
calculate A∞ , what we’re doing is basically asking the probability
of being in a node after a random walk of infinite length. Since
122 the atlas for the aspiring network scientist

the length is infinite, it does not really matter from which node
you originally started. That’s why all rows of A∞ are the same –
remember that the row indicates the starting point while the column
indicates the ending point.
This row vector – you can pick any of them, since they’re all
the same – is so important that we give it a name. We call it the
“stationary distribution” – or π, for short. π tells us that, if you have
a path of infinite length, the probability of ending up on a destination
is only dependent on the destination’s location and not on your point
of origin. In practice, if you apply the transition probability (A) to the
stationary distribution (π), you still obtain the stationary distribution:
πA = π. Having a high value in the stationary distribution for
a node means that you are likely to visit it often with a random
walker – by the way, this is almost exactly what PageRank estimates,
plus/minus some bells and whistles, see Section 11.4.
Note that it is not necessary to calculate A∞ to know the stationary
distribution. At least for undirected networks, π is quite literally the
normalized degree of the nodes: the degree divided by the sum of all
degrees (2| E|).
But... wait! This stationary distribution formula is oddly familiar:
πA = π. Haven’t we seen something similar to it? This kind of
looks like our eigenvector specification (Av = λv, see Section 5.2),
with a few odd parts. First, where’s the eigenvalue? Well, we can
always multiply a vector to 1 and we won’t change anything in the
equation. So: π A = π1. This is cool, because we already know that
1 is the largest eigenvalue (λ1 ) of a stochastic matrix. Second, the
vector π is on the left, not on the right. Putting these things together:
the stationary distribution π is the vector associated with the largest
eigenvalue, if multiplied on the left of A. Therefore: π is the leading
left eigenvector.
If you’re dealing with an undirected graph, there is a relationship
between right and left eigenvectors. If you were to transpose the
stochastic adjacency matrix, that is making it column-normalized
instead of row-normalized, the left and right eigenvectors would
swap. In different words: the left eigenvectors of A are exactly the
same as the right eigenvectors of A T . Thus the vector of constant
and π are the right and left leading eigenvectors of A, and they swap
roles in A T .
What do you do if your graph is not connected? No matter how
many powers of A you take, how infinitely long your walks are,
some destinations are unreachable from some origins. We end up
with two stationary distributions, one for one component, and one
for the other. Figure 8.3 shows an example. These two stationary
distributions are not directly comparable one with the other. They are
 random walks 123

3 Figure 8.3: The result when

2
calculating the stationary dis-
4
6 1 tribution for an unconnected
9 graph.
5

8 7

effectively telling you something about two different networks: one

made by the first component, and the other composed by the second
one.
This makes it clear why the eigenvector contains zeros for the
entries corresponding to the nodes that are not part of the connected
component we’re looking at. The eigenvector contains the probability
of ending up in a node after a random walk of infinite length. The
probability of ending up in those nodes via a random walk – no
matter how long – is zero, because there is no edge that you can use.

8.2 Non-Backtracking Random Walks

This is a good place to mention that there is an almost infinite num-

ber of different matrix representations you can build for a graph.
Each of those representations are useful to describe some sort of
process on the graph. Since we’re in the random walk section, I
will mention another useful matrix representing random walks: the
non-backtracking matrix. However, be aware that this is only one 1
Prabhaker Mateti and Narsingh Deo.
arbitrary choice about the many possible, you can have: cycle ma- On algorithms for enumerating all
trices1 , 2 , cut-set matrices3 , path and distance matrices, modularity circuits of a graph. SIAM Journal on
Computing, 5(1):90–99, 1976
matrices4 , to name a few. 2
Frank Harary. Graphs and matrices.
So what’s a non-backtracking matrix? As the name suggests, SIAM Review, 9(1):83–90, 1967
it’s a matrix describing non-backtracking walks. A walk is non- 3
Jørn Vatn. Finding minimal cut sets
in a fault tree. Reliability Engineering &
backtracking when we forbid the walker to re-use the same edge System Safety, 36(1):59–62, 1992
twice in a row in its walk. Figure 8.4 shows an example. 4
Mark EJ Newman. Modularity and
If we want to represent such a process with a matrix, we need community structure in networks.
Proceedings of the national academy of
to build it quite differently from an adjacency matrix. Rather than
sciences, 103(23):8577–8582, 2006b

Figure 8.4: A non-backtracking

random walk. The green arrows
No backtracking!
show the state transitions.
124 the atlas for the aspiring network scientist

having a row and a column per node, we instead have a node and 5
Ki-ichiro Hashimoto. Zeta functions
column per edge5 . Each cell contains a one if we can use the edge for of finite graphs and representations of
our non-backtracking walk, zero otherwise. Formally: p-adic groups. In Automorphic forms
and geometry of arithmetic varieties, pages
 211–280. Elsevier, 1989
1 if u 6= z
NBuv,vz =
0 otherwise.

So you see what’s going on here: we can only transition to node z

from v only if we got into v via u and u is not the same node as z. If
you’re a graphical thinker, Figure 8.5 might help you.

Figure 8.5: A non-backtracking

matrix.
0 1 1 1 0
1 0 1 0 1
1 1 0 1 1
1 0 1 0 1
0 1 1 1 0

As the figure shows, the non backtracking matrix has zero on

the diagonal, because the diagonal contains all edges to themselves.
Thus, a one in the diagonal is exactly a backtracking move: you
used the u, v edge to get to v and then you use it again to get to u.
Naughty backtracking walker!
On the other hand, if we go to the blue node from the red node,
the only legal move is to go to the purple node. If we did the oppo-
site move, from blue to red, we could reach either the green or the
purple node. Using these rules, you can figure out each entry in the 6
Travis Martin, Xiao Zhang, and
matrix. Of course, you can have a directed non-backtracking matrix, Mark EJ Newman. Localization and
centrality in networks. Physical review E,
in which you need to respect the direction of the edge as well. 90(5):052808, 2014
Non-backtracking matrices are useful for a bunch of applications: 7
Florent Krzakala, Cristopher Moore,
fixing eigenvector centrality degeneration6 (Section 11.4); helping Elchanan Mossel, Joe Neeman, Allan
Sly, Lenka Zdeborová, and Pan Zhang.
with community detection7 (Part IX); describing percolation pro- Spectral redemption in clustering sparse
cesses8 (Chapter 19); and even helping with counting motifs in networks. Proceedings of the National
Academy of Sciences, 110(52):20935–20940,
graphs9 (Chapter 39). 2013
8
Brian Karrer, Mark EJ Newman, and
Lenka Zdeborová. Percolation on sparse
8.3 Hitting Time networks. Physical review letters, 113(20):
208702, 2014
The stationary distribution allows you to calculate the probability 9
Leo Torres, Pablo Suárez-Serrato, and
of visiting a node given an infinite length random walk. Another Tina Eliassi-Rad. Non-backtracking
cycles: length spectrum theory and
important thing you might be interested in discovering is how long graph mining applications. Applied
you have to wait before a node gets visited by a random walker. Of Network Science, 4(1):41, 2019
 random walks 125

course you have an intuition: if it is very likely to visit the node –

high value in the stationary distribution – then probably it won’t take
long before we “hit” that node. But the two quantities, probability
and average hitting time, are not the same. Especially since the
hitting time of node v depends from the starting point u.
We use Hu,v to indicate the expected hitting time of v for a random
walk starting in u. How can we calculate Hu,v ? Well, if we want to
reach v from u, first we have to go to a neighbor of u. Let’s call it z.
Once we are in z, it will take us Hz,v steps to reach v, by definition. 10
László Lovász et al. Random walks on
The probability of ending up in z from u is one over u’s degree (k u ) graphs: A survey. Combinatorics, Paul
erdos is eighty, 2(1):1–46, 1993
since we picked it at random. Thus the formula expands to: 11
So, here we go. The main formula
can be rewritten in matrix form: H =
1
Hu,v = 1 +
ku ∑ Hz,v . J + AH − F, with J being the matrix of
ones, and A the stochastic adjacency.
z∈ Nu
What’s that F, though? It’s a diagonal
Applying this formula naively wouldn’t lead you very far, as matrix we have to remove from H
because, by definition, the diagonal
you’ll find yourself needing to calculate Hu,v in order to find out of H must be zero: the hitting time of
Hu,v ’s value. There is a way to find Hu,v using – what else? – the the origin from the origin is zero, it
is not the time it takes for a random
eigenvectors and eigenvalues of the adjacency matrix10 . The exact walker to go somewhere and coming
mathematical derivation is not for the faint of heart and can be ap- back, which is what you’d get if you
preciated by the brave readers who will dare to look at this obscene didn’t take F out. So did we just make
it worse by adding something more
footnote11 , and ends with the following formula: we don’t know? Actually, we can
derive F. Let’s rewrite the equation as
|V |
! F = J + AH − H. Let’s multiply the
1 w2n,v wn,u wn,v
Hu,v = 2| E | ∑ − √ , stationary distribution to both sides:
n =2
1 − λn kv ku kv Fπ = Jπ + H ( A − I )π (I grouped
the H terms). Note that Aπ = π by
where |V | and | E| are the number of nodes and edges. λ and w definition of a stationary distribution
and that Iπ = π by definition of an
are eigenvalues and eigenvectors of a special decomposition of A, identity matrix. So the whole H ( A − I )π
namely N = ∆1/2 A∆1/2 . ∆ = D −1 is the diagonal matrix with the term disappears, leaving Fπ = Jπ.
Again by definition of π, a matrix of
inverse of the degree on the diagonal – much like D is the diagonal
ones times π equals the scalar one,
matrix with the degree on the diagonal, we met it when calculating giving us Fπ = 1. So F is a diagonal
the graph Laplacian. λn is the nth eigenvalue – sorted in descending matrix with 1/πu = 2| E|/k u on
its uth diagonal entry. We use D to
order –; wn,u is the uth entry of the nth eigenvector. Finally, k u is the specify the diagonal matrix with the
degree of node u. degree on the diagonal, giving us the
equation H ( I − A) = J − 2| E| D −1 .
The formula looks threatening, but it ought not to be. Let’s look
So we can derive H easily now, right?
at the stupidest example possible, in Figure 8.6. Here we have a Ahahah. Wrong. ( I − A) is singular,
simple chain graph. Its degrees k are (1, 2, 1), as the second node has thus non-invertible: you can’t calculate
( I − A)−1 . So we need to multiply the
two neighbors and the other nodes have only one. The eigenvalues left and the right side by something
of N are (1, 0, −1) – we know the largest must be one because N is that will make ( I − A) disappear.
That something is the special matrix
stochastic. w shows the corresponding eigenvectors. Note how, in
p √ Z = ( I − A + A∞ )−1 . This gives us
w1 , w1,u = (1/ 2| E|) k u , which you can derive following what you H = ( I − A + A ∞ ) −1 ( J − 2 | E | D −1 ).
know about the stationary distribution of A and the way we derived To cut a long story short, we can
decompose A using its eigenvectors,
N from A. obtaining the formula in the main
If you want to know H1,3 , you need to do two things. First, for text. It took me one day to write this
1
 2
0.7 0.7 × −0.7
 footnote. I’m not paid nearly enough
n = 2, = 1 and − √ = 1. Then, for n = 3, for this.
1 − λ2 1 1
126 the atlas for the aspiring network scientist

 2
0.5 × 0.5

1 0.5
= 1/2 and − √ = 0. So, for n = 2 the part on
1 − λ3 1 1
the right side of the sum evaluates to 1 and for n = 3 it evaluates to
zero. Thus the total sum is one. Multiplied to 2| E| you obtain four.
This is super intuitive. How long does it take to get from node 1 to
node 2? Well, node 1 has only one connection and it goes to node 2,
so it will always take one step. But to get from node 2 to node 1, you
only have a 50% chance of doing it in one step. The other 50% of the
times the random walker will go to node 3. It will always come back
after another step, and then we’ll have another 50% chance to go to
node 1. You sum that to infinity, and you get an expected hitting time
of three.

λ1 λ2 λ3
1 k1 = 1 H Figure 8.6: The elements
needed to calculate the hit-
ting time of a graph. From left
to right: the graph and its de-
2
gree vector, the eigenvalues and
k2 = 2
eigenvectors of N, the resulting
hitting time matrix H.

3 k3 = 1
w1 w2 w3

From the formula and the example it is easy to see that H is

asymmetric, given that one of its parts is dependent on the degree
of the destination k v and not on k u , the degree of the origin. For this
reason, another object of interest is the commute matrix C: the time
it takes to go from u to v and back to u. This is C, and it is simply
defined as: Cu,v = Hu,v + Hv,u , trivially symmetric.
There’s one fun connection with the stationary distribution here.
We call this the “Random Target Lemma”. Suppose you start from
node u and you pick destinations v at random. What’s the expected
hitting time? This is basically averaging Hu,v over all possible destina-
tions v. If you do that, you’ll find out that the result is:

|V |
1
∑ πv Hu,v = ∑ 1 − λn
.
v ∈V n =2
Noticing something weird? The right hand side has no trace of u.
This means that the average time to hit something doesn’t depend on
your starting point u, exactly like, in the stationary distribution, the
probability of ending your random walk somewhere didn’t depend
on from where you started.
 random walks 127

Note that this is a very short and incomplete treatment of the

subject, just to let you know how to calculate hitting and commute
times. You really should check out Lovász’s paper (footnote 10) for a
full treatment of the subject.

8.4 Mincut Problem

I already said in Section 7.4 that smallest eigenvector of L isn’t so

special after all. It plays the very same role as the largest eigenvector
of A: it is a vector of constant, telling us to which component the
node belongs. The reason why L is interesting lies with the second
smallest eigenvector (or the eigenvector associated to the smallest
non-zero eigenvalue, if the graph has multiple components).

.37 2 1

+
.37 Figure 8.7: (a) The Laplacian
.28 matrix of a graph. We show the
3 4
.28 2-cut solution for this graph
.12 5 with the red and blue blocks.
-.23 (b) The second smallest eigen-
- -.38
-.38
7
6

8
vector of (a). (c) The graph view
of (a) – we color the nodes ac-
-.43 cording to the value attached to
v2 9 them in the (b) vector.
(a) (b) (c)
One classical problem in graph theory is to find the minimum
(or normalized) cut of a graph: how to divide nodes in two disjoint
groups such that the number of edges running across groups is
minimized. Turns out that the second smallest eigenvector of the 12
Miroslav Fiedler. Laplacian of graphs
Laplacian is a very good approximation to solve this problem12 . and algebraic connectivity. Banach Center
Publications, 25(1):57–70, 1989
How? Consider Figure 8.7. In Figure 8.7(a) I show the Laplacian
matrix of a graph. I arranged the rows and columns of the matrix
so that the 2-cut solution is evident: by dividing the matrix in two
diagonal blocks there is only one edge outside our block structure
that needs to be cut.
Now, why did I label the two blocks as “+” and “-”? The reason
lies in the magical second smallest eigenvector of the Laplacian – also
known as the Fiedler vector –, which is in Figure 8.7(b). We can see
that the top entries are all positive (in red) and the bottom are all
negative (in blue). This is where L shines: by looking at the sign of
the value of a node in its second smallest eigenvector we know in
which group the node has to be to solve the 2-cut problem!
Not only that, but the values in Figure 8.7(b) are clearly in de-
scending order. If we look at the graph itself – in Figure 8.7(c) – and
128 the atlas for the aspiring network scientist

we use these values as node colors, we discover that there is much

more information than that in the eigenvector. The absolute value
tells us how embedded the node is in the group, or how far from the
cut it is. Node 5 is right next to it, while node 9 is the farthest away.
Now – you’re thinking – you’re itching to tell me you can use this
eigenvector to solve all the k-cut problems, for any k larger than two. But
you can’t, because the Fiedler vector is just a simple monodimensional vec-
tor. Or can I? True: the second smallest eigenvector cannot solve, by
itself, the arbitrary k-cut problem, finding the minimum cuts to divide
the graph in k parts. That’s why we have all the other eigenvectors of
the Laplacian.

4
6 0.5
16 12
2 15
0.4 11
10

18 0.3 9
3rd Eigenvector
8
5
0.2
1 13
3
7
17 0.1
14 0
7 13 1
9
-0.1 17
1514 23
18 16 45
8 -0.2 6

11
-0.3
-0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4
10
12 2nd Eigenvector
(a) (b)

Figure 8.8: (a) A graph in which

Solving the 3-cut problem involves looking at the eigenvectors in
the node colors represent the
a two dimensional space. I show an example of this in Figure 8.8.
best solution to the 3-cut prob-
Figure 8.8(b) is a 2D representation of the second and third smallest
lem. (b) The eigenspace of the
eigenvectors of the Laplacian of the graph in Figure 8.8(a). We can
Laplacian of the (a) graph. We
see that there is a clear pattern: each node takes a position in this
plot the second smallest eigen-
space on a different axis, depending on the block to which it belongs.
vector in the x-axis, and the
Farther nodes on the axis are more embedded in the block, while
third smallest eigenvector on
nodes closer to the cuts are nearby the origin (0, 0). You can imagine
the y-axis. Data point color cor-
that we could solve the 4-cut problem looking at a 3D space, and the
responds to the node color in
k-cut problem looking at a (k − 1)D space. I can’t show it right now
(a). Each data point is labeled
because, although it’s truly remarkable, the margin of my Möbius
with its corresponding node ID.
paper is too small to contain it.
Of course, at the practical level, real world networks are not
amenable to these simple solutions. Most of the times, the best
way to solve the 2-cut problem is to put in one group a node with
degree equal to one and put all other nodes of the network in the
 random walks 129

other group. If you want to find non-trivial k-cuts of the network

that are meaningful for humans... well... you have to do community
discovery (and jump to Part IX).

8.5 Random Walks and Consensus

One thing that might be left in your head after reading the previous
section is: why? Why do the eigenvectors of the Laplacian help with
the mincut problem? What’s the mechanism? To sketch an answer for
this question we need to look at what we call “consensus dynamics”.
This is a subclass of studying the diffusion of something (a disease,
word-of-mouth, etc) on a network – which we’ll see more in depth 13
Michael T Schaub, Jean-Charles
in Part V. This section is sketched from a paper13 that you should Delvenne, Renaud Lambiotte, and
Mauricio Barahona. Structured networks
read to have a more in-depth explanation of the dynamics at hand. and coarse-grained descriptions: A
Consensus dynamics were originally modeled this way by DeGroot14 . dynamical perspective. Advances in
Network Clustering and Blockmodeling,
In this section I’m going to use the stochastic adjacency matrix
pages 333–361, 2019
of the graph, but what I’m saying also holds for the Laplacian. The 14
Morris H DeGroot. Reaching a
difference between the two – as I also mention in Section 30.2 – is consensus. Journal of the American
Statistical Association, 69(345):118–121,
that the stochastic adjacency matrix describes the discrete diffusion
1974
over a network. In other words, you have a clock ticking and nothing
happens between one tick of the clock and the other. The Laplacian,
instead, describes continuous diffusion: time flows without ticks in
a clock, and you can always ask yourself what happens between two
observations. Besides this difference, the two approaches could be
considered equivalent for the level of the explanation in this section.
How does the stochastic adjacency help us in studying consensus
over a network? Let’s suppose that each node starts with an opinion,
which is simply a number between 0 and 1. We can describe the
status of a network with a vector x of |V | entries, each corresponding
to the opinion of each node. One valid operation we could do is
multiplying x with A, the stochastic adjacency matrix, since A is a
|V | × |V | matrix.
What does this operation mean? Mathematically, from Section 5.2,
|V |
the result is a vector x 0 of length |V | defined as xv0 = ∑ xu Auv . In
u =1
practice, the formula tells you that node v is updating its opinion
by averaging the opinion of its neighbors. Non-neighbors do not
contribute anything because Auv = 0, and this is an average because
we know that the rows of the adjacency matrix sum to 1 – thus each
xu is weighted equally and xv0 will still be between 0 and 1.
We can reach a consensus by multiplying x with A an infinite
number of times. If you do that, x will converge to a vector of con-
stant – which is the average of the initial x, in this case 0.5 since
values were extracted uniformly at random between 0 and 1. Figure
130 the atlas for the aspiring network scientist

1
Figure 8.9: The value of x
0.8 (y-axis) for each node in the
0.6 graph from Figure 8.8 over time
v(T)

0.4 (x-axis). At each time step, I up-

date x’s values by multiplying
0.2
them to A. The line color tells
0 you to which community the
1 10 100
Time
node belongs. The black vertical
line highlights step 6.

8.9 shows an example process on the network from Figure 8.8. Each
node starts with a uniform random value and the line tracks this
value over time.
Now the connection with eigenvectors: from Section 8.1 you
remember that the vector of constant is the leading eigenvector of A.
This operation showed you why: if a network has separate connected
components, it cannot reach a unique consensus; every connected
component will reach its own consensus independently because
there’s no exchange of information across components.
The second eigenvector, instead, tells you how quickly the nodes
will reach the consensus. In the figure, the line color tells you the
community of the node. You might notice that nodes bundle up
with their community mates before reaching the network’s final
consensus. This is described by the inverse of the second eigenvalue
of the Laplacian. For that network, it is 1/λ2 ∼ 5.13. This tells you
that the nodes are expected to converge to their community’s opinion
between step 5 and 6, which is the step highlighted in Figure 8.9. You
might notice that one blue and one red node don’t seem to converge
to their community, but that’s because they are nodes 1 and 13 and,
as you can see from Figure 8.8, they are in between communities, i.e.
they are close to the cut.
So the reason why the Fiedler vector allows you to solve the
mincut is because it tells you how much it will take for the node
to converge to the consensus. Nodes farther from the cut will take
longer time. Moreover, you can use the sign to know on which side
of the cut you are, because the nodes will first tend to converge to the
value of their own community, which is the opposite of the value in
the other community.

8.6 Summary

1. Raising the stochastic adjacency matrix to high powers will make

it converge to the stationary distribution, which is the probability
 random walks 131

of ending in a node in the network after an infinite length random

walk. This is also the leading left eigenvector, or the normalized
degree.

2. A non-backtracking matrix is a matrix describing a non-backtracking

random walk, with a row/column per edge telling you whether
you can move from one edge to another. It has zero on its main
diagonal, because the main diagonal contains all backtracking
moves.

3. The hitting time is the number of expected steps you need to take
in a random walk to reach one node starting from another. It is
related to a special eigenvector decomposition of the adjacency
matrix.

4. The normalized cut problem aims to find the way to partition

the network in n balanced parts such that we “cut” the minimum
number of edges (the ones flowing from one group to another).
You can approximate the solution by looking at the n − 1 smallest
eigenvectors of the Laplacian (skipping the first).

5. This is because the eigenvectors of the Laplacian tell you when

a node will reach a consensus, and to which intermediate value
it will converge before doing so. Nodes on the same side of a cut
will converge to the same intermediate value.

8.7 Exercises

1. Calculate the stationary distribution of the network at http://

[Link]/exercises/8/1/[Link] in three ways: by
raising the stochastic adjacency to a high power, by looking at the
leading left eigenvector, and by normalizing the degree. Verify that
they are all equivalent.

2. Calculate the non-backtracking matrix of the network used for the

previous question. (The network is undirected)

3. Calculate the hitting time matrix of the network at [Link]

[Link]/exercises/8/3/[Link].

4. Draw the spectral plot of the network at [Link]

eu/exercises/8/4/[Link], showing the relationship between
the second and third eigenvectors of its Laplacian. Can you find
clusters?
9
Density

9.1 Density & Real Networks

In Chapter 6, we saw that there is a quick way to get a sense of how

well connected the nodes of your network are. You can calculate
their average degree, that is to say the average number of edges
each node uses to connect to its neighbors. However, in the same
chapter, we also saw that the degree distribution is usually extremely
broad, which makes the average degree an incomplete information.
Moreover, depending on the size of your network, the same value of
average degree could mean different things. A network with three
nodes and average degree equal to two is fully connected. A network
with the same average degree, but ten thousand nodes is quite sparse.
See Figure 9.1 for a graphical example.

Figure 9.1: (a) A network with

three nodes and average degree
equal to two. (b) A network
with around 650 nodes and
average degree equal to two.

(a) (b)
To quantify the difference between the two cases, network scien-
tists defined the concept of network density. Informally, this is the
probability that a random node pair is connected. Or, the number of
edges in a network over the total possible number of edges that can
exist given the number of nodes. We can estimate this latter quantity
 density 133

quite easily – from now on, I’ll just assume the network is undirected,
unipartite and monolayer, for simplicity.
Here’s the problem, rephrased as a question: how many edges
do we need to connect |V | nodes? Well, let’s start by connecting one
node, v, to every other node. We will need |V | − 1 edges – we’re
banning self loops. Now we take a second node, u. We need to
connect it to the other nodes in V minus itself – seriously, no self
loops! – and v, because we already added the (u, v) edge at the
previous step. So we add |V | − 2 edges. If you go on and perform
the sum for all nodes in v, you’ll obtain that the number of possible
edges connecting |V | nodes is |V |(|V | − 1)/2. In other words, you
need |V | − 1 edges to connect a node in V with all the other nodes,
and you divide by two because each edge connects two nodes. In
fact, the number of possible edges in a directed network is simply
|V |(|V | − 1), because you need an edge for each direction, as (u, v) 6=
(v, u).
Now we can tell the difference between the networks in Figures
9.1(a) and 9.1(b). The first one has three nodes. We would expect
3 ∗ 2/2 = 3 edges, and that’s exactly what we have. Its density is
the maximum possible: 100%. The network on the right, instead,
contains 650 nodes. Since the average degree of the network is also
two, we know it also contains 650 edges. This is a far cry from the
650 ∗ 649/2 = 210, 925 we’d require. Its density is just 0.31%, more
than three hundred times lower than the example on the left!
With all this talk about real world networks having low degree, we
should expect them to be quite sparse. They are, in fact, even sparser
than you think. A few examples.
The network connecting the routers forming the backbone of the
Internet? It contains |V | = 192, 244 nodes. So the possible number of
edges is |V |(|V | − 1)/2 = 18, 478, 781, 646. How many does it have,
really? 609, 066, which is just 0.003% of the maximum. How about
the power grid? The classical studied structure has 4, 941 nodes,
which means – potentially – 12, 204, 270 edges. Yet, it only contains
6, 594 of them, or 0.05%. Well, compared to the Internet that’s quite
the density! Final example: scientific paper citations. A dataset from
arXiV contains 449, 673 papers. The theoretical maximum number
of citations is 101, 102, 678, 628. Physicists, however, are quite stingy:
they only made 4, 689, 479 citations, or 0.004% of the theoretical
maximum. (All figures are from classical network structures studied 1
Albert-László Barabási et al. Network
in the literature1 ) science. Cambridge university press, 2016
You might have spotted a pattern there. The density of a network
seems to go down as you increase the number of nodes. While not
an ironclad rule, you might be onto something. The problem is that
the denominator of the density formula grows quadratically. Each v
134 the atlas for the aspiring network scientist

1400
Potential Edges
1200 Figure 9.2: The red line shows
Actual Edges
1000
800
the number of possible edges
|E|

600 (|V |(|V | − 1)/2) in a network

400
with |V | nodes (x axis). The
200
0
blue line shows the number
5 10 15 20 25 30 35 40 45 50
|V|
of actual edges, assuming the
average degree being k̄ = 3.

you add to V allows |V |(|V | − 1) to grow pretty rapidly. Figure 9.2

shows that as a red line. The numerator, instead, grows practically
linearly – the blue line in Figure 9.2. Each added node will bring only
few edges – three in Figure 9.2 –, as we know that the average degree
of real world networks is low.

9.2 Clustering Coefficient

Density doesn’t solve all ambiguities you had in the case of the
average degree. Two networks can have the same density and the
same number of nodes, but end up looking quite different from each
other. That is why the ever industrious network scientists created
yet another measure to distinguish between different cases: the
clustering coefficient.

Figure 9.3: Both networks

have 13 nodes and 32 edges.
However, their topologies are
different: in the example to
the left the edges are more
(a) (b) “clustered” together.
Consider Figure 9.3: it contains two networks with the same
number of nodes and the same number of edges – thus the same
density. However, they look quite different. There’s more a sense of
“order” in Figure 9.3(a). That is because, in Figure 9.3(a), the edges
are more “clustered” together. That is what the clustering coefficient
aims at estimating quantitatively.
I can sum up the intuition behind the clustering coefficient as
the old adage “The friend of my friend is my friend”. If you have
two friends it’s overwhelmingly likely that they know each other,
because they have something in common: you. You might invite
them – even by accident – to the same event. This sort of dynamics
in network is called “triadic closure”. A set of three connected nodes
is a triad – Figure 9.4(a). If one member of the triad is connected to
the other two, more often than not the triad will “close”, meaning
 density 135

that the other two nodes will connect to each other to form a triangle.
“Triangle” is the name we give to a specific network pattern: three
nodes that are all connected to each other, and I show one in Figure
9.4(b).

Figure 9.4: The two possible

connected network patterns
involving three nodes.

(a) Triad (b) Triangle

A note for computer scientists: do not confuse the clustering

coefficient with the operation you know as “clustering”: the process
of grouping similar rows of a matrix. The latter is a process that in
network science is called community detection – or discovery – and
will be the subject of Part IX. The clustering coefficient is simply a
number you return that describes quantitatively how “clustered”
a network looks. A way to dispel the confusion is to use the term
“transitivity”. This takes inspiration from the transitive property: if u
is connected to v and v is connected to z, then u is connected to z too.
To sum up, the clustering coefficient answers the question: how
often does a triad closes down into a triangle? What’s the likelihood
that the “friend of my friend is my friend” rule holds in the network?
To answer this question we have to count the number of triads and
the number of triangles in the network. Then the global clustering
2
Paul W Holland and Samuel Lein-
hardt. Transitivity in structural models
coefficient (CC) is simply2 : CC = 3 × #Triangles/#Triads. Why do we of small groups. Comparative group
have to multiply the number of triangles by three? studies, 2(2):107–124, 1971
Consider Figure 9.5(a). I highlighted a triangle in there. From the
perspective of the node highlighted in blue. The triad is closed by the
edge highlighted in blue. The same holds for the nodes highlighted
in green and purple. That single triangle is closing three triads, that
is the reason why we multiply the number of triangles in the network
by three.
Counting triads is a bit more confusing, but in the end it’s going to
be easy to remember, because it connects with something you already
know. I provide a representation of the process in Figure 9.5(b). In
there, I count the number of triads centered on node v, meaning
that we only count the triads that have v connected to both of its
members. This makes it easier, because we only have to look at v’s
neighbors. I start by selecting its first neighbor, with the blue arrow.
How many triads do v and the blue neighbor generate? Well, one
for each of the remaining neighbors of v, so I add a blue dot to each
136 the atlas for the aspiring network scientist

Figure 9.5: (a) Counting the

number of triads closed by a
triangle. From each node’s
perspective, a different triad is
v closed by the same triangle. (b)
Counting the number of triads
in the network.

(a) (b)

of the neighbors. When I move to the neighbor highlighted by the

green arrow, I perform the same operation adding the green dot. I
don’t have to add one to the neighbor with the blue arrow, because I
already counted that triad.
Sounds familiar? That is because this is the very same process
you apply when you have to count the number of possible edges of
a graph. The number of triads centered on node v is nothing more
that the number of possible edges among k v nodes, with k v being v’s
degree. So, if we want to know the number of triads in a graph, we
simply need to add k v (k v − 1)/2 for every v in the graph.
Note that, as you might expect, the clustering coefficient takes 3
Jukka-Pekka Onnela, Jari Saramäki,
different values for weighted3 , 4 and directed5 graphs. János Kertész, and Kimmo Kaski.
I primed you to expect that many statistical properties can be Intensity and coherence of motifs in
weighted complex networks. Physical
derived via matrix operations. This is true also for the clustering
Review E, 71(6):065103, 2005
coefficient. It is done via the powers of the binary adjacency matrix – 4
Jari Saramäki, Mikko Kivelä, Jukka-
see Section 7.1. Triangles are closed paths of length 3, while triads are Pekka Onnela, Kimmo Kaski, and
Janos Kertesz. Generalizations of
paths of length 2. The number of closed walks of length 3 centered the clustering coefficient to weighted
on u is A3uu , while the number of walks of length 2 passing through u complex networks. Physical Review E, 75
is A2uv , with u 6= v, which results in the formula: (2):027105, 2007
5
Giorgio Fagiolo. Clustering in complex
directed networks. Physical Review E, 76
∑ A3uu (2):026107, 2007
u
CC = .
∑ A2uv
u6=v

So, let’s calculate the global clustering coefficient for the graph in
Figure 9.6(a). We know how many triads there are in the graph. How
many triangles are there? Here I made my life easier, because it is
rather trivial to count the number of triangles in a planar graph – a
graph you can draw on a 2D plane without intersecting any edges.
There are eight triangles and 48 triads in the network. Thus the
global clustering coefficient of the network is CC = 3 × 8/48 = 0.5.
Half of the triads in the network close to form a triangle.
The fact that I keep calling this the “global” clustering coefficient,
 density 137

3 Figure 9.6: (a) Estimating the

6 global clustering coefficient of
3
1 4 a graph. Each node is labeled
1 2 10 3 1 2 with the number of triads cen-
5
15 6 3 tered on it, and the numbers
among the edges count the tri-
7 6 4 angles. (b) Estimating the local
8 5
clustering coefficient of a node,
1 3
counting the triangles to which
(a) (b) it belongs.

should tip you off about the existence of a “local” clustering coef-
ficient. Its definition is rather similar, but it is focused on a single
node. It is the number of triangles to which the node belongs over
the number of triads centered on it. We do not multiply by three the
numerator, because we’re focusing exclusively on the triads of v, that
can then be closed only in one way.
Looking at Figure 9.6(b), let’s try to calculate the local clustering
coefficient of the node highlighted by the arrow. Again, we use
our planar graph to have an easy time counting triangles: there
are five that include node v. We already counted the number of
triples before: it was 15. Thus, the local clustering coefficient of v is
CCv = 5/15 = 0.3̄.

2/3 Figure 9.7: I label each node

1/2 in the network with its local
1 clustering coefficient CCv value.
1/2 2/3

1/3

1/2

1 2/3

And, hopefully, that’s it. Oh, who am I kidding. Of course there’s

more. Once you have a local clustering coefficient, one might get
curious and desire to calculate the average local clustering coefficient
1
of the network. This is simply CCavg = ∑ CCv . You would
| V | v ∈V
hope that CCavg = CC. No such luck. For the network in Figure
9.7, we know that CC = 0.5. Unfortunately, if you average the CCv
values I used to label each node, you’ll find out that CCavg = 0.648.
So you have to remember that the global and the average clustering
138 the atlas for the aspiring network scientist

coefficient are two different things, and not to report one as the other.
I closed the previous section with a mantra – “real world networks
are sparse” –, so I want to do it again. However, there’s a surprise
here. If average degrees are low and networks are sparse, wouldn’t
you expect real world networks to have a low clustering too? Instead,
the opposite holds: real world networks are clustered. The power
grid example I used before? It has a CC of 0.1032, which is 150 times
higher than you would expect if its edges were distributed randomly.
The scientific paper citations? It has CC = 0.318, more than 200 times
higher than expected.
This means that these systems might have few connections per
node, but these connections tend to be clustered in the same neigh-
borhood. Nodes tend to close triangles. This is especially true for
social systems. In fact, the protein-protein network I used in Chapter
6 has a clustering of 0.0236, which is still higher than expected. But
in this biological case, we only have a factor of 16, a far cry from the
factor of 200 in the social paper citation system.

9.3 Cliques

When it comes to clustering and density, you cannot do any better

than having all possible edges among the nodes in your network.
A network will contain many subsets of nodes for which this is
true: you pick k nodes in the network and all possible connections
among them are present. This happens often in social networks:
these complete graphs – as we call them – represent tightly knit
groups of friends. They also get a special name: cliques.

Figure 9.8: The clique zoo.

(a) 2-clique. (b) 3-clique. (c) 4-clique. (d) 3,2-clique.

Formally, the definition of a clique is a subgraph of k nodes and
k(k − 1)/2 edges – assuming we’re in the usual case of undirected
unipartite networks. The most general way to refer to a clique of size
k is by calling it a k-clique. So an edge, which is a clique of size two,
is a 2-clique (Figure 9.8(a)). A clique with three nodes is a 3-clique
(Figure 9.8(b)), with four nodes we have a 4-clique (Figure 9.8(c)) and,
hopefully, you can generalize from that. A few cliques get special
names too, given to their popularity. We already named the 3-clique
in the previous section as a triangle.
The case of bipartite networks is a peculiar one worth mentioning.
 density 139

As we know from Section 4.1, in bipartite graphs we’re not allowed

to connect nodes of the same type. So, when we define a clique as
a subset of nodes where “all possible connections among them are
present”, we mean something radically different in a bipartite net-
work. Here, we simply mean that all nodes of V1 type are connected
to all nodes of V2 type. So we call this structure a biclique. Figure
9.8(d) shows an example of biclique made of three nodes of type
1 and two nodes of type 2. We need to modify the way to refer to
specific instances of a biclique: from k-clique to n, m-clique. So the
one in Figure 9.8(d) is a 3,2-clique.

Figure 9.9: An example of

maximal 6-clique containing
a non-maximal 4-clique, high-
lighted with the blue outline.

It follows from the definition of a clique that any k-clique will

contain many (k − 1)-cliques, down to 2-cliques. If you have all
possible edges between six nodes, you can pick any five (four, three,
...) of those six nodes and they’re all connected to each other – by
definition. Figure 9.9 provides a depiction of this reasoning. We want
to distinguish between cliques that are contained in other cliques,
and cliques that aren’t. We call the latter maximal cliques: the set of
nodes to which you cannot add any other node and still make it a
clique.

9.4 Independent Sets

Just like matter has anti-matter, also cliques have anti-cliques. By

definition, a clique is a set of nodes all connected to each other. The
anti-clique, which we call “independent set” is a set of nodes none 6
Chris Godsil and Gordon F Royle.
of which are connected to each other6 . Figure 9.10 shows a possible Algebraic graph theory, volume 207.
subdivision of a graph into independent sets. Springer Science & Business Media,
2013
Note that, in Figure 9.10, I force each node to have a color, i.e. to
belong to at least an independent set. I could find a larger indepen-
dent set, for instance the purple node could make an independent set
with the two red nodes it isn’t connected to – since they are not con-
nected to each other. The task of finding an independent set coverage
of a graph is called graph coloring and it’s a classical graph theory
140 the atlas for the aspiring network scientist

Figure 9.10: A graph with

several independent sets, rep-
resented by the color of the
nodes.

7
Tommy R Jensen and Bjarne Toft.
problem7 . Solving graph coloring tells you, for instance, how many Graph coloring problems, volume 39. John
colors you need to use for your map such that no two neighboring Wiley & Sons, 2011
countries share the same hue – in this case you represent countries as
nodes and connect two countries if they share a land border.
When it comes to independent sets, you should not confuse the
maximal independent set with the maximum independent set. A maxi-
mal independent set, just like a maximal clique, is an independent set
to which you cannot add any node and still make it an independent
set. The green set in Figure 9.10 is maximal because the two green
nodes are connected to all other nodes in the graph.
On the other hand, the maximum independent set is simply the
largest possible independent set you can have in your network. In
Figure 9.10, the red set is the maximum independent set, or at least
one of the many possible independent sets of size 3. Finding the
largest possible independent set is an interesting problem, because it
tells you something about the connectivity of the graph. It also has
applications in graph mining – see Section 39.4.

9.5 Summary

1. We define a network’s density as the number of its edges divided

by the total possible amount of edges, which is a different number
depending whether your network is directed or not.

2. Real world networks tend to be sparse, meaning that the density is

usually lower than a few percentage points.

3. A way to estimate a local density by looking at the neighborhood

of a node is the clustering coefficient: the number of common
neighbors around node u connected to each other. There are
global, local, and average versions of the clustering coefficient,
sometimes known as trasitivity.

4. Differently from density, many real world networks have very

high clustering.
 density 141

5. A (sub)graph with density equal to one is a clique: a set of nodes

all connected to each other. Bipartite networks have bicliques.

6. The opposite of a clique is an independent set: a group of nodes

none of which connects to any other member of the group.

9.6 Exercises

1. Calculate the density of hypothetical undirected networks with

the following statistics: |V | = 26, | E| = 180; |V | = 44, | E| = 221;
|V | = 8, | E| = 201. Which of these networks is an impossible
topology (unless we allow it to be a multigraph)?

2. Calculate the density of hypothetical directed networks with the

following statistics: |V | = 15, | E| = 380; |V | = 77, | E| = 391;
|V | = 101, | E| = 566. Which of these networks is an impossible
topology (unless we allow it to be a multigraph)?

3. Calculate the global, average and local clustering coefficient for

the network in [Link]
txt.

4. What is the size in number of nodes of the largest maximal clique

of the network used in the previous question? Which nodes are
part of it?

5. What is the size in number of nodes of the largest independent

set of the network used in the previous question? (Approximate
answers are acceptable) Which nodes are part of it?
 Part III

Centrality
10
Shortest Paths

The degree (Chapter 6) is the most direct measure of importance of

a node in a network. The more connections a node has, the more
important it is. However, there are alternative ways to estimate
the importance of a node. Sometimes, it doesn’t matter how many
connections you have, but how many people someone can reach
by passing through you. Normally, the two are correlated – more
connections mean more possibilities – but that’s not always the case.
We explore these differences in this part of the book.
Before we start ranking nodes in Chapter 11 we need to lay down
some groundwork. A significant chunk of measures of node im-
portance are based on the concept of shortest paths, which is what
we’re exploring in this chapter. We start by defining how to explore a
graph and we build up from there.

10.1 Graph Exploration

When we first encounter a graph, how do we know its topology and

properties? Humans can “see” and parse simple graphs, but how
does a computer do it? If we start from a node, how do we access
information about its connections, its neighbors, and the connections
among them? We need to perform a graph exploration – or graph
traversal. There are two main ways to do it: Depth First Search (DFS)
and Breadth First Search (BFS).

Depth & Breadth First

In Depth First Search (DFS), you start by picking a root node. Then
you put its neighbors in a Last-In-First-Out queue. You pick the last
neighbor you added (you “pop” the queue) and you perform the
same operation: you add its neighbors to the queue, making sure you
don’t add nodes you already explored. You continue until you have
explored all nodes in the graph. Figure 10.1(a) provides an example,
144 the atlas for the aspiring network scientist

1 1

2 12 1922 27 2 3 4 5 6

3 10 13 2023 2829 7 8 9 1011 1213

4 9 1114182124 303234 36 14 151617181920 212223 24

5 6 7 8 1516 1725263133353738394041 252627282930 3132333435363738394041

(a) DFS (b) BFS

Figure 10.1: Exploring a graph
showing the exploration order. Since you’re using a Last-In-First-Out by DFS and BFS. In both cases, I
queue, the very first neighbor of the root node will be the last node to label each node with the order
be explored – unless you encounter it again as the neighbor of some in which it gets explored. I use
other node down the exploration tree, of course. the node color to encode the
Breadth First Search (BFS) is practically speaking the exact same same information, from light
algorithm as DFS, with a tiny change. Rather than putting the neigh- (early explored) to dark (late
bors of the root node in a Last-In-First-Out queue, you put them into explored).
a First-In-First-Out queue. This changes fundamentally the way you
explore the graph: you explore all neighbors of the root node before
passing to the first neighbor of the first neighbor of the root node.
Figure 10.1(b) provides an example, showing the exploration order.
DFS tends to make a gradient over followed paths until it back-
tracks because it explored the entire neighborhood – in the example
from Figure 10.1(a) it backtracks, for instance, from the eighth ex-
plored node back to the third explored node. BFS tends to make a
gradient from the origin node to the farthest nodes in the network.

Random Node/Edge Access

In day to day computing, you might find yourself exploring the
graph in two other ways. These are dependent on the way we store
graphs on a computer’s hard disk. We can call these two methods
random edge access and random node access.

Src Trg Node Neighbors

Figure 10.2: Two different ways
1 2 1 2, 3
to store graphs on disk.
2 4 2 1, 4
5 3 5 3, 4
1 3 3 1, 5
4 5 4 2, 5
(a) Edge list. (b) Adjacency list.
 shortest paths 145

Random edge access is when you read the file containing your
graph one line at a time, and each line contains an edge. We call this
type of graph storing format an “edge list”, because it’s a list of one
edge per line. In this case, you may or may not have sorted the edges
in a particular way, but the baseline assumption is that they’re in a
random order. See Figure 10.2(a) for an example.
Random node access is the same, but the file records, in each line,
the complete list of a node’s neighbors. We call this type of graph
storing format an “adjacency list”, because it’s a list of the adjacency
of one node per line. Also in this case, you may or may not have
sorted the nodes – and their neighbors – in a particular way, but the
baseline assumption is that they’re in a random order. See Figure
10.2(b) for an example.

10.2 Finding Shortest Paths

The problem of these ways to explore the graph is that they are
not optimal, they cannot find the “best” (shortest) way to go from
v to u. Well, they can, but only in very specific cases under very
specific assumptions. For instance, BFS finds shortest paths only for
undirected unweighted graphs. This can be useful, for instance, to
1
Shimon Even. Graph algorithms.
Cambridge University Press, 2011
find the shortest path out of a maze1 , 2 , 3 . But we still need a better, 2
Edward F Moore. The shortest path
more general way. through a maze. In Proc. Int. Symp.
To see why finding “best paths” is important, suppose you have Switching Theory, 1959, pages 285–292,
1959
to deliver a letter to a person, as I show in Figure 10.3. If you know 3
Konrad Zuse. Der Plankalkül. Num-
them, no problem: you just give it to them. What if you don’t? You ber 63. Gesellschaft für Mathematik und
might know one of their friends, and pass through them. Or you Datenverarbeitung, 1972

might know that one of your friends knows one of theirs. But if
none of this is true, you have to know the shape of the entire social
network – the part in gray in the figure – and discover what’s the
least amount of people you have to bother to get your letter to the
recipient. This we call the “shortest path problem” in networks.

Figure 10.3: A vignette rep-

resenting the problem of
delivering a letter through
acquaintances: how do you
know the best path is at the
top since you’re unaware of the
existence of the people in gray?
146 the atlas for the aspiring network scientist

The formal specification of the shortest path problem is the follow-

ing. You’re given a start node v and a target node u. You have to find
the path going from v to u crossing the fewest possible number of
edges. Figure 10.4 provides a visualization of a shortest path between
two nodes. In fact, the figure highlights a feature of this problem: it
provides not one but two solutions. That is because, in unweighted
undirected graphs, it is quite common to find multiple shortest paths
between a given origin-destination pair. In other words, the solution
to the shortest path problem is not necessarily unique.

Figure 10.4: Finding the short-

est path – edges colored in
purple – between the start node
(in blue) and the target node (in
green). Note that (a) and (b) are
both valid shortest paths which
have the same length.

(a) (b)
This is for the case of undirected, unweighted networks. If you
have directed networks you obviously have to respect the edge direc-
tions – see Figure 10.5(a). If you have weighted networks, you might
want to minimize the weight (as in Figure 10.5(b)), assuming that the
edge weight represents its traversal cost. If your edge weights repre-
sent proximities rather than costs, for instance they are the capacity
of a trait of road as explained in Section 3.3, you’d do the opposite.

3 Figure 10.5: Finding the short-

2 2 1 est path – edges colored in
1
purple – between the start node
4 2
4 (in blue) and the target node (in
4
1 3 green). (a) Directed network. (b)
3 Weighted network, where edge
5 3 weights represent the cost of
2
traversal.
1

(a) (b)
How do we find the shortest path? Depending on the properties
of the graph (e.g. direct/undirected, weighted/unweighted), there
are different algorithms for finding shortest paths. We also need to
know if we just want to find a path from v to u (single-origin single-
 shortest paths 147

2 Figure 10.6: A quick refer-

ence of the most well known
algorithms used to solve spe-
cific shortest path problems.
BFS Dijkstra* Dijkstra* Columns (from left to right):
simple, weighted, directed.
Rows (top to bottom): single-
origin, all pairs shortest path.
Dijkstra is marked with a star
Floyd- Floyd- Floyd-
because variants of the base
Warshall Warshall Warshall
algorithm can outperform it in
special cases and they are used
in most real world scenarios.
destination shortest path), or from v to all other nodes (single-origin
shortest path), or between every single pair of nodes.
Figure 10.6 provides you with a quick reference on which algo-
rithm to use given each use case. For instance, as mentioned before, if
you have an undirected unweighted network and you are interested
in single-origin shortest paths, you can find them by performing a
simple BFS exploration.
If you still have a single-origin in mind but your network contains
directions and/or weights, you’ll probably use one of the many 4
Edsger W Dijkstra. A note on two
problems in connexion with graphs.
flavors of the classical Dijkstra’s algorithm4 . Dijkstra’s algorithm Numerische mathematik, 1(1):269–271,
works as follows. You start by your origin, which you mark as you 1959
“current node”. 5
For example, if the current node u is
at distance of 6 from the source, and
the edge connecting it with a neighbor
1. You look at all the unvisited neighbors of the current node and v has length 2, then the distance to v
calculate their tentative distances through the current node. through u is 6 + 2 = 8. If you previously
marked v with a distance greater than 8,
you will change it to 8. Otherwise you
2. Compare this tentative distance to the current assigned value and will keep the current value.
assign the smallest one.5 6
[Link]
wikipedia/commons/5/57/Dijkstra_
3. When you are done considering all of the unvisited neighbors of [Link]
the current node, mark the current node as visited. You will never 7
Robert B Dial. Algorithm 360: Shortest-
path forest with topological ordering
check a visited node twice. [h]. Communications of the ACM, 12(11):
632–633, 1969
4. If the current node, the one you’re marking as visited, is the 8
Ravindra K Ahuja, Kurt Mehlhorn,
destination node, you can stop. Otherwise, you can continue by James Orlin, and Robert E Tarjan.
selecting the unvisited node that is marked with the smallest Faster algorithms for the shortest path
problem. Journal of the ACM (JACM), 37
tentative distance, as your new current node. Then go back to step (2):213–223, 1990
1. 9
Rajeev Raman. Recent results on the
single-source shortest paths problem.
I cannot include in the book the best visual representation of the ACM SIGACT News, 28(2):81–87, 1997
10
Mikkel Thorup. On ram priority
Dijkstra algorithm I know, because it is an animated GIF6 . queues. SIAM Journal on Computing, 30
Faster variations of the Dijkstra algorithm7 , 8 , 9 , 10 use clever data (1):86–109, 2000
148 the atlas for the aspiring network scientist

structures and a few optimizations – often under assumptions about

the edge weights – that are of no interest here.
The only other algorithm in the hall of fame of shortest path 11
Bernard Roy. Transitivité et connexité.
algorithms we consider here is Floyd-Warshall11 , 12 , 13 , 14 . That is Comptes Rendus Hebdomadaires Des
because it is the most used algorithm for the all-pairs shortest path Seances De L Academie Des Sciences, 249
(2):216–218, 1959
problem, when you’re not limiting yourself to a single origin and/or 12
Stephen Warshall. A theorem on
a single destination – or to specific constraints on topology and/or boolean matrices. Journal of the ACM
edge weights. The algorithm uses recursive programming which, to (JACM), 9(1):11–12, 1962
this day, I still consider borderline magic.
13
Robert W Floyd. Algorithm 97:
shortest path. Communications of the
Suppose you have a function sp(u, v, K ) that calculates the shortest ACM, 5(6):345, 1962
path between u and v using only nodes in the set K. K is a special 14
Bernard Roy, who actually discovered
set, it contains all nodes of the network with id equal to or lower the algorithm first, for mysterious
reasons gets no naming rights.
than K. Obviously, if K = 0, then it is an empty set. Then sp(u, v, 0)
simply returns the weight of the edge between u and v – if they are
connected –, because we’re not using any node in the path:

sp(u, v, 0) = Auv .
If K > 0 it means that we are adding a node as a possible member
of the shortest path. When we do it, either of two things can happen:
(i) adding the extra node allowed us to find a better (shorter) path, or
(ii) it didn’t. So:

sp(u, v, K ) = min(sp(u, k, K ) + sp(k, v, K ), sp(u, v, K − 1)).

1 3 2

1 8 4

3 1 1 3

1 6
2 1 3
1 1
8 4
1 3 2 2 4
2 2 4 1 1 3 1 2
3 2

2 3 6 4 3 1 2 2 4 1 1 3 1 2 2 4

(a) Input (b) K = 0 (c) K = 2 (d) K = 3

Figure 10.7: (a) The input for
Figure 10.7 shows an example run. Figure 10.7(a) is an hypothet-
the Floyd-Warshall algorithm.
ical input. At the first step, K = 0, we can only consider directly
(b-d) The temporary short-
connected origins and destinations, setting the edge weights as the
est paths at each step of the
length – Figure 10.7(b). For K = 1 (not pictured) nothing happens:
algorithm.
node 4 cannot use node 1 to go anywhere, because their edge is very
costly, and nodes 2 and 3 have low cost connections to node 1, but
 shortest paths 149

they are already directly connected by the minimum weight in the

network. For K = 2 (Figure 10.7(c)) we’re also allowed to use node 2
for our paths. Both node 1 and node 3 use it to get to node 4, given
that their direct connection to node 4 is costly. For K = 3 (Figure
10.7(d)) we can also use node 3 in our paths. The path 1 → 3 → 2 is
the sum of two paths we already know from Figure 10.7(b): 1 → 3
an 3 → 2. It costs less than 1 → 2, so we select it. To go from node 1
to node 4 we sum two paths we already know: 1 → 3 (from Figure
10.7(b)) and 3 → 2 → 4 (from Figure 10.7(c)). We discover then that
the actual distance between the nodes 1 and 4 is four, rather than five
– as we though in Figure 10.7(c) – or eight – as we though in Figure
10.7(b).

Figure 10.8: (a) A weighted net-

-1 -1 -2
work with negative weights
which results in degener-
1 ate shortest paths – in blue –
2 2 2 2 2 2
over preferred non-shortest
paths – in green. (b) A directed
1 1 1 weighted network with negative
weights but without the infinite
(a) (b) (c) negative weight problem. (c) A
directed weighted network with
negative cycles.
A final word about negative weights. As presented earlier, there’s
no shame if your network contains them (see Section 3.3). However,
you need to be careful when computing shortest paths. The reason
is evident, as one can see from Figure 10.8(a). The problem with
negative weights is that we might think that it is trivial to find a
shortest path (in green in the figure), but by going back and forth
over a negative weight we can find an equivalent path. At that point,
we can be stuck in an infinite loop of shorter and shorter paths
without ever reaching the destination (in blue in the figure).
Directed networks can allow negative weights, because you’re not
allowed to follow the edge against its direction, as in Figure 10.8(b).
However, if there is a negative cycle – see Figure 10.8(c) – you are in
the same situation as before. A negative cycle is a cycle whose total
edge weight sum is lower than zero.
If you’re writing shortest path algorithms, you have to take care
of these situations. Usually, you have to explicitly say that you’re
looking for paths, not walks. In paths, you cannot re-use the same
edge twice (see Chapter 7), no matter how cool it would make your
path length.
150 the atlas for the aspiring network scientist

10.3 Path Length Distribution

Just like with the degree, knowing the length distribution of all
shortest paths in the network conveys a lot of information about its
connectivity. A tight distribution with little deviation implies that all
nodes are more or less at the same distance from the average node in
the network. A spread out distribution implies that some nodes are
in a far out periphery and others are deeply embedded in a core.

# Paths
Figure 10.9: The path length
distribution (left) of a graph
(right). Each bar counts the
number of shortest paths of
length one, two and three,
which is the maximum length
in the network.

Path Length

To generate a path length distribution you perform the same

operation you used to get the degree distribution: you have the path
length on the x axis and the number of paths of a given length on
the y axis. See Figure 10.9 for an example. I’m not going to go on
a tangent on log-log spaces and power laws like last time because
usually path lengths distribute quasi-normally: you’ll find a lot of
classical bell shapes.
Some values in the distribution are fixed. For instance, the number
of paths of length one is twice the number of edges, because each
edge is used for two paths of length one (u → v, and v → u). It goes
without saying that things are different in directed networks. The
number of total shortest paths is |V |(|V | − 1), because each origin
has to reach each destination, minus one because we don’t count the
paths of length zero, from the origin to the origin.

Diameter
The rightmost column of the histogram in Figure 10.9 is important.
It records the number of shortest paths of maximum length. These
are the “longest shortest paths”. Since this is an important concept,
such a long mouthful name won’t do. We’re busy people and we
got places to be. So we use a different name for them or, to be more
precise, to their length. We call it the diameter of the network.
Why do we care about the diameter? Because that’s the worst case
 shortest paths 151

for reachability in the network. The diameter is the measure of the

maximum possible separation between two nodes. A long diameter
means that the problem of finding a shortest path for some pairs of
nodes might be too hard because there are too many hypothetical
paths and splits to consider. With a small diameter, everybody is
reachable in one or two hops. With a large diameter, a full traversal
of the graph might be impossible, especially if we only have local
information about our neighborhood.
Let’s go over a few values of diameter, just to get a grasp of the
concept:

• Diameter = 1 → You know everyone;

• Diameter = 2 → Your friends know everyone;

• Diameter = 3 → Your friends know someone who knows every-

one;

• ...

It’s now easy to see that a network with diameter equal to three
is easy to navigate. As the diameter grows, the number of people to
rely on for a full traversal starts becoming unwieldy.
If your network has multiple connected components (Section 7.4),
we have a convention. Nodes in different components are unreach-
able, and thus we say that their shortest path length is infinite. Thus,
a network with more than one connected component has an infinite
diameter. Usually, in these cases, what you want to look at is the
diameter of the giant connected component.

Average
The diameter is the worst case scenario: it finds the two nodes that
are the most far apart in the network. In general, we want to know
the typical case for nodes in the network. What we calculate, then, is
not the longest shortest path, but the typical path length, which is the
average of all shortest path lengths. This is the expected length of the
shortest path between two nodes picked at random in the network.
If Puv is the path to go from u to v and | Puv | is its length, then the
∑ | Puv |
u,v∈V
average path length of the network is APL = . Figure
|V |(|V | − 1)
10.10 shows that, even in a tiny graph, the diameter and the APL
can take different values, with the former being more than twice the
length of the latter.
With APL, we can fix the origin node. For instance, in a social
network, you can calculate your average separation from the world.
152 the atlas for the aspiring network scientist

Diameter = 3
Figure 10.10: The diameter and
the APL in a graph can be quite
different.

APL ~ 1.4

Mean = 3.57
125
Figure 10.11: The path length
100 distribution for Facebook in
FB Users (M)

2012.
75

50

25

2.5 2.7 2.9 3.1 3.3 3.5 3.7 3.9 4.1 4.3 4.5 4.7
Average Degree of Separation

This would be an APLv , the average path length for all paths starting
at v. Then you can generate the distribution of all lengths for all
origins. How does this APLv distribution look like for a real world
network? One of the most famous examples I know comes from 15
Sergey Edunov, Carlos Diuk, Is-
Facebook15 . I show it in Figure 10.11. The remarkable thing is how mail Onur Filiz, Smriti Bhagat, and
ridiculously short the paths are even in such a gigantic network. Moira Burke. Three and a half degrees
of separation. Research at Facebook, 2016
This is in line with classical results of network science, showing
that the diameter and APL typically grow sublinearly in terms of 16
Mark EJ Newman. The structure and
number of nodes in the network16 . In other words, there are dimin- function of complex networks. SIAM
ishing returns to path lengths: each additional person contributes review, 45(2):167–256, 2003b

less and less to the growth of the system in terms of reachability. In 17

Jure Leskovec, Jon Kleinberg, and
fact, some researchers have found that adding people might even Christos Faloutsos. Graphs over
time: densification laws, shrinking
shrink the diameter17 , 18 : as people join a social network, they create diameters and possible explanations. In
shortcuts and new paths that bring close together people that were Proceedings of the eleventh ACM SIGKDD
international conference on Knowledge
previously far apart.
discovery in data mining, pages 177–187.
The most notorious enunciation of the surprising small average ACM, 2005a
path length in large networks is the famous “six degrees of sepa- 18
Jure Leskovec, Jon Kleinberg, and
Christos Faloutsos. Graph evolution:
ration”. This concept says that, on average, you’re six handshakes
Densification and shrinking diame-
away from meeting any person in the world, being a fisherman in ters. ACM Transactions on Knowledge
Cambodia or an executive in Zimbabwe. People used this concept to Discovery from Data (TKDD), 1(1):2,
2007b
describe the famous – failed – Milgram experiment. 19
Stanley Milgram. The small world
In 1967, Milgram published a paper19 detailing the travels of a problem. Psychology today, 2(1):60–67,
1967
 shortest paths 153

series of envelopes. He handed a destination address to people in

the Midwest of the United States. The destination was in Boston,
Massachusetts. The idea was that each recipient needed to attempt
to have the letter reach its final destination. However, they could
not mail it directly: they needed to hand it over to a person they
knew on a first name basis. So they needed to figure out who in
their acquaintances was most likely to know somebody (who knew
somebody, who knew somebody, ...) in Massachusetts. Each handler
of the envelope would have to write their name on it. When the
envelope reached the destination, counting the names in it would
give an approximation of the degrees of separation between the
origin and destination individuals.
The number turned out to be 5.5 on average, which gave fuel to
the “six degrees of separation” urban legend. However, the experi-
ment was arguably a failure given that, of the more than 400 letters
sent, less than a hundred actually arrived at the destination. The
problem is that obviously there is no way to account for the fact that
a letter might not successfully reach its target because some people
in the chain were unreliable, rather than unconnected with the desti-
nation. Fascinating as it is, this theory might be wrong because the
degrees of separation could be lower than six: people have proposed 20
Lars Backstrom, Paolo Boldi, Marco
four20 , as we see in Facebook (Figure 10.11). Rosa, Johan Ugander, and Sebastiano
Diameter and average path length are only the two most famous Vigna. Four degrees of separation. In
Proceedings of the 4th Annual ACM Web
and most used measures derived from the shortest path length
Science Conference, pages 33–42. ACM,
distribution. There is a collection of other measures you might find 2012
in network science papers and books. Two other examples are the
eccentricity of a node and the radius of a network. You can think of
the eccentricity as a node-level diameter. It is the longest shortest
path leading from node u to the farthest possible node v in the
network. Thus, by definition the diameter is equal to the highest
eccentricity among the nodes of the network. The radius of a network
is, conversely, equal to the smallest eccentricity in the network.

10.4 Spanning Trees & Other Filtered Graphs

I conclude this chapter with a look at spanning trees and other

ways to filter down a graph. These methods are usually deployed to
reduce a network to its minimum terms and finding its fundamental
structure in a way that is parsable by humans. They are also at the
basis of some network backboning techniques (Chapter 24).
A spanning tree of an undirected graph is a subgraph that: (i) is
a tree (see Section 7.2), and (ii) it includes all of the vertices of the
graph. In practice, it is that subgraph that can connect all nodes of
the graph with the minimum number of edges, and no cycles.
154 the atlas for the aspiring network scientist

Figure 10.12: (a) A graph with

one of its possible spanning
trees highlighted in green. (b)
The minimum spanning tree
of a weighted graph, with the
edge width proportional to
its weight. (c) The maximum
spanning tree of a weighted
graph.

(a) (b) (c)

Figure 10.12(a) shows you an example of a spanning tree inside a

graph. If your graph has multiple connected components you cannot
find a single spanning tree, because you don’t have a way to connect
nodes in different components. However, you can make a spanning
forest, by finding the spanning trees of each component separately.
Spanning trees are nice, but they get used mostly in weighted
networks. In that case, you have to distinguish between weights
as proximities and weights as distances (Section 3.3): is an edge
with a high weight expressing the cost of going from u to v, or is
it saying how much u and v interact? In the first case we have a
“distance” weight: we want to minimize costs. Imagine finding the
tree connecting all your road intersections that minimizes driving
distance – the cost of an edge. 21
Ronald L Graham and Pavol Hell. On
When your weights are distances you want a minimum spanning the history of the minimum spanning
tree21 : the spanning tree among all spanning trees of a graph that tree problem. Annals of the History of
Computing, 7(1):43–57, 1985
has the minimum possible total edge weight. Figure 10.12(b) shows 22
Otakar Borůvka. O jistém problému
an example. When your weights are proximities – maybe because minimálním. 1926
they tell you the capacity of the road – then you want the maximum 23
Robert Clay Prim. Shortest connection
networks and some generalizations. Bell
spanning tree: the spanning tree among all spanning trees of a graph
system technical journal, 36(6):1389–1401,
that has the maximum possible total edge weight. Figure 10.12(c) 1957
shows an example. 24
Joseph B Kruskal. On the shortest
spanning subtree of a graph and the
Of course, the algorithm to find the minimum and the maximum
traveling salesman problem. Proceedings
spanning tree is the same, you just flip the sign of the comparison. of the American Mathematical society, 7(1):
There’s a good range of algorithms, from classical ones to more mod- 48–50, 1956
25
Bernard Chazelle. A minimum
ern which use special data structures: Borůvka22 , Prim23 , Kruskal24 ,
spanning tree algorithm with inverse-
Chazelle25 . They are usually all implemented in standard network ackermann type complexity. Journal of
analysis libraries. the ACM (JACM), 47(6):1028–1047, 2000
26
Eugene L Lawler, Jan Karel Lenstra,
Note that finding the minimum spanning tree doesn’t really solve
AHG Rinnooy Kan, David Bernard
the traveling salesman problem26 , although it sounds like it should. Shmoys, et al. The traveling salesman
A quick recap: the traveling salesman problem is the quest to find problem: a guided tour of combinatorial
optimization, volume 3. Wiley New York,
the shortest possible route that visits each city and returns to the 1985
 shortest paths 155

origin city, given a list of cities and the distances between each pair of
cities. We can represent the problem as a weighted graph, with city
distances as edge weights. The minimum spanning tree doesn’t solve
the problem: it creates a tree, which has no cycles. Thus, to get back
to the origin city, you have to backtrack all the way through the tree –
not ideal.

Figure 10.13: An example of

12 a weighted graph with a non
unique minimum spanning
12
tree.
6

Another thing to keep in mind is that rarely minimum/maximum

spanning trees are unique: a weighted network can and will have
multiple alternative minimum/maximum spanning trees. Consider
the graph in Figure 10.13. Suppose we want to find its minimum
spanning tree. The first choice is obvious: we use the edge of weight
6. Then, we have to connect the final node. Each of the edges of
weight 12 is a valid addition to the tree: they will connect the node
and the result will be a tree, an acyclic graph. So the graph has two
valid minimum spanning trees.
There is an easy rule to remember to know whether a graph
will have a unique minimum/maximum spanning tree or not. If
each edge has a distinct weight then there will be only one, unique
minimum spanning tree. As soon as you have two edges with the
same weight, you open the door to the possibility of having more
than one minimum spanning tree. In fact, in an unweighted graph
where we assume that all edges have the same weight equal to one,
then every spanning tree of that graph is minimum.
Spanning trees have some closely related cousins that are worth-
while mentioning. The first one is the planar maximally filtered
27
Michele Tumminello, Tomaso Aste,
graph27 . As the name suggests, this is a technique to reduce any arbi-
Tiziana Di Matteo, and Rosario N Man-
trary graph into a planar version of itself, such that the edge weight tegna. A tool for filtering information
sum is maximal (or minimal, depending on the meaning of your edge in complex systems. Proceedings of the
National Academy of Sciences, 102(30):
weights). Since a spanning tree is a tree, it means that it must have 10421–10426, 2005
|V | − 1 edges. On the other hand, a planar maximally filtered graph
must have 3(|V | − 2) or fewer edges.
Just like in the case of the tree, also in this case some motifs cannot
appear. In a tree you cannot have cycles. In a planar graph you
cannot have a motif that is impossible to draw as planar – i.e. on a
2D surface without edge crossings –, for instance a 5-clique or a 3,3-
156 the atlas for the aspiring network scientist

Figure 10.14: Two examples

of non planar graphs that can-
not be included in any planar
maximally filtered graph. (a) A
5-clique; (b) a 3,3-biclique.

(a) (b)

biclique. Look at Figure 10.14 and try to draw those graphs in two
dimensions without having any edge crossing another one. You’ll
find out that is not possible.
The second cousin of spanning trees is the triangulated maximally 28
Guido Previde Massara, Tiziana
filtered graph28 . This was originally proposed as a more efficient Di Matteo, and Tomaso Aste. Network
algorithm to extract planar maximally filtered graphs from larger filtering for big data: Triangulated
maximally filtered graph. Journal of
graphs. However, it also allows to specify different topological con-
complex Networks, 5(2):161–178, 2016
straints, which are not necessarily making the graph planar.

10.5 Classic Combinatorial Problems

Graph exploration in general, and shortest paths in particular, are

linked with some of the most famous problems discussed in com-
puter science. We already saw one in Section 9.4 – graph coloring:
how many colors do I need to make sure that I don’t give the same
one to two connected nodes? Here I mention another, related to the
classic Traveling Salesman Problem. In the Traveling Salesman Prob-
lem, we have a set of cities and we want to find the path that allows
us to visit all cities by covering the minimum possible distance.

Figure 10.15: A graph with two

Hamiltonian cycles highlighted
using the edge color. Red =
minimum Hamiltonian; green
= maximum Hamiltonian. The
edge’s thickness is proportional
to its weight.

In this scenario, we are assuming that cities live in a two dimen-

sional space and there is a path between any two cities. However, we
could impose the existence of a road graph that makes some city-city
connections impossible. In this case, we want to find the path of
minimum cost in a graph that visits each node exactly once (i.e. the
minimum Hamiltonian path – see Chapter 7). Figure 10.15 shows an
 shortest paths 157

example, with two Hamiltonian paths of different costs highlighted

in red and green.
Such problems have a huge importance in computer science
because they are classical examples of NP-hard problems. These
problems have no known polynomial-time solution, meaning that
we can usually only find approximate solutions in a reasonable time.
Finding the best solution would require brute force algorithms whose
time complexity make them unsuitable for problems of large size –
i.e. if your graph has more than a handful nodes.
Combinatorics and graphs have a much deeper relationship that
this one, though. A vast number of problems in combinatorics can be
represented as a graph problem, and often graphs are the best tool to
solve them. Two other examples are the classic SAT problem, where
we want to know if there is a true/false assignment so that a set of
logical propositions is not contradictory; and vehicle routing, where
we want to find the optimal set of routes for several vehicles to reach
their destinations from their origins.

10.6 Summary

1. There are many ways to explore a graph structure. Breadth-First

Search means to explore all neighbors of a node before exploring
their neighbors; Depth-First Search means to explore a neighbor’s
neighbors before moving on to the next direct neighbor; random
node and edge access means to explore one node or edge at a time
ignoring the graph’s topology.

2. Shortest paths are the paths connecting two arbitrary nodes in the
network using the minimum possible number of edges. In directed
networks you have to respect the edge’s direction, in weighted
networks you have to minimize (or maximize, depending on the
problem definition) the sum of the edge weights.

3. The most common algorithms to solve shortest path finding are

Dijkstra (if you have a fixed origin and destination) or Floyd-
Warshall (if you are calculating the shortest paths between all pairs
of nodes in the network).

4. Two important network connectivity measures are the diameter

and the average path length. The diameter is the length of the
longest shortest path. The average path length is the average
length of all shortest paths in the network.

5. A minimum spanning tree is a tree connecting all nodes in the

network which minimizes the sum of edge weights.
158 the atlas for the aspiring network scientist

10.7 Exercises

1. Label the nodes of the graph in Figure 10.12(a) in the order of

exploration of a BFS. Start from the node in the bottom right
corner.

2. Label the nodes of the graph in Figure 10.12(a) in the order of

exploration of a DFS. Start from the node in the bottom right
corner.

3. Calculate all shortest paths for the graph in Figure 10.12(a).

4. What’s the diameter of the graph in Figure 10.12(a)? What’s its

average path length?
11
Node Ranking

The most direct way to find the most important nodes in the network
is to look at the degree. The more friends a person has, the more
important she is. This way of measuring importance works well in
many cases, but can miss important information. What if there is a
person with only few friends, but placed in different communities –
just like in Figure 11.1? The removal of such person will create iso-
lated groups, which are now unable to talk to each other. Shouldn’t
this person be considered a key element in the social network, even
with her puny degree?

Figure 11.1: An example of a

social network in which the
degree does not necessarily con-
vey all the information about
node importance.

Many networks scientists agree that she should, and developed

different centrality measures accordingly. Here we focus on a few
examples.

11.1 Closeness
1
Alex Bavelas. A mathematical model
If we want to know the closeness centrality1 of a node v, first we for group structures. Human organization,
calculate all shortest paths starting from that node to every possible 7(3):16, 1948

destination in the network: P. Each of these paths Pvu has a length,

which is the number of edges you need to cross to get to your desti-
nation. Let’s call it | Pvu | – the length to go from v to u. We sum these
distances in a total distance measure: ∑u | Pvu |. We take the average
of this value by dividing it by the number of all possible destinations
u, which is the number of nodes in the network minus one (the ori-
gin): ∑u | Pvu |/(|V | − 1). Then, since the measure is called closeness,
160 the atlas for the aspiring network scientist

we don’t want to look at it directly. Closeness is the opposite of dis-

tance. So we actually want the opposite of what we just calculated,
or (|V | − 1)/ ∑u | Pvu |. If this looks familiar, that’s because it is. The
closeness centrality of v is nothing more than its inverse average path
length (see Section 10.3), or 1/APLv .

0.53 Figure 11.2: A sample network.

0.66 Node labels represent the close-
0.53 ness centrality value for the
0.72 0.53 node.

0 .8

0.66

0. 5 0.57

Let’s look at an example – in Figure 11.2. Let’s consider the node

in the bottom left, labeled with 0.5. That is its closeness centrality.
How do we get to that value? First, we start with the nodes directly
connected to it. The shortest paths to get to them is to follow the
direct connections, thus only one edge is crossed. Both neighbors
contribute | Pvu | = 1. Moving on, the two neighbors allow our v node
to access to four mode nodes. These four nodes require to cross an
additional edge, thus they contribute | Pvu | = 2. We are left with two
more nodes that require a third edge to be reached: | Pvu | = 3. So to
recap, the total distance of this node is 1 + 1 (the two direct neighbors)
+2 + 2 + 2 + 2 (the four nodes at distance two) +3 + 3 (the final two
nodes at distance three) = 16. We then take the average (16/(9 − 1))
and convert this into a closeness: (9 − 1)/16 = 0.5.
The advantage of closeness centrality is that it has a spatial intu-
ition: the closer you are on average to anybody, the more central you
are. Exactly like standing in the middle of a room makes you closer
on average to each member of the crowd in a party than standing in a
corner. Empirically, in the vast majority of networks I analyzed, close-
ness centrality is distributed on a classical bell shape, i.e. normally. If
you use closeness centrality, most of your nodes will have an average
importance. This is not realistic for many networks: we know that
degree distributions are very skewed – the vast majority of nodes are
unimportant, while only a few selected superstars take all the glory.
Why does closeness centrality behave so differently from the de-
gree? How can two nodes with very low degree – for instance equal
to one – have different closeness centrality values so that they end up
 node ranking 161

Figure 11.3: The closeness cen-

trality lottery. The blue and
green nodes are new to the net-
work and only have one edge
to attach. The green node is
lucky, and connects to a central
hub. The blue node is unlucky
and connects to a peripheral
node. Thus nodes with the very
same low degree end up with
distributing normally instead of on a skewed arrangement? One pos- radically different closeness
sible explanation is that edge creation is a lottery. The many nodes centrality values.
with degree equal to one that you have in broad degree distributions
can get lucky with their choice of neighbor. Sometimes, like in the
case of the green node in Figure 11.3, the neighbor is a hub. The
green node’s closeness centrality will then be high, because it is just
one extra hop away from the hub itself – which is very central. Some-
times the new node will attach itself to the periphery – like the blue
node in Figure 11.3 –, and thus have a very low closeness centrality.

11.2 Betweenness
2
Jac M Anthonisse. The rush in a
directed graph. Stichting Mathematisch
Network scientists developed betweenness centrality2 , 3
to fix some of Centrum. Mathematische Besliskunde, (BN
the issues of closeness centrality. Differently from closeness, with be- 9/71), 1971
tweenness we are not counting distances, but paths. We still calculate
3
Linton C Freeman, Douglas Roeder,
and Robert R Mulholland. Centrality in
all shortest paths between all possible pairs of origins and destina- social networks: Ii. experimental results.
tions. Then, if we want to know the betweenness of node v, we count Social networks, 2(2):119–141, 1979
the number of paths passing through v – but of which v is neither an
origin nor a destination. In other words, the number of times v is in
between an origin and a destination. If there is an alternative way of
equal distance to get from the origin to the destination that does not
use v, we discount the contribution of the path passing through v to
v’s betweenness centrality. I provide an example in Figure 11.4.
The total number of paths that can pass through a node – ex-
cluding the ones for which it is the origin or the destination – are
(|V | − 1)(|V | − 2) in a directed network, and (|V | − 1)(|V | − 2)/2 in
an undirected one.
One intuitive way to think about betweenness centrality is asking
yourself: how many paths would become longer if node v would
disappear from the network? How much is the network structure
dependent on v’s presence? Since real world networks have hubs
which are closer to most nodes, the shortest paths will use them
often. As a result, betweenness centrality distributes over many
162 the atlas for the aspiring network scientist

Figure 11.4: An example on

how to calculate the between-
ness centrality of the node
marked with the gray arrow.
The shortest paths passing
through it – in green – con-
tribute to its betweenness
centrality. If there are n alterna-
tive paths not passing through
the node, then the path con-
tributes only 1/n to the node’s
centrality – I show an example
orders of magnitude, just like the degree. Unlike the degree, it takes in blue.
into account more complex information than simply the number of
connections.
The concept underlying betweenness centrality can be extended to
go beyond nodes. You can use it to gauge the structural importance
of edges. The definition is the same: the betweenness of an edge is
the (normalized) count of shortest paths using the edge. If applied to
connections, we call this measure “edge betweenness”. This is a key
concept especially for the field of community discovery, as we will
find out in Part IX.

Figure 11.5: In this network the

edge thickness is proportional
to its edge betweenness value.
The node size is proportional to
the node betweenness.
Figure 11.5 shows an example of edge betweenness centrality.
The intuition here is the same: the edge betweenness is the number
of paths that would get longer if the edge were to disappear. Note,
though, that if we remove the edge from the network, almost all of
the edge betwenness centralities will have to be recalculated. It is
very hard to figure out the second order effects of the edge disappear-
ance on the shortest paths of the network. The edge betweenness of
some edges might increase by one, but it is not easy to understand
which ones.
In some cases – and actually this might be very likely – the net-
work will be broken up into multiple components, meaning that
no edge will increase its betweenness and, instead, many will lose
part of their centrality. That is because now there will be many node
pairs that cannot reach each other any more. Consider again Figure
11.5: once we remove one of the two most central edges, no node
 node ranking 163

in one clique can reach the nodes in the other one. All those paths
passing through the removed edge are lost forever. The surviving
edge between the two most central ones will have a much reduced
edge betweenness centrality: it cannot be used to move between
cliques any more. This consideration holds true not only for the edge
betweenness, but also for the node betweenness.
A relaxed version of betweenness centrality does not use shortest
paths, but random walks (Chapter 8). This simulates the spreading
of information into the network. The definition is similar: this “flow”
centrality is the number of random walker passing through the node 4
Mark EJ Newman. A measure
during the information spread event4 , 5 . Just like with the regular of betweenness centrality based on
betweenness centrality, also in this case you can take an edge-centric random walks. Social networks, 27(1):
39–54, 2005a
approach, and count the number of random walks going through a 5
Ulrik Brandes and Daniel Fleischer.
specific edge. This has been used, for instance, to solve the problem Centrality measures based on current
of community discovery6 , 7 . flow. In Annual symposium on theoretical
aspects of computer science, pages 533–544.
Springer, 2005
6
Santo Fortunato, Vito Latora, and
11.3 Reach Massimo Marchiori. Method to
find community structures based on
Reach centrality is only defined for directed networks. The local information centrality. Physical review E,
reach centrality of a node v is the fraction of nodes in a network that 70(5):056104, 2004
7
Vito Latora and Massimo Marchiori.
you can reach starting from v8 . From this definition, one can see why Vulnerability and protection of infras-
it doesn’t make much sense for undirected networks. If your network tructure networks. Physical Review E, 71
has a single connected component, then all nodes have the same (1):015103, 2005
8
Enys Mones, Lilla Vicsek, and Tamás
reach centrality, which is equal to one. That is also the case if your Vicsek. Hierarchy measure for complex
directed network has only one strongly connected component. In a networks. PloS one, 7(3):e33799, 2012
strongly connected component there are no “sinks” where paths get
trapped, thus every node can reach any other node.

Figure 11.6: A wheel graph

with a flipped edge. The central
node has the maximum reach
centrality.

However, when you have multiple (or no) strongly connected

components, reach centrality tells you how much you can command
in your network if you’re node v. Consider Figure 11.6. There is a
clear boss in this figure: the central node. Following its edges, you
can reach the entirety of the network. Thus its reach centrality is
equal to one. On the other hand, the node on the top right has an
out-degree of zero. You cannot reach anything from it, thus its reach
164 the atlas for the aspiring network scientist

centrality is zero. Reach centralities progressively increase if you

follow the wheel clockwise, as more and more of the network gets
reachable from the nodes’ perspective.
Calculating the reach centrality is trivial. You start from node v
and you explore the graph with a BFS strategy. Once you cannot ex-
plore any more, you stop. The number of nodes you touched divided
by the number of nodes in the network is your reach centrality. This
is linear in the number of edges in the graph.
Reach centrality is a key concept we use to detect the hierarchical
structure of networks, as we will see in Chapter 29.

11.4 Eigenvector

Betweenness centrality shares with closeness a drawback: compu-

tational complexity. Both measures require to calculate all shortest
paths in the network. For large structures, this becomes unfeasible.
The reason fully lies in the shortest paths calculation, which is very
computationally expensive. One could approximate the node’s impor-
tance for connectivity by looking not at shortest paths, but at random
walks.
This is different from the flow centrality I explained at the end of
Section 11.2 because, in this case, we’re not simulating a spreading
event. Instead, we’re looking at infinite length random walks and
we’re not bounded by origin-destination pairs of spreading events.
Here, we are interested in knowing the expected probability of end-
ing up in a node when we perform a random walk in the network.
That is, we start from a random node and we keep choosing to tra-
verse random edges. What’s the likelihood of ending up in node
v?
Calculating all shortest paths takes |V |3 operations. By using
clever linear algebra, running infinite length random walks could
take only |V |2 . In fact, we already saw how to calculate the proba-
bility of ending in a node after an infinite length random walk: it is
the stationary distribution, as I discussed in Section 8.1. By replacing
the expensive step at the basis of betweenness centrality with simple
9
John R Seeley. The net of recipro-
random walks, you can obtain phenomenal speedups. cal influence. a problem in treating
We call methods based on this technique “Eigenvector Centrali- sociometric data. Canadian Journal of
ties”, as the stationary distribution is the leading left eigenvector of Experimental Psychology, 3:234, 1949
10
Mark EJ Newman. Mathematics of
the stochastic adjacency matrix. If you take the straight up stationary networks. The new Palgrave dictionary of
distribution, you obtain what we call the eigenvector centrality9 , 10 . economics, pages 1–8, 2016b
11
Lawrence Page, Sergey Brin, Rajeev
Motwani, and Terry Winograd. The
PageRank pagerank citation ranking: Bringing
order to the web. Technical report,
By far, the most famous approach in this category is PageRank11 : the Stanford InfoLab, 1999
 node ranking 165

algorithm that Google invented in 1998 to rank webpages in their

nascent search engine. PageRank is nothing more than calculating a
stationary distribution over a directed adjacency matrix. PageRank
differs from eigenvector centrality in one tiny – but rather salient –
aspect.
If you remember Section 8.1 you’ll recall a small issue with the
stationary distribution. If your network is not connected, meaning
that it has more than one connected component (Section 7.4), you
will obtain multiple incomparable stationary distributions. This is
bad news for the use case of PageRank: you want to use it to sort out
webpages, and the users want to see a single ranking, not one per
connected component!

3 Figure 11.7: The practical im-

2
plementation of the PageRank’s
4
6 1 teleportation trick: adjacency
matrix (a) and resulting graph
9 (b).
5

8 7

(a) (b)

Google’s solution for the PageRank algorithm was to give the

walker a teleportation device. At each step, the walker has a minus-
cule chance to request a teleportation, which then might land it on
a different component. This mathematical trick is embarrassingly
easy to implement. It is equivalent to the creation of ghost edges with
very little weight connecting the entire graph. On matrix notation,
this is the same as adding a tiny constant to the (not yet normalized)
adjacency matrix: A∗ = A + e. Figure 11.7 shows this teleportation
trick in practice.
However, PageRank is not immune from downsides. PageRank is
very close to the degree. How closely the degree approximates the 12
Santo Fortunato, Marián Boguñá,
PageRank depends on the value of our teleportation parameter e: in Alessandro Flammini, and Filippo
Menczer. Approximating pagerank from
the literature, we call 1 − e the “damping factor”. The magic value of
in-degree. In International Workshop on
e is 0.15, that is what Brin and Page used originally. If we set e = 0, Algorithms and Models for the Web-Graph,
PageRank is equivalent to π, and therefore to the degree12 , 13 . pages 59–71. Springer, 2006
13
Gourab Ghoshal and Albert-László
Of course, nowadays Google uses a much more complex algorithm
Barabási. Ranking stability and super-
to sort the results. Most of the tricks are either secret or too special- stable nodes in complex networks.
ized to include here. However, there are a few tweaks of note. For Nature communications, 2:394, 2011
14
Taher H Haveliwala. Topic-sensitive
instance, a very popular variant of PageRank is the personalized
pagerank. In Proceedings of the 11th
PageRank14 . In practice, one can split the network to a multilayer international conference on World Wide
one depending on the topic of the hyperlink (e.g. its keywords) and Web, pages 517–526. ACM, 2002
166 the atlas for the aspiring network scientist

calculate a set of PageRanks, one per topic. There are other possible 15
Arda Halu, Raúl J Mondragón, Pietro
ways to define a multilayer PageRank15 . Panzarasa, and Ginestra Bianconi.
Multiplex pagerank. PloS one, 8(10):
e78293, 2013
Katz
16
Leo Katz. A new status index derived
Another popular variant of eigenvector centrality is Katz centrality16 . from sociometric analysis. Psychometrika,
At a philosophical level, the difference between the two is that Katz 18(1):39–43, 1953

says that nodes that are farther away from v should count less when
estimating v’s importance. So it matters whether v is reached at the
first step of the random walk, rather than at the second, or at the
hundredth. For eigenvector centrality when you meet v in a random
walk makes no difference, for Katz it does.
If we were to write the eigenvector centrality not as an eigenvector,
but as a sum, we would end up with something that looks a bit like
this:

∞
ECv = ∑ ∑ ( Ak )uv ,
k =1 u ∈V

which means that v’s importance is the sum of the probabilities

of getting from any u to v in k steps, with k going to infinity. Katz
simply adds a term, α, which is lower than one. He plugs it in the
formula as follows:

∞
KCv = ∑ ∑ αk ( Ak )uv .
k =1 u ∈V

Since 0 < α < 1, as k grows the contribution of ( Ak )uv becomes

more and more insignificant. Which is what Katz wants: longer
walks contribute less to v’s centrality.

UBIK
A paper of mine presents UBIK, which is the lovechild between Katz
17
Michele Coscia, Giulio Rossetti, Diego
Pennacchioli, Damiano Ceccarelli, and
centrality and the personalized PageRank I presented before17 . The Fosca Giannotti. You know because
weird acronym is short for “you (U) know Because I Know” and we i know: a multidimensional network
approach to human resources problem.
developed it with networks of professional in mind (like Linkedin). In Proceedings of the 2013 IEEE/ACM
Each professional has skills, which allow her to perform her job. International Conference on Advances in
However, sometimes she is required to do something using a skill she Social Networks Analysis and Mining,
pages 434–441. ACM, 2013b
doesn’t have. In many cases, she might be able to perform the task
anyway because she can ask for help in her social network. Think
about any time you asked a friend to fix something in your script, or
scrape some data, or patch a leaking water pipe.
Of course, if the task you need to perform requires only knowl-
edge you have, you can do it quickly. Every level of social interaction
you add will slow you down. If you’re a computer scientist you can
 node ranking 167

Skills THEY have

Figure 11.8: How UBIK works.
93
Each node (you) has different
4 Skills YOU get Skills YOU have
proficiency for different skills,
here we have three: writing
12
(red), statistical analysis (blue),
13 3
and finance (yellow). But what
Skills THEY have each node really can do is the
3 84 sum of what it knows plus a
77 combination of what its neigh-
8 7 22 bors know. You get from your
social network some skills, in
38 this case the sum of their skills
raised to the power of −1/l,
where l is the degrees of sep-
aration plus one. So, in this
case, the neighbors provide 13
think about this as memory layers. What you know is in your brain,
writing skill points, because
your cache: it is ready to run on your CPU. What your friends know
(93 + 77)1/2 ∼ 13.
is the main memory, the RAM. The main memory is slower than the
cache, because the data need to travel from the memory to the cache
before it can be used. Your friends’ friends are like a hard disk, and
the friends of the friends of your friends are the Internet: a limitless
amount of distributed information that is hard to search and collect.
Figure 11.8 shows a vignette of this process.
So in UBIK we take the stance that a person’s knowledge is the
sum of her own knowledge plus some combination of her social net-
works and, ultimately, of mankind. This lofty philosophical picture
boils down to just adding a few bells and whistles to Katz centrality.
First, we don’t use a simple graph, but a multilayer network. Dif-
ferent types of friends might have different levels of willingness or
reactivity when asked to help. A colleague is just down the corridor,
a close friend might want to do anything for you, that person you
dated once during college maybe will pick up the phone if you call.
So we have different adjacency matrices A with a different topol-
ogy and a different coefficient favoring or hampering the centrality
contribution.
Second, rather than giving a single centrality score to each node,
we have multiple. Each node gets a different score for each skill.
You might be a dragon when it comes to do multivariate regression
analysis, but unable to make yourself understood in an email. As a
consequence, the initial condition is also different. The skills aren’t
distributed equally in the network. The nodes don’t all start from
the same level in all skills. Each node has its own personal story, and
might start with higher scores in some skills and lower in others.
168 the atlas for the aspiring network scientist

Figure 11.9: The difference

in input between Katz and
UBIK centralities. The node’s
color determines its initial
condition: how much of a
skill/centrality it possesses (dif-
ferent colors represent different
skills/centralities). The edge’s
color determines its layer. Note
how all nodes and all edges in
Katz have the same color.
(a) Katz (b) UBIK

So the difference between UBIK and Katz is basically in the input

data. Where Katz works with a single layer network, a single central-
ity measure, and a uniform initial condition, UBIK uses a multilayer
network, with multiple centrality scores, initialized differently for
different nodes. Figure 11.9 depicts this difference. Then the process
is practically the same: direct neighbors have a big effect on your
centrality scores and, as you go to more and more degrees of sep-
aration, the contributions fade away to zero. Sure, UBIK has to do
this multiple times for each skill and needs the extra parameters to
distinguish between different layers but, at the end of the day, UBIK
is a glorified Katz centrality.
Where UBIK shines is in the analysis of so-called “expertise net-
works”: web-based communities of experts helping each other with 18
Jun Zhang, Mark S Ackerman, and
problems related to their professions18 , 19 . One could also use it to Lada Adamic. Expertise networks
investigate the question whether team formation is a process that in online communities: structure and
algorithms. In Proceedings of the 16th
happens better if it is organized from the top – like in organizations
international conference on World Wide
– or spontaneously from the bottom, like it happens for instance in Web, pages 221–230, 2007a
large open source software projects20 . 19
Lada A Adamic, Jun Zhang, Ey-
tan Bakshy, and Mark S Ackerman.
Knowledge sharing and yahoo answers:
everyone knows something. In Proceed-
Alpha ings of the 17th international conference on
World Wide Web, pages 665–674, 2008
Another variant of eigenvector centrality is Bonacich’s Alpha central- 20
Christian Bird, David Pattison,
ity. If Katz wanted to penalize long walks, Bonacich wants to add an Raissa D’Souza, Vladimir Filkov, and
Premkumar Devanbu. Latent social
external source of importance to the node’s centrality21 . Practically, structure in open source projects. In
we are saying that, to know how important a node is in a network, Proceedings of the 16th ACM SIGSOFT
International Symposium on Foundations
we don’t have to look exclusively at the topology of the network. The
of software engineering, pages 24–35, 2008
node might get its importance from somewhere else. A Web without 21
Phillip Bonacich and Paulette Lloyd.
Google would be poorer even if [Link] would not be the most Eigenvector-like measures of centrality
central node in the hyperlink network. for asymmetric relations. Social networks,
23(3):191–201, 2001
If, as we saw before, we can express the vanilla eigenvector central-
ity as an infinite sum:
 node ranking 169

∞
ECv = ∑ ∑ ( Ak )uv ,
k =1 u ∈V

then we can express Alpha centrality as the same sum, plus am

external source of non-network importance:

∞
ECv = (1 − α)ev + ∑ ∑ α( Ak )uv .
k =1 u ∈V

Differently from Katz, α doesn’t change as the length k increases:

it just regulates how much weight we give to the traditional part
of the eigenvector centrality. If α = 0, then 100% of the node’s
importance comes from the vector e. Each entry ev of e is the external
importance of node v. In the Web network, Google’s ev would be
through the roof. On the other hand, if α = 1, this reduces to the
classical eigenvector centrality.

11.5 HITS
22
Jon M Kleinberg, Ravi Kumar, Prab-
HITS22 , 23is an algorithm designed by Jon Kleinberg and collabora- hakar Raghavan, Sridhar Rajagopalan,
and Andrew S Tomkins. The web as
tors to estimate a node’s centrality in a directed network. It is part a graph: measurements, models, and
of the class of eigenvector centrality algorithms from Section 11.4, methods. In International Computing
and Combinatorics Conference, pages 1–17.
but it deserves its own section due to its interesting characteristics. Springer, 1999
Differently from other centrality measures, HITS assigns two values 23
Jon M Kleinberg. Authoritative sources
to each node. In fact, one can say that HITS assigns nodes to one of in a hyperlinked environment. Journal of
the ACM (JACM), 46(5):604–632, 1999
two roles – we will see more node roles in Chapter 12. The two roles
are “hubs” and “authorities”.

Figure 11.10: Hubs and authori-

ties in directed networks.

(a) Hub. (b) Authority.

In a sense, both hubs and authorities are central nodes in the

network. However, when you’re dealing with directed networks,
there are two ways in which a core member of a community can
play its role. A core member might be a person who maybe does not
know many things, but knows the people who know them. You will
go to this member with a question and she will point to someone who
knows the answer. We call such linking resource a “hub”. Figure
11.10(a) provides an illustration.
170 the atlas for the aspiring network scientist

The converse role of a hub is an authority. This is in principle

the exact opposite of a hub – although it’s possible for a node to
be partly a hub and partly an authority at the same time –: this
person might not know many people in the social circle, but she has
mastered her own topic of specialization. Everybody knows that,
and so she is pointed by everyone when someone asks about that
particular topic. This happens because she is an “authority” on the
subject. Figure 11.10(b) provides an illustration.
Hubs and authorities are an instance in which the quantitative
approach of the centrality measures and the qualitative approach
of the node roles meet. There is a way to estimate the degree of
“hubbiness” and “authoritativeness” in a network. This is what the
HITS algorithm does. The underlying principle is very simple. A
good hub is a hub that points to good authorities. A good authority
is an authority which is pointed by good hubs. These recursive
definitions can be solved iteratively – or, more efficiently, with clever
linear algebra – and they eventually converge.

1|1 .5|.25 .6|.36

1|1 0|1 0|1
1|1 .5|0 .6|0
1|1 1|1 1|.25 0|.75 1|.3 0|.88

1|1 .75|.75 .57|.88

1|1 1|0 .84|0

1|1 1|1 .25|.25 0|.75 .18|.2 0|.57

(a) 0th iteration. (b) 1st iteration (c) n-th iteration.

Figure 11.11: A sample pro-
Figure 11.11 shows the progress when calculating the measure.
gression of the HITS algorithm
Before the first iteration we assume that each node is equal. They
to estimate hub and authority
thus have all the same hub score and authority score, equal to one
scores. Node labels are their
– Figure 11.11(a). At each iteration, we sum all the incoming hub
authority (left) and hub (right)
scores of a node to determine its authority score. At the same time,
scores, separated by a pipe.
we sum all the outgoing authority scores of a node to obtain its
new hub score. We then normalize so that the maximum hub and
authority score is one. At the first iteration – Figure 11.11(b) – hub
and authority scores are equivalent to a normalized out- and in-
degree, respectively.
After a sufficient number of iterations – Figure 11.11(c) – the scores
stabilize. We can see that nodes with the same in-degree can have
different authority scores – the same holds for hub scores. Consider
the two nodes with in-degree four: one has the maximum score of
one, while the other has a score of 0.84. This is because the more
 node ranking 171

authoritative node has, on average, incoming connections from more

reputable hubs.
HITS is an important algorithm in the computer science portion of
the network analysis community. It was modified and extended in a 24
Tamara Kolda and Brett Bader. The
number of ways, notably to work on multilayer networks24 , enabling tophits model for higher-order web link
topic-dependent hub-authority scores. SALSA25 is also a related analysis. In Workshop on link analysis,
counterterrorism and security, volume 7,
method.
pages 26–29, 2006
25
Ronny Lempel and Shlomo Moran.
Salsa: the stochastic approach for link-
structure analysis. ACM Transactions
11.6 Harmonic on Information Systems (TOIS), 19(2):
131–160, 2001
PageRank solves the problem of networks with multiple connected
components. This is a common problem to have: all centrality mea-
sures based on shortest paths or random walks are ill defined when
your network has pairs of unreachable nodes. This includes close-
ness, betweenness, reach, ... practically everything. But PageRank
and its teleportation trick is not the only way to deal with multiple
components.
The crux of the issue is that you cannot compare the closeness
centrality of two nodes from different connected components, be-
cause one might have a higher closeness simply because its connected
component is smaller. One approach to fix this issue is to consider
the component size in the measure. We want the desirable property
Nan Lin. Foundations of social research.
26

of saying that a central node in a large component is more impor- McGraw-Hill Companies, 1976
tant than a central node in a smaller component. Lin’s centrality26
achieves this by multiplying the closeness centrality of a node by the
size of its connected component, which – incidentally – just means to
square the numerator:

LCv = (|Vv | − 1)2 / ∑ | Pvu |, 27

Massimo Marchiori and Vito Latora.
u∈Vv Harmony in the small-world. Physica A:
Statistical Mechanics and its Applications,
285(3-4):539–546, 2000
with Vv ⊆ V here being the set of nodes part of the component in 28
Yannick Rochat. Closeness centrality
which v resides. extended to unconnected graphs: The
Harmonic centrality represents another alternative which has harmonic centrality index. Technical
report, 2009
been discussed in many slight different variations and scenar- 29
Paolo Boldi and Sebastiano Vigna. Ax-
ios27 , 28 , 29 , 30 , 31 . In practice, you calculate the harmonic mean of ioms for centrality. Internet Mathematics,
all distances – even those between unreachable nodes. Thus: 10(3-4):222–262, 2014
30
Edith Cohen and Haim Kaplan.
Spatially-decaying aggregation over a
1 network. Journal of Computer and System
HCv = ∑ | Pvu | . Sciences, 73(3):265–288, 2007
u 31
Raj Kumar Pan and Jari Saramäki.
Path lengths, correlations, and centrality
The harmonic centrality handles unreachable nodes properly, in temporal networks. Physical Review E,
based on the assumption that 1/∞ = 0. 84(1):016105, 2011
172 the atlas for the aspiring network scientist

11.7 k-Core

When it comes to node centrality, one common term you’ll hear

thrown around is one of “core” node. This is usually a qualitative
distinction – see Chapter 28, but sometimes we need a quantitative
one. With k-core centrality we look for a way to say that a node
is “more core” than another. A k-core in a network is a subset of
its nodes in which all nodes have at least k connections to each 32
Stephen B Seidman. Network structure
other32 . A connected component of a network is always a 1-core: and minimum degree. Social networks, 5
each node in the component has at least one connection to the rest (3):269–287, 1983

of the component. In a 2-core, each node must have at least two

connections to the other nodes in the 2-core.
One can easily identify the k-core of a network via the k-core 33
Vladimir Batagelj and Matjaz Zaver-
decomposition algorithm33 . In Figure 11.12 we represent a stylized snik. An o (m) algorithm for cores
version of it. Figure 11.12(a) shows the original network. decomposition of networks. arXiv
preprint cs/0310049, 2003

3
3

3 3
3
2 2
3
2 2
1 1 2 1 2

1 1 1 1

(a) (b) (c) (d) (e)

Figure 11.12: The steps to de-
The first step is identifying the nodes with degree one. They termine the k-core value for
are labeled as part of the 1-core of the network, and removed from each node in the network. (a)
the structure. – Figure 11.12(b). We need to apply this step recur- The starting network. (b-e) The
sively: there could be nodes that originally had degree two, but now steps of the algorithm.
have lower degree because we removed one of their neighbors (or
both!). Also these nodes are part of the 1-core of the network – Figure
11.12(c).
Once the minimum degree in the network is higher than one we
can proceed to the next phase. In this phase we identify the nodes
that are part of the 2-core of the network. These are, unsurprisingly,
the nodes with degree two – and all nodes whose degree lowers to
two or less once we remove their neighbors during this step (Figure
11.12(d)). We continue the procedure to detect 3-, 4-, ..., k-cores until
there are no remaining nodes in the network – Figure 11.12(e).
Note that the k-core decomposition approach is only the most
famous among many similar which define a structure of interest and
 node ranking 173

use it to define a centrality measure. Among popular examples we 34

Christos Giatsidis, Dimitrios M
find D-cores34 (for directed networks), k-shells35 , k-coronas36 , and Thilikos, and Michalis Vazirgiannis.
more. D-cores: Measuring collaboration of
directed graphs based on degeneracy.
In Data Mining (ICDM), 2011 IEEE
11th International Conference on, pages
11.8 Centralization 201–210. IEEE, 2011
35
Shai Carmi, Shlomo Havlin, Scott
This chapter is all about knowing which nodes are central in a net- Kirkpatrick, Yuval Shavitt, and Eran
Shir. A model of internet topology using
work. So it is a node-centric chapter. To wrap it up, let’s change the
k-shell decomposition. Proceedings of
perspective a little. Let’s see how node centrality can say something the National Academy of Sciences, 104(27):
about your network as a whole. This would be the centralization 11150–11154, 2007
36
Alexander V Goltsev, Sergey N Doro-
of the network. A network is centralized when there is one node
govtsev, and Jose Ferreira F Mendes.
in it that is so much more central than everything else. Consider a k-core (bootstrap) percolation on com-
star, where one node is in the middle, it is connected to every other plex networks: Critical phenomena and
nonlocal effects. Physical Review E, 73(5):
node in the network and there are no other connections between its 056101, 2006
neighbors. A network cannot get more centralized that that37 . 37
Linton C Freeman. Centrality in social
Note that this is a meta-definition, because it rests on however networks conceptual clarification. Social
networks, 1(3):215–239, 1978
you define “centrality”. If you use closeness or betwenness centrality,
you’re going to get two different centralization values. The procedure
is always the same two steps. First, you sum the centrality differ-
ences between the most central node in the network and all other
nodes. Then you calculate what would be the largest theoretical sum
of differences in networks of comparable size. Usually the maxi-
mum is obtained by a star graph with the same number of nodes of
your original network. The ratio between the two is the degree of
centralization.
There are variants of centralization measures. For instance, you
can allow the graph to have multiple central points, not just the one
with the maximum centrality. Once you add this degree of freedom,
you can also ask yourself: given that the graph has multiple centers,
are these centers close together, or are they scattered far apart? In the
former case the graph is more centralized than in the latter.

11.9 Summary

1. We’ve seen many alternatives to the degree to estimate a node’s

importance. Many are based on shortest paths. The first measure
is closeness centrality, answering the questions: how far is on av-
erage a node from every other node in the network? Betweenness
centrality, instead, asks: how many shortest paths would become
longer if this node were to disappear?

2. Alternatively, you can look at random walks, since they’re less

computationally expensive to calculate. You can calculate a family
of eigenvector centralities, of which PageRank is one of the most
174 the atlas for the aspiring network scientist

famous examples.

3. HITS is another famous eigenvector centrality measure for di-

rected networks, which divides nodes in two classes: hubs, who
dominate out-degree centrality; and authorities, who dominate
in-degree centrality.

4. Harmonic centrality is a version of closeness centrality which

solves the issue of networks with multiple connected components.
In such networks, there are pairs of nodes that cannot reach each
other, thus other approaches based on shortest paths and random
walks wouldn’t work.

5. k-Core decomposition also works with networks with multiple

components. It recursively removes nodes from the network with
increasing degree thresholds. At iteration k, we say that surviving
nodes are part of the kth core.

6. Regardless of your centrality measure, you can estimate how

centralized your network is by comparing the highest observed
centrality with the theoretical maximum centrality of a network
with the same number of nodes: a star graph.

11.10 Exercises

1. Based on the paths you calculated for your answer in the previous
chapter, calculate the closeness centrality of the nodes in Figure
10.12(a).

2. Calculate the betweenness centrality of the nodes in Figure

10.12(a). Use to your advantage the fact that there is a bottle-
neck node which makes the calculation of the shortest paths easier.
Don’t forget to discount paths with alternative routes.

3. Calculate the reach centrality for the network in [Link]

[Link]/exercises/11/3/[Link]. Keep in mind that
the network is directed and should be loaded as such. What’s the
most central node? How does its reach centrality compare with the
average reach centrality of all nodes in the network?

4. What’s the most central node in the network used for the previous
exercise according to PageRank? How does PageRank compares
with the in-degree? (for instance, you could calculate the Spear-
man and/or Pearson correlation between the two)

5. Which is the most authoritative node in the network used for

the previous question? Which one is the best hub? Use the HITS
 node ranking 175

algorithm to motivate your answer (if using networkx, use the

scipy version of the algorithm).

6. Based on the paths you calculated for your answer in the previous
chapter, calculate the harmonic centrality of the nodes in Figure
10.12(a).

7. Calculate the k-core decomposition of the network in [Link]

[Link]/exercises/11/7/[Link]. What’s the highest
core number in the network? How many nodes are part of the
maximum core?

8. What’s the degree of centralization of the network used in the

previous question? Compare the answer you’d get by using, as
your centrality measure, the degree, closeness, and betweenness
centrality.
12
Node Roles

Not all nodes perform the same role in the network. Sometimes, the
differences between nodes can be estimated quantitatively. A person
is measurably more or less connected in a social network. That is
what we described in the previous chapter: if you can estimate the
importance of the node (number of connections, centrality, etc), you
do so by calculating the corresponding quantitative measure (degree,
betweenness, etc).
Sometimes you cannot put a number to what you’re trying to de-
scribe. What the person is doing in the social network does not have
a quantity, but a quality: she is playing a specific role, which does
not have a countable result. This could be explicitly represented in
your data as a node attribute (Section 4.5): for instance, in a corporate
network, nodes might be explicitly labeled as managers, executive,
technicians, etc.
If you don’t have explicit qualitative data, you might want to put
a label on the nodes based on the structural network data. Rather
than being a characteristic of the person by itself, the node role is
determined by her position in the network.
There are many ways to define node roles, dependent on the
aspect of the network you want to describe. The main split in the
literature is on the type of procedure you’re following: unsupervised
or supervised. In unsupervised role learning, no node in your data
has a role and you’re making up your own definition of roles de-
pending on what’s meaningful to you. This is the classical approach,
which I dissect in Sections 12.1 and 12.2. In supervised node learning,
you already have partially labeled data and you want to figure out
what are the underlying rules determining the node roles. This is the
theme of Section 12.3.
 node roles 177

12.1 Classic Node Classification

In this section we introduce the concept of node roles by picking

network communities as our focus, just to give an example. If your
focus is different, you will probably define different roles – for in- 1
Kathryn Cooper and Mauricio Bara-
stance, you could look at paths in a directed network1 . In fact there hona. Role-based similarity in directed
are countless centrality measures developed to identify specific node networks. arXiv preprint arXiv:1012.2726,
2010
roles in complex networks2 . 2
Stephen P Borgatti and Martin G
Let’s consider the case of social circles. A social circle is a group of Everett. A graph-theoretic perspective
people interacting with each other because of shared interests and/or on centrality. Social networks, 28(4):
466–484, 2006
characteristics. We can say that Figure 12.1 shows two connected
social circles. The communities are not completely homogeneous:
they have structure. They have a boundary, members that are more
or less central, and outsiders connecting to them. We could define
four roles in this network: brokers, gatekeepers, core, and periphery
(the latter two not to be confused with the core-periphery mesoscale
structure that we will see in Chapter 28).

Figure 12.1: Two hypothetical

social circles: groups of nodes
connected to each other on the
left and on the right, with an
intermediary in the middle. Are
the nodes highlighted by the
gray arrows all performing the
same “role” in the network?

Broker. Suppose we have two social circles. If the two communi-

ties do not share any member it means that they cannot communicate.
However, sometimes you have people who are not part of either
community – because they only have few connections to each of
them – but they still have friends in both. These nodes can enable
communication to happen between the communities, and so they are
performing the role of information brokers. Figure 12.2(a) provides
an illustration.
Gatekeeper. It is rare for a social circle to be completely isolated
from the rest of society. Some of its members still have connections
with people outside the community. If they do, they are managing
how the community relates to society: both in the flow of information
getting inside the community from the outside, and in what the
community sends outside. These nodes are the gatekeepers. Figure
178 the atlas for the aspiring network scientist

Figure 12.2: (a) An example of

a broker. Color indicates the
membership to a social circle.
The red node isn’t part of the
two social circles it connects,
so it brokers information be-
tween them. (b) An example of
(a) (b) a gatekeeper. The blue person is
not part of the red community.
12.2(b) provides an illustration.
The member of the community
Core. Some members of the community have a more central role
to which it connects is man-
than others. They connect exclusively with other members of the
aging the information access
community, without establishing relations with outsiders. They also
to the community. Thus, it is
have many connections. They are the heart of the social circle, and
gatekeeping it.
thus composing its core. Figure 12.3 provides an illustration.
Periphery. The other side of the coin of core members. A periph-
eral member does not have many connections in the community.
Differently from brokers and gatekeepers, this is not because they
also have connections to the external world. They just do not have
many relations, and all they have are in their own community. They
are thus peripheral to it. Figure 12.3 provides an illustration.

Figure 12.3: Core and periphery

nodes in a community. The
red element of the social circle
is very embedded in it: she is
a core member. On the other
hand, the purple person only
has two friends in the social
circle and no other relation. She
is in the periphery.

3
Keith Henderson, Brian Gallagher,
Rolx3 is one of the best known computer science approaches for Tina Eliassi-Rad, Hanghang Tong,
the extraction of node roles in complex networks. The way it works Sugato Basu, Leman Akoglu, Danai
Koutra, Christos Faloutsos, and Lei
is by representing nodes as vectors of attributes. Attributes can be,
Li. Rolx: structural role extraction &
for instance, the degree, the local clustering, betweenness centrality, mining in large graphs. In Proceedings
and so on. In practice, you decide which node features are relevant of the 18th ACM SIGKDD international
conference on Knowledge discovery and
to determine the roles your nodes should be playing in the network. data mining, pages 1231–1239. ACM,
This means that, selecting the right set of features, you can recover all 2012
the roles I discussed so far – core, periphery, broker, gatekeeper.
Rolx works in the way you would expect from a standard machine
 node roles 179

learning framework. The features are represented in a space where

redundancies are eliminated, for instance by running principal
component analysis (Section 5.4). This space is then fed to a classifier,
which tries to find the salient differences between different vector
prototypes. These classes are the different roles a node can play.

Figure 12.4: An example of Rolx

output. Node color encodes the
node’s role.

Figure 12.4 shows an example of Rolx’s output. The network

represents co-authorship in a scientific network. Nodes are scientists
connected if they collaborated on a paper. Rolx is able to find the
different roles played by different authors. Specifically, authors found
four roles of interest:

• Bridges (red): these are the hubs keeping the network together;

• Tightly knit (blue): these are the authors who have a reliable group
of co-authors, and are usually embedded in cliques;

• Pathy (green): authors who are part of long stretches;

• Mainstreram (purple): everything else.

Note that, with Rolx, you can also estimate how much each role
tends to connect with nodes in a similar role. For instance, by their
very nature, bridges tend to connect to nodes with different roles,
while tightly knit nodes band together. This is related to the concepts
of homophily and disassortativity, which we’ll explore in Chapter 26.

12.2 Node Similarity

Structural Equivalence
When two nodes have the same role in a network they are, in a sense,
similar to each other. Researchers have explored this observation
and derived measures of “node similarity”. We can also call this
180 the atlas for the aspiring network scientist

“Structural Equivalence”, as two nodes with similar roles in similar

areas of the network are keeping the network together in the same
way. In fact, structural equivalence is the stricter test of node similarity,
which we can relax to obtain alternative measures.
In this part of the book we have assumed a structural view of
nodes, unless otherwise specified. What this means is that, for
betweenness centrality or the k-core algorithm, nodes don’t have
metadata, or internal statuses, or attributes. The only way to tell
the difference between one node and another is by looking at their
degrees and the nodes they connect to.

1 1
Figure 12.5: (a) An example
of two structurally equivalent
nodes (nodes 1 and 2). (b) Here,
3 4 5 6 3 4 5 6 nodes 1 and 2 are not struc-
turally equivalent, because
node 1 has a neighbor that
2 2 node 2 does not.

(a) (b)
This is important to point out in this section, because it helps
understanding the definition of structural equivalence. For two nodes
to be structurally equivalent they have to be connected to the same 4
Robert A Hanneman and Mark Riddle.
neighbors4 . If they do, they are indistinguishable from one another, Introduction to social network methods.
therefore they cannot be any more similar. Consider Figure 12.5(a): 2005

nodes 1 and 2 have the same neighbors and no other additional one.
If I were to flip their IDs, you would not be able to tell. There is
no extra information for you to do so, because all you have is their
neighbor set.
On the other hand, we can tell the difference between nodes 1
and 2 in Figure 12.5(b). That is because we know that node 1 also
connects to node 3, which node 2 does not. So the two nodes are not
structurally equivalent. You can use any vector similarity measure
to estimate structural equivalence. For instance, you can calculate
the Jaccard similarity of the neighbor sets between u and v. In Figure
12.5(b), nodes 1 and 2 have three common neighbors out of four
possible, thus their structural equivalence is 0.75.
Alternatively, one could use cosine similarity, Pearson correlation
coefficients, or inverse Euclidean distance. In all these cases, you
have to transform the neighbor set into a numerical vector. If you sort
the nodes consistently, each node can be represented as a vector of
zeros and ones. Zeros correspond to nodes not connected to u, while
ones are u’s neighbors. These are the rows in the adjacency matrix
corresponding to the nodes, as Figure 12.6 shows. You can input the
 node roles 181

1 Node 1 Node 2 Figure 12.6: (a) A simple graph.

0 0 (b) The adjacency vector repre-
0 0 sentations of node 1 and node
3 4 5 6 1 0 2.
1 1
1 1
2
1 1
(b)
(a)

vectors corresponding to u and v to any of the mentioned measures

and obtain their structural equivalence. For instance, the Pearson
correlation coefficient of nodes 1 and 2 in Figure 12.6 is around 0.7.

Automorphic Equivalence
Automorphic equivalence is a more relaxed version of structural equiv-
alence. To understand it, we need to introduce the concepts of iso-
morphism and automorphism. We call two graphs “isomorphic” if they
have the same topology: the graph in Figures 12.5(a) and 12.5(b)
would be isomorphic if you were to add an edge in Figure 12.5(b)
between nodes 2 and 3. As a mnemonic trick: “iso” = same, and
“morph” = “shape” – two isomorphic graphs have the same shape.
We’ll see how to determine whether two graphs are isomorphic in
Section 39.2.
A graph is “automorphic” if it is isomorphic with itself. This
means that you can shuffle all node IDs of your graph such that you
preserve the neighborhoods of all nodes. If node 1 was connected
only to nodes 2 and 3 in G, you can only swap its ID with another
node that only has two connections and those connections lead to
nodes that have swapped their IDs with nodes 2 and 3. The graph
in Figure 12.7(a) is automorphic because we can swap around labels
respecting this rule – as I do in Figure 12.7(b).
So, for automorphic equivalence, two nodes are equivalent if you

1 2

3 4 5 6 6 4 5 3
Figure 12.7: An example of rela-
beling of the graph to highlight
2 1 the automorphic equivalence
between nodes 1 and 2.
(a) (b)
182 the atlas for the aspiring network scientist

can perform this re-labeling. To understand what this means consider

nodes 1 and 2 in Figure 12.6(b). They are NOT structurally equivalent
because, if we swap their labels, there is no further relabeling we can
do to render the graph isomorphic. We’re not allow to swap nodes 3
and 6, because they have a different number of neighbors. In Figure
12.7(a), instead, nodes 1 and 2 are automorphically equivalent. We
can swap their labels, which forces us to swap node 3 and 6’s labels
too. The resulting graph, in Figure 12.7(b), is identical to the original.
In this case, nodes 1 and 2 are not structurally equivalent, because
they both have neighbors that the other node doesn’t have, but
they are automorphically equivalent, because you can perform
the re-labeling. Every structurally equivalent pair of nodes is also
automorphically equivalent, but two automorphically equivalent
nodes might not be structurally equivalent.
In an automorphic graph, all nodes are by definition automorphic
equivalent. In non-automorphic graphs, some nodes can still be auto-
morphic equivalent if you can perform a local relabeling. You could
imagine Figure 12.7(a) to be embedded in a larger non-automorphic
graph, but that would not affect the automorphism between nodes 1
and 2.
While structural equivalence focused on nodes that had literally
the same neighbors in the network, automorphic equivalence focuses
on nodes that belong to the same structure type, no matter who their
actual neighbors are.

Regular Equivalence

The most relaxed variant of node similarity is regular equivalence. In

regular equivalence, nodes can be equivalent to each other if they
have connections to equivalent nodes. This is most easily understood
in hierarchies. Consider Figure 12.8. In the figure, we can find three
equivalence classes containing, respectively: {1}, {2, 3}, and {4, 5, 6}.
The third class is defined by those nodes connected to nodes of class
two but without connections to class one. Class two connects to both

2 3
Figure 12.8: A graph with three

4 5 6
regularly equivalent classes of
nodes.
 node roles 183

class one and class three, even if the number of connections to the
members of class three can vary. Class one is the mirror of class three:
it connects to nodes in class two, but to no node in class three.
To sum up the difference between structural, automorphic, and
regular equivalence, consider familial bonds. Two women with the
same husband and the same children are structurally equivalent: they
have the same relationships with the same people (in fact, they would
be the same person – although this is not a necessary requirement
for structural equivalence). Two women can be automorphically
equivalent if they have the same number of husbands and the same
number of children. Finally, to be regularly equivalent to a married
woman with children you have to be a married woman with children,
even if you have a different number of relations – the woman with
fewer husbands will definitely lead a less frustrating life.
There are many other measures of node similarity. Researchers
usually define new ones to better solve a problem called “link predic-
tion”, under the assumption that two similar nodes are more likely
to connect to each other. We will see more node similarity measures
than you want to know in Chapter 20. Node similarity can be used to
estimate network similarity, which is the topic of Chapter 41.

12.3 Node Embeddings

So far we’ve been pretty rigid in the way we wanted to classify nodes
into roles. Either we explicitly defined the roles with strict rules, or
we adopted the similarity approach, finding which node plays a
similar structural role to which other node. This is a sort of “zero-
dimensional” approach, where everything collapses in a single label.
One could use instead node embeddings, which determine the role
of a node with a vector of numbers and then classifies the node with
it. Recently, the most common way to discover such roles has become
the use of graph convolutional techniques. These techniques are not
the only way to go about solving this task, but I’ll focus on them for
the remaining of this chapter.
As an initial extreme simplification, graph convolutional uses
machine learning techniques – and, specifically, neural networks – 5
Jie Zhou, Ganqu Cui, Zhengyan
to learn a function classifying nodes5 , 6 . Graph convolutional is a Zhang, Cheng Yang, Zhiyuan Liu, and
supervised method, meaning that you already have a set of nodes Maosong Sun. Graph neural networks:
A review of methods and applications.
for which the label is known and you’re trying to infer the function arXiv preprint arXiv:1812.08434, 2018
behind this label assignment. In this way, you can assign a role to the 6
Zonghan Wu, Shirui Pan, Fengwen
nodes for which you don’t know the label yet. Chen, Guodong Long, Chengqi Zhang,
and Philip S Yu. A comprehensive
This should not be confused with graph embedding techniques, survey on graph neural networks. arXiv
which perform a similar task, but to learn graph embeddings. The preprint arXiv:1901.00596, 2019
difference between the two is that graph embeddings are then used
184 the atlas for the aspiring network scientist

as an input for the machine learning task, while node embeddings

provide you the output. Even more dumbed down: node embeddings
return you a label – the node role – while graph embedding returns
you the vector – a complex node representation – that is the basis
on which you’ll learn whatever it is that you want to learn about the
node – or the graph. I’m going to explore graph embedding in the
graph mining part, in Chapter 37.

Figure 12.9: Different neural

networks. The node color de-
termines the layer type: input
(red), hidden (blue), output
(green).

(a) Feedforward. (b) Recurrent. (c) Modular.

I’m going to discuss only the grandfather of modern graph convo- 7

Marco Gori, Gabriele Monfardini,
and Franco Scarselli. A new model
lutional techniques7 , just to give you the flavor of the approach. To
for learning in graph domains. In
understand what graph convolutional means, one needs to start from Proceedings. 2005 IEEE International Joint
the basics of neural network learning. I already mentioned what a Conference on Neural Networks, 2005.,
volume 2, pages 729–734. IEEE, 2005
neural network is (Section 4.6), a network with three classes of nodes:
input, hidden, and output. The data is a specific weighted activation
of the input nodes, it is processed by activating the hidden nodes,
and finally end up activating the output node(s), which is the answer
you’re looking for – hopefully! I reproduce in Figure 12.9 the general
schema.
Now, what’s the obvious problem in Figure 12.9? The input nodes
are not connected to each other. Specifically, in traditional neural
networks, the input is a vector. Thus there is no explicit structural
relationship between the elements in the vector – in fact, learning
such structure could be considered the whole point of performing
machine learning in the first place. The revolutionary idea of graph
convolutional networks is to allow the input to be the graph itself,
rather than reducing each node to an entry into a monodimensional
vector.
Figure 12.10 shows a general schema for graph convolutional net-
works. We start from some training data. This could be the graph
itself with some nodes labeled and some not, or a graph or a collec-
tion of graphs with labeled nodes. We pass this graph as the input to
the hidden layers. The role of the hidden layers is to learn a function
f that can explain why each node has a specific label/value. Once f
is learned, you will obtain the labels of the non-classified nodes, or
you’ll be able to classify a new, previously unseen, graph.
 node roles 185

Figure 12.10: A general schema

? for graph convolutional learn-
ing. The gray nodes in the
? Learning f
input network are unclassified.
By learning the function f be-
? hind the classification of nodes,
we can classify the rest of the
Input Layer Hidden Layers Output Layer
network (yellow outline in the
output layer).

Typically, f assumes that each node can be represented by a vector, 8

Joan Bruna, Wojciech Zaremba, Arthur
called state. But there are many different ways to construct different Szlam, and Yann Lecun. Spectral net-
works and locally connected networks
f s – and to learn them. The main split is between spectral and spatial on graphs. In ICLR, 2014
methods. 9
Michaël Defferrard, Xavier Bresson,
In spectral methods8 , 9 , 10 , one uses the graph spectrum (read: and Pierre Vandergheynst. Convolu-
tional neural networks on graphs with
eigenvectors of the Laplacian) as “filters”: we see the graph as a fast localized spectral filtering. In NIPS,
whole as the processor of a signal which determines the activation of pages 3844–3852, 2016
the nodes11 and – if we can reconstruct the signal – we can also know 10
Thomas Kipf and Max Welling. Semi-
supervised classification with graph
which value each unclassified node should have. One disadvantage convolutional networks. In ICLR, 2017
of these approaches is that calculating the spectrum of a graph isn’t 11
David Shuman, Sunil Narang, Pascal
easy, and you need to process the entire graph to determine the Frossard, Antonio Ortega, and Pierre
Vandergheynst. The emerging field of
status of a node. signal processing on graphs: Extending
In spatial methods12 , 13 , 14 , f is a sort of message-passing func- high-dimensional data analysis to
networks and other irregular domains.
tion. Each node wants to determine its own status by looking at the
IEEE Signal Processing Magazine, 3(30):
statuses of all its neighbors. Thus, these methods are heavily influ- 83–98, 2013
enced by the topology of the network. One advantage is that the 12
Franco Scarselli, Marco Gori,
Ah Chung Tsoi, Markus Hagenbuchner,
information to determine the status of a node is essentially local, thus
and Gabriele Monfardini. The graph
you don’t need to know the entire topology of the graph to learn a neural network model. IEEE Transactions
specific node’s label. on Neural Networks, 20(1):61–80, 2008
13
Kyunghyun Cho, Bart van Mer-
Notable examples are: GraphSage15 , which samples information
rienboer, Caglar Gulcehre, Dzmitry
from the neighbors and then aggregates it (thus the target node Bahdanau, Fethi Bougares, Holger
“pulls” information from some of its neighbors, the closer they are Schwenk, and Yoshua Bengio. Learn-
ing phrase representations using rnn
the more information they send); DCNN16 , which uses a diffusion encoder–decoder for statistical machine
process (which is more akin as “pushing” the information out to the translation. In EMNLP, pages 1724–1734,
2014
targets, so that similar information add to each other); and PATCHY- 14
Justin Gilmer, Samuel S Schoenholz,
SAN17 , which transforms the graph into a grid. Patrick F Riley, Oriol Vinyals, and
But modifying the way you learn f is not the only possible variant. George E Dahl. Neural message passing
for quantum chemistry. In ICML, pages
You could also modify the input and output layers, to change the
1263–1272. JMLR. org, 2017
task itself. For instance, you could try to predict spatial-temporal 15
Will Hamilton, Zhitao Ying, and Jure
networks18 , 19 , 20 , 21 : by having as input a dynamic network with Leskovec. Inductive representation
learning on large graphs. In NIPS, pages
changing node states, you could predict a future node state. Figure
1024–1034, 2017
12.11 shows a simplified schema for the task. Think, for instance, 16
James Atwood and Don Towsley.
about traffic load: given the way traffic evolves, you want to be able Diffusion-convolutional neural net-
works. In NIPS, pages 1993–2001,
to predict how many cars will hit a specific road straight (edge) or
2016
186 the atlas for the aspiring network scientist

intersection (node). 17
Mathias Niepert, Mathias Ahmed,
Other possible applications are the generation of a realistic net- and Konstantin Kutzkov. Learning
convolutional neural networks for
work topology (Chapter 15), the prediction of a link (Chapter 20), or graphs. In ICML, pages 2014–2023, 2016
summarizing the graph (Chapter 38). Given that these are not related 18
Yaguang Li, Rose Yu, Cyrus Shahabi,
to node roles, I’ll deal with such applications in the proper chapters. and Yan Liu. Diffusion convolutional
recurrent neural network: Data-driven
traffic forecasting. arXiv preprint
1,2,1,2,1,? 1,2,1,2,1,2 arXiv:1707.01926, 2017b
Figure 12.11: A schema for
spatial-temporal neural net-
works. We have an activation
timeline for each node (here
Learning f
showing only one). The task is
predicting the activation state in
the next timestep.
19
Bing Yu, Haoteng Yin, and Zhanxing
Input Layer Hidden Layers Output Layer Zhu. Spatio-temporal graph convo-
lutional networks: a deep learning
framework for traffic forecasting. In
Moreover, graph neural network techniques are not necessar- IJCAI, pages 3634–3640. AAAI Press,
ily limited to the convolutional approach. For instance, different 2018
20
Sijie Yan, Yuanjun Xiong, and Dahua
approaches have been used also to solve the graph isomorphism Lin. Spatial temporal graph convo-
problem22 , the problem of deciding whether two graphs are the same lutional networks for skeleton-based
action recognition. In Thirty-Second
graph – we’ll see more about this in Chapter 39.
AAAI Conference on Artificial Intelligence,
2018
21
Ashesh Jain, Amir R Zamir, Sil-
12.4 Summary vio Savarese, and Ashutosh Saxena.
Structural-rnn: Deep learning on spatio-
temporal graphs. In Proceedings of the
1. Going beyond node centrality, we can attach to nodes qualitative IEEE Conference on Computer Vision and
roles, rather than quantitative estimations of their importance. Pattern Recognition, pages 5308–5317,
These qualitative roles are dependent on the node’s position in 2016
22
Christopher Morris, Martin Ritzert,
the network topology. Traditionally, this is an “unsupervised” Matthias Fey, William L Hamilton,
learning task, in which you don’t know any node role and you’re Jan Eric Lenssen, Gaurav Rattan, and
Martin Grohe. Weisfeiler and leman
substantially inventing your own definition.
go neural: Higher-order graph neural
networks. In Proceedings of the AAAI
2. There are many ways to define roles, some well known are brokers Conference on Artificial Intelligence,
– in between communities –, gatekeepers – on the border of a volume 33, pages 4602–4609, 2019

community –, and more. A popular algorithm to detect node roles

is Rolx.

3. We can detect nodes playing the same role in a topology by esti-

mating their structural similarity: their tendency of connecting to
the same set of neighbors.

4. Node role could also be a supervised learning problem, where

you have some node roles in your data and you want to discover
the latent rules that determine them. Graph convolutional net-
works are a popular way to do so. In this technique, we don’t have
 node roles 187

(necessarily) a definition of what the role is, but some already

labeled data on which we can train the algorithm.

12.5 Exercises

1. For the network at [Link]

1/[Link], I precomputed communities ([Link]
eu/exercises/12/1/[Link]). Use betweenness centrality to
distinguish between brokers (high centrality nodes equally con-
necting to different communities) and gatekeepers (high centrality
nodes connecting with different communities but preferring their
own).

2. Use the network from the previous question to distinguish be-

tween core community nodes (high degree nodes with all their
connections going to members of their own community) and
peripheral community nodes (low degree nodes with all their
connections going to members of their own community).

3. Calculate the structural equivalence of all pairs of nodes from the

network used in the previous question. Which two nodes are the
most similar? (Note: there could be ties)
 Part IV

Synthetic Graph Models

13
Random Graphs

To explore the properties of a network – the degree distribution, the

clustering, the centrality of its nodes – there is one fundamental re-
quirement. You have to have a network. If you don’t have a network, 1
Just like searching in a dark room
you’re going to look pretty silly when you try to analyze it1 . At the for a black cat that isn’t there (https:
very beginning of network analysis, there was a widespread lack of //[Link]/wiki/Black_cat_
analogy).
data. Thus, some of the most brilliant mathematical minds in the
field determined different ways to create synthetic network data by
defining network models. These models are the subject of this part of
the book.
There are fundamentally three reasons to generate synthetic data
today. The first is explanatory in nature. After you analyze a bunch
of real world networks, you may realize they all seem to have a com-
mon property. Maybe they have a broad degree distribution (Section
6.3), incredibly high clustering (Section 9.2), or they contain commu-
nities (Part IX). And you ask yourself: how did these properties arise?
You might want to apply very simple rules, to see if they can repro-
duce the property of interest. If you succeed, you might be closer to
explain their origins. This is what I explore in Chapter 14.
The second reason is to have a way to test your algorithms and
analyses. If that’s your aim, you want to generate fake networks that
are as similar as possible to real world networks. They must have the
same properties, especially the ones that are important for testing
your method. This is what I explore in Chapter 15.
The third and final class focuses on description. As I just said,
you might find a network with a peculiar property. You might ask
yourself not how this property arose, but if it is something you’d ex-
pect any network to have, given other characteristics on its topology.
In other words, you want to estimate the statistical significance of
that observation. It’s pretty hard to talk about statistical significance
when you have a single observation – a single network. So you might
want to generate random networks with the same properties of your
observed one and see if they all have that characteristic of interest.
190 the atlas for the aspiring network scientist

Chapter 16 is all about this task.

13.1 Building Random Graphs

Before diving deep into these more complex models, I need to spend
some time with the grandfather of all graph models. It is the family
of network generating processes created by Paul Erdős and Alfréd 2
P Erdős and A Rényi. On random
Rényi in their seminal set of papers2 , 3 , 4 (some credit goes also to graphs. Publicationes Mathematicae
Gilbert5 for a few variants of the model). Debrecen, 6:290–297, 1959
These are simply known colloquially as “Random graphs”. I
3
Paul Erdos and Alfréd Rényi. On the
evolution of random graphs. Publ. Math.
can divide them fundamentally in two categories: Gn,p and Gn,m Inst. Hung. Acad. Sci, 5(1):17–60, 1960
models. The way they work is slightly different, but their results 4
Paul Erdos and Alfred Renyi. On
are mathematically equivalent. The difference is there simply for random matrices. Magyar Tud. Akad. Mat.
Kutató Int. Közl, 8(455-461):1964, 1964
convenience in what you want to fix: Gn,p allows you to define the 5
Edgar N Gilbert. Random graphs. The
probability p that two random nodes will connect, while Gn,m allows Annals of Mathematical Statistics, 30(4):
to fix the number of edges in your final network, m. 1141–1144, 1959

Ok, but... what is a random graph? We’re all familiar to the con-
cept of random number. You toss a die, the result is random. But
what does “random” mean in the context of a graph? What’s ran-
domized here? For this chapter, I will answer these questions assum-
ing uncorrelated random graphs – meaning that you can mentally
replace “random” with “statistical independence”. This is not strictly
speaking necessary: in the same way that you can study the statistics
of correlated coins, you can study correlated random graphs. How-
ever, that would make for a nasty math, and it isn’t super useful for
the aim of this book.

Let me Figure 13.1: In a social setting,

introduce you
to my friend! friends introduce each other,
thus edges are correlated: hav-
ing a common friend increases
the chance of being connected
(see example on the left). In
random graphs, edges are in-
dependent: blindfolded people
establish the connections, as in
the example on the right.

For network scientists, “random” applies to the edges. In a social

setting, connections are not independent: it is more likely for you to
know people your friends know, because they can introduce you. In
random graphs, this is strictly forbidden, as I show in the vignette
in Figure 13.1. If you are getting into a random graph and deciding
where to put your new connection, you’re going in completely blind.
 random graphs 191

This means that you don’t know anything about the connections that
are already there.
If you’re tired to read a book, this is the perfect occasion for a
physical exercise. Here’s a process you can follow to make your
own random network. First, take a box of buttons and pour it on the
floor. Yup, you heard me: just make a mess. Then take a yarn, cut
some strings, and drop them on the buttons. The buttons are now
your nodes, and the strings the edges. Congratulations! You have a
random network! Time to calculate!

Figure 13.2: How a Gn,m model

works: the n and m parameters
determine the box from which
you will extract your graph.
The box contains all possible
graphs with n nodes and m
The Big Box edges.
of Graphs with
8 Nodes and
12 edges

Less facetiously, this process can illustrate clearly the distinction

between Gn,p and Gn,m models. In Gn,m you first fix the characteristics
of the graph you want. You decide first how many nodes the graph
should have – which is the n parameter –, say 8. This is the number
of buttons. Then you fix the number of edges it should have – which
is the m parameter –, say 12. This is the number of yarn strings you
cut out. With those in mind, you go and take all possible graphs with
n nodes and m edges, and you pick one at random. All the graphs,
by the power of having the right number of nodes and edges, are
equally likely to be the result of our operation. Alternatively, you
could simply extract m random node pairs and connect them. The
result will be the same.
In the Gn,p variation we still say how many nodes we want: n.
However, rather than saying how many edges we want, we just
decide what’s the probability that two nodes are connected to each
other: p. It can be fifty-fifty, a coin toss. Then we consider all possible
pairs of nodes, and for each one we toss the coin. If it lands heads
we connect the nodes, if it lands tails we do not. Gn,p is the perfect
example of what we mean by “random graph”. Given a pair of nodes
we toss the coin and if it lands on heads we connect them. Another
pair, same procedure. The two tosses are independent: the fact that
one landed on heads does not influence the other. The coin is also the
same, so each edge has an equal probability to appear.
192 the atlas for the aspiring network scientist

1 2 1 3 Figure 13.3: How a Gn,p model

works: the p parameter deter-
Independent tosses mines whether a node pair con-
nects. The coin is not loaded,
so it always has the same prob-
ability of landing on heads.
The tosses are independent of
Same coin each other, so the result of one
doesn’t affect the result of the
1 2 1 3 other.

Figure 13.3 depicts the process. Note that throwing coins for each
node pair isn’t exactly the most efficient way to go about generating a
Gn,p – although I invite you to try. A few smart folks determined an 6
Vladimir Batagelj and Ulrik Brandes.
algorithm to generate Gn,p efficiently6 . Efficient generation of large random
Since Gn,m and Gn,p generate graphs with the same properties, it networks. Physical Review E, 71(3):036113,
2005
means that p and m must be related. Since the graphs have n nodes,
we can derive the number of edges (m) from p. p is applied to each
pair of nodes independently. We know how many pairs of nodes the
graph has, which is n(n − 1)/2. Thus we have an easy equation to
derive m from p: the number of edges is the probability of connecting
any random node pair times the number of possible node pairs, or

n ( n − 1)
p = m.
2

This is useful if you use Gn,p but you want to have, more or less,
control on how many edges you’re going to end up with.
By the way this gives you an idea of what’s the typical density
of a random Gn,p graph. The density (see Section 9.1) is the number
of links over the total possible number of links. We just saw that
n ( n − 1)
the number of links in a random graph is p and the total
2
n ( n − 1)
possible number of links is . One divided by the other gives
2
you p. So, if you want to reproduce the sparseness of real world
networks, you can do that at will. Just tune the p parameter to be
exactly the density you want.
In the following sections we explore each property of interest of
random graphs, to see when they model the properties of real world
networks well, and when they don’t. The latter is the starting point
of practically any subsequent graph model developed after Erdős and
Rényi.
 random graphs 193

13.2 Degree Distribution

What’s the expected degree of a node in a Gn,p network with 9 nodes?

Well, we start by looking at the first potential connection and toss a
coin. We do that for every possible node in the network. Since this
is independent, if the probability of connection is 50% and we make
8 tosses – one per potential neighbor –, on average we expect 8 × 0.5
head flips: 4, plus minus a small random fluctuation.

% nodes % nodes
with k = x with k = x
Real World
Figure 13.4: (a) The typical
degree distribution of Gn,p net-
k works. (b) Comparing a Gn,p
p(|V| - 1)

G(n, p)
degree distribution with one
that you would typically get
x x
from a real world network.
(a) (b)

A process like that generates a binomial distribution, where most

nodes have a specific degree: p times the number of nodes minus one
– because we avoid creating self loops. This is the average degree of
the network. Figure 13.4(a) shows an example of a random degree
distribution. Very few nodes have a much lower or much higher
degree, because of how rarely you will get many heads or tails in a
row from a fair coin.
Although the result is a binomial, many papers studying the
degree distributions of random graphs use a Poisson distribution
instead. They are practically identical, so this choice doesn’t really
matter – specifically a binomial becomes equivalent to a Poisson
when the number of trials is very large and the expected number of
successes remains fixed (see Section 2.7). We use a Poisson because
the parameters regulating it make it easier to calculate the things that
interest us – they all depend on a single parameter: k̄, the average
degree.
The random graph’s degree distribution is at odds with most real
world networks which, as we saw in Section 6.3, have many nodes
with low degree. Moreover, the outliers with many connections – the
hubs – in real networks have a much higher degree than the highest
degrees you’ll find in a random network. See Figure 13.4(b) for an
example. So this is the first pain point of random networks: their
degree distributions don’t match the skewed ones we observe in the
real world. Where is the real world broad degree distribution coming
from? That’s a question for Section 14.3.
194 the atlas for the aspiring network scientist

13.3 Connected Components

Besides broad degree distributions, we’re interested in giant compo-

nents, because all real world networks seem to have them. Remember
that giant components are components that include the vast major-
ity of – or even all – the nodes of the network, see Section 7.4 for a
refresher.

% nodes in LCC Figure 13.5: (a) The effect of

p on Gn,p ’s connectivity. (b)
High p The evolution of the number
of nodes in the largest con-
nected component of Gn,p as p
changes.

x
Low p p
(a) (b)

In a Gn,p graph, the presence of a giant component is dependent

on p. As it is easy to see from Figure 13.5(a), if p is high there are
a lot of edges, if it is low there are few and the graph could be dis-
connected. One could make a plot, showing for which values of
p we have how many nodes in the largest connected component.
One could expect a linear relationship, but that is not what we see:
there is a special value of p for which we observe a phase transi-
tion – which I show in Figure 13.5(b) –: if p is lower than that value
7
Paul Erdős and Alfréd Rényi. On the
there is no giant component, if p is higher then most nodes are in the strength of connectedness of a random
GCC7 , 8 , 9 , 10 . graph. Acta Mathematica Hungarica, 12
Can we determine this magical value of p? Logically, a node (1-2):261–267, 1961
8
Paul Erdos and Alfréd Rényi. On the
cannot be part of any component if it doesn’t have an edge. If the existence of a factor of degree one of a
average degree is less than one, many nodes won’t have edges. Thus connected random graph. Acta Math.
they cannot be part of the largest connected component. When the Acad. Sci. Hungar, 17(3-4):359–368, 1966
9
Dimitris Achlioptas, Raissa M D’souza,
average degree is higher than one, the giant component appears. and Joel Spencer. Explosive percolation
Thus the magical value of p is 1/|V |. in random networks. Science, 323(5920):
So this is the value beyond which we start to see the largest con- 1453–1455, 2009
10
Raissa M D’Souza and Michael
nected component gobbling up the majority of the network’s nodes. Mitzenmacher. Local cluster aggregation
But there is another question. Is there a value of p such that all nodes models of explosive percolation. Physical
are part of the giant component? This is equivalent of asking: when review letters, 104(19):195702, 2010

do we have fewer than one node without connections to the giant

component? We start with the probability of connecting a node u
with a node v. This is p. It follows that the probability of u and v
not to connect is 1 − p. Generalizing this, having no connection to a
 random graphs 195

component with |V | nodes is the same of landing tails for |V | times

in a row: (1 − p)|V | . The number of nodes without a connection to a
component with |V | nodes is equal to make |V | attempts – since we
have |V | nodes in our network –, all with that probability of success.
The result is |V |(1 − p)|V | .
Our original question was knowing when there are no nodes
outside the GCC. This is equivalent to say that we want to have fewer
than one node outside GCC. We just said that the number of nodes
outside the GCC is |V |(1 − p)|V | . Thus we want to know for which p 11
The full mathematical derivation rests
we have |V |(1 − p)|V | < 1. Which is p = ln |V |/|V |.11 on the assumption that (1 − x/n)n ∼
Note that ln |V |/|V | tends to be a rather small number as |V | e− x if n is large. At that point the
derivation follows: |V |(1 − p)|V | =
grows, given that it pits a logarithmic growth in the numerator
| V | p |V |
 
against a linear one in the denominator. Thus we discover that ran- |V | 1 − = |V |e−|V | p . If
|V |
dom graphs tend to have giant components as they grow larger, |V |e−|V | p = 1, as we said we want in the
which is exactly what we see happening in real world networks. One text, then:
|V |e−|V | p = 1
point to team Erdős-Rényi! e−|V | p = |V |−1
e |V | p = | V |
|V | p = ln |V |
13.4 Average Path Length p = ln |V |/|V |.

Figure 13.6: A simplification of

what happens to the average
path length in Gn,p random
graphs. If the average degree
ends up being four, a random
node – at the center – will have
four neighbors. Each of them,
will contribute on average three
more neighbors when consider-
ing paths of length two.

In a Gn,p network, the number of nodes directly connected to

a node is the average degree. This is the usual expectation as the
connection probability is fixed to p, as we saw in Figure 13.4(a).
In Gn,p we can easily calculate the number of nodes at two hops 12
Ithiel de Sola Pool and Manfred
from v. This is expected to be the average degree squared because, Kochen. Contacts and influence. Social
networks, 1(1):5–51, 1978
on average, each neighbor gives you access to its average degree 13
Time for another mathematical
number of neighbors – see Figure 13.6 for an example. This goes on derivation!
for any number of hops l. The number of nodes at l hops away is k̄l – k̄l = |V |
l ln k̄ = ln |V |
remember that k̄ indicates the average degree of the network.
l = ln |V |/ ln k̄.
If we know when k̄l = |V |, then we know the average path length Since in a Gn,p network k̄ = |V | p, you
of Gn,p : ln |V |/ ln(|V | p)12 , 13 . Note that this is relatively short, unless get the final derivation in the text.
196 the atlas for the aspiring network scientist

p is really low. Which is another thing that Gn,p graphs have in

common with real world networks. It seems that these random
graphs are not too shabby when it comes to reproducing real world
properties!

13.5 Clustering

What’s the clustering of a Gn,p network? Remember Section 9.2:

the local clustering CCv of a node v is number of triangles over the
number of triads centered in v. The number of triads, as we saw then,
is the number of possible edges among neighbors: k̄ (k̄ − 1)/2. We can
use k̄ instead of k v , because any v in a Gn,p graph is expected to have
an average degree. That’s the denominator of CCv .
The numerator of CCv is the number of triangles centered of
v. This is the probability an edge exists (p), times the number of
possible edges among neighbors. Again, on the basis of Section 9.2,
we know that the number of possible edges among neighbors is
k̄ (k̄ − 1)/2.

# Trianglesv n * (n - 1) Figure 13.7: The derivation of

CCv = # Triplets =
# of Tripletsv 2 the clustering coefficient of a
# edges among # neighbors = k random Gn,p network.
neighbors

p * # possible
edges among
neighbors

k * (k - 1) k * (k - 1)
p*
2 2
Much lower than the
one of real world networks! p Every node has the same
(So it’s also the average cc)

If we say, for simplicity, that k̄(k̄ − 1)/2 = x, then our CCv formula
looks like: CCv = px/x = p. This means that the clustering coeffi-
cient doesn’t depend on any node characteristic, and it’s expected
to be the same – equal to p – for all nodes. Figure 13.7 provides a
graphical version of this derivation.
Compared to real world networks, p is usually a very low value
for the clustering coefficient. In real world networks, it’s more likely
to close a triangle than to establish a link with a node without com-
mon neighbors. Thus the clustering coefficient tends to be higher
than simply the probability of connection. This is a second pain point
of Erdős-Rényi graphs when it comes to explain real world prop-
erties, after the lack of a realistic degree distribution, as we saw in
Section 13.2. Such a low clustering usually implies also the absence of
 random graphs 197

a rockstar feature of many real world networks: communities.

To recap, Gn,m and Gn,p models correctly estimate the emergence
of giant components and the relatively small diameters of real world
systems. However they fail three of the most crucial tests: degree
distribution, clustering, and communities. This is cause enough for
researchers to push forward in the quest for better graph models.
You can find a deeper treatment of random graph models in Béla Bollobás. Random graphs. In
14

Bollobás’s seminal work14 . Modern graph theory, pages 215–252.

Springer, 1998

13.6 Summary

1. Random graph models are useful for testing your algorithms,

explain how specific properties might arise in real world networks,
and test whether an observed network is really as special as you
think it could be, or if its properties are due to random chance.

2. The oldest and most venerable random graph model is the Gn,p
(or Gn,m ) model, where we fix the number of nodes n and the
probability p of connecting a random node pair, and we extract
edges uniformly at random.

3. These random graphs have a binomial degree distribution, which

is very different from the broad degree distributions of real world
networks.

4. There is a phase transition when it comes to the largest connected

component: if your random graph has an average degree higher
than one, you’ll have a connected component including most of
the nodes; if the average degree is lower, you won’t have such
component.

5. Random graphs have a short average path length just like real
world networks typically have. However, they have a much lower
clustering coefficient than what you find in the wild.

13.7 Exercises

1. Consider the network in [Link]

13/1/[Link]. Generate an Erdős-Rényi graph with the same
number of nodes and edges. Plot both networks’ degree CCDFs,
in log-log scale. Discuss the salient differences between these
distributions.

2. Generate a series of Erdős-Rényi graphs with 1, 000 nodes and

an increasing p value, from .00025 to .0025, with increments of
.000025. Make a plot with the p value on the x axis and the size of
198 the atlas for the aspiring network scientist

the largest connected component on the y axis. Can you find the
phase transition?

3. Generate a series of Erdős-Rényi graphs with p = .02 and increas-

ing number of nodes, from 200 to 1, 400 with increments of 200.
Make a plot with the |V | value on the x axis and the average path
length on the y axis. Since the graph might not be connected, only
consider the largest connected component. How does the APL
scale with the number of nodes?

4. Generate an Erdős-Rényi graph with the same number of nodes

and edges as the network used for question 1. Calculate and
compare the networks’ clustering coefficients. Compare this with
the connection probability p of the random graph (which you
should derive from the number of edges and number of nodes
using the formula I show in this chapter).
14
Understanding Network Properties

We now move on to the class of network models developed primarily

to explain network properties. The two most famous examples in this
class are the small world model proposed by Watts and Strogatz and
the preferential attachment model, independently discovered in many
variants multiple times across many decades but usually attributed
to Albert and Barabási. The first aims at explaining the high level of
clustering and small diameter of real world networks. The second
focuses on power law degree distributions.
The interest in these models is twofold. First, it is a historic in-
terest: these models were the first developed in the new wave of
network science in the late 90s. Their impact in the development
of the field was huge – the original papers both accumulated more
than 35 thousand citations. Second, they give an idea of what are
some of the original guiding principles of a certain flavor of network
science: the hunt for universal patterns that apply to any network
representation of a complex system. In practice, nowadays you’d sel-
dom use these vanilla models, as they have been superseded by more
sophisticated – albeit often less mathematically tractable – models.

14.1 Clustering

Before the Stone Age, a caveman society was very simple. You had
tribes living in their own caves. The tribes were very small, they were
families. Everybody knew everyone else in their cave, but between
caves there was almost no communication. Maybe there could have
been one weak link if the two caves were close enough.
This metaphor was the starting point for Watts in developing 1
Duncan J Watts. Networks, dynamics,
his “cavemen” model1 . The cavemen model is part of the family of and the small-world phenomenon.
simple networks (see Section 4.6). It takes two parameters: the cave American Journal of sociology, 105(2):
493–527, 1999
size (Figure 14.1(a)) and the number of caves (Figure 14.1(b)). The
cave size is the number of people living in each cave. A cave is a
clique: as said, everyone in the cave knows every cavemate (Figure
200 the atlas for the aspiring network scientist

Figure 14.1: (a) First step of

cavemen: decide the size of
the cave. (b) Second step: de-
cide the number of caves. (c)
Third: make each cave in a
(a) (b) (c) (d) clique. (d) Finally connect the
nearest caves via random cave
14.1(c)). Each cave “elects” a random member which will connect to a members.
random member of the nearest cave on the left, and another member
to connect to the nearest cave to the right (Figure 14.1(d)). That’s it:
the cavemen model.
By construction, the cavemen model has only one component,
because the caves are always connected with their nearest neighbors.
However, the cavemen model is worse than Gn,p in approximating
realistic diameters. To go from one cave to the farthest one in the
network it takes a really long path. The degree distribution is also
weird: in the example of Figure 14.1, all nodes inside a cave have
the same degree (equal to three) and the nodes in between caves all
have degree equal to four. Needless to say, this system with only two
distinct degree values isn’t found anywhere in natural networks.
So why do we want this type of graph? Well, differently from
Gn,p , cavemen gives us clustering and communities. Having such
well separated groups – the caves – makes it an ideal dataset to test
whether your community discovery algorithm is working or if it
is returning random results. You can’t get anything clearer than a
clique with just two edges pointing outwards.

14.2 Path Lengths

The small-world model is a more famous model developed by the

2
Duncan J Watts and Steven H Stro-
gatz. Collective dynamics of ‘small-
same author2 . It also models high clustering, but its primary target world’networks. nature, 393(6684):440,
was to explain small diameters, which were discovered in real world 1998
social networks by Milgram, as I showed in Section 10.3. In a small
world we start from you. You are standing in a certain point in
space. Then there are other people, also in their spots. You can only
communicate with people that are nearby you, because they are the
ones who can listen to you. Their sets of listeners overlap with yours,
because you’re all nearby each other. But, since they are not exactly
occupying your position, there are some folks whom you can reach
and they cannot, and vice versa. They can talk to an extra neighbor.
And so can their neighbors, to infinity.
This creates a regular network, a lattice, where each node is the
same as each other node. This is a simple network. Figure 14.2
 understanding network properties 201

Figure 14.2: The first step of

a small world model: each in-
dividual can talk to their four
most immediate neighbors.

provides a simple visualization. To use a more formal language, in

the first step of a small world model you put nodes into a single
dimensional space and connect them with their k nearest neighbors –
with k being the first parameter of the model that you can specify.

Figure 14.3: The second step of

a small world model: random
individuals can talk to someone
in the network, no matter their
physical locations.

However, sometimes, two folks can talk at distance, maybe because

they have each other phone number. And so a shortcut is made, as I
show in Figure 14.3. Formally, you establish a rewiring probability p
– the second parameter of the model. For each edge you toss a coin: if
it lands on heads you delete the edge and you rewire it by picking a
3
Mark EJ Newman and Duncan J Watts.
random destination. In variants of the model3 you do not delete the
Renormalization group analysis of the
original edge: for each existing edge you have a certain probability to small-world network model. Physics
pick an additional random destination for one of the two connected Letters A, 263(4-6):341–346, 1999

nodes.

Figure 14.4: The steps of the

small world mode: (a) Place
nodes in a one-dimensional
ring space. (b) Connect each
node with its k nearest neigh-
bors. (c) Randomly rewire
(a) (b) (c) edges according to the parame-
ter p.
Figure 14.4 shows the full process. Again, by construction we
don’t need fancy math to prove that this model generates a single
component. Since each node connects to a few nearest neighbors we
have one component by default. In the rewiring model you could
divide the network in separated components by unlucky rewiring
draws, but this is pretty unlikely, especially for high k values. Any-
how, given how unlikely this is, we can say that the small world
202 the atlas for the aspiring network scientist

model properly recovers the giant connected component property of

real world networks.
Differently from cavemen, this time we have short paths. They are
regulated by the rewiring probability p. Rewiring creates bridges that
span across the network. Even a tiny bridge probability can connect
parts of the network that are very far away. Thousands of shortest
paths will use it and will be significantly shorter.
Still, the degree distribution of a small world model is very weird.
If p is low, it looks like a cavemen graph, because almost all nodes
will have the same number of neighbors. If p is high it means that
we are practically rewiring every edge in the graph. At that point,
randomness overcomes every other feature of the model, and the
result would be almost indistinguishable from a Gn,p model. We have
high clustering because each connection that we don’t rewire will 4
Unless you set k = 2, then the
create triangles with some of the neighbors of the connected nodes4 . clustering would be zero. But why
would you set k = 2 in a small world
model? People are weird.
Figure 14.5: A small world
graph with 200 nodes, aver-
age degree k̄ equal to 8, and
rewiring probability p = 0.01.

However, high clustering does not necessarily mean that you are
going to have communities. In fact, a small world model typically
doesn’t have them. The triangles are distributed everywhere uni-
formly in the network. There are no discontinuities in the density,
no differences between denser and sparser areas. This is especially
evident for high k and low p: as I show in Figure 14.5, small world
networks with such parameter combinations just look like odd
snakes without clear groups of densely connected nodes. This is a
precondition to have communities, and so you cannot find them in a
small world model.

14.3 Degree Distribution

If we want to reproduce the degree distribution of a scale free net-

work we need a process that can generate a power law degree dis-
tribution. One of the most popular approaches is cumulative advan-
 understanding network properties 203

5
Jonathan R Cole and Stephen Cole.
tage5 . This is a fundamentally dynamic model. You have an initial Social stratification in science. 1974
condition and then you keep adding one element at a time. Each
element you add does not contribute to the preexisting ones uni-
formly at random, but prefers to contribute to specific older elements,
according to a rule you determine.
The preferential attachment model starts from the assumption
that the rich get richer. For instance, suppose you have one coin and
you invest in the stock market. If you’re lucky, after a while, you
will have another coin. Consider instead somebody who has a lot
of coins. Not only she can match your returns, she can probably do
better, because she can have a diversified portfolio which is resilient
to market shocks and black swans – highly improbable but also
massively impactful events, like the one at the basis of the mortgage 6
Nassim Nicholas Taleb. The black
crisis6 . Moreover, she can probably pay better advisers, and capital – swan: The impact of the highly improbable,
according to Piketty7 – just has better returns at scale. In the time it volume 2. Random house, 2007
takes for you to make a coin, she makes hundreds. Being already rich
7
Thomas Piketty. Capital in the 21st
century. 2014
makes her proportionally richer than you.

Figure 14.6: We both have crazy

hair, so the only difference
between Einstein and me is
that he got an unfair starting
advantage which accumulates
over time. Obviously.

The textbook case of cumulative advantage is in scientific publish- 8

Robert K Merton. The matthew
ing8 . In terms of networks, consider a citation network. I can write effect in science: The reward and
a paper, and maybe at some point somebody will read it and cite it. communication systems of science are
considered. Science, 159(3810):56–63,
On the other hand, we might have an actual researcher with a paper 1968
that has been cited hundreds of thousands of times. If we have a
newcomer to the citation network, what is she more likely to see, and
therefore to cite? The paper everybody knows. And so she will add 9
Herbert A Simon. Models of man;
to the pool, further increasing the odds that the paper will be seen social and rational. 1957
by another newcomer. Figure 14.6 shows a vignette depicting this 10
Derek de Solla Price. A general theory
process. of bibliometric and other cumulative
advantage processes. Journal of the
As I said, this reasoning is at the basis of a dynamical preferential American society for Information science,
attachment model9 , 10 , 11 . In preferential attachment you start from 27(5):292–306, 1976
an initial (set of) node(s) and you keep adding more. Each time you
11
Albert-László Barabási and Réka
Albert. Emergence of scaling in random
add a new node, it will connect to m of the old ones, where m is a networks. science, 286(5439):509–512,
parameter of the model. It will connect at random, but preferentially to 1999
204 the atlas for the aspiring network scientist

1/4 Figure 14.7: (a) Adding a fourth

1/4 node in a preferential attach-
ment model creates uneven
Loaded! probabilities of attachment
2/4
(floating next to the nodes that
are already part of the network).
(a) (b) A possible end result of the
(b) preferential attachment after
adding ten nodes.
nodes with higher degree: the higher a node’s degree, the more likely
it is it will gather more connections. Let’s follow the process step by
step.
The initial condition, meaning the topology of the network from
which you start, is not specified by the model. You can have practi-
cally what you want: a clique, a chain, a star. The idea is that, after
enough steps, what you started from doesn’t matter. Of course, this
seed has to have some characteristics that make it compatible with
your model. For instance, since you are going to add a new node
with m connections, it means that the initial condition has to have at
least m nodes. An accepted convention is to have them be connected
in a clique, so that they all have the same degree. One downside
of this flexible initial condition is that it makes the model a bit less
tractable mathematically, as you don’t really have a formula describ-
ing an arbitrary graph. There are some variants of the model that fix 12
Béla Bollobás, Oliver Riordan, Joel
this issue, for instance the linearized chord diagram12 . Spencer, and Gábor Tusnády. The degree
When you add your first new node, since you only have m initial sequence of a scale-free random graph
process. Random Structures & Algorithms,
nodes in the seed and you have to place m connections, there isn’t
18(3):279–290, 2001
much choice in deciding to whom you connect it. The new node will
connect to all nodes in the seed. When you add more nodes, you flip
a coin m times to decide who gets the edges from the new node. By
the time you’re adding the third node, you have more than m nodes
to choose from, and they do not have the same degree: some have
degree m, others have degree m + 1. You still flip a coin to decide
where the edges go, but now it’s a loaded one – see Figure 14.7(a),
where I fix m = 1. The new edges are more likely to go to the nodes
with more connections. If you keep repeating the process, you end up
with something looking like Figure 14.7(b).
As I said earlier, the law determining the connection probabil-
ities floating next to the nodes in Figure 14.7(a) is the number of
connections the node already has over twice the number of edges.
In practice, newcomers prefer to attach to high degree nodes. This
advantage accumulates over time: if Einstein is the highest degree
node, he is the most likely to get a new edge, which makes it even
more likely for him to get the next edge, and so on. You see that new-
 understanding network properties 205

1/6
1/2
1/6 Figure 14.8: Adding a new
node with the link selection
1/2 1/2
1/3 model. (a) Pick an existing
1/3
1/6 link at random. (b) Pick one
1/3
of the two nodes connected by
(a) (b) (c) that link. (c) Connect to it. The
probabilities of connecting to
comers have an ever decreasing chance to get the new connections. each node in this process are
This is not the only way to create a cumulative advantage. In fact, the ones floating next to it.
the model has some defects. Preferential attachment requires the
newcoming nodes to have global information about all the existing
nodes’ degree. This might be unrealistic in some cases – you may not
know the number of citations of every paper when you are making
a citation. It is also not a necessary feature to generate a cumulative
advantage. 13
Sergey N Dorogovtsev and Jose FF
An alternative to preferential attachment is link selection13 . In Mendes. Evolution of networks.
link selection, the newcoming node selects a link at random from the Advances in physics, 51(4):1079–1187,
2002
ones that exist in the network (Figure 14.8(a)). Then, it connects with
one of the two nodes connected by that edge – choosing uniformly
at random between the two (Figure 14.8(b)). Cumulative advantage
arises because nodes with more links are more likely to be selected,
thus getting more links on average (Figure 14.8(c)). No matter which
link you select, the central hub is connected to it.

1/4 1 / 12
1/4 1 / 12 Figure 14.9: Adding a new
node with the copying model.
1/4 3/4
(a) Pick an existing node at
1/4 1 / 12
random. (b) Copy one of its
connections. (c) The probabili-
(a) (b) ties of connecting to each node
(c)
in this process are the ones
A third alternative from preferential attachment and link selection floating next to it.
is the copying model. Just like in link selection, the newcoming node
has no information about the network, it just picks something uni-
formly at random. Differently from the link selection model, here it
picks another existing node, rather than a link (Figure 14.9(a)). It then
copies one of its connections (Figure 14.9(b)). You can see again how
it’s more likely to connect to the hub: the hub has more neighbors,
thus it is more likely to select one of its neighbors. Moreover, the
14
Jon M Kleinberg, Ravi Kumar, Prab-
hakar Raghavan, Sridhar Rajagopalan,
neighbor of a hub is likely to be low degree, increasing the chances and Andrew S Tomkins. The web as
of selecting the hub in the copying step (Figure 14.9(c)). The copy- a graph: measurements, models, and
methods. In International Computing
ing model is based on an analogy on how webmasters create new and Combinatorics Conference, pages 1–17.
hyperlinks to pre-existing content on the web14 . Springer, 1999
206 the atlas for the aspiring network scientist

It is easy to see why a network generated with either of these three

models has a single connected component. Since you always connect
a new node with one that was already there, there is no step in which
you have two distinct connected components. Thus, any cumulative
advantage network following any of these models will have all its
nodes in the same giant connected component, as it should.

6
Gn,m Figure 14.10: The average short-
5
4 PA
est path length (y axis) for
APL

3
2
increasing number of nodes (x
1 axis) for Gn,m (blue) and prefer-
10 100 1000
ential attachment (red) models,
|V|
with the same average degree.

These networks also have short diameters and average path

lengths. Mechanically it is easy to see why. Hubs with thousands
of connections can be used as shortcuts to traverse the network.
Mathematically, the diameter of a preferential attachment network
log |V |
grows as , thus very slowly, slower than a random graph.
log log |V |
Figure 14.10 shows some simulations, comparing the average path
length of a Gn,m and a preferential attachment network with the same
number of nodes and the same number of edges, as their size grows.
These models also reproduce power law degree distributions –
that’s what they were developed for. In fact, you can calculate the
exact degree distribution exponent for the standard preferential
attachment model, which is α = 3. This is independent from the
m parameter, meaning that you cannot tune it to obtain different
exponents (obviously, you might get different exponents because of
the randomness of the process, but as |V | → ∞, then α → 3). If you
want to reproduce a real world network with α = 2, you cannot use
the basic preferential attachment model.
However, there is a peculiar aspect about their degree distributions
that is worth considering. As you saw from all examples, the cumu-
lative advantage applies especially to “old” nodes. The earlier the
node entered in your structure, the more likely it is to become a hub.
This is especially easy to see in preferential attachment: the first node
getting the second edge in Figure 14.7 already has an advantage that
no newcomer can match.
If you look at a degree distribution of a preferential attachment
model, you’ll find the old nodes in the tail – the hubs – and the
head is going to be mostly composed by “young” nodes. I show
an example in Figure 14.11. Put it another way, there is a positive
correlation between a node’s age and its degree. This is particularly
 understanding network properties 207

0
10
Figure 14.11: The age effect in
10-1
the degree distribution of the
-2
10 preferential attachment model.
p(k>=x) “Young”
-3
10 nodes
-4
10
“Old” nodes
10-5
-6
10
0 1 2 3 4 5 6
10 10 10 10 10 10 10
x

interesting because that is not something we observe in real world 15

Lada A Adamic and Bernardo A
systems15 . For instance, you could argue that, on the web, the num- Huberman. Power-law distribution of
ber one website at the moment is [Link]. However, [Link] the world wide web. science, 287(5461):
2115–2115, 2000
isn’t the oldest website in the world. The web was already four years
old when [Link] was created. Moreover, readers a hundred 16
I’m very optimistic about the success
years from now16 might not even know what [Link] is, because of this book.
something replaced it.
There are ways to tweak the classical preferential attachment
model to fix some of its issues. For instance, one way is to balance 17
Fragkiskos Papadopoulos, Maksim
popularity and similarity17 . To determine where the connections Kitsak, M Ángeles Serrano, Marián Bo-
of a new node attach to, the classical preferential attachment uses guná, and Dmitri Krioukov. Popularity
versus similarity in growing networks.
exclusively the popularity of the already present nodes: the more
Nature, 489(7417):537–540, 2012
connections a node has, the more it’ll gather. However, nodes will
want to connect to other nodes that are similar to them – we’ll see
this real world tendency when we’ll discuss about homophily in
Chapter 26. Thus, if you have metadata about how similar two
nodes are, you can create a model where this similarity score is as
important as a node’s popularity to determine which nodes will
connect to it when they first arrive in the network.
From the examples made so far, you probably figured out that
there’s another thing missing: clustering. In particular, so far I made
simple examples where a newcoming node will add only one edge
to the network (the parameter m is equal to one). If we add one node
and one edge at a time it is impossible to create triangles.
You could set the parameter m > 1 to add two or more edges per
new node, but that helps only to a certain point: it’s not so likely
to strike two already connected nodes thus creating a triangle. The
preferential attachment model has a higher clustering than a random
Gn,p one, but not by much. It is still a far cry from the clustering
levels you see in real world networks. Moreover, this is only true for
m > 1. In that case, you add more than one edge per newcomer node,
which means you end up losing the head of your distribution. The
208 the atlas for the aspiring network scientist

network will not contain a single node with degree equal to one.
Thus the clustering in these cumulative advantage models is much
lower than real world networks, and there are no communities –
because everything connects to hubs which make up a single core.
There are some extensions of the model which try to include clus- 18
Petter Holme and Beom Jun Kim.
tering18 . At every step of this model you have a choice. You either Growing scale-free networks with
add a node with its links, or you just add links between existing tunable clustering. Physical review E, 65
(2):026107, 2002
nodes without adding a new one. The probability of taking that step
regulates the clustering coefficient of the network.
However, triangles close randomly, thus we have no communities
just like in the small world model. If we want to look at models
which generate more realistic network data, we have to look at the
ones I discuss in the next chapter.

14.4 Summary

1. To explain the high clustering and small diameter in real world

networks we could use the small world model by Watts and Stro-
gatz. In it, we place nodes on a regular distance in a low dimen-
sional space and connect them to k of their neighbors, ensuring
high clustering. We then create few shortcuts connecting pairs of
nodes at random with probability p, ensuring a small diameter.

2. The small world model has no communities, which you could

generate with a caveman graph: a ring of cliques. However, the
caveman graph has a long diameter.

3. To explain power law degree distributions you could use a prefer-

ential attachment model. In it, you grow the network one node at
a time. Each node brings m random connections. The probability
of connecting to a node u already present in the graph is propor-
tional to u’s degree. This model has low diameter, but also low
clustering.

4. Alternative models recreating power law degree distributions are

the link selection and the copying models.

5. Another unrealistic effect of the preferential attachment model is

the correlation between a node’s age (how long ago we added it
to the network) and its degree. Such correlation might not exist in
real world networks.

14.5 Exercises

1. Generate a connected caveman graph with 10 cliques, each with

10 nodes. Generate a small world graph with 100 nodes, each
 understanding network properties 209

connected to 8 of their neighbors. Add shortcuts for each edge

with probability of .05. The two graphs have approximately the
same number of edges. Compare their clustering coefficients and
their average path lengths.

2. Generate a preferential attachment network with 2, 000 nodes and

average degree of 2. Estimate its degree distribution exponent
(you can use either the powerlaw package, or do a simple log-log
regression of the CCDF).

3. Implement the link selection model to grow the graph in http:

//[Link]/exercises/14/3/[Link] to 2, 000 nodes
(for each incoming node, copy 2 edges already present in the net-
work). Compare the number of edges and the degree distribution
exponent with a preferential attachment network with 2, 000 nodes
and average degree of 2.

4. Implement the copying model to grow the graph in [Link]

[Link]/exercises/14/4/[Link] to 2, 000 nodes (for
each incoming node, copy one edge from 2 nodes already present
in the network). Compare the number of edges and the degree
distribution exponent with networks generated with the strategies
from the previous two questions.
15
Generating Realistic Data

Both the small world and the preferential attachment models are
useful because they give us ideas on how some real world network
properties arise. The small world model tells us that small diameters
happen because a clustered network might have some random short-
cuts. The preferential attachment model tells us that broad degree
distributions arise because of cumulative advantage: having many
links is the best way to attract more links.
Yet, neither of them is able to reproduce all the features of a real
world network. If we want to do so, we have to sacrifice the explana-
tory power of a model. We have to fine tune the model so that we
force it to have the properties we want, regardless of what realistic
process made them emerge in the first place. This is the topic of this
chapter.

15.1 Configuration Model

The easiest way to ensure that your network will have a broad degree
distribution is to force it to have it. No fancy mechanics, no emerging
properties. You first establish a degree sequence and then you force
each node to pick a value from the sequence as its degree. This 1
Mark EJ Newman. The structure and
simple idea is at the basis of the configuration model1 . In fact, the function of complex networks. SIAM
configuration model is more general than this. You can use it to review, 45(2):167–256, 2003b
match the degree sequence of any real world graph, regardless of the
simplicity or complexity of its actual degree distribution.
The configuration model starts from the assumption that, if we 2
Michael Molloy and Bruce Reed. A
want to preserve the degree distribution, we can take it as an input of critical point for random graphs with
our network generating process. We know exactly how many nodes a given degree sequence. Random
structures & algorithms, 6(2-3):161–180,
have how many edges. So we forget about the actual connections, 1995
and we have a set of nodes with “stubs” that we have to fill in. Figure 3
Mark EJ Newman, Steven H Strogatz,
15.1 shows an example. and Duncan J Watts. Random graphs
with arbitrary degree distributions and
There’s a relatively simple algorithm to generate a configuration their applications. Physical review E, 64
model network, the Molloy-Reed approach2 , 3 . First, as we saw, you (2):026118, 2001
 generating realistic data 211

# nodes
Figure 15.1: In a configuration
model, you start from the de-
gree histogram to determine
2 how many nodes have how
4 many open “edge stubs”.
1 1 1

create a degree sequence, in which each value represents a node’s

degree. This sequence has a few constraints: the most important is
that it has to sum to an even number. If it were to sum to an odd
number, you’d have a node which cannot assign its last stub to any
other neighbor – i.e. the sequence is not “graphic”.
Second, each node gets a unique identifier. As third step, you
generate a list of these identifiers. You repeat each identifier in this
list as many times as its assigned degree. For instance, if you know
that node 1 has degree four, the list will contain four 1s. If node 2 has
degree twenty, you add twenty 2s.
Finally, you create the actual connections. You pick two elements
at random from this list, which are two node identifiers. If the iden-
tifiers are different and the two nodes are not already connected to
each other, you connect them. The two conditions are necessary to
avoid the creation of self loops and parallel edges – if you’re ok with
either, you can skip these checks. Note that you might end up with a
few unassigned edges, but usually these are an insignificant number
which will not affect the degree distribution too much.
Note that, each time we pick two ids from the list, we remove
them from it. This ensures that each node will be picked only as
many times as its degree – or fewer times if we cannot find any legal
connections at the end of the process. Figure 15.2 shows a depiction
of a connection step in the configuration model.
The Molloy-Reed approach is not the only way to generate a ran-
dom graph with a given degree distribution. For starters, there are 4
Fan Chung and Linyuan Lu. Con-
closely related alternatives like the Chung-Lu model4 . An alterna- nected components in random graphs
tive is the double swap edge algorithm which I describe in Section with given expected degree sequences.
Annals of combinatorics, 6(2):125–145,
16.1. As you’ll see, in that case one doesn’t need to worry about
2002b
self-loops or parallel edges. The two algorithms are in different chap-
ters because of their different aims. If you simply need a realistic
graph with a given degree distribution for testing an algorithm or
212 the atlas for the aspiring network scientist

Figure 15.2: A depiction of the

algorithm to generate a network
1 2 3 4 5 6 7 8 9 following the configuration
model: the edge stubs on the
[1, 2, 3, 4, 5, 5, 6, 6, 7, 7, 7, nodes with their identifiers
8, 8, 8, 8, 9, 9, 9, 9, 9] (top) and the list with node
ids from which we pick nodes
(bottom). The red circled ids are
do asymptotic mathematics, the Molloy-Reed configuration model is
the ones picked, so we connect
good enough. But, if you want to compare a random graph to data,
node 5 to node 8.
the differences are crucial5 and that’s why the edge swap algorithm
is in the chapter dedicated to evaluating statistical significance.
5
Bailey K Fosdick, Daniel B Larremore,
Joel Nishimura, and Johan Ugander.
You can add a few features to the configuration models that were Configuring random graph models
not trivial to add to either small world or preferential attachment. For with fixed degree sequences. SIAM
Review, 60(2):315–355, 2018
instance, you can make a directed version of it. The only thing you
need is to generate two degree distributions: one for the in-degree
and one for the out-degree. This makes the connection step a bit
more complex, as you have to pick the source from one list and the
destination from another, but it is not a big deal.
You can see that this model will be spot on in replicating the vast
majority degree distributions you pass to it. The way to estimate the
difference between two distributions is by performing a Kolmogorov- 6
Andrey Kolmogorov. Sulla deter-
Smirnov test6 , 7 . The test identifies the point of maximum separation minazione empirica di una lgge di
between two distributions, along with a p-value telling you how distribuzione. Inst. Ital. Attuari, Giorn., 4:
83–91, 1933
likely it is that the two distributions are indistinguishable from each 7
Nickolay Smirnov. Table for estimating
other. the goodness of fit of empirical distribu-
The configuration model tends to generate giant connected com- tions. The annals of mathematical statistics,
19(2):279–281, 1948
ponents just like a random Gn,p graph would, although this heavily 8
William Aiello, Fan Chung, and
depends on the α parameter of your power sequence8 (see Section Linyuan Lu. A random graph model
6.4 for a refresher about the meaning of α). Deriving the expected for massive graphs. In Proceedings of
the thirty-second annual ACM symposium
average shortest path length is a bit trickier, but it can be done9 and it
on Theory of computing, pages 171–180.
is realistically short in most cases. Acm, 2000
The clustering coefficient of a configuration model also depends 9
Fan Chung and Linyuan Lu. The
average distances in random graphs
on α. For the majority of realistic values of α – between 2 and 3 –, the with given expected degrees. Proceedings
clustering coefficient of a configuration model tends to zero, which of the National Academy of Sciences, 99
is very unrealistic. There are a few valid values of α generating a (25):15879–15882, 2002a

properly high clustering, but these are rare enough that researchers
needed to modify the configuration model to explicitly include the 10
Mark EJ Newman. Random graphs
generation of triangles10 , 11 . with clustering. Physical review letters,
This is usually achieved by generating a joint degree sequence. 103(5):058701, 2009
Rather than simply specifying the degree of each node, we now have
11
Joel C Miller. Percolation and epi-
demics in random clustered networks.
to fix two values. The first is the number of triangles to which the Physical Review E, 80(2):020901, 2009
node belongs. The second is the number of remaining edges the
node has that are not part of a triangle. One can see that we’re still
 generating realistic data 213

specifying the degree, because the number of triangles is simply

half the number of edges we’re adding to that node: each node in
a triangle connects to other two nodes. If you know the number of
triangles and the total degree of each node you know the clustering
coefficient of the network (Section 9.2), thus you can generate a
sequence that will have the desired clustering coefficient.

Figure 15.3: A simplicial con-

figuration model with 8 nodes
(in red) and two simplicial com-
plexes (in blue). The nodes have
the following open stubs (from
left to right): {1, 1, 1, 3, 2, 2, 2, 1}.

An alternative – and more flexible – way to build higher order

structures in your configuration model is to allow it to include sim-
plicial complexes. I introduced simplicial graphs when talking about
hypergraphs: these are graphs including relations between multiple
nodes, rather than just normal edges, which are binary relationships.
In a simplicial configuration model, besides nodes with open stubs,
you also have simplicial shapes, each with the number of stubs de- 12
Owen T Courtney and Ginestra Bian-
termined by their dimension (from 3 up)12 , 13 . Figure 15.3 shows an coni. Generalized network structures:
example. The configuration model and the canon-
ical ensemble of simplicial complexes.
You can connect each node stub with either another node stub, or
Physical Review E, 93(6):062311, 2016
with a simplicial complex, until there are no more open stubs. Note 13
Jean-Gabriel Young, Giovanni Petri,
that, in this model, the number of open stubs is not the degree any Francesco Vaccarino, and Alice Patania.
Construction of and efficient sampling
more. Every time you connect a node to a complex, the node’s degree
from the simplicial configuration model.
increases by the dimensionality of the stub minus one (thus by 2 if Physical Review E, 96(3):032312, 2017
you connect it to a triangular complex, by 3 if you connect it to a
square, and so on). Thus controlling the exact degree of a node is
trickier.
Another complication is that you have more topolgical constraints.
Forget about simply generating a graphical degree sequence: you
now have a set of forbidden moves. A node can be part of multiple
simplicial complexes, but you cannot make two distinct simplicial
complexes using the very same nodes (Figure 15.4(a)). Nor you can
use two stubs from the same node in the same simplicial complex
(Figure 15.4(b)). The first case is forbidden because you’d be creating
two indistinguishable complexes – and therefore only one instead of
two. In the second case, you wouldn’t be constructing a complex at
all!
The result is a network that has exactly the desired number of sim-
214 the atlas for the aspiring network scientist

Figure 15.4: Two forbidden

moves in the simplicial configu-
ration model.

(a) (b)

plicial complexes. It might have, however, more triangles than you

expect, because you can connect three nodes independently. There’s
an example of this mishap in Figure 15.3: one of the two triangles is
not filled, because it’s made by three independent edges, rather than
being a three-way relationship, like the blue-filled simplicial complex.
You don’t have to necessarily end up with a simplicial network
or a hypergraph: once you have placed the shapes you can “forget”
that they are higher order structures, and consider you network as a
simple graph.

15.2 Communities

The configuration model can be tuned to include a high clustering,

but it will still close triangles randomly in the network. This means
that the number of triangles is correct, but their distribution in the
network is random. As I mentioned multiple times, this is not the
case for real world networks. The triangles tend to correlate and
form denser areas we call communities – specifically, assortative
communities, the concept of community is a bit more complex than
that and requires to read Part IX, for now let’s just roll with this
simplification. Thus, the configuration model is still inadequate
to fully reproduce a quasi-real network. Doing so is the task of a
few models: stochastic block models, GN and LFR benchmark, and
Kronecker graphs.

Stochastic Block Models

The easiest way to create communities in your graph model is to
make a simple observation. Since communities are dense areas –
with nodes connecting to each other – separated by sparse areas, it
just means that nodes have two different probabilities to connect
to each other. One probability, say pin , determines the probability
of a node u to connect to another node inside the same community.
Another probability, pout , determines the likelihood of connecting
 generating realistic data 215

to a node from a different community. Obviously, if we want dense

communities, pout < pin .
pin and pout are the first two parameters of a Stochastic Block 14
Paul W Holland, Kathryn Blackmond
Model14 (SBM). To fully specify the model you need a few additional Laskey, and Samuel Leinhardt. Stochas-
ingredients. First, you have to specify |V |, the number of nodes in tic blockmodels: First steps. Social
networks, 5(2):109–137, 1983
the graph. Then you have to plant a community partition. In other
words, for each node – from one to |V | – you have to specify to which
community it belongs. Otherwise we don’t know how to use the pin
and pout parameters.
It’s easy to see that, if pin = pout , then the SBM is fully equivalent
to a Gn,p model: each node has the same probability to connect to
any other node in the network. However, if we respect the constraint
that pout < pin , the resulting adjacency matrix will be block diagonal.
Most of the non zero entries will be close to the diagonal, whose
blocks are exactly the communities we planted in the first place!

Figure 15.5: (a) A matrix con-

taining, for each entry, the
probability of the two nodes
to connect to each other. (b) A
realization of the SBM using the
matrix from (a) as input. Each
dark entry is an edge in the
(a) (b) network.

One could go even deeper and determine that each pair of nodes
u and v can have its own connection probability. This would generate
an input matrix for the SBM that looks like the one in Figure 15.5(a).
Figure 15.5(b) shows a likely result of the SBM using Figure 15.5(a)
as an input. It’s easy to see why we call this matrix “block diagonal”.
The blocks are.... on the diagonal, man.
In one swoop we obtained what we were looking for: both com-
munities and high clustering. The very dense blocks contribute a lot
to the clustering calculation, more than the sparse areas around the
communities can bring the clustering down. One observation we will
come back to is that, in real world networks, the community sizes dis-
tribute broadly, just like the degree: there are few giant communities
and many small ones. This can be embedded in SBM, since we’re free
to determine the input partition as we please.
If you set pout to a relatively high value, you might make your
communities harder to find, but you gain something else: smaller
diameters. You’re also free to set pin < pout in which case you’d find
a disassortative community structure, where nodes tend to dislike
nodes in the same community. See Section 26.2 to know more about
216 the atlas for the aspiring network scientist

what disassortativity means.

Many real world networks will have overlapping communities
sharing nodes. The standard SBM cannot handle this: in the input
phase we can only put a node in a single community. However, smart 15
Edoardo M Airoldi, David M Blei,
folks have created Mixed-Membership Stochastic Block Models15 , Stephen E Fienberg, and Eric P Xing.
in which nodes are allowed to span across communities. Another Mixed membership stochastic block-
models. Journal of Machine Learning
important variation of SBM is the degree-correlated SBM16 , which
Research, 9(Sep):1981–2014, 2008
allows you to fix the degree distribution just like the configuration 16
Brian Karrer and Mark EJ Newman.
model does. Stochastic blockmodels and community
structure in networks. Physical review E,
83(1):016107, 2011

GN Benchmark
17
Michelle Girvan and Mark EJ New-
The GN benchmark is a modification of the cavemen graph and one
man. Community structure in social
of the first network models designed to test community discovery and biological networks. Proceedings of
algorithms17 . The first defining characteristic of this model is set- the national academy of sciences, 99(12):
7821–7826, 2002
ting some of the parameters of the cavemen graph as fixed. In the
benchmark, we have only four caves and each cave contains 32 nodes.
Differently from the caveman graph, the caves are not cliques: each
node has an expected degree of 16, thus it can connect at most to half
of its own cave.

Figure 15.6: Results from the

GN benchmark for increasing
values of µ.

(b) µ = 0.0625
(c) µ = 0.125
(a) µ = 0 (d) µ = 0.1875

The GN benchmark then introduces a parameter, usually called

“mixing” parameter, or µ. This is the share of edges that a node has
to nodes that are not part of its own cave. You can use µ to introduce
the amount of noise you want in the community structure. If µ = 0,
then all nodes will exclusively connect with fellow members of the
same cave. This results in four isolated connected components with
no paths among them. If µ = 0.5, half of the edges of a node point
outside its community, to a random other cave. You can see the effect
of µ in the sequence from Figure 15.6.
The GN benchmark isn’t particularly realistic. It has a fixed num-
ber of nodes, rather low when compared with real world networks.
Its degree distribution is binomial, which rarely happens in the real
world. It is also rare to have equally-sized communities. The LFR
benchmark fixes all these issues.
 generating realistic data 217

LFR Benchmark
The LFR Benchmark was developed to serve as a test case for com- 18
Andrea Lancichinetti, Santo Fortu-
munity discovery algorithms18 . The objective is to generate a large nato, and Filippo Radicchi. Benchmark
number of benchmarks to test a new algorithm such that we know graphs for testing community detection
algorithms. Physical review E, 78(4):
the “true” allegiance of each node. Once an algorithm returns us a 046110, 2008
possible node partition, we can compare its solution with the true
communities.
Since you want to have networks with lots of realistic properties,
some of which are difficult to reproduce organically, the LFR bench-
mark takes lots of input parameters. If you want an LFR network,
you have to specify:

• The α exponent of the power law degree distribution of the graph;

• The β exponent of the power law size distribution of the communi-

ties in the graph;

• The |V | number of nodes in your graph;

• The k̄ average degree of the nodes – you can also set k min and k max
as the minimum and maximum degree, respectively;

• smin and smax , as the minimum and maximum community size;

• The µ mixing parameter, regulating the fraction of edges going

outside their planted communities;

Optional on : the fraction of nodes overlapping between multiple

communities, if using the overlapping variant of LFR;

Optional om : the number of communities to which an overlapping

node belongs.

As you can see, the LFR assumes that both the degree distribution
and the size of your communities distribute like a power law. The

Figure 15.7: A messy real world

network in which I highlighted
with red outlines the three
largest communities.
218 the atlas for the aspiring network scientist

latter is regulated by the β parameter, with one gigantic community

including the majority of nodes and many trivial communities of
size smin . In Figure 15.7 I show a real world network and its three
largest communities, showing how their sizes rapidly decline. This is
a rather realistic assumption, although there are obvious exceptions.
You can take care of such exceptions by forcing the maximum com-
munity size to a known – and lower – smax size. You can also set the
minimum community size if you don’t want to have too many trivial
communities in your network, using the smin parameter.
The mixing parameter µ regulates how hard it is to find communi-
ties in the network. If µ = 0 all edges run between nodes part of the
same community, i.e. each community becomes a distinct connected
component of the network (Figure 15.8(a)). In this scenario, recover-
ing community information is trivial. If µ = 1 then nodes in the same
community do not connect at all (Figure 15.8(d)). In this scenario,
recovering the community wiring is impossible. Usually, you want to
set µ to a reasonably low non-zero value. From Figures 15.8(b-c) you
can see why this is the case: even the seemingly low value of µ = 0.2
generates hard to distinguish communities (Figures 15.8(c)).

Figure 15.8: Examples of LFR

benchmarks at different µ
levels.

(c) µ = 0.2 (d) µ = 1

(b) µ = 0.05
(a) µ = 0

The basic LFR algorithm, which generates disjoint communities,

works using the following strategy:

1. Generate the degree distribution of the network, with exponent

α and whose minimum and maximum degree are k min and k max
(Figure 15.9(a));

2. Mark a fraction µ of its edges as connecting outside the commu-

nity and the rest as connecting inside the community (Figure
15.9(b));

3. Generate the community size distribution, with exponent β and

whose minimum and maximum size are smin and smax ;
 generating realistic data 219

4. Assign each node v to a community c at random, ensuring that

k v (1 − µ) < sc , i.e. that the community will have enough nodes for 19
If k v (1 − µ) > sc , once you gave
v to connect to them19 (Figure 15.9(c)); to v sc internal edges, you still have
internal edges to assign to v that
5. Apply a modified configuration model to establish the edges. cannot be attached to nodes inside c,
because v is already connected to all its
Each node v connects to k v (1 − µ) random nodes in its community community mates, thus breaking the
c, and to k v µ random nodes outside c (Figure 15.9(d)). model. Remember that sc is the size of
community c, in number of nodes.

(a) Step #1 (b) Step #2 (c) Steps #3 & #4

(d) Step #5
Figure 15.9: A run through a
Note that, in light of step #4, you have some constraints in your simple LFR model.
choice of parameter. Specifically k min < smin and k max < smax ,
otherwise the nodes with minimum and maximum degree will never
belong to any community. Even with such constraints, sometimes
the combination of parameters will require the generation of an
impossible graph, so the LFR benchmark will always be some sort of
approximation of your desires. In practice, differences are going to be
relatively tiny and insignificant.
Since we’re plugging in a power law degree distribution and
communities, it is obvious that LFR benchmarks will reproduce these
characteristics of real world networks well – although now you’re
actually forced to have a power degree distribution, which in some
cases you might not want. They also respect clustering and small
diameters, making them the most realistic model we have.

Kronecker Graphs

The idea of a Kronecker graph originates from the Kronecker product

operation. The Kronecker product is the matrix equivalent of the
outer product of two vectors. We call the outer product the following
operation:
220 the atlas for the aspiring network scientist

   
u1 u1 v1 u1 v2 u1 v3
u h i u v u2 v2 u2 v3 
u ⊗ v = uv> =  2  v1
 2 1
v2 v3 =  .
  
 u3   u3 v1 u3 v2 u3 v3 
u4 u4 v1 u4 v2 u4 v3

xA xA xA xA
Figure 15.10: An example of
xA xA xA xA Kronecker product. (a) A ma-
xA xA xA xA trix A. (b) The operation we
xA xA xA xA perform to obtain A ⊗ A.

(a) (b)
So, in practice, the outer product of two vectors u and v is a
|u| × |v| matrix, whose i, j entry is the multiplication of ui to v j .
The Kronecker product is the same thing, applied to matrices. Fig-
ure 15.10 shows an example. To calculate A ⊗ B, we’re basically
multiplying each entry of A with B20 . 20
G Zehfuss. Über eine gewisse
determinante. Zeitschrift für Mathematik
When it comes to generating graphs, the matrix we’re multiplying und Physik, 3(1858):298–301, 1858
is the adjacency matrix. We usually multiply it with itself. So we’re
calculating A ⊗ A, as I show in Figure 15.10(b). This generates a new
squared matrix, whose size is the square of the previous size. We can
multiply this new adjacency matrix with our original one once more,
for as many times as we want. We stop when we reach the desired
number of nodes21 , 22 .
21
Jurij Leskovec, Deepayan Chakrabarti,
Jon Kleinberg, and Christos Faloutsos.
Figure 15.11 shows the progression of the Kronecker graph. Fig- Realistic, mathematically tractable
ure 15.11(a) is our seed graph which we multiply to itself (Figure graph generation and evolution, using
kronecker multiplication. In European
15.11(b)) twice (Figure 15.11(c)). Conference on Principles of Data Mining
One small adjustment that is customary to do when generating and Knowledge Discovery, pages 133–145.
a Kronecker graph is to fill the diagonal with ones instead of zeros. Springer, 2005b
22
Jure Leskovec and Christos Faloutsos.
If you remember my linear algebra primer, this means we consider Scalable modeling of real graphs using
every node to have a self-loop to itself. This is because we want kronecker multiplication. In Proceedings
the Kronecker graph to be a block-diagonal matrix, with lots of of the 24th international conference on
Machine learning, pages 497–504. ACM,
connections around the diagonal. This is required if we want them to 2007
show a sort of community partition.
By how the Kronecker product is defined you can see that, if
the seed matrix had an empty diagonal, we would not get a block

Figure 15.11: An example of

Kronecker graph. (a) The seed
adjacency matrix. (b) Kronecker
product of (a) with itself. (c)
Kronecker product of (b) with
(a).
(a) (b) (c)
 generating realistic data 221

diagonal matrix after applying the Kronecker product. Figure 15.12

shows an example, in which you can see the devastating effect on the
graph’s density of leaving the main diagonal empty. We can always
reset the main diagonal to zero once we’re done with the Kronecker
products.

Figure 15.12: An example of

Kronecker graph, similar to the
one in Figure 15.11, but without
setting the main diagonal to
one.
(a) (b) (c)
The question underlying generating graphs with an iterative Kro-
necker product is: why? Well, for starters, Kronecker graphs are
fractals. Personally, I don’t need any other reason that that. Look
at Figure 15.11(c): if you tell me it doesn’t speak to your heart then
I question whether you’re really human. If you’re not an incurable
fractal romantic like me, the deceptively simple process that gener-
ates Kronecker graphs solves all the issues we want from a graph
generating process. In some cases, it is even better than LFR.

Figure 15.13: Some properties

of Kronecker graphs. (a) Com-
munities – circled in red –; (b)
Communities (red) with their
(c) overlap (blue); (c) Small diame-
(a) (b)
ter – as the highlighted node in
Kronecker graphs have high clustering and communities (Figure red is connected to every other
15.13(a)), even hierarchical and overlapping (Figure 15.13(b)). They node in the network making the
even have a hint of core-periphery structures, which I’ll present fully diameter equal to two.
in Chapter 28. They are small world (Figure 15.13(c)), and with a
power-lawish degree distribution (usually shifted because they have
few low degree nodes). LFR is preferred because it leaves space for
the randomness of real world noise, but Kronecker graphs have the
advantage of being more simple to understand and implement.

15.3 Random Geometric Graph

Another property you might want to preserve in your graph is the

spatial structure. Many networks live on a physical space, and this
physical space constraints the edge generating process. If two nodes
are too far way from each other, they cannot connect. For instance,
if two cities are at the antipodes of the globe, there might not be a
222 the atlas for the aspiring network scientist

plane able to fly directly from once city to the other. We can model 23
Mathew Penrose et al. Random
these constraints using random geometric graphs23 , 24 . geometric graphs, volume 5. Oxford
The concept is simple. You first decide the dimensionality of your university press, 2003
space: is it a 2D plane, three dimensional, or n-dimensional? Then
24
Jesper Dall and Michael Christensen.
Random geometric graphs. Physical
you generate |V | points in this space, by extracting them uniformly at review E, 66(1):016121, 2002
random. Finally, you connect two points if they are at r distance – or
less – from each other. Every point will be at distance zero from itself
but, for the sake of simplicity, we simply ignore self-loops. Note that
you are free to decide how to calculate the distance between points:
you’re not forced to use the Euclidean.

Figure 15.14: Random geomet-

ric graphs with 200 nodes. X
and Y positioning of nodes are
the same for all figures, but the
r parameter increases from left
to right, generating a higher
number of longer edges.
(a) r = 0.08 (b) r = 0.14 (c) r = 0.2
This is a rather simple way of generating random graphs. A ran-
dom geometric graph is fully described by a handful of parameters:
the number of nodes |V | – which is the number of points you ex-
tracted –; the maximum distance r; the number of dimensions; and
the measure you used to calculate point-point distances. Figure
15.14 shows a few results. In the figure, I used a 2D plane and the
Euclidean distance measure. The networks have the same number
of nodes – in fact their coordinates are the same –, and I simply play
with the r parameter.
There is a simple naive algorithm to generate random geometric
graphs. First, you extract |V | uniform random tuples – depending on
your chosen dimensionality, for a 2D plane they’d be pairs. Then you
calculate the pairwise distance between all of them and connect the 25
Daniel Funke, Sebastian Lamm,
nodes if the distance is lower than r. There are smarter algorithms25 , Ulrich Meyer, Manuel Penschuck,
especially designed to avoid computing all pairwise distances – and Peter Sanders, Christian Schulz, Dar-
ren Strash, and Moritz von Looz.
exploiting parallel processing. Communication-free massively dis-
Assuming that you use the Euclidean distance, you can derive the tributed graph generation. Journal of
Parallel and Distributed Computing, 131:
probability of having a given number of isolated vertices or a value 200–217, 2019
of clustering coefficient by looking at the parameters you used to
generate the network. In general, giant connected components appear
2
easily in these types of graphs, provided that |V |e−πr |V | < 1. This
magic value derives from the fact that the expected degree of a node
is πr2 |V |, given that it will connect to all nodes in a circle around it
– which has an area of πr2 . For instance, in Figure 15.14(a), r = 0.08
2
and thus |V |e−πr |V | ∼ 3.6, which allows for a few isolated nodes;
 generating realistic data 223

while in Figure 15.14(c) this value is ∼ 2 × 10−9 , which makes the

presence of a single connected component almost certain. 26
Bernard M Waxman. Routing of
You can also have probabilistic random geometric graphs26 . The multipoint connections. IEEE journal
difference is that you do not always connect nodes at distance lower on selected areas in communications, 6(9):
1617–1622, 1988
than r with probability 1, but rather with some probability p < 1.

15.4 Graph Generative Networks

One thing that all models discussed so far have in common is that
they are engineered to have specific properties. These are the prop-
erties we think are salient in real world networks: broad degree
distributions, community structures, etc. But what if we are wrong?
Maybe some of these properties are not the most relevant things
about a network we want to model. Moreover: what if there are other
properties that we aren’t seeing? Edges are dependent on each other,
but these dependencies can be complex and it’s difficult to put them
in simple measures we can then optimize. 27
Yujia Li, Oriol Vinyals, Chris Dyer,
The field of graph generative networks27 , 28 , 29 aims at tackling Razvan Pascanu, and Peter Battaglia.
this problem. Here we want to generate networks that look like Learning deep generative models of
graphs. arXiv preprint arXiv:1803.03324,
specific real world networks, without us knowing what “looking like” 2018
actually means. In other words, we want the generative process to 28
Nicola De Cao and Thomas Kipf.
Molgan: An implicit generative model
“learn” how a real world network looks like, so that it can generate for small molecular graphs. arXiv
synthetic versions at will. preprint arXiv:1805.11973, 2018
The most trivial way you can do this is by feeding the adjacency 29
Aleksandar Bojchevski, Oleksandr
Shchur, Daniel Zügner, and Stephan
matrix of your graph – or a suitably modified version of it – to a Günnemann. Netgan: Generating graphs
standard neural network. The neural network will learn the depen- via random walks. In International
dencies between edges. You can think of this approach as a SBM Conference on Machine Learning, pages
609–618, 2018
process without inferring the communities beforehand. SBM wants
to preserve the community structure and, on this basis, learns edge
probabilities that only depend on the community affiliation. Here,
we want to preserve the general network properties, thus each edge
probability is dependent on the entirety of the adjacency matrix.
However, this approach has two problems. First, it will only gener-
ate a graph with the same number of nodes as the input, while you
might want to vary the size of your synthetic networks. Second, it
can only learn from a single graph at a time. Sometimes, you might
want to model a class of graphs.
These limitations are solved in a variety of ways. Just to give an 30
Jiaxuan You, Rex Ying, Xiang Ren,
example, GraphRNN30 allows for two moves: a graph-level update William Hamilton, and Jure Leskovec.
and an edge-level update. In the first step, GraphRNN adds a new Graphrnn: Generating realistic graphs
with deep auto-regressive models. In
node into the network. Every time a new nodes is added, the edge- International Conference on Machine
level update is triggered, determining to which nodes the new node Learning, pages 5694–5703, 2018
connects. This is achieved by representing the graph as a sequence.
For each node, in order, we list to which of the previous nodes it con-
224 the atlas for the aspiring network scientist

[1, 1, 0, 0, 1, 1, 0, 0, 1, 1]
2 Figure 15.15: A graph and its
1, sequence representation in
4 1, 0, GraphRNN. Each element in
1 the sequence belongs to a node
0, 1, 1, (character color) and records
3 5 0, 0, 1, 1 whether that node connects to a
specific node preceding it in the
sequence (underline color).
nects. For instance, the sequence [1, 1, 0, 0, 1, 1, 0, 0, 1, 1] corresponds to
the graph in Figure 15.15.
To see why, it is useful to break down the sequence in sections,
each one referring to a node, as the figure does. The first node has
no element in the sequence, because it has no preceding nodes.
The second node contributes only one element to the sequence: 0
if it doesn’t connect to the first node, 1 otherwise. The third node
contributes two values, one for its edge with the first node and one
for the second node, and so on.

2 2 2 2

4 4
1 1 1 1 1

3 3 3 5

1 1,0 0,1,1 0,0,1,1

Figure 15.16: The GraphRNN

In practice, GraphRNN expands the sequence, by adding the nth
workflow. From left to right, we
node (graph-level update) as a new subsequence of length n − 1 (edge-
progressively add new nodes,
level update). Figure 15.16 shows a simple iteration. These updates
in the form of an extension of
are implemented via autoregressive models.
the sequence representing the
connections.
15.5 Summary

1. The configuration model is a way to have a synthetic network with

an arbitrary degree distribution. However, if you don’t allow for
parallel edges or self loops, the degree distribution is likely only
going to be approximated.

2. Stochastic block models can recreate a community structure by

taking as input a node partition and the probabilities of connecting
to nodes inside the same community and between communities.

3. The GN and LFR benchmark were created to test community

 generating realistic data 225

discovery algorithms. The GN benchmark creates equal size

communities and normal degree distributions, while LFR is able to
return power-law degree distributions and communities of varying
size.

4. Kronecker graphs are generated from a simple seed matrix to

which you recursively apply the Kronecker product, creating high
clustering networks with shifted power law degree distributions
and communities.

5. Alternatively, you can make a random geometric graph, by plac-

ing nodes uniformly on an n-dimensional space and connecting
nodes to all their closest neighbors, at a maximum distance that
you can set as parameter.

6. Finally, you can learn a neural network representation from your

original (set of) network(s), which will be able to generate more
synthetic networks with comparable properties to your original
one(s).

15.6 Exercises

1. Generate a configuration model with the same degree distribution

as the network in [Link]
[Link]. Perform the Kolmogorov-Smirnov test between the two
degree distributions.

2. Remove the self-loops and parallel edges from the synthetic

network you generated in the previous question. Note the % of
edges you lost. Re-perform the Kolmogorov-Smirnov test with the
original network’s degree distribution.

3. Generate an LFR benchmark with 100, 000 nodes, a degree expo-

nent α = 3.13, a community exponent of 1.1, a mixing parameter
µ = 0.1, average degree of 10, and minimum community size of
10, 000. (Note: there’s a networkx function to do this). Can you
recover the α value by fitting the degree distribution?

4. Use kron function from numpy to implement a Kronecker graph

generator. Plot the CCDF degree distribution of a Kronecker graph
with the following seed matrix multiplied 4 times (setting the main
diagonal to zero once you’re done):
 
1 1 1 0
1 1 1 0
A=
 

1 1 1 1
0 0 1 1
16
Evaluating Statistical Significance

One of the big issues when it comes to analyzing complex networks

is that, usually, you only have one network to base your observations
on. Therefore, whenever you observe a given property – power
law degree distribution, reciprocity, clustering – you don’t have the
statistical power to claim that what you’re observing is interesting.
You need to have multiple versions of your network, a null model, to
test your observation. If keeping everything fixed about a network
minus the property of interest gives you something indistinguishable
from your observation, then you know that the particular feature
arose at random. There is no fundamental non-random force behind
it.
To do so, you need to generate a (set of) synthetic graph(s), which
is why I put this task in this part of the book. In other words, you
consider your observed network as part of a family of networks,
which all have the same fixed properties. Then you ask if the one you
did not fix is also a typical characteristic of this family of networks. If
it is, then it’s not an interesting discovery. If it isn’t, the deviation of
the network from its family is interesting.
As far as I know, there are two ways to generate this network
family: the easy way – network shuffling, Section 16.1 –, and the right
way – the Exponential Random Graph approach, Section 16.2.

16.1 Network Shuffling

Network shuffling is a way to generate synthetic networks that is

based on directly manipulating your observed network. At a fun-
damental level, it is a process of rewiring edges for a given number
of times, until we think that we are sufficiently far from the start-
ing point. I call this the “easy” way because, as we’ll see in a mo-
ment, the process is usually straightforward. On the other hand, this
method is significantly less rigorous than the Exponential Random
Graph model (ERGM), and thus should be used only when you don’t
 evaluating statistical significance 227

1 3 1 3 Figure 16.1: The edge swap

procedure.

2 4 2 4
(a) (b)

need the statistical power ERGM can give you.

The fundamental basis of the network shuffling model is the edge
swap operation. Figure 16.1 depicts it in all its simplicity. You pick
two pairs of connected nodes, in this examples node 1 is connected
to node 2, and node 3 is connected to node 4. Then, you flip the edge
around, deleting (1, 2) and replacing it with (1, 3), and deleting (3, 4)
replacing it with (2, 4). If you do this enough times, the resulting
network will be quite different from your original one. However, it
will still have the same number of nodes, the same number of edges,
and the same degree distribution – also in case of a directed network,
provided that you always swap edges in the correct direction.
This procedure is usually performed in network games, where 1
Noga Alon, Erik D Demaine, Moham-
edges are rewired with some objective in mind1 . Note that the num- mad T Hajiaghayi, and Tom Leighton.
ber of swaps to perform before stopping is a non trivial quantity to Basic network creation games. SIAM
Journal on Discrete Mathematics, 27(2):
evaluate2 .
656–668, 2013
An attentive reader will surely notice that this result is practically 2
Giulio Bottazzi and Davide Pirino.
the same as the one you would obtain from a configuration model. Measuring industry relatedness and
corporate coherence. 2010
However the two approaches have completely different objectives.
The configuration model wants to simply generate a network with
more or less the same degree distribution. The networks generated
by shuffling are significantly more similar to the original network
than the ones obtained from a configuration model, because they
need to be compared to it.
This becomes more obvious once you explore the differences with
the configuration model more in depth. First, edge swapping is
always possible, while at some point in the configuration model pro-
cess you might have to create self-loops or parallel edges. You cannot
create self-loops in network shuffling unless there were already self
loops in the original network. You can easily avoid parallel edges by
checking that your node pairs are not connected – for instance, you
can reject the operation in the example in Figure 16.1 if either the
(1, 3) or the (2, 4) edges are already in the network.
Second, what I explained is only the simplest way to perform
network shuffling. You can add a few more features that are hard to
embed in a configuration model. For instance, you can keep fixed
the number of connected components, by rejecting an edge swap if it
228 the atlas for the aspiring network scientist

would disconnect more nodes, or join two different components. You

can keep the clustering fixed, by making sure to keep the number of
triads and triangles constant. You can preserve the communities, by
only allowing edge swaps inside the clusters. And so on.
If all you want to do is to have a randomized version of your orig-
inal network, then you’re done. But, since I mentioned the problem
of determining the statistical significance of your observations, let’s
push on. How would you use the networks generated via edge swap
for such a task? I usually apply the following procedure:

1. Fix all reasonable properties of the network (at least number of

nodes, edges, and degree distribution) except the one of interest;

2. Generate a large set of shuffled networks, with independent

shuffles – from 100 to 10, 000;

3. Calculate the property of interest in the observed and in the

generated networks;

4. Estimate the distribution of the property in the shuffled networks

and calculate how far from the expectation the observed value is.

1200 900
Figure 16.2: The edge swap
900
# Null Models

# Null Models

600 statistical test. The plot shows

600 how many null models (y-axis)
300
300 scored a given value of the
property of interest (in this
0 0
0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5 0.6 case clustering, x-axis). The
Clustering Clustering
blue vertical bar shows the
(a) (b) observation.
Usually, this ends up with a plot looking like Figure 16.2(a) or
16.2(b). The histogram shows how many null models scored a value
of the property of interest in a given interval. The blue vertical bar
shows the value for the observed network.
In the easiest scenario, the null model will show a nice normal or
pseudo-normal distribution, making the estimation of statistical sig-
nificance easier. That’s the case for the figures I show, which means I
can simply calculate how many standard deviations from the average
my observation is. In the case of Figure 16.2(a) the observation is
significantly higher than expectation, given it’s three standard devi-
ations away from the average. That is not true for Figure 16.2(b): in
that case, we cannot reject the null explanation. The observation is
less than a standard deviation away from null expectation.
By the way, the thing that I call “number of standard deviations
above (or below) average” is know as z-score. You can automatically
 evaluating statistical significance 229

convert from the z-score into a p-value, provided that you know
whether you’re interested in a one-sided or a two-sided test. The
one-sided test means that your success is exclusively on one side of
the distribution – e.g. you want to score more than average, you’re 3
When discussing discoveries in
not interested whether your score is significantly below average3 (or physics, you’ll hear often the term “five
vice versa). sigma” (5σ) thrown around. This means
a z-score equal to 5. In turn, this can
Of course, if the null model distribution is not pseudo-normal,
be converted to a (one-sided) p-value
estimating the statistical significance is a bit trickier. We don’t need to lower than 10−6 , way lower than the
go into that, because we’re about to learn how to perform this task in p < 0.01 you’ll see in other fields. For
p < 0.01, you’re looking at a z-score a
the “right” way, using ERGMs. bit higher than 2.3. I’m simplifying a lot
here, since this is not – and never will
be – a statistics book.
16.2 Exponential Random Graphs

As introduced in this chapter, ERGM is a technique to generate a

set of graphs that have the same properties of an observed network. 4
Carolyn J Anderson, Stanley Wasser-
ERGM is also know in the literature as p* model4 , 5 . The observed man, and Bradley Crouch. A p* primer:
Logit models for social networks. Social
network is seen as the result of a stochastic (random) process with
networks, 21(1):37–66, 1999
a set of parameters. ERGM creates other networks using the same 5
Garry Robins, Pip Pattison, Yuval
process and the same parameters. The problem we need to solve Kalish, and Dean Lusher. An intro-
duction to exponential random graph
to generate an exponential random graph ensemble of networks is
(p*) models for social networks. Social
figuring out which parameter values to use. This is usually achieved networks, 29(2):173–191, 2007
through maximizing likelihood.

Figure 16.3: An illustrative ex-

ample of the ERGM process. (a)
Observed network. (b) Unlikely
random network. (c) Random
network more likely to be in the
same family of (a) than (b). (d)
Most likely random network.
(a) (b) (c) (d) The parameters used to gener-
The sketch of the solution is the following. Suppose you’re observ- ate it are more likely to be the
ing a graph G. There is an immensely large set of other random G ones of the family of random
graphs we could have observed: they are those we could generate graphs to which (a) belongs.
with a random process with a given set of parameters (same number
of nodes, edges, ...). However, some of these random graphs are more
likely than others to be observed – namely, the ones most similar to
G. Knowing this, we can identify the parameters values these likely
Gs have in common. Once we find these values, we can generate
an arbitrary number of graphs with them. We can use them as new
synthetic graphs for our purposes, or we can test them against the
observed G to verify whether they also share with it some other prop-
erty we did not fix. Figure 16.3 provides a vignette of this process.
230 the atlas for the aspiring network scientist

If you talk statistics, the following process might give you an

inkling about how ERGMs work. In this scenario, we consider an
edge as a random variable. In the simplest case of unweighted
network, this will be a binary variable, equal to one if the edge is
present, and to zero if it isn’t. You hypothesize what sort of process
might be the one determining the edge presence in your network.
This is sort of similar to estimating a logistic regression. You have
a binary outcome (edge present/absent) and a set of variables that
might be able to predict its value.

12 9
Au,v ku kv Au,v ku kv
10
4 1 9 4 .92 9 4
2 8
11
1 9 3 .87 9 3
10
7 1 9 3 .87 9 3 12
3
1 1 9 2 .8 9 2 5
11 1 9 2 .8 9 2 1
6
5
9 1 9 2 .8 9 2
6 3
0 9 1 .7 9 1 4
... ... ... ... ... ... 7
8 2
(b) (c)
(a) (d)
Let’s make an example. Figure 16.4(a) represents our observed Figure 16.4: Step-by-step exam-
graph. I generated it with a configuration model with the degree ple of a simple ERGM proces.
sequence (9, 4, 3, 3, 2, 2, 2, 1, 1, 1, 1, 1), but the ERGM doesn’t know (a) Observed network. (b) Ob-
that. The edge presence, the outcome, is our adjacency matrix A. So served edge table (only first
the edge between u and v is Au,v . I now make an hypothesis: the seven rows). Au,v is one if the
degree of a node influences its likelihood of getting a connection. Or, edge is present, zero otherwise;
in mathematical terms, Au,v = β 1 k u + β 2 k v . The degrees of u and k u and k v are the degrees of
v (k u and k v ) can be used to predict the probability of existence of the two nodes. (c) Result of the
an edge. This is equivalent of running a logit regression on an edge logit regression (only first seven
table like the one in Figure 16.4(b): we’re trying to predict the binary rows). Au,v is the estimated
Au,v variable using the degrees of u and v. probability of the edge existing.
Once the logit model is done, for each (u, v) pair we have a proba- (d) An extracted ERGM from
bility of its existence: Au,v (Figure 16.4(c)). We can now flip a loaded the edge probabilities in (c).
coin for each node pair and add the edge in case of success. Au,v is
determined by the β 1 and β 2 parameters. Since they are the result
of a logit model estimation, they are the ones most likely to describe
the family of random graphs from which we extracted the observed
G. By using Figure 16.4(c) to generate a new graph (e.g. the one in
Figure 16.4(d)), we’re sure to extract a graph from the same family
that generated the original one – at least when it comes to its degree
distribution.
So far, I simplified the process for the sake of intuition. For in-
 evaluating statistical significance 231

stance, I assumed that the likelihood of an edge only depends on a

node’s characteristic – in this case the degree. This is not necessarily
the case in the general ERGM. You can plug in all complex structures
you can express mathematically. For instance, you can ensure triadic
closure to preserve the clustering coefficient or other, more complex,
motifs. I also assumed the functional form of the model, namely a
linear one. The degrees of the two nodes interact linearly to give us
the result. That might not be the case.
We can describe the full model making no such assumptions as:
!
1
0
Pr ( A = A ) = exp
B ∑ β g g A0 .
g

There’s a bit to unpack here:

• Pr ( A = A0 ) is the probability that the adjacency matrix A we

extract is equal to a given adjacency matrix A0 , dependent on A0 ’s
characteristics.

• exp is the exponential function. We use it to define the probability

because exponentials come from maximum entropy distributions.
We want to use a function that can have the highest possible
entropy while still having a positive and definite mean – which
is necessary to define a probability (see Section 2.2). “Highest
possible entropy” simply means that whatever statistic we haven’t
incorporated in the model will be “as random as possible”.

• g is a graph configuration. It can be any pattern, for instance a

triangle, or a clique of four nodes, or even just an edge.

• β g is the parameter corresponding to this particular configuration.

In our previous example, it is the thing telling you how much the
degree of a node influences the connection probability. This is
the knob you have to use to maximize your quality function. The
“right” β g value is the one best describing your data.

• g A0 is a function applying pattern g to A0 . It tells you whether the

pattern is in the network. Mathematically: g A0 = ∏ A0uv , which
A0uv ∈ A0
means that g A0 = 1 if and only if all parts of g are in A0 .

• B is simply a normalization parameter needed to ensure that the

rest of the equation is a proper probability distribution – i.e. that it
sums to one.

To use human language: the probability of observing an adja-

cency matrix A is the probability of extracting a random A0 from all
possible adjacency matrices, weighted by how well A0 ’s topological
232 the atlas for the aspiring network scientist

properties fit the β parameters we observed in our original graph,

over the patterns g that interest us – any other pattern not in g is
assumed to appear entirely at random.
Such a model can have lots of β parameters. That is why usually
there is an additional step of parameter reduction, through what
we call “homogeneity constraints”. For instance you could have
a parameter for each node, telling us how likely that node is to
reciprocate connections. However, the homogeneity assumption says
that – most likely – all nodes in the same network have more or less
the same tendency of reciprocating connections. Thus, rather than
having |V | reciprocity parameters, you have a single, network-wide,
one.
This functional form is a general version of more specific ones that 6
Paul W Holland and Samuel Lein-
were studied in the literature in the eighties: p1 models6 and Markov hardt. An exponential family of
graphs7 . probability distributions for directed
graphs. Journal of the american Statistical
To understand a bit more the magic behind the formula I just
association, 76(373):33–50, 1981
presented, let’s consider a few special cases. Given their general form, 7
Ove Frank and David Strauss. Markov
ERGMs include many of the network models we saw so far. For graphs. Journal of the american Statistical
association, 81(395):832–842, 1986
instance, we can represent a Gn,p model, by noting that, in this case,
edges are all independent to each other. Without the homogeneity
assumption, we would have a parameter for each pair of nodes,
giving us the equation:
!
1
0
Pr ( A = A ) = exp
B ∑ β u,v A0u,v .
u,v

In this case, the graph pattern g is a single u, v edge. Since g A0

is equal to one if A0 contains pattern g, it reduces to A0u,v , which is
one if A0 contains the u, v edge. Further, we have a different β u,v
(β g ) per edge. Edges present in our observed network will have
corresponding high β u,v parameters, while absent edges will have
β u,v values close to zero. This in turn implies that an A0 is likely to be
extracted if it has edges attached to high β u,v values.
However, as we said, this is too many parameters. If we apply the
homogeneity assumption, we will just say that any pair of nodes has
the same probability p of connecting – which is the same assumption
of the Gn,p model. This means that we get rid of a lot of parameters,
which are substituted by the single parameter p:
!
1
0
Pr ( A = A ) = exp
B ∑ pA0u,v .
u,v

Since ∑ A0u,v is simply the number of edges | E0 |, this simplifies to:

u,v

1
Pr ( A = A0 ) = exp p| E0 | ,


B
 evaluating statistical significance 233

which is exactly a Gn,p model: a graph whose edges are all equally
likely to be observed. We can also simulate a stochastic blockmodel
by not reducing all connection probabilities to p but by having mul-
tiple ps for each block (and for inter-block connections). If you have
a directed graph you can represent reciprocity with the probability
p1 of a node to reciprocate the connection, adding a term to the Gn,p
model:

1
Pr ( A = A0 ) = exp p| E0 | + p1 R( A0 ) .


B
Here R( A0 ) is the number of reciprocated ties. You can make 8
Emmanuel Lazega and Marijtje
edges dependent on node attributes as researchers do in p2 models8 , 9 . Van Duijn. Position in formal structure,
Finally, you can also plug higher-order structures in the model, for personal characteristics and choices
of advisors in a law firm: A logistic
instance: regression model for dyadic network
data. Social networks, 19(4):375–397, 1997
1
Pr ( A = A0 ) = exp p| E0 | + τT ( A0 ) ,

9
Marijtje AJ Van Duijn, Tom AB
B Snijders, and Bonne JH Zijlstra. p2:
a random effects model with covariates
where – under the homogeneity assumption – T ( A0 ) is the number for directed graphs. Statistica Neerlandica,
of triangles in A0 . This way, you can also control the transitivity of 58(2):234–254, 2004
the graph.
These models can be very difficult to solve analytically for all
but the simplest networks. Modern techniques rely on Monte Carlo 10
Tom AB Snijders. Markov chain monte
maximum likelihood estimation10 , 11 . We don’t need to go too much carlo estimation of exponential random
into details on how these methods work, but these work in the line of graph models. Journal of Social Structure,
3(2):1–40, 2002
any Markov chain Monte Carlo method12 . However, if your network 11
Tom AB Snijders, Philippa E Pattison,
is dense, your estimation might need to take an exponentially large Garry L Robins, and Mark S Handcock.
number of samples to estimate your βs13 . There are ways to get New specifications for exponential
random graph models. Sociological
around this problem by expanding the ERG model14 , but by now methodology, 36(1):99–153, 2006
we’re already way over my head and I don’t think I can characterize 12
Walter R Gilks, Sylvia Richardson,
this fairly. and David Spiegelhalter. Markov chain
Monte Carlo in practice. Chapman and
How does all of this look like in practice? The result of your model Hall/CRC, 1995
might look like something from Figure 16.5. Here we decide to have 13
Shankar Bhamidi, Guy Bresler, and
four parameters: a single edge (this is always going to be present Allan Sly. Mixing time of exponential
random graphs. In 2008 49th Annual
in any ERGM), a chain of three nodes, a star of four nodes, and IEEE Symposium on Foundations of
a triangle. Each motif has a likelihood parameter: the higher the Computer Science, pages 803–812. IEEE,
2008
parameter the more likely the pattern. Negative values mean that the 14
Arun G Chandrasekhar and
pattern is less likely than chance to appear. Matthew O Jackson. Tractable and
The negative value for simple edges means that the network is consistent random graph models.
Technical report, National Bureau of
sparse: two nodes are unlikely to be connected. The positive value Economic Research, 2014
for the triangle means that triangles tend to close: when you have a
triad, it is more likely than chance to have the third edge. The other
two configurations are not significantly different from zero (you can’t
tell because I omitted the standard errors, but trust me on that). Thus
we should not emphasize their interpretation too much.
On the right side of the figure you can see a potential network
234 the atlas for the aspiring network scientist

β
Figure 16.5: On the left we
have the estimated parameters
-4.27
from the observation for four
patterns, with positive values
1.09 indicating a “more than chance”
occurrence of the pattern, and
-0.67 negative values a “less than
chance”. On the right we have
a likely network extracted from
1.32
the set of ERGM with the given
parameters.
15
John F Padgett and Christopher K
that is very likely to be extracted by this ERGM. In fact, I cheated a
Ansell. Robust action and the rise of the
bit, because that is the network on which I fitted the model. It is the medici, 1400-1434. American journal of
famous graph mapping the business relationship between Florentine sociology, 98(6):1259–1319, 1993
16
Skyler J Cranmer and Bruce A
families in the Renaissance15 .
Desmarais. Inferential network analysis
In this chapter I presented only the simplest of the ERGM forms. with exponential random graph models.
Recent research has shifted to more sophisticated models. A few of Political analysis, 19(1):66–86, 2011

those are:
17
Steve Hanneke, Wenjie Fu, Eric P
Xing, et al. Discrete temporal models
of social networks. Electronic Journal of
• Longitudinal ERGMs16 , which are specialized to deal with net- Statistics, 4:585–605, 2010
works that are evolving over time, for instance co-sponsorship of 18
Pavel N Krivitsky and Mark S Hand-
cock. A separable model for dynamic
bills in the US Congress – two representatives might co-sponsor a networks. Journal of the Royal Statistical
bill in one year, but not in another; Society. Series B, Statistical Methodology,
76(1):29, 2014
• Similarly, TERGMs17 introduce the temporal aspect in ERGMS. 19
Bruce A Desmarais and Skyler J
This contains the “separable” TERGMs18 which works on discrete Cranmer. Statistical inference for
valued-edge networks: The generalized
models, rather than modeling the evolution as happening on a exponential random graph model. PloS
continuous time flow; one, 7(1):e30136, 2012
20
Pavel N Krivitsky. Exponential-family
• ERGMs that can take into account edge weights, initially only random graph models for valued
continuous weights19 , but subsequently also discrete ones20 ; networks. Electronic journal of statistics, 6:
1100, 2012
• ERGMs for multilayer networks21 . 21
Alberto Caimo and Isabella Gollini. A
multilayer exponential random graph
modelling approach for weighted
ERGMs have been successfully applied in many fields. For in-
networks. Computational Statistics & Data
stance, they help in cases in which longitudinal network data col- Analysis, 142:106825, 2020
lection is unfeasible – e.g. informal face to face contacts in certain 22
Pierre-Alexandre Balland, José An-
tonio Belso-Martínez, and Andrea
business clusters22 , or inside firms23 . In economics they are partic-
Morrison. The dynamics of technical
ularly useful because of their ability to estimate structural network and business knowledge networks in
parameters, extending conventional analyses that use, for instance, industrial clusters: Embeddedness,
status, or proximity? Economic Geography,
gravity models24 . To make an example, in migration a gravity model 92(1):35–60, 2016
would say that the number of migrants from country u to v is directly 23
Tom Broekel and Matté Hartog.
proportional to the size – in number of inhabitants – of the two coun- Explaining the structure of inter-
organizational networks using exponen-
tries, and inversely proportional to their distance. ERGMs allow you tial random graph models. Industry and
to model more complex interdependencies25 . Innovation, 20(3):277–295, 2013
 evaluating statistical significance 235

16.3 Summary 24
Tom Broekel, Pierre-Alexandre
Balland, Martijn Burger, and Frank van
Oort. Modeling knowledge networks
1. Network shuffling is a way to create a null version of your net- in economic geography: a discussion
work, created through edge swapping. In edge swapping, you pick of four methods. The annals of regional
two pairs of connected nodes and you rewire the edges to connect science, 53(2):423–452, 2014
25
Michael Windzio. The network of
nodes from the other pair. global migration 1990–2013: Using
ergms to test theories of migration
2. Once you generate thousands of null versions of your network, between countries. Social Networks, 53:
you can test a property of interest and obtain an indication of 20–29, 2018
how statistically significant your observation is, by counting the
number of standard deviations between the observation and the
null average.

3. In Exponential Random Graphs you use a series of characteristics

of the network of interest as a predictor of the presence of an edge
between two nodes.

4. Once you know the relationship between these parameters and

the presence of an edge, you can randomly extract graphs that are
likely results of such predictors.

5. There is a trade off between the number of parameters you can

use and the complexity of the extraction process. Many different
heuristics have been proposed to sample the space of all possible
ERGMs.

16.4 Exercises

1. Perform 1, 000 edge swaps, creating a null version of the network

in [Link] Make
sure you don’t create parallel edges. Calculate the Kolmogorov-
Smirnov distance between the two degree distributions. Can you
tell the difference?

2. Do you get larger KS distances if you perform 2, 000 swaps? Do

you get smaller KS distances if you perform 500?

3. Generate 50 Gn,m null versions of the network in [Link]

[Link]/exercises/16/3/[Link], respecting the num-
ber of nodes and edges. Derive the number of standard deviations
between the observed values and the null average of clustering
and average path length. (Consider only the largest connected
component) Which of these two is statistically significant?

4. Repeat the experiment in the previous question, but now generate

50 Watts-Strogatz small world models, with the same number of
nodes as the original network and setting k = 16 and p = 0.1.
 Part V

Spreading Processes
17
Epidemics

So far, we’ve seen some dynamics you can embed in your network. In
Section 4.4 I showed you how to model graphs whose edges might
appear and disappear, while in the previous book part we’ve seen
models of network growth: nodes arrive steadily into the network
and we determine rules to connect them such as in the preferential
attachment model. This part deals with another type of dynamics on
networks. Here, edges don’t change, but nodes can transition into
different states.
The easiest metaphor to understand these processes is disease.
Normally, people are healthy: their bodies are in a homeostatic state
and they go about their merry day. However, they also constantly
enter into contact with pathogens. Most of the times, their immune
systems are competent enough to fend off the invasion. Sometimes
this does not happen. The person transitions into a different state:
they now are sick. Sickness might be permanent, but also temporary.
People can recover from most diseases. In some cases, recovery is
permanent, in others it isn’t.
These are all different states in which any individual might find
themselves at any given time. Like individuals, nodes too can change
state as time goes on. This book part will teach you all the possible
models we have to study these state transitions. In this chapter
we look at three models we defined to study the progression of 1
Romualdo Pastor-Satorras, Claudio
diseases through social networks1 . Note that such models can easily Castellano, Piet Van Mieghem, and
represent other forms of contagion, for instance the spread of viruses Alessandro Vespignani. Epidemic
processes in complex networks. Reviews
in computer and mobile networks2 . of modern physics, 87(3):925, 2015
We’re going to complicate these models in Chapter 18, to see how 2
Pu Wang, Marta C González, César A
different criteria for passing the diseases between friends affect the Hidalgo, and Albert-László Barabási.
Understanding the spreading patterns
final results. Then, in Chapter 19, we’ll see how the same model of mobile phone viruses. Science, 324
can be adapted to describe other network events, such as infrastruc- (5930):1071–1076, 2009
ture failure and word-of-mouth systems to aid a viral marketing
campaign.
Another complication is the one introduced by simplicial spread-
238 the atlas for the aspiring network scientist

ing. If you remember (Section 4.3 is there if you don’t), simplicial

networks contain these simplicial complexes, linking together multi-
ple nodes in higher-order structures (triangles, squares, etc). In some
cases, to be infected you might need to be part of such a complex
structure. For instance, peer-pressure might not work well if you’re
only connected by an edge. However, if you’re part of a simplicial
complex of three nodes, that might be enough to trigger you. The
combined pressure from your two friends overcomes your resistance.
For the purpose of this book, I’m going to leave it at that, and point 3
Iacopo Iacopini, Giovanni Petri, Alain
you to some recent literature on the subject3 . Barrat, and Vito Latora. Simplicial
models of social contagion. Nature
communications, 10(1):2485, 2019

17.1 SI

Sickness can be fatal for the individual and extremely debilitating

for entire societies. If tomorrow an epidemic sends to bed 90% of
the population at the same time – even if it doesn’t kill anyone –
4
This chapter was drafted before
COVID-19 happened, and it shows.
it can grind the planet to a halt4 . For this reason, humans have a
strong incentive to study contagion dynamics at large, to predict
whether such a situation might occur in the future. Researchers have
5
William Ogilvy Kermack and Ander-
son G McKendrick. A contribution to
developed simple models to describe the dynamics of diseases5 , 6 , 7 . the mathematical theory of epidemics.
These are usually known as “Compartmental models” – although I’ve Proceedings of the Royal Society of London.
Series A, Containing Papers of a Mathe-
been calling “state” what is traditionally known as “compartment”.
matical and Physical Character, 115(772):
The model divides individuals into two states. The first state is 700–721, 1927
called Susceptible. A person in the Susceptible state is... well... sus- 6
David Clayton, Michael Hills, and
A Pickles. Statistical models in epidemiol-
ceptible to contract a disease. This marks healthy people that show
ogy, volume 161. Oxford university press
no symptoms and are functioning properly. The second state is called Oxford, 1993
Infected. People in the Infected state – you’ll never believe it – have 7
Abdel R Omran. The epidemiologic
transition: a theory of the epidemiology
contracted the disease. We use S to indicate the set of individuals in
of population change. The Milbank
the Susceptible state, and I to indicate the set of individuals in the Quarterly, 83(4):731–757, 2005
Infected state.
The model allows for only one possible transition between states.

Figure 17.1: The schema un-

derlying the SI Model: two
possible states and one possible
S I transition.
 epidemics 239

The only thing that can happen in this model is the transition from
the Susceptible to the Infected state: S → I. In this world, the only
possible action is for a healthy person to contract the disease. Noth-
ing else is allowed.
Given that there are only two states (S and I) and only one transi-
tion (S → I), we call this the SI Model. Figure 17.1 shows the schema
fully defining the model. In practice, SI models diseases with no
recovery. An example would be some variants of the herpes virus.
Love goes by, herpes is forever.
There is one assumption underlying the traditional SI Model:
homogenous mixing – keep this in mind because it’s important. In
homogenous mixing, we assume that each susceptible individual has
the same probability to come into contact with an infected person.
This is simply determined by the current fraction of the population in
the infected state. Once the susceptible individual meets an infected,
there is a probability that they will transition into the I state too. This
probability is a parameter of the model, traditionally indicated by β.
If β = 1, any contact with an infected will transmit the disease, while
if β = 0.2, you have an 20% chance to contract the disease.

1
0.9 Figure 17.2: The solution of the
0.8
SI Model for different β values.
Infected Ratio

0.7
0.6
β = 0.175 The plot reports on the y axis
0.5
0.4 β = 0.15
0.3 the share of infected individu-
β = 0.125
0.2
0.1
als (i = | I |/(| I | + |S|)) at a given
β = 0.1
0
0 20 40 60 80 100 120 140 160 180 200
time step (x axis).
Time

Once you have β you can solve the SI Model. Usually, the way
it’s done is assuming that at the first time step you have a set of one
or more patient zeros scattered randomly into society. Then, you
track the ratio of people in the I status as time goes on, which is
| I |/(| I | + |S|). This usually generates a plot like the one in Figure 17.2.
SI models have the same signature. At first, the ratio of infected
individuals grows slowly, because there are few people in the I state.
Then, as soon as I expands a little, we see an exponential growth, as
more and more people have a chance to meet an infected individual.
After a critical point, the growth of I slows down, because there
aren’t many people left in S to infect.
Eventually, all SI models stop when every single individual is in
the set I and so no one else can transition. All SI Models, no matter
the value of β will end up with a complete infection, where S is
empty and I contains the entirety of society. The only thing β affects
240 the atlas for the aspiring network scientist

– as you can see from Figure 17.2 – is the speed of the system: when
the exponential growth of I starts to kick in and when S gets emptied
out.
We can re-tell the story I’ve just exposed in mathematical form.
In our SI model, the probability that an infected individual meets a
susceptible one is simply the number of susceptible individuals over
the total population, because of the homogenous mixing hypothesis:
|S|/|V | (remember |V | is our number of nodes). There are | I | infected
individuals, each with k̄ meetings (the average degree). Thus the total
| I ||S|
number of meetings is k̄ . Since each meeting has a probability
|V |
| I ||S|
β of passing the disease, at each time step there are βk̄ new
|V |
infected people in I.
We can simplify the equation a bit, because | I |/|V | and |S|/|V | are
related. They sum to one, since S and I are the only possible states
in which you can have a node. So, if we say i = | I |/|V |, that is, the
fraction of nodes in I, then |S|/|V | = 1 − i. So our formula becomes: 8
Note that, for simplicity, I’m only
it+1 = βk̄it (1 − it )8 , where t is the current time step. If we integrate including the addition to it+1 , not
over time, we can derive the fraction of infected nodes depending its full composition. So, pedantically,
the correct formula should be it+1 =
solely on the time step9 :
it + βk̄it (1 − it ), but that would make
the discussion harder to follow. This
i0 e βk̄t warning applies to all formulas with the
i= . time subscript.
1 − i0 + i0 e βk̄t 9
Albert-László Barabási et al. Network
This is the mathematical solution to the SI model with homoge- science. Cambridge university press, 2016
nous mixing, generating the plot in Figure 17.2. You can see why you
have an initial exponential growth at the beginning and a flat growth
at the end. If i0 ∼ 0, then the denominator is 1 and the numerator is
dominated by the e βk̄t factor: exponential growth (very slow at the
beginning because multiplied with the small i0 ). When i0 ∼ 1, both
the denominator and the numerator reduce to e βk̄t , which means that,
in the end, i ∼ i0 ∼ 1, so there’s no growth.
Why did we go to the trouble of all this math? Because, at this
point, we have to tear down the homogenous mixing hypothesis. The
formulas will allow to see the difference better.
Homogenous mixing is based on the assumption that the more
people are infected, the more likely you’re going to be infected. In
practice, it assumes everybody is the same. In homogenous mixing,
the global social network is a lattice: a regular grid where each node
is connected only to its immediate neighbors. Figure 17.3 shows an
example of square lattice (Section 4.6): each node connects regularly
to four spatial neighbors. On a lattice, the infection spreads like water
10
This is a useful mental image: https:
filling a surface10 . //[Link]/wikipedia/
We know that real networks are not neat regular lattices. The commons/a/a6/SIR_model_simulated_
using_python.gif.
degree is distributed unevenly, with hubs having thousands of con-
 epidemics 241

Figure 17.3: The underlying

assumption of an SI model:
that social networks look like a
uniform lattice (a). Instead, the
degree is distributed differently
(b), with hubs – pointed by the
red arrow – having many more
connections and, thus, infection
(a) (b) chances.

nections – see Section 6.3. When the infection hits such a hub, it will
accelerate faster through the network. In fact, it is extremely easy to
infect an hub early on. Hubs have more connections, thus they are
more likely to be connected to one of your patient zeros. Those same
connections make them super-spreaders: once infected, the hub will
allow the disease to reach the rest of the network quickly. In fact,
when searching information in a peer-to-peer network, your best 11
Lada A Adamic, Rajan M Lukose,
guess is always to ask your neighbor with highest degree11 . Amit R Puniyani, and Bernardo A
To treat the SI model mathematically you have to first group nodes Huberman. Search in power-law
networks. Physical review E, 64(4):046135,
by their degree. Rather than solving for i – the fraction of infected
2001
nodes –, you solve for ik : the fraction of infected nodes of degree k.
The formula for a network-aware SI model is similar as the one we
saw for the vanilla SI model:

ik,t+1 = βk f k (1 − ik,t ).

The two differences are that: (i) we replace the average degree k̄
with the actual node’s degree k, and (ii) rather than using ik,t we use
f k – a function of the degree k. This is because real world networks
typically have degree correlations: if you have a degree k the degree
of your neighbors is usually not random (see Section 27.1 for more).
If it were random, then we could simply use ik,t , because the number
of infected individuals around you should be proportional to the
current infection rate. But it isn’t: in presence of degree correlations,
if you have k neighbors then there exists a function f k able to predict
how many neighbors they have. Thus the likelihood of a node of
degree k of having infected neighbors is specific to its degree, and not
(only) dependent on ik,t . 12
Romualdo Pastor-Satorras and
If you do the proper derivations12 , you’ll discover that in a Gn,p Alessandro Vespignani. Epidemic dy-
network the dynamics have the same functional form to the ones of namics and endemic states in complex
networks. Physical Review E, 63(6):066117,
the homogeneous mixing, as Figure 17.4 shows. In Gn,p the exponen- 2001a
tial rises faster at the beginning – due to the few outliers with high
degree – and tails off slower at the end – due to the outliers with
low degree – but the rising and falling of the infection rates is still an
242 the atlas for the aspiring network scientist

1
0.9 Figure 17.4: The solution of the
0.8
SI Model for different β values
Infected Ratio

0.7
0.6
β = 0.175 (no net) β = 0.175 (Gn,p)
0.5 in homogeneous mixing (reds)
0.4 β = 0.15 (no net) β = 0.15 (Gn,p)
0.3 and Gn,p graphs (blues). The
β = 0.125 (no net) β = 0.125 (Gn,p)
0.2
0.1 β = 0.1 (Gn,p)
plot reports on the y axis the
β = 0.1 (no net)
0
0 50 100 150 200 250 300
share of infected individuals
Time (i = | I |/(| I | + |S|)) at a given
time step (x axis).

exponential. It also depends, obviously, on the average degree you

give to the lattice and to the Gn,p graph.
Is that it? Did I really throw Greek letters at you for such an
underwhelming discovery? Of course not. Remember that Gn,p is a
poor approximation of social networks, especially when it comes to
degree distributions. Let’s look at what happens when you have a
network with a power law degree distribution.

1
0.9 Figure 17.5: The solution of the
β = 0.175 (α = 2) β = 0.125 (α = 3)
0.8
SI Model for different β values
Infected Ratio

0.7 β = 0.15 (α = 2) β = 0.1 (α = 3)

0.6
β = 0.125 (α = 2) β = 0.175 (Gn,p)
0.5 for networks with power law
0.4 β = 0.1 (α = 2) β = 0.15 (Gn,p)
0.3 degree distribution with expo-
β = 0.175 (α = 3) β = 0.125 (Gn,p)
0.2
0.1 β = 0.1 (Gn,p)
nent α = 2 (reds), α = 3 (blues)
β = 0.15 (α = 3)
0
0 50 100 150 200 250 300
and Gn,p graphs (greens). The
Time plot reports on the y axis the
share of infected individuals
(i = | I |/(| I | + |S|)) at a given
Figure 17.5 shows you the results of a bunch of simulations on time step (x axis).
networks with different degree distributions. The slowest infections
(in green) happen for Gn,p graphs. When looking at a power law
random network, the thing that matters the most is the exponent of
the degree distribution, α (for a refresher on its meaning, see Section
6.3).
If α = 3 we have not-so-large hubs. These hubs contribute enor-
mously to the speed of infection: it is easy to catch them and, once
you do, the disease spreads faster. You can see how the exponential
growth regimes, for the blue data series, are much steeper than in
the green Gn,p cases. Even for β = 0.1, a mildly contagious disease, a
power law degree distribution with α = 3 gets infected faster than a
Gn,p network with a much more aggressive disease (with β = 0.175,
almost twice as infectious!).
The same comparison applies when pitting the α = 3 case with
α = 2 networks. In the latter case, there’s not even a recognizable
 epidemics 243

exponential warm up any more (in red in Figure 17.5). You know
that, no matter where you started, you’re going to hit the largest hub
of the network at the second time step t = 2, because it is connected
to practically every node. And, since it is connected to practically
every node, at t = 3 you’ll have almost the entire network infected.
In fact, I ran the simulations from Figure 17.5 on imperfect and
finite power law models. Theoretically, if you had a perfect infinite
power law network, infection would be instantaneous for any non-zero
value of β. Meaning that, no matter how infectious a disease is, with
α = 2 it will infect the entire network almost immediately. And
things get even more complicated when you add to the mix the fact 13
Eugenio Valdano, Luca Ferreri, Chiara
that networks evolve over time13 . Scary thought, isn’t it? Poletto, and Vittoria Colizza. Analytical
computation of the epidemic threshold
on temporal networks. Physical Review X,
17.2 SIS 5(2):021005, 2015

Just like in the SI model, also in the SIS model nodes can either be 14
Herbert W Hethcote. Three basic
Susceptible or Infected14 . However, the SIS model adds a transi- epidemiological models. In Applied
tion. Where in SI you could only get infected without possibility of mathematical ecology, pages 119–144.
Springer, 1989
recovery (S → I), in SIS you can heal (I → S).
Thus the SIS model requires a new parameter. The first one,
shared with SI, is β: the probability that you will contract the disease
after meeting an infected individual. Once you’re infected, you also
have a recovery rate: µ. µ is the probability that you will transition
from I to S at each time step. High values of µ mean that recovering
from the disease is fast and easy. Note that recovery puts you back
to the Susceptible state, thus you can catch the disease again in the
future.

Figure 17.6: The schema un-

derlying the SIS Model: two
possible states and two possible
transitions.

S I
Figure 17.6 shows the schema fully defining the model. In practice,
SIS models disease with recovery and relapse. An example would
be the general umbrella of the flu family. Once you heal from a
particular strain of the flu you’re unlikely to fall ill again under the
244 the atlas for the aspiring network scientist

same strain. However, you can easily catch a similar strain, thus
cycling each year between the S and I states.
The presence of µ changes the outcome of the model. SI models
always reach full saturation: eventually, every node will end up in
status I. For SIS models that is not true, because a certain fraction
of nodes – µ – heal at each time step. The interplay between the
recovery rate µ, the infection rate β, and the average degree k̄ deter-
mines the asymptotic size of I: the share of infected nodes as time
approaches infinity (t → ∞). To see how, let’s look at the math again.

% Infected
Figure 17.7: The typical evolu-
Depends on μ and β
tion of an endemic SIS model:
< the equilibrium state is the one
100%
in which a constant fraction
Endemic state i < 1 contracted the disease.
The rate at which infected peo-
ple recover and the infection
rate are perfectly balanced, as
all things should be.

Time

The SI model could be described by the formula it+1 = βk̄it (1 − it ).

If, at each time step, a fraction µ of the infected nodes i recovers,
we just have to remove it from i. Thus, the SIS model is simply
it+1 = βk̄it (1 − it ) − µit . This should raise your eyebrow. As it grows,
so does µit , obviously. Eventually, βk̄it (1 − it ) = µit : that is when
the share of infected nodes i doesn’t grow any more. We reached the
endemic state where the number of people recovering is perfectly
balanced by the new infected. Figure 17.7 depicts this situation.
Is it possible that βk̄it (1 − it ) < µit ? Meaning: is there a situation
when people are recovering faster than new infected pop up? Yes!
We can get rid of a disease in the SIS model. If you do the proper 15
Romualdo Pastor-Satorras and
derivations15 , you discover that the magic value of µ for that to Alessandro Vespignani. Epidemic
spreading in scale-free networks.
happen is βk̄. If µ < βk̄, recovery isn’t fast enough to escape the Physical review letters, 86(14):3200, 2001b
endemic state, and you’re in the situation we saw in Figure 17.7. But,
if µ > βk̄, eventually the endemic state is the one for which i = 0!
Congratulations! No more infected people. You defeated the disease.
The evolution of the outbreak looks like what I sketch in Figure 17.8.
In my simulations for Figure 17.8 I set k̄ = 1, to make the compar-
isons easier. You can see that, when µ ≥ β, eventually the disease
dies out. The case for which µ < β reaches the endemic state, show-
ing that the disease will persist in the population. For µ = β, the
disease dies out, although not as quickly as for µ > β.
 epidemics 245

100

10-1 Figure 17.8: The solution of the

SIS Model, keeping β fixed but
Infected Ratio

-2
10

10-3 varying µ. The plot reports on

β = 0.2, µ = 0.1
10-4 the y axis the share of infected
β = 0.2, µ = 0.2
10-5 individuals (i = | I |/(| I | + |S|))
β = 0.2, µ = 0.3
10-6 at a given time step (x axis).
0 50 100 150 200
Time

The relationship between µ and β is so important that we can

study the evolution of infections in a network according to their ratio
λ = β/µ. Since we ignore the degree when calculating λ, we know
that λ depends exclusively by the pathogen’s characteristics. We
just saw in Figure 17.8 that, in some cases, the SIS model predicts
a non-zero endemic state – there are always at least i > 0 infected
individuals – and, in other cases, the disease dies out – thus i = 0. So
there must be a critical value of λ that make us transition between
the endemic and non-endemic state.
In Gn,p networks with homogeneous mixing this critical λ value
depends on the average degree of the network k̄. Specifically, if
λ > 1/(k̄ + 1) then the disease will be endemic. If λ is below that
threshold, then the pathogen will eventually disappear. Note that,
in a Gn,p graph, k̄ is always positive and equal to p|V | (see Section
13.2 for a refresher). This mean that you can find a value of λ below
the critical endemic threshold: the only way for 1/(k̄ + 1) to be equal
to zero would be if k̄ = ∞, which is clearly nonsense. The average
degree in a Gn,p graph cannot be infinite. Thus any Gn,p graph will
be resistant to a disease with a λ lower than 1/(k̄ + 1)
Surprising absolutely no one, when we drop the homogeneous
mixing assumption and we look at a preferential attachment network
the situation changes radically. Here, the critical value is k̄/k¯2 : the
average degree over the average squared degree – note that we
square the degrees and then we take the average, we don’t simply
raise the average degree to the power of 2. Here’s the problem: the
average degree in a preferential attachment network is low. But
the network contains large hubs with a ridiculously high degree:
squaring it eclipses the small contributions from the peripheral nodes
that kept k̄. In other words, as you add more and more nodes to
the network, k̄ remains constant and low – because you’re adding
peripheral nodes with low degree – but k¯2 grows fast. Each of those
new nodes tend to add to the degree of the largest hubs, because of
preferential attachment – shooting k¯2 in the stratosphere.
The consequence? Well, if k̄ stays constant – or even decreases –
246 the atlas for the aspiring network scientist

and k¯2 grows relatively to it, the critical threshold k̄/k¯2 tends to zero.
If we say that you have an endemic value if λ > k̄/k¯2 and k̄/k¯2 = 0,
then any disease, no matter β and µ, will be endemic in a network
with a power law degree distribution. Oops.

% Infected
at time ∞ Figure 17.9: The solution of the
SIS Model for λ. As λ grows
(x axis), I show the share of
infected individuals i at the
λ=β/μ endemic state (t → ∞).

Power Law
Random

I sum up the situation in Figure 17.9: in a power law random

network, no matter λ, the pathogen can always be endemic, even
if it’s not very infectious. In a Gn,p network you see that for some
values of λ greater than zero you do not have endemic infections,
thus the network’s topology has a big effect on the dynamics of the
epidemic. Heavy tailed degree distributions, which are ubiquitous in
reality, are closer to the power law line than to the Gn,p line, meaning
that we should expect to see a similar behavior in real networks.

17.3 SIR

The next step in modeling epidemics on networks is by considering

those diseases you can catch only once in your lifetime. Think about
the bubonic plague. If you have the bad luck of encountering the
Yersinia pestis, there are only two possible outcomes. Either you die,
or you survive. If you survive, your immune system is now trained to
recognize the bacterium and will not allow you to be infected again.
In either case, you are Removed from the outbreak.
Removed is exactly the state we add to the SI model in the SIR
model. Now the only two possible state transitions are S → I – when
you contract the disease – and I → R – when you heal or die. Figure
17.10 shows the schema fully defining the model.
The defining characteristic of a SIR model is its lack of endemicity.
Either the disease kills everybody, or every individual still alive has
 epidemics 247

Figure 17.10: The schema un-

derlying the SIR Model: three
possible states and two possible
transitions.

S I R

had the disease and healed. Figure 17.11 shows such a typical evolu-
tion. At the beginning, everybody is susceptible. Then, people start
getting infected, so I grows. R cannot start growing immediately,
as I is still too small for the recovery parameter µ to significantly
contribute to R size. As I grows, though, there are enough infected
individuals that start being removed. Eventually every I individual
transitions to R.

% of Pop
Figure 17.11: The typical evolu-
Susceptible Removed tion of an SIR model: after an
initial exponential growth of
the infected, the removed ratio
takes over until it occupies the
entire network.
Infected

Time

Note that the evolution of Figure 17.11, where eventually there

are only people in the R state, isn’t necessarily the only possible. If
you’re lucky, I empties out before S, meaning that the disease dies
out in R before every susceptible individual has had the privilege of
sneezing.
Mathematically speaking, the evolution of the recovery ratio
r = | R|/|V | is the simplest possible. At each time step, a fraction
µ of I transitions into R. In SIR, just like in SIS, µ is the recovery
parameter. So rt+1 = µit . The evolution of i is a bit trickier, but it 16
Michael Y Li and James S Muldowney.
boils down to making sure of removing the nodes in | R| from the Global stability for the seir model in
epidemiology. Mathematical biosciences,
potential pool of infected. 125(2):155–164, 1995
Of course, in the quest of making models more and more accurate 17
Michael Y Li, John R Graef, Liancheng
to fit the actualy dyamics of infections, you don’t have to stop with Wang, and János Karsai. Global
dynamics of a seir model with varying
the SIR model. In the literature you can find: SEIR, adding an “Ex- total population size. Mathematical
posed” status before the infection triggers in an individual16 , 17 ; you biosciences, 160(2):191–213, 1999
248 the atlas for the aspiring network scientist

can have an immune status M; relapsing to susceptibility in a SIRS 18

Chun-Hsien Li, Chiung-Chiou
model after being removed18 ; and, of course, combining everything Tsai, and Suh-Yuh Yang. Analysis of
together in a warm and fuzzy pile of states, in the MSEIRS model epidemic spreading of an sirs model
in complex heterogeneous networks.
(I wish I was kidding). As you might expect, the math becomes
Communications in Nonlinear Science and
fiendishly complicated and it’s just not worth delving into that for an Numerical Simulation, 19(4):1042–1054,
introductory chapter to network epidemics such as this one. 2014

This chapter is also by necessity just a superficial sketch of net-

work epidemics. There’s plenty more research on endemic and 19
Yang Wang, Deepayan Chakrabarti,
epidemic states and their relationship with network topology19 , 20 , 21 Chenxi Wang, and Christos Faloutsos.
that you can check if you find the topic fascinating. Epidemic spreading in real networks:
An eigenvalue viewpoint. In 22nd
International Symposium on Reliable
Distributed Systems, 2003. Proceedings.,
17.4 Summary pages 25–34. IEEE, 2003
20
Rick Durrett. Some features of the
1. In simple contagion epidemics models, nodes are in specific states spread of epidemics and information
given their exposure to the disease and can transition in different on a random graph. Proceedings of the
National Academy of Sciences, 2010
states according to simple contact rules. 21
Claudio Castellano and Romualdo
Pastor-Satorras. Thresholds for epidemic
2. In SI models there are two states: Susceptible and Infected. Nodes spreading in networks. Physical review
transition from S to I with a certain probability β if they have at letters, 105(21):218701, 2010

least an I neighbor.

3. All SI models end up with the entire network in the I state. β

determines how quickly this happens. Networks with a degree
distribution characterized by a low α exponent are infected more
quickly.

4. SIS models are like SI models, but nodes can transition back to S
state with a stochastic probability µ at each time step.

5. The λ = β/µ ratio determines whether the disease will be en-

demic or it will die out. In power law random networks, no matter
λ, the disease will always be endemic.

6. In SIR models, nodes in state I recover at a µ rate rather than

moving back to S. Eventually, all network will move to the R state,
and no disease can be endemic.

17.5 Exercises

1. Implement an SI model on the network at [Link]

eu/exercises/17/1/[Link]. Run it 10 times with different β val-
ues: 0.05, 0.1, and 0.2. For each run (in this and all following
questions) pick a random node and place it in the Infected state.
What’s the average time step in which each of those β infects 80%
of the network?
 epidemics 249

2. Run the same SI model on the network at [Link]

eu/exercises/17/2/[Link] as well. One of the two networks is
a Gn,p graph while the other has a power law degree distribution.
Can you tell which is which by how much the disease takes to
infect 80% of the network for the same starting conditions used in
the previous question?

3. Extend your SI model to an SIS. With β = 0.2, run the model

with µ values of 0.05, 0.1, and 0.2 on both networks used in the
previous questions. Run the SIS model, with a random node as a
starting Infected set, for 100 steps and plot the share of nodes in
the Infected state. For which of these values and networks do you
have an endemic state? How big is the set of nodes in state I com-
pared to the number of nodes in the network? (Note, randomness
might affect your results. Run the experiment multiple times)

4. Extend your SI model to an SIR. With β = 0.2, run the model for
400 steps with µ values of 0.01, 0.02, and 0.04 and plot the share of
nodes in the Removed state for both the networks used in Q1 and
Q2. How quickly does it converge to a full R state network?
18
Complex Contagion

You may or may not have noticed that, in the previous chapter, all
our models of epidemic contagion shared an assumption. Every time
a susceptible individual comes in contact with an infected individual,
they have a chance to become infected as well. If that doesn’t happen,
the healthy person is still in the susceptible pool. The next time step
represents a new occasion for them to contract the disease. And so
on, ad infinitum.
Without that assumption, the models wouldn’t be mathematically
tractable. For instance, if each node gets only one chance to be in-
fected, you can easily see how it is not given that a SI model would
eventually infect the entire network. In fact, it takes any β < 1 to
make that impossible. The first time you fail to infect somebody you
won’t get the chance to try again.
SI, SIS, and SIR models are useful and generated tons of great
insights. But this limitation allows them to model only rather specific
types of outbreak. We usually consider them models of simple
contagion. There are fundamentally two ways to make such models
more complex and realistic. They involve changing two things: (i) the
triggering mechanism, which is the condition regulating the S → I
transition, and (ii) the assumption that each individual gets infinite
chances to infect their neighbors.
We deal with the triggering mechanisms in Section 18.1 and
infection chances in Section 18.2. We also explore the possibilities of
interfering with the outbreak in Section 18.3, dedicated to epidemic
interventions.

18.1 Triggers

As I mentioned before, in the simple contagion we explored in Chap-

ter 17, one contact with an infected node is enough for you to have a
chance to be infected. The main difference between simple and com-
plex contagion is that, in the latter, you require reinforcement. You
 complex contagion 251

Figure 18.1: A simple introduc-

tion to complex contagion. (a)
A simple contagion where any
contact can and will transmit
the disease. (b) A complex
contagion where you need two
(a) (b) contacts to contract the disease.

can consider Figure 18.1 as the simplest possible introduction to this

concept. If we require a node to enter into contact with two infected
individuals rather than one, the figure shows that the outbreak from
a single seed is impossible.
In reality, complex contagion is more nuanced than this. A single
contact may or may not infect you, but if you have multiple con-
tacts your likelihood to transition into the I state grows. There are
fundamentally two types of reinforcement we can consider, which
are subtypes of complex contagion: Cascade and Threshold. Before
looking at them, though, let’s consider an easy extension of simple
contagion since, in a sense, it can be turned into the simplest possible
reinforcement mechanism.

Classical
In classical reinforcement you have an independent probability of
being infected for each of your neighbors that are infected. Note that
this is different from the simple contagion of Chapter 17: in there,
you get β chance to transition regardless whether you have one or
more infected neighbors. Here, more infected neighbors mean more
chances of infection.
If you have n sick friends, and you visit them one by one, at each
visit you toss a coin. To calculate the probability you are going to be
infected, it is easier to calculate the probability of not being infected
by any contact, and then invert it. If our parameter β tells us the
probability of being infected by a single contact, then (1 − β) is
the probability of not being infected. Since the coin tosses are all
independent, the probability of never being infected by any of the n
contacts is (1 − β)n . So the probability that at least one contact will
infect us is 1 − ((1 − β)n ).
Figure 18.2 shows a vignette of this process. The healthy individ-
ual has four neighbors, all of which are infected. Thus she has to
make four independent coin tosses, each of which has β chance to
succeed. Thus, the more infected neighbors the more likely she will
contract the disease. The difference with a simple SI model without
reinforcement is that in the simple SI model you always toss a sin-
252 the atlas for the aspiring network scientist

Figure 18.2: An example of

classical contagion. The healthy
individual performs an inde-
pendent check for contagion –
a coin toss – with each of their
neighbors.

gle coin at each time step, no matter how many infected neighbors
you have – as long as you have at least one. So, at each time step,
in simple SI the infection probability is β if you have 1 or n infected
neighbors. In classical complex SI, you have 1 − ((1 − β)n ) probability
of being infected. The whole difference between the two models is
that the latter depends on n, the number of your friends that are
infected.
The vignette makes clear why, in the classical model, it’s easy to
infect hubs: they have more neighbors. More neighbors mean that
they toss their coins much more often. This is what generates the
super-exponential – theoretically instantaneous – outbreak growth in
power law models with large hubs.

Threshold

The threshold model is a sophisticated version of the introductory

example I made with Figure 18.1. To be infected, you need multiple
infected neighbors. The threshold model adds a parameter, let’s call
it κ. If more than κ of your neighbors are infected, then they pass the 1
Mark Granovetter. Threshold models of
collective behavior. American journal of
infection to you1 . sociology, 83(6):1420–1443, 1978
For instance, if κ = 4, you need four infected friends to have a
chance to be infected. Note that you can still inject in this model the
β chance of infection, by saying that, once you clear the κ threshold,
you have a chance β < 1 to contract the disease. In this latter case, if
κ = 1, this model is the same as the simple SI without reinforcement:
as long as you have at least one infected neighbor, you toss your β
coin.
Figure 18.3 shows a vignette of this process. If we were in classical
contagion, the hub at the top would toss three coins with β chance of
getting infected at each check. In threshold contagion, since κ = 4,
 complex contagion 253

the probability of her being infected is zero. Threshold models

usually find an easy time to infect hubs, because we usually set κ
to be low. Any hub will have more infected friends than that. Any
κ > 1 renders peripheral nodes safe, since most of them have only
one connection.

I’m fine! Figure 18.3: An example of

threshold contagion. The
healthy individual checks
how many infected neighbors
she has. If they’re less than
κ, she’s fine. In this example,
κ = 4.

The threshold model is where epidemiology starts to blend in with 2

Mark Granovetter and Roland Soong.
Threshold models of diffusion and col-
sociology. Rather than modeling the spread of a virus, the threshold lective behavior. Journal of Mathematical
assumption works best when explaining the spread of a behavior. sociology, 9(3):165–179, 1983
The assumption is that individuals’ behavior depends on the number 3
Mark Granovetter and Roland Soong.
Threshold models of diversity: Chinese
of other individuals already engaging in that behavior2 . This can be restaurants, residential segregation,
used to explain racial segregation3 and customer demand4 . We’re and the spiral of silence. Sociological
going to dive in deep on the racial segregation angle when we’ll methodology, pages 69–104, 1988
4
Mark Granovetter and Roland Soong.
deal with homophily in social networks in Chapter 26. The customer
Threshold models of interpersonal
demand angle explains why variations of the threshold models are effects in consumer demand. Journal of
one of the favorite instruments of researchers involved in studying Economic Behavior & Organization, 7(1):
83–99, 1986
viral marketing. We’ll see more of that later on in this chapter.
You can spice up the threshold model by allowing κ to be a node-
dependent parameter, rather than a global one. This means that each
node v has a different κv activation threshold. Some might be con-
vinced to change their behavior by a single individual contact. Or,
to keep our epidemic metaphor, they might have a weak immune
system, prone to concede defeat to the disease after the first exposure.
Others require a high κv : their defining characteristic is being stub-
born, whether in their head or in their antibodies. When it comes to
social behavior, individuals’ thresholds may be influenced by many
factors: social economic status, education, age, personality, etc.
254 the atlas for the aspiring network scientist

Cascade

The cascade model is a straightforward variant of the threshold

model, in fact they were developed together by the same researchers.
However, their differences are important enough to mention them
separately. In the cascade model you also need reinforcement from
more than one neighbor to transition to the I state. However, while
in the threshold model this was governed by an absolute parameter κ,
here we use a relative one.
In other words, in the cascade model you need a fraction of your
5
Duncan J Watts. A simple model of
global cascades on random networks.
neighbors to be infected in order for you to be infected5 . In the cas- Proceedings of the National Academy of
cade model, the size of your neighborhood influences your likelihood Sciences, 99(9):5766–5771, 2002
to transition. In the threshold model it didn’t: whether you have one
or one hundred neighbors, you always need the same κ number of
them in the I state to consider transitioning.
So if you have four friends, but you need a fraction β > .75 to
transition, if only one, two, or three of them are infected you’re fine.
In such a condition, you need all of your neighbors to be infected in
order to get sick. Only when the fourth neighbor is infected you’ll
be triggered. Here, I use the same notation β I had in the classical
contagion, but note that its meaning is slightly different. β is not the
probability of infection given a contact, but the share of neighbors
in set I required for you to transition to I. If in classical contagion a
low β means low infection chance, in cascade a low β means that it’s
more likely to get infected, as you need fewer neighbors to make you
transition.
Why do I separate the cascade and the threshold model? Because
of hubs. We saw that, in the threshold model, infecting hubs was easy.
Since they have lots of connections, the likelihood of them having
at least κ infected friends is high. In the cascade model the opposite
holds. It’s harder to infect hubs in a cascade than in a threshold
model, because – for a hub – the β fraction of neighbors required to
be infected usually includes hundreds or thousands of nodes. So, in a
threshold model, hubs are the primary spreaders of the disease. In a
cascade model, they’re the last bastion of defense. Once the hubs fall,
there’s no more chance for salvation.
Again, you’re allowed to vary β to account for the heterogeneity of
gullibility. With proper, rather complicated, fine tuning of κv in the
threshold model and β v in the cascade model, you can render them
equivalent. However, it’s useful to know that there is a simple way to
model contagion in the two cases where you want to simulate a high
or low resistance of hubs to the new spreading behavior.
Separating the cascade and threshold models in different com-
partments would make you think they obey completely different
 complex contagion 255

Figure 18.4: The three classes of

complex contagion as regulated
by the φ parameter. The node
color represent the activation
time of the node from dark
(early) to bright (late).
(a) φ = 0 (b) φ > 0 (c) φ < 0

rules and they are just different phenomena. This needs not to be the
case. The separation is mostly done out of a pedagogical need. In
fact, there is a universal model of spreading dynamics concentrating 6
Baruch Barzel and Albert-László
on hubs6 . We don’t need to delve deep into the details on how this Barabási. Universality in network
model works, but it mostly hinges on a parameter: φ. φ determines dynamics. Nature physics, 9(10):673,
2013b
the interplay between the degree of a node and its propensity of be-
ing part of the epidemics. If φ = 0, the likelihood of contagion of the
node is independent with its degree. If φ > 0 we are in the threshold
scenario: hubs have a stronger impact on the network. With φ < 0, as
you might expect, the opposite holds. Figure 18.4 shows a vignette of
the model.
Sprinkling a bit of economics into the mix, you can relate the
threshold or the cascade parameter with the utility an actor v gets
from playing along or not. Each individual calculates their cost and
benefit from undertaking or not undertaking an action. There is a
cost in adopting a behavior before it gets popular, and in not doing 7
Thomas C Schelling. Hockey helmets,
so after it did7 . Being aware of these effects makes for very effective concealed weapons, and daylight
strategies to make your own decisions while you’re in doubt. You saving: A study of binary choices with
externalities. Journal of Conflict resolution,
can establish a Schelling point which determines whether or not
17(3):381–428, 1973
you’re going to undertake an action8 , which effectively means you 8
[Link]
consciously set your own κv . However, this is getting dangerously posts/Kbm6QnJv9dgWsPHQP/
schelling-fences-on-slippery-slopes
close to a weird blend of economics, philosophy, and game theory. If
you’re interested in learn more, you’d be best served by closing this 9
Herbert Gintis. The bounds of reason:
book and looking elsewhere9 . Game theory and the unification of the
behavioral sciences. Princeton University
Press, 2014
18.2 Limited Infection Chances

So far we have mainly looked at diseases spreading through a net-

work of contacts as a bad thing that we want to minimize. If you
want to look at the opposite problem – how to spread things faster
and faster through a social network – without looking like a so-
ciopath, you need to slightly change the perspective. You can concoct
a scenario in which, for instance, you want to sell a product, thus
you want people to talk about it and convince each other. In practice,
256 the atlas for the aspiring network scientist

you want them to infect themselves with the idea that the product is 10
Pedro Domingos and Matt Richard-
good10 , 11 . son. Mining the network value of
The obvious strategy would be to target hubs, since they have customers. In Proceedings of the seventh
ACM SIGKDD international conference
more connections. However, this heavily depends on your triggering
on Knowledge discovery and data mining,
model, and hubs come with a disadvantage. First, by being promi- pages 57–66. ACM, 2001
nent, hubs are targeted by many things, thus they have a very high 11
Dashun Wang, Zhen Wen, Hanghang
Tong, Ching-Yung Lin, Chaoming Song,
barrier to attention. Second, they have many connections: if the trig-
and Albert-László Barabási. Information
gering mechanism requires reinforcement, most of their connections spreading in context. In Proceedings of
might not get it, thinning out the intervention. A third and final prob- the 20th international conference on World
wide web, pages 735–744. ACM, 2011b
lem might be that you have only one shot at convincing a person. If
you fail, it’s game over forever. If a hub fails, you might not have a
second shot to get to all their peripheral nodes.

Ok. Try it!

NO! NO! Figure 18.5: An example of in-
NO! dependent cascade. The node’s
Ok. Try it! state is encoded by the color:
Try it!
blue = susceptible, red = in-
Ok.
fected and contagious, green =
NO!
infected but not contagious.
NO! NO!

NO! Ok. Ok. Try it!

(a) t = 1 (b) t = 2 (c) t = 3

You can think of this model as a modified SI or SIR, as I show in
Figure 18.5. Suppose that an infected neighbor makes you transition
from S to I at time t. At the next time step t + 1 you will attempt to
do the same to your S neighbors. However, at t + 2 you transition
to a non-contagious stage, where you won’t attempt to convert
your neighbors any more. You will be in I forever, thus why this
is a SI model, but you won’t propagate the disease. Or, you could
see yourself as transitioning to R, with µ = 1: every i individual
will always transition to R immediately. The difference is that R
individuals are still infected, they just cannot pass the disease.
Note how the node at the bottom in Figure 18.5(a) resisted the hub
at time t = 1, but in Figure 18.5(b) gave up on the second attempt
by another node at time t = 2. At the same time, the rightmost node
has to give two answers because of two independent attempts from
its two neighbors. At time t = 3 (Figure 18.5(c)) we see only one per-
suasion attempt, given that the other infectious nodes are connected
to already infected ones. The cascade ends with unconvinced nodes,
due to the lack of any possible further move in t = 4.
It should be clear by now that, once you tried the first time to
convince me to buy a product, any further attempts won’t work if
you didn’t convince me immediately. Maybe another person will
 complex contagion 257

persuade me, just not you. This is a crucial difference with regard to
SI models. We know that any SI model will eventually fill the entire 12
Jacob Goldenberg, Barak Libai, and
network. The independent cascade model12 won’t: the nodes we Eitan Muller. Using complex systems
choose to start the infection with are very important to maximize the analysis to advance marketing theory
development: Modeling heterogeneity
reach of our message.
effects on new product growth through
In the simplest model, node u has a probability pu,v of convincing stochastic cellular automata. Academy of
node v. However, the past history of attempts to convince v might Marketing Science Review, 9(3):1–18, 2001

influence this probability, that’s why you should get to hubs when
you’re the most sure you’re going to convince them. So we can
modify that probability as pu,v (S), with S being the set of nodes 13
David Kempe, Jon Kleinberg, and
who already tried to influence v13 , 14 , 15 . The process ends when all Éva Tardos. Maximizing the spread of
infected nodes exhausted all their chances of convincing people so no influence through a social network. In
Proceedings of the ninth ACM SIGKDD
more moves can happen.
international conference on Knowledge
So you get the problem: find the set of cascade initiators I0 such discovery and data mining, pages 137–146.
that, when the infection process ends at time t, the share of infected ACM, 2003
14
David Kempe, Jon Kleinberg, and
nodes in the network it is maximized. Kempe et al. solve the prob- Éva Tardos. Influential nodes in a
lem with a greedy algorithm. We start from an empty I0 . Then we diffusion model for social networks. In
calculate for each node its marginal utility to the cascade. We add International Colloquium on Automata,
Languages, and Programming, pages
the node with the largest utility, meaning the number of potential 1127–1138. Springer, 2005
infected nodes, to I0 and we repeat until we reach the size we can 15
Note that, if we only use pu,v we
afford to infect. Of course, each node we add to I0 changes the ex- call this independent cascade model,
because the previous attempts do not
pected utility of each other node, because they might have common influence future attempts. When we
friends, thus we cannot simply choose the | I0 | nodes with the largest introduce pu,v (S) the cascades are not
independent any more. Specifically, for
initial utility.
the paper I’m citing, we have decreasing
There are many improvements for this algorithm, focused on cascades because, the more people
improving time efficiency, lowering the expected error, and integrat- try, the hardest it is to convince v,
i.e. pu,v (S) < pu,v (S ∪ z). If we did
ing different utility functions. However, things get more interesting the opposite, pu,v (S) > pu,v (S ∪ z),
when you start adding metadata to your network. For instance, Gu- then this model would be practically
equivalent to the threshold model: the
rumine16 is a system that lets you create influence graphs, as I show
more infected neighbors you have, the
in Figure 18.6. You start from a social network (Figure 18.6(a)) and a more likely you’re going to turn.
table of actions (Figure 18.6(b)). You know when a node did what. 16
Amit Goyal, Francesco Bonchi, and
Laks VS Lakshmanan. Learning influ-
You can use the data to infer that node v does action a1 regularly
ence probabilities in social networks. In
after node u performed the same action. In the example, for two Proceedings of the third ACM international
actions a1 and a2 you see node 2 repeating immediately after node conference on Web search and data mining,
pages 241–250. ACM, 2010
1. Since these two nodes are connected, maybe node 1 is influencing
node 2. You can use that to infer pu,v = 0.66 (Figure 18.6(c)) – or, if
you’re really gallant, to infer pu,v (S) by looking at all neighbors of v
performing a1 before it.
Note that node 6 performed the same action at the same time as
node 3. Node 6 could only be influenced by node 2. For node 3 we 17
Eytan Bakshy, Jake M Hofman,
Winter A Mason, and Duncan J Watts.
prefer inferring that node 1 did it, because we know that it influenced Everyone’s an influencer: quantifying
node 2 too, so that’s the most parsimonious hypothesis. The size in influence on twitter. In Proceedings of
number of nodes of these cascades can be approximated by – you the fourth ACM international conference on
Web search and data mining, pages 65–74.
guessed it – a power law17 . ACM, 2011
258 the atlas for the aspiring network scientist

3 1
1 Time Node Action Figure 18.6: (a) The underlying
1 1 a1 social network. (b) The actions
0.66 0.33
nodes made. (c) A possible
2 2 1 a2
4 inferred influence graph.
2 2 a1
2 3
6 3 1 a3
5 3 2 a2
7 0.5
3 3 a1
9 3 6 a1
8 6
(b)
(a) (c)

When running such models on real data you can find funny
things. For instance, I ran it with some co-authors on LastFm data,
a social network recording which user listened to which musical 18
Diego Pennacchioli, Giulio Rossetti,
artist at which time18 – the artist is considered the “action”. In doing Luca Pappalardo, Dino Pedreschi, Fosca
so, we discovered that we could build these influence graphs and Giannotti, and Michele Coscia. The
three dimensions of social prominence.
describe their trade offs. For instance, the more intensely a user was
In International Conference on Social
influenced by a prominent friend – meaning that they listened the Informatics, pages 319–332. Springer,
new artist a lot – the fewer friends the influencer hit. In other words: 2013

the stronger you want to influence people, the fewer people you can
influence.

Figure 18.7: Anatomies of two

Cascade
#1 different cascades. Time flows
from left to right. A node at
a given position on the x axis
denotes when they share the
Cascade
#2 content on their profile. An
arrow indicates from where
they re-shared an item, i.e. who
Time influenced them.

Similar studies on Facebook tried to find which early signs we

can use to predict the size of a cascade. A cascade is when I share
something on my profile, then other people share it too and so on
until it hits the news. Counterintuitively, the answer seem to have
very little to do with the actual content of the idea per se, but with
Justin Cheng, Lada Adamic, P Alex
19

the speed with which it triggers other people19 , 20 . For instance, in Dow, Jon Michael Kleinberg, and Jure
Figure 18.7 we have two hypothetical cascades with the same number Leskovec. Can cascades be predicted? In
WWW, pages 925–936. ACM, 2014
of shares and the same topological pattern: one re-sharer then two. 20
Justin Cheng, Lada A Adamic, Jon M
However, the fact that the first cascade happened faster is enough for Kleinberg, and Jure Leskovec. Do
us to infer that it’s much more likely to end up being much larger cascades recur? In WWW, pages 671–681.
ACM, 2016
than the second, slower, one. I’m going to explore more in depth this
 complex contagion 259

idea about memes spreading when talking about classical results in

network analysis in Chapter 45.
You can further complicate models by having competing ideas
spreding into the network. There are some people who are com-
plete enthusiasts about iPhones, while others really hate them. The
love/hate opinions are both competing to spread through the net-
work. You can see them, for instance, as a physical heating/cooling
process which will eventually make nodes converge to a given tem- 21
Hao Ma, Haixuan Yang, Michael R
perature21 . The classic survey of viral marketing applications of Lyu, and Irwin King. Mining social
network analysis22 is a good starting point for diving deeper into the networks using heat diffusion processes
for marketing candidates selection. In
topics only skimmed in this section.
Proceedings of the 17th ACM conference on
Information and knowledge management,
pages 233–242. ACM, 2008
18.3 Interventions 22
Jure Leskovec, Lada A Adamic, and
Bernardo A Huberman. The dynamics
Once we have a disease spreading through a social network, we of viral marketing. ACM Transactions on
the Web (TWEB), 1(1):5, 2007a
might be interested in using our knowledge to prevent people to
become sick. In practice, if this were a SIR model, we want to flip
some people directly from the S to the R state, without passing by
I. This is equivalent to vaccinate them and, if done properly, would
stop the epidemics in its tracks. You can try an online game with this
premise and see how much of a network you can save from an evil 23
[Link]
disease23 .
The first question is: who should we vaccinate? The answer is
rather obvious once you run your simulation numbers: the hubs.
If the disease attacks the hubs, it will spread to the entirety of the
network almost instantly. This assumes that its degree exponent is
α < 3 and we know that, unfortunately, this is true for the majority of
social systems we know.
However, now we have a second question: how do we find hubs?
We might not have a complete – or even a partial! – picture of our
social network. Luckily, this book has prepared you to figure out a
way to find hubs even if you know nothing about the network’s topol-
ogy. You can exploit the fact that hubs have lots of connections. The
simplest and unreasonably effective vaccination strategy is to pick 24
Reuven Cohen, Shlomo Havlin,
a node at random in the network and vaccinate one of its friends24 . and Daniel Ben-Avraham. Efficient
immunization strategies for computer
Statistically speaking, the friend of our random sampled individual is networks and populations. Physical
more likely to have a higher degree than our first choice. review letters, 91(24):247901, 2003
To see why, consider Figure 18.8. Here we apply the “vaccinate-
a-friend” strategy and report the probability of choosing each node.
Note that this is done completely blindly, we don’t know anything
about the topology of this network. If we were to vaccinate the
randomly sampled node, we would have only one chance out of nine
to find the hub, given that the example has nine nodes. However, the
probability of vaccinating the hub with our strategy is almost three
260 the atlas for the aspiring network scientist

. 02 Figure 18.8: The probability

. 05 of vaccinating a node with the
“vaccinate-a-friend” strategy.
.14
.05 .3 2

.1 5
.06
.1 1
.08

times as high. This is related to a curious network effect on hubs,

known as the “Friendship Paradox”, which we’ll investigate further
in Section 27.2.
Of course, this strategy makes a number of assumptions that
might not hold in practice. For instance, we only consider a simple
SIR model, without looking at the possibility of complex contagion.
Luckily, there is a wealth of research relaxing this assumption and
proposing ad hoc immunization strategies that can work in realistic 25
Chen Chen, Hanghang Tong,
scenarios25 . One of the most historically important approaches in this B Aditya Prakash, Charalampos E
category is Netshield26 . Tsourakakis, Tina Eliassi-Rad, Christos
Faloutsos, and Duen Horng Chau. Node
How do we know if we did a good job? How can we evaluate the
immunization on large graphs: Theory
impact of an intervention? There are two things we want to look at. and algorithms. IEEE Transactions on
First, we look at the size of the final infected set and simply subtract Knowledge and Data Engineering, 28(1):
113–126, 2015
the predicted infected share without immunization with the one with 26
Hanghang Tong, B Aditya Prakash,
immunization. The higher the difference the better. Figure 18.9 gives Charalampos Tsourakakis, Tina
you a sense of this. An SI model without immunization reaches sat- Eliassi-Rad, Christos Faloutsos, and
Duen Horng Chau. On the vulnerability
uration when all nodes are infected. A smart immunization strategy of large graphs. In 2010 IEEE Interna-
can make sure that the outbreak stops at a share lower than 100%. tional Conference on Data Mining, pages
1091–1096. IEEE, 2010
A second criterion might be just delaying the inevitable. Once

% Infected
No intervention
100%

x%

With immunization

Figure 18.9: The first criterion

of immunization success: the
share of infected nodes at the
end of the outbreak is lower
Time than 100% in a SI model.
 complex contagion 261

immunized, the nodes can revert to the S state, and therefore to I,

after a certain amount of time. This time can be used to develop
a real vaccine or might be a feature in itself, preventing having
too many people transitioning to I at the same time. Figure 18.10
provides an example. In this case, we might want to either calculate
the time t at which the system reaches saturation, or compute the
area between the two curves as a more precise sense of the delay we
imposed.

% Infected
Figure 18.10: The second crite-
100%
rion of immunization success:
No intervention
temporary immunity can delay
propagation.

With immunization

Time

We can combine the two criteria at will. By immunizing nodes, we

make the disease unable to reach saturation at 100% infection AND
we delay its spread in the network. Thus the two scenarios are not
mutually exclusive.
Obviously, here I only assumed the perspective of limiting the
outbreak of a disease. If you’re in the viral marketing case you
can invert the perspective: your interventions wants to favor the
spread of the idea in the social network. In this case, the second
scenario makes more sense: even if the idea was bound to reach
everyone eventually, if it does so faster it can have great repercussion.
Think about the scenario of condom use to prevent HIV infections.
You want to convince as many people as fast as you can, even if
eventually your message was going to reach everybody anyway. 27
Yang-Yu Liu, Jean-Jacques Slotine,
and Albert-László Barabási. Controlla-
bility of complex networks. nature, 473
18.4 Controllability (7346):167, 2011
28
Jianxi Gao, Yang-Yu Liu, Raissa M
A related problem is the classical scenario of the controllability of D’souza, and Albert-László Barabási.
complex networks27 , 28 , 29 . Here the task is slightly different: nodes Target control of complex networks.
Nature communications, 5(1):1–8, 2014
can change their state freely and there can be an arbitrary number of 29
Gang Yan, Georgios Tsekenis, Baruch
states in the network. What we want to ensure is that all – or most Barzel, Jean-Jacques Slotine, Yang-
– nodes in the network end up in the state we desire. To do so, we Yu Liu, and Albert-László Barabási.
Spectrum of controlling and observing
need to identify driver nodes: the smallest possible subset of nodes complex networks. Nature Physics, 11(9):
we have to manipulate so that they will influence the other nodes to 779–786, 2015
262 the atlas for the aspiring network scientist

switch to the state we want them to assume.

There already is a branch of mathematics dedicated to figure out
how to control engineered and natural simple systems, unoriginally 30
Ernest Bruce Lee and Lawrence
named control theory30 , 31 . However, we define complex systems Markus. Foundations of optimal control
exactly on their nature of being difficult to predict, as their parts theory. Technical report, Minnesota
Univ Minneapolis Center For Control
interact with each other and thus let non-obvious properties and
Sciences, 1967
behaviors emerge. 31
B Francis. A course in H 1 control theory.
In complex systems, controllability is a bit more complicated. Lectures notes in control and information
sciences, volume 88. Springer Verlag
Figure 18.11 shows a few simple examples. In Figure 18.11(a), a chain, Berlin, 1987
we only need to control the origin of the chain and the rest of the
system will fall into place. In Figure 18.11(b), somewhat surprisingly,
one needs to control at least two among the blue nodes besides the
hub in green to ensure control the system. One also needs to control
three of the four blue nodes in Figure 18.11(c). Unfortunately, the
mathematical details to reach this conclusion are beyond the scope of
this book, and I invite you to read the papers cited at the beginning
of this section if you’re interested in them.

Figure 18.11: Different control-

lability scenarios. The nodes
we’re forced to select as drivers
are in green, the ones we could
or could not choose are in
(a) (b) (c) blue, the ones we don’t need as
As you might expect from a complex network paper, the final con- drivers are in red. Similarly for
clusion is that sparse networks with a power law degree distribution links, the links we need to have
characterized by a low α exponents are extremely difficult to control: to ensure controllability are
they require a large number of driver nodes. In another twist going in green, the ones we could or
against our intuition, driver nodes tend not to be hubs. You would could not have are in gray, and
expect nodes connecting to the majority of the network to be natural the ones we could remove from
choices to control the system, yet it seems that peripheral nodes have the network without hampering
their role. controllability are in red.
There are numerous applications of controllability in complex 32
Jonathan D Power, Alexander L
systems, for instance in networks modeling the brain32 . Cohen, Steven M Nelson, Gagan S Wig,
Kelly Anne Barnes, Jessica A Church,
Alecia C Vogel, Timothy O Laumann,
18.5 Summary Fran M Miezin, Bradley L Schlaggar,
et al. Functional network organization of
1. In simple contagion at each timestep you have the same chance the human brain. Neuron, 72(4):665–678,
2011
of getting infected if you have one or more infected neighbors.
In complex contagion more infected neighbors reinforce the in-
fection chances. Different models work with different triggering
mechanisms.

2. Classically, you can have an independent β probability to transi-

 complex contagion 263

tion for each infected neighbor. In the threshold model you need
at least κ neighbors, independently of your degree. In the cascade
model you need at least a fraction of neighbors.

3. This changes the behavior of hubs: in the threshold model it is

easy to have at least κ infected contacts because hubs have so
many neighbors, but for the very same reason it is difficult to
reach the relative limit in the cascade model.

4. You can estimate which type of infection model a real world out-
break follows by estimating the universality class of the spreading,
via its parameter φ: φ > 0 is similar to a threshold model (positive
correlation between degree and chance of infection), φ < 0 is sim-
ilar to a cascade model (negative correlation between degree and
chance of infection).

5. In viral marketing models of word-of-mouth you don’t have

infinite chances to infect a node. The problem becomes identifying
the set of initial infected seeds so that you maximize the number
of infected nodes in the network.

6. One effective strategy to prevent a global outbreak is to immu-

nize the friends of randomly chosen nodes. This strategy works
because the randomly picked neighbors of randomly picked nodes
are more likely to be hubs.

18.6 Exercises

1. Modify the SI model developed in the exercises of the previous

chapter so that it works with a threshold trigger. Set κ = 2 and run
the threshold trigger on the network at [Link]
eu/exercises/18/1/[Link]. Show the curves of the size of
the I state for it (average over 10 runs, each run of 50 steps) and
compare it with a simple (no reinforcement) SI model with β =
0.2.

2. Modify the SI model developed in the previous exercise so that

it works with a cascade trigger. Set β = 0.1 and compare the I
infection curves for the three triggers on the network used in the
previous exercise (average over 10 runs, each run of 50 steps).

3. Modify the simple SI model so that nodes become resistant after

the second failed infection attempt. Compare the I infection curves
of the SI model before and after this operation on the network
used in the previous exercise, with β = 0.3 (average over 10 runs,
each run of 50 steps).
264 the atlas for the aspiring network scientist

4. Run a classical SIR model on the network used in the previous

exercise, but set the recovery probability µ = 0. At each timestep,
before the infection phase pick a random node. Pick one random
neighbor in status S, if it has one, and transition it to the R state.
Compare the I infection curves with and without immunization,
with β = 0.1 (average over 10 runs, each run of 50 steps).
19
Catastrophic Failures

In Section 18.2 we saw an interesting thing: when limiting the infec-

tion chances nodes get, the disease might be unable to reach some
parts of the network. This is great when fighting a disease, but net-
works model much more than social systems hosting pathogens. The
roads you use every day for your commute are part of a network.
The power grid is a network. The beloved cat pictures you look at
every day flow through edges of the interwebz. The fact that some-
thing might prevent them to reach you is alarming and deserves to
be studied.

Figure 19.1: The effect of node

failures on the connectivity of
a network. Node color: green =
active; red = failing. (a) Starting
condition, all nodes active. (b)
First failure. (c) Propagating
failure disconnects the two
(a) (b) (c) nodes on top from the main
In this chapter we do exactly that. We again slightly change the component.
perspective of our epidemic model to study the conditions under
which networks break down. Rather than propagating a disease, we
propagate failures. In their standard status, nodes are active and
fulfill their duties. See Figure 19.1(a) for an example. However, for
random or deliberate reasons they might transition into an R status:
they might fail. The fundamental question is: how does the network
react to such failures? Can information still flow through the Internet
if some routers go down? How many blocked roads does it take for
cars not to be able to drive around town?
Networks are usually resilient to small failures: a single node go-
ing down does not affect communication in the structure (see Figure
19.1(b)). Our criterion to say whether a network is still fulfilling its
266 the atlas for the aspiring network scientist

purpose is the share of nodes part of its largest component. If there

still is a path between all or most nodes in the network, even if it
becomes longer, the network still works. However, when nodes start
breaking down in multiple components and getting isolated – as in
Figure 19.1(c) – then the network is failing.
We start by looking at random failures in Section 19.1 to move
then to deliberate attacks in Section 19.2. We then put some dynamics
on failures by considering correlated cascade failures in Section 19.3,
giving a special attention to a specific case of multilayer structures:
interdependent networks (Section 19.4).
This chapter is related to the mathematical problem of percolation 1
Dietrich Stauffer and Ammon Aharony.
theory1 , which has then been adapted to the network scenario2 , 3 , 4 . Introduction to percolation theory. Taylor &
Note that I’m using the power grid example mostly as a way to give Francis, 2014
color to the math – and for traditional reasons. However, power grids
2
Réka Albert, Hawoong Jeong, and
Albert-László Barabási. Error and attack
failures don’t necessarily follow such percolation approach. The tolerance of complex networks. nature,
model here propagates failures linearly and locally, but real failures 406(6794):378, 2000
are neither, due to the underlying laws governing electrical flows.
3
Paolo Crucitti, Vito Latora, Massimo
Marchiori, and Andrea Rapisarda.
You can use these methods in other scenarios, but only if you’re Error and attack tolerance of complex
sure that the assumptions made here are respected in the phe- networks. Physica A: Statistical mechanics
and its applications, 340(1-3):388–394,
nomenon you’re studying. 2004
4
Jianxi Gao, Baruch Barzel, and Albert-
László Barabási. Universal resilience
19.1 Random Failures patterns in complex networks. Nature,
530(7590):307–312, 2016
In this section we look at random failures. In this case, nodes can
spontaneously break for uncorrelated and not deliberate reasons.
Think about normal wear and tear. Any power generator can only
take so much. Moreover, slight differences in the manufactory pro-
cess, or in the model, can give different failure rates. Thus it is dif-
ficult to predict when one component will fail. The error rate will
appear to be more or less random.
How does a network respond to these random failures? We’re
assuming that all its nodes are in the same Giant Connected Compo-
nent (GCC), so that power can flow freely through the grid. When
will the network lose its giant component? In other words: when will
we need to rely on local generators rather than on the entire grid?
The answer depends on the original topology of the network. Let’s
start by considering the case of a random Gn,p network. What we
want to see is how much the probability of a node being part of the
GCC changes as we put more and more nodes in the failure state R. I
ran a few simulations and they generate a plot similar to what Figure
19.2 shows.
Now, if this seems the very same plot as one you’ve already seen,
calm down: you’re not taking crazy pills. You have indeed seen
something like this. It was back to Figure 13.5(b), when we were talk-
 catastrophic failures 267

% nodes in LCC
Figure 19.2: The probability of
being part of the largest con-
nected component as a function
of the number of failing nodes
in a random Gn,p graph.

x
|R|

ing about the probability of a node being part of the GCC in a Gn,p
model. In that case, the function on the x-axis was the probability
p of establishing an edge between two nodes. In fact, the two are
practically equivalent: if you have a Gn,p graph with failures it is as if
you’re manipulating n and p.
What Figure 19.2 says is that a Gn,p network will withstand small
failures: a few nodes in R will not break the network apart. However,
the failure will start to become serious very quickly, until we reach a
critical value of | R| beyond which the GCC disappears and the net-
work effectively breaks down. Just like the appearance of a GCC for
increasing p in a Gn,p model is not a gradual process, so are random
failures. At some point there is a phase transition, from having to not
having a GCC.
Ah – you say – but we’re not amateurs at this. Who would engineer
a random power grid network? For sure it won’t be a Gn,p graph. Good
point. In fact that’s true: the power grid’s degree distribution is
skewed. For the sake of the argument – and the simplicity of the
math – let’s check the resilience to random failures of a network with
a power law degree distribution.
Good news everybody! Power law random networks are more
resilient than Gn,p networks to random failures. The typical signature
of a power law network under random node disappearances looks
something like Figure 19.3. In the figure you see no trace of the phase
transition. The critical value under which the GCC disappears is
much higher than in the Gn,p case. Of course the size of the largest
connected component goes down, because you’re removing nodes
from the network. However, the nodes that remain in the network
still tend to be able to communicate to each other, even for very high
| R |.
Why would that be the case? The reason is always the power law
degree distribution. If you remember Section 6.3, having a heavy
tailed degree distribution means to have very few gigantic hubs and
268 the atlas for the aspiring network scientist

% nodes in LCC
Figure 19.3: The probability of
being part of the largest con-
nected component as a function
of the number of failing nodes
in a network with a skewed
degree distribution.

|R|

a vast majority of nodes of low degree. When you pick a node at

random and you make it fail, you’re overwhelmingly more likely to
pick one of the peripheral low degree ones. Thus its impact on the
network connectivity is low. It is extremely unlikely to pick the hub,
which would be catastrophic for the network’s connectivity.
Since, by now, you must be a ninja when it comes to predict the ef-
fect of different degree exponents on the properties of a network with
a power law degree distribution, you might have figured out what’s
next. The exponent α is related to the robustness of the network to
random failures. An α = 2, remember, means that there are fewer
hubs and their degree is higher. If α > 3, the hubs are more common
and less extreme.

% nodes in LCC
Figure 19.4: The probability of
being part of the largest con-
nected component as a function
of the number of failing nodes
in a network for different α ex-
α=2
ponents of its power law degree
α=4 distribution.

|R|

More common hubs equals higher likelihood of picking them up

in a random extraction. Thus the failure functions for different α
values follow the pattern I show in Figure 19.4. The lower your α the
fewer hubs, the more resilient the network. By now you probably
start to get an inkling on why network scientists are so obsessed
about finding that their networks are scale free. If they are, then
there are tons of properties you can infer by just knowing its degree
exponent and relatively simple math. In this part we already saw two:
 catastrophic failures 269

robustness to random failures (here) and outbreak size and speed in

SI and SIS models (in Chapter 17).
By the way, so far we’ve been looking at random node failures,
i.e. a generator blowing up in the power grid. Edge failures can be
equally common: think about road blocks. However, the underlying
math is rather similar and the functions describing the failures are 5
Duncan S Callaway, Mark EJ Newman,
not so different than the ones I’ve been showing you so far5 , 6 . For Steven H Strogatz, and Duncan J Watts.
this reason we keep looking at node failures. Network robustness and fragility:
Percolation on random graphs. Physical
review letters, 85(25):5468, 2000
19.2 Targeted Attacks 6
Reuven Cohen, Keren Erez, Daniel
Ben-Avraham, and Shlomo Havlin.
Resilience of the internet to random
So far we’ve assumed the world is a nice place and, when things breakdowns. Physical review letters, 85
break down, they do so randomly. We suspect no foul play. But what (21):4626, 2000
if there was foul play? What if we’re not observing random failures,
but a deliberate attack from an hostile force? In such a scenario, an
attacker would not target nodes at random. They would go after
the nodes allowing them to maximize the amount of damage while
minimizing the effort required.
This translates into prioritizing attacks to the nodes with the
highest degree. Taking down the node with most connections is
guaranteed to cause the maximum possible amount of damage. What
would happen to our network structure?

1
Random
0.9 Targeted Figure 19.5: The probability
0.8
of being part of the largest
Nodes in GCC

0.7
0.6
0.5 connected component as a func-
0.4
0.3 tion of the number of failing
0.2
0.1
nodes in a Gn,p network, for
0
1 10 100
random (blue) and targeted
|R| (red) failures.

Let’s start again by considering a Gn,p network. I ran a few simula-

tions and Figure 19.5 shows the result. We knew that Gn,p networks
aren’t particularly good under random failures. It turns out that
targeted attacks don’t change the scenario much. Sure, the critical
threshold is a bit lower, but the failure function is fundamentally the
same.
Why? Remember that a Gn,p model generates a normal degree
distribution. This means that hubs are less common and their degree
isn’t much different from the average degree of all other nodes. If
you pick up nodes randomly, you are likely to pick a node with
higher-than-average degree and, even if you don’t, whatever you pick
isn’t much different.
270 the atlas for the aspiring network scientist

1
Random
0.9 Targeted Figure 19.6: The probability of
0.8
being part of the largest con-
Nodes in GCC

0.7
0.6
0.5 nected component as a function
0.4
0.3 of the number of failing nodes
0.2
0.1
in a degree skewed network,
0
1 10 100 1000
for random (blue) and targeted
|R| (red) failures.

The case is oh-so-much different when we turn our attention to

networks with power law degree distributions. Since they have large
hubs, prioritizing them for your attack will have devastating effects,
as Figure 19.6 shows. Removing even a single node brings down the
GCC size by almost 20% in this case. To make a similar damage to a
Gn,p network, you have to remove around 40 nodes.
In fact, networks with power law degree distributions break down
more easily than Gn,p equivalents, when under a targeted attacks.
As a consequence, different topologies should be used for different
failure scenarios. If we’re talking about random failures, your should
plan your network to be scale free. If you want to defend from hostile
takeovers, you probably want something similar to a random Gn,p 7
Paul Baran. Introduction to distributed
graph or, even better, a mesh-like network7 . communications networks. Technical
report, Memorandum RM-3420-PR,
Rand Corporation, 1964
1
α = 2.5, kmin = 1
0.9 α = 2.5, kmin = 2 Figure 19.7: The probability
0.8 α = 3, kmin = 1
α = 3, kmin = 2
of being part of the largest
Nodes in GCC

0.7
0.6
0.5 connected component as a func-
0.4
0.3 tion of the number of failing
0.2
0.1
nodes in a degree skewed net-
0
1 10 100 1000
work, for different α and k min
|R| combinations.

As you might expect, the α exponent of the power law degree

distribution has something to do with the fragility of a network to
8
Reuven Cohen, Keren Erez, Daniel
Ben-Avraham, and Shlomo Havlin.
deliberate attacks. However, it is a non-linear relationship, which Breakdown of the internet under
also depends on the minimum degree of the network k min 8 , 9 . Figure intentional attack. Physical review letters,
86(16):3682, 2001
19.7 shows the results of a few simulations. If k min is low, higher α 9
Béla Bollobás and Oliver Riordan.
exponents tend to make your network rather fragile, so it’s better to Robustness and vulnerability of scale-
have α = 2.5 rather than α = 3. However, if we increase k min , then the free random graphs. Internet Mathematics,
1(1):1–35, 2004
opposite holds true: higher α actually make your network stronger.
 catastrophic failures 271

19.3 Chain Effects

So far in this chapter we’ve relying on an unreasonable assumption:

failure doesn’t propagate. We said that a power generator goes
boom randomly and studied what this means for the structure of
the power grid. However, it is important to note that energy demand
does not go down just because there was a failure. People will still
turn their light bulbs on. Therefore, whatever power that generator
was providing has to come from somewhere else. However, if that
generator was there, there was a reason. Maybe the other nodes in
the network cannot satisfy the additional demand. Thus there is a
high chance that they will themselves go out of business.
In this scenario, the failure of one node propagates in a cascade
and causes more correlated failures. This sort of snowball effect can
turn into an avalanche and shut the entire network down. And it has 10
Ian Dobson, Benjamin A Carreras,
happened, many times10 , also in structures that have nothing to do Vickie E Lynch, and David E Newman.
Complex systems analysis of series of
with power grids such as airline schedules11 . blackouts: Cascading failure, critical
The models we use to simulate such propagating failures are yet points, and self-organization. Chaos:
An Interdisciplinary Journal of Nonlinear
another family of variations of the threshold model from Granovetter
Science, 17(2):026103, 2007
(Section 18.1), for instance the Failure Propagation model12 . You 11
Pablo Fleurquin, José J Ramasco,
can define failure propagation as being literally equivalent to the and Victor M Eguiluz. Systemic delay
propagation in the us airport network.
threshold model, by having a node V failing if a fraction f v of its Scientific reports, 3:1159, 2013
neighbors are failing. However, things become more interesting when 12
Ian Dobson, Benjamin A Carreras,
you take into account more information. and David E Newman. A loading-
dependent model of probabilistic
All nodes start in the state S. They are characterized by a current
cascading failure. Probability in the
load and by a total capacity. Think of this as road intersections: the Engineering and Informational Sciences, 19
load is how many cars pass on average through the intersection (1):15–32, 2005

and the capacity is the maximum amount that it can pass before
congestion happens.
At time t = 1 we shut down a node in the network. Maybe the
traffic light failed and so no one can pass through until we repaired
it. This means that the node transitions to state I. People still need
to do their errands, so we have to redistribute the load of cars that
wanted to pass through that intersection through alternative routes:
the neighbors of that node. However, that means that their load
will increase. If the new load exceeds the capacity of the node, also
this node shuts down due to congestion. So its load has also to be
redistributed to its neighbors and so on and so forth.
Figure 19.8 shows an example of failure propagation with this
load-capacity feature. You can see that the network was built with
some slack in mind: its normal total load is 37 – the sum of all loads
of all nodes – for a maximum capacity of 90 – the sum of all nodes’
capacities. Yet, shutting down the top node whose load was only 6
and redistributing the loads causes a cascade that, eventually, brings
272 the atlas for the aspiring network scientist

6/10 0 /10 0/ 1 0 0/ 1 0

2/3 9 /1 0 4/3 11/10 0/3 0/10 0/3 0/10

2/12 4 /12 10/12 13/12

1 / 10 1 / 10 4 /1 0 4/ 10
6/7 4 /1 6 6/7 4 /1 6 8/7 8/16 0/7 10/16

2/ 14 5/8 2/ 14 5/8 2 /1 4 5/8 5 /1 4 5/8

(a) t = 0 (b) t = 1 (c) t = 2 (d) t = 3

0/10 0/1 0 0/ 1 0 0/ 1 0

0/3 0 /1 0 0/3 0/10 0/3 0/10 0/3 0/10

0/12 0/ 12 0/12 0/12

4 / 10 11/10 0/ 1 0 0/ 10
0/7 23/16 0/7 0/ 1 6 0/7 0/16 0/7 0/16

5/ 14 5/8 13/14 13 / 8 37/14 0/8 0 /1 4 0/8

(e) t = 4 (f) t = 5 (g) t = 6 (h) t = 7

Figure 19.8: A simulated fail-
the whole network down. ure propagation. Each node is
One could represent the failure cascade from Figure 19.8 as the labeled with “load / capacity”.
branches of a tree. The first node failing is the root. We then connect Green nodes are active, red
each node to the nodes it causes to fail. The final structure would be nodes have failed. If load >
something like Figure 19.9. capacity, the node will fail at
the next time step.
0/10 0/10 0/10
Figure 19.9: The branch model
representing the same cascade
0/3 0/7 0/12 0/16 0/8 0 / 14 failure of Figure 19.8.

Using this perspective has its own advantages. It makes the failure
propagation model more amenable to analysis. The final size of the
failure cascade depends on the average degree of nodes in this tree
k̄. The critical value here is k̄ = 1. If, on average, the failure of a node
generates another node failure – or more – the cascade will propagate
indefinitely, until all nodes in the network will fail. If, instead, k̄ < 1,
the failure will die out, often rather quickly.
It’s easy to see why if you have the mental picture of a domino
snake: each domino falling will cause the fall of another domino,
until there’s nothing standing. If, however, there is as much as a
single gap in this chain, the rest of the system will be unaffected.
Quick show of hands: how many of you expect the size of a failure
 catastrophic failures 273

cascade to be a power law? Good, good: by now you learned that

every goddamn thing in this book distributes broadly. The exponent
of the cascade size is also related to the α exponent of your degree
distribution. With a power degree exponent α > 3, networks behave
like Gn,p graphs, but for α < 3 then the cascade size will grow with
exponent α/(α − 1).

19.4 Interdependent Networks

There is an additional thing you have to consider when describing

cascading failures in real world structures. So far, we have considered
our networks as living in isolation. A failure in the power grid propa-
gates only through the power grid. In our interconnected world that
is not the case. In fact, once you realize how fragile networked sys-
tems can be, why would you rely on such systems without additional
fail-safes? For instance, you might want to control what’s happening
on a power grid with an automatic computerized controller, so that it
can try to isolate the failure and prevent it from propagation.
But, yeah, how are you going to do that with a single computer?
You need a network of terminals all close to the action. And how
are you going to provide the power they need? Exactly: through the
power grid itself. So now you have two interdependent networks:
the power grid needs computers to work and the computers need
power to work. And interdependent networks don’t behave like 13
Jianxi Gao, Sergey V Buldyrev,
isolated networks13 . Researchers have studied propagating failures H Eugene Stanley, and Shlomo Havlin.
in such interdependent systems and found out that, spoiler alert, Networks formed from interdependent
networks. Nature physics, 8(1):40, 2012
even if the two networks are resilient to random failures in isolation, the
inter-dependencies cause them to be fragile to failures propagating back and 14
Sergey V Buldyrev, Roni Parshani,
forth between them14 . Ouch. Gerald Paul, H Eugene Stanley, and
The way you model this problem is by using multilayer networks Shlomo Havlin. Catastrophic cascade
of failures in interdependent networks.
(Section 4.2). Figure 19.10 shows a simple example. The blue layer Nature, 464(7291):1025, 2010
represents the power stations and the green layer represents the
computers. A power station needs the coupled computer to work and
vice versa. The network can only work via connected components in
both layers: if nodes get isolated in one layer, the nodes on the other
layer coupled to different components get disconnected. A power
station needs to know the statuses of its neighboring stations, but
if they are on a different computer component it cannot know it, so
their links become inactive.
If a node in one layer fails (Figure 19.11(a)) it breaks its connec-
tions and causes its coupled node to lose its connections too. Now
we have multiple connected components in one layer, so the links in
the other layer going across components start to fail as well (Figure
19.11(b)). These failures propagate in a chain reaction until we end up
274 the atlas for the aspiring network scientist

Figure 19.10: A multilayer

network representing inter-
dependencies between two
layers: the nodes in one layer
depend on the nodes on the
other layer connected to them
via an inter-layer coupling.

in a situation where practically every node in both layers is isolated,

and the network almost completely failed (Figure 19.11(c)).

X X X

X X X

(a) t = 1 (b) t = 2 (c) t = 3

Figure 19.11: Propagating
When describing failures in single layer networks, we asked
failures in interdependent
ourselves what’s the value of | R| such that the network breaks down.
networks. The purple cross
In other words: what’s the fraction of initially failing nodes that
shows the original failing nodes.
will make the GCC disappear. We can ask the same question here,
At each time step, nodes de-
realizing that, in interdependent networks, this | R| value is much
pending on nodes in different
lower than the corresponding one for single layer networks. In fact,
components lose their connec-
if you were to calculate the critical | R| value for each layer separately you
tions (indicated by a faded edge
would obtain a result much higher than the one for the interdependent
color).
network as a whole. Meaning that, if you were to analyze the layers
independently, you’d grossly underestimate the risk of a catastrophic
failure propagating through the entire network.
Remember when, in Section 19.1, I said that networks with a heavy
tail in their degree distribution are particularly robust to random
failures? The reason was that large hubs keep the network together
and there are very few of them, so it’s unlikely to pick them up at
random. Well... In two interdependent networks it is likely that
hubs in one layer will couple to nodes with a lower degree in the
other. Guess what: that makes coupled power law networks fragile
to random failures. In fact, they’re more fragile than random Gn,p
 catastrophic failures 275

% nodes in LCC
Figure 19.12: The relationship
Gn,p
α=3
between the degree exponent α
α = 2.7 of coupled power law networks
α = 2.3 and the fraction of nodes in R
state (x axis) needed to destroy
the GCC (shades of red). In
blue the equivalent plot for
coupled random Gn,p graphs.

|R|

graphs, contrarily to what was the case before.

Figure 19.12 shows a general schema of fragility for different cou-
pled network topologies. I have only considered the case of random
failures, but even in the case of targeted attacks interdependency is
not a good thing to have: failures will spread more easily than in 15
Xuqing Huang, Jianxi Gao, Sergey V
isolated networks15 . Buldyrev, Shlomo Havlin, and H Eu-
Fixing this issue is not easy. First, one needs to estimate the gene Stanley. Robustness of interdepen-
dent networks under targeted attack.
propensity of the network to run the risk of failure. This has been Physical Review E, 83(6):065101, 2011b
done in two-layer multiplex networks16 . One would think that the 16
Rebekka Burkholz, Matt V Leduc,
best thing to do is to create more connections between nodes, to Antonios Garas, and Frank Schweitzer.
Systemic risk in multiplex networks
prevent breaking down in multiple components. However that’s with asymmetric coupling and thresh-
not a trivial operation, as these networks are embedded in a real old feedback. Physica D: Nonlinear
Phenomena, 323:64–72, 2016b
geographical space, where creating new power lines might not be
possible. However, there are also theoretical concerns that show how 17
Charles D Brummitt, Raissa M
D’Souza, and Elizabeth A Leicht.
more connections could render the network more fragile, as it would
Suppressing cascades of load in inter-
give the cascade more possible pathways to generate a critical fail- dependent networks. Proceedings of the
ure17 . A better strategy involves so-called “damage diversification”: National Academy of Sciences, 109(12):
E680–E689, 2012
mitigating the impact of the failure of a high degree node18 . 18
Rebekka Burkholz, Antonios Garas,
Note that we assumed that power law networks are randomly and Frank Schweitzer. How damage
coupled: hubs in one layer will pick a random node to couple to in diversification can reduce systemic risk.
Physical Review E, 93(4):042313, 2016a
the other layer. As a consequence, they’ll likely to pick a low degree 19
Byungjoon Min, Su Do Yi, Kyu-Min
node. Other papers study the effect of degree correlations in inter- Lee, and K-I Goh. Network robustness
layer coupling19 : what if hubs in one layer tend to connect to hubs in of multiplex networks with interlayer
degree correlations. Physical Review E, 89
the other layer? If such correlations were perfect, we’d obtain again (4):042811, 2014
the robustness of power law networks to random failures. These 20
Roni Parshani, Celine Rozenblat,
correlations are luckily observed in real world systems20 , 21 , showing Daniele Ietri, Cesar Ducruet, and
Shlomo Havlin. Inter-similarity between
how they’re not as fragile as one might fear. Phew. coupled networks. EPL (Europhysics
Letters), 92(6):68002, 2011
21
Saulo DS Reis, Yanqing Hu, Andrés
19.5 Summary Babino, José S Andrade Jr, Santiago
Canals, Mariano Sigman, and Hernán A
Makse. Avoiding catastrophic failure in
1. Gn,p networks are fragile to random failures: beyond a critical correlated networks of networks. Nature
number of removed nodes the giant connected component will Physics, 10(10):762, 2014
276 the atlas for the aspiring network scientist

disappear. Networks with skewed degree distributions are instead

robust because they rely on few hubs which are unlikely to be
picked by random failures.

2. In targeted attacks we take down nodes from the most to least

connected. Power law random networks are very fragile and break
down quickly under this scenario.

3. When one node fails, all its load needs to be redistributed to non-
failing nodes. This can and will make the failure propagate on
the network in a cascade event which might end up bringing the
entire network down.

4. In interdependent networks we have a multilayer network whose

nodes in one layer are required for the functioning of nodes in
the others. Depending on the degree correlations among layers,
failures can propagate across layer and bring down power law
networks even under random accidents.

19.6 Exercises

1. Plot the number of nodes in the largest connected component

as you remove 2, 000 random nodes, one at a time, from the net-
work at [Link]
(Repeat 10 times and plot the average result)

2. Perform the same operation as the one from the previous exercise,
but for the network at [Link]
19/2/[Link]. Can you tell which is the network with a power
law degree distribution and which is the Gn,p network?

3. Plot the number of nodes in the largest connected component

as you remove 2, 000 nodes, one at a time, in descending degree
order, from the networks used for the previous exercises. Does the
result confirm your answer to the previous question about which
network is of which type?

4. The network at [Link]

[Link] has nodes metadata at [Link]
exercises/19/4/node_metadata.txt, telling you the current load
and the maximum load. If the current load exceeds the maximum
load, the node will shut down and equally distribute all of its
current load to its neighbors. Some nodes have a current load
higher than their maximum load. Run the cascade failure and
report how many nodes are left standing once the cascade finishes.
 Part VI

Link prediction
20
For Simple Graphs

Link prediction is the branch of network analysis that deals with the
prediction of new links in a network. In link prediction, you see the
network as fundamentally dynamic, it can change its connections.
Suppose you’re at a party. You came there with your friends, and
you’re talking to each other, using the old connections. At some
point, you want to go and get a drink so you detach from the group.
On the way, you could meet a new person, and start talking to them.
This creates a new link in the social network. Link prediction wants
to find a theory to predict these events – in this case, that alcohol is
the main cause of new friendships at parties, or so I’m told –, as I
show the vignette in Figure 20.1.

Figure 20.1: A vignette explain-

ing the link prediction aim. at
a party, red individuals know
each other and express their
relationships (or links) by talk-
ing. One member might detach
from the group for any reason
and, in doing so, she exposes
herself to the possibility of es-
tablishing a new link with the
blue individual. Link prediction
In other words, given a network with nodes and edges, we want
is all about finding the true
to know which link is the most likely to appear in the future. Or, if
reason that might make this
we think we’re seeing an incomplete version of the network, we ask
happen.
ourselves which edges are currently missing from the structure.
Link prediction happens in three steps. The starting point is
your desire to place a new link in the network: which edge will
appear next? The first thing you do is to observe the current links.
On the basis of this observation you formulate a hypothesis on how
nodes decide to link in the network. Finally, you operationalize this
 for simple graphs 279

hypothesis: if nodes are created via process x, you apply x to the

current status of the network and that will tell you which link is most
likely to appear next.
In this chapter, we are going to focus on the simplest possible 1
Lise Getoor and Christopher P Diehl.
Link mining: a survey. Acm Sigkdd
case for link prediction: predicting new links in a simple graph. We Explorations Newsletter, 7(2):3–12, 2005
delve deep into the classical approaches to link prediction, which 2
David Liben-Nowell and Jon Klein-
are the simplest and most used in the literature1 , 2 , 3 , 4 , 5 : Preferential berg. The link-prediction problem for
Attachment, Common Neighbor, Adamic-Adar, Hierarchical Random social networks. Journal of the Amer-
ican society for information science and
Graph models, Resource Allocation, and Graph Evolution Rules. We
technology, 58(7):1019–1031, 2007
also briefly mention other approaches, to give justice to a gigantic 3
Linyuan Lü and Tao Zhou. Link
subfield of network analysis in computer science. prediction in complex networks: A
survey. Physica A: statistical mechanics and
We will deal with multilayer link predictions later, in Chapter 21.
its applications, 390(6):1150–1170, 2011
Once we understand how to assign a score to every possible future 4
Peng Wang, BaoWen Xu, YuRong Wu,
link, it’s time to estimate whether we did a good job. This will be and XiaoYu Zhou. Link prediction in
social networks: the state-of-the-art.
covered in Chapter 22.
Science China Information Sciences, 58(1):
1–38, 2015a
5
Víctor Martínez, Fernando Berzal, and
20.1 Preferential Attachment Juan-Carlos Cubero. A survey of link
prediction in complex networks. ACM
Computing Surveys (CSUR), 49(4):69,
2017

Figure 20.2: An example of a

Preferential Attachment link
prediction. Two hubs (Einstein
and Curie) have a lot of con-
nections (in gray), while a third
author only has few. For PA,
the most logical link to predict
is in blue between the hubs,
because rich get richer, and
thus they will attract the new
connections.
Let’s start with Preferential Attachment (PA). Consider scientific
publishing. We have three authors: two of them – Einstein and Curie
– have a lot of collaborators, while the third – me – has only few – see
Figure 20.2. If we have to make a guess of what collaboration is more
likely to happen next, which one would we expect? It’s more likely
to see the two high degree hubs to connect, because they’re more
6
Mark EJ Newman. Clustering and
preferential attachment in growing
prominent, and thus visible to each other. networks. Physical review E, 64(2):025102,
If we want to predict links, we have to formulate a hypothesis 2001a
7
Albert-Laszlo Barabâsi, Hawoong
and then translate it into a score of u connecting to v for any pair of Jeong, Zoltan Néda, Erzsebet Ravasz,
u, v nodes: score(u, v). In PA, the hypothesis is that “rich get richer”, Andras Schubert, and Tamas Vicsek.
nodes with lots of edges will attract more edges6 , 7 . So we look for Evolution of the social network of
scientific collaborations. Physica A:
pairs of nodes that have attracted so far the most edges. Our PA Statistical mechanics and its applications,
model would consider it strange if they are not connected to each 311(3-4):590–614, 2002
280 the atlas for the aspiring network scientist

other. Our best guess is that they will connect soon. In practice, the
probability of connecting two nodes is directly proportional to their
current degree: score(u, v) = k u k v , where k u and k v are u’s and v’s
degrees, respectively.

20.2 Common Neighbor

Figure 20.3: An example of a

Common Neighbor link predic-
X tion. The hubs (Einstein and
Curie) do not share any connec-
tion, thus it’s less likely they
will connect to each other. But
they share a lot of connections
with other lower degree nodes.
CN will give the possibility of
linking to them a boost.

The preferential attachment example in Figure 20.2 has one defect:

its prediction is wrong, Curie and Einstein never collaborated. This
is because PA fails to consider the social element: it is more likely
to collaborate not only if one is good at collaborating, but also if the
two people are likely to meet. Given that Curie and Einstein are from
slightly different fields, it is difficult for a meeting between the two
to stick into a collaboration. On the other hand, they might have a lot
of common collaborators with other people in the same field: Curie
shared a Nobel prize with her husband Pierre Curie, and Einstein
owes a great debt to Marić – see Figure 20.3. Neither Pierre nor Marić
had as many collaborations as Curie or Einstein, but for the Common
Neighbor (CN) model the thing that matters most is the number of
neighbors they share with them.
Common Neighbor’s basic theory is that triangles close: the more 8
Mark EJ Newman. Clustering and
common neighbors u and v have, the more triangles we can close preferential attachment in growing
networks. Physical review E, 64(2):025102,
with a single edge connecting u to v8 . So the likelihood of connecting 2001a
two nodes is proportional to the number of shared elements in
their neighbor sets: score(u, v) = | Nu ∩ Nv |, where Nu and Nv are
the set of neighbors of u and v, respectively. A variant controls
for how many neighbors the two nodes have: the same number of
common neighbors weighs more if it’s the total set of connections
the two neighbors have. This is the Jaccard variant: score(u, v) =
| Nu ∩ Nv |/| Nu ∪ Nv |.
 for simple graphs 281

20.3 Adamic-Adar

Figure 20.4: An example of a

Adamic-Adar link prediction.
As a hub, Hamilton (top) does
not have enough time to intro-
duce all possible pairs of her
many collaborators. Thus it’s
less likely Johnson (middle
bottom) can connect to Boehm
(left bottom) than she can con-
nect to Vaughan (right bottom),
with whom she shares com-
mon connections with fewer
collaborators than Hamilton.

Common Neighbor has a problem which is bandwidth. Even

if you have Hamilton as a collaborator, she has collaborated with
so many other people that the likelihood of Johnson to connect to
Boehm – both Hamilton’s collaborators – is low, because Hamilton
does not have enough bandwidth to make the introduction. On the
other hand, few common collaborators, if they have few connections,
can represent a stronger attraction between two people. In this case,
they are more likely to make the introduction, because they have the
time to do so. Thus they make the new collaboration happen, as is
the case for Johnson and Vaughan – see Figure 20.4.
9
Lada A Adamic and Eytan Adar.
In Adamic-Adar (AA)9 we say that common neighbors are im- Friends and neighbors on the web. Social
portant, but the hubs contribute less to the link prediction than two networks, 25(3):211–230, 2003
common neighbors with no other links, because the hubs do not have
enough bandwidth to make the introduction. In AA, our score func-
tion discounts the contribution of each node with the logarithm of its
1
degree: score(u, v) = ∑ . The formula says that, for each
z∈ Nu ∩ Nv log kz
common neighbor, instead of counting one – as we do in Common
Neighbor when we look at the intersection –, we count one over the
common neighbor’s degree (log-transformed).

20.4 Resource Allocation

10
Tao Zhou, Linyuan Lü, and Yi-Cheng
The Resource Allocation index10 is almost identical to Adamic-Adar. Zhang. Predicting missing links via
It stems from the very same principle: nodes have bandwidth. The local information. The European Physical
Journal B, 71(4):623–630, 2009
likelihood of u connecting to v is proportional to the amount of
resources u can send to v and vice versa. Thus we have score(u, v) =
282 the atlas for the aspiring network scientist

1
∑ . The only difference with Adamic-Adar is that the scaling
z∈ Nu ∩ Nv k z
is assumed to be linear rather than logarithmic. Thus, Resource
Allocation punishes the high-degree common neighbors more heavily
than Adamic-Adar. You can see that the difference between k z and
log k z is practically nil for low values of k z , but balloons when k z is
high.
One could make a more complex version of the Resource Alloca-
tion index by assuming that the bandwidth of each node and of each
link is not fixed. Thus the amount of resources u sends can change,
and the amount of resources that can pass through the (u, v) link can
also be different from the one passing through other edges.

20.5 Hierarchical Random Graphs

11
Aaron Clauset, Cristopher Moore,
With the Hierarchical Random Graph (HRG) model11 we start to look and Mark EJ Newman. Hierarchical
at different approaches to link prediction. Its main difference with structure and the prediction of missing
links in networks. Nature, 453(7191):98,
what we saw so far is that in HRG we’re not just looking at pairs of
2008
nodes and their neighbors, but at the entire network. First we look
at all connections and we create a hierarchical representation of it
that fits the data. In practice, we want to group nodes in the same
part of the hierarchy if they have a high chance of connecting. In our
recurring example, the field of study of the scientist is a good way to
group nodes. Then we say that it is more likely for nodes in the same
part of the hierarchy to connect in the future if they haven’t done so
yet. Making a long path through this hierarchy to establish a new
connection is less likely – see Figure 20.5.

STEM
Figure 20.5: An example of a
Physics NASA Hierarchical Random Graph
link prediction. The hierarchy
Radioactivity Human
Computers fits the observed connections,
showing that researchers in the
same field are more likely to
connect. Then HRG looks at
pairs of nodes in the same part
of the hierarchy that are not yet
Likely Unlikely connected, and gives them a
higher score.
In HRG we’re basically saying that communities matter: it is more
likely for nodes in the same community to connect. Thus we fit the
hierarchy and then we say that the likelihood of nodes to connect is
proportional to the edge density of the group in which they both are.
If the nodes are very related, the group containing both nodes might
 for simple graphs 283

be a semi-clique with almost maximum density; if the nodes are

far apart, the group containing both nodes might be just the entire
network. In a schematic way: score(u, v) = | Ec |/e(| Ec |), where c is the
community to which u and v belong, | Ec | is the number of edges it
contains, and e(| Ec |) is the number of edges we expect it to contain,
under a null random configuration model assumption (see Section
15.1).

20.6 Association Rules

12
Michele Berlingerio, Francesco Bonchi,
Just like HRG, GERM12 , 13 is a peculiar approach to link prediction Björn Bringmann, and Aristides Gionis.
Mining graph evolution rules. In joint
that has almost nothing in common with the standard approach of European conference on machine learning
simply evaluating node-node similarity. GERM is short for Graph and knowledge discovery in databases,
pages 115–130. Springer, 2009
Evolution Rule Mining and it is rarely considered in link prediction 13
Björn Bringmann, Michele Berlingerio,
surveys because it’s a bit harder to implement and its only known Francesco Bonchi, and Arisitdes Gionis.
implementation is proprietary software. But the approach deserves to Learning and predicting the evolution
of social networks. IEEE Intelligent
be mentioned, given its cleverness. Systems, 25(4):26–35, 2010
GERM looks at any possible network motif (see Section 39.1)
and counts how many times each appears in the network. This is a
spectacularly hard problem, and I’ll tell you how to perform it in the
part of this book dedicated to graph mining (Chapter 39). For now,
let’s just assume that an oracle told us how many times each pattern
occurs in our network.
The idea is to identify all graph patterns that are a simple exten-
sions of other, simpler patterns. By “simple extension” I mean that
the consequent should have at most one additional edge – and, pos-
sibly, one additional node – added to its antecedent. This is the hard
part. Once you detect all possible antecedent-consequent pairs, you
can generate the rules, as Figure 20.6 shows.

Frequency Some Rules

Figure 20.6: An example of
12 75% GERM. The frequency of each
pattern is on the left. The graph
9
evolution rules with their rela-
66%
tive frequency is on the right.
Note from the last two rules
6 how the same consequent can
50%
be predicted with different
4 levels of confidence by two
75% different antecedents.

3
284 the atlas for the aspiring network scientist

Suppose you have two patterns G 0 and G 00 , which differ only by

one edge – with G 00 including G 0 , for instance the top two patterns
in Figure 20.6’s left column. Given their frequencies, you know that,
75% of the times you see G 0 , you’ll also see G 00 . So you can infer, with
75% confidence, that G 0 evolves into G 00 .
A crucial difference between GERM and whatever we saw so far is
that it doesn’t directly assign a similarity score to any two particular
nodes. Each of the methods listed so far has a score(u, v) for each u, v
pair. In GERM, rather than iterating over each pair of unconnected
nodes to estimate their score, we iterate over rules and identify which
pairs should be connected.
To understand the process, consider Figure 20.7. Imagine our
starting network is on the left. Suppose that we found two rules
whose antecedents match the data. The first (on top) is a classic triad
closure pattern that we see in many real world networks (see Section
9.2 for a refresher on clustering). The second (on the bottom) says
that these “square” patterns attract a fifth node.

Data Matching Rules Result

Figure 20.7: An example of link
prediction in GERM. The pat-
tern matches two rules – in the
middle –, which we apply to all
combinations of it to obtain, at
the next time step, the extended
pattern on the right.

With the power of these two rules we know we can close the two
open triads we have and add a new neighbor to each node in the
original data. The end result is on the right. We now have more open
triads and could apply the rules again, which would in turn create
more square patterns and so on and so forth. In fact, one could use
GERM not only as a link predictor but also as a graph generator (and
put it in Chapter 14).
Note that I made two simplifications to GERM that the original
papers don’t make. First, in Figure 20.7, I assumed that each rule
applies with the same priority. I ignored its frequency and its confi-
dence. Of course, that would be sub-optimal, so the papers describe a
way to rank each candidate new edge according to the frequency and
confidence of each rule that would predict it.
Second, in all my examples I always assumed that the rules add a
new edge in the next time step and that all edges in the antecedent
are present at the same time. In reality, GERM allows to have more
 for simple graphs 285

complex rules spanning multiple time steps. You could have a rule
saying something like: you have a single edge at time t = 0, you add
a second edge at time t = 1 creating an open triad, and then you close
the triangle at time t = 2. This triad closure rule spans three time
steps, rather than only two.
GERM has a final ace up its sleeve. We can classify new links
coming into a network into three groups: old-old, old-new, and new-
new. We base these groups according to the type of node they attach
to. I show an example in Figure 20.8. An “old-old” link appearing
at time t + 1 connected two nodes that were already present in the
network at time t. These are two “old” nodes. You can expect what
an “old-new” link is: a link connecting an old node with a node
that was not present at time t – a “new” node. New nodes can also
connect to each other in a “new-new” link. If the network represents
paper co-authorships, this would be a new paper published by two or
more individuals who have never published before.

Figure 20.8: Two observations

of the graph G at time t0 (left)
and t00 (right). The node color
encodes its type (red = “old”,
green = “new”). The edge color
Gt' Gt'' encodes its type: gray = original
link; blue = a new “old-old”
Every method we saw so far – and the ones we’ll see – predict link between two old nodes;
exclusively “old-old” links. They work by creating a score between purple = a new “old-new” link
nodes u and v and, if either of those nodes were not present at time between an old node and a new
t, then their score is undefined. GERM is the only method I know node; orange = a new “new-
that is able to predict also old-new and new-new links. Look again at new” link, between two nodes
Figure 20.7: the result of GERM’s prediction has more nodes than the which were not in the graph at
original graph. That is because GERM predicted a few old-new links. time t0 .
There’s nothing stopping GERM to, in principle, predict also
new-new links. However, estimating its precision in doing so is
tricky. Thus the papers presenting the algorithm did not explore that
dimension.
Finally, note how all the rules I mentioned so far have no infor-
mation on the nodes. This is a simplification I made which can be
thrown away with ease. We can have labels on the nodes, so that
we will obtain different link prediction scores for the same edges
depending on the characteristics of the nodes. Figure 20.9 provides
an example. Only one qualitative information can be represented at a
time in this scenario. Also, implementing quantitative node attributes
– such as the degree – is non-trivial, as one would ideally want to
286 the atlas for the aspiring network scientist

Rule Confidence Figure 20.9: An example of link

prediction in GERM consider-
ing also node types. The node
85% color here represent the node’s
label.

45%

encode the fact that the attribute values 1 and 2 should be considered
“more similar” to each other than 1 and 1, 000. Extensions taking care
of these limitations might be possible, but I’m not aware of one.
This is not a function unique to GERM, though. Many link pre-
diction methods can be extended to take into consideration node
attributes as well. In fact, this is also a key ingredient in some net-
work generating processes. Node attributes are used, for instance,
when modeling exponential random graphs, as we saw in Section
16.2. In this case, differently than GERM, quantitative attributes
represent no issue.

20.7 Other Approaches

Link prediction is a vast subfield of network analysis. It is not quite

as vast as community discovery (Part IX) is, but we’re not that far off.
I cannot give justice to all methods out there. I chose the ones for the
previous sections because they are the simplest and most didactic
examples that everyone knows. In this section, I group together the
most prominent examples of “all the rest”.
I don’t have the mental firepower to give you refined intuitions of
the following methods as I did so far. Thus this is going to be just a
big lump of concepts and formulas, necessarily superficial. If I did
it any other way, this would not be a network analysis book, but a
link prediction book. If you want to specialize in the field and really
understand all of these methods – and more! – please go and read
the review papers I cited at the beginning of the chapter, and a few 14
Amir Ghasemian, Homa Hossein-
newer ones14 . Understood? Good. mardi, Aram Galstyan, Edoardo M
And now: God helps us all, we’re going in. Airoldi, and Aaron Clauset. Stacking
models for nearly optimal link predic-
tion in complex networks. Proceedings of
Graph Embeddings. This is a very big family of methods which the National Academy of Sciences, 117(38):
has blasted into scene recently – not only in link prediction, but in 23393–23400, 2020

network analysis as a whole. We’re going to see in details what a

“graph embedding” is in Chapter 37. For now, suffice to say that it
 for simple graphs 287

is a way to represent a node as a vector of values. Depending on

how you construct this vector of values, you could argue that similar
nodes tend to connect to each other. On this basis, you can use any
vector similarity measure to predict links in your network.
This is a dumbed-down version of the general approach shared
by most of graph embedding link predictors. As you might expect,
this general template has been applied in multiple ways to tackle
different challenges. For instance, embeddings can feed their node 15
Muhan Zhang and Yixin Chen. Link
representation to a deep neural network15 ; they can easily go be- prediction based on graph neural
yond purely structural methods, because node attributes are just networks. In Advances in Neural
Information Processing Systems, pages
another entry in the vector that can be fed to a machine learning
5165–5175, 2018
framework16 ; finally, they have been used to predict relations in se- 16
Lizi Liao, Xiangnan He, Hanwang
mantic graphs17 , 18 , 19 , which encode relationships between different Zhang, and Tat-Seng Chua. Attributed
social network embedding. IEEE
entities such as “king marries queen”. Transactions on Knowledge and Data
The list goes on and on and it would deserve a specialized book Engineering, 30(12):2257–2270, 2018
on the subject, but you get the picture so let’s move on. 17
Guoliang Ji, Shizhu He, Liheng Xu,
Kang Liu, and Jun Zhao. Knowledge
graph embedding via dynamic map-
Katz20 . We already saw Katz’s name when we talked about cen- ping matrix. In IJCNLP, pages 687–696,
trality (in Section 11.4). His idea of centrality was one where you get 2015
a centrality contribution from your neighbors, their neighbors, the 18
Théo Trouillon, Johannes Welbl,
Sebastian Riedel, Éric Gaussier, and
neighbors of their neighbors, and so on. With each additional degree Guillaume Bouchard. Complex em-
of separation, Katz established a penalty: the farther away a node is, beddings for simple link prediction.
the less it contributes to your centrality. International Conference on Machine
Learning (ICML), 2016
We can apply a similar strategy to derive our score(u, v). Two 19
T Dettmers, P Minervini, P Stenetorp,
nodes are strongly related if there are many short paths between and S Riedel. Convolutional 2d
them. So one would calculate all paths between u and v and sum knowledge graph embeddings. In 32nd
AAAI Conference on Artificial Intelligence,
their count. Of course, short paths contribute more because they AAAI 2018, volume 32, pages 1811–1818.
represent a closer relationship. Thus the formula would be something AAI Publications, 2018
∞ 20
Leo Katz. A new status index derived
like: score(u, v) = ∑ αl | Pu,v
l |.
from sociometric analysis. Psychometrika,
l =1 18(1):39–43, 1953

Figure 20.10: An example of the

Katz correction for using paths
as a score for link prediction.
u v The shade of blue of a path is
proportional to its contribution
to score(u, v), with darker paths
contributing more.

The assumption is that, the more short paths are between u and v,
the more the two nodes are related. This is regulated by the 0 < α <
1 parameter: a lower α penalizes long paths more – because they have
a high l. Here, α plays the exact same role it did in Katz centrality. If
288 the atlas for the aspiring network scientist

we choose a very small α, the Katz score is practically equivalent to

counting common neighbors, as α2 ∼ 0.
Figure 20.10 shows an example of this correction, with longer
paths fading to almost no contribution.

Hitting Time. We all tried to forget the shenanigans of Section 8.3,

when I defined the hitting time: the expected number of steps it takes
for a random walker to reach v from u. Alas, we’re reminded of it
now, as there is a way to use Hu,v as a basis for score(u, v). After all,
if u and v have very low hitting times, doesn’t it mean that they are
very related, and thus likely to connect?
The easiest way to use hitting time as a score is to simply negate
it (score(u, v) = − Hu,v ) or negate the commute time (score(u, v) =
−( Hu,v + Hv,u )). For instance, in Figure 20.11, nodes 2 and 4 are more
likely to connect (score(2, 4) = −( H2,4 + H4,2 ) = −16) than nodes 1
and 5 (score(1, 5) = −( H1,5 + H5,1 ) = −32) because they are closer.
However, this has the problem of greatly favoring connections to
hubs, since their hitting times as destinations are quite low. Thus
some authors normalize them by multiplying Hu,v with the stationary
distribution of the target (score(u, v) = − Hu,v πv ). Hubs have higher
stationary distribution values, thus they are penalized more.

1 Figure 20.11: (a) A chain of five

nodes. (b) The corresponding
5 2 hitting time matrix H.

4 3
(a) (b)
Another problem of using Hu,v is that the hitting time might
increase even if u and v are nearby in the graph, simply because the
graph has many vertices and edges that can lead the random walkers
astray. To counteract this problem, a solution could be allowing the
random walker to restart from u. This is practically equivalent to
calculate the PageRank, with the difference that you fix the origin of
the random walker. For this reason, since you random walker has a
root (u), it is usually called “rooted PageRank”.
21
Glen Jeh and Jennifer Widom. Scaling
personalized web search. In Proceedings
SimRank21 . As the name suggests, SimRank is based on an idea of
of the 12th international conference on
node similarity. The more similar two nodes are, the more likely they World Wide Web, pages 271–279. Acm,
are to connect. Similarity here is defined recursively: two nodes are 2003

similar if they are connected to similar neighbors. This is a definition

in line with the philosophy of the regular equivalence we saw in
 for simple graphs 289

Section 12.2. Thus, if we say that score(u, u) = 1, we can define all

other scores as:

∑ ∑ score( a, b)
a∈ Nu b∈ Nv
score(u, v) = γ .
ku kv
γ is a parameter you can tune. This is surprisingly similar to the
hitting time approach. The expected value of a SimRank score is γl ,
where l is the length of an average random walk from u to v.
22
Elizabeth A Leicht, Petter Holme, and
Vertex similarity22 . The name of this approach should tip you off Mark EJ Newman. Vertex similarity in
regarding its relationship with SimRank. However, it’s actually much networks. Physical Review E, 73(2):026120,
2006
closer to the Jaccard variant of common neighbor. In fact, the only
difference with Jaccard is the denominator. While Jaccard normalizes
the number of common neighbors by the total possible number of
common neighbors – which is the union of the two neighbor sets –
this approach builds an expectation using a random configuration
graph as a null model. This is a definition in line with the philosophy
of the structural equivalence we saw in Section 12.2.
In practice, score(u, v) = | Nu ∩ Nv |/(k u k v ). This is because two
nodes u and v with k u and k v neighbors are expected to have k u k v
common neighbors (multiplied by a constant derived from the aver-
age degree which would not make any difference as it is the same for
all node pairs in the network).
The same authors in the same paper also make a global variant of
this measure. Their inspiration is the Katz link prediction, where they
again provide a correction for a random expectation in a random
graph with the same degree distribution as G. I won’t provide the
full derivation, which you can find in the paper, but their score is:
  −1
φA
score(u, v) = 2| E|λ1 D −1 I− D −1 .
λ1
The elements in this formula are the usual suspects: | E| is the
number of edges, λ1 is the leading eigenvalue of the adjacency matrix
A (not the stochastic, as that would be equal to one), D is the degree
matrix, and I is the identity matrix. The only odd thing is 0 < φ <
1, which is a parameter you can set at will. This is similar to the
parameter of Katz: smaller φ give more weight to shorter paths.
23
Weiping Liu and Linyuan Lü. Link
Local and superposed random walks23 . These two methods are a close prediction based on local random walk.
EPL (Europhysics Letters), 89(5):58007,
sibling to the hitting time approach. To determine the similarity
2010
between u and v, we place a random walker on u and we calculate
the probability it will hit node v. Note that, if we were to do infinite
length random walks, this would be the stationary distribution π.
This would be bad, as you know that this only depends on the degree
290 the atlas for the aspiring network scientist

of v, not on your starting point u. For this reason, the authors limit
the length of the random walk, and also add a vector q determining
different starting configurations – namely, giving different sources
different weights.
To sum up, the local random walk method determines score(u, v) =
qu πu,v + qv πv,u . The superposed variant works in the same way, with
the difference that the random walker is constantly brought back to
its starting point u. This tends to give higher scores to nodes closer in
the network.
24
Roger Guimerà and Marta Sales-
Stochastic block models24 . We saw the stochastic block models (SBM) Pardo. Missing and spurious interac-
as a way to generate graphs with community partitions (Section 15.2) tions and the reconstruction of complex
– and we will see them again as a method to detect communities networks. Proceedings of the National
Academy of Sciences, 106(52):22073–22078,
(Section 31.1). In fact, any link prediction approach, in a sense, is a 2009
graph generating model. Given the close relationship of SBMs with
community discovery, this class of solutions is particularly related to
the Hierarchical Random Graph approach.

Figure 20.12: (a) The adjacency

matrix of a simple graph. (b)
One of the possible connection
probability matrices that could
generate the graph in (a). Each
(a) (b) cell reports the probability of
Suppose you’re observing a graph, in the form of its adjacency observing the edge.
matrix (Figure 20.12(a)). Given an adjacency matrix, we can infer a
matrix of connection probabilities (Figure 20.12(b)), telling us the
likelihood of observing each edge. We don’t need to know how this
works – we’ll study this process in details when it comes to use SBMs
for community detection – but for now suffice to say we use the
Expectation Maximization algorithm25 .
25
Todd K Moon. The expectation-
maximization algorithm. IEEE Signal
This matrix will give you a probability of observing a connection processing magazine, 13(6):47–60, 1996
that isn’t there (yet). You can use that probability as your score(u, v) 26
Nir Friedman, Lise Getoor, Daphne
for your link prediction. The cool thing about this approach is that, Koller, and Avi Pfeffer. Learning
differently from the ones we saw so far, it can also tell you when an probabilistic relational models. In IJCAI,
volume 99, pages 1300–1309, 1999
observed link is likely to be spurious, because it is associated with a 27
David Heckerman, Chris Meek, and
low probability. Daphne Koller. Probabilistic entity-
This is a family of solutions, because you can bake in different relationship models, prms, and plate
models. Introduction to statistical relational
assumptions on how your stochastic blockmodel works. For instance, learning, pages 201–238, 2007
you can have mixed-membership ones where nodes are allowed to 28
Kai Yu, Wei Chu, Shipeng Yu, Volker
be part of multiple blocks. Or you can have versions working with Tresp, and Zhao Xu. Stochastic relational
models for discriminative link predic-
multilayer networks. tion. In Advances in neural information
processing systems, pages 1553–1560,
Probabilistic models26 , 27 , 28 . In this subsection I group not a single 2007
 for simple graphs 291

method, but an entire family of approaches to link prediction. They

have their differences, but they share a common core. In probabilistic
models, you see the graph as a collection of edges and attributes at-
tached to both nodes and edges. The hypothesis is that the presence
of an edge is related to the values of the attributes.
In practice, the hypothesis is that there exist a function taking
as input the attribute values of each node and edge. This function
models the observed graph. Then, depending on the values of the
attributes for nodes u and v, the function will output the probability
that the u, v edge should appear – and with which attributes.
29
Fei Tan, Yongxiang Xia, and Boyao
Mutual information29 . In Section 2.8 I introduced the concept of Zhu. Link prediction in complex
mutual information: the amount of information one random variable networks: a mutual information per-
gives you over another. This can be exploited for link prediction. If spective. PloS one, 9(9):e107056, 2014

you remember how it works, you’ll remember that MI allows you to

calculate the relationship between two non-numerical vectors, which
is not really possible using other correlation measures – not without
doing some non-trivial bending of the input. In link prediction, this
advantage is crucial: you can define your function as score(u, v) =
MIuv , where the “events” that allow you to calculate MIuv are the
common neighbors between nodes u and v.

CAR Index30 . In this index you favor pairs of nodes that are part of
30
Carlo Vittorio Cannistraci, Gregorio
Alanis-Lobato, and Timothy Ravasi.
a local community, i.e. they are embedded in many mutual connec- From link-prediction in brain connec-
tions. This is a variant of the idea of common neighbor: each shared tomes and protein interactomes to the
local-community-paradigm in complex
connection counts not equally, but proportionally more if it also networks. Scientific reports, 3:1613, 2013
shares neighbors with u and v. This basic idea can be implemented
in multiple ways, depending on which of the traditional link predic-
tion methods we want to extend. For instance, if we extend vanilla
common neighbors, you’d say that:

| Nu ∩ Nv ∩ Nz |
score(u, v) = ∑ 1+
2
.
z∈ Nu ∩ Nv

Note how here we simply added a second intersection to the basic

common neighbors, the one with the common neighbor z. Figure

a
Figure 20.13: Comparing the
u v CAR index contribution to
score(u, v) for nodes a and b.
Node size is proportional to the
b contribution.
292 the atlas for the aspiring network scientist

20.13 shows an example. Node a in this case contributes much more

than node b, because it shares four common neighbors with u and v.
Node b, on the other hand, lies outside this local community.
One could use this CAR approach to create a new family of mea-
sures. For instance, we can have a CAR version of the resource
allocation approach:

| Nu ∩ Nv ∩ Nz |
score(u, v) = ∑ | Nz |
.
z∈ Nu ∩ Nv

Here, we normalize by z’s bandwidth.

31
Daniel M Dunlavy, Tamara G Kolda,
Katz Tensor Factorization31 . This approach doesn’t really add a and Evrim Acar. Temporal link predic-
tion using matrix and tensor factoriza-
new idea to link prediction, but it is worthwhile mentioning for a tions. ACM Transactions on Knowledge
few reasons. It is an application of the Katz criterion we saw earlier. Discovery from Data (TKDD), 5(2):10,
2011
However, it implements it via tensor factorization. If you recall
Section 5.1, we could view particular tensors as “3D” matrices. In
practice, they are a collection of adjacency matrices. In that section
I introduced the idea to represent multilayer networks, where each
adjacency is a layer of the network. Here, instead, each layer is a
temporal snapshot of the network.
The advantage of this approach is that, like GERM (Section 20.6),
one could predict links not necessarily at time t + 1, but also at t + 2,
t + 3, etc... This is possible because the adjacency matrix at any time
t is simply a slice of the tensor. Thus, there’s nothing fundamentally
different between predicting a link in the slice t + 1 or in the slice
t + 3 – your precision might be a bit lower because of the strongly
sequential nature of link appearance, but it’s still possible to provide
a good guess.
32
Cynthia Dwork, Ravi Kumar, Moni
Naor, and Dandapani Sivakumar. Rank
Rank aggregation techniques32 . When it comes to link prediction,
aggregation methods for the web. In
you can go full meta. And I never drop an opportunity to go full Proceedings of the 10th international
meta. You can take all – yes, I mean all – the methods listed so far. conference on World Wide Web, pages
613–622. ACM, 2001
Each method will rank unobserved edges differently. Meaning that
they have an ordered list of preferences as to which new link should
be the next one observed. You can use a rank aggregation method to
create a final list that uses all the information from all the methods.
Rank aggregation is the general process of having two ordered list
and producing a single ordered list that is the one agreeing the most
with both your inputs.
There is a classical way to solve the issue, which is the Borda’s 33
[Link]
Borda_count
method33 . This is what happens in those electoral systems where
citizens will vote not for one candidate, but will rank their top n
candidates. Each candidate receives a number of points proportional
to its rank, and the candidate with most points win.
 for simple graphs 293

A more sophisticated aggregation methods uses the Kendall τ.

The Kendall τ counts the number of pairwise disagreements: pairs
of edges that have the opposite rankings in the two lists. If in one
list u1 , v1 is ranked higher than u2 , v2 while the opposite holds in the
other list, then you have a disagreement – this is sort of similar to the 34
Douglas G Bonett and Thomas A
Spearman rank correlation34 . Wright. Sample size requirements
for estimating pearson, kendall and
spearman correlations. Psychometrika, 65
20.8 Summary (1):23–28, 2000

1. In link prediction we want to take an observed network and infer

the most likely connections to appear in the future, or the ones
that might already be there but for some reason we aren’t seeing
yet.

2. The most common approach is to compute a score for each pair

of unconnected nodes by using some theory about the topology of
the network. For instance, we can say that usually triangles close
and thus count the number of common neighbors between two
nodes.

3. Other approaches model mesoscale structures of the network

such as communities, or find overexpressed graph patterns in the
network and rank node pairs on whether they are likely to make
more of these patterns appear in the network.

4. Many approaches from other branches of network science can

be used to predict links, for instance ranking algorithms (Katz),
random walk hitting time, stochastic blockmodels, mutual informa-
tion, etc.

5. Nothing stops you from using all the link prediction methods at
once and then aggregate their results. Really, it’s a free country.

20.9 Exercises

1. What are the ten most likely edges to appear in the network at
[Link] accord-
ing to the preferential attachment index?

2. Compare the top ten edges predicted for the previous question
with the ones predicted by the jaccard, Adamic-Adar, and resource
allocation indexes.

3. Use the mutual information function from scikit-learn to im-

plement a mutual information link predictor. Compare it with the
results from the previous questions.
294 the atlas for the aspiring network scientist

4. Use your code to calculate the hitting time (from exercise 3 of

Chapter 8) to implement a hit time link predictor – use the com-
mute time since the network is undirected. Compare it with the
results from the previous questions.
21
For Multilayer Graphs

Link prediction takes a distinctive new flavor when your input is a

multilayer network. In single layer networks, all you have to do is
asking the question: “who will be the next two people to become
friends with each other?” The question becomes harder and more
interesting in multilayer networks. Here you don’t want to know only
which two people will connect next, but also how. Are they going
to be best buds? Work colleagues? Lovers? Enemies? You see that
this new dimension adds a lot of spice to the problem. You don’t
want to make a friend suggestion on Facebook for two people with
a strong connection prediction if you also knew that the connection
type between the two would be an enmity link – and researchers in 1
Jiliang Tang, Shiyu Chang, Charu
social media studies have looked at this problem1 . Aggarwal, and Huan Liu. Negative link
Multilayer link prediction is the topic of this chapter. We start prediction in social media. In Proceedings
of the eighth ACM international conference
from the simplest case where we have only two layers in the network on web search and data mining, pages
with a very precise semantic: link prediction in signed networks (Sec- 87–96. ACM, 2015b
tion 21.1). We then move on to the generalized case of an arbitrary
number of layers with no clear semantic relationship with each other
(Section 21.2).

21.1 Signed Networks

Social Balance Theory

There are two reasons why signed networks represent the simplest
case of link prediction when your network has multiple different
types of connections. First, signed networks are a subtype of multi-
layer networks with strong constraints on the edges. You can only
have two edge types: positive and negative. Moreover, these edge
types are exclusive: if you have a positive edge between u and v, you
cannot have also a negative one – unless the network is directed and
the edge direction flows in the opposite way. This reduces the search
space for a link predictor.
296 the atlas for the aspiring network scientist

The second reason why signed networks are easier to predict

is because the positive/negative sign of an edge gives it a precise
meaning. We have strong priors as to which structures we can see
in a signed network. These are discussed in what we call “Social 2
Fritz Heider. The psychology of inter-
Balance Theory”2 , 3 . According to the theory, positive and negative personal relations. Psychology Press,
relationships are balanced. For instance, if I have two friends, they 2013
are more likely to like each other. On the other hand, if I have an
3
Tibor Antal, Pavel L Krapivsky, and
Sidney Redner. Dynamics of social
enemy, I expect my friends to be enemies of them as well. balance on networks. Physical Review E,
72(3):036121, 2005

Figure 21.1: The four possible

types of triangles when consid-
ering a mutually exclusive pair
of positive (in green) and neg-
(a) (b) (c) (d) ative (in red) relationships. (a)
Social balance theory looks predominantly at triangles – although and (b) are balanced triangles
there are ways to look at longer cycles4 . It divides them in two because they have an odd num-
classes: balanced and unbalanced, see Figure 21.1. Balanced triangles ber of positive relationships.
can be understood with common sense: friend of friend is my friend, (c) and (d) are classically con-
enemy of my friend is my enemy. Unbalanced triangles are relation- sidered unbalanced, although,
ships that we expect to change in the future. In fact, the prediction is under certain circumstances, (c)
that balanced triangles are overexpressed in real networks over our can be considered a balanced or
expectation – and unbalanced triangles are underexpressed –, and neutral configuration.
that is generally observed.
One note about the all-negative triangle (Figure 21.1(c)): in some 4
Kai-Yang Chiang, Nagarajan Natara-
jan, Ambuj Tewari, and Inderjit S
views it is not considered unbalanced, and it is in fact more com- Dhillon. Exploiting longer cycles for
monly found in real networks than the other unbalanced triangle link prediction in signed networks. In
(Figure 21.1(d)). A typical case of balanced all-negative triangle is Proceedings of the 20th ACM international
conference on Information and knowledge
campanilism in Tuscany: the worst enemy of a person from Pisa is management, pages 1157–1162. ACM,
a person from Livorno. The second worst enemy for both of them is 2011
a person from Lucca. And people from Lucca hate indiscriminately
both Pisa and Livorno. And this has gone on for centuries. Pretty
balanced.
You can calculate a summary statistics telling how much, on
average, your whole network is balanced. There are many ways 5
Frank Harary. On the measurement of
structural balance. Behavioral Science, 4
to do this, but I think the most popular one is called frustration5 . (4):316–323, 1959
In frustration, you count the number of edges whose removal –
or negation – would result in a perfectly balanced network. You
can normalize this over the total number of edges in the network.
Frustration is a bit computational complex to calculate, but there are 6
Samin Aref and Mark C Wilson.
Balance and frustration in signed
heuristics you can use to speed up your calculations6 . Figure 21.2 networks. Journal of Complex Networks, 7
shows an example network with two unbalanced triangles: (1, 2, 3) (2):163–189, 2019
and (2, 3, 6). Both triangles would turn balanced if we were to flip the
sign of edge (2, 3) – or, alternatively, frustration would dissipate if
 for multilayer graphs 297

we were to remove the edge altogether. Thus, the frustration of this

graph is 1/9, since it contains 9 edges.

4 1 2 Figure 21.2: A graph with two

unbalanced triangles, the ones
including edge (2, 3).

5 3 6

This relates to link prediction when we consider evolving signed

networks. If we find a configuration with three nodes connected
by two positive edges, it is overwhelmingly more likely that, in the
future, the triangle will close with a positive relationship (Figure
21.1(a)) rather than with a negative one (Figure 21.1(d)). On the other
hand, if we find a positive and a negative relationship, we expect the
triangle to close with a negative edge (Figure 21.1(b)). The case with
an initial condition of two negative edges is more difficult to close,
but we prefer to close it with a positive edge (Figure 21.1(b)) than
with a negative one (Figure 21.1(c)).
So you see that you can perform signed link prediction by first
predicting the pair of nodes that will connect, calculating a score(u, v)
with any of the methods presented in the previous chapter. Then you
will decide the sign of the link, by using social balance theory.

Social Status Theory

There is a competing theory to social balance, which is the status
7
Jure Leskovec, Daniel Huttenlocher,
theory7 . This arises from a different interpretation of the sign. A
and Jon Kleinberg. Signed networks
positive sign in a social setting might mean that the user originating in social media. In Proceedings of the
the link feels to be lower status than – and thus giving social credit SIGCHI conference on human factors in
computing systems, pages 1361–1370.
to – whomever receives the link. Conversely, a negative link is a way ACM, 2010a
for a higher status node to shoot down a lower status one. Note that
here we started talking about the direction of an edge, meaning that
we have more than four types of triangles. In fact, we have 32.
Figure 21.3 shows the 16 main configurations of these triangles.
The closing edge connecting u to v can be either positive or negative,
generating 32 final possible configurations. Status theory generates
predictions that are more sophisticated and – sometimes – less imme-
diately obvious. Some cases are easy to parse. For instance, consider
Figure 21.3(a). The objective is to predict the sign of the (u, v) edge.
In the example, u endorses z as higher status. z endorses v. If v is on
a higher level than z, and z is on a higher level than u, then it’s easy
to see how v is also on a higher level than u. Thus the edge will be
298 the atlas for the aspiring network scientist

v v v v
Figure 21.3: The 16 templates
of directed signed triangles
u z u z u z u z in social status networks. The
(a) (b) (c) (d) color of the edge determines
v v v v its status: green = positive, red
= negative, blue = the edge we
are trying to predict – can be
u z u z u z u z either positive or negative.
(e) (f) (g) (h)
v v v v

u z u z u z u z
(i) (j) (k) (l)
v v v v

u z u z u z u z
(m) (n) (o) (p)

of a positive sign. In fact, this specific configuration is grossly over

represented in real world data.
The situation is not as obvious for other triangles. For instance,
the one in Figure 21.3(i). Here we have the z node endorsing both u
and v. We don’t really know anything about their relative level, only
that they are both on a higher standing with respect to z. The paper
presenting the theory makes a subtle case. The edge connecting u to
v is more likely to be positive than the generative baseline on u, but
less likely to be positive than the receiving baseline of v. So, suppose
that 50% of edges originating from u are positive, while 80% of the
links v receives are positive. The presence of a triangle like the one in
Figure 21.3(i) would tell us that the probability of connecting u to v
with a positive link is higher than 50%, but lower than 80%.
Given this sophistication, and the fact that social status works with
more information than social balance – namely the edge’s direction
–, it is no wonder that there are cases in which social status vastly
outperforms social balance. For instance, the original authors apply
social status to a network of votes in Wikipedia. Here the nodes
are users, who are connected during voting sessions to elect a new
admin. The admin receives the links, positive if the originating
user voted in favor, negative if they voted against. Triangles in this
network connect with the patterns predicted by social status theory.
 for multilayer graphs 299

Atheoretical Sign Prediction

Of course, calling onto us the powers we unlocked in Section 20.6,
we can apply a strategy similar to GERM to extract graph association 8
Giacomo Bachi, Michele Coscia, Anna
rules also in this scenario8 . Figure 21.4 shows two possible associa- Monreale, and Fosca Giannotti. Clas-
tion rules extracted from two different datasets. Looking at the figure, sifying trust/distrust relationships in
online social networks. In 2012 Inter-
two advantages for this strategy emerge. national Conference on Privacy, Security,
First, using a variation of GERM we free ourselves from the Risk and Trust and 2012 International
tyranny of the triangles. We can look at an arbitrary set of rules, Confernece on Social Computing, pages
552–557. IEEE, 2012
not necessarily involving three nodes and triadic closure, which may
not apply for all networks.

Figure 21.4: Two possible graph

Second, what social balance and status have in common is that
association rules extracted from
they will make the same prediction no matter the network you’re
a directed signed network.
going to have as input. They establish universal laws that might
apply in general, but overlook specific laws that might apply for
the phenomenon we’re observing right now. For instance, voting in
Wikipedia might be very different from trusting someone’s opinion in
a product review database. Consider the rule on the right in Figure
21.4. That is a voting pattern in Wikipedia. It makes sense from a
social status point of view: the node receiving the last negative link
should expect to receive it, because it already received one, so it
received a signal of being of low status.
But that rule makes little sense when moving to the scenario of
trusting reviews. What the negative link means here is that the origi-
nator of the edge doesn’t trust the recipient of the edge. However, we
already know that the node originating the last link disagrees with
the bottom two nodes, who trust each other. So we would expect it
to trust – to send a positive rather than a negative link – to the node
in the top right. In fact, while the pattern is widely popular in the
Wikipedia network, it doesn’t appear in the social review dataset.

21.2 Generalized Multilayer Link Prediction

So far we have only considered the case of two possible edge types.
Moreover, these two types have a clear semantic: one type is positive,
300 the atlas for the aspiring network scientist

the other is negative. Both assumptions make the link prediction

problem easier: there are few degrees of freedom and we move in
a space constrained by strong priors. It is now time to drop these
assumptions and face the full problem of multilayer link prediction
as the big boys we are.
Generalized multilayer link prediction is the task of estimating
the likelihood of observing a new link in the network, given the
two nodes we want to connect and the layer we want to connect
them through. Nodes u and v might be very likely to connect in the
immediate future, but they might do so in any, some, or even just a
single layer. Thus, we extend our score function to take the layer as
an input: from score(u, v) to score(u, v, l ).

Layer Independence

As you might expect, there are tons of ways to face this problem.
The most trivial way to go about it is to apply any of the single layer
link prediction methods from Chapter 20 to each layer separately.
9
Manisha Pujari and Rushed Kanawati.
Then, you can create a single ranking table by merging all these Link prediction in multiplex networks.
predictions9 . NHM, 10(1):17–35, 2015

b Figure 21.5: The easiest way

e Nodes Layer CNs
a, b l2 1 to perform multilayer link
c
f a, b l3 1 prediction. Given the input
a d
l1 b, d l1 2 network, perform single layer
g
b, d l2 1 link prediction on each of the
l2 b, d l3 1 layer separately. In this case, we
a, g l1 1 count the number of common
a, g l2 1 neighbors between pairs of
l3
a, g l3 0 nodes. We then predict the one
with the overall highest score.
(b)
(a)

Figure 21.5 depicts an example for this process. Note that here
I use a rather trivial approach to aggregate, by comparing directly
the various scores. One could also apply to this problem the rank
aggregation measures presented in the previous chapter. In this way,
you could also aggregate different scores using different criteria:
common neighbors, preferential attachment, and so on.
This is practically a baseline: it will work as long as we have an
assumption of independence between the layers. As soon as having a
link in a layer changes the likelihood of connecting into another layer,
we expect to grossly underperform.
 for multilayer graphs 301

Blending Layers
A slightly more sophisticated alternative is to consider the multilayer
network as a single structure and perform the estimations on it. For
instance, consider the hitting time method. This is based on the
estimation of the number of steps required for a random walker
starting on u to visit v. We can allow the random walker to, at any
time, use the inter layer coupling links exactly as if they were normal
edges in the network. At that point, a random walker starting from
u in layer l1 can and will visit node v in layer l2 . The creation of
our connection likelihood score is thus well defined for multilayer
networks. Figure 21.6 depicts an example for this process.

b
e
Figure 21.6: A slightly more so-
phisticated way to perform mul-
c
f
a d tilayer link prediction. Given
l1
g the input network, perform the
link prediction procedure on
l2
the full structure. In this case,
the gray arrow simulates a ran-
l3
dom walker going from node g
in layer l3 to node a in the same
layer, passing through node
These paths crossing layers are often called meta-paths. The in- c in layer l2 . The mutlilayer
formation from these meta-paths can be used directly as we just random walker contributes to
saw, informing a multilayer hitting time. Or we can feed them to a the score( g, a, l3 ).
classifier, which is trying to put potential edges in one of two cate-
10
Mahdi Jalili, Yasin Orouskhani,
gories: future existing and future non-existing links. Any classifier
Milad Asgari, Nazanin Alipourfard,
can perform this job once you collect the multilayer information from and Matjaž Perc. Link prediction in
the meta-path: naive Bayes, support vector machines (SVM), and multiplex online social networks. Royal
Society open science, 4(2):160863, 2017
others10 . 11
Darcy Davis, Ryan Lichtenwalter,
Other extensions to handle multilayer networks have been pro- and Nitesh V Chawla. Multi-relational
posed11 . These studies show that multilayer link prediction is indeed link prediction in heterogeneous
information networks. In ASONAM,
an interesting task, as there is a correlation between the neighbor- pages 281–288. IEEE, 2011
hood of the same nodes in different layer. The classical case involves
the prediction of links in a social media platform using information 12
Desislava Hristova, Anastasios
about the two users coming from a different platforms12 . Such layer- Noulas, Chloë Brown, Mirco Musolesi,
layer correlations are not limited to social media, but can also be and Cecilia Mascolo. A multilayer
approach to multiplexity and link
found in infrastructure networks13 . prediction in online geo-social networks.
EPJ Data Science, 5(1):24, 2016
13
Kaj-Kolja Kleineberg, Marián Boguná,
Multilayer Scores M Ángeles Serrano, and Fragkiskos
Papadopoulos. Hidden geometric
The last mentioned strategy is better, but it still doesn’t consider correlations in real multiplex networks.
all the wealth of information a multilayer network can give you. To Nature Physics, 12(11):1076, 2016
302 the atlas for the aspiring network scientist

see why, let’s dust off the concept of layer relevance we introduced
in Section 6.1. That is a way to tell you that a node u has a strong
tendency of connecting through a specific layer. If a layer exclusively
hosts many neighbors of u, that might mean that it is its preferred
channel of connection.

Figure 21.7: The multilayer

neighborhood of nodes u and v.
Edge color indicates the layer to
which the connection belongs.
u v

This suggests that other naive ways to estimate node-node sim-

ilarity for our score should be re-weighted using layer relevance14 .
14
Giulio Rossetti, Michele Berlingerio,
and Fosca Giannotti. Scalable link
Consider Figure 21.7. We see that the two nodes have many common prediction on multidimensional net-
neighbors in the blue layer. They only have one common neighbor works. In 2011 IEEE 11th International
Conference on Data Mining Workshops,
in the red layer. However the blue layer, for both nodes, has a very pages 979–986. IEEE, 2011
low exclusive layer relevance. There is no neighbor that we can reach
using exclusively blue edges. In fact, in this case, the exclusive layer
relevance is zero.
The opposite holds for the layer represented by red edges. There
are many neighbors for which red links are the only possible choice.
In this particular case, we might rank the red layer as more likely
to host a connection for nodes u and v. In practice, this boils down
to multiplying the layer relevance to the common neighbor score.
Such weighting schema can be applied to most of the link prediction
strategies we saw so far.
A related approach tries to estimate not whether a link will exist
in the future, but its strength. If we have an unweighted multilayer
network, we might still be able to estimate how strong a connection
is. By looking at the various layers connecting two nodes, one could
estimate such tie strength15 .
15
Luca Pappalardo, Giulio Rossetti, and
Dino Pedreschi. " how well do we know
Another approach to multilayer link prediction is the usage of each other?" detecting tie strength in
tensor factorization. We briefly mentioned tensor factorization at the multidimensional social networks. In
2012 IEEE/ACM International Conference
end of the previous chapter, for single layer link prediction. In that on Advances in Social Networks Analysis
case, the third dimension of our tensor was representing time. In and Mining, pages 1040–1045. IEEE,
this case, we can apply the same technique by changing the meaning 2012

of this third dimension. Rather than using it to represent time, we

can use it to represent the layer in which the edge appear. The same
technique can now be applied, to discover in which layer new edges
 for multilayer graphs 303

are likely to pop up.

And, since we’re mentioning flexible methods that can be applied
in multiple scenarios, why don’t we dust off GERM again? We
already saw how graph association rules can be extracted in signed
networks without batting an eye. There is, in principle, no issue in 16
Michele Coscia and Michael Szell.
extending the algorithm to deal with multilayer networks16 – as long Multiplex graph association rules
as you can properly and efficiently solve the graph isomorphism for link prediction. arXiv preprint
arXiv:2008.08351, 2020
problem (Section 39.2) for labeled multigraphs.
Figure 21.8 shows multilayer association rules in all their glory. In
this case I encode also node attributes – because why not? – render-
ing the rules extracted by multilayer GERM extremely multifaceted.
By collecting all the rules I showed so far in this book part, you real-
ize that there are really a lot of ways to close a triangle in complex
networks!
By using the edge labels to represent the layer in which an edge
appears, we lose one of the powers of GERM. Namely, we are not
able to make predictions at time steps farther than one. Remember 17
Suhang Wang, Jiliang Tang, Charu
that, with GERM, we could predict that a link will appear at time t + 2. Aggarwal, Yi Chang, and Huan Liu.
Signed network embedding in social
This is not the case any more here, because the way we were able to
media. In Proceedings of the 2017 SIAM
do that was by encoding the edge arrival time in its label. But here international conference on data mining,
we’re using the edge label to indicate the layer in which it appears. pages 327–335. SIAM, 2017b
18
Antoine Bordes, Nicolas Usunier,
This is an acceptable price to pay if we’re able to perform multilayer
Alberto Garcia-Duran, Jason Weston,
link prediction. and Oksana Yakhnenko. Translating
Finally, as in the single layer case, there are some promising ap- embeddings for modeling multi-
relational data. In Advances in neural
proaches using multilayer network embedding to predict links in information processing systems, pages
multilayer networks, both in the singed case17 and in the multilayer 2787–2795, 2013
proper case18 , 19 , 20 . Hongming Zhang, Liwei Qiu, Lin-
19

gling Yi, and Yangqiu Song. Scalable

multiplex network embedding. In IJCAI,
Heterogeneous Networks volume 18, pages 3082–3088, 2018
20
Ryuta Matsuno and Tsuyoshi Murata.
In computer science, specifically in data mining and machine learn- Mell: effective embedding method for
multiplex networks. In Companion
ing, multilayer networks are often called “heterogeneous” networks, Proceedings of the The Web Conference
because they have edges of different, heterogeneous, types. Heteroge- 2018, pages 1261–1268, 2018

Rule Confidence

85% Figure 21.8: Two possible graph

association rules extracted with
the multilayer version of GERM.
Node color represents the
45% node’s label, while edge color
represents the edge’s layer.
304 the atlas for the aspiring network scientist

neous link prediction is one of the main tasks tackled in this subfield. 21
Yizhou Sun, Rick Barber, Manish
This is actually where metapaths were firstly developed21 . Figure Gupta, Charu C Aggarwal, and Jiawei
21.9 shows examples of possible metapaths in a co-authorship net- Han. Co-author relationship prediction
in heterogeneous bibliographic net-
work. These metapaths form the input of a classifier, which will then works. In 2011 International Conference on
spit out the most likely new metapaths involving specific nodes. Advances in Social Networks Analysis and
Mining, pages 121–128. IEEE, 2011
Other common approaches use a ranking factor graph model22 , 22
Yuxiao Dong, Jie Tang, Sen Wu, Jilei
which searches for common general patterns shared by the various Tian, Nitesh V Chawla, Jinghai Rao, and
layers of the network; or consider link prediction as a matching Huanhuan Cao. Link prediction and
recommendation across heterogeneous
problem23 .
social networks. In 2012 IEEE 12th
By the way, the converse of what I said about GERM and tensor International conference on data mining,
factorization applies also to heterogeneous link predictions. There is pages 181–190. IEEE, 2012
23
Xiangnan Kong, Jiawei Zhang, and
research showing how you can use this class of approaches to predict
Philip S Yu. Inferring anchor links
when an edge will appear, rather than its type24 . across multiple heterogeneous social
I should also mention that link prediction, community discovery, networks. In Proceedings of the 22nd ACM
international conference on Information &
and generating synthetic networks are sides of the same weirdly Knowledge Management, pages 179–188.
triangular coin. This holds also for multilayer networks. There are ACM, 2013
efforts to create models generating multilayer networks than can
24
Yizhou Sun, Jiawei Han, Charu C
Aggarwal, and Nitesh V Chawla. When
then be applied to predict new links on already existing real-world will it happen?: relationship prediction
multilayer networks25 , 26 . in heterogeneous information networks.
In Proceedings of the fifth ACM interna-
In this chapter, as in all chapters of this book, I presented only
tional conference on Web search and data
the most prominent methods to tackle the issue at hand, and the mining, pages 663–672. ACM, 2012
ones I’m most familiar with. The study of a deeper review work27 25
Caterina De Bacco, Eleanor A Power,
Daniel B Larremore, and Cristopher
is necessary if you want to make a living off solving multilayer link
Moore. Community detection, link
prediction. prediction, and layer interdependence
in multilayer networks. Physical Review
E, 95(4):042317, 2017
26
A. Roxana Pamfil, Sam D. Howison,
and Mason A. Porter. Edge correlations
in multilayer networks. arXiv preprint
arXiv:1908.03875, 2019

Figure 21.9: Some examples of

metapaths in a heterogeneous
network with multiple node
types, in a scientific publication
scenario. From top to bottom,
connecting authors because:
they co-author a paper (both
nodes of type authors are con-
nected to the same node of type
paper); they cite each other (a
node of type author connects
NetSci
to a node of type paper citing
another paper-type node); or
they publish in the same venue.
 for multilayer graphs 305

21.3 Summary 27
Yizhou Sun and Jiawei Han. Mining
heterogeneous information networks: a
structural analysis approach. Acm Sigkdd
1. In multilayer link prediction, besides predicting the appearance of Explorations Newsletter, 14(2):20–28, 2013
a new edge, you also need to guess in which layer the new edge
will appear. It’s not only about whether two nodes will connect, it’s
also about how they will connect.

2. A simplified version of multilayer link prediction involve signed

networks. In this case, real world networks have a preference for
balanced structures, which you can use to predict the sign of the
relationship.

3. An alternative approach is by using status theory. Whether you

should use social balance or social status depends on what your
network represents, for instance trust would follow balance, while
voting would follow status.

4. For generalized multilayer link prediction the common approach

is to create multilayer generalizations of single layer predictors,
with some strategy to aggregate multilayer information.

5. Other approaches rely on multilayer extensions of graph mining

and on the use of metapaths: paths connecting nodes across layers.

21.4 Exercises

1. You’re given the undirected signed network at [Link]

[Link]/exercises/21/1/[Link]. Count the number of
triangles of the four possible types.

2. You’re given the directed signed network at [Link]

eu/exercises/21/2/[Link]. Does this network follow social bal-
ance or social status? (Consider only reciprocal edges. For social
balance, the reciprocal edges should have the same sign. For social
status they should have opposite signs)

3. Consider the multilayer network at at [Link]

eu/exercises/21/3/[Link]. Calculate the Pearson correlation
between layers (each layer is a vector with an entry per edge. The
entry is 1 if the edge is present in the layer, 0 otherwise). What
does this tell you about multilayer link prediction? Should you
assume layers are independent and therefore apply a single layer
link prediction to each layer?
22
Designing an Experiment

As the name of the problem suggests, link prediction is fundamen-

tally a task that involves making claims about the future. Evaluating
the performance of an oracle in getting things right is harder than
it might seem. There are surprising ways to get it wrong. Luckily,
making predictions is the bread and butter of machine learning. Thus
we have a large set of best practices we can follow. This chapter is
a crash course on those which apply particularly to link prediction. 1
Yang Yang, Ryan N Lichtenwalter,
This is mostly taken from the literature1 , which you should check out and Nitesh V Chawla. Evaluating link
to get a deeper view on the problem. prediction methods. Knowledge and
Information Systems, 45(3):751–782, 2015
The chapter is divided in two parts. First, we have to figure out
how to perform the test (Section 22.1). Since link prediction is about
the future, one option would be to just wait until new data comes in.
This is often not ideal, because you don’t really know when you’ll
get new information, and you want to publish your paper right now.
So you have to work with the data you have. Once you make your
prediction, you have to then evaluate how well you perform (Section
22.2).

22.1 Train/Test Sets

The Basics
When it comes to evaluate your prediction algorithm, you have to
distinguish between the training and the test datasets. The training
dataset is what your model uses to learn the patterns it is supposed
to predict. For instance, if you’re doing a common neighbor link
predictor, the training dataset is what you use to count the number of
shared connections between two nodes. Once you’re done examining
the input data, you have generated the results of the score(u, v)
function for all possible pairs of u, v inputs.
The test dataset is a set of examples used to assess performance.
When your model is done learning on the training dataset, it is
 designing an experiment 307

unleashed on the test dataset and will start making predictions.

Every time it gets it right you increase its performance, every time it
gets it wrong you decrease it.
Figure 22.1 shows an example of the difference between the two
sets. Figure 22.1(a) is the training dataset: it contains all the informa-
tion we can use to infer out scores. Figure 22.1(b) are the scores we
calculate based on the data from Figure 22.1(a). Specifically, for each
pair of nodes I calculate the number of common neighbors shared
by the two nodes. Figure 22.1(c) shows the test set in blue. These
are the actual new edges that appeared in the network. I include the
training set in gray because it makes it easier to put the new edges
into context – besides, when performing a prediction you need to
make sure you remember what was in the training set and discard
any prediction you might have done about those edges. For instance,
in this case, we need to throw away the (1, 2) edge prediction.

Edge Score
5 5 Figure 22.1: An example of
1, 2 1
4 6 1, 3 2 4 6 train and test sets for a network.
The information (a) we use to
1, 4 1
8 1, 5 1 8 build the score table (b), using
3 7 3 7 the common neighbor approach.
1, 6 2
I highlight the test edges (c) in
2, 4 2
2 1 2 1 blue.
... ...
(a) Train (c) Test
(b) Scores
In machine learning there is also what you’d call a “validation”
dataset, for the tuning of the parameters, but that usually doesn’t
apply to link prediction. Link predictors usually have no or a trivial
number of parameters, therefore you can safely conflate training and
validation in the same set.
There is a fundamental tenet for making data-driven predictions
that still holds. You can never ever ever use the data that trained
your model to test it. In other words, training and test sets have to
be disjoint. If you test your method on the same data that trained
it, you’re going to overfit: your method is going to learn only the
specifics of the training set and nothing about the general forces that
shaped it the way it is.
What this means in link prediction is that you cannot claim to have
predicted a link that was already in your data. You have to focus
only on those pairs of nodes that were not connected in the training
set. That is why in Figure 22.1(c) the edges that were already in the
training set are gray rather than blue: we won’t make predictions on
those, because we already know they exist.
So now the problem is: how do you do that? If you have a net-
308 the atlas for the aspiring network scientist

t=1 Figure 22.2: Two approaches to

Temporal t=1
build your train and test sets
Info for link prediction. On the left
t=2
Train
you have the input data. On the
Test right, the partition of links into
the two sets: train (green) and
Cross
Validation
test (blue).

work, how do you divide it into training and test sets?

You have two options, as Figure 22.2 shows. If you have temporal
information on your edges you can use earlier edges to predict the
later ones. Meaning that your train set only contains links up to time
t, and the test set only contains links from time t + 1 on. If you don’t
have the luxury of time data, you have to do n-fold cross validation:
divide your dataset in a train and test set (say 90% of edges in train
and 10% in test) and then perform multiple runs of train-test by 2
Ron Kohavi et al. A study of cross-
rotating the test set so that each edge appears in it at least once2 . validation and bootstrap for accuracy
estimation and model selection. In Ijcai,
volume 14, pages 1137–1145. Montreal,
Specific Issues Canada, 1995

Link prediction comes with a few peculiarities that might not be a

problem in other machine learning tasks. I focus on two: size of the
search space and sparsity of positives.
The first refers to the fact that, as we saw in Section 9.1, real
networks are sparse. I made the example of the Internet back-
bone: with its 192, 244 nodes, the number of possible edges is
|V |(|V | − 1)/2 = 18, 478, 781, 646. However, it only contains 609, 066
actual edges. This means that, if you were to use its current state
as a train set, the score function would have to compute a result for
18, 478, 781, 646 − 609, 066 = 18, 478, 172, 580 potential edges. That
is an unreasonable burden, both for computation time and memory
storage.
For this reason, when you perform link prediction you will often
sample your outputs. You will not calculate score(u, v) for every
possible u, v pair, but you will sample the pairs according to some
expectation criterion. Such criterion can be as hard to pin down as
the link prediction problem itself.
The second problem is intertwined with the first. Suppose that the
Internet backbone adds edges at a 5% rate per time step. That means
that, if at time t you had 609, 066 edges, at time t + 1 you will observe
 designing an experiment 309

609, 066 × .05 ∼ 30, 453 new edges. As we just saw, the number of
potential edges is just above 18B. Putting these two facts together
lets us reach an absurd conclusion: we can build a link prediction
method that will tell us that no new link will ever appear. If we do
so, we would be right 99.999% of the times. We would make 18B
correct predictions – no edge – and we would get it wrong only 30k
times. The accuracy of the “always negative” predictor in Figure 22.3
is ∼ 85%: not bad!

Edge Prediction Correct?

Figure 22.3: Estimating the
5 1, 3 0 1
performance of the “always
4 6 1, 4 0 1
negative” predictor on our test
1, 5 0 1
set.
8 1, 6 0 0
3 7 2, 4 0 1
2, 5 0 0
2 1 2, 6 0 1
... ...
(a) Test
(b) Scores

However that’s... kind of not the point? We’re in this business be-
cause we want to predict new links. Returning a negative prediction
for all possible cases is not helpful. The usual fix for this problem 3
Ryan N Lichtenwalter, Jake T Lussier,
is building your test set in a balanced way3 , 4 . Rather than asking and Nitesh V Chawla. New perspectives
about all possible new edges, you create a smaller test set. Half of the and methods in link prediction. In
Proceedings of the 16th ACM SIGKDD
edges in the test set is an actual new edge, and then you sample an international conference on Knowledge
equal number of non-edges. This would make our Internet test set discovery and data mining, pages 243–252,
2010
containing 60k edges, not 18B. 4
Ryan Lichtnwalter and Nitesh V
Chawla. Link prediction: fair and
effective evaluation. In 2012 IEEE/ACM
22.2 Evaluating International Conference on Advances in
Social Networks Analysis and Mining,
pages 376–383. IEEE, 2012
Let’s assume that we have competently built our training and test set.
We made our model learn on the former. We now have two things:
prediction – the result of the model – and reality – the test set. We
want to know how much these two sets overlap.
There are four possible cases:

• True Positives (TP): you predict a link that really appeared;

• False Positives (FP): you predict a link that didn’t appear;

• True Negatives (TN): you correctly didn’t predict a link that, in

fact, didn’t appear;

• False Negatives (FN): you didn’t predict a link that appeared.

310 the atlas for the aspiring network scientist

These are simple counts on which we can build several quality

measures. Two basic combinations of these counts are the True
Positive Rate (TPR) and False Positive Rate (FPR). TPR – also known
as sensitivity or recall – is the ratio between true positives and all
positives: TPR = TP/( TP + FN ). It tells you how many times you got
it right over the maximum possible number of times you could. Or,
what’s the share of correct results you found.
FPR is defined similarly: FPR = FP/( FP + TN ). This is the share
of your wrong answers over all the possible instances of a negative
prediction.

Confusion Matrix
Humans like single numbers, because seeing a number going up
tingles our pleasure centers (wait, what? You don’t feel inexplicable
arousal while maximizing scores? I question whether you’re in the
right line of work...). However, we should beware of what we call
“fixed threshold metrics”, i.e. everything that boils down a complex
phenomenon to a single number. Usually, to reduce everything to
a single measure you have to make a number of assumptions and
simplifications that may warp your perception of performance.

Actual
Yes No Figure 22.4: The schema of
a confusion matrix for link
prediction. From the top-left
Yes corner, clockwise: true positives,
false positives, true negatives,
false negatives.
Predicted

No

That is why one of the first thing you should look at is a confusion
matrix. A confusion matrix is simply a grid of four cells, putting 5
Stephen V Stehman. Selecting and
interpreting measures of thematic
the four counts I just introduced in a nice pattern5 . You can see an
classification accuracy. Remote sensing of
example in Figure 22.4. Confusion matrices are nice because they Environment, 62(1):77–89, 1997
don’t attempt to reduce complexity, but at the same time you see
information in an easy-to-parse pattern.
By looking at two confusion matrices you can say surprisingly
sophisticated things about two different methods. The one in Figure
22.5(a) does a better job in making sure a positive prediction really
 designing an experiment 311

Figure 22.5: Two distinct con-

fusion matrices for different
predictors.

(a) (b)

corresponds to a new link: there are very few false positives (one)
compared to the true positives (15). The one in Figure 22.5(b) mini-
mizes the number of false negatives, with the downside of having a
lot of false positives.
By combining the cells of a confusion matrix, you can easily de-
rive measures like TPR or FPR, or many others. They are simple
operations on the rows and columns.
If you didn’t balance your test set, the confusion matrix can end
up being irrelevant, as the vast majority of your observations will end
up in the true negative cell, obliterating all the rest.
Another disadvantage of the confusion matrix is that you have
to pick a threshold in your score. In other words, you predict the
appearance of a link if it obtains a score higher than the specific
threshold, otherwise you don’t. This is in itself a problematic choice,
thus it is common to show the evolution of your accuracy as you
change that threshold. For high values of the threshold you only
report high confidence predictions, which become less and less
confident as you decrease the threshold. This is the topic explored in
the rest of the chapter.

ROC Curves & AUC

The classic evaluation instrument for classification tasks is the Re-
ceiver Operating Characteristic (ROC) curve. This is a plot, with the
false positive rate on the x-axis and the true positive rate on the y- 6
James A Hanley and Barbara J McNeil.
axis (see Figure 22.6(a))6 , 7 . We sort all our predictions by their score The meaning and use of the area under
such that we look at the highest scores first. Then we keep track of a receiver operating characteristic (roc)
curve. Radiology, 143(1):29–36, 1982
the evolution of TPR and FPR. 7
Tom Fawcett. An introduction to roc
In a ROC curve, the 45 degree line corresponds to the random analysis. Pattern recognition letters, 27(8):
guess (Figure 22.6(b)). Suppose 80% of possible links did not appear 861–874, 2006
and 20% did, and there are a total of 20 new links. If we make ten
random guesses, we’ll get eight false positives and two true positives.
The two true positives represent 10% of all the positives, so TPR =
2/20 = 0.1. On the other hand, we know that there are 80 negatives.
Since we got eight false positives, FPR = 8/80 = 0.1. This shows that
FPR and TPR grow at the same rate for a random predictor.
What we want to see is that our best guesses are more likely to be
312 the atlas for the aspiring network scientist

1
True Positive Rate

( + )

TPR
Highest score 2nd highest Random guess

False Positive Rate

/( + ) FPR 1
(a) (b)
Figure 22.6: Schema of ROC
true positives, and thus contribute to the y-axis more than they do to curves.
the x-axis. Just like in the confusion matrix, there are multiple ways
for this to happen. We can be very precise at high scores, or at all
scores on average. The two classifiers in Figure 22.7 will be used in
different scenarios with different requirements.

Fewer false positives

at low scores Figure 22.7: An example of
More true ROC curves. The gray line cor-
positives at responds to random guesses.
high scores The blue and red lines corre-
spond to two different predic-
TPR

tors, with different behaviors at

Better if we want to avoid
different score levels.
false positives and don’t care
about missing many actual ones
Better if we want all actual
positives and don’t care about
getting a few false ones

FPR

ROC curves are great – you might even say that they ROC – but,
at the end of the day, you might want to know which of the two
classifiers is better on average. ROC curves can be reduced to a single
number, a fixed threshold metric. Since we just said that the higher
the line on the ROC plot the better, one could calculate the Area
Under the Curve (AUC). The more area under that curve, the better
your classifier is, because for each corresponding FPR value, your
TPR is higher – thus encompassing more area.
You don’t need to know calculus to estimate the area under the
curve, because it’s such a standard metric that any machine learning
package will output it for you. The AUC is 0.5 for the random guess:
that’s the area under the 45 degree line. An AUC of 1 – which you’ll
 designing an experiment 313

never see and, if you do, it means you did something wrong – means
a perfect classifier.
Note that ROC curves and AUCs are unaffected if you sample
your test set randomly, namely if you only test potential edges at
random from the set of all potential edges – I discussed before how
this is a common thing to do because of the unmanageable size
of the real test set. However, that is not true if you perform a non-
random sampling. This means choosing potential edges according
to a specific criterion. If your criterion is “good”, meaning that your
sampling method is correlated with the actual edge appearance
likelihood, you’re going to see a different – lower – AUC value. That
is because, if you don’t sample, the vast number of easy-to-predict
false negatives increases your classifier’s accuracy.

Precision & Recall

Another way of putting a number to evaluate the quality of the 8
David Martin Powers. Evaluation:
prediction is to look at Precision and Recall8 . Precision means that, from precision, recall and f-measure
when we predict that a link exists, it exists (even if we fail to predict to roc, informedness, markedness and
correlation. 2011
actual links). Recall means that there are very few existing links we
do not predict, even if we might have predicted many that didn’t ac-
tually exist. So Precision is true positives over all predicted positives
(including false positives): TP/( TP + FP). Recall is another name for
the True Positive Rate: TP/( TP + FN ). Figure 22.8 shows a visual
example.

Figure 22.8: A representation of

precision and recall.
Precise Complete
“When I say positive “When I say that’s all
it’s all goddamn positives” it’s all goddamn positives”

( + ) ( + )
You can do a few things with precision and recall. First, you
can transform them into fixed threshold metrics. This is done by
calculating what we call “Precision@n”, defining n as the number of
predictions we want to make. For instance, in Precision@100 we only
consider as an actual prediction the 100 pairs of nodes that have the
 314 the atlas for the aspiring network scientist

highest scores. Everything else is classified as “no link”.

You can also combine precision and recall to generate a derived
score, balancing them out. This is known as the F1-score, which is
their harmonic mean: F1 = 2( Precision × Recall )/( Precision + Recall ).
This is a single number, like AUC, capturing both types of errors:
failed predictions and failed non-predictions.

Very few Figure 22.9: A representation of

results, precision-recall curves.
but all of
them positive!

Got all the

Precision

positives,
at the price
of having
many false
ones.

Recall

A powerful way to use precision and recall is by using them as an

alternative to ROC curves. The so-called Precision-Recall curves have
the recall on the x-axis and the precision on the y-axis (see Figure
22.9). They tell you how much your precision suffers as you want
to recover more and more of the actual new edges in the network.
Recall basically measures how much of the positive set your recover.
But, as you include more and more links in that set, you’re likely to
start finding lots of false positives. That will make your recall go up,
but precision go down.
A final way to use precision for evaluating link prediction methods
9
Carlo Vittorio Cannistraci, Gregorio
is to use the prediction power9 . This is a measure that compares the
Alanis-Lobato, and Timothy Ravasi.
precision of your classifier with the one you would obtain from a From link-prediction in brain connec-
random classifier returning random links without looking at the tomes and protein interactomes to the
local-community-paradigm in complex
network topology. If we say that your precision is P and the random networks. Scientific reports, 3:1613, 2013
precision is Pr , then the prediction power PP is

P
PP = 10 log10 .
Pr
This is a decibel-like logscale: a PP = 1 implies your predictor
is ten times better than random, while PP = 2 means you are one
hundred times better than random. You can also create PP-curves by
having on the x-axis the share of links you remove from your training
 designing an experiment 315

set. By definition, the random predictor is an horizontal line at 0. The

more area your PP curve can make over the horizontal zero, the most
precise your predictor is.
In closing, I should also mention another popular measure: accu-
racy. This is simply ( TP + TN )/( TP + TN + FP + FN ): the number
of times you got it right over all the attempts. The lure of accuracy is
its straightforward intuition. However, it hides the difference between
type I and type II errors – false positives and false negatives – and
thus it should be handled with care.

22.3 Summary

1. To evaluate the quality of a link prediction you need to train your

algorithm and then test it. To do so, you need to divide the data in
mutually exclusive train and test sets.

2. If your data has temporal information you can decide a cutoff

date to divide the two sets. Otherwise you have to perform cross
validation: divide the data in ten blocks and rotate one block as
test set using the other nine as training, until you tested on all
data.

3. Since real networks are sparse, there are more non-edges than
edges. Thus a link prediction always predicting non-edge would
have high performance. That is why you should balance your test
sets, having an equal number of edges and non-edges.

4. A classical evaluation strategy is the ROC curve, recording your

true positive rate against your false positive rate. The more area
under this curve you have (AUC) the better your prediction perfor-
mance.

5. Precision is the ability of returning only true positive results at

the price of missing some. Recall is the ability of returning all
positive results, at the price of returning also lots of false positives.
You can draw precision-recall curves, again with the objective of
maximizing their AUC.

22.4 Exercises

1. Divide the network at [Link]

22/1/[Link] into train and test sets using a ten-fold cross vali-
dation scheme. Draw its confusion matrix after applying a jaccard
link prediction to it. Use 0.5 as you cutoff score: scores equal to or
higher than 0.5 are predicted to be an edge, anything lower is pre-
dicted to be a non-edge. (Hint: make heavy use of scikit-learn
316 the atlas for the aspiring network scientist

capabilities of performing KFold divisions and building confusion

matrices)

2. Draw the ROC curves on the cross validation of the network used
at the previous question, comparing the following link predictors:
preferential attachment, jaccard, Adamic-Adar, and resource alloca-
tion. Which of those has the highest AUC? (Again, scikit-learn
has helper functions for you)

3. Calculate precision, recall, and F1-score for the four link predictors
as used in the previous question. Set up as cutoff point the nineti-
eth percentile, meaning that you predict a link only for the highest
ten percent of the scores in each classifier. Which method performs
best according to these measures? (Note: when scoring with the
scikit-learn function, remember that this is a binary prediction
task)

4. Draw the precision-recall curves of the four link predictors as used

in the previous questions. Which of those has the highest AUC?
 Part VII

The Hairball
23
Bipartite Projections

Reality does not usually match expectations. Let’s consider three

examples:

1. Degree distributions;

2. Epidemics spread;

3. Communities.

Many papers have been written on how power law degree distribu-
tions are ubiquitous1 , 2 , 3 , 4 . Chances are that any and all the networks 1
Albert-László Barabási and Réka
you’ll find on your way as a network analyst do not have even a hint Albert. Emergence of scaling in random
networks. science, 286(5439):509–512,
of a power law degree distribution. In the best case scenario you are 1999
going to have shifted power laws, or exponential cutoffs – if you’re 2
Albert-László Barabási and Eric
lucky – (for a refresher on these terms, see Section 6.4). Bonabeau. Scale-free networks. Scientific
american, 288(5):60–69, 2003
My second example is epidemics spread – Figure 23.1. As we 3
Reka Albert. Scale-free networks in cell
saw in Part V, SIS/SIR models tell us exactly when the next node is biology. Journal of cell science, 118(21):
going to be activated. In practice, data about real activation times 4947–4957, 2005
4
Albert-László Barabási. Scale-free
has (a) high levels of noise, (b) many exogenous factors that have as networks: a decade and beyond. science,
much power in influencing how the infection spreads as the network 325(5939):412–413, 2009
connections have.
Third, and more famously, communities. We are not going to dive
deeply into the topic only until Part IX. But, very superficially, when

Figure 23.1: (a) Theory-driven

mechanically explained acti-
vation times, represented by
the node color (from dark to
bright). (b) Real data swamped
with noise, which only mildly
conforms to the network topol-
ogy.
(a) (b)
 bipartite projections 319

it comes to community discovery, the vast majority of papers propose

a very naive standard definition of what constitute communities
in a network: “Groups of nodes that have a very large number of
connections among them and very few to nodes outside the group”.
Many papers claim that most networks have this kind of organization
– references provided in Part IX. 99% of networks will instead look
like a blobbed mess. We have not one but three names for this useless
visualization of an (apparently) useless network structure: ridiculo- 5
Petter Holme, Mikael Huss, and
gram – a term which you can find sneaking around in some papers5 Sang Hoon Lee. Atmospheric reaction
and attributed to Marc Vidal –; spaghettigraph – a term I’m fond of systems as null-models to identify struc-
tural traces of evolution in metabolism.
due to my Italian origins; and hairball – the term I’ll use from now
PLoS One, 6(5):e19759, 2011
on in the book. See Figure 23.2 for an example.

Figure 23.2: (a) Well-separated

groups internally densely con-
nected. (b) The ubiquitous and
mighty hairball.

(a) (b)
There are a few ways in which hairballs arise, which are the focus
of this book part. First, many networks are not observed directly:
they are inferred. If the edge inference process you’re applying does
not fit your data, it will generate edges it shouldn’t. Second, even
if you observe the network directly, your observation is subject to
noise, connections that do not reflect real interactions but appear due
to some random fluctuations. Finally, you might have the opposite
problem: you’re looking at an incomplete sample, and thus missing
crucial information.

Reality Inference You Reality Noise You Reality Data Collection You

(a) (b) (c)

Figure 23.3: The typical breed-
In the chapters of this book part, we tackle each one of these
ing grounds for hairballs: (a)
problems to see some examples in which you can avoid giving birth
Indirect observation, (b) Noise
to yet another hairball. Chapter 24 deals with network backboning:
in the data, (c) Incomplete
how to clear out noise from your edge observations. Chapter 25
samples.
320 the atlas for the aspiring network scientist

focuses on the problem of network sampling: if you have a huge

network in front of you, how do you extract a part of it so that your
sample is representative?
Here, we start by tackling the first problem: how to deal with
indirectly observed networks. Most of the times, you want to connect
things because they are somehow similar, or they do similar things,
or they relate to similar things. For instance, you want to connect
users because they watch the same movies on Netflix. The most
natural way to represent these cases is with bipartite networks (see
Section 4.1): in my example, a network connecting each user to the
movie they watched. However, you don’t want a bipartite network,
you want a normal, down-to-earth, honest-to-god unipartite network.
What can you do in this case?
Project! Bipartite projection means that you have a bipartite net-
work with nodes of type V1 and V2 , and you want to create a uni-
partite network with only nodes of type V1 (or V2 ). In my Netflix
example, all you observe is people watching movies. As I said before,
this is a bipartite network: nodes of type V1 are people, nodes of type
V2 are movies, and edges go from a person to a movie if the person
watched the movie. However, the holy grail is to know which movies
are similar, to make recommendations to similar users.
In the following sections we explore the different ways in which
one can project a bipartite network. They all boil down to the same
strategy: we use a different criterion to give the projected edges
a weight, we establish a threshold, and drop the edges below this
minimum acceptable weight.
6
Note that, hereafter, I ignore the fact
that in Netflix you could also rate the
movie, i.e. that the bipartite network
23.1 Simple Weights is weighted. In my example, I treat the
bipartite network as unweighted.
Let’s stick with our Netflix example6 . Naively, you might think that 7
Mark EJ Newman. Scientific collabora-
you can connect movies because the same people watched them7 – as tion networks. i. network construction
and fundamental results. Physical review
in Figure 23.4. The problem is that – as we saw – degree distributions E, 64(1):016131, 2001b

2
1 6
3
5 7
4

5
3 8
Figure 23.4: An example of
6
naive bipartite projection,
7 4 2 where we connect nodes of one
8 “Connecting movies type if they have a common
because the same
Users Movies users watch them” neighbor.
 bipartite projections 321

are broad. This means that there are going to be some users in your
bipartite user-movie network with a very high degree. These are
power users, people who watched everything. They are a problem:
under the rule we just gave to project the bipartite networks, you’ll
end up with all movies connected to each other. A hairball. The key
lies in recognizing that not all edges have the same importance. Two
movies that are watched by three common users are more related to
each other than two movies that only have one common spectator.

1
Figure 23.5: An example of Sim-
2 1 6
ple Weight bipartite projection,
3
5 7 where we connect nodes of one
4 2 2 type with the number of their
5
2
common neighbors.
3 2 8
6
3
7 4 2
8
Wu,v = |Nu ∩ Nv|

The easiest way to take this information into account is to perform

simple weighting. For each pair of nodes you identify the number
of common neighbors they have, and that’s the weight of the edge
– see Figure 23.5. In practice, you don’t simply require that movies
are connected if there is at least one person who has watched both of
them. You connect movies with a weighted link, and the weight is the
number of people who watched them both: wu,v = | Nu ∩ Nv |. This
weighting scheme is similar to Common Neighbors in link prediction
(Section 20.2), and of course you can do a Jaccard correction by
normalizing it with the size of the union of the neighbor sets: wu,v =
| Nu ∩ Nv |/| Nu ∪ Nv |.
If you like to think in terms of matrices (Chapter 5), this is equiva-
lent to multiplying the bipartite adjacency matrix with its transpose.
Of course, you need to pay attention to the dimension onto which
you’re projecting. If A is a |V1 | × |V2 | matrix, then AA T is a |V1 | × |V1 |
matrix, while A T A is a |V2 | × |V2 | one. When multiplying binary
matrices, the result in cell Auv is the number of common entries set to
one between the uth and the vth rows, which is exactly the number
of common neighbors between nodes u and v. The diagonal will tell
you the degree of the node, which you can simply set to zero.
8
Fabio Saracco, Mika J Straka, Riccardo
This approach can be integrated with a second step8 . In this Di Clemente, Andrea Gabrielli, Guido
second step, one wants to evaluate the statistical significance of the Caldarelli, and Tiziano Squartini.
edge weights you obtained by counting the number of common Inferring monopartite projections of
bipartite networks: an entropy-based
neighbors. This is sort of the same thing as first projecting and then approach. New Journal of Physics, 19(5):
performing network backboning – a task we’ll see in Chapter 24. 053022, 2017
322 the atlas for the aspiring network scientist

The main difference is that this backboning is specially defined to

clean up the result of bipartite projections. One can define a series
of null bipartite network models, either via exponential random
graphs (Section 16.2) or configuration model (Section 15.1). These
null models will give birth to a bunch of null projections, which
will give an expected weight for all possible edges in the unipartite
network. Then, you can keep in your projection only those links
significantly exceeding random expectation.

23.2 Vectorized Projection

There are many criticisms of the simple counts as a weighting ap- 9

Menghui Li, Ying Fan, Jiawei Chen,
proach. Here we see the one called saturation problem9 . Another Liang Gao, Zengru Di, and Jinshan
issue is the bandwidth problem, which I explain in Section 23.3, Wu. Weighted networks of scientific
communication: the measurement and
along with the projection methods designed to fix it. topological role of weight. Physica A:
Some authors noticed that the simple count scheme has what they Statistical Mechanics and its Applications,
350(2-4):643–656, 2005
call a “saturation” problem. As an illustration, consider the following
example: suppose you are an author and you collaborated with
another scientist on a new paper. The contribution of that new paper
to your similarity is not linear. If in your previous history you only
had a single other paper with this person, then the new paper is your
second collaboration. This is a strong contributor: it represents 50%
of your entire scientific output. If, instead, this was your hundredth
collaboration, this new paper only adds little to your connection
strength. Giving the same weight in these two different scenarios is
not a good proxy to estimate the similarity in the original network.
We can exploit edge weights to solve the saturation problem. Edge
weights are something that simple counting cannot handle easily,
and if you try to handle them by doing a weighted simple counting,
you probably end up doing something similar to what I present in
this section anyway. In this scenario, you don’t want to count each
common V2 neighbor equally. You need your adjacency matrix to
contain non-zero values different than one. For simplicity, I’m going
to make the following examples with a binary matrix anyway, also to
show that these techniques can handle this simpler scenario as well.
Our sophisticated needs imply that we need to change the way
we look at the problem. As the title of this section suggests, we
are considering nodes as vectors. Specifically, consider the binary
adjacency matrix of our bipartite network. Each row is a node of type
V1 . Each entry tells us whether it is connected to a node of type V2 .
So we can see a node as a vector of zeroes and ones.
Once we do – as Figure 23.6 shows –, we discover that we can
apply a large number of distance metrics between two numerical
vectors. If these numerical vectors represent two V1 nodes, then the
 bipartite projections 323

Simple Weight = 2 Figure 23.6: An example of

vectorized bipartite projection,
CosineSim = 0.66
where we connect nodes of one
Pearson + 1 = 1.52 type with the inverse of some
vector distance measure of their
1 / (Euclidean + 1) = 0.41 rows in the bipartite adjacency
matrix.

distance between them must be – inversely – related to how similar

they are. Popular choices to establish the strength of the connection
between these two nodes are the Euclidean distance, cosine similarity
and Pearson correlation – but the list could be much longer and
you can get inspiration from the set of vector distance measures 10
[Link]
implemented in any statistical library10 . reference/[Link]
One nice thing about many of these measures, besides properly
handling edge weights, is that they handle also common zeroes. In
simple weighting, you only count common neighbors. However,
two nodes might be similar also based on the neighbors they don’t
connect to. This is elegantly handled in the Pearson correlation, for
instance. Such indirect effects are not always good: for instance they 11
Baruch Barzel and Albert-László
are a problem when performing link prediction11 . Barabási. Network link prediction by
You need to be aware of a few problems with this approach. First, global silencing of indirect correlations.
Nature biotechnology, 31(8):720–725,
it’s not always immediately obvious how to translate a distance
2013a
into a similarity while preserving its properties. You cannot always
take the inverse, or multiply by minus one, or doing one minus the
distance. Each of these solutions might work with some measures,
but catastrophically fail with others.
The second issue is more subtle. None of these measures were
really developed with network data in mind. So they might not work
because they don’t take into account what the edge creation process
of the bipartite network looks like. They are not going to necessarily
solve the issues simple weighting has, because they’re still prone to
fall into the trap of large hubs and very skewed degree distributions.

23.3 Hyperbolic Weights

The second problem is similar to the saturation one, but cannot

be solved by looking at edge weights. If you’re in a CERN paper,
you coauthor with hundreds of people, but you don’t really know
all of them. In practice, we’re acknowledging that “bandwidth” is
finite: having too many coauthors implies having only a superficial
relationship with all of them. This bandwidth argument is not new,
324 the atlas for the aspiring network scientist

we saw a similar one when we introduced link prediction methods

like Adamic-Adar in Section 20.3 and Resource Allocation in Section
20.4.

1
Figure 23.7: An example of
2 1 6
Hyperbolic Weight bipartite
3
5 .46 7 projection, where each common
4 .46 neighbor z contributes k− 1
z to
5
.46
the sum of the edge weight.
3 .46 8
6
.79
7 4 2
8 1
Wu,v = ΣzϵN ∩N
u v kz

In hyperbolic weight we recognize that hubs contribute less to the 12

Mark EJ Newman. Scientific collab-
connection weight than non-hubs12 . Such a weight scheme is similar oration networks. ii. shortest paths,
to the link prediction strategies I just mentioned: each common weighted networks, and centrality.
Physical review E, 64(1):016132, 2001c
neighbor z contributes k− 1
z rather than 1 to the weight of the edge
1
connecting the two nodes: wu,v = ∑ . The final result
z∈ Nu ∩ Nv k z − 1
in this example is similar to simple weight – see Figure 23.7 –, but it
exaggerates the differences, so that thresholding becomes easier.
Note that the minus one in the denominator – which we do be-
cause u never checks its similarity with itself – means that we’re
effectively ignoring all papers with only one author. And, if an au-
thor only wrote with herself, she won’t appear in the network. This
makes sense at some level – how can you connect with anybody else
if you never collaborate? – but it also implies that there is going to be
no information in the diagonal of the resulting adjacency matrix.
Again, this projection is simple to implement as a matrix operation.
Rather than multiplying the bipartite adjacency matrix with its
transpose, you multiply the degree normalized stochastic with its
transpose. If you do so, rather than counting the common ones, you
sum up all the 1/k z entries. Again, pay attention to the dimension
over which you project, because normalizing by row sum or by
column sum will change the result.

23.4 Resource Allocation

In resource allocation we do the same thing as hyperbolic weight,

but considering two steps instead of one. Rather than only looking
13
Tao Zhou, Jie Ren, Matúš Medo, and
at the degree of the common neighbor, we also look at the degree
Yi-Cheng Zhang. Bipartite network pro-
of the originating node13 . In the paper-writing example, not only jection and personal recommendation.
it is unlikely to be strongly associated with a co-author in a paper Physical Review E, 76(4):046115, 2007
 bipartite projections 325

with hundreds of authors, it is also difficult to give attention to

a particular co-author if you have many papers with many other
people. So each common neighbor z that node u has with node
v contributes not k− 1
z as in hyperbolic weights, but ( k u k z )
−1 – see

Figure 23.8. The weight is then:

1
wu,v = ∑ ku kz
.
z∈ Nu ∩ Nv

This generates the unipartite weight matrix W.

1
1/2 Figure 23.8: An example of
1/3 2
Resource Allocation bipartite
1/8
1/2 3
1 0.229 projection, where each com-
4 mon neighbor z contributes
5 (k u k z )−1 to the sum of the
6 0.153 2 edge weight. When connecting
node 1 to node 2, from node
7
1’s perspective the edge weight
8 1
Wu,v = ΣzϵN ∩N u v kukz is (1/2 ∗ 1/3) + (1/2 ∗ 1/8),
because the two common
This strategy also works for weighted bipartite networks. If B is neighbors have degree of
your weighted bipartite adjacency matrix, the entries of W are: 3 and 8, respectively, and
node 1 has degree of two.
Buv
wu,v = ∑ ku kz
. However, from node 2’s per-
z∈ Nu ∩ Nv spective, the edge weight is
In practice, you replace the 1 in the numerator with the edge (1/3 ∗ 1/3) + (1/3 ∗ 1/8),
weights connecting z to v and u. Moreover, we can also have node because node 2 has three neigh-
weights, noticing that some nodes might have more resources than bors.
others. Suppose that you have a function f giving each node in the
network a resource weight. After you perform the resource allocation
projection, each node will have a new amount of resources f 0 = W f .
Note that, in this case, W is not symmetric: in the scenario with a
single common neighbor z, u’s score for v would be (k u k z )−1 , while
v’s score would be (k v k z )−1 . If k u 6= k v , then the scores are different.
In many cases, this provides a better representation of the network
than one ignoring asymmetries. You might be the most similar
author to me because I always collaborated with you, but if you also
contributed to many other papers with other people, then I might not
be the author most similar to you.
1 1 1
W has a well-defined diagonal: wu,u = ∑ = ∑ .
z∈ Nu k u k z | Nu | z∈ Nu k z
In fact, this diagonal is the maximum possible similarity value of the
row: only a node v with the very same neighbors and nothing else
can have a weight wu,v = wu,u .
326 the atlas for the aspiring network scientist

In some other cases you might consider having a directed pro-

jection an inconvenience, because you really want an undirected
network as a result. You can make the result of resource allocation
symmetric by always choosing the minimum or maximum between
wu,v and wv,u , or simply their average: (wu,v + wv,u )/2. Also self-loops
can be annoying sometimes. If you have no use for them, you can
manually set W’s diagonal to zero.
The resource allocation as presented so far is only one of the many
possible variants following the same idea. The one I explained so
far is known as ProbS and uses k u , the degree of the origin of the
two-step random walk, as the normalizing factor. A variant known 14
Tao Zhou, Zoltán Kuscsik, Jian-Guo
as HeatS14 uses instead k v , the destination of the random walk. Thus, Liu, Matúš Medo, Joseph Rushton
1 Wakeling, and Yi-Cheng Zhang. Solv-
the weight of the u, v connection is now wu,v = ∑ .
z∈ Nu ∩ Nv v k z
k ing the apparent diversity-accuracy
Surprising absolutely no one, some authors decided to combine dilemma of recommender systems.
Proceedings of the National Academy of
ProbS and HeatS in a single Hybrid framework15 . The combination Sciences, 107(10):4511–4515, 2010
1 15
Linyuan Lü and Weiping Liu. Informa-
is exactly what you would expect: wu,v = ∑ ( 1 −λ)
. This
z∈ Nu ∩ Nv k λ tion filtering via preferential diffusion.
u kv kz
Physical Review E, 83(6):066119, 2011
introduces a parameter in the equation: λ. This should be a number
between zero and one, determining how much importance the degree
of the origin has compared to the degree of the destination. For λ = 0
you have HeatS, for λ = 1 you have ProbS, and for λ = 1/2 you have
the middle point between HeatS and ProbS.
In matrix terms, ProbS is the same as the hyperbolic projection
(Section 23.3), but now you normalize differently. In hyperbolic,
you multiply the stochastic adjacency matrix A with its transpose.
In ProbS you multiply the stochastic with a stochastic version of
the transpose. Meaning, in hyperbolic first you normalize then you
transpose, in ProbS first you transpose and then you normalize.
Finally, HeatS is the transpose of ProbS.

23.5 Random Walks

In Random Walks, we take the resource allocation to the extreme.

Rather than looking at 2-step walks, we look at infinite length ran-
dom walks. Which means that the strength between u and v is the
probability of visiting v starting from u. If we have infinite random
walks, this means that we can use the stationary distribution to
estimate the edge weight: wu,v = πv Au,v , where A is a transition
16
Muhammed A Yildirim and Michele
probability matrix (recording the probability of the path u → z → v, Coscia. Using random walks to generate
for any z)16 . Note that A here is different than a simple binary ad- associations between objects. PloS one, 9
(8):e104813, 2014
jacency matrix, as it encodes the probabilities of all random walks
of length two. This means that its interpretation is slightly different
than what I originally presented for π in Section 11.4.
 bipartite projections 327

1
Figure 23.9: An example of Ran-
2
dom Walks bipartite projection,
3
1 0.049 where the connection strength
4 between u and v is dependent
5 on the stationary distribution
6 0.022 2 π, telling us the probability
of ending in v after a random
7
walk.
8 Wu,v = πAu,v

In matrix terms, you take the result ProbS’ multiplication, which

is a square |V1 | × |V1 | matrix, and you multiply it with its stationary
distribution.
As in the resource allocation case, this means that the measure
is not symmetric, and the differences between nodes now are more
extreme than before: the 1 → 2 edge weight is now more than twice
as 2 → 1, while in resource allocation it was just about 50% higher.
See Figure 23.9 for an example. Another parallelism between these
two approaches is the presence of a well-defined diagonal, which you
can use in case you’re not afraid of self-loops (I am).

23.6 Comparison in a Practical Scenario

I showed you how these different methods approach the projection

process and the different results they obtain in a toy example. Do
these differences in simple scenarios translate to big practical dif-
ferences in real-world cases? To answer this question, let’s just take
a superficial look at the projections I get using a bipartite network
extracted from Twitter. In the network, the nodes of type V1 are web-
sites, and nodes of type V2 are Twitter users. I connect a Twitter user
to a website if the user included the URL of the website in one of her
tweets.
I project the network so that I have a unipartite version with only
V1 nodes: websites are connected if the same users tweet about them.
This is a sort of website similarity index. Now let’s see how different
the space of edge weights looks like if we use different approaches.
This is what Figure 23.10 is all about.
The figure shows that the space of the edge weights looks pretty
different according to different projection methods. For instance, the
simple projection (Figure 23.10(a)) shows a power law distribution of
edge weights, with more than four million edges with weight equal
to one and one edge with weight equal to 2, 252, while the average
328 the atlas for the aspiring network scientist

107 106 106 106

6 5 5 5
10 10 10 10
105 10
4
10
4
10
4
# Edges

# Edges

# Edges

# Edges
104
103 103 103
103
102 102 102 102
101 101 101 101
100 100 100 100
100 101 102 103 10-3 10-2 10-1 100 10-2 10-1 100 100
Edge Weight Edge Weight Edge Weight Edge Weight

(a) Simple (b) Jaccard (c) Cosine (d) Pearson

106 106 106 106
105 105 105 105
4 4 4 4
10 10 10 10
# Edges

# Edges

# Edges

# Edges
103 103 103 103
102 102 102 102
101 101 101 101
100 100 100 100
10-6 10-5 10-4 10-3 10-2 10-1 100 10-6 10-5 10-4 10-3 10-2 10-1 10-8 10-6 10-4 10-2 100 10-11 10-9 10-7 10-5 10-3
Edge Weight Edge Weight Edge Weight Edge Weight

(e) Hyperbolic (f) ProbS (g) Hybrid (λ = 0.5) (h) Random Walks
Figure 23.10: The distributions
weight is 2.14. This is very much not the case for other projection of edges weights in the pro-
strategies such as Jaccard (Figure 23.10(b)), where there is no trace jected Twitter network for eight
of a power law. And, in many cases such as cosine and Pearson different projection methods.
(Figures 23.10(c-d)), the highest edge weight is actually the most The plot report the number
common value, rather than being an outlier such as in the hyperbolic of edges (y axis) with a given
projection (Figure 23.10(e)). weight (x axis).
Is the difference exclusively in the shape of the distribution, or
do these approaches disagree on the weights of specific edges? To
answer this question we have to look at a scattergram comparing the
edge weights for two different projection strategies. This is what I do
in Figure 23.11.
I picked three cases to show the full width of possibilities. In
Figure 23.11(a), I compare the cosine projection against the Jaccard
one. This is the pair of projections that, in this dataset, agree the most.
Their correlation is > 0.94. Looking at the figure, it is easy to see that
there isn’t much difference. You can pick either method and you’re
going to have comparable weights. The opposite case compares two
method that are anti-correlated the most. This would be HeatS and
the random walks approach, in Figure 23.11(b). They correlation in a
log-log space is a staggering −0.7. From the figure you can probably
spot a few patterns, but the lesson learned is that the two methods
build fundamentally different projections.
Ok, but these are extreme cases. How does the average case looks
like? To get an idea, I chose a particular pair of measures: HeatS and
ProbS (Figure 23.11(c)). You might expect the two to be more similar
than the average method: after all, one is the transpose of the other.
You’d be very wrong. In this dataset, HeatS and ProbS are actually
anti correlated, at −0.34 in the log-log space. HeatS and ProbS would
be positively correlated if the nodes of type V1 with similar degrees
 bipartite projections 329

(a) (b) (c)

Figure 23.11: The comparison

connect to the same nodes of type V2 . But that is not the case in this between the edge weights ac-
specific Twitter dataset. Here, it is not true that the people sharing cording to different network
lots of URLs share the same URLs. projections. Each point is an
At this point, you might be asking yourself how do you choose edge. The x-y coordinates
the projection method that is most suitable for your application. encode its weight in the two
The general guideline is to study what each method does and see different projections. The color
if it aligns with your expected edge generation process. However, I encodes how many edges share
feel it’s a bit too early to ask this question. That is because network she same x-y score. (a) Cosine
projection is rarely the only thing you’re going to do. Almost all vs Jaccard; (b) HeatS vs YCN;
these methods return the same set of non-zero weighted edges. They (c) HeatS vs ProbS.
also return extremely dense projections, as a single common node is
enough to create an edge in the projection.
In fact, the Twitter data I just used has ∼ 15k users and ∼ 14k
domains, with ∼ 175k edges connecting them. The undirected projec-
tions return ∼ 5.3M edges, meaning a density of 2 × 5.3M/14k2 ∼ 5%,
or an average degree of ∼ 713. This is usually way too much for an
intelligible network. That is why, if we want to avoid hairballs and re-
lated problems, these techniques – while necessary – are not usually
sufficient. The process to get rid of hairballs has two steps: first one
performs the bipartite projection, and then she applies a threshold to
throw away low-weighted edges. The next chapter expands on how
to perform this second step properly.

23.7 Summary

1. Most network analysis algorithms work with unipartite networks,

but many phenomena have a natural bipartite representation. To
transform a bipartite network into a unipartite network you need
to perform the task of network projection.

2. In network projection you pick one of the two node types and you
connect the nodes of that type if they have common neighbors
of the other type. Normally you’d count the number of common
neighbors they have (simple weighted) and then evaluate their
330 the atlas for the aspiring network scientist

statistical significance.

3. Real world bipartite networks have broadly distributed degrees

which might make your projection close to a fully connected
clique. Then you need a smart weighting scheme to aid you in
removing weak connections.

4. You could use standard vector distances (cosine, euclidean, cor-

relation) but we have specialized network-aware techniques. For
instance, considering nodes as allocating resources to their neigh-
bors, inversely proportional to the number of neighbors they have
(hyperbolic).

5. In resource allocation, you also have nodes sending resources, but

you take two steps instead of one: you’re not discounting only for
the degree of nodes of type one, but also for the degree of nodes of
type two.

6. Finally, you can also do resource allocation with infinite length

random walks by looking at the stationary distribution. The
resulting edge weights from all these techniques can create very
different network topologies.

23.8 Exercises

1. Perform a network projection of the bipartite network at http:

//[Link]/exercises/23/1/[Link] using simple
weights. The unipartite projection should only contain nodes of
type 1 (|V1 | = 248). How dense is the projection?

2. Perform a network projection of the previously used bipartite

network using cosine and Pearson weights. What is the Pearson
correlation of these weights compared with the ones from the
previous question?

3. Perform a network projection of the previously used bipartite

network using hyperbolic weights. Draw a scatter plot comparing
hyperbolic and simple weights.
24
Network Backboning

Network backboning is the problem of taking a network that is

too dense and removing the connections that are likely to be not
significant – or “strong enough”. If you ever found yourself in a
situation thinking “there are too many edges in this network, I’m
going to filter some out”, then you performed network backboning.
Even if it is rarely explicitly labeled like that, network backboning is
one of the most common tasks performed in network analysis.
There are many reasons why you would want to backbone your
network. First, this is a book part about hairballs. If your network
is a hairball, meaning that the tangle of connections is too dense
to reach any meaningful conclusion, you might want to sparsify 1
Daniel A Spielman and Shang-Hua
your network. Graph sparsification1 , 2 – sometimes called “pruning” Teng. Nearly-linear time algorithms
in combinatorics3 and neural networks4 – could be an alternative for graph partitioning, graph sparsifi-
cation, and solving linear systems. In
name for backboning, but it is often used in a more narrow context, Proceedings of the thirty-sixth annual ACM
namely the second application field of backboning: your network symposium on Theory of computing, pages
simply has too many connections to be computationally tractable 81–90, 2004
2
Venu Satuluri, Srinivasan
and so you need to filter out the ones that are unlikely to affect Parthasarathy, and Yiye Ruan. Lo-
your computation. Finally, a third scenario might be the presence of cal graph sparsification for scalable
noise: you don’t know whether the edges you’re observing are real clustering. In Proceedings of the 2011
ACM SIGMOD International Conference
connections and you need a statistical test to determine that. on Management of data, pages 721–732,
When wearing its “graph sparsification” hat, network backboning 2011
3
Daniel Damir Harabor, Alban Grastien,
could be confused with graph summarization: the task of taking a et al. Online graph pruning for pathfind-
large complex network and reducing its size so that we can describe ing on grid maps. In AAAI, pages
it better. However, there is a crucial difference between the two tasks: 1114–1119, 2011
4
Zhuang Liu, Mingjie Sun, Tinghui
one of the central objectives of graph summarization is to reduce Zhou, Gao Huang, and Trevor Darrell.
the number of nodes of the network as much as possible, often even Rethinking the value of network
merging them into “meta nodes”. This is exactly the opposite of what pruning. arXiv preprint arXiv:1810.05270,
2018c
backboning wants to do: in network backboning you do not merge
nodes and you want to keep as many as possible in your network.
The reason is that you want to let the strong connections emerge,
but you want to preserve all the entities in your data. If you remove
nodes from your network, you cannot describe them directly any
332 the atlas for the aspiring network scientist

more in your analysis. In a nutshell, network backboning wants to

allow you to perform node- and global-level analyses, while graph
summarization only focuses on empowering meso-level analysis (Part
VIII) where you lose sight of the single individual nodes. For this
reason, graph summarization has its own chapter (Chapter 38) in a
totally different part of this book.
There are several network backboning methods which aim to
tackle this problem. I’ll look at a few techniques divided in two
macro categories: structural approaches (naive thresholding, Doubly
Stochastic, High Salience Skeleton, convex network reduction), and
statistical ones (Disparity Filter, Noise Corrected). These are, to the
best of my knowledge, the most used and are the ones that I’m the
most familiar with. There are other backboning methods, many of
which are based on the same “urn extraction” procedure we’ll see in
depth when we talk about the noise-corrected backbone: for bipartite 5
Michele Tumminello, Salvatore
networks5 , using Polya urns6 , and more. Another common approach Micciche, Fabrizio Lillo, Jyrki Piilo,
is to create a null version of the observed network and testing the and Rosario N Mantegna. Statistically
validated networks in bipartite complex
edge weights against such expectation7 , in line with the disparity
systems. PloS one, 6(3):e17994, 2011
filter we’ll see. 6
Riccardo Marcaccioli and Giacomo
Note that finding the maximum spanning tree, planar maximally Livan. A pólya urn approach to
information filtering in complex
filtered graphs, and the triangulated maximally filtered graphs could
networks. Nature Communications, 10(1):
also be considered a way to perform structural network backboning, 745, 2019
and they were covered in Section 10.4. 7
Filippo Radicchi, José J Ramasco, and
Santo Fortunato. Information filtering
The vast majority of methods in this area of research work on
in complex weighted networks. Physical
weighted networks. You could, in principle, apply some of them to Review E, 83(4):046101, 2011
unweighted networks as well, but you might fall off the use cases
that were taken in consideration when developing such algorithms.
I conclude this section by talking about network measurement
error, which is something we network scientists talk surprisingly little
about, but it is intimately intertwined with the way noise-corrected
backboning works.

24.1 Naive

The reason not many network researchers mention this problem is

because they usually apply a limited set of naive strategies and do
not recognize it as a problem in itself. In fact, there is an easy naive
solution that most researchers apply without a second thought. If
we have a weighted network and we want to keep the “strongest
connections”, we sort them in decreasing order of intensity. We
decide a threshold, a minimum strength we accept in the network.
Everything not meeting the threshold is discarded.
Figure 24.1 provides a vignette of this procedure. There are two
problems with the naive strategy.
 network backboning 333

Discarded
Figure 24.1: A vignette of the
naive thresholding procedure.
Each red bar is an edge in
the network. The bar’s width
is proportional to the edge’s
weight. Here, I sort all edges
Threshold in decreasing weight order. I
then establish a threshold and
Broad Weight Distributions discard everything to its right.

The first problem is that, in real world networks, edge weights dis-
tribute broadly in a fat-tail highly skewed fashion, much like the
degree (Section 6.3). Let’s take a quick look again at the edge weight
distribution we got using the simple projection in the previous chap-
ter for our Twitter network. I show the distribution again in Figure
24.2.

7
10
Figure 24.2: The distributions of
106
edges weights in the projected
105 Twitter network using the sim-
# Edges

104 ple projection strategy. The plot

3
10 reports the number of edges
2
10 (y axis) with a given weight (x
101 axis).
0
10 0 1 2 3
10 10 10 10
Edge Weight

In this network, 82% of the edges have weight equal to one. The
smallest possible hard threshold would remove 82% of the network,
without allowing for any nuance. Moreover, since we have a fat
tailed edge weight distribution, it is hard to motivate the choice of a
threshold. Such a highly skewed distribution lacks of a well-defined
average value and has undefined variance. You cannot motivate your
threshold choice by saying that it is “x standard deviations from the
average” or anything resembling this formulation.

Local Edge Weight Correlations

The second problem is that edge weights are usually correlated.
Nodes that connect strongly tend to connect strongly with everybody.
In our sample Twitter network, let’s consider a user u who only
had shared a single URL. If u connects to any v, there can be only
one possible edge weight in the simple projection: one. All edges
334 the atlas for the aspiring network scientist

around u will have weight equal to one. On the other hand, if u had
shared thousands of URLs, it will likely connect to another user with
similar sharing patterns, because statistically speaking they have high
odds of sharing at least few of the same URLs, even if it happens by
chance. Thus many edges around u will have high weights.

9
Figure 24.3: The average weight
Avg Neighbor Weight

8
7 of edges sharing a node with
6 a focus edge (y axis) against
5 the weight of the focus edge (x
4
axis). Thin lines show the stan-
3
2 dard deviation. One percent
1 sample of the Twitter network.
0
0 1 2
10 10 10
Edge Weight

This is what I mean when I say that the weight of an edge is

correlated with the weights of the edges of the nodes it connects.
Figure 24.3 shows how this correlation looks like in the Twitter
network. The higher an edge weight, the higher on average the
weights of edges sharing a node with it. Here, the correlation is
∼ 0.69. The figure has the edge weights in log scale, since they are
broadly distributed. The correlation of the average neighbor weight
against the logarithm of the edge weight is ∼ 0.84.
This means that there are areas of the network with high edge
weights and areas with low weights. If we impose the same threshold
everywhere, some nodes will retain all their connections and others
will lose all of theirs, without making the structure any clearer. Fig-
ure 24.4 provides a vignette of this issue. In the figure, we completely
destroy the topological information in the rightmost clique, while at
the same time being unable to sparsify the leftmost clique.

Figure 24.4: Establishing a hard

threshold in a network with
correlated edge weights. (a) I
represent the edge weight with
(a) (b) the width of the line. (b) I elimi-
An alternative “naive” strategy you could apply is to simply pick nate all the edges with a weight
the top n strongest connections for each node. This would not be lower than a given threshold,
affected by the issues I mentioned. However, by applying it you’re equal for all edges.
effectively determining the minimum degree of the network to be
n. This is a heinous crime against the God of power law degree
 network backboning 335

distributions and, if you commit it, you will be tormented by scale

free demons in network hell for all eternity.

24.2 Doubly Stochastic

The next approach we look at is the doubly stochastic strategy. Re-

member what a stochastic matrix is: it is the adjacency matrix nor-
malized such that the sum of the columns is 1 (Section 5.3). A doubly
stochastic matrix is a matrix in which the sums of both rows and
columns are equal to one. You can transform an adjacency matrix
into its corresponding doubly stochastic by alternatively normalizing
rows and columns until they both sum to 1.

Figure 24.5: An example of

Doubly Stochastic network
backboning. (a) The adjacency
matrix has areas of the network
with different edge weight
scales. (b) Its doubly stochas-
(a) (b)
tic counterpart has no such
correlations.
After you perform such normalization, the scale of all edges is
the same, and you break local correlations – as we show in Figure
24.5. You can now threshold the edges without fearing for the issues 8
Paul B Slater. A two-stage algorithm for
we mentioned before8 . The original paper proposing this technique extracting the multiscale backbone of
has specific guidelines on how to perform this thresholding. You complex weighted networks. Proceedings
of the National Academy of Sciences, 106
should pick the threshold that allows your graph to be a single (26):E66–E66, 2009
connected component. However, in many cases you might have
different analytic needs. Thus you can specify your own threshold.
The downside of this approach is that not all matrices can be
transformed into a doubly stochastic. Only strictly positive matrices 9
Richard Sinkhorn. A relationship
can9 , 10 , meaning that the matrix cannot contain zero elements. Since between arbitrary positive matrices and
real world networks are sparse, they actually contain lots of zeros. doubly stochastic matrices. The annals
of mathematical statistics, 35(2):876–879,
So this solution cannot be always applied, although, in practice, 1964
my experience is that failure to convergence is the exception rather 10
Richard Sinkhorn and Paul Knopp.
than the rule. The easy solution of adding a small e to the matrix Concerning nonnegative matrices and
doubly stochastic matrices. Pacific
to get rid of zero entries does not always make sense. As e → 0, Journal of Mathematics, 21(2):343–348,
meaning that A + e → A, the normalization of A + e does not 1967
converge.
Note also that a doubly stochastic matrix must be square. This
is easy to see: if all rows sum to one, then the sum of all entries in
the matrix must be the number of rows. On the other hand, if all
columns sum to one, the sum of all the entries of the matrix must be
the number of columns. Thus, the number of rows and the number
336 the atlas for the aspiring network scientist

of columns are the same number. This cheeky proof means that you
cannot apply the doubly stochastic backboning to bipartite networks,
unless |V1 | = |V2 |.
Doubly stochastic matrices have other fun properties. If you
remember Section 8.1, the leading left eigenvector of a stochastic
adjacency matrix is the stationary distribution, while the leading
right eigenvector is a constant – assuming the graph is connected. In
a doubly stochastic matrix, both the left and the right eigenvectors
are equal to a constant or, in other words, the stationary distribution
of a doubly stochastic matrix is constant. This isn’t really a necessary
thing to know while doing network backboning, but I though it was
cool, so do with this information what you will.

24.3 High-Salience Skeleton

The intuition behind the high salience skeleton (HSS) is that a net-
work is a structure facilitating the exchange of information or goods.
Thus, some connections are more important than others because they
keep the network together in a single component. The main impera-
tive is to allow all nodes to reach all other nodes in the most efficient
and high-throughput way possible. Thus you need to interrogate
each node and ask them what are the most efficient paths from their
perspective. This cannot be done repurposing measures such as edge
betweenness – whose objective is also telling us how structurally
important an edge is (Section 11.2) – because these measures adopt
a “global” point of view: they are the salient connections for the
network as a whole, but they might leave some nodes poorly served.
To build an HSS we loop over the nodes and we build their short-
est path tree: a tree originating from a node, touching all other nodes
in the minimum number of hops possible and maximum amount of
edge weight possible. In practice we start exploring the graph with
a BFS and note down the total edge weight of each path. When we
reach a node that we already visited we consider the edge weights of
the two paths and the one with the highest one wins.
Note that we have the constraint of the structure originating from
a node to be a tree. Thus it cannot contain a triangle. Consider Figure
24.6 as an example. In the bottom example, we might want to save
two edges at the same time. Our origin node, the one at the top of
the network connects strongly with one node which also connects
strongly to the node on the left. However, we cannot have both
edges in the shortest path tree, as that would create a cycle. The final
salience skeleton is allowed to have triangles and cycles, because it is
the sum of all the shortest path trees.
We perform this operation for all nodes in the network and we
 network backboning 337

+ Figure 24.6: An example of

High Salience Skeleton network
backboning. The original graph
+ is used to create a shortest path
tree for each node in the net-
work. In each tree, I highlight
+ the focus node with the orange
outline. The trees are then
summed, and the result is new
obtain a set of shortest path trees. We sum them so that each edge edge weights for the original
now has a new weight: the number of shortest path trees in which graph that can be thresholded.
it appears11 . The network can now be thresholded with these new 11
Daniel Grady, Christian Thiemann,
weights. and Dirk Brockmann. Robust clas-
sification of salient links in complex
Forbidding the creation of cycles in shortest path trees causes the networks. Nature communications, 3:864,
main difference with the edge betweenness measure (Section 11.2). 2012
One could think that the edges are simply sorted according to their
contributions to all shortest paths in the network, but that is not the
case. By forcing the substructures to be trees, we are counting the
edges that are salient from each node’s local perspective, rather than
the network’s global perspective. The authors in the paper show
the subtle difference between shortest path tree counts and edge
betweenness, also showing how a hypothetical skeleton extracted
using edge betweenness performs more poorly.

180
160 Figure 24.7: A typical “horn”
140 plot for the edge weight distri-
120 bution in HSS. The plot reports
# Edges

100
how many edges (y axis) are
80
part of a given share of shortest
60
40 path trees (x axis).
20
0
0 0.2 0.4 0.6 0.8 1
HSS Score

The HSS makes a lot of sense for networks in which paths are
meaningful, like infrastructure networks. However, it requires a
lot of shortest path calculations – which makes it computationally
expensive. Moreover, the edges are either part of (almost) all trees or
of (almost) none of them. Figure 24.7 shows an example of this edge
weight distribution, showcasing the typical “horn” shape of the HSS
score attached to the original edges. You can see clearly that there
are two peaks: one at zero – the edge is in no shortest path tree –; the
338 the atlas for the aspiring network scientist

other at one – the edge is part of all shortest path trees.

This can be nice, because it means HSS can be almost parameter
free: the thresholding operation does not have many degrees of
freedom. On the other hand, when there are few edges with weights
close to one your skeleton might end up being too sparse and it is
difficult to add more edges without lowering the threshold close to
zero.

24.4 Convex Network Reduction

A subgraph of a network G is convex if it contains all shortest paths 12

Frank Harary, Juhani Nieminen,
existing in the main network G between its V 0 ⊆ V nodes12 . We can et al. Convexity in graphs. Journal of
expand this concept of convexity to apply to a full network G. To Differential Geometry, 16(2):185–190, 1981
do so, we need to introduce the concept of “induced” subgraph: an
induced subgraph is a graph formed from a subset of the vertices
of the graph and all of the edges connecting pairs of vertices in that
subset. Figure 24.8 shows an example of the inducing procedure.

2
1 Figure 24.8: An example of
2 induced graph. (a) The original
4 3
graph. I highlight in red the
5 nodes I pick for my induced
9
8 3 graph. (b) The induced graph
6
of (a), including only nodes in
11
7
10 9 red and all connections between
8 them.
12 6
(a) (b)
A network is convex if all its induced subgraph are convex. No
matter which set of nodes you pick: as long as they are part of a
single connected component, they are all going to be convex. This
might look like a weird and difficult to understand concept, but you
can grasp it with the help of elementary building blocks you already
saw in this book.
Figure 24.9 shows the two basic alternatives for a convex network.
In a tree – Figure 24.9(a) – any set of connected nodes is a convex
subgraph. There are no other edges in G you can use to make short-
cuts, because they’d create a cycle and trees cannot contain a cycle.
A clique – Figure 24.9(b) – is a convex network as well: all possible
connections are part of G, so picking any subset V 0 of nodes will
also result in a clique. Since all nodes are connected to each other
in a clique, you have all the shortest paths between them, making it
convex.
 network backboning 339

Figure 24.9: Two examples of

convex networks. (a) A tree. (b)
A clique.

(a) (b)

You can build an arbitrary convex network by stitching together

trees and cliques. In practice, it’s just stitching together cliques,
because the “tree-like” parts are nothing more than 2-cliques.
One could use the concept of convex networks to create a skeleton 13
Lovro Šubelj. Convex skeletons of
of a real world network13 . Convex networks are almost impossible in complex networks. Journal of The Royal
nature, because adding a single edge to a tree or removing a single Society Interface, 15(145):20180422, 2018
edge from a clique completely destroys convexity. However, one
could make a real world network into a convex network by finding
the minimal set of edges to remove to reduce the network into a tree
of cliques. This is a possible way of backboning your network.

24.5 Disparity Filter

In this and the following sections, we’re slightly turning the perspec-
tive on network backboning. You could consider these as a different
subclass of the problem. They all apply a general template to solve
the problem of filtering out connections, which relate to the “noise
reduction” application scenario of network backboning. Up until
now, we adopted a purely structural approach which re-weights
nodes according to some topological properties of the graph. Here,
instead, given a weighted graph, we adopt a template composed by
three main steps: (1) define a null model based on node distribution
properties; (2) compute a p-value for every edge to determine the
statistical significance of properties assigned to edges from a given
distribution; (3) filter out all edges having p-value above a chosen
significance level, i.e. keep all edges that are least likely to have
occurred due to random chance.
The disparity filter (DF) is the first example in this class of solu-
tions. It takes a node-centric approach. Each node has a different
threshold to accept or reject its own edges. This is done by modeling
an expected typical “node strength”, for instance the average of its
edge weights. Then we keep only those edges which are higher than
340 the atlas for the aspiring network scientist

1 2 3 Figure 24.10: A schematic sim-

plification of Disparity Filter
(1+2+3+4+1+2) / 6 network backboning. The node
~ 2.16 determines its customized
threshold by building an expec-
4 1 2 tation of its average connection
strength. Every edge weight
higher than this expectation in
the expected edge weight for this node, making sure that this differ- a statistically significant way is
ence is statistically significant14 . Figure 24.10 depicts a simplification kept.
of the method.
More precisely, the disparity filter defines u’s p-value for an edge 14
M Ángeles Serrano, Marián Boguná,
u, v of weight wu,v as: and Alessandro Vespignani. Extracting
the multiscale backbone of complex
 (| Nu |−1) weighted networks. Proceedings of the
national academy of sciences, 106(16):
wu,v  6483–6488, 2009
p((u, v), u) = 1 − ,

∑ wu,v0

v0 ∈ Nu

where Nu is the set of u’s neighbors (thus, | Nu | is a fancy way to

represent u’s degree). The original paper also shows how to calculate
the expected edge weight and its variance, from which you can
derive this p-value, but the procedure is a bit too convoluted to be
included here. All you need, really, is the p-value. You can easily see
that, if | Nv | 6= | Nu | and/or ∑ wu,v0 6= ∑ wv,v0 , the p-values for
v0 ∈ Nu v0 ∈ Nv
the same edge u, v will be different depending whether we focus on u
or on v.
The disparity filter doesn’t take into account that some nodes have
inherently stronger connections. For instance, consider a mobility
network, tracking commuters between cities in the United States.
Figure 24.11 provides an example. New York has a lot of people
and thus will have strong mobility links with any place in the US.
In the disparity filter, edges are checked twice from both nodes’
perspectives: few of New York’s links are stronger than its average,
but almost all of them are the strongest in the perspective of the
smaller towns to which New York connects.
We check New York against a small town in the south, for instance
Franklington in Louisiana. Let’s say that New York’s connections, on
average, involve 10k travelers. The traveler traffic with Franklington
involves only 1k. This is way less than New York’s average so, when
we check this edge from New York’s perspective, we mark it for dele-
tion. However, on average, Franklington’s connections involve only
500 travelers. Thus, when we check the edge from Franklington’s
perspective, we will find it significant and so we will keep it.
 network backboning 341

Figure 24.11: An example of

hub dominance in the DF filter-
ing schema. Edge thickness is
proportional to the weight. We
check the same edge from both
perspectives (blue arrows).

Since you need one success out of the two attempts to keep the
edge, you end up with strong hubs connected to the entire net-
work, and few peripheral connections (hub-spoke structure, or core-
periphery, with no communities). In other words, the disparity filter
tends to create networks with high centralization (Section 11.8),
broad degree distributions, and weak communities. In many cases,
that is fine. For some other scenarios, we might want to consider an
alternative.
In summary, this means that DF ignores the weights of the neigh-
bors of a node when deciding whether to keep an edge or not. There 15
Navid Dianati. Unwinding the hairball
is a collection of alternatives15 , 16 that take this additional piece of graph: pruning algorithms for weighted
information into account and are thus less biased. complex networks. Physical Review E, 93
(1):012304, 2016
In this section I explained the disparity filter only in the case of 16
Valerio Gemmetto, Alessio Cardillo,
undirected networks. You can apply the same technique also for di- and Diego Garlaschelli. Irreducible
rected networks. In this case, you need to make sure that you’re prop- network backbones: unbiased graph
filtering via maximum entropy. arXiv
erly accounting for direction in your p-value calculation: the edge preprint arXiv:1706.00230, 2017
must be significant either when compared to the out-connections of
the node sending the edge, or when compared to the in-connection
weights of the node receiving it.

24.6 Noise-Corrected

The noise-corrected (NC) approach attempts to fix the issues of the 17

Michele Coscia and Frank MH Neffke.
disparity filter17 . In spirit, it is very similar to it. However, the focus Network backboning with noisy data. In
is shifted towards an edge-centric approach: each edge has a different 2017 IEEE 33rd International Conference
on Data Engineering (ICDE), pages
threshold it has to clear if it wants to be included in the network. 425–436. IEEE, 2017
The assumption is that an edge is a collaboration between the nodes.
It has to surpass the weight we expect given both nodes’ typical
connection strength. Again, we have to make sure that this difference
is statistically significant. Figure 24.12 depicts a simplification of the
method.
Formally speaking, the p-value of NC is calculated by looking at
the CDF of a binomial distribution. The observed value (number of
342 the atlas for the aspiring network scientist

successes) is the weight of the edge wu,v , the number of trials is the
total sum of edge weights in the network ∑ wu,v , and the probability
u,v
of success is given by:

∑ wu,v0 × ∑ wu0 ,v
v0 ∈ Nu u0 ∈ Nv
pu,v = !2 .
∑ wu0 ,v0
u0 ,v0

So, in practice, we’re looking at the probability of having a weight

higher than wu,v in a binomial distribution with ∑ wu,v trials and a
u,v
probability of success pu,v . Given that we use a binomial as a null
model, you can see that NC works only for discrete counts as edge
weights, because the binomial is a discrete distribution. Moreover, all
the elements here (wu,v , ∑ wu,v , and pu,v ) are the same in the perspec-
u,v
tive of u and v, thus this measure is u, v specific, differently from the
disparity filter. Of course, if your network is directed, wu,v 6= wv,u
and you’ll get a different null expectation for either direction of the
edge, because the u, v edge is different from the v, u edge.
In the mobility network example I used before, a way to under-
stand the difference between DF and NC is that, in NC, we require
both nodes to agree to keep the edge. So, in this case, the edge be-
tween Franklington and New York will not be kept, because New
York voted for deletion. If you attempt to create backbones with the
same number of edges, Franklington will end up connecting to its
local neighborhood, because those edges are more likely to be agreed
upon by all the smaller towns nearby our focus.
Figure 24.13 abstracts from our geographical example to show the
crux of the difference between DF and NC. DF favors the centraliza-
tion around a hub. NC favors the horizontal peripheral connections
which are the basis of the community structure of the network.
As you might expect, these methods exist because they give very

Figure 24.12: A schematic sim-

1 2 4 4 plification of Noise Corrected
network backboning. The edge
determines its customized
1+2+1+3 / 4 4+4+3+2 / 4
~1.75
3 ~3.25 threshold by building an expec-
tation of the average connection
strength of its two nodes. If
1 3 3 2 its weight is higher than the
expectation in a statistically
1.75 + 3.25 / 2 significant way then the edge is
= 2.5 kept.
 network backboning 343

9 3 Figure 24.13: Different choices

between DF and NC backbon-
ing. Edge width is proportional
7 2 to its weight. Edge color: red
= selected by both DF and NC;
1
blue = DF only; purple = NC
only; gray = neither.
8 4

6 5

different results. It is up to you to decide which of their assumptions

best fits the network you are analyzing and the type of things you
want to say about the network. A naive threshold fixes the same
obstacle for all nodes no matter how strong, favoring the connections
of the hub; HSS can include weaker links if they’re the only path to a
node; DF is similar to naive, but can recognize important weak edges;
and NC overweights peripheries and communities: it is the most
punishing method for the central hubs.

24.7 Measurement Error

To any person who has ever worked with real world data, it should
come as no surprise that datasets are often disappointing. They
contain glaring errors, incomprehensible omissions, and a number
of other issues that make them borderline useless if you don’t pour
hours of effort into fixing them. In fact, I’d say that 80% of data
science is just about cleaning data, and only 20% about shiny and fun
analysis techniques. This obviously applies to network data as well.
You’ll find edges in your networks that shouldn’t be there, and you’ll
have plenty of missing or unobserved connections.
Admittedly, techniques to clean network data would deserve their
own chapter but, frankly, I don’t know many of them. This section
is awkwardly placed here because intimately related to the initial
assumption of noise-corrected backboning – that connections are
noisy. But I could have placed it in the link prediction part as well,
since it’s not only about throwing away observed connections but
also inferring missing ones. In fact, a survey paper about measure- 18
Dan J Wang, Xiaolin Shi, Daniel A
ment error in network data18 points out that measurement error is McFarland, and Jure Leskovec. Mea-
routinely considered only a problem about missing data, rather than surement error in network data: A
re-classification. Social Networks, 34(4):
the more general framing as uncertainty. 396–409, 2012a
Network data cleaning is thus the lovechild of network backboning
and link prediction, but that’s a rather barren marriage – as far as
344 the atlas for the aspiring network scientist

19
Tiago P Peixoto. Reconstructing
I know. In fact, one of the few papers I know19 delivers the truth networks with unknown and heteroge-
in a brutal and deadpan way: “[in network analysis] the practice of neous errors. Physical Review X, 8(4):
041011, 2018
ignoring measurement error is still mainstream”. I hope this tiny
section will contribute to make things change.
To give you an idea of the significance of the measurement error
blind spot in network science, consider the Zachary Karate Club
network. As I’ll explain in details in Section 46.4, everyone in our
field is madly in love with this toy example. The paper presenting the 20
Wayne W Zachary. An information
network20 has been cited more than 4.5k times – and not everybody flow model for conflict and fission in
using this network cites it. The fun thing about this graph is that we small groups. Journal of anthropological
research, 33(4):452–473, 1977
actually don’t know whether it has 77 or 78 edges. The bewildering
thing about this graph is that almost no one even mentions this problem!
The basic way to go about estimating (and correcting) measure- 21
M Newman. Network recon-
ment error is by measuring network data multiple times21 . This is struction and error estimation with
a way to reconstruct a primary error estimate, i.e. to diagnose how noisy network data. arXiv preprint
arXiv:1803.02427, 2018a
good or bad our data collection is. The paper I cited in the previous
paragraph creates a clever Bayesian framework, which enables a
similar result, but does not require multiple measurements. There are
some works outside network science proper that also cite measure-
ment error as one of the many things you should think about when 22
Arun Advani and Bansi Malde.
working with networked data22 . Empirical methods for networks data:
Social effects, network formation and
measurement error. Technical report, IFS
24.8 Summary Working Papers, 2014

1. Backboning is the process of removing edges in a network. Rea-

sons to do so span from a simple need of getting a sparser net-
work, to facilitate computation on large networks, to the removal
of connections that are not statistically significant. Most methods
are developed assuming weighted networks, although you could
apply some of them to unweighted networks as well.

2. One cannot simply establish a fixed threshold and remove all

edges with a weight lower than the threshold. Edge weights
usually distribute broadly and are correlated in different parts of
the network, both factors that make the naive threshold approach
not reasonable.

3. In doubly stochastic backboning, you transform the adjacency ma-

trix in a doubly stochastic matrix (whose rows and columns sum
to one) to break the local edge correlations. Such transformation is
not always possible.

4. In high-salience skeleton, you calculate the short path tree for each
node and you re-weight the edges counting the number of trees
using them. Then you keep the most used edges. This is usually
 network backboning 345

computationally expensive.

5. In disparity filter and noise corrected, you create a null expecta-

tion of the edge weight and keep only the ones whose weight is
significantly higher than the expectation. This expectation is node-
centric in the disparity filter and edge-centric in noise-corrected.

24.9 Exercises

1. Plot the CCDF edge weight distribution of the network at http:

//[Link]/exercises/24/1/[Link]. Calculate its
average and standard deviation. NOTE: this is a directed graph!

2. What is the minimum statistically significant edge weight – the

one two standard deviations away from the average – of the previ-
ous network? How many edges would you keep if you were to set
that as the threshold?

3. Can you calculate the doubly stochastic adjacency matrix of

the network used in the previous exercise? Does the calculation
eventually converge? (Limit the normalization attempts to 1,000. If
by 1,000 normalizations you don’t have a doubly stochastic matrix,
the calculation didn’t converge)

4. How many edges would you keep if you were to return the dou-
bly stochastic backbone including all nodes in the network in a
single (weakly) connected component with the minimum number
of edges?
25
Network Sampling

Sometimes, having a good edge induction or network backboning

technique still doesn’t help you. Sometimes you’re observing a
network directly and it’s just a hairball. In these cases, it’s useful to
make a step back and consider that the act of observation in itself is
not neutral. We decide what to focus on, whether we do it because
of our interests, or simply because of data availability. If we could
zoom out, we would see the structure, as Figure 25.1 shows. When
you’re unable or unwilling to look at the entire network you have to
perform network sampling.

Figure 25.1: A representation

of the sampling conundrum.
(a) The sample you’re able to
observe looks like a hairball,
with everything connected with
everything else. (b) Zooming
out to the whole structure
(a) (b) shows a different story, with a
“Network sampling” means to extract from your network a clear community structure we
smaller version of it. This smaller version, the sample, should be could not observe due to the
a representative subset of the data. By “representative” we mean that improper sample.
the property you’re interested in studying should be more or less the
same in the sample as in the network at large. For instance, if you’re
interested in estimating the clustering coefficient, extracting the only
triangle from a large network which otherwise has none wouldn’t be
a good sampling. The sample’s clustering is one, while the network
at large has a clustering approaching to zero. Put it in other words, a
proper network sampling will ensure that the tiny sliver you observe
is carrying the properties of the whole structure you’re interested in.
To put in perspective how bad the problem is, consider Twitter. As
of writing this paragraph, Twitter has more than 300 million active
users. According to its API, it takes a bit more than a minute on
 network sampling 347

Figure 25.2: The gray circle

~330M monthly users represents the set of users in
~1.1m per user
Twitter. Given the platform’s
~21.78B seconds
~690 years API constraints, a non-stop
one-year crawl of the Twitter
1 year of
crawling network would yield the set of
nodes encompassed by the red
circle.

average to fully know the connections of a user. This means that it 1

Minas Gjoka, Maciej Kurant, Carter T
Butts, and Athina Markopoulou.
takes more than 20 billion seconds to crawl the entirety of Twitter,
Walking in facebook: A case study of
or just a bit less than 700 years. If you were to crawl constantly for unbiased sampling of osns. In 2010
one year, you’d get a bit more than 0.1% of Twitter. If you’re a visual Proceedings IEEE Infocom, pages 1–9.
Ieee, 2010
thinker, Figure 25.2 shows a depiction of the fact I just narrated. You 2
Minas Gjoka, Maciej Kurant, Carter T
can understand that what ends up in your 0.1% has to be the best Butts, and Athina Markopoulou.
possible representation of the whole, and thus it has to be chosen Practical recommendations on crawling
online social networks. IEEE Journal on
carefully. Selected Areas in Communications, 29(9):
We already saw some ways to explore a graph: BFS and DFS 1872–1892, 2011
(Section 10.1). They are reasonable ways to explore a graph, but their
3
Anirban Dasgupta, Ravi Kumar, and
D Sivakumar. Social sampling. In
underlying assumption is that, eventually, they will cover the entire Proceedings of the 18th ACM SIGKDD
network. Here we focus on a slightly different perspective. We don’t international conference on Knowledge
discovery and data mining, pages 235–243.
want the entire network: we want to prevent biases to creep into our ACM, 2012
sample. 4
Neli Blagus, Lovro Šubelj, and Marko
We can classify network sampling strategies – in the broadest Bajec. Empirical comparison of network
sampling techniques. arXiv preprint
terms possible – as induced and topological techniques. These are arXiv:1506.02449, 2015
the focus of the next sections. What I’m writing is based on review 5
Daniel Stutzbach, Reza Rejaie, Nick
works on network sampling1 , 2 , 3 , 4 . I’m going to mostly focus on Duffield, Subhabrata Sen, and Walter
Willinger. Sampling techniques for
the case in which the sampled network is stable, or it is evolving large, dynamic graphs. In Proceedings
too slowly to make any significant difference during the sampling IEEE INFOCOM 2006. 25TH IEEE
International Conference on Computer
procedure. There are specialized methods to sample graphs when
Communications, pages 1–6. IEEE, 2006
this assumption is not true. For instance streaming or evolving 6
Amir H Rasti, Mojtaba Torkjazi, Reza
graphs, whose properties might significantly change as you explore Rejaie, D Stutzbach, N Duffield, and
W Willinger. Evaluating sampling
them5 , 6 , 7 .
techniques for large dynamic graphs.
Nowadays, you rarely want to sample a large graph that you Univ. Oregon, Tech. Rep. CIS-TR-08, 1,
fully own. We have enough computing and storing capabilities to 2008
7
Nesreen K Ahmed, Jennifer Neville,
process humongous structures. The case is different when you rely and Ramana Kompella. Network
on an external data source. Most of the times, such data source will sampling: From static to streaming
graphs. ACM Transactions on Knowledge
be a large social media platform. In this scenario, one has to apply
Discovery from Data (TKDD), 8(2):7, 2014
348 the atlas for the aspiring network scientist

double carefulness. API-based sampling is affected by fundamental

issues. Works in the past have shown that one has to be careful when
working with data sources that potentially yield non-representative 8
Fred Morstatter, Jürgen Pfeffer, Huan
samples of the phenomenon at large8 , 9 . Liu, and Kathleen M Carley. Is the
Note that you are not the only person in the world performing sample good enough? comparing data
from twitter’s streaming api with
network sampling. In most cases, you’re going to work with data
twitter’s firehose. In Seventh international
that has already been sampled by somebody else and you have no AAAI conference on weblogs and social
control over how they extracted that sample from reality. This is true media, 2013
9
Fred Morstatter, Jürgen Pfeffer, and
also if you’re convinced that you are at the data source itself, for Huan Liu. When is it biased?: assessing
instance the API of the social media platform. But then you should the representativeness of twitter’s
ask yourself a few questions. Who has decided to use the platform? streaming api. In Proceedings of the 23rd
international conference on world wide web,
Who is active and who is present but inactive? What data does the pages 555–556. ACM, 2014
provider make available? In such cases, you might need to carefully
consider what you do with the data and/or decide to perform a
network completion process (Section 25.5) – if it is possible at all.

25.1 Induced

Induced sampling works with a guiding principle. You specify a

set of elements that must be in your sample. Then, you collect all 10
Jure Leskovec and Christos Faloutsos.
information that is connected to the elements you selected10 . Sampling from large graphs. In
Proceedings of the 12th ACM SIGKDD
This is related, but not the same thing as, the concept of induced
international conference on Knowledge
subgraph, a graph formed from a subset of the vertices of the graph discovery and data mining, pages 631–636.
and all of the edges connecting pairs of vertices in that subset – see ACM, 2006

Section 24.4. When performing an induced subgraph, you only focus

on nodes, and you won’t obtain new nodes from your induction
procedure. When performing induced samples, instead, you usually
want to add nodes to your sample as well, besides edges.
Differently from simply making an induced graph, you can do
induced sampling in two ways: by focusing on nodes or by focusing
on edges.

Node Induced
If you focus on nodes, it means that you are specifying the IDs of a
set of nodes that must be in your sample. Then, usually, what you do
is collecting all their immediate neighbors. The issue here is clearly
deciding the best set of node IDs from which to start your sampling.
There are a few alternatives you could consider.
The first, obvious, one is to choose your node IDs completely
at random. Random sampling is a standard procedure in many
other scenarios, and has its advantages. If the properties you’re
interested in studying are normally distributed in your population,
a large enough random sample will be representative. However,
 network sampling 349

Figure 25.3: (a) A graph with

a thousand nodes. I select uni-
formly at random 1% of the
nodes, in green. I then induce a
graph with the selected nodes,
all their neighbors, and all con-
nections between them. (b) The
resulting node-induced graph.

(a) (b)

when it comes to real world networks, such expectation might not be

accurate. For two reasons.
First, if your network is large – and if your network isn’t large why
the heck are you sampling it? – choosing node IDs at random might
end up reconstructing a disconnected sample. The likelihood of two
random nodes – or their neighbors – being connected is stupidly
low. Figure 25.3 shows an example of this issue. Even with a very
generous 1% random node sampling – which, in the Twitter example
I made earlier, would mean three million nodes! – the resulting node-
induced graph breaks down in multiple components. This might not
be a problem but, usually, large social networks are connected. Thus
ending up with a disconnected network, by definition, will mean that
you don’t have a representative sample.
Second, one of the properties most network scientists are inter-
ested in is the degree distribution. The degree distribution is emphat-
ically not distributed normally in your population (Section 6.3). Thus,
a random node-induced sample is unlikely to fairly represent the
hubs in your network.
Standard solutions for these two issues are simple. One can weight
their samples. Nodes are more likely to be extracted and be part of
the sample if they have a higher degree or PageRank. However, this
requires knowing this information in advance, which is not feasible if
you’re crawling your network from an API system.

Edge Induced
Another way to generate induced samples is to focus on edges rather
than nodes. This means selecting edges in a network and then crawl
their immediate neighbors. There are a few techniques to do so. One
is the obvious extension of random node induced sampling: random
edge induced sampling. You select edges at random and you collect
350 the atlas for the aspiring network scientist

all their direct neighbors. Two more sophisticated approaches are 11

Nesreen Ahmed, Jennifer Neville,
Totally Induced Edges Samples (TIES)11 and Partially Induced ones and Ramana Rao Kompella. Network
(PIES)12 . sampling via edge-based node selection
with graph induction. 2011
The idea behind edge sampling is that it counteracts the down- 12
Nesreen K Ahmed, Jennifer Neville,
ward bias when it comes to the degree. In a network with a heavy- and Ramana Kompella. Space-efficient
tailed degree distribution, most nodes have a low degree. Thus, if sampling from social activity streams.
In Proceedings of the 1st international
you pick one at random, it’s overwhelmingly likely that it will be
workshop on big data, streams and heteroge-
a low degree node. On the other hand, most edges are attached to neous source mining: algorithms, systems,
large hubs. Thus, if you pick an edge at random, it is likely that programming models and applications,
pages 53–60. ACM, 2012
a hub will be attached to it. This is a similar consideration of the
vaccination strategy we saw in Section 18.3.
There is an obvious downside to the edge sampling technique.
You cannot easily use it when interfacing yourself with a social media
API system. Very rarely such systems will allow you to start your
exploration from a randomly selected edge. Thus, in one way or
another, you’re always going to perform some form of node-induced
sampling.

25.2 Topological Breadth First Search Variants

The alternative to induced sampling is topological sampling. In

topological sampling you also start from a random seed, but then
you start exploring the graph. You’re not limited to the immediate
neighborhood of your seed as in the induced sampling, but you can
get arbitrarily far from your starting point. That is why one of the
key differences between induced sampling and topological sampling
is the seed set size. In induced sampling you have to have the largest
possible seed set, while in topological sampling you can start from a
single seed and explore from there. 13
Sang Hoon Lee, Pan-Jun Kim, and
One of the key advantages of topological sampling is that it works Hawoong Jeong. Statistical properties of
well with API systems. There is also research showing that topolog- sampled networks. Physical Review E, 73
(1):016102, 2006
ical sampling is, in general, less biased than induced sampling13 . If
used for sampling purposes, DFS and BFS graph exploration fall into
this category.
There are fundamentally two families of topological sampling. The
first is a modification of the BFS approach. The idea is to perform
a BFS, but then adding a few rules to prevent some of the issues
affecting that strategy. This is what we focus on in this section. The
second big family is based on random walks and it will be the topic
of the next section. Note that this division is largely arbitrary, as
there is cross-pollination between these two categories, but it is a
useful way to organize this chapter.
 network sampling 351

Snowball

In Snowball sampling we start by taking an individual and asking 14

Leo A Goodman. Snowball sampling.
her to reveal k of her connections14 , 15 . She might have more than k The annals of mathematical statistics,
pages 148–170, 1961
friends, but we only take k. Then, we use these new individuals and
15
Patrick Biernacki and Dan Waldorf.
we ask them the same question: to name k friends. We do so with Snowball sampling: Problems and
a BFS strategy. In practice, Snowball is BFS, but imposing a cap in techniques of chain referral sampling.
the number of connections we collect at a time: k. Figure 25.4 depicts Sociological methods & research, 10(2):
141–163, 1981
the process. Note that k is not the maximum degree of the network,
because a node might be mentioned by more than k neighbors, if they
have them.

Name k
of your
friends
Figure 25.4: Snowball sampling.
Name k Your sampler (blue) starts
of your from a seed (red) and asks for
friends
k = 3 connections. Red names
their green friends, but not
the gray ones. The interviewer
then recursively asks the same
question to each of the newly
sampled green individuals. If
no one ever mentions the gray
ones, those are not sampled and
won’t be part of the network.

Snowball has some advantages. It is cheap to perform in the

real world, where the cost of identifying nodes is high, because the
nodes identify themselves as a part of the survey process. This is less
relevant for social media, where node discovery is relatively easy.
Snowball has a smaller degree bias: with the “nominate-a-friend”
strategy we’re likely to encounter hubs. However, their degree is
somewhat capped, since they can only name k of their friends, rather
than the full list. This generates weird degree distributions with a
sharp cutoff, which aren’t very realistic.
When it comes to sampling from social media, Snowball has a
surprising advantage. It works well with pagination: in API systems,
when you ask the connections of a node, you rarely get all of them.
Social media paginate results, so you only get k connections at a time.
With Snowball you can easily decide the maximum number of pages
you want.
352 the atlas for the aspiring network scientist

Forest Fire

In Forest Fire, like in Snowball, the base exploration is a BFS. How-

ever, once we get all neighbors of a node, we do not explore them
all. Instead, for each of them, we flip a coin and we explore the node
only with probability p. The advantage of Forest Fire is usually 16
Jure Leskovec and Christos Faloutsos.
linked with a proper estimation of the clustering coefficient of the Sampling from large graphs. In
network, since with a BFS we would overestimate it – because we Proceedings of the 12th ACM SIGKDD
international conference on Knowledge
fully explore the neighborhood of nodes16 . discovery and data mining, pages 631–636.
ACM, 2006

Name
all your
friends
Figure 25.5: Forest fire sam-
Name pling. Your sampler (blue)
all your starts from a seed (red) and
friends
asks for all the connections a
node. If the probability test
succeeds, the neighbor turns
green and is also explored. If it
fails, the neighbor remains gray
and is not explored further.

Figure 25.5 provides an example. After sampling a node and

getting all its neighbors, we continue the BFS exploration. But, before
sampling the neighbors of a neighbor, we flip a coin. If the test fails,
we skip the neighbor and we go to the next one. Usually, one won’t
try to visit again the neighbors that have been skipped.
Forest Fire has an interesting relationship with your sampling
budget. Usually, you’re in a scenario in which you have a limited
amount of resources to gather your network – normally, the time it
takes to perform the crawl. Assuming your network is sufficiently
large, all sampling methods seen so far will eventually use up all
your budget. However, if you set p sufficiently low, you might end
up in a situation where your Forest Fire crawl ends before you used
up your budget. In this case you have to decide whether you want to
stop your crawl and forgo the rest of your budget, or you’re allowed
to re-visit skipped nodes. The decision should be made depending
on what’s most important to keep: if the sample’s topological proper-
ties are paramount, you cannot re-visit skipped nodes and you will
have to make peace with having wasted part of your budget.
 network sampling 353

25.3 Random Walk

The random walk sampling family does exactly what you would
expect it to do given its name: it performs a random walk on the
graph, sampling the nodes it encounters. After all, if random walks
are so powerful and we can use them for ranking nodes (Section
11.4) or projecting bipartite networks (Section 23.5), why can’t we use
them for sampling too? I’ll start by explaining the simplest approach
and its problems, moving into sophisticated variants that address its
downsides.

Vanilla
In Random Walk (RW) sampling, we take an individual and we ask
them to name one of their friends at random. Then we do the same
with her and so on. Figure 25.6 shows the usual vignette applied to
this strategy.

Name 1
of your
friends
Figure 25.6: Random walk
Name 1 sampling. Your sampler (blue)
of your starts from a seed (red) and
friends
asks for all the connections a
node (green + gray). One of the
neighbors is picked at random
and becomes the new seed
(green) and, when asked, will
name another green node to
become the new seed.

This is an easy approach which can be very effective, but it has

problems. First, you might end up trapped in an area of the network
where you already explored all nodes, thus unable to find new ones.
This can be easily solved by allowing a random teleportation prob-
ability, just like PageRank does to avoid being stuck in a connected
component of the network.
More importantly, RW sampling has a degree bias. Remember the
stationary distribution (Section 8.1): the probability of ending in a
node with a random walk is known and constant no matter where we
started. And the stationary distribution has a 1-to-1 correspondence
to the degree. This means that high degree nodes are very likely
to be sampled, while low degree nodes not so much. Thus, with
RW, your sample is not representative – at least when it comes to
354 the atlas for the aspiring network scientist

representing nodes with all degrees fairly. 17

Sho Tsugawa and Hiroyuki Ohsaki.
Note that not all biases are entirely bad, some are useful17 . Specifi- Benefits of bias in crawl-based network
cally, we could compare this upward degree bias with the downward sampling for identifying key node set.
IEEE Access, 8:75370–75380, 2020
degree bias of node induced sampling. Arguably, if we have to be
biased, at least let’s oversample the important nodes in the network,
rather than the unimportant ones. This philosophy is implemented 18
Arun S Maiya and Tanya Y Berger-
by the Sample Edge Counts (SEC) method18 . SEC ranks the neigh- Wolf. Benefits of bias: Towards better
bors of all the sampled nodes according to their degree and then characterization of network sampling.
In Proceedings of the 17th ACM SIGKDD
explores the neighbor with the highest edge count towards already
international conference on Knowledge
explored nodes. discovery and data mining, pages 105–113.
In this vein, one could avoid the limit of performing a single ACM, 2011

random walk at a time. A simple extension of RW sampling is m- 19

Bruno Ribeiro and Don Towsley.
dependent Random Walk (MRW)19 . This involves performing m Estimating and sampling graphs with
random walks at once. The random walkers are not independent: multidimensional random walks. In
Proceedings of the 10th ACM SIGCOMM
we choose which of the m random walker will take the next step by
conference on Internet measurement, pages
looking at the degree of the nodes they are currently visiting. Thus, if 390–403, 2010
there are three random walkers and they are currently on nodes with
degrees 3, 2, and 1, we will continue from the first random walker
with 3/(3 + 2 + 1) = 0.5 probability.

Metropolis-Hastings

One way in which we could fix the issues of random walk sampling
is by perform a “random” walk. Meaning that we still pick a neigh-
bor at random to grow our sample, but we become picky about
whether we really want to sample this new node or not.
In the Metropolis-Hastings Random Walk (MHRW), when we
select a neighbor of the currently visited node, we do not accept
it with probability 1. Instead, we look at its degree. If its degree is
higher than the one of the node we are visiting, we have a chance of
rejecting this neighbor and trying a different one. This probability is
the old node’s degree over the new node’s degree. The exact formula
for this decision is p = k v /k u , assuming that we visited v and we’re 20
Daniel Stutzbach, Reza Rejaie, Nick
Duffield, Subhabrata Sen, and Walter
considering u as a potential next step20 , 21 . Willinger. On unbiased sampling for
Thus, if the current node v has degree of 3, and its u neighbor unstructured peer-to-peer networks.
has degree of 100, the probability of transitioning to u is only 3% – IEEE/ACM Transactions on Networking
(TON), 17(2):377–390, 2009
note that this is after we selected u as the next step of the random 21
Balachander Krishnamurthy, Phillipa
walk, thus the visit probability is actually lower than 3%: first you Gill, and Martin Arlitt. A few chirps
have a 1/k v probability of being selected and then a k v /k u probability about twitter. In Proceedings of the first
workshop on Online social networks, pages
of being accepted. If we were, instead, to transition from u to v, we 19–24. ACM, 2008
would always accept the move, because 100/3 > 1, thus the test
always succeeds. In practice, we might refuse to visit a neighbor if
its degree is higher than the currently visited node. The higher this
difference, the less likely we’re going to visit it. A random walk with
 network sampling 355

Name 1 Figure 25.7: Metropolis-

of your
Hastings Random Walk sam-
friends
pling. Your sampler (blue)
starts from a seed (red) and
v u
asks for all the connections
a node (green + gray). One
of the neighbors is picked at
random and we attempt to
Mmm…
Name make it the new seed (green).
another However, since u has so many
connections, it is likely that the
sampler will ask for a different
neighbor.
this rule will generate a uniform stationary distribution. Figure 25.7
shows the mental process of our Metropolis-Hastings sampler.

Re-Weighted
In Re-Weighted Random Walk (RWRW) we take a different approach.
We don’t modify the way the random walk is performed. We extract
the sample using a vanilla random walk. What we modify is the way
we look at it. Once we’re done exploring the network, we correct the 22
Matthew J Salganik and Douglas D
result for the property of interest22 , 23 . Say we are interested in the Heckathorn. Sampling and estimation in
hidden populations using respondent-
degree. We want to know the probability of a node to have degree driven sampling. Sociological methodology,
equal to i. We correct the observation with the following formula: 34(1):193–240, 2004
23
Amir Hassan Rasti, Mojtaba Tork-
∑ i −1 jazi, Reza Rejaie, Nick Duffield, Wal-
v∈Vi ter Willinger, and Daniel Stutzbach.
pi = . Respondent-driven sampling for charac-
∑ xv−0 1 terizing unstructured overlays. In IEEE
v 0 ∈V INFOCOM 2009, pages 2701–2705. IEEE,
2009
The formula tells us the probability of a node to have degree equal
to i (pi ). This is the sum of i−1 – the inverse of the value – for all
nodes in the sample with degree i (Vi ), over 1/ degree (xv−0 1 ) of all
nodes in the sample (V). This is also known as Respondent-Driven 24
H Russell Bernard and Harvey Rus-
sell Bernard. Social research methods:
Survey24 , because it is used in sociology to correct for biases in the Qualitative and quantitative approaches.
sample when the properties of interest are rare and non-randomly Sage, 2013
distributed throughout the population. Figure 25.8 attempts to break
down all parts of the formula.
Let’s make an example. Suppose you want to estimate the proba-
bility of a node to have degree i = 2. First, you perform your vanilla
random walk sample. Say you extracted 100 nodes. Twenty of those
nodes have degree equal to two. So your numerator in the formula
will be the sum of i−1 = 1/2 for |Vi | = 20 times: 20 ∗ 1/2 = 10. If
we assume that there were 50 nodes of degree 1, 10 of degree 3, 8 of
356 the atlas for the aspiring network scientist

p of nodes
with value i Figure 25.8: The Re-Weighted
Random Walk formula, esti-
Set of mating the probability pi of
nodes observing the i value in a prop-
with erty of interest, using the set
value i of sampled nodes Vi with that
particular value in the total set
of v sampled nodes.

Value
Set of nodes for v’
in the sample

degree 4, 7 of degree 5, and 5 of degree 6, our denominator would be:

(50/1) + (20/2) + (10/3) + (8/4) + (7/5) + (5/6),

which is 67.56̄. Hopefully, you can spot what I did there. To bring
the formula together, we discover that p2 = 10/67.56̄ ∼ 0.148. So
RWRW is telling us that the overall probability of a node having
degree equal to 2 is not 20% as we would have inferred from the –
biased – random walk sample. It is actually lower, it is 14.8%. 25
Murray Rosenblatt. Remarks on some
nonparametric estimates of a density
Note that the formula reported here only works when the variable
function. The Annals of Mathematical
of interest is discrete, i.e. i is an integer, like in the case of the degree. Statistics, pages 832–837, 1956
You can still apply RWRW sampling even if the variable you want 26
Emanuel Parzen. On estimation of a
probability density function and mode.
to study is continuous, for instance the local clustering coefficient.
The annals of mathematical statistics, 33(3):
However, you’ll have to perform a kernel density estimate25 , 26 , 27 , and 1065–1076, 1962
I’m not particularly keen of going into that nest of vipers. You’re on 27
[Link]
Kernel_density_estimation
your own, have a blast. 28
Douglas D Heckathorn and Christo-
RWRW works particularly well when there are hidden populations pher J Cameron. Network sampling:
who might actively try not to be sampled28 . For instance, it has been From snowball and multiplicity to
respondent-driven sampling. Annual
successfully applied to the sampling of drug users29 . review of sociology, 43:101–119, 2017
RWRW has a crucial downside. While it is excellent to estimate 29
David C Bell, Elizabeth B Erbaugh,
the distribution of a property in a network, it will still return a biased Tabitha Serrano, Cheryl A Dayton-
Shotts, and Isaac D Montoya. A
vanilla random walk sample. So, if what you need was the sample comparison of network sampling
rather than the estimation of a simple measure, you’re out of luck. designs for a hidden population of drug
You cannot use this method to have a representative sample. users: Random walk vs. respondent-
driven sampling. Social science research,
62:350–361, 2017
Neighbor Reservoir Sampling
30
Xuesong Lu and Stéphane Bressan.
Sampling connected induced subgraphs
Neighbor Reservoir Sampling30 (NRS) is one of those methods uniformly at random. In International
Conference on Scientific and Statistical
blending between the two families of sampling I talked about. It Database Management, pages 195–212.
happens in two phases. In the first phase, NRS builds its set of core Springer, 2012
 network sampling 357

nodes and connections. Starting from the seed we provide as an

input, NRS performs a normal random walk, including in the sample
all nodes and edges it finds during this exploration.
However, the majority of NRS’s budget is spent in the second
phase. Once we have a core, we start modifying it. Suppose that, after
the first phase, you sampled nodes in a set V 0 . In this second phase,
you make a loop. At each iteration i of the loop, you pick two nodes
at random: u and v. Node v is a member of V 0 , the set of explored
nodes. Node u is not a member of V 0 – meaning that you haven’t
explored it yet, but it is a neighbor of a node in V 0 .
Our objective is to add u to V 0 and to remove v from V 0 at the
same time. We can do it only if two conditions are met. First condi-
tion: we want our sample to be a single connected component. We
cannot remove v if that would break the graph into multiple com-
ponents – adding u isn’t going to add new components, because
we only consider us that are connected to a node in V 0 ensuring
connectivity. Note that u and v usually are not connected to each
other.
The second condition is a random test. We extract a uniform
random number 0 < α < 1. We perform the switch if and only if
α < |V 0 |/i, where i is set to be equal to |V 0 | at the beginning and
it is increased by one at each attempt. In practice, this has a few
consequences. By swapping u and v, we ensure that the size of our
sample stays constant, i.e. |V 0 | doesn’t change. Moreover, at the
beginning, since i ∼ |V 0 | all initial attempts to modify V 0 succeed. As
we progress, the chances of accepting a new node in the set vanish.
This isn’t really a random walk nor a BFS, because the random
neighbor selected can be from any node in V 0 . So you can see how
hard it is to fit it into a neat category.
NRS ensures a realistic clustering coefficient distribution. How can
that happen? The trick lies in the connectivity test. We only perform
the u-v swap if it doesn’t break the network into distinct connected
components. Which means that not all vs have the same probability
of being removed from the sample. The v with higher clustering
have higher probability to be replaced, because by removing them
it is more likely that the graph will stay connected. High clustering
coefficient means that their neighbors are connected to each other
(see Section 9.2), thus making it more likely there are alternative
paths to keep the network together.
To see why it’s the case, consider Figure 25.9. NRS will pick a
node in green and a node in red at random. It will then remove the
red node and add the green node. However, it will always refuse to
perform the operation if the red node you pick is node 5. Removing
that node will create two connected components, which is unaccept-
358 the atlas for the aspiring network scientist

19
20 Figure 25.9: Neighbor Reser-
voir sampling. Nodes in the
12
11 explored set V 0 are in red.
8 21
Neighbors of V 0 – the reservoir
3
– are in green.
18 9 7 5

2
4 14
6
17 10
1

15 16
13

able. Other unlucky u-v draws are forbidden too. For instance, you
cannot perform the swap if you pick nodes 3 and 12.
Figure 25.10 shows an example of how some of these different
strategies would explore a simple tree. I don’t show RWRW, because
the samples it extracts are indistinguishable from the vanilla random
walk ones. I also don’t include NRS, because it’s too subtle to really
be appreciated in a figure like this one.

25.4 Sampling Issues

When talking about Snowball sampling I mentioned the issue of

pagination. To recap: social media APIs will rarely give you all
connections of a user when you ask for them. Rather, they will send
you a list of k, chosen with some criterion that is opaque to you
(likely in the order they are stored in their internal database). If
you want the remaining ones, you have to ask again. You’re always
getting k connections at a time. Each request is a “page”, with k
being the page size.
It can be tricky to know how pagination will affect crawl time.
Imagine two different API policies. The first returns big pages –
say 100 edges per page – but requires a long waiting time between
queries – say two seconds. The second policy returns small pages
– ten edges per page – but more often – you only need to wait one
second between queries. We can calculate the edge throughput of
these two policies. In this case, the one with big pages returns more
edges per unit of time, on average: 50 edges per second versus 10
edges per second. However, how will these policies behave on a real
world network?
As we saw in Section 6.3, real world networks have broad degree
distributions, like the one we show in Figure 25.11. For some of these
 network sampling 359

(a) BFS (b) DFS (c) Snowball

(d) Random Walk (e) MHRW (f) Forest Fire

Figure 25.10: Examples of how
nodes, the second policy is better: if they have 10 or less edges, we different network sampling
can fully explore them with a single query, thus we’re going faster strategies explore a given net-
because we have lower waiting times between nodes. In Figure 25.11, work. Each node is labeled
we color in blue the part of the degree distribution for which this with the order in which it is
holds true. If the node has a degree higher than 10, then the first explored. The node color shows
policy is better, because it requires to perform fewer queries, even if whether the node was sampled
they are spaced out more in time. In Figure 25.11, we color in purple (green) or not (red), assuming
the part of the degree distribution for which this holds true. a budget of 15 units and a con-
The second policy is in theory 5 times slower than the first (10 stant cost of 1 unit per node.
edges/sec versus 50 edges/sec, on paper), however it will allow you Snowball assumes n = 3 (un-
to crawl this network in half of the time31 . This is because, in a broad labeled nodes are not explored
degree distribution, we have way more nodes with low degree – for due to this parametric restric-
which the second policy is faster. In Figure 25.11, out of 500k nodes, tion), while Forest Fire has a
492k have degree of 10 or less. burn probability of .5.
You could conclude that the best API policy possible is the one
31
Michele Coscia and Luca Rossi.
that gives you only one node at a time, imposing no waiting time Benchmarking api costs of network
between requests. This is true only in theory. In practice there are a sampling strategies. In 2018 IEEE
few things you need to consider, which you can lump into the issue International Conference on Big Data (Big
Data), pages 663–672. IEEE, 2018
of network latency. First, it still takes time for the information to
travel from the server to your computer. This is not exactly the speed
of light, so the requests will never be truly instantaneous. Second,
a server which gets hit too frequently with too many requests will
also naturally slow down, often in unpredictable ways. Thus some
level of pagination and waiting time will always be part of an API
360 the atlas for the aspiring network scientist

106
Figure 25.11: A power law de-
105
gree distribution, showing the
104 count of nodes (y-axis) with
Count
3
10 a given degree (x-axis). The
2
10 colors in the plot represent in
which cases the first API policy
101
described in the text is faster
100
100 101 102 103 than the second (purple) and
k when the second is faster than
the first (blue).
system. Which means that there are going to be trade-offs when
reconstructing the underlying network.
Pagination is often not the only thing you need to worry about.
Other challenges might be sampling a network in presence of hostile 32
Edward Bortnikov, Maxim Gure-
behavior32 . For instance, some hostile nodes will try to lie about their vich, Idit Keidar, Gabriel Kliot, and
connections and it’s your duty to reconstruct the true underlying Alexander Shraer. Brahms: Byzantine
resilient random membership sampling.
structure. Or not: there are reasonable and legit reasons to lie about
Computer Networks, 53(13):2340–2359,
one’s connection, for instance to protect one own privacy. 2009
In another scenario, you might not be interested in the topological
properties of the full network. What’s interesting for you is just
estimating the local properties of one – or more – nodes. In that case, 33
Manos Papagelis, Gautam Das, and
specialized node-centric strategies can be used33 . Nick Koudas. Sampling online social
networks. IEEE Transactions on knowledge
and data engineering, 25(3):662–676, 2013
25.5 Network Completion

Network completion is a related – but not identical – problem. Like

sampling, it wants to establish a topological strategy for the explo-
ration of a network. Different from sampling, its aim is not to extract
a smaller version of the full dataset. Rather, we want to complete the
sample. The idea is that you downloaded from somewhere a sample
of a network, but you are able to process a larger dataset. Rather than
starting collecting data from scratch, you can use what you have as a
seed and try to complete it.
The question now is: what’s the most efficient way to do so? What
strategy would give you the most information about the full structure
with the least amount of effort? Specifically, you want to obtain the
highest possible number of new nodes by asking the lowest amount
possible of new queries. You could simply apply any of the network
sampling strategies I explained so far. However, there are dedicated
techniques developed to solve this problem.
Note that here you don’t know the strategy originally used to
collect the sample you’re given. If you knew that it was a Metropolis-
Hastings random walk you’d probably use a different strategy than
 network sampling 361

if it was a standard BFS. But, since you don’t know this piece of
information, you need a general strategy working regardless of the
shape of the initial sample.
Naively, you might think to just go and probe the nodes with the
highest degree. However, there are a few considerations to make.
First, since – by definition – your sample is incomplete, you don’t
really know the true degree of a node. You only know how many
neighbors it has in your sample. Second, since the node has a high
degree in your sample, there’s some chance you already explored all
its neighbors, thus probing it won’t help you. 34
Sucheta Soundarajan, Tina Eliassi-
The first technique, MaxReach34 , estimates the true degree of a Rad, Brian Gallagher, and Ali Pinar.
node and its clustering coefficient using the information gathered Maxreach: Reducing network incom-
pleteness through node probes. In 2016
so far. It does so with a technique similar to Re-Weighted Random
IEEE/ACM International Conference on
Walk. The difference is that, in RWRW, we only want to know how Advances in Social Networks Analysis
many nodes have a given degree i. In MaxReach, we want to also and Mining (ASONAM), pages 152–157.
IEEE, 2016
know which nodes have that given degree value. At this point, the
score of a node is the difference between its estimated degree and its
degree in the sample. Nodes with higher scores are probed earlier.
After each probe, since we gathered more information in the sample,
MaxReach will recalculate the degree estimates. 35
Sucheta Soundarajan, Tina Eliassi-
e-wgx35 is a more recent alternative. Rad, Brian Gallagher, and Ali Pinar.
ε-wgx: Adaptive edge probing for
enhancing incomplete networks. In
25.6 Summary Proceedings of the 2017 ACM on Web
Science Conference, pages 161–170. ACM,
2017
1. Network sampling is a necessary operation when the network you
need to analyze is too large and/or you need to gather data one
node/edge at a time from a high latency source (e.g. the API of a
social media platform). Sometimes the decision is not up to you
and all you can access is a sample made by somebody else.

2. The main objective is to extract a sample that is representative of

the network at large for the property you’re interested in studying.
For instance, it has to have a comparable degree distribution if you
want to infer its shape (e.g. whether it is a power law).

3. Sampling methods can be induced or topological. In induced

sampling you extract a random sample of nodes/edges and you
collect all that it is attached to it. In topological sampling you
explore the structure one node at a time.

4. Variants of BFS explorations are: Snowball, in which we impose a

maximum number k of explored neighbors of a node; and Forest
Fire, in which we have a probability of rejecting some edges.

5. Variants of random walk exploration are: Metropolis-Hastings,

where we have a probability of refusing visiting high degree nodes;
362 the atlas for the aspiring network scientist

and Re-Weighted, where we correct the statistical properties of the

network after we collected it.

6. When sampling from real API systems one has to be careful that
the throughput in edges per second is not necessarily a good
indicator of how quickly you can gather a representative sample.
Due to pagination, high-throughput sources might return smaller
samples.

7. A related problem is network completion: given an incomplete

sample of a network, find the best strategy to complete the sample
in the least number of queries possible.

25.7 Exercises

1. Perform a random walk sampling of the network at [Link]

[Link]/exercises/25/1/[Link]. Sample 2,000 nodes
(1% of the network) and all their connections (note: the sample
will end up having more than 2,000 nodes).

2. Compare the CCDF of your sample with the one of the original
network by fitting a log-log regression and comparing the ex-
ponents. You can take multiple samples from different seeds to
ensure the robustness of your result.

3. Modify the degree distribution of your sample using the Re-

Weighted Random Walk technique. Is the estimation of the expo-
nent of the CCDF more precise?

4. Modify your random walk sampler so that it applied the Metropolis-

Hastings correction. Is the estimation of the exponent of the CCDF
more precise? Is MHRW more or less precise than RWRW?
 Part VIII

Mesoscale
26
Homophily

“Mesoscale” is the term we use to indicate network analyses that

operate at the level that lies in between the global and the local one.
At the global level, we have analyses that sum up the topological
characteristics of a network with a single number. For instance, the
exponent of the degree distribution, the global clustering coefficient,
or the diameter. At the local level, we sum up individual node’s
characteristics with a single number. They can be its degree, its local
clustering coefficient, or closeness centrality.
At the mesoscale we want to describe groups of nodes. How do
they relate to each other? How does their local neighborhood look
like? There are many different meso-level analyses you can perform.
This part of the book groups almost all of them together, leaving one
out: community discovery. Community discovery is, by far, the most
popular meso-level analysis of complex networks. Given its size, it
deserves a part on its own, which will be the next one. Here, we’re
talking about all the meso-rest.

Figure 26.1: Some examples

of homophily driven by spa-
tial, temporal, and attribute
similarity.

Space Time Similarity

 homophily 365

We start with homophily: the love (philia) of the similar (homo).

“Birds of a feather flock together” is a popular way of saying. It
originates from the fact that many species of birds flock with in-
dividuals of their own kind and coordinate when moving around.
This phrase has been adopted in sociology to exemplify the con-
cept of homophily: people will tend to associate with people with
similar characteristics as their own. In a social network, homophily
implies that nodes establish edges among them if they have similar 1
Miller McPherson, Lynn Smith-Lovin,
attributes1 . If you have a particular taste in movies, and there are two and James M Cook. Birds of a feather:
potential friends, you are more likely to choose the one with similar Homophily in social networks. Annual
review of sociology, 27(1):415–444, 2001
tastes as yours, because you have more things to talk about and have
less potential for conflict.
Many factors influence and favor homophily, and they are not
necessarily exclusively explained by individual preference: some-
times homophily is a property emerging from access. It’s not only
the fact that you don’t like the different, but rather that you cannot
access the different. In other words, homophily is not only the result
of our preferences, but also of social constructs. That is why the term
“homophily” is problematic, and we use it only because of historic
reasons.
With that said, let me go through a carousel of examples of ho-
mophily, some of which I represent in Figure 26.1. There are so many
observed examples in real world social networks that I have to push
their references down to the next page otherwise my Latex won’t
2
Juliette Stehlé, François Charbonnier,
compile. So have a picture of my cat Ferris. He is, incidentally, a
Tristan Picard, Ciro Cattuto, and Alain
great example of homophily, in that he hates everything that is not Barrat. Gender homophily from
himself. spatial behavior in a primary school: a
sociometric study. Social Networks, 35(4):
604–613, 2013

Figure 26.2: My cat Ferris. In

the picture, I color in orange the
parts of the cat that are orange.
3
Marta C González, Hans J Herrmann,
J Kertész, and Tamás Vicsek. Commu-
nity structure and ethnic preferences in
school friendship networks. Physica A,
379(1):307–316, 2007
4
Kara Joyner and Grace Kao. School
racial composition and adolescent racial
homophily. Social science quarterly, pages
810–825, 2000
5
Elizabeth Aura McClintock. When does
race matter? race, sex, and dating at an
elite university. Journal of Marriage and
First, gender2 and race3 are glaring examples. School children are Family, 72(1):45–72, 2010
6
Kelly Raley, Megan Sweeney, and
more likely to make friends with people sharing their gender or race4 .
Danielle Wondra. The growing racial
We observe this in adults too: in marriage ties it is so overwhelmingly and ethnic divide in us marriage
likely to date5 or marry6 someone of the same race that sociologists patterns. Future of children, 25(2):89, 2015
366 the atlas for the aspiring network scientist

don’t study this fact any more because it’s so boringly obvious. In 7
Yuexin Jiang, Daniel I Bolnick, and
this, we’re truly similar to other animals we often look down to7 . Mark Kirkpatrick. Assortative mating in
Rather than asking whether romantic ties show homophily, it’s animals. The American Naturalist, 181(6):
E125–E138, 2013
more interesting to use the degree of homophily of romantic ties to
compare societies.
8
Salvatore Scellato, Anastasios Noulas,
In Figure 26.3 you see an example of mixed marriage in the United
Renaud Lambiotte, and Cecilia Mascolo.
States. To that diagonally dominated matrix, you have to add the Socio-spatial properties of online
consideration that the United States is probably one of the most location-based social networks. In
ICWSM, 2011
diverse countries in the world. Imagine how this would look like 9
Kerstin Sailer and Ian McCulloh.
elsewhere! Social networks and spatial configura-
tion—how office layouts drive social
interaction. Social networks, 34(1):47–58,
2012

Figure 26.3: The mixing ma-

trix of interracial marriage in
the US: share of husbands per
race with a wife of a given race
(Census Bureau).
10
Ling Heng Wong, Philippa Pattison,
and Garry Robins. A spatial model for
social networks. Physica A, 360(1):99–120,
2006
11
Dashun Wang, Dino Pedreschi,
Chaoming Song, Fosca Giannotti, and
Albert-Laszlo Barabasi. Human mobility,
social ties, and link prediction. In
SIGKDD, pages 1100–1108. Acm, 2011a
12
Jameson L Toole, Carlos Herrera-
Yaqüe, Christian M Schneider, and
Another example is spatial homophily: living in the same place Marta C González. Coupling human
mobility and social ties. Royal Society
makes it easier to have stronger connections, a factor that overcomes
Interface, 12(105):20141128, 2015
other correlates such as race8 , 9 , 10 . A sub-type of spatial homophily 13
Eunjoon Cho, Seth A Myers, and Jure
is mobility homophily: going to the same places influences the Leskovec. Friendship and mobility:
user movement in location-based social
likelihood of connecting socially11 , 12 . The reverse is also true – as it
networks. In SIGKDD, pages 1082–1090.
might seem obvious –: being friends increases the likelihood to go ACM, 2011
to the same places13 . The connection between geographical space 14
Jukka-Pekka Onnela, Samuel
Arbesman, Marta C González, Albert-
and social space is very strong, showing how, even in presence of László Barabási, and Nicholas A
(almost) limitless communication ranges, social ties still decay with Christakis. Geographic constraints on
distance14 , 15 , 16 . social network groups. PLoS one, 6(4):
e16939, 2011
Another factor of homophily is time, meaning that being in the 15
Michele Coscia and Ricardo Haus-
same age range favors connections. Think about school friends: 38% mann. Evidence that calls-based and
of a person’s friends are within a 2-year age gap – this figure comes mobility networks are isomorphic. PloS
one, 10(12):e0145091, 2015
from McPherson’s paper. 16
Pierre Deville, Chaoming Song,
In the rest of the chapter we are going to explore some techniques Nathan Eagle, Vincent D Blondel,
to study the mesoscale, such as the usage of ego networks. We’re Albert-László Barabási, and Dashun
Wang. Scaling identity connects human
going to quantify homophily and see some of the consequences in mobility and social interactions. PNAS,
network dynamics. 113(26):7047–7052, 2016
 homophily 367

26.1 Ego Networks

Ego networks are a common technique to explore the meso-level 17

Nick Crossley, Elisa Bellotti, Gemma
around a node17 . “Ego” in Latin means “I”, the self. An ego network Edwards, Martin G Everett, Johan
is a subset of a larger network. You first have to identify your “ego”: Koskinen, and Mark Tranmer. Social
network analysis for ego-nets: Social
the node on which the ego network is centered. Then, you select network analysis for actor-centred networks.
all of its neighbors and the connections among them. The resulting Sage, 2015
network formed by all the nodes and edges you selected is an ego
network. Figure 26.4 provides an example of this procedure.

Figure 26.4: The procedure to

extract an ego network from
a larger network. (a) We se-
lect the ego (in red) and all
its neighbors (in green). (b)
We create a view only using
red and green nodes, and all
connections among them.

(a) (b)
Once you have an ego network, you can start investigating its
“global” properties such as the degree distribution or its homophily,
and these are not properties of the global network as a whole, but of
the local neighborhood of the ego, the ego network, which lives in
the mesoscale. Ego networks are frequently used in social network 18
Stephen P Borgatti, Ajay Mehra,
analysis18 , 19 , for instance to estimate a person’s social capital20 . Daniel J Brass, and Giuseppe Labianca.
Network analysis in the social sciences.
A consequence of this procedure is that we know that an ego science, 323(5916):892–895, 2009
node is connected to all nodes in its ego network. This is unfortunate 19
Jure Leskovec and Julian J Mcauley.
in some cases, depending on our analytic needs. For instance, all Learning to discover social circles in
ego networks. In Advances in neural
ego networks have a single connected component and will have information processing systems, pages
a diameter of two. If those forced properties are undesirable, one 539–547, 2012
can extract an ego network and then remove the ego and all its 20
Stephen P Borgatti, Candace Jones,
and Martin G Everett. Network
connections. measures of social capital. Connections,
21(2):27–36, 1998

26.2 Assortativity & Disassortativity

When it comes to homophily, we want to have an objective way to

quantify how much it is driving a network’s connections. This means
that the nodes connect to other nodes depending on the value of one
of their attributes. There are two possible scenarios. The attribute
driving the connections could be quantitative (e.g. age) or qualitative
(e.g. gender). When the attribute is quantitative, you can use the
368 the atlas for the aspiring network scientist

same technique to estimate the degree assortativity, which we cover

in the next chapter.
Here we focus on the case of a qualitative attribute. Let’s start by
making a simple scenario: biological sex. In humans, this is – barring
rare and exceptional cases – a categorical binary attribute.

Figure 26.5: A toy example to

test our measures of homophily.
We represent the categorical
binary attribute with node color,
with two possible values: red
and green.

In this scenario, you can estimate the probability of an edge to

connect alike nodes, and compare it to the probability of connection
in the network. Consider Figure 26.5. We have 20 edges connecting
nodes with the same color over 22 total edges in the graph. Therefore,
the observed probability of edges between alike nodes is 20/22. In
the graph we have 11 nodes, thus the number of possible edges is
|V |(|V | − 1)/2 = 55 (with |V | = 11). So the probability of having
a connection between any node pair is 22/55. Thus we see that the
probability of an edge being between alike nodes is more than twice
what we would have expected: (20/22)/(22/55) ∼ 2.27. Values
higher than one imply homophily, while values lower than one mean
that nodes tend to connect with similar nodes less than we expect
– i.e. the network is disassortative, nodes don’t like to connect to
similar nodes, another totally valid thing that can happen often in
social networks (in Section 26.4 I call this concept “heterophily”).
This approach breaks down if you have more than two possible
values for your attribute, and also if some values are more popular
than others. In these cases, you might conclude that there is assorta-
tivity in a non-assortative network, simply because you’re assuming
the incorrect null model of equal attribute value popularity. 21
Mark EJ Newman. Assortative mixing
In this case, you should use a different approach21 , 22 . You want to in networks. Physical review letters, 89(20):
look at the probability of edges connecting alike nodes per attribute 208701, 2002
22
Mark EJ Newman. Mixing patterns in
value i, and then compare it to the probability of an edge to have at networks. Physical Review E, 67(2):026126,
least one node with attribute value i. The formula is: 2003a

∑ eii − ∑ ai bi
i i
r= ,
1 − ∑ a i bi
i

where eii is the probability of an edge to connect two nodes which

both have value i, ai is the probability that an edge has as origin a
 homophily 369

node with value i, and bi is the probability that an edge has as desti-
nation a node with value i. In an undirected network, the latter two
are equal: ai = bi . This formula takes values between −1 (perfect dis-
assortativity) and 1 (perfect assortativity: each attribute is a separate
component of the network).

2 2
((8 /22)+(12/ 22))−((10/ 22) +(14 / 22) ) Figure 26.6: How to calculate
2 2
1−((10 / 22) +(14 / 22) )
homophily using the formula in
the text.

~0.766

Fraction of edges Fraction of edges

connecting nodes
with attribute
∑ eii −∑ ai bi ending to a node
with attribute
equal to i r= i i
equal to i
1−∑ ai bi
i

In Figure 26.6 we have two values i: red and green. There are 22
edges in the graph: eight green-green edges – thus the probability
is 8/22 – and 12 red-red edges – thus the corresponding eii value is
12/22. Ten edges originate (or end) in a green node: ai = bi = 10/22;
and 14 originate (or end) in a red node: ai = bi = 14/22. The final
value of homophily is ∼ 0.766. This value is interpretable as a sort of
Pearson correlation coefficient, which means that 0.766 is pretty high.

26.3 Strength of Weak Ties

Is homophily a good thing? In some aspects yes. A person who is

surrounded by people with similar tastes and behaviors is happy.
But suppose this person is looking for a job. It is very difficult, in
presence of high homophily, for a message to arrive to the job seeker,
because she only has close ties who cannot broker to her new infor-
mation from the outside – assuming that the network has a strong
assortative community structure.
The ties that bind different communities with different people
are the so-called “weak ties” and they have been shown to be funda- 23
Mark S Granovetter. The strength
mental in the job market by Granovetter23 , 24 . To put it simply: it’s of weak ties. In Social networks, pages
rarely your closest friends who make you find a job, but that far ac- 347–367. Elsevier, 1977
24
Mark Granovetter. The strength of
quaintance with whom you rarely speak, because your close friends
weak ties: A network theory revisited.
usually access the same information as you do and so cannot tell you 1983
anything new. Figure 26.7 shows an example of the weak ties effect.
Note that Granovetter divides ties in three types: weak, strong,
370 the atlas for the aspiring network scientist

Figure 26.7: An example of

strength of weak ties. The
green individual is part of a
different community and thus
only weakly linked with the
red community. However, by
being exposed to different infor-
mation, she can bring it to the
community she is not part of,
but connected to it via a weak
tie.

and absent. The terminology should not fool you. In this case, we
are not referring to the edge’s weight (Section 3.3). This is rather
a categorical difference, more akin to multilayer networks (Section
4.2). A weak tie is established between individuals whose social
circles do not overlap much. A strong tie is the opposite: an edge
between nodes well embedded in the same community. The absent
tie is more of a construct in sociology, which lacks a well defined
counterpart in network science. It can be considered as a potential
connection lurking in the background. For instance, there is an
absent tie between you and that neighbor you always say “hello” to
but never interact beyond that. You could consider an absent tie as
one of the most likely edges to appear next, if you were to perform a
classical link prediction (Part VI).
You can see now that you can have strong, weak, and absent ties
in an unweighted network. We can, of course, expect a correlation
between being a weak tie and having a low weight. However, we can
construct equally valid scenarios in which there is an anti-correlation
instead. For instance, we could weight the edges by their edge be-
tweenness centrality (Section 11.2). A weak tie must have a high edge
betweenness, because by definition it spans across communities and
thus all the shortest paths going from one community to the other
must pass through it.
Note that, notwithstanding their usefulness in favoring informa-
tion spread, weak ties are not the only game in town in a society. The
competing concept of the “strength of strong ties” shows that strong
ties are important as well. They are specifically useful in times of 25
David Krackhardt, N Nohria, and
uncertainty: “Strong ties constitute a base of trust that can reduce B Eccles. The strength of strong ties.
Networks in the knowledge economy, 82,
resistance and provide comfort25 ”. 2003
 homophily 371

26.4 Homophily & Social Contagion

Homophily can lead to a surprising number of counter-intuitive

social dynamics. This section is intimately linked with Part V, where
we looked at spreading events in networks. Here, we explore some
more social explanations behind behavioral change in networks,
mostly fueled by the right combination of homophily and heterophily
(the love of the different).

Figure 26.8: A dating network.

The node color encodes the
gender (red = female, blue =
male).

What’s heterophily? Some things in social networks are very dis-

assortative. For instance, consider sexual partners. When looking
at some attributes, it is a network driven by homophily: people try
to find mates with similar characteristics. They like the same music,
movies, food. On the other hand, other attributes are very disassor-
tative, for instance gender. Notwithstanding notable exceptions, the
majority of edges in this network are between unlike genders – as
Figure 26.8 shows.
If we live in a network governed by homophily, we know that
connections are driven by the characteristics of the nodes. In some
cases that is the only possible explanation. For instance, race is given:
With the possible exception of
26
one cannot change their race26 and race homophily means that one’s Michael Jackson.
race influences which social connections are more or less likely to be
established.
But consider the other side of the coin: if we observe a strong
homophily, it could be because our social connections are influencing
us into adopting behaviors we would not otherwise. For instance,
drug use. One can decide whether to use drugs, and will be more
likely to do so if the majority of their friends are drug users. It turns
out that the right network topology can create an illusion of majority.
Even if the majority of people do not use drugs, we can draw a 27
Kristina Lerman, Xiaoran Yan, and
network in which everybody thinks that the opposite is true27 ! Xin-Zeng Wu. The" majority illusion" in
You can look at Figure 26.9 to see an example of this counter social networks. PloS one, 11(2):e0147617,
2016
intuitive result. Or, you can play a simple game showing how to
build networks fooling people into thinking everybody is binge-
372 the atlas for the aspiring network scientist

Figure 26.9: The majority illu-

sion in a toy network. Nodes
in red are drug users, nodes in
green are not. For every node,
its neighbors include a majority
of drug users.

28
[Link]
drinking28 .
Since humans are social animals and tend to succumb to peer
pressure, homophily can be a channel for behavioral changes. In a
health study, researchers looked at health indicators from thousands
of people in a community over 32 years. They saw that behavior and
health risks that are not contagious actually are. For instance obesity:
if you have an obese friend, the likelihood of you becoming obese 29
Nicholas A Christakis and James H
increases by 57% in the short term29 . This is like the Susceptible- Fowler. The spread of obesity in a
Infected epidemic models we saw, even if obesity is not a biological large social network over 32 years.
New England journal of medicine, 357(4):
virus. It is rather a social type of virus.
370–379, 2007
Same with smoking, although in this case it worked the opposite: 30
Nicholas A Christakis and James H
people were quitting in droves30 . This is due to social pressure and Fowler. The collective dynamics of
homophily: a behavior you might not adopt by yourself is brokered smoking in a large social network. New
England journal of medicine, 358(21):
by your social circle, which you trust because it is made by people
2249–2258, 2008
like you – it speaks to your identity.

Figure 26.10: The network of

political blogs. Each node is a
blog. Node’s color encodes its
political leaning (blue = demo-
crat, red = republican). Two
nodes are connected if either
blog links to the other.

31
Lada A Adamic and Natalie Glance.
Another paper shows strong homophily in political blogs31 . In The political blogosphere and the
Figure 26.10 we see a visualization of how people writing online 2004 us election: divided they blog.
In Proceedings of the 3rd international
about politics connect to each other. A common political vision is the
workshop on Link discovery, pages 36–43.
clear driving force behind the creation of an hyperlink from one blog ACM, 2005
to another.
 homophily 373

Homophily arises very strongly even with mild preferences. One 32

[Link]
classical example is segregation. The famous “Parable of Polygons32 ”
starts from a simple assumption: people want to live with at least
some similar people next to them. Even if they do not seek a majority
of alike neighbors the end result is very clustered. Try to make an
experiment and set the threshold to 40%, which means that people 33
Ciro Cattuto, Wouter Van den Broeck,
are happy being in the minority. You’ll still end up with segrega- Alain Barrat, Vittoria Colizza, Jean-
tion. There’s no network in this interactive example, but one could François Pinton, and Alessandro
Vespignani. Dynamics of person-to-
easily introduce one by allowing nodes to rewire their friendship person interactions from distributed
preferences according to the same rules. Experiments building rela- rfid sensor networks. PloS one, 5(7), 2010
tion graphs via RFID tags show that these dynamics may shape the 34
Michela Del Vicario, Antonio Scala,
Guido Caldarelli, H Eugene Stanley,
topology of networks of face-to-face interactions33 . and Walter Quattrociocchi. Modeling
confirmation bias and polarization.
Scientific reports, 7:40391, 2017
Figure 26.11: Homophily driv-
ing echo chambers. Science-
oriented people (in red) rescind
connections from conspir-
(a) (b) (c) (d)
acy theorists (green) creating
communities which have no
We are venturing now in new territory. So far we have seen ho- possibility of communicating.
mophily as a constructive force, meaning that people with similar 35
Michela Del Vicario, Alessandro Bessi,
characteristics link to each other. But with segregation we’re doing Fabiana Zollo, Fabio Petroni, Antonio
Scala, Guido Caldarelli, H Eugene
something different. We’re seeing homophily as a destructive force: Stanley, and Walter Quattrociocchi. The
polygons are moving away if their expectation of uniformity isn’t met. spreading of misinformation online.
PNAS, 113(3):554–559, 2016a
In network terms, people who are connected and discover differences 36
Michela Del Vicario, Gianna Vivaldo,
in their characteristics might decide to rescind their connection. Alessandro Bessi, Fabiana Zollo,
Recently, researchers have started investigating this effect: rather Antonio Scala, Guido Caldarelli,
and Walter Quattrociocchi. Echo
than preferring to connect to similar strangers, we preferably rescind
chambers: Emotional contagion and
connections from dissimilar friends. Suppose you’re on Facebook group polarization on facebook. Scientific
and you share a lot on scientific topics. One of the members of your reports, 6:37825, 2016b
37
Eytan Bakshy, Solomon Messing,
community has outside connections, which could be convincing
and Lada A Adamic. Exposure to
them of something like anti-vaccination. This person starts sharing ideologically diverse news and opinion
anti-vax content, and the rest of the community is likely to rescind on facebook. Science, 348(6239):1130–
1132, 2015
its connections. Which ends up creating groups that cannot connect 38
Chengcheng Shao, Giovanni Luca
any more people with different ideas, and thus reinforce each other Ciampaglia, Onur Varol, Alessandro
convictions without any debate34 . Figure 26.11 shows a vignette of Flammini, and Filippo Menczer. The
spread of fake news by social bots. arXiv,
this process. pages 96–104, 2017
This is particularly problematic, as there is a large body of re- 39
Emilio Ferrara, Onur Varol, Clayton
search showing how easy it is for misinformation to spread through Davis, Filippo Menczer, and Alessandro
Flammini. The rise of social bots.
social media35 and how strong online echo chambers can be36 , 37 . In Communications of the ACM, 59(7):
fact, a sufficiently determined single actor can magnify their impact 96–104, 2016
online, as the challenge in creating and operating difficult-to-detect
40
Alessandro Bessi and Emilio Fer-
rara. Social bots distort the 2016 us
bot nets is easy to overcome38 , 39 . There is suggestive research show- presidential election online discussion.
ing how this might already have happened40 . 2016
374 the atlas for the aspiring network scientist

26.5 Summary

1. Homophily or assortativity is the tendency of nodes in a network

to connect with nodes that are similar to them in some attribute.
For instance, people tend to be friends in a social network with
other people of similar age or same race.

2. A way to study this meso scale property is by creating ego net-

works: you pick one node as ego and then you create a network
view including only its neighbors and the connections among
them.

3. There are measures to estimate attribute assortativity, usually

interpreted as a correlation coefficient taking values from +1
(perfect assortativity) to −1 (perfect disassortatvitiy).

4. Disassortativity is the opposite of assortativity: nodes tend to

connect to other nodes with different attributes from their own.
For instance, the dating network tends to be disassortative by
gender.

5. Homophily interacts with network process. Links lowering ho-

mophily connect nodes with different attributes, which can favor
information spread (the “strength of weak ties”).

6. In other cases, nodes can be fooled into seeing a minority attribute

as always the majority option in their friends: the majority illusion.

26.6 Exercises

1. Load the network at [Link]

26/1/[Link] and its corresponding node attributes at http:
//[Link]/exercises/26/1/[Link]. Iterate over
all ego networks for all nodes in the network, removing the ego
node. For each ego network, calculate the share of right-leaning
nodes. Then, calculate the average of such shares per node.

2. What is the assortativity of the leaning attribute?

3. What is the relative popularity of attribute values “right-leaning”

and “left-leaning”? Based on what you discovered in the first
exercise, would you say that there is a majority illusion in the
network?
27
Quantitative Assortativity

In the previous chapter we talked about homophily, the love of the

similar. It is our tendency of liking the people who are similar to us:
similar race, similar places we hang around, similar movies we watch.
These are all qualitative attributes. In this chapter, we make the jump
towards quantitative homophily.
Many node attributes are quantitative: age, number of friends, etc.
We can still estimate the level of homophily in a network based on
these attributes. In this case, we perform a small change in terminol-
ogy. We use the term “assortativity” instead. This change is largely
arbitrary, but can help you in differentiating between the concepts.
Just like “(qualitative) homophily = (quantitative) assortativity”, we
have “(qualitative) heterophily = (quantitative) disassortativity”.
Shifting our attention to quantitative attributes means we can use
slightly different tools to estimate homophily, since there is a clear
sorting in the attribute values and an intuition of similarity. If two
nodes of values 1 and 2 connect, it is true that they have a different
attribute value, but it still counts more towards assortativity than, say,
connecting a node of value 1 to a node of value 100.
The set of possible quantitative attributes can be vast. For this
chapter, I’m going to focus mainly on one example, which is the most
studied case: the degree. However, don’t be fooled: any technique for
the estimation of degree assortativity can be employed to estimate
any other quantitative attribute’s assortativity. However, by focusing
on the degree, I can introduce other fun assortativity-related network
effects, such as the friendship paradox.

27.1 Degree Correlations

The degree is the most studied example of assortativity because it is

directly related to the edge creating process. In a degree-assortative
network we see that hubs connect preferentially to hubs, while
peripheral nodes connect preferentially to other peripheral nodes.
376 the atlas for the aspiring network scientist

This is like a dating network, where celebrities hook up with each

other much more than you would expect if the dating network would 1
Which is clearly the only reason why
be fair1 . I’ve been unsuccessful in getting a
date with Jennifer Lawrence. No other

kv
possible explanation.
Figure 27.1: A scatter plot we
can use to visualize degree
assortativity. For each edge, we
have the degree of one node on
the x axis and of the other node
u-v on the y axis.

ku
v u

In a disassortative network, hubs connects to periphery, as it 2

Peter Uetz, Loic Giot, Gerard Cagney,
happens for instance in protein networks2 . This is more similar to Traci A Mansfield, Richard S Judson,
James R Knight, Daniel Lockshon, Vaib-
the classical preferential attachment, where newcoming low-degree
hav Narayan, Maithreyan Srinivasan,
nodes will connect more often to older and high-degree hubs. Pascale Pochart, et al. A comprehensive
A way to visualize degree assortativity is to consider each edge analysis of protein–protein interactions
in saccharomyces cerevisiae. Nature, 403
as an observation. We create a scattegram, recording the degree of (6770):623, 2000
one node on the x-axis and of the other node on the y-axis. So each
point in this scatter plot is an edge of the network. Remember that
the degree in real world networks follows a skewed distribution
spanning many orders of magnitude. So, usually, these plots will
have a log-log scale. Figure 27.1 shows the skeleton of such a scatter
plot. For an example network, such scatter would look like the one in
Figure 27.2.
Such a visualization suggests us a way to compute a possible
index of degree assortativity. This is the first of two options you have

Figure 27.2: (a) A network with

1
nodes labeled with their degree.
4
8 (b) A scatter plot we can use to
2 5 7
2
visualize (a)’s degree assorta-
2 6
# Edges (x2)

3 4 tivity. Each point is a possible

4
2
5
kv

3 4 degree combination of an edge.

3 4 3
4 2 3
The data point color tells you
2
4 3 2
3 5 1 how many edges have that
1 2 3 4 5
2
3 3 ku particolar degree combination.
Usually, this count should also
1 1
(b) be log-transformed.
(a)
 quantitative assortativity 377

if you want to quantify the network’s assortativity. You iterate over

all the edges in the network and put into two vectors the degrees of
the nodes at the two endpoints. Note that each edge contributes two
entries to this vector – unless your network is directed. So, if your
network only contains a single edge connecting nodes 1 and 2, your
two vectors are x = [k1 , k2 ] and y = [k2 , k1 ], with k v being the degree
of node v. Then, assortativity is simply the Pearson correlation
coefficient of these two vectors.
There is only one way to achieve perfect degree assortativiy. In
such a scenario, each node is connected only to nodes with the exact
same degree. This is true only in a clique. Thus, a perfectly degree
assortative network is one in which each connected component is a
clique.

avg(kN(u)) 3 4 Figure 27.3: A second strategy

to visualize degree assortativity.
The scatter plot has a point
for each node in the network,
u reporting its degree (x axis)
against the average degree of its
neighbors (y axis).

5 3
ku

Figure 27.3 shows the second strategy to estimate degree assorta-

tivity in a network. Rather that plotting each edge, we plot each node.
We compare a node’s degree with the average degree of its neighbors.
In a degree assortative network, we expect to see a positive correla-
tion: the more connections the node has, the more connections, on 3
Romualdo Pastor-Satorras, Alexei
average, its neighbors have3 . Vázquez, and Alessandro Vespignani.
Dynamical and correlation properties of
Again, you shouldn’t forget to log-transform the degree values, the internet. Physical review letters, 87(25):
given the broad degree distributions of most real world networks. 258701, 2001
This also means that you should perform a power fit. The exponent
of such a fit tells you whether the network is degree assortative (if
it’s positive), disassortative (if it’s negative), or non assortative (if it’s
statistically indistinguishable from zero).
Figure 27.4 shows few examples of assortativity in real world net-
works. Coauthorship is assortative: if I have a lot of coauthors, on
378 the atlas for the aspiring network scientist

100 1000

Figure 27.4: A collection of

assortativity plots for four
avg(kN )

avg(kN )
u

u
100
real world networks: (a) co-
authorship in scientific pub-
1 10 100
10
1 10 100 1000 10000
lishing, (b) P2P network, (c)
ku ku
Internet routers, (d) Slashdot
social network.
(a) (b)
1000 1000

100
avg(kN )

avg(kN )
u

u
100

10

1 10
1 10 100 1000 1 10 100 1000
ku ku

(c) (d)

average, they have a lot of coauthors too. Gowalla is disassortative:

users with few friends likely attach to hubs. Slashdot is still disas-
sortative, but it is the closest we could find to show what a neutral
network looks like: one where my degree doesn’t tell me anything 4
Marián Boguñá, Romualdo Pastor-
about my neighbors’ degree. Satorras, and Alessandro Vespignani.
Note that, for real world networks, we usually aggregate in the Absence of epidemic threshold in scale-
free networks with degree correlations.
same data point all nodes with the same degree. Thus what we’re Phys. Rev. Lett., 90:028701, Jan 2003. doi:
plotting is actually the average degree of all neighbors of all nodes of 10.1103/PhysRevLett.90.028701
degree k. Otherwise, we would have an unreadable cloud of points 5
Alexei Vázquez and Yamir Moreno.
Resilience to damage of graphs with
at low degree values, since most nodes in real world networks have a degree correlations. Phys. Rev. E, 67:
degree equal to one or two. 015101, Jan 2003. doi: 10.1103/Phys-
Degree assortativity is a super important property for your net- RevE.67.015101
6
F Sorrentino, M Di Bernardo,
work. Degree correlations radically change many network dynam- G Huerta Cuellar, and S Boccaletti.
ics4 , 5 , 6 , 7 . This is especially true when you have multilayer networks Synchronization in weighted scale-free
and we’re looking at assortativity inter-layer besides intra-layer8 . networks with degree–degree correla-
tion. Physica D: nonlinear phenomena, 224
Meaning: if I am a hub in one layer, am I also a hub in the other (1-2):123–129, 2006
layers? We touched the topic in Section 19.4. If the answer to this 7
Márton Pósfai, Yang-Yu Liu, Jean-
Jacques Slotine, and Albert-László
question is yes, failure-resistant layers become failure-prone9 , 10 .
Barabási. Effect of correlations on
There is a curious tension between degree assortativity and other network controllability. Scientific reports,
common statistical properties of real world networks. For instance, 3:1067, 2013
8
Zhen Wang, Lin Wang, Attila Szolnoki,
we just saw that scientific collaboration is an assortative network.
and Matjaž Perc. Evolutionary games
However, we also know it has a broad degree distribution. on multilayer networks: a colloquium.
The two properties clash against each other: assortativity means The European physical journal B, 88(5):124,
2015b
that hubs connect to hubs, but in a network with a heavy-tailed 9
Jianxi Gao, Sergey V Buldyrev, Shlomo
degree distribution there are few huge hubs and many one-degree Havlin, and H Eugene Stanley. Robust-
nodes. The likelihood of connecting a hub to many small nodes ness of a network of networks. Phys.
Rev. Lett., 107:195701, Nov 2011. doi:
seems too high. In fact, if we were to generate a random version of 10.1103/PhysRevLett.107.195701
 quantitative assortativity 379

the co-authorship network respecting its degree distribution – for 10

Jia Shao, Sergey V Buldyrev, Shlomo
instance via a configuration model (Section 15.1) –, we would obtain Havlin, and H Eugene Stanley. Cascade
of failures in coupled network systems
a degree disassortative network11 . This makes quite interesting with multiple support-dependence
networks that both have a skewed degree distribution and are degree relations. Phys. Rev. E, 83:036116, Mar
2011. doi: 10.1103/PhysRevE.83.036116
assortative! They have some non-trivial machinery driving their 11
Marián Boguná, Romualdo Pastor-
nodes’ connections that cannot be captured by simple models. Satorras, and Alessandro Vespignani.
The network generators I discussed in Part IV weren’t really de- Cut-offs and finite size effects in scale-
free networks. The European Physical
signed with assortativity in mind. As we just saw, the configuration Journal B, 38(2):205–209, 2004
model is naturally disassortative, as is preferential attachment and
anything which imposes a power law degree distribution. By defini-
tion, random graphs such as Gn,p are non-assortative.
However, if you start with any synthetic network, there are al-
gorithms to rewire the edges such that the degree distribution will
be preserved, but you will obtain an assortative (or disassortative) 12
Marián Boguñá and Romualdo Pastor-
network12 , 13 , 14 . Satorras. Class of correlated random
This happens through edge swap, as Figure 27.5 shows. First, you networks with hidden variables. Phys.
Rev. E, 68:036112, Sep 2003. doi:
select two connected node pairs. Then you sort them according to 10.1103/PhysRevE.68.036112
their degree, in the figure the order is nodes 4, 2, 3, 1 (k4 = 6, k2 = 3, 13
Ramon Xulvi-Brunet and Igor M
k3 = 2, k1 = 1). The next move depends on whether you want to Sokolov. Reshuffling scale-free networks:
From random to assortative. Physical
induce assortativity or disassortativity. In the first case, you connect Review E, 70(6):066102, 2004
the two nodes with the highest degree to each other, and the two 14
R. Xulvi-Brunet and I. M. Sokolov.
with lowest degree to each other (Figure 27.5(b)). In the second case, Changing Correlations in Networks:
Assortativity and Dissortativity. Acta
you do the opposite: connect the highest degree node with the lowest, Physica Polonica B, 36:1431, 5 2005
and the two middle ones (Figure 27.5(c)).

1 1 1
Figure 27.5: The
2 2 2
(dis)assortativity inducing
4 3 4 3 4 3 model. (a) Select two pairs
of connected nodes (in green
the edges we select). (b) As-
sortativity inducing move. (c)
Disassortativity inducing move.
(a) (b) (c)
Note that this swap doesn’t always change the topology nor
alter the characteristics of the network. For instance, if all nodes
have the same degree, the move would not affect assortativity. But,
after enough trials in a large enough network, you’ll see that these
operations will have the desired effect.
Degree assortativity, as I discussed it so far, is defined for undi-
rected networks. There are straightforward extensions for directed 15
Jacob G Foster, David V Foster, Peter
networks15 . The standard strategy is to look at four correlation coeffi- Grassberger, and Maya Paczuski. Edge
cients: in-degree with in-degree, in-degree with out-degree (and vice direction and the structure of networks.
Proceedings of the National Academy of
versa), and out-degree with out-degree.
Sciences, 107(24):10815–10820, 2010
Obviously, everything I wrote so far on degree assortativity also
380 the atlas for the aspiring network scientist

applies to any other quantitative attribute you might have on your

nodes. For instance, if you have a social network, it applies to a
person’s age, height, weight, and so on. You can build the scatter-
grams and calculate the best fit to figure out if your social circle sort
themselves according to their height or income.

27.2 Friendship Paradox

You might want to make a double take on Figure 27.4, because it

contains a not-so-obvious but intriguing – and enraging – message.
In Figure 27.6 I focus on one of those assortativity plots, the one
about scientific collaborations. I add an additional line to the plot:
the identity line. This line runs through the part of the plane where
the x axis and the y axis have the same value.

Way more nodes here than here

100 Figure 27.6: The friendship

paradox. The plot shows a
I have fewer
friends than my node’s degree (x axis) against
average neighbor
the average degree of its neigh-
avg(kN )
u

bors (y axis). The blue line

is the best fit, while the gray
line is the identity line. Nodes
I have more
friends than my above the identity line have
average neighbor fewer friends than their friends’
1 10 100 average.
ku
Identity Line

In other words, the identity line divides the space in two. Above
the identity line we have all the nodes for which, on average, the
neighbor degree is higher than the node’s degree. Below the identity
line it’s the opposite: the node’s degree is higher than the neighbors
degree.
At first glance, the situation seems balanced. There are as many
points above the identity line as there are below. However, remember
that we’re aggregating all nodes with the same degree value in a sin- 16
Scott L Feld. Why your friends have
gle point. We know that the degree has a broad distribution, because more friends than you do. American
Journal of Sociology, 96(6):1464–1477,
we visualized it for the co-authorship network before. Therefore,
1991
there are way more nodes above the identity line than below. 17
Ezra W Zuckerman and John T Jost.
That’s the friendship paradox: your friends are, on average, more What makes you think you’re so pop-
ular? self-evaluation maintenance and
popular than you16 , 17 ! This means that, for the average node, its
the subjective side of the" friendship
degree is lower than the average degree of their neighbors. This is paradox". Social Psychology Quarterly,
actually pretty obvious once you think about it: a node with degree pages 207–223, 2001
 quantitative assortativity 381

k appears in k other nodes’ averages, and hence is “over-counted”

by an amount equal to how much larger it is than the network’s
mean degree. A high degree node appears in many more node
neighborhoods than does a low degree node, and hence it skews
many local averages. The only way to escape such paradox is by
having a network whose degree distributes mostly regularly: for
instance small-world networks (Section 14.2) are usually immune to
the friendship paradox because most nodes have the same degree
(the probability of rewiring is low).
The friendship paradox sounds pretty depressing, but we actually
already made use of it in a rather uplifting scenario. The effective
“vaccinate-a-friend” scheme I discussed in Section 18.3 is nothing else
than a practical application of this network property.

27.3 Distribution of Quantitative Attributes

Quantitative assortativity does not stop at the degree. There are

examples of papers analyzing quantitative attributes discovering all
sort of interesting phenomena. I’m going to give you one example I 18
Mauro Barone and Michele Coscia.
know well, as it came from a paper I wrote18 . Birds of a feather scam together: Trust-
In the paper, I analyze a business to business network, connecting worthiness homophily in a business
network. Social Networks, 54:228 – 237,
businesses if they are customers or suppliers of each other. Each 2018. ISSN 0378-8733
edge is a B2B transaction and both businesses have to report it, as
Figure 27.7 shows. Thus, if they report a different amount, we know
someone is lying. I create a measure of trustworthiness, which is
the average value of mismatch a business has, weighted by how
trustworthy its neighbors are.

80
Figure 27.7: The data model
100 of the business to business net-
work. The edge color tells corre-

100
sponds to the node making the
claim about the transaction. For

90
instance, the green node reports
selling 90 to the red node and
buying 80 back.
This trustworthiness score is a quantitative attribute. It is strongly
correlated with the likelihood that the business was in fact cheating
on their taxes, as I have information whether the audited businesses
were fined and, if they were, how much they had to pay.
Simulations show that, with this correction, in a randomly wired
network the score should be disassortative. Instead, in the real ob-
served network, the score is assortative. Figure 27.8 shows the re-
lation: there are more nodes above the identity line than below – it
382 the atlas for the aspiring network scientist

100
Figure 27.8: The average trust-

Avg Non-Neigh Trust Diff

1000
worthiness score difference
10-1 between neighbors (x axis) and

# Nodes
100
non-neighbors (y axis). The
blue line shows the identity,
10-2 and the point color the number
10
of nodes with the given value
combination.
10-3 -3 1
10 10-2 10-1 100
Avg Neigh Trust Diff

might appear that the opposite is true, but you need to take into ac-
count the color of the dots. Being above the identity line means that
the trust score difference between neighbors is lower than between
non-neighbors: a sign of assortativity.
Given the connection with the actual tax fines, the assortativity
analysis can make us conclude something tangible about business
connections. In this case, that fraudulent and untrustworthy busi-
nesses band together. If I know your customer/suppliers are scam-
mers, I should update my priors on whether you’re a scammer as
well.
A more famous example of quantitative attribute assortativity is
related to the friendship paradox that I just described in the previous
section, and it is ten times more enraging. We humans are social
animals. Notwithstanding introverted cavemen like myself, in general
our level of happiness is correlated with the number of friends we
have. In fact, just like the degree, happiness is assortative in social 19
Johan Bollen, Bruno Gonçalves,
networks: happy people tend to befriend each other19 . Guangchen Ruan, and Huina Mao.
What I just stated is that more friends imply more happiness. And Happiness is assortative in online social
networks. Artificial life, 17(3):237–251,
the friendship paradox tells us that our friends have more friends
2011
than us. Do I mean to tell that, like with friendship, there is also a 20
Johan Bollen, Bruno Gonçalves, Ingrid
happiness paradox? Why, yes there is20 . If you ask people about van de Leemput, and Guangchen Ruan.
their level of happiness in a social network, you will find out that the The happiness paradox: your friends
are happier than you. EPJ Data Science, 6
average happiness level of one’s friends tends to be higher than their
(1):4, 2017
own happiness level. That is probably why you think everyone is
having such a great time on social media. Everyone but you. It’s not
you, it’s the system. Luckily, the researchers behind this discovery
have a few guidelines on how to unplug from social media toxicity 21
Johan Bollen and Bruno Gonçalves.
and live a more fulfilling life21 . Network happiness: How online social
In general, everything that correlates with degree – be it happiness, interactions relate to our well being. In
Complex Spreading Phenomena in Social
income, or tax fraud – will get its own paradox for free.
Systems, pages 257–268. Springer, 2018
 quantitative assortativity 383

27.4 Summary

1. Assortativity works not only on qualitative, but also on quanti-

tative attributes. The most studied case is degree assortativity:
the tendency of high degree nodes to connect to other high de-
gree nodes (degree assortative) or to low degree nodes (degree
disassortative).

2. You can calculate degree assortativity – or any quantitative assor-

tativity – by correlating the attribute values at the endpoints of
each edge. Alternatively, you can correlate a node’s attribute value
with the average of their neighbors.

3. Graph generators usually are unable to provide degree assortative

networks, but there are postprocessing techniques that can induce
either degree assortativity or degree disassortativity.

4. By a cruel mathematical property of degree distributions, social

systems are affected by the friendship paradox: the average person
has fewer friends than their friends on average.

5. Even crueler, since in social system happiness is usually correlated

with the number of friends a person has, the friendship paradox
also implies than the average person is less happy than their
friends on average. So it’s not just you.

27.5 Exercises

1. Draw the degree assortativity plots of the network at http://

[Link]/exercises/27/1/[Link] using the first
(edge-centric) and the second (node-centric) strategies explained
in Section 27.1. For best results, use logarithmic axes and color the
points proportionally to the logarithmic count of the observations
with the same values.

2. Calculate the degree assortativity of the network from the previ-

ous question using the first (edge-centric Pearson correlation) and
the second (node-centric power fit) strategies explained in Section
27.1.

3. Prove whether the network from the previous questions is affected

or not by the friendship paradox.
28
Core-Periphery

When you obtain a new network dataset and you plot it for the
first time, in the vast majority of cases you will see a blobbed mess.
This is usually due to the fact that raw network data is usually a
hairball, and you need to backbone it, or perform other data cleaning
tasks, as I detailed in Part VII. However, in some cases, there is an
unobjectionable truth. It might be that, deep down, your network
really is a hairball.
Many large scale networks have a common topology: a very
densely connected set of core nodes, and a bunch of casual nodes at-
taching only to few neighbors. This should not be surprising. If you
create a network with a configuration model and you have a broad
degree distribution, the high degree nodes have a high probability
of connecting to each other – see Section 15.1. The surprising part
is that the cores of some empirical networks are even denser than
what you’d anticipate by looking at the degree distribution of the 1
Shi Zhou and Raúl J Mondragón. The
network1 ! rich-club phenomenon in the internet
topology. IEEE Communications Letters, 8
Since everything that departs from null expectation is interesting,
(3):180–182, 2004
this phenomenon in real world networks has attracted the attention
2
Petter Holme. Core-periphery orga-
of network scientists. They gave a couple of names to this special
nization of complex networks. Phys.
meso-scale organization of networks: core-periphery2 , 3 , with the core Rev. E, 72:046111, Oct 2005. doi:
sometimes dubbed as “rich club”4 . 10.1103/PhysRevE.72.046111

We already saw the concept of core and periphery when we dis-

3
Peter Csermely, András London, Ling-
Yun Wu, and Brian Uzzi. Structure and
cussed node roles and k-core centrality in Section 11.7. Such method dynamics of core/periphery networks.
is not included here because it finds a fundamentally different type Journal of Complex Networks, 1(2):93–123,
2013b
of core-periphery structure, which is more similar to a hierarchi- 4
Vittoria Colizza, Alessandro Flammini,
cal decomposition of the network estimating the centrality of the M Angeles Serrano, and Alessandro
nodes5 . In this chapter I discuss ways in which you can detect a Vespignani. Detecting rich-club ordering
in complex networks. Nature physics, 2
core-periphery structure that is less hierarchical and more “hub- (2):110–115, 2006b
and-spoke”, in which we want to determine which nodes are in 5
Ryan J Gallagher, Jean-Gabriel Young,
the core and which others are in the periphery. I also discuss the and Brooke Foucault Welles. A clarified
typology of core-periphery structure in
tension between the ubiquity of core-periphery structures and the networks. arXiv preprint arXiv:2005.10191,
equally common and (only) apparently contradictory presence of 2020
 core-periphery 385

communities in complex networks. Finally, I’ll connect core-periphery

structures with a few real world dynamics that might be able to
generate them.

28.1 Models

There are many ways to extract core-periphery structures from

your networks. However, two methods dominate in the literature,
especially in sociology. These are the discrete and the continuous 6
Stephen P Borgatti and Martin G
Everett. Models of core/periphery
model6 .
structures. Social Networks, 21(4):
375 – 395, 2000. ISSN 0378-8733.
doi: [Link]
Discrete 8733(99)00019-2

Core-periphery networks emerge when all nodes belong to a single

group. Some nodes are well connected while others, while still
being part of that group, are not. In a pure idealized core-periphery
network the nodes can be classified strictly into two classes. The
core nodes are the one with a high degree of interconnectedness.
The periphery nodes are the rest, the ones that are only sparsely
connected in the network.

C p Figure 28.1: A toy example of

the Discrete Model for core-
periphery structures. This
adjacency matrix shows, high-

p Nothing
here
lighted in blue, a dense area of
the network with many connec-
tions. In green, a sparser area:
the periphery. Connections only
go to (or from) a core member,
There can be only two types of connections: between core nodes meaning that in the main diago-
– which is the most common edge type, since the core is densely nal in the peripheral area there
connected – and between a core-periphery pair. Peripheral nodes do are no entries larger than zero.
not connect to each other. In the adjacency matrix, which I show in
Figure 28.1, there’s a big area with no connections. This is known
as the “Discrete Model”. It is a very strict one, and rarely real world
networks comply with this standard. A perfect discrete model in
which the core is composed by a single node is a star.
If you want to detect the core-periphery structure using the dis-
crete model, you have a simple quality measure you want to maxi-
mize. This is ∑ Aij ∆uv , with A being the adjacency matrix, and ∆ a
uv
matrix with a value per node pair. An entry in ∆ is equal to one if
either of the two nodes is part of the core.
386 the atlas for the aspiring network scientist

Since A is immutable, your quest is to find the best ∆ such that

the sum is maximized, i.e. you are capturing all edges established
between a core node and all other nodes in the network. In a network
following the perfect discrete model, this sum is equal to | E| the
number of edges in a network.
Of course, you cannot compare the “coreness” of two different
networks unless they have the same number of nodes and edges,
because this measure will take different expected values. That is
why, sometimes, you can simply calculate the Pearson correlation
coefficient between A and ∆.
Without going into details of specialized algorithms, one can find
the best ∆ using classical randomization algorithms. Options are
genetic algorithms, simulated annealing, or basin hopping.

Continuous
Reality rarely conforms with strict expectations. Having only two
classes in which to put nodes is exceedingly restrictive. What if nodes
can be sorted in three classes? What if a semi-periphery exists? This
is an enticing opportunity, until you realize that you could also ask:
why three classes? Why not four? Why not five? Why not... you get
the idea.

Figure 28.2: A toy example

of the Continuous Model for
core-periphery structures.
This network shows a densely
connected core (blue), a pure
periphery whose nodes do not
connect to each other (green),
and an intermediate stage
which is not dense enough to
be part of the core, but whose
nodes still connect to each other
(purple).
This is where the Continuous Model comes into play. Looser than
the Discrete Model, in the Continuous Model we have an arbitrary
number of classes for the nodes beyond the core. The intermediate
classes can interconnect with each other and with the periphery
proper, just in a looser way than the core proper. We show an exam-
ple in Figure 28.2.
To be more precise, rather than creating an arbitrary number of
classes, we assign to each node a “coreness” value. Mathematically
speaking, the difference with the discrete model is tiny. The quality
 core-periphery 387

measure we want to optimize is still ∑ Auv ∆uv . However, now the

uv
entries of ∆ are not binary any more. Instead we have ∆uv = cu cv ,
with cu being the coreness value of node u.
For instance, we could say that, in Figure 28.2, the nodes in the
blue circle have coreness equal to 1, the ones in purple have coreness
equal to 0.5, and the rest has coreness equal to 0. If you force nodes
to have a coreness of either 1 or 0, you’re back in the discrete model.
How could one establish the cu values for the Continuous Model?
I already mentioned the k-core centrality algorithm from Section 11.7
in this chapter. This could be a good choice, because we know that
nodes with low k-core centrality have a low degree, while nodes with
a high k-core value tend to connect to each other in a core.
Of course, that is an a priori approach: it fixes your ∆, which may
or may not fit your data well. The alternative is to use a technique
similar to the ones I mentioned before, to build your ∆ via the c
vector in such a way that your quality measure is maximized.

Other Approaches
The continuous model is powerful, but it doesn’t really tell you much 7
M Puck Rombach, Mason A Porter,
on how you should build your ci values. Rombach et al.7 propose James H Fowler, and Peter J Mucha.
a way to build such vector, introducing two parameters, α and β. β Core-periphery structure in networks.
SIAM Journal on Applied mathematics, 74
determines the size of the core, from the entirety of the network to
(1):167–190, 2014
an empty core. α regulates the c score difference between the core
classes. If a node u at a specific core level has a score cu , the node v
at the closest highest core level will have cv = α + cu – or, really, any
function taking α as a parameter.

Figure 28.3: Some matrix masks

you can use to build ∆. The
darkness of the color is directly
proportional to how much
the core value combination
(a) ∆uv = cu cv (b) ∆uv = cu + cv
p
(c) ∆uv = 5
c5u + c5v contributes to ∆. Matrices are
Another freedom you can take is to build ∆ differently. In the stan- sorted so that the top-left corner
dard continuous model, we build it via multiplication: ∆uvq= cu cv . shows the value for cu = cv = 1
p p
An alternative is to build it via a p-normalization: ∆uv = cu + cv .
p
and the bottom-right corner
The higher p, the more weight you’re putting into a classic discrete shows the value for cu = cv = 0.
model core. Figure 28.3 shows the different effects of different criteria
to build ∆.
Other approaches in the literature make use of the Expectation
8
Xiao Zhang, Travis Martin, and
Mazimization or the Belief Propagation algorithms8 .
Mark EJ Newman. Identification of
All methods discussed so far detect the core via a statistical infer- core-periphery structure in networks.
ence or the development of a null model. Thus there are issues of Physical Review E, 91(3):032803, 2015
388 the atlas for the aspiring network scientist

scalability when you need to infer the parameters of the model to

make the proper inference. Alternative methods exploit the fact that
the core is densely connected and nodes have a high degree. Thus,
the expectation is that a random walker would be trapped for a long 9
Athen Ma and Raúl J Mondragón.
time in the core9 . By analyzing the behavior of a random walker, one Rich-cores in networks. PloS one, 10(3):
could detect the boundary of the core. e0119678, 2015
What about multilayer networks (Section 4.2)? Does it make sense
to talk about a core in a network spanning multiple interconnected
layers? The analysis of some naturally occurring multilayer networks, 10
Federico Battiston, Jeremy Guillon,
for instance the brain connectome10 , 11 , suggest that it is. However, I Mario Chavez, Vito Latora, and Fabrizio
would say that at the moment of writing this paragraph, a systematic De Vico Fallani. Multiplex core–
periphery organization of the human
investigation of core-periphery in multilayer networks, along with
connectome. Journal of the Royal Society
a general method to detect them, is an open problem in network Interface, 15(146):20180514, 2018
science. 11
Jeremy Guillon, Mario Chavez, Fed-
erico Battiston, Yohan Attal, Valentina
La Corte, Michel Thiebaut de Schotten,
28.2 Tension with Communities Bruno Dubois, Denis Schwartz, Olivier
Colliot, and Fabrizio De Vico Fallani.
Disrupted core-periphery structure
There is a tension between core-periphery structures (CP) and the of multimodal brain networks in
classical community discovery (CD) assumption: in CP there isn’t alzheimer’s disease. Network Neuro-
science, 3(2):635–652, 2019
space for communities, given that there’s only one dense area and
everything connects to it. In CD, there’s little space for peripheries,
and there are multiple cores.
You can see this mathematically. For simplicity, let’s consider the
discrete model: ∑ Aij ∆uv . There is a strong correlation between being
uv
an high degree node and being in the core: after all, the nodes in
the core are highly connected. We’ll see that a community is a set of
nodes densely connected to each other. Thus, nodes in a community
have a relatively high degree and should be considered part of the
core. So, for all nodes deeply embedded in a community, ∆uv = 1.

Figure 28.4: (a) A classical

core-periphery structure. (b) A
classical community structure.

(a)
(b)
However, a traditional community is also sparsely connected to
nodes outside the community. This means that, if nodes u and v
are in different communities, likely Auv = 0. But we just saw that
their ∆uv should be 1 because they have high degree! All of that
score is wasted! Maximizing the quality function would imply to
 core-periphery 389

put all nodes in the same core, sacrificing the defining characteristic
of a core: the fact that nodes in it should tend to connect to each
other. Figure 28.4 shows the difference between the two archetypal
meso-scale organizations.
This is problematic since we have evidence that core-periphery
structures are ubiquitous, and so are communities. There are a
couple of explanations we can use to restore our sanity.
The first explanation is realizing that every network lives on a
core-periphery to community structure continuum. The real world
networks we observe distribute through this continuum in such a
way that perfect instances are extremely rare – as Figure 28.5 shows.
You’ll very rarely find a natural discrete model, exactly as rarely
as finding a real world network organizing like a caveman graph
(Section 14.1) – the quintessential community structure. As a conse-
quence, one way to solve this conundrum is to admit that a network
might have multiple cores. Thus, one first performs community dis-
covery to find the multiple cores and then applies the core-periphery 12
Sadamori Kojaku and Naoki Masuda.
detection algorithm independently on each community12 . Core-periphery structure requires
something else in the network. New
Journal of Physics, 20(4):043012, 2018
# Networks
The blend is always different! Figure 28.5: The number of
networks with a pure core-
periphery network and with a
pure community structure is
actually tiny.

Almost never Almost never

In most cases, networks are in a middle way. Another important

thing to keep in mind is that this continuum is not monodimensional
– although on this page I had to squeeze it on a line. The way in
which each network blends core-peripheries with communities is
always different and difficult to fully characterize.
There are many mechanisms that makes this the case. One is
related to overlapping communities (Chapter 34). These communities
allow nodes to belong to multiple groups at the same time. These
nodes in between communities can be considered a special core of
390 the atlas for the aspiring network scientist

13
Jaewon Yang and Jure Leskovec.
the network13 , bringing together the community structure with the Overlapping communities explain core–
core-periphery organization. periphery organization of networks.
Proceedings of the IEEE, 102(12):1892–
Alternative explanations use the power of random walkers to ex-
1902, 2014
plain core-peripheries14 , an approach that is also commonly used in 14
Fabio Della Rossa, Fabio Dercole, and
community discovery. Other models attempt to embed nodes into a Carlo Piccardi. Profiling core-periphery
network structure by random walkers.
spatial dimentions, showing how this can create core-periphery struc- Scientific reports, 3:1467, 2013
tures15 . It is following this example that I’ll try to draw a connection 15
Daniel A Hojman and Adam Szeidl.
between core-periphery and some well studied aspects of economics Core and periphery in networks. Journal
of Economic Theory, 139(1):295–309, 2008
in the next section.

28.3 Emergence from Social Behavior

The emergence of core-periphery structures can be observed in many

systems. I’ll make one example from economic geography. 16
Harold Hotelling. Stability in competi-
Consider Hotelling’s law16 . Suppose that you are on a beach tion. The Economic Journal, 39(153):41–57,
with two ice cream vendors of equal quality. People want ice cream 1929

and, since the two vendors offer the same quality, the customers
will go to the closest vendor. Therefore, a rational vendor would
move their stand so that it can capture the people in the middle. The
other vendor would do the same. The solution is an equilibrium in
which vendors concentrate in the middle, even though that means
increasing the walk length for every customer.

(a) t = 1 (b) t = 2 (c) t = 3 (d) t = 4

Figure 28.6: A depiction of
Figure 28.6 captures the law in all of its enraging, negative-sum,
Hotelling’s Law. Each customer
glory. The situation hardly changed for the businesses, while the
(circle) walks to the closest
customers are served less efficiently.
ice-cream stand. As times goes
This is observed in reality, at multiple levels. Consider a federal
by, the stands get closer and
government structure, where state governments coalesce into state
closer, to capture the people in
capitals that need to be visited by their peripheries. In turn, federal
the middle.
capitals provide even higher level services and thus attract people
from each state. 17
Constantinos Apostolos Doxiadis et al.
This is related to ekistics, the science of human settlements17 . In Ekistics; an introduction to the science
of human settlements. 1968
ekistics, Doxiadis shows how improvements in human transporta-
tion have introduced a cumulative advantage for the best connected
city centers. These are swallowing up their surroundings, sparsify-
ing periphery-periphery connections and attracting all wealth and
connections for themselves, creating a centralized rich club.
If we look at the types of networks generated with such theory
in mind – for instance Figure 28.7 –, we realize they do not conform
 core-periphery 391

Figure 28.7: A toy example of

a network built accordingly to
Doxiadis’ ekistics. (Left) Graph
view: the most central 5-clique
is the union of 5 settlements
that make arise the best con-
nected metropolis. The smaller
3-cliques represent other com-
peting cities connecting to their
(a) (b) peripheries, which are bound to
be eventually absorbed by the
very much to the strict core-periphery structure imposed by the core. (Right) Adjacency matrix
Discrete Model, and they are not quite the same as the Continuous view.
Model, due to their multi-core nature.
However, their schematic structure brings us some sort of closure
when it comes to the tension between core-periphery and commu-
nity discovery. The networks do have a core, and also some sort of
communities, coalescing around the secondary cores. Figure 28.8
shows a possible model representing a temporary core-periphery
organization – which ekistics predicts to eventually collapse into a
single core. These networks are very similar to the Kronecker graphs
we saw in Section 15.2, in all their fractal glory.

Figure 28.8: Two schematic

representations of ideal core-
periphery structures: the dis-
Communities? crete model on the left and the
ekistics model on the right. The
blue area represents a densely
connected set of nodes, while
the green area is a sparser
periphery.
Discrete Model "Ekistics" Model
18
Walter Christaller. Central places in
The Central Place Theory in economic geography18 is an alterna- southern Germany. Prentice Hall, 1966

tive and less pessimistic interpretation of geographical core-periphery

structures. It says that settlements function as “central places” pro-
viding services to surrounding areas. The more sophisticated the
service the harder it is to provide, and thus it requires more skill
integration and thus more centralization. If we keep centralizing
sophisticated services, we end up with a high-level central core, sev-
eral secondary lower level cores and various peripheries, in a fractal
way. However, the conceptual jump to core-periphery structures is
392 the atlas for the aspiring network scientist

larger here, because to explain central place we need to introduce

this service-providing system – with sophistication and skills –, while
ekistics uses the simpler cumulative advantage mechanics that we
already know are a part of many networked systems.

28.4 Nestedness

Core-periphery structures are a generalization of a specific meso-

scale organization of complex systems that is relevant in multiple 19
Sang Hoon Lee et al. Network nest-
fields: nestedness19 . A nested system is one where the elements edness as generalized core-periphery
containing few items only contain a subset of the items of elements structures. Physical Review E, 93(2):
022306, 2016
with more items.
In terms of networks, an ideal nested system has a hub which is
connected to all nodes in the network. The node with the second
largest degree is connected to a subset of the neighbors of a hub. 20
Jordi Bascompte, Pedro Jordano,
The third largest degree node is connected, in turn, to a subset of Carlos J Melián, and Jens M Olesen.
The nested assembly of plant–animal
the neighbors of the second node. It rarely connects to nodes not mutualistic networks. Proceedings of the
connected to its antecedent in this hierarchy20 . National Academy of Sciences, 100(16):
9383–9387, 2003
Nestedness was originally developed in ecology studies21 , 22 , 23 21
Robert H Mac Arthur and Edward Os-
– specifically biogeography. It originated from an observation of bornecoaut Wilson. The theory of island
related ecosystems. Suppose you have a mainland with a set of biogeography. Technical report, 1967
species. Then you have an archipelago, with many islands at an
22
Bruce D Patterson. The principle of
nested subsets and its implications for
increasing distance from the coast. The closest island contains all biological conservation. Conservation
the species able to cross water. The second island contains species Biology, 1(4):323–334, 1987
that can cross water and have a slightly higher range. The third
23
Ugo Bastolla, Miguel A Fortuna,
Alberto Pascual-García, Antonio Ferrera,
island contains only species with a further increased range, and so on. Bartolo Luque, and Jordi Bascompte.
Clearly, the species which did not make it to the second island are The architecture of mutualistic net-
works minimizes competition and
unable to get to the third. Thus the third island is a perfect subset of increases biodiversity. Nature, 458(7241):
the second. 1018, 2009
I should point out that this explanation of nestedness is not the 24
Claudia Payrató-Borras, Laura
Hernández, and Yamir Moreno. Break-
only one in literature: other authors suggest that nestedness could ing the spell of nestedness: The entropic
arise simply from the degree distribution24 , and that there are fewer origin of nestedness in mutualistic
nested system than we originally thought. systems. Physical Review X, 9(3):031024,
2019
A typical nested network is bipartite. One node type, in our ecol- 25
Wirt Atmar and Bruce D Patterson.
ogy case, is the species, and the other is the ecosystem. If you were The measure of order and disorder
to plot the adjacency matrix of such network, you’d end up with a in the distribution of species in frag-
mented habitat. Oecologia, 96(3):373–382,
picture that looks like Figure 28.9. To highlight the nested pattern, 1993
you want to plot the matrix sorting rows and columns by their sum 26
Paulo R Guimaraes Jr and Paulo
in descending order. If you do so, most of the connections end up in Guimaraes. Improving the analyses of
nestedness for large sets of matrices.
the top-left of the matrix. That is why we often call nested matrices Environmental Modelling & Software, 21
“upper-triangular”. (10):1512–1513, 2006
Just like in the discrete model, it’s also rare for a real world net-
27
Samuel Jonhson, Virginia Domínguez-
García, and Miguel A Muñoz. Factors
work to be perfectly nested. Thus, researchers developed methods determining nestedness in complex
to calculate the degree of nestedness of a matrix25 , 26 , 27 . I’m going to networks. PloS one, 8(9):e74025, 2013
 core-periphery 393

Figure 28.9: A matrix view of

a nested network. The matrix
has species on the columns and
ecosystems on the rows.

give you a highly simplified view of the field.

These measures are usually based on the concept of temperature,
using an analogy from physics. A perfectly nested matrix has a
temperature of zero, because all the “particles” (the ones in the
matrix) are frozen where they should be: in the upper-triangular
portion. Every particle shooting outside its designated spot increases
the temperature of the matrix, until you reach a fully random matrix
which has a temperature of 100 degrees.
A key concept you need to calculate a matrix’s temperature is the
isocline. The isocline is the ideal line separating the ones from the
zeroes in the matrix. You should try to draw a line such that most
of the ones are on one side and most of the zeros are on the other.
Usually, there are two ways to go about it.
The parametric way is to figure out which function best describes
the curve in your upper triangular matrix: a straight line, a parabolic,
a hyperbolic curve, the ubiquitous and mighty power-law. Then you
fit the parameters so that the isocline snugs as close as possible to
said border.
The non-parametric way is to simply create a jagged line following
the row sum or the column sum. If your row (or column) sums to 50,
then you expect a perfectly nested matrix to have 50 ones followed
only by zeros. So your isocline should pass through that point.
Once you have your isocline, you can simply calculate how many
mistakes you made: how many ones are on the side of the zeroes,
and vice versa? Figure 28.10 shows an example of the different levels
of nestedness two toy matrices can have.
I’m mentioning nestedness in this book because I have encoun-
tered it a suprisingly high amount of times in my research, in com-
plex systems that have nothing to do with ecology – economics for
instance.
394 the atlas for the aspiring network scientist

Figure 28.10: Two matrices at a

different level of nestedness: (a)
high nestedness (low tempera-
ture), (b) low nestedness (high
temperature). The line in blue is
the best isocline.

(a) (b)

Some colleagues built a bipartite network connecting countries to 28

Sebastián Bustos, Charles Gomez,
the products they can export successfully in the global market28 , 29 . Ricardo Hausmann, and César A
The most competitive export-oriented economies (Japan, Germany, Hidalgo. The dynamics of nestedness
predicts the evolution of industrial
...) are able to have an export edge in almost any product, no matter
ecosystems. PloS one, 7(11):e49393, 2012
how complex. As you go down the ladder of competitiveness, the 29
Matthieu Cristelli, Andrea Tacchella,
countries are able to export a subset of what their immediate higher and Luciano Pietronero. The het-
erogeneous dynamics of economic
ranked country can. When you get to the least diversified economies
complexity. PloS one, 10(2):e0117174,
in the world, you end up with countries that are able to export only 2015
the products that every country in the world can make.
I personally found nested patterns in a supermarket matrix, with 30
Diego Pennacchioli, Michele Coscia,
customers and products as the two node types30 . A connection goes Salvatore Rinzivillo, Fosca Giannotti,
from a customer to the product they bought in significant quantities. and Dino Pedreschi. The retail market as
a complex system. EPJ Data Science, 3(1):
Nestedness emerges as there are different customer types, organized
33, 2014
in a continuum: from those who buy everything they need in the
shop we studied, to those who only buy immediate necessities.
Note that I constantly used adjacency matrix pictures to convey
the ideas behind core-peripehry and communities – from Figure
28.1 to Figure 28.10. This is because one could unify core-peripehry
and communities in a single model using a mixing matrix, which
is at the basis of using stochastic blockmodels (Section 15.2) to find
communities (Section 31.1) and/or cores in networks.

28.5 Summary

1. A core-periphery structure is a meso-level organization of a com-

plex network in two parts: one set of nodes densely interconnected
with each other (the core) and a sparsely connected set of nodes
with just one or few connections per node to the core (the periph-
ery).

2. The discrete model allows to detect such structures by penalizing

periphery-periphery connections. Other approaches, such as the
continuous model, are more flexible and allow for varying degrees
of “coreness”.
 core-periphery 395

3. Core-periphery structures are ubiquitous just as community

structures are, yet a pure core-periphery structure is incompatible
with the general notion of community. In reality, these two meso-
scale organizations of complex networks co-exist on a spectrum.

4. Many real world dynamics may be at the basis of a core-periphery

structure. For instance, geographical agglomeration – i.e. the
creation of a localized core – makes sense when combining skills
to provide complex services in an economy.

5. Nestedness in ecology and economics is another classical core-

periphery structure for bipartite networks. In an archipelago,
islands with the most species have all species, while islands with
few species only have the species that are present in all islands.

28.6 Exercises

1. Install the cpalgorithm library (sudo pip install cpalgorithm)

and use it to find the core of the network using the discrete model
([Link]) and Rombach’s model ([Link]) on the network
at [Link] Use
the default parameter values. (Warning, Rombach’s method will
take a while) Assume that Rombach’s method puts in the core all
nodes with a score higher than 0.75. What is the Jaccard coefficient
between the cores extracted with the two methods?

2. The network at [Link]

2/[Link] has multiple cores/communities. Use the Divisive
algorithm from cpalgorithm to find the multiple cores in the
network.

3. [Link] contains
a nested bipartite network. Draw its adjacency matrix, sorting
rows and columns by their degree.
29
Hierarchies

We usually represent organizations with networks. Each person in

the company is a node. Directed edges connect nodes. They usually
flow from the superior to the subordinate, from the coordinator to
its team. These networks have a particular structure. In an ideal 1
Aaron Clauset, Samuel Arbesman,
organization, there is a single head. If this were a corporation, that and Daniel B Larremore. Systematic
inequality and hierarchy in faculty
would be the CEO. The head commands a small group, its top level hiring networks. Science advances, 1(1):
executives. They, in turn, have their own team of managers. The e1400005, 2015
managers command their own groups, and so on and so forth, until
2
Erzsébet Ravasz, Anna Lisa Somera,
Dale A Mongru, Zoltán N Oltvai, and
we get to the foot soldiers. A-L Barabási. Hierarchical organization
Hierarchical networks arise not only in social systems1 , but also – of modularity in metabolic networks.
science, 297(5586):1551–1555, 2002
and especially – in biological ones2 , 3 , 4 , 5 , 6 , 7 . 3
Erzsébet Ravasz and Albert-László
This is some sort of core-periphery structure (Chapter 28), with a Barabási. Hierarchical organization in
few distinctions. complex networks. Physical review E, 67
(2):026112, 2003
First, in core-periphery there’s still some degree of horizontal con- 4
Haiyuan Yu and Mark Gerstein.
nections. People at the same level are able to connect to each other. Genomic analysis of the hierarchical
In a perfect hierarchy that is not the case: horizontal connections are structure of regulatory networks.
Proceedings of the National Academy of
banned. You can assign each worker to a level, and workers can only Sciences, 103(40):14724–14731, 2006
connect to lower level – and being connected by higher levels. 5
A Vazquez, R Dobrin, D Sergi, J-
Second, usually hierarchical networks are directed, while core- P Eckmann, ZN Oltvai, and A-L
Barabási. The topological relationship
periphery normally doesn’t care all that much about the direction of between the large-scale attributes and
the edges. local interaction patterns of complex
networks. Proceedings of the National
We can then consider hierarchies as some sort of special case of
Academy of Sciences, 101(52):17940–17945,
core-periphery structures. In this chapter we’ll explore the concept 2004
of hierarchical networks, as it can be interpreted in multiple ways. 6
Peter Csermely, Tamás Korcsmáros,
Huba JM Kiss, Gabor London, and Ruth
We’re then going to introduce key concepts – and the main methods
Nussinov. Structure and dynamics of
using such concepts – to estimate the “hierarchicalness” of a directed molecular networks: a novel paradigm
network. of drug discovery: a comprehensive
review. Pharmacology & therapeutics, 138
(3):333–408, 2013a
29.1 Types of Hierarchies 7
Samuel Johnson and Nick S Jones.
Looplessness in networks is linked to
trophic coherence. Proceedings of the
When talking about “hierarchies” in complex networks, researchers National Academy of Sciences, 114(22):
mainly refer to three related but distinct concepts. We can classify 5618–5623, 2017
 hierarchies 397

8
Enys Mones, Lilla Vicsek, and Tamás
them in three categories: order, nested, and flow hierarchy8 . I’ll Vicsek. Hierarchy measure for complex
present the three of them in this section, noting how this chapter will networks. PloS one, 7(3):e33799, 2012
then only focus on flow hierarchy. Order and nested hierarchies are
covered elsewhere in this book with different names.

Order
In an order hierarchy, the objective is to determine the order in which
to sort nodes. We want to place each node to its corresponding level,
according to the topology of its connections. Usually, this is achieved
by calculating some sort of centrality score. The most central nodes
are placed on top and the least central on the bottom.

5
8 Figure 29.1: (a) A toy network.
(b) Its order hierarchy. I place
2
9 3 nodes in descending order of
5 betweenness centrality from top
7 3 1
to bottom.
9
6

4
6 7 8 2 4 1
(a)
(b)
Figure 29.1 provides an example of detecting an order hierarchy in
a toy network, using betweenness centrality as the guiding principle.
Node 5 has the highest betweenness centrality, followed by node
3 and then node 9. All other nodes have the same betweenness
centrality – equal to zero.
One can easily see that we already covered this sense of hierarchi-
cal organization of complex networks. The order hierarchy is nothing
more than a different point of view of node centrality. Thus, I refer to
Chapter 11 for a deeper discussion on the topic.

Nested
Nested hierarchy is about finding higher-order structures that fully
contain lower order structures, at different levels ultimately ending
in nodes. In the corporation example, the largest group is the cor-
poration itself, encompassing all workers. We can first subdivide
the corporation into branches, if it is a multinational, they could
be regional offices. Each office can be broken down into different
departments, which have teams and, finally, the workers in each
team.
Figure 29.2 provides an example of detecting a nested hierarchy in
a toy network. This is usually done by detecting smaller and smaller
398 the atlas for the aspiring network scientist

8
Figure 29.2: (a) A toy network.
2 Colored circles delineate nested
9
5 substructures. (b) Its nested
hierarchy, according to the
7 3 1
6 highlighted substructures. Each
node and substructure is con-
8 9 5 6 7 2 3 4 1
4 nected to the substructure it
(b) belongs to.
(a)

densely connected units in the network. Note how, in this case, the
hierarchy does not place nodes on levels, but organizes the detected
substructures.
This is equivalent to performing hierarchical community discovery
on complex networks. Thus, I refer to Chapter 33 for further reading.

Flow
In a flow hierarchy, nodes in a higher level connect to nodes at the
level directly beneath it, and can be seen as managers spreading infor-
mation or messages to the lower levels. We call it a “flow” hierarchy
because you can see the highest level node as the origin of a flow,
which hits first the nodes at the level directly beneath it, and so on
until it reaches the leaves of the network: the nodes at the bottom
layer.

8
5 Figure 29.3: (a) A toy network.
2 (b) Its flow hierarchy.
9
5
8 9 7 3 2
7 3 1
6
6 4 1
4
(b)
(a)
Figure 29.2 provides an example of detecting a flow hierarchy in a
toy network. Note how it tends to have high centrality nodes on top,
like the order hierarchy, but it creates a substantially different orga-
nization. Nodes directly connected to a given level tend to belong to
the level immediately beneath it, no matter their different centrality
values. One could think that the order hierarchy is a special case of
flow hierarchy, but that is incorrect: in a flow hierarchy all nodes
belonging to a level need to connect to the level directly below and
above, while that’s not the case for the order hierarchy. Moreover,
 hierarchies 399

the order hierarchy is just something we add on top of a node-level

measure (centrality), while the flow hierarchy is a meso-level analysis:
it describes how groups of nodes – the ones at different hierarchical
levels – relate to each other.
Since this concept is not covered elsewhere in this book, I focus
on flow hierarchies for the rest of this chapter. There are four main
approaches to detect hierarchies and quantify the hierarchicalness of
real world directed networks. They are: cycle-based Flow Centrality,
Global Reach Centrality (GRC), Agony, and Arborescence.
From now on, we always assume that the network we’re analyzing
is directed.

29.2 Cycles

I introduced the concept of cycles in Section 7.2: special paths that

start and end in the same node. Cycles are natural enemies of hierar-
chies. There is no way to have a perfect hierarchy if you have a cycle
in your network. In a perfect hierarchy, you can always tell who’s
your boss. Your boss will never take orders from you, nor from your
peer, and even less so from any of your underlings.
However, if you have a cycle, that is not true. A cycle means
that your boss gives you an order, you pass it down to one of your
underlings and, somehow, they give it back to your boss. This is clear
nonsense and should be avoided at all costs.

1 1
Figure 29.4: (a) A directed net-
2 3 2 3 work. In the figure, I highlight
in blue the edges partaking
2
6 12 13 9 9 in cycles. (b) The condensed
version of (a), where all nodes
4 5 14 15 part of a strongly connected
component are condensed in a
7 8 16 10 11 7 8 16 10 11 node (colored in blue).
(a) (b)

Thus, the simplest way to estimate the hierarchicalness of a di-

rected network is to count the number of edges involved in a cycle.
The fewer edges are part of a cycle in a network, the more hierarchi- 9
Jianxi Luo and Christopher L Magee.
cal it is9 . You can see in Figure 29.4(a) that the somewhat hierarchical Detecting evolving patterns of self-
organizing networks by flow hierarchy
network has only a handful of edges involved in cycles.
measurement. Complexity, 16(6):53–61,
You can calculate the flow hierarchicalness of a network by simply 2011
condensing the graph. Graph condensation follows two steps. First,
you reduce all the graph’s strongly connected components each to
a single node. Then you connect that node to all nodes that were
400 the atlas for the aspiring network scientist

connected to a node part of that component. When condensing the

graph in Figure 29.4(a), you’ll obtain the graph in Figure 29.4(b).
Note that, if you’re merging two edges, you need to keep track of
this information, for instance in the edge weight. The (2, 5) and (2, 6)
edges collapse in a single edge outgoing form node 2, which now
must have a weight of two.
The ratio between the sum of the edge weights in the condensed
graph and the number of edges in the original graph is the flow
hierarchy of your network. The original graph in Figure 29.4(a) had
20 edges. Since the condensed graph in Figure 29.4(b) has 11 edges
with a total weight sum of 12 (because of the edge of weight two
coming out of node 2), we can conclude that the network’s flow
hierarchy is equal to 12/20 = 0.6.
A consequence of this definition is that any directed network
composed by a single strongly connected component has a hierarchi-
calness of zero by definition.
Flow hierarchy has a major flaw: it’s too lenient. In fact, it says
that any and all directed acyclic graphs are perfect hierarchies. It
is very easy to construct toy examples of less-than-ideal structures
that this cycle-based flow hierarchy will consider perfect. Figure 29.5
shows two of such examples.

Figure 29.5: Two directed

acyclic graphs not conforming
to our intuition of a perfect
hierarchy. (a) A wheel graph
with one flipped edge. (b) A
“hierarchy” with more bosses
than workers.
(b)
(a)
In Figure 29.5(a) we have no cycles because I flipped one edge.
However, arguably, one should not be able to go from a perfect
hierarchy to less than 50% hierarchicalness by simply flipping one
direction. Figure 29.5(b) shows another case where there are more
bosses than workers: you don’t need to have a master in management
to see that this is no way to run an organization! Yet, since they
are both directed acyclic graphs, for this cycle-based definition of
hierarchy, both toy examples are ideal hierarchies.

29.3 Global Reach Centrality

I introduced the concept of reach centrality back in Section 11.3:

the reach centrality of node v in a directed network is the fraction
 hierarchies 401

of nodes it can reach using directed paths originating from itself.

This measure is often dubbed “local” reach centrality, because the 10
Enys Mones. Hierarchy in directed
same authors defined a “global” reach centrality10 (GRC). GRC random networks. Physical Review E, 87
is not a measure for nodes any more: it is a way to estimate the (2):022817, 2013
hierarchicalness of a network.
The intuition behind GRC is simple. A network has a strong
hierarchy if there is a node which has an overwhelming reaching
power compared to the average of all other nodes. Or, to put it in
other words, if there is an overseer that sees all and knows all. To
calculate GRC you first estimate the local reach centrality of all nodes
in the network. You then find the maximum value among them, say
LRC MAX . Then:

1
|V | − 1 v∑
GRC = LRC MAX − LRCv .
∈V
In practice, you average out its difference with all reach centrality
values in the network. This is an effective way of counteracting the
degeneracy of cycle-based hierarchy measures. In both toy examples
from Figure 29.5, GRC is well behaved, returning values of 0.555 and
0.22, respectively.

Figure 29.6: A network we

could consider a perfect hierar-
chy, for which GRC fails to give
a perfect score.

However, GRC has a blind spot of its own. Since we’re averaging
the differences between the most central node against all others, we
know we will never get a perfect GRC score if there is more than one
node with non-zero local reach centrality. Consider Figure 29.6. I
don’t know about you, but to me it looks like a pretty darn perfect
hierarchy. Yet, we know that the two nodes connected by the root
don’t have a zero local reach centrality. In fact, the GRC for that
network is 0.89.
So, if cycle-based flow hierarchy is too lenient – every directed
402 the atlas for the aspiring network scientist

acyclic graph is a perfect hierarchy –, GRC is too strict: even flawless

hierarchies might fail to get a 100% score. If for cycle-based flow
hierarchy the perfect hierarchy is a DAG, for GRC the only perfect
hierarchy is a star: a central node connected to everything else, and
no other connections in the network.

29.4 Arborescences

We introduced the concept of arborescence in Section 7.2, as a stricter

definition of a directed acyclic graph. To sum up: an arborescence
is a directed tree in which all nodes have in-degree of one, except
the root, which has in-degree of zero. For instance, Figure 29.6 is an
arborescence. Given their properties, arborescences seem particularly
well suited to inform us about hierarchies. Every arborescence is a
perfect hierarchy: all nodes have a single boss, there are no cycles,
and there is one node with no bosses: the CEO. 11
Michele Coscia. Using arborescences
That is why I used them to create my own hierarchicalness score11 . to estimate hierarchicalness in directed
I take an approach similar to the one developed from the cycle-based complex networks. PloS one, 13(1):
e0190825, 2018
flow hierarchy. In fact, the first step is almost identical: take your
directed graph and condense all its strongly connected components
into a single node. The only difference is that here we ignore edge
weights. This gives us a directed acyclic graph version of the original
network. Note that there are alternative methods to reduce a generic 12
Can Lu, Jeffrey Xu Yu, Rong-Hua Li,
directed network to a DAG12 , 13 , which can preserve more edges. and Hao Wei. Exploring hierarchies in
To transform it in an arborescence, we need to remove all edges online social networks. IEEE Transactions
on Knowledge and Data Engineering, 28(8):
going “against” the flow. We cannot allow any node to have an in- 2086–2100, 2016
degree larger than one. So all edges contributing to a larger-than-one 13
Jiankai Sun, Deepak Ajwani, Patrick K
Nicholson, Alessandra Sala, and Srini-
in-degree have to be removed. We cannot remove them at random:
vasan Parthasarathy. Breaking cycles
we need to remove the ones pointing towards the root and keep the in noisy hierarchies. In Proceedings of
ones pointing away from it. I use closeness centrality to determine the 2017 ACM on Web Science Conference,
pages 151–160. ACM, 2017
which edges to keep: the ones coming from the node with the lowest
closeness centrality. This is a bit counter-intuitive: the closer a node
is to the root, the lower its closeness centrality is. This is due to the
fact that these nodes have more possible paths originating from them,
and they tend to be longer because they can reach a larger portion of
the network.
Once all edges breaking the arborescence requirements are elim-
inated, we can count how many connections survived. This is also
equivalent to the final step of the cycle-based flow hierarchy. The
more edges we needed to remove to obtain an arborescence, the less
the original network was resembling a perfect hierarchy. In the exam-
ple from Figure 29.4, we would preserve nine edges out of 20, giving
us an arborescence score of 9/20 = 0.45. Differently from the cycle-
based measure, the arborescence score is not fooled by the examples
 hierarchies 403

Figure 29.7: The additional

step to go from cycle-based
hierarchy to arborescence. (a) I
highlight in blue the two edges
going “against the flow”. (b)
The final result, reduced from
Figure 29.4(a): an arborescence
forest.
(a) (b)

in Figure 29.5, returning a score of 0.5 and 0.33, respectively.

Note from Figure 29.7 that, technically speaking, this technique
reduces an arbitrary directed network into an arborescence forest,
not an arborescence. This is another difference with the cycle-based
method, as condensing a graph will never break it into multiple
weakly connected components. Arborescence is a very punitive
measure, much more than cycle-based flow hierarchy, but less so than
GRC.

29.5 Agony

In the agony measure we start from the assumption that we can

partially order nodes into levels. The CEO lives at the top of the hi-
erarchy (level 1), its immediate executive are at level 2, the managers
beneath them are at level 3, and so on. Let’s say that lv tells us the
level of node v. In this scenario, a perfect hierarchy only has edges
going from a node in a lower (more important) level to a node in a
higher level. If lu < lv then a u → v edge is ordinary and expected.
On the other hand, a u ← v edge will cause “agony”: something isn’t
as it is supposed to be.
How much agony does it cause? Well, this is proportional to the
level difference between the nodes. You won’t be shocked if the
CEO accepts orders from another top executive, but you’ll go to the
madhouse if she does what the most recently hired intern says. In the 14
Mangesh Gupte, Pravin Shankar, Jing
original paper14 , the authors define the agony of the u ← v edge as: Li, Shanmugauelayut Muthukrishnan,
lv − lu + 1. The +1 is necessary because, if we were to exclude it, we and Liviu Iftode. Finding hierarchy
in directed online social networks.
could put all nodes in the same level and obtain zero agony, which In Proceedings of the 20th international
would defeat the purpose of the measure. conference on World wide web, pages
557–566. ACM, 2011
Ultimately, this reduces to calculating the result of:

A( G, l ) = ∑ max(lv − lu + 1, 0).
(u,v)∈ E

Every time lu < lv , we contribute zero to the sum. Note that agony
404 the atlas for the aspiring network scientist

requires you to specify the lu value for all nodes in the network. This
is not usually something you know beforehand. So the problem is to
find the lu values that will minimize the agony measure. There are 15
Nikolaj Tatti. Hierarchies in directed
efficient algorithms to estimate the agony of a directed graph15 . networks. In 2015 IEEE international
conference on data mining, pages 991–996.
IEEE, 2015
Figure 29.8: Two hierarchies
with different values of agony.
The vertical positioning of each
node determines its level, from
top (lu = 1) to bottom (lu = 4).

(a) (b)
Consider Figure 29.8. In both cases, we have only one edge going
against the flow. Agony, however, ranks these two structures differ-
ently. In Figure 29.8(a), the difference in rank is only of one, thus the
total agony is 2. In Figure 29.8(b), the difference in rank is 3, resulting
in a higher agony. Also a cycle-based flow hierarchy measure takes
different values, as Figure 29.8(b) involves more edges in a cycle (four
edges, versus just two in Figure 29.8(a)).
Ultimately, the resting assumption of agony is the same of the
cycle-based flow hierarchy. Agony considers any directed acyclic
graph as a perfect hierarchy. Thus it will give perfect scores to the
imperfect hierarchies from Figure 29.5.

29.6 Drawing Hierarchies

To wrap up this chapter, note that all these methods have a various
degree of graphical flavor to them. Meaning that you can use them to
create a picture of your hierarchy, which might help you to navigate
the structure. The most rudimentary method is the cycle-based flow
hierarchy, because it just reduces the graph to a DAG, which doesn’t
help you much.
Global reach centrality is better, as you can place nodes on a ver-
tical level according to their local reach centrality value. In Figure
29.9(b) I apply a reach centrality informed layout to the directed
graph from Figure 29.9(a). The reach centrality layout is quite rudi-
mentary, as the resulting picture looks more akin to an order hierar-
chy than a flow hierarchy. In the figure, I had to do a bit of manual
work to make it look more like a flow hierarchy, which you might not
be able to do for larger graphs. Since the method allows you to find
flow hierarchies, this mismatch could be confusing. However, at least
 hierarchies 405

Figure 29.9: (a) A directed

graph. (b) The graph from (a),
layout according to local reach
centrality – with the most cen-
tral nodes on top and the least
central on the bottom. (c) The
graph from (a) layered accord-
ing to its arborescence scheme
(a) (c)
or its agony levels – the two are
(b)
equivalent.

it allows you to find out the root of the hierarchy, the node(s) with
the highest reach centrality, which the previous method could not do.
The arborescence approach is a further step up. Since it reduces
the network to an arborescence, one can draw the resulting con-
densed graph, identifying not only the root of the hierarchy, but at
which level each node lies. The same can be said for agony: it assigns
each node to a level, thus you can plot the network by layering nodes
vertically according to their assigned rank. Figure 29.9(c) shows a
possible layout informed by arborescence (or agony).

29.7 Summary

1. There are many different ways to intend the meaning of “hier-

archy” in complex networks. Order hierarchy is like centrality:
sorting nodes according to their importance. Nested hierarchy is
like communities: grouping nodes in teams and teams of teams.

2. Here we look at flow hierarchy: a structural organization where

we have nodes working at different levels and information always
flows in one direction, from nodes in a higher level to nodes in the
directly lower level. We assume networks are directed.

3. There are many ways to estimate the hierarchicalness of a network.

A perfect hierarchy cannot have cycles, which are nodes at a lower
level linking against the flow to higher levels. One can simply
remove cycles, or calculate how much “agony” a connection brings
to the structure.

4. We can identify the head of the hierarchy as the node with the
highest reach. Alternatively, arborescences are prefect hierarchies
– directed acyclic graphs with all nodes having in-degree of one,
except the head of the hierarchy having in-degree of zero.
406 the atlas for the aspiring network scientist

29.8 Exercises

1. Calculate the flow hierarchy of the network at [Link]

[Link]/exercises/29/1/[Link]. Generate 25 ver-
sions of the network with the same degree distributions of the
observed one (use the directed configuration model) and calculate
how many standard deviations the observed value is above or
below the average value you obtain from the null model.

2. Calculate the global reach centrality of the network at http://

[Link]/exercises/29/1/[Link] (note: it’s much
better to calculate all shortest paths beforehand and cache the
result to calculate all local reaching centralities). Is there a single
head of the hierarchy or multiple? How many?

3. The arborescence algorithm is simple: condense the graph to

remove the strongly connected components and then remove
random incoming edges from all nodes remaining with in-degree
larger than one, until all nodes have in-degree of one or zero.
Implement the algorithm and calculate the arborescence score.

4. Perform the null model test you did for exercise 1 also for global
reach centrality and arborescence. Which method is farther from
the average expected hierarchy value?
30
High-Order Dynamics

Networks are a great tool to represent a complex system in a simple

and elegant way. They embed most of their subtleties into a coherent
structure. However, if you only look at the structure you might
miss part of the story. This is especially true if you’re interested in
knowing how different agents act in it.
For instance, consider air travel. The airlines give you the structure,
by deciding from where their planes take off and to where they fly.
And, if you’re interested only in studying how different airports
connect together, that is all you need to know. But you might instead
want to study the behavior of the passengers taking those flights.
In this case, the structure itself might be misleading. If you ever
made some air travel, you know that, in most cases and especially for
long trips, you would take connecting flights. Meaning that you will
hop through one or more airports before you reach your intended
destination from your origin.
But, if all you look at is the structure of flights, you’re not going to
be able to recover such information. When a traveler boards in u and
jumps off in v, if all you have is the structure, you only know that
they are going to either stay in v or go to one of v’s neighbors, maybe
even back to u. If you want to really model the traveler’s behavior,
you need some sort of memory, you need to know they arrived from u
into v. The next step is not dependent exclusively on the fact we’re in
v.
Figure 30.1 shows an example of this. The network might have
equal weights on the edges, because from the central airport there is
an equal number of flights or passengers in each link. Thus, all we
can say is that all steps from the central node are equally likely (left).
However, from observed data, we might see that some second steps
– from our given origin – are much more likely than others (right).
In this case, we cannot trust the unweighted edges at face value: we
need to take this additional information into account.
When you start talking about memory, you’re talking about higher
408 the atlas for the aspiring network scientist

Figure 30.1: An example of pos-

sible high order dynamics. The
blue arrow shows the first step,
while the green arrows show
the potential second steps.

No Memory With Memory

order dynamics – for a refresher on the terminology, see Chapter 2.

In this chapter, we’ll explore two ways of embedding this memory
into your network analysis workflow. You can either modify your net-
work data, embedding the higher order dynamics into the structure;
or you can modify your algorithm.

30.1 Embedding Dynamics into the Structure

The first natural way to have higher order memory in your network
analysis is by embedding it into the structure itself. This is a pow-
erful approach, because it allows you to use any non-high-order
algorithm you want. You have the entirety of the network analysis
toolbox at your disposal. The price you have to pay is that you need
to keep track of your operation. You need to reconstruct the original
structure if you want to properly interpret your results.
This practically boils down to performing a pre-processing on your
data structure and a post-processing on your results. Here we focus
mainly on the pre-processing as, hopefully, how to post-process the
results should be straightforward. Unfortunately, the pre-process is
not going to be as simple as I make it to be in Figure 30.1. You cannot
simply re-weight your edges, because the re-weighting would be
dependent on the current position of the agent in the network. Thus
you’d have to have a re-weighting for every node in the network.
Worse still, if you do that you’re just implementing second-order
dynamics: if you need to go to a higher order than that (say, you
need to remember the last two nodes through which you passed)
you’re in no better position than before.
I’m going to specifically focus on a few papers in this line of
research, but hopefully you could see how the general approach
in this category of high-order analysis works. Also note that other
structures described elsewhere in the book can count as high order
structures. For instance, hypergraphs connecting multiple nodes at
the same time, and networks with simplicial complexes, which are
 high-order dynamics 409

higher order structures. Check Section 4.3 for a refresher.

High Order Network

1
Jian Xu, Thanuka L Wickramarathne,
What we want to build here is a High Order Network1 (HON). Let’s and Nitesh V Chawla. Representing
higher-order dependencies in networks.
go back to our airline travel example. In the original network, nodes Science advances, 2(5):e1600028, 2016
are airports and edges represent flows of passengers between them.
Now, the crucial assumption we’re making here is that, if I’m landing
in New York in a plane coming from Boston, this is fundamentally
a different travel than the one which makes me land in the same
airport, but from London. Thus, in the HON representation, we split
the New York node in two meta nodes. One of the two meta nodes
captures the passenger flow from Boston, the other from London.
Figure 30.2 shows an example of this procedure.

BOS CPH BOS CPH

NY Figure 30.2: (a) A simple di-
rected network of airplane
NY|BOS
travel. (b) Its corresponding
NY
High Order version, intro-
NY|LON
ducing conditional nodes
LON ALB LON ALB
depending on the origin of the
flow.
(a) (b)

What happens here is that now we have a way to represent dif-

ferent flows. A passenger from London might be much more likely
to want to go to Alberta, while the Bostonian would instead go to
Copenhagen. Thus we can re-part the outgoing edge weight of New
York into those two meta nodes, to make them a more accurate repre-
sentation of the data.

CPH|NY Figure 30.3: A small example

NY|.
of a full High Order Network
BOS|. CPH|NY,BOS
with complex dependencies up
NY|BOS
to an order of four. the dashed
CPH|NY,LON
outlines group all the meta
NY|LON
LON|. nodes originating from the
same original node.
PAR|. LON|PAR NY|LON,PAR
ALB|NY,LON

LYO|. PAR|LYO LON|PAR,LYO NY|LON,PAR,LYO

ALB|NY,LON,PAR
410 the atlas for the aspiring network scientist

Of course, this represents only a single step in the creation of the

HON structure. The origin nodes that brought us to New York were
themselves the product of another high order transition. Thus they
are also split into several meta nodes. The general HON structure
looks like the one in Figure 30.3.
You might have noticed that we use the conditional probability
notation introduced in Chapter 2. Each node represents the transition
probability of getting into node v given that we come from node
u – and, possibly, given that we reached u from another node, and
so on. While airline travel might have short dependency chains –
rarely a trip involves more than two transfers – other networks show
dependency chains up to length five: the global shipping network,
for instance. When I label a node v as, for instance, v|., it means that
this specific node has no dependencies: it represents passengers who
are starting their first step in v.
If you think HON structures looks like a big and complex Bayesian
network – see Section 4.6 – don’t worry: you’re not alone.
There are potential downsides of using a HON representations.
First, as you can see from Figure 30.3, the structure can become really
unwieldy. The original network only had 7 nodes and 6 edges, and
ballooned into having 17 nodes and 16 edges. Therefore you need
to find the right trade off between the complexity of your HON
representation and the analytic gains it gives you. In the original
paper, for instance, the authors limit themselves to an order of five,
even if the shipping network they analyze might have even longer
dependencies. The increase in complexity simply wasn’t worth it.
Finally, HON networks tend to transform into weakly connected
graphs, or even not connected. The original network from Figure 30.3
had a single connected component, while its HON representation
splits into 5 connected components. If your algorithm cannot handle
multiple connected components, you might be in trouble.

Memory Network

Alternatives to HON exist. For instance, researchers built what they 2

Martin Rosvall, Alcides V Esquivel,
Andrea Lancichinetti, Jevin D West, and
call a “memory network2 ”. To model second-order dynamics we Renaud Lambiotte. Memory in network
can create a line graph. If you recall Section 3.1, a line graph is a flows and its effects on spreading
transformation of the original graph. The edges of the original graph dynamics and community detection.
Nature communications, 5:4630, 2014
become the nodes of the network and they are connected to each
other if the original edges shared a node in the network. Figure
30.4 is a reproduction of Figure 3.4 to remind you of the building
procedure. Line graphs will pop up also in overlapping community
discovery (Section 34.5).
You can model third order dynamics by making a more compli-
 high-order dynamics 411

3-5
3 Figure 30.4: (a) A graph. (b) Its
4-5 linegraph version.
2 4 3-4
5 2-4

1-4
1
1-2
(a)
(b)

cated version of a line graph, in which nodes stand in for paths of

length three. You can see how memory graphs are a generalization
of line graphs, and they start looking extremely similar to the HON
model. In fact, once you have built your memory network with the
desired order, then the simple memoryless Markov processes on the
memory network describe the high-order processes, of the order you
used to build the network in the first place. The adjacency matrix
of a memory network of second order is a non-backtracking matrix
(Section 8.2), provided that the memory network has no self-loops.
Non-backtracking random walks are another example of high order
network analysis.
The difference between memory networks and HONs is that
HONs are a bit more flexible, because they allow you to have nodes
of any order mixed together in the same structure. In the memory
network, all nodes represent transitions of the same order.

30.2 Embedding Dynamics into the Algorithm

The alternative approach to deal with high order dependencies is to

leave your structure alone and to embed the high order logic directly
into your algorithm. In practice, you “hide” both the pre- and post-
process from the previous section into your analysis. This way, you
don’t have to deal with the complexity yourself.
There is a bit more diversity in this category of solutions, due to
the many different valid ways one could incorporate high orders into
network analysis.

Motif Dictionary
The first approach I consider is the one building motif dictionaries. In
this approach, one realizes that there are different motifs of interest
that have an impact on the analysis. For instance, one could focus
specifically on triangles. Once you specify all the motifs you’re
interested in, you take a traditional network measure and you extend
412 the atlas for the aspiring network scientist

it to consider these motifs.

This move isn’t particularly difficult once you realize that low-
order network measures still work with the same logic. It’s just that
they exclusively focus on a single motif of the network: the edge.
An edge is a network motif containing two nodes and a connection
between them. Once you realize that a triangle is nothing else but a
motif with three nodes and three edges connecting them, then you’re
in business.
To make this a bit less abstract, let’s consider a specific instance 3
Austin R Benson, David F Gleich,
of this approach3 , 4 . In the paper, the idea is to use motifs to inform and Jure Leskovec. Higher-order
community discovery, the topic of Part IX. I’m going to delve deeper organization of complex networks.
Science, 353(6295):163–166, 2016
into the topic in that book part, for now let’s just say that we’re 4
Hao Yin, Austin R Benson, Jure
interested in solving the 2-cut problem (Section 8.4): we want to Leskovec, and David F Gleich. Local
divide nodes in two groups such that we minimize the number of higher-order graph clustering. In
Proceedings of the 23rd ACM SIGKDD
edges connecting nodes in different groups. International Conference on Knowledge
In the classical problem we want to make a normalized cut such Discovery and Data Mining, pages
that the number of edges flowing from one group to the other is 555–564. ACM, 2017

minimum, considering some normalization factor. So we have a

fraction that looks something like this:

| ES,S̄ |
φ(S) = ,
min (| ES |, | ES̄ |)
where S is one of the two groups – i.e. a set of nodes on one side
of the cut –, S̄ is its complement – i.e. S̄ = V − S, the set of nodes
on the other side of the cut. Ex is the set of edges in the set x, and
Ex,y is the set of edges established between a node in x and a node
in y. In practice, let’s find S such that φ(S) is minimum. We do so
by minimizing the number of edges between S and its complement,
normalized by their sizes (so that we don’t find trivial solutions by
cutting off simply a dangling leaf node).
But here we say: No! Not the number of edges! We are interested in
higher order structures! We want to minimize the number of triangles
between groups! How would that look like? Exactly the same. We
just count not the number of the edges, but the number of arbitrary
motifs M spanning between S and non-S:

| MS,S̄ |
φ(S, M ) = .
min (| MS |, | MS̄ |)
Boom. By finding an S minimizing this specific φ(S, M ) we just
made a high-order normalized cut. This can be done exactly like 5
Austin R Benson, David F Gleich, and
finding the “normal” normalized cut, by examining the eigenvectors Jure Leskovec. Tensor spectral clustering
for partitioning higher-order network
of a specially constructed Laplacian5 . Figure 30.5 shows an example.
structures. In Proceedings of the 2015
In the figure, the two groups still have tons of edges going from one SIAM International Conference on Data
group to another, this is hardly a solution for a regular normalized Mining, pages 118–126. SIAM, 2015
 high-order dynamics 413

Figure 30.5: (a) The motif we

want to minimize the cut for. (b)
A high order normalized cut
solution, which I represent via
the node’s color.
(a)
(b)

cut. But there are few triangles between S and S̄, thus this is a proper
solution for the higher order normalized cut. The shape of this
solution can be applied to many other problems.

Processes with Memory

In this category of solutions you find all the approaches who take
the equation of the Markov process and add a temporal factor. For
instance, suppose that we’re observing random walks (Chapter
8). We have a vector of probability pk that tells us the status of the
random walkers at time k, namely the probability of the walkers to
be in a node (pk is a vector of length |V |). Now, if we were doing
a normal random walk and perform one step, we’d know that the
next step is simply given by the stochastic adjacency matrix, i.e.
pk+1 = pk D −1 A (assuming A is the normal adjacency matrix, and
D the degree diagonal matrix). This is a discrete random walk on a
graph.
But what if we want higher order dynamics? What if we want to
look at the result of the process after t steps? Well, the idea is to add
the temporal information to the equation: pk+t = pk Tt . So what’s Tt ?
Tt is a matrix telling us the transition probability from u to v after t
steps. If you followed the linear algebra math in Chapter 8 you know
that, to perform a random walk of length t, you can simply raise
D −1 A to the power of t: ( D −1 A)t .
The limitation here is that time ticks discretely, i.e. the random 6
Michael T Schaub, Renaud Lambiotte,
and Mauricio Barahona. Encoding
walker makes one move per timestep. In many cases, you might want dynamics for multiscale community
to simulate the passage of time as a continuous flow6 . If you want detection: Markov time sweeping for
to do it, you need to derive a different Tt . First, rewrite D −1 A as the map equation. Physical Review E, 86
(2):026112, 2012b
− D −1 L. This changes the random walk from discrete to continuous7 . 7
Renaud Lambiotte, J-C Delvenne, and
This allows us to let time flow and take the result of the random Mauricio Barahona. Laplacian dynamics
−1
walk at time t: Tt = e−tD L . If we set t = 1, we recover exactly the and multiscale modular structure in
networks. arXiv preprint arXiv:0812.1770,
transition probabilities of a one-step random walk. But now we’re 2008
free to change t at will, to get second, third, and any higher order
Markov processes.
414 the atlas for the aspiring network scientist

Other Approaches
There is a whole bunch of other approaches to introduce high order
dynamics into your network structure. More than I can competently
cover. I provide here some examples with brief, but overly simplistic,
explanations.
One way is to use tensors (Section 5.2). In practice, you create a
multidimensional representation of the topological features of the
network. Each added dimension of the tensor represents an extra 8
Michael Chertok and Yosi Keller.
order of relationships between the nodes8 , 9 . Then, by operating on Efficient high order matching. IEEE
this tensor, you can solve any high order problem: in the papers I cite Transactions on Pattern Analysis and
Machine Intelligence, 32(12):2205–2215,
the problem the authors focus on is graph matching.
2010
Another general category of solutions is a collection of techniques 9
Olivier Duchenne, Francis Bach, In-So
to take high order relation data and transform it into an equiva- Kweon, and Jean Ponce. A tensor-
based algorithm for high-order graph
lent first order representation10 , 11 . The first order solution in this matching. IEEE transactions on pattern
structure then translates into the high order one, much like in the analysis and machine intelligence, 33(12):
2383–2395, 2011
HON and memory network approach. The cited techniques are also 10
Hiroshi Ishikawa. Higher-order clique
generally applied to the problem of finding a high order cut in the reduction in binary graph cut. In 2009
network. IEEE Conference on Computer Vision and
Pattern Recognition, pages 2993–3000.
Being able to study high order interactions can help you making
IEEE, 2009
sense of many complex systems. For instance, they have been used 11
Alexander Fix, Aritanan Gruber,
to explain the remarkable stability of biodiversity in complex eco- Endre Boros, and Ramin Zabih. A graph
cut algorithm for higher-order markov
logical systems12 . Other application examples include the study of random fields. In 2011 International
infrastructure networks13 – how to track the high order flow of cars Conference on Computer Vision, pages
in a road graph –, and aiding in solving the problem of controlling 1020–1027. IEEE, 2011
12
Jacopo Grilli, György Barabás,
complex systems14 – which we introduced in Section 18.4. Matthew J Michalska-Smith, and
Stefano Allesina. Higher-order interac-
tions stabilize dynamics in competitive
30.3 Summary network models. Nature, 548(7666):210,
2017
1. Classical network analysis is single-order: only the direct connec- 13
Jan D Wegner, Javier A Montoya-
Zegarra, and Konrad Schindler. A
tions matters. But many phenomena they represent are high-order:
higher-order crf model for road net-
in a flight passenger networks, the nodes you just visited greatly work extraction. In Proceedings of the
influence to which node you’ll move next. IEEE Conference on Computer Vision and
Pattern Recognition, pages 1698–1705,
2013
2. There are two approaches to high-order networks: embedding the 14
Abubakr Muhammad and Magnus
dynamics in the structure, meaning that we modify the network Egerstedt. Control using higher order
data so that it has “memory”; or embedding it in the analysis, laplacians in network topologies. In
Proc. of 17th International Symposium
giving to the algorithm the task of remembering previous moves.
on Mathematical Theory of Networks and
Systems, pages 1024–1038. Citeseer, 2006
3. In High Order Networks you split each node to represent all the
paths that lead to it. In memory networks you instead model
second order dynamics with a line graph, third order dynamics
with the line graph of the line graph, and so on.

4. Embedding memory in the analysis can be done by either build-

ing dictionaries of temporal motifs, adding a temporal factor in
 high-order dynamics 415

the classical Markov equation (e.g. for random walks), and other
approaches.

30.4 Exercises

1. Generate the two-order line graph of the network at [Link]

[Link]/exercises/30/1/[Link], using the average
edge betweenness of the edges as the edge weight.

2. Assume that the edge weight is proportional to the probability

of following that edge. Which 2-step node transitions became
more likely to happen in the line graph compared to the original
network? (For simplicity, assume that the probability of going back
to the same node in 2-steps is zero for the line graph)
 Part IX

Communities
31
Graph Partitions

We have reached the part of network analysis that has probably re-
ceived the most attention since the explosion of network science in
the early 90s: community discovery (or detection). To put it bluntly,
community discovery is the subfield of network science that pos-
tulates that the main mesoscale organization of a network is its
partition of nodes into communities. Communities are groups of
nodes that are very related to each other. If two nodes are in the
same community they are more likely to connect than if they are in
two distinct communities. This is an overly simplistic view of the
problem, and we will decompose this assumption when the time
comes, but we need to start from somewhere.
So the community discovery subfield is ginormous. You might
ask: “Why?” Why do we want to find communities? There are many
reasons why community discovery is useful. I can give you a couple
of them. First, this is the equivalent of performing data clustering
in data mining, machine learning, etc. Any reason why you want
to do data clustering also applies to community discovery. Maybe
you want to find similar nodes which would react similarly to your
interventions. Another reason is to condense a complex network into
a simpler view, that could be more amenable to manual analysis or
human understanding.
More generally, decomposing a big messy network into groups
is a useful way to simplify it, making it easier to understand. The
reason why there are so many methods to find communities – which,
as we’ll see, rarely agree with each other – is because there are innu-
merable ways to simplify a network.
It is difficult to give you a perspective of how vast this subfield of
network science is. Probably, one way to do it is by telling you that
there are so many papers proposing a new community discovery
algorithm or discussing some specific aspect of the problem, that
making a review paper is not sufficient any more. We are in need
of making review papers of review papers of community discovery.
418 the atlas for the aspiring network scientist

This is what I’m going to attempt now.

I think that we can classify review works fundamentally into four
1
Santo Fortunato. Community detection
in graphs. Physics reports, 486(3-5):
categories, depending on what they focus on the most. Review pa- 75–174, 2010
pers on community discovery usually organize community discovery 2
Santo Fortunato and Darko Hric.
algorithms by: Community detection in networks: A
user guide. Physics Reports, 659:1–44,
2016
• Process. In this subtype of review paper, the guiding principle 3
Srinivasan Parthasarathy, Yiye Ruan,
is how an algorithm works1 , 2 , 3 , 4 , 5 , 6 , 7 . Does it use random walks and Venu Satuluri. Community discov-
ery in social networks: Applications,
rather then eigenvector decomposition? Does it use a propagation
methods and emerging trends. In Social
dynamic or a Bayesian framework? network data analytics, pages 79–113.
Springer, 2011
• Performance. Here, all that matters is how well the algorithm 4
Mason A Porter, Jukka-Pekka Onnela,
works in some test cases8 , 9 , 10 , 11 , 12 . The typical approach is to find and Peter J Mucha. Communities in
many real world networks, or creating a bunch of synthetic bench- networks. Notices of the AMS, 56(9):
1082–1097, 2009
mark graphs (usually using the LFR benchmark – see Section 5
Mark EJ Newman. Detecting commu-
15.2) and rank methods on how well they can maximize a quality nity structure in networks. The European
function. Physical Journal B, 38(2):321–330, 2004b
6
Leon Danon, Albert Diaz-Guilera,
• Definition. More often than not, the standard community defi- Jordi Duch, and Alex Arenas. Compar-
nition I gave you earlier (nodes in the same community connect ing community structure identification.
Journal of Statistical Mechanics: Theory
to each other, nodes in different communities rarely do so) isn’t and Experiment, 2005(09):P09008, 2005
exactly capturing what a researcher wants to find. We’ll explore 7
Natali Gulbahce and Sune Lehmann.
later how this definition fails. Some review works acknowledge The art of community detection. BioEs-
says, 30(10):934–938, 2008
this, and classify methods according to their community def- 8
Jure Leskovec, Kevin J Lang, and
inition13 , 14 , 15 . Different processes might be based in different Michael Mahoney. Empirical comparison
definitions, so there’s overlap between this category and the first of algorithms for network community
detection. In WWW, pages 631–640.
one I presented, but that is not always the case. ACM, 2010b
• Similarity. Finally, there are review works using a data-driven
9
Zhao Yang, René Algesheimer, and
Claudio J Tessone. A comparative
approach to figure out which algorithms, on a practical level, analysis of community detection
return very similar communities for the same networks16 , 17 , 18 , 19 . algorithms on artificial networks.
Scientific Reports, 6:30750, 2016a
This is similar to the performance category, with the difference 10
Steve Harenberg, Gonzalo Bello,
that we’re not interested in what performs better, only in what L Gjeltema, Stephen Ranshous, Jitendra
performs similarly. Harlalka, Ramona Seay, Kanchana
Padmanabhan, and Nagiza Samatova.
Community detection in large-scale
The process approach is the most pedagogical one, because it
networks: a survey and empirical
focuses on us trying to understand how each method works. Thus, it evaluation. Wiley Interdisciplinary
will be the one guiding this book part the most. However, I’ll pepper Reviews: Computational Statistics, 6(6):
426–439, 2014
around the other approaches as well, when necessary. This book part 11
Günce Keziban Orman and Vincent
will necessarily be more superficial than what one of the excellent Labatut. A comparison of community
surveys out there can do in each specific subtopic of community detection algorithms on artificial
networks. In DS, pages 242–256.
discovery, so you should check them out. Springer, 2009
In this chapter I’ll limit myself discussing the most classical view 12
Andrea Lancichinetti and Santo
of community discovery, the one sheepishly following the classical Fortunato. Community detection
algorithms: a comparative analysis.
definition. We’re going to take a historical approach, exploring how Physical review E, 80(5):056117, 2009
the concept of network communities came to be as we intend it today. 13
Satu Elisa Schaeffer. Graph clustering.
Later chapters will complicate the picture. Buckle up, this is going to Computer science review, 1(1):27–64, 2007
 graph partitions 419

be a wild ride. 14
Michele Coscia, Fosca Giannotti,
and Dino Pedreschi. A classification
for community discovery methods in
31.1 Stochastic Blockmodels complex networks. SADM, 4(5):512–546,
2011
15
Fragkiskos D Malliaros and Michalis
Classical Community Definition Vazirgiannis. Clustering and community
detection in directed networks: A
When people started mapping complex systems as networks, they survey. Physics Reports, 533(4):95–142,
2013
realized that the edges didn’t distribute randomly across nodes. We 16
Nguyen Xuan Vinh, Julien Epps, and
already saw that deviating from random expectation is a source of James Bailey. Information theoretic
interest when we talked about degree distributions (Section 6.2). measures for clusterings comparison:
Variants, properties, normalization and
On top of the rich-get-richer effect, researchers realized that edges
correction for chance. J. Mach. Learn. Res,
distributed unevenly among different groups of nodes. Especially, 11(Oct):2837–2854, 2010
but not only, in social networks, there were lumps of connections, 17
Vinh-Loc Dao, Cécile Bothorel, and
Philippe Lenca. Estimating the similarity
separated by very sparse areas.
of community detection methods
The ideal scenario is something resembling Figure 31.1(a). The nat- based on cluster size distribution.
ural next step was trying to see if we could separate these lumps of In International Workshop on Complex
Networks and their Applications, pages
connections into coherent groups: communities. This created the first 183–194. Springer, 2018b
and most commonly accepted definition of a network community: 18
Vinh-Loc Dao, Cécile Bothorel, and
Philippe Lenca. Community structure:
Communities are groups of nodes densely connected to each other and A comparative evaluation of community
detection methods. arXiv preprint
sparsely connected to nodes outside the community.
arXiv:1812.06598, 2018a
19
Amir Ghasemian, Homa Hossein-
I would call this the classical definition of a network community. mardi, and Aaron Clauset. Evaluating
This definition can be attacked and deconstructed from multiple overfit and underfit in models of net-
parts but, for now, let’s accept it. Note that, for now, we assume that work community structure. TKDE,
2019
a node can only be part of a single community. We use the term
“partition” to refer to the assignment of nodes to their community.

Figure 31.1: (a) A representa-

tion of a classical ideal commu-
nity structure. I encode with
the node color the community
to which the node belongs. (b)
The adjacency matrix of (a).

(b)
(a)
In the early community discovery days20 – before we even had
20
Paul W Holland, Kathryn Blackmond
Laskey, and Samuel Leinhardt. Stochas-
coined the term “community” –, the main approach was using tic blockmodels: First steps. Social
stochastic blockmodels. We already introduced them in Section networks, 5(2):109–137, 1983
15.2 as a method to generate a synthetic graph. How do we apply
them to the problem of finding communities? The first step in our
quest is changing the perspective over the graph from Figure 31.1.
420 the atlas for the aspiring network scientist

Figure 31.1(b) shows you its corresponding adjacency matrix. The

diagonal blocks are the ones referred by the name “blockmodel”.

Maximum Likelihood

Figure 31.2: (a) An observed

adjacency matrix. (b) A possible
SBM representation, darker
cells indicate edges with higher
probability. Finding the most
likely (b) explaining (a) is the
task of SBM-based community
(a) (b) detection.
What we want to do now is to find the SBM model that most
accurately reconstructs the adjacency matrix in Figure 31.1(b). To do
so, we will need to plant a community structure in the SBM model.
If the resulting network is similar to the original one, it is very likely
that the partition we planted corresponds to the “real” partition in
the original graph. As the vignette in Figure 31.2 shows, we do so by 21
Gary King. Unifying political method-
employing the principle of maximum likelihood21 , 22 . I’m going to ology: The likelihood theory of statistical
give you a very incomplete and simplified view of what that means, inference. University of Michigan Press,
1998
prompting you to read more if you’re interested in the topic. 22
Edwin T Jaynes. Probability theory: The
Maximum likelihood estimation means to estimate the parame- logic of science. Cambridge university
ters of a model that are more likely to generate your observed data. press, 2003

Suppose that you have your parameters in a vector θ. The likelihood

function L tells you how good of a job you made using θ to approx-
imate your observed adjacency matrix A. In practice, you’re after
those special parameters θ̂ such that Lθ̂,A is maximum. Mathemati-
cally:

θ̂ = arg max Lθ,A ,

θ ∈Θ

where Θ is the space of all possible parameters, and A is your

given adjacency matrix.
So let’s make an extremely simplified example of what that means
when using SBM to find communities. This sketched example has
a planted partition, only two parameters, and it exclusively focuses
on the simplest definition of community – the one I gave you earlier:
simple non-overlapping assortative communities. I’m imposing these
limitations for pedagogical purposes: in reality, SBMs are much more
powerful than this.
I already mentioned in Section 15.2 why: SBM can find disassorta-
tive communities, communities in which the nodes tend to connect
 graph partitions 421

Figure 31.3: (a) A network with

a disassortative community
structure. (b) A possible disas-
sortative SBM representation,
darker cells indicate edges with
higher probability.

(b)
(a)

to their community mates less than chance. The classical definition

I gave you earlier is only for assortative communities. You can com-
pare the disassortative community structure from Figure 31.3(a)
with the classical assortative one in Figure 31.1(a). SBMs don’t break
a sweat in this scenario: you can simply flip the parameters and –
presto! – you switch from the classical model in Figure 31.2(b) to the
disassortative model in Figure 31.3(b). I haven’t even started explain-
ing you the first community discovery method and we already found
the first hole in the classical definition!
However, let’s take it slow. To understand the SBM basics it is bet-
ter to start with a cartoonishly simplified example and then introduce
the features you can add to SBMs every time it’ll be relevant in the
book.
As mentioned, in the simplest SBM, we only have three param-
eters. The first two are the probabilities of two nodes connecting if
they are – or are not – in the same community: θ1 and θ2 , respectively.
The third (hyper)parameter θ3 is the planted partition: it contains all
node pairs that are in the same community.
Let’s have the simplest possible L function. Let’s define a helper
variable defined for a pair of nodes:



 θ1 − 1, if Auv = 1 and (u, v) ∈ θ3

θ − 1, if A

= 1 and (u, v) 6∈ θ3
2 uv
lθ,A,u,v =
 − θ1 ,

 if Auv = 0 and (u, v) ∈ θ3


−θ , if Auv = 0 and (u, v) 6∈ θ3 .
2

In practice, if two nodes are connected, we subtract one from their

probability of connections in the SBM – since θ1 > θ2 , this means
we reward the case in which we put the two nodes in the same
community. If they are not, we penalize ourselves proportionally to
how sure we were they were connected: in this case we get the lowest
penalty if we assumed them not being in the same community.
Then, we go over every node pair to check whether we assigned
422 the atlas for the aspiring network scientist

a high probability of edge existence to two nodes that were indeed

connected, and we aggregate the scores:

Lθ,A = ∑ lθ,A,u,v .
u,v∈ A

Let’s fix θ1 = 1 and θ2 = 0.01 for simplicity, since we’re only

interested in the partition (θ3 ). If we apply L to the perfect partition,
the one following the colors in Figure 31.1(a), we get Lθ,A = −8.78
(I’m ignoring the diagonal because I don’t allow self loops). Why?
We get zero contribution from the blocks in the diagonal, since
they’re fully connected and also in the same community (case 1
in the formula). We get 4 × (θ2 − 1) contribution from the edges
connecting nodes in different communities (case 2 in the formula)
– note that I don’t double count because I assume the network is
undirected. We get zero contribution from case 3, as there are no
disconnected nodes that we put in the same community. Finally, we
get 482 × −θ2 contribution from the 482 disconnected node pairs in
different communities (case 4).
Any other partition would return a lower likelihood. For instance,
having only two communities, each fully including two blocks, has
(following the four cases): Lθ,A = 0 + (2 × (θ2 − 1)) + (160 × −θ1 ) +
(322 × −θ2 ) = −165.2.
One of the problems of this approach is that the space of all pos-
sible partitions (θ3 ) is huge. How do we know which one is best?
Besides, how do we even know what’s the correct number of commu-
nities? A reliable way to estimate maximum likelihoods in presence
of unknowns such as these, is the Expectation Maximization algo- 23
Arthur P Dempster, Nan M Laird,
and Donald B Rubin. Maximum
rithm23 .
likelihood from incomplete data
Note that here I showed you an intuitive, but cartoonish, likeli- via the em algorithm. Journal of
hood function. This will fail in most, if not all, real world networks. the Royal Statistical Society: Series B
(Methodological), 39(1):1–22, 1977
There are smarter alternatives out there, taking into account the 24
Brian Karrer and Mark EJ Newman.
degree distributions24 , and more that we will see in later chapters. Stochastic blockmodels and community
There are also better heuristics to find the best partition25 . structure in networks. Physical review E,
83(1):016107, 2011
Don’t be deceived by the fact that the SBM approach I just de- 25
Tiago P Peixoto. Efficient monte carlo
scribed is clunky. I introduced it as a historic approach because it and greedy heuristic for the inference of
was the one used by sociologists before the network science renais- stochastic block models. Physical Review
E, 89(1):012804, 2014a
sance of the late nineties. However, as I mentioned, SBMs are more
sophisticated than this.
Many community detection methods use different approaches, and 26
Ulrike Von Luxburg. A tutorial
some can be shown as equivalent to SBMs. One such key algorithm is on spectral clustering. Statistics and
modularity maximization. However, since this is closely related with computing, 17(4):395–416, 2007

evaluating the quality of a partition, I’m going to defer presenting

27
Mark EJ Newman. Finding commu-
nity structure in networks using the
modularity maximization methods to Section 32.1. eigenvectors of matrices. Physical review
Another approach uses the spectrum of the graph26 , 27 . Well- E, 74(3):036104, 2006a
 graph partitions 423

separated communities will show large gaps in the eigenvalues. The

eigenvectors are simply a coordinate system that you can use to
place nodes in a multidimensional space and then use a standard
clustering algorithm to find communities, such as k-means. I already
covered the main concepts for this approach in Section 8.4, so I won’t
go into details here. Hereafter, we explore alternatives to the problem
of discovering communities.

31.2 Random Walks

Figure 31.4: A random walk

in a network with communi-
ties. The node color indicates
to which community a node
belongs. The arrows show each
step of the random walker. The
orange arrows are transitions
between nodes in the same
community. The brown arrow
is a transition between nodes in
different communities.
A common approach for detecting communities looks at network
processes. Random walks (Chapter 8) are a popular choice. These
walks tell us a lot about the community structure of the network.
Consider the example in Figure 31.1(a). In the network, there are
eight “border nodes” with a connection to a different community, out
of 36 nodes. And only one edge out of nine they have points outside 28
Stijn Dongen. A cluster algorithm for
graphs. 2000
the community. So we know that the probability for a random walker 29
Pascal Pons and Matthieu Latapy.
to get out of the community it is visiting is very low. Computing communities in large
Appearing together in a random walk is a strong indication that networks using random walks. J. Graph
Algorithms Appl., 10(2):191–218, 2006
two nodes are in the same community. Figure 31.4 shows why. When
30
Vinko Zlatić, Andrea Gabrielli, and
we perform our random walk, only one step out of the 13 we took Guido Caldarelli. Topologically biased
crossed communities. Let’s do some napkin math. The probability of random walk and community finding
being a “border node” is 8/36 = 0.2̄. The probability of picking the in networks. Physical Review E, 82(6):
066109, 2010
one edge going outside of the community if you are in a border node 31
Venu Satuluri and Srinivasan
is 1/9 = 0.1̄, because the node has degree nine and only one edge Parthasarathy. Scalable graph clustering
using stochastic flows: applications
going out. So the probability of transitioning between communities
to community discovery. In SIGKDD
in a random walk is (8/36) × (1/9) ∼ 0.025. Pretty low! conference, pages 737–746. ACM, 2009
The guiding principle of detecting communities with random 32
E Weinan, Tiejun Li, and Eric Vanden-
walks is that a random walk is likely to be trapped inside a commu- Eijnden. Optimal partition and effective
dynamics of complex networks. Proceed-
nity and to keep visiting nodes belonging to the same community. ings of the National Academy of Sciences,
This has been exploited in a number of different ways28 , 29 , 30 , 31 , 32 . 105(23):7907–7912, 2008
424 the atlas for the aspiring network scientist

These are only a few examples of the many papers using this ap-
proach – you’re going to hear this excuse from me a lot, to save
myself from citing literally everything and making this a book about
community discovery.

101101
1001 110011 Figure 31.5: A binary node ID
schema you’d use to encode
a random walk. Each colored
arrow points to the ID of the
three nodes involved in the
orange walk.

1001101101110011
The most known and best performing of these approaches is usu- 33
Martin Rosvall and Carl T Bergstrom.
ally considered the map equation approach, or Infomap33 , 34 . The Maps of random walks on complex
networks reveal community structure.
map equation is what you use to encode the random walk informa- Proceedings of the National Academy of
tion with the minimum possible number of bits – i.e. minimizing the Sciences, 105(4):1118–1123, 2008
“code length”. Suppose that you give each node a binary ID. Since 34
Ludvig Bohlin, Daniel Edler, Andrea
Lancichinetti, and Martin Rosvall.
we have 36 nodes, we need around 5 bits, but we can save a little if Community detection and visualization
we give shorter codes to central nodes (they are going to be visited of networks with the map equation
more often). Then the cost of describing the random walk is simply framework. In Measuring Scholarly
Impact, pages 3–34. Springer, 2014
the length of the code of the node multiplied by the number of times
we’re going to see it in a random walk, which is given by the station-
ary distribution (Section 8.1). In the example from Figure 31.5, the
orange walk is fully described by the bits in the figure: the node id
sequence.
Infomap saves bits by using community prefixes. Nodes in the
same community get the same prefix. So now we need fewer bits to
uniquely refer to each node, because we can prepend the community
code the first time and then omit it as long as we are in the same
community. Since a community contains, in this case, 9 nodes instead
of 36, we can use shorter codes. We need to add an extra code that
allows us to know we’re jumping out of a community. This is an
overhead, but the assumption here is that a random walker will
spend most of its time in the community, so this community prefix
and jump overhead is rarely used. Figure 31.6 shows this re-labeling
process.
 graph partitions 425

00 01
101 Figure 31.6: The large two-digit
100 101101 1100 codes are the IDs of the com-
1001 110011
munities. Each node gets a
new shorter ID, given that IDs
need to be community-unique,
rather than network-unique.
Now the random walk uses
the prefix (in red) to indicate in
Prefix Node IDs in the walk Jumping out
which community it is, then the
01 1100 101 100 1111 new shorter node IDs (in blue)
10 11 Rarely used! and finally adds an extra ID to
indicate it’s jumping out of a
community (in green).
If the partition is good, we can compress the random walk in-
formation by a lot. Consider the example in Figure 31.7. Without
communities we have no overhead, but we need to fully encode our
36 nodes. The path in orange is simply the sequence of node IDs and
can be stored in 72 bits. If we have community partitions, we add
the community prefixes and the jump overhead (for the community
jump in brown), but the node IDs are shorter. The encoding of the
same walk is 56 bits, and we can see that the overhead parts are tiny
compared with the rest.

00 01 Figure 31.7: The cost of en-

10000 10001 1101 1100
coding a random walk with
10100 100 the naive scheme (left) com-
101101 001
1011001 10101 010 1110 pared with the Infomap scheme
1001 000 (right). In the Infomap scheme,
1111 I underline in red the com-
0000 100
00010 00101 1110 1101 munity prefixes, in green the
001101 1100 inter-community jump over-
00111 010
head, and in blue the node ID
0001100100 000 001 encoding.

10 11
1000010001101001010110110110110011001000000010001101000
Without 11001000011100101
72 bits
With 00110111001001110001010000111110100111011000000010101101 56 bits

If my explanation still makes little sense, you can try out an in-
teractive system showing all the mechanics of the map equation 35
[Link]
approach35 . Infomap has been adapted to numerous scenarios. Many [Link]

involve hierarchical, mutlilayer and overlapping community detec-

426 the atlas for the aspiring network scientist

tion, which we will explore in later chapters. Other modifications 36

Michael T Schaub, Renaud Lambiotte,
include adding some “memory” to the random walkers36 , 37 – effec- and Mauricio Barahona. Encoding
tively using higher order networks (Chapter 30). dynamics for multiscale community
detection: Markov time sweeping for
This means that the walker is not randomly selecting destinations
the map equation. Physical Review E, 86
any more, but it follows a certain logic. Consider a network of flight (2):026112, 2012b
travelers. If you fly from New York to Los Angeles, your next leg trip 37
Martin Rosvall, Alcides V Esquivel,
Andrea Lancichinetti, Jevin D West, and
isn’t random. You’re much more likely, for instance, to come back to Renaud Lambiotte. Memory in network
New York, since you were in LA just for a vacation or visiting family. flows and its effects on spreading
Some approaches do not use vanilla random walks, but also dynamics and community detection.
Nature communications, 5:4630, 2014
consider the information encoded in node attributes in the map 38
Laura M Smith, Linhong Zhu,
equation38 . Kristina Lerman, and Allon G Per-
Since these methods are based on a fundamentally random pro- cus. Partitioning networks with node
attributes by compressing information
cess – random walks – they tend to be non-deterministic. This means
flow. ACM Transactions on Knowledge
that running the same algorithm on the same input twice might Discovery from Data (TKDD), 11(2):15,
return different results. 2016

31.3 Label Percolation

Random walks are running a dynamic process to detect communities,

meaning that we’re performing some sort of event on the network to
uncover the community structure. Another very popular dynamic ap-
proach is having nodes deciding for themselves to which community
they belong by looking at their neighbors’ community assignments.
We use node labels to indicate to which community each node
belongs. We start with a network whose node labels are scattered
randomly. Then each node looks at its neighbors and adopts the
most common labels it sees (if there is a tie, it will choose a random
one among the most popular). As a result, the labels will percolate
through the network until we reach a state in which no more signif-
icant changes can happen. The assumption is that, in a community,
nodes will end up surrounded by nodes with the same label. That is
why this class of solutions is usually know as “label percolation” (or
propagation).
At the beginning, nodes will switch their labels randomly. How-
ever, by chance, some nodes will eventually adopt the same label. If
they are in the same community, all of a sudden this label is the only
one with two nodes in the cluster. It starts becoming the majority
label for many nodes in the cluster, and thus it will be eventually
adopted by everybody, as Figure 31.8 shows.
This approach is fairly straightforward and computationally
simple. In fact, one of the claims to fame of such an algorithm is its
time complexity: it runs linearly in terms on the number of edges in
the network. You just have to iterate over your edge list a few times
before convergence.
 graph partitions 427

Figure 31.8: (a) Starting condi-

tion of label propagation algo-
rithm, with labels distributed
randomly in the network. (b)
After some iterations, randomly,
some neighbors will adopt the
same label. The highlighted
pink nodes will take over the
community at the next time
(a) (b) step.

The most important dimension along which the many papers

implementing label propagation community detection differ is in the
strategy they employ for the nodes to look around their neighbor- 39
Usha Nandini Raghavan, Réka Albert,
hood. We can classify them in three classes: asynchronous39 – which and Soundar Kumara. Near linear
is also the original formulation of the label propagation principle –, time algorithm to detect community
structures in large-scale networks.
semi-synchronous40 , and synchrnous41 .
Physical review E, 76(3):036106, 2007
The asynchronous case uses the labels that the nodes had at the 40
Gennaro Cordasco and Luisa
previous iteration. For instance, at the ith iteration a node will decide Gargano. Community detection
via semi-synchronous label propagation
which label to adopt by looking at the majority label between its
algorithms. In 2010 IEEE International
neighbors at the (i − 1)th iteration. If some neighbors have, in the Workshop on: Business Applications of
mean time, updated their label, this information is ignored, and will Social Network Analysis (BASNA), pages
1–8. IEEE, 2010
be used only at the (i + 1)th iteration. 41
Jierui Xie, Boleslaw K Szymanski,
In the synchronous approach this is not the case: you always use and Xiaoming Liu. Slpa: Uncovering
the most up-to-date information you have. If some of your neighbors overlapping communities in social net-
works via a speaker-listener interaction
already changed their label, you look the their ith iteration label. dynamic process. In 2011 IEEE 11th
Otherwise, you look at their (i − 1)th iteration label. The semi- International Conference on Data Mining
Workshops, pages 344–349. IEEE, 2011
synchronous case is, as you might expect, a combination of the
two.
Note that, just like in the random walk case, the label propagation
algorithms are typically non-deterministic. The random choices
nodes make when breaking ties among the most popular labels
around them can lead to differences in the detected communities.

31.4 Temporal Communities

Evolutionary Clustering

So far I’ve framed the community discovery problem as essentially

static. You have a network and you want to divide it into densely
connected groups. However, we saw in Section 4.4 that many of the
graphs we see are views of a specific moment in time. Networks
evolve and you might want to take that information into account.
428 the atlas for the aspiring network scientist

42
Qing Cai, Lijia Ma, Maoguo Gong,
A couple of good review works42 , 43 focus on dynamic community and Dayong Tian. A survey on network
discovery and can help you obtaining a deeper understanding of this community detection based on evolu-
tionary computation. IJBIC, 8(2):84–98,
problem. Let’s explore what can happen to your communities over
2016
time.

Time
Figure 31.9: Two things that can
happen to your communities in
an evolving network: growing
Grow and shrinking.

43
Giulio Rossetti and Rémy Cazabet.
Community discovery in dynamic
networks: a survey. ACM Computing
Surveys (CSUR), 51(2):35, 2018
Shrink

One possibility is that the community will grow: it will attract new
nodes that were previously unobserved. The other side of the coin is
shrinking: nodes that were part of the community disappear from the
network. Figure 31.9 shows visual examples of these events.

Time
Figure 31.10: Two things that
can happen to your commu-
nities in an evolving network:
Merge merging and splitting.

Split

Another way for a community to grow is by merging with other

communities. The difference with the previous case is that in the
growth case the added nodes to the community were not previously
observed. In this case they were, and they were classified in a differ-
ent community. “Grow” happens with the addition of nodes, while
“merge” usually happen with the addition of edges. Again, we can
have the opposite case: a community which splits into two or more
new communities, due to the loss of edges (rather than nodes, as
it was the case in the “shrink” scenario). Figure 31.10 shows visual
 graph partitions 429

Time
Figure 31.11: Two things that
can happen to your commu-
nities in an evolving network:
Birth birth and death.

Death

examples of these events.

Communities can also arise from nothing. This is like “grow”, ex-
cept that none of the nodes forming the community were previously
observed in the network. The converse is community death: every
node which was part of it disappears from the network. Figure 31.11
shows visual examples of these events.
Note that, as everything else in this chapter, also this description
is cartoonish. What happens in real world networks is way messier.
Communities grow, shrink, split, and merge at the same time. Merges
could also be partial, with communities “stealing” nodes from each
other, resulting in an evolution that is not nearly as neat as the one
I depicted here. Don’t assume that you’re going to be able to say,
unequivocally, something like “community C split in C1 and C2 at
time t”.
How do you detect communities in an evolving graph? One 44
John Hopcroft, Omar Khan, Brian
possible approach could be to define a series of network snapshots, Kulis, and Bart Selman. Tracking
evolving communities in large linked
apply a community discovery algorithm to each of these snapshots, networks. Proceedings of the National
and then combine the results into a single clustering44 , 45 , 46 . This is Academy of Sciences, 101(suppl 1):5249–
5253, 2004
fine in some scenarios, but is generally not advisable, as it makes the 45
Sitaram Asur, Srinivasan
quiet assumption that snapshots are sort of independent. Parthasarathy, and Duygu Ucar. An
It becomes challenging to link the community discovery of each event-based framework for charac-
terizing the evolutionary behavior of
snapshot to the next. For each community at time t you’re trying
interaction graphs. ACM Transactions on
to find the community at time t + 1 that is the most similar to it, Knowledge Discovery from Data (TKDD),
and say that the most recent community is an evolution of the older 3(4):16, 2009
46
Gergely Palla, Albert-László Barabási,
one. A possible similarity criterion would be calculating the Jaccard and Tamás Vicsek. Quantifying social
coefficient. group evolution. Nature, 446(7136):664,
A better solution is performing evolutionary clustering47 . This 2007
47
Deepayan Chakrabarti, Ravi Kumar,
means that we add a second term to whatever criterion we use to and Andrew Tomkins. Evolutionary
find communities in a snapshot – a procedure sometimes called clustering. In Proceedings of the 12th
“smoothing”. Suppose you’re using Infomap. The aim of the algo- ACM SIGKDD international conference
on Knowledge discovery and data mining,
rithm, as I presented earlier, is to encode random walkers with the pages 554–560. ACM, 2006
430 the atlas for the aspiring network scientist

lowest number of bits. Let’s say this is its quality function – which is
known as code length (CL).
In evolutionary clustering you don’t just optimize CL. You have
CL as a term in your more general quality function Q. The other
term in Q is consistency. For simplicity sake, let’s just assume it is
some sort of Jaccard coefficient between the partitions at time t and
the partition at time t − 1. To sum up, a very simple evolutionary
clustering evaluates the partition pt at time t as:

Q pt = αCL pt + (1 − α) J pt ,pt−1 .

16
16 10 16
10 10
11 13 14 Figure 31.12: (a) The commu-
13 14 9 13 14
9 11 9 11
15
nity partition of a graph at time
15 12 15
12 12
t. (b) A partition of the graph at
6
time t + 1 exclusively optimizing
3 6 3 3 6
the code length, using Infomap.
2 4 5 7 2 4 5 7 2 4 5 7
(c) A partition of the graph at
1 8 1 8 1 8
time t + 1 balancing a good code
(a) (b) (c) length and consistency with the
Here, α is a parameter you can specify which regulates how much partition at time t + 1.
weight you want to give to your previous partitions. For α = 1 you
have standard clustering, while for α = 0 the new information is
discarded and you only use the partition you found at the previous
time step. Figure 31.12 shows you that maximizing CL pt might yield
significantly different results than maximizing a temporally-aware
Q pt function.
This is only one – the simplest – of the many ways to perform
smoothing, which the other review works I cited describe more
in details. However, all these methods (and the ones that follow)
have something in common: they are all at odds with the classical
definition of community that I gave you earlier. That is because,
at time t + 1, we’re not simply trying to group nodes in the same
community according to the density of their connections. Eventually,
we’re going to end up with a partition with many edges running
between communities, which is against the traditional definition of
community. Together with the ability of SBMs to find disassortative 48
Mark Goldberg, Malik Magdon-
communities, these are yet more cracks appearing in the classical Ismail, Srinivas Nambirajan, and James
Thompson. Tracking and predicting
community detection assumption of assortative communities.
evolution of social communities. In
Smoothing is not necessarily applied to adjacent snapshots: you SocialCom, pages 780–783. IEEE, 2011
can have a longer memory looking at t − 2, t − 3, and so on48 , 49 . In 49
Matteo Morini, Patrick Flandrin,
alternative approaches, you can skip the smoothing altogether. You Eric Fleury, Tommaso Venturini, and
Pablo Jensen. Revealing evolutions in
can identify a “core-node” which is the center of the community and dynamical networks. arXiv preprint
will identify it for all snapshots. You then find communities around arXiv:1707.02114, 2017
 graph partitions 431

50
Zhengzhang Chen, Kevin A Wilson,
that node50 , 51 . Ye Jin, William Hendrix, and Nagiza F
Samatova. Detecting and tracking
community dynamics in evolutionary
Other Approaches networks. In ICDMW, pages 318–327.
IEEE, 2010
Alternatives to evolutionary clustering exist. You could find an
51
Yi Wang, Bin Wu, and Xin Pei. Comm-
tracker: A core-based algorithm of
optimal partition only for the very first snapshot of your network. tracking community evolution. In
As you receive a new snapshot, rather that starting from scratch ADMA, pages 229–240. Springer, 2008b
and then smoothing, you can adapt the old communities to the new 52
K Miller and Tina Eliassi-Rad. Contin-
network, whether you do it via global optimization52 , or using a uous time group discovery in dynamic
graphs. Technical report, LLNL, 2010
specific set of rules to update the old communities53 , 54 , 55 . 53
Giulio Rossetti, Luca Pappalardo,
Another approach consists in defining a dynamic null model: a Dino Pedreschi, and Fosca Giannotti.
null version of your evolving network which has no communities56 , Tiles: an online algorithm for com-
munity discovery in dynamic social
much like a random graph. Then you look at deviations from this ex-
networks. Machine Learning, 106(8):
pected null model in the network as the potential sources of dynamic 1213–1241, 2017
communities. 54
Yizhou Sun, Jie Tang, Jiawei Han,
Manish Gupta, and Bo Zhao. Commu-
You can use SBMs in this case as well – you can use SBMs for
nity evolution detection in dynamic
any case, really. SBMs tend to be more principled than evolutionary heterogeneous information networks. In
clustering approaches, because they model temporal communities MLGraphs, pages 137–146. ACM, 2010
55
Lei Tang, Huan Liu, Jianping Zhang,
directly57 , rather than chasing communities around as your network
and Zohreh Nazeri. Community
evolves. As an example, you could use a tensor representation in evolution in dynamic multi-mode
which each slice of the tensor is a snapshot of the network. Since a networks. In SIGKDD, pages 677–685.
ACM, 2008
tensor is nothing more than a high dimensional matrix, and SBMs 56
Danielle S Bassett, Mason A Porter,
understand matrices, you can make a tensor-SBM58 . One nice thing Nicholas F Wymbs, Scott T Grafton,
about the SBM approach is that it allows to estimate from data the Jean M Carlson, and Peter J Mucha. Ro-
bust detection of dynamic community
timescale at which the community structure changes – which is structure in networks. Chaos Journal, 23
inferred by the coupling between snapshots. This is nice because then (1):013142, 2013
you don’t need to decide yourself the granularity of the temporal
57
Tiago P Peixoto. Inferring the
mesoscale structure of layered, edge-
observation. Basing your inferences on data rather than taking a valued, and time-varying networks.
guess is always a plus! Physical Review E, 92(4):042807, 2015
A final approach is not to consider the different snapshots as
58
Marc Tarrés-Deulofeu, Antonia
Godoy-Lorite, Roger Guimera, and
separate, but taking the entire structure of the network as input all at Marta Sales-Pardo. Tensorial and bi-
once, as if it were a single structure. For instance, you can split each partite block models for link prediction
in layered networks and temporal net-
node v into many meta-nodes vt1 , vt2 , ... connected to each other by works. Physical Review E, 99(3):032307,
special edges59 , 60 , 61 , 62 , 63 . This is similar to performing multi-layer 2019
community discovery, which we’ll see later. 59
Jimeng Sun, Christos Faloutsos,
Spiros Papadimitriou, and Philip S Yu.
Graphscope: parameter-free mining of
large time-evolving graphs. In SIGKDD,
31.5 Local Communities pages 687–696. ACM, 2007a
60
Laetitia Gauvin, André Panisson, and
In some cases, you are not interested in grouping every node into Ciro Cattuto. Detecting the community
a community. I’m not just referring to allowing nodes to be part of structure and activity patterns of
temporal networks: a non-negative
no communities – a feature included in many algorithms, regardless tensor factorization approach. PloS one, 9
of their guiding principle. Sometimes, you want to find local com- (1):e86028, 2014
munities: you’re interested in knowing the communities around a
specific (set of) node(s), regardless of the rest of the network. This
432 the atlas for the aspiring network scientist

makes sense if the network is very large and some nodes are just too 61
Leto Peel and Aaron Clauset. De-
far to ever influence the results on your specific objectives. Or you tecting change points in the large-scale
structure of evolving networks. In
cannot analyze it fully because it would take too much memory. Or it Twenty-Ninth AAAI Conference on Artifi-
might take too much time to access the entire network, imposing you cial Intelligence, 2015
to sample it (see Chapter 25).
62
Amir Ghasemian, Pan Zhang, Aaron
Clauset, Cristopher Moore, and Leto
This is usually done by exploring the graph one node at a time, Peel. Detectability thresholds and
putting nodes into different bins according to their exploration status optimal algorithms for community
structure in dynamic networks. Physical
– and their community affiliation. For instance, you start from a seed Review X, 6(3):031005, 2016
node v0 , which by definition is part of your local community C . All of 63
Tiphaine Viard, Matthieu Latapy, and
its neighbors are part of the unexplored node set U . Clémence Magnien. Computing maxi-
mal cliques in link streams. Theoretical
Computer Science, 609:245–252, 2016

Figure 31.13: The process of

discovering local communities.
Red: nodes in C . Blue: nodes in
U . Green: nodes maximizing
the number of edges in C and
therefore being added to the
community.
(a) (b) (c) (d) 64
Aaron Clauset. Finding local com-
munity structure in networks. Physical
review E, 72(2):026132, 2005
You iterate over all members of U , trying to find the one that 65
James P Bagrow. Evaluating local
community methods in networks.
would maximize some community quality function. For instance, it
Journal of Statistical Mechanics: Theory
could be simply the number of edges connecting inside C – with ties and Experiment, 2008(05):P05001, 2008
broken randomly. We add to C the v1 node with the most edges to 66
Feng Luo, James Z Wang, and Eric
Promislow. Exploring local community
the local community. Then, U is updated with the new neighbors, the
structures in large networks. Web
ones v1 has but v0 did not. Intelligence and Agent Systems: An
We continue until we have reached our limit: we explored the International Journal, 6(4):387–400, 2008
67
Symeon Papadopoulos, Andre Skusa,
number of nodes we wanted to test, or we ran out of time, or we
Athena Vakali, Yiannis Kompatsiaris,
actually explored all nodes in the component to which v0 belongs. and Nadine Wagner. Bridge bounding: A
Figure 31.13 shows some steps of the process. As you can see, we local approach for efficient community
discovery in complex networks. arXiv
can terminate after we explore a certain set of nodes. At that point, preprint arXiv:0902.0871, 2009
we detected the local community of node v0 , without exploring the 68
Lucas GS Jeub, Prakash Balachandran,
entire network. We did not explore the blue nodes – although we Mason A Porter, Peter J Mucha, and
Michael W Mahoney. Think locally, act
know they exists – and we’re absolutely clueless about the existence locally: Detection of small, medium-
of the grey nodes. sized, and large communities in large
networks. Physical Review E, 91(1):012821,
The algorithm I just described is one64 of the many possible65 , 66 , 67 .
2015
All these algorithms are variation of this exploration approach. More 69
Pasquale De Meo, Emilio Ferrara,
alternatives have been proposed, for instance using random walks Giacomo Fiumara, and Alessandro
Provetti. Mixing local and global
like Infomap to explore the network68 , 69 . You can explore the liter-
information for community detection in
ature more in depth using one of the survey papers I cited at the large networks. Journal of Computer and
beginning of the chapter. System Sciences, 80(1):72–87, 2014
 graph partitions 433

31.6 Using Clustering Algorithms

I barely started scratching the complex landscape of different ap-

proaches to community discovery. We’re going to have time in the
next chapters to explore even more variations. However, a ques-
tion might have already dawned on you. If there are such different
approaches to detecting communities, how do I find the one that works for
me? And, how do I maximize my chances of finding high quality communi-
ties? As you might expect, the answers to these questions are difficult
and often tend to be subjective.
Let’s start from the second one: designing a strategy to ensure you
find close-to-optimal communities. In machine learning, we discov-
ered a surprising lesson. If you want to improve accuracy, designing
the most sophisticated method in the world usually helps only up
to a certain point. Having many simple methods and averaging the
results could potentially yield better results.
This observation is at the basis of what we know as “consensus 70
Alexander Strehl and Joydeep Ghosh.
clustering” (or ensemble clustering)70 . This strategy has been applied Cluster ensembles—a knowledge reuse
to detecting communities71 in the way you’d expect. Take a network, framework for combining multiple
partitions. Journal of machine learning
run many community discovery algorithms on it, average the results. research, 3(Dec):583–617, 2002
Figure 31.14 shows an example of the procedure. Note how none of 71
Andrea Lancichinetti and Santo
the methods (Figure 31.14(a-c)) found the best communities, which Fortunato. Consensus clustering in
complex networks. Scientific reports, 2:
is their consensus: Figure 31.14(d). Note also how the third method 336, 2012
finds rather absurd and long stretched sub-optimal communities.
However, its evident blunders are easily overruled by the consensus
between the other two methods, and its tiebreaker improves the
overall partition.

(a) Method #1 (b) Method #2 (c) Method #3 (d) Consensus

Figure 31.14: An example of
This is a fine strategy, but you should not apply it blindly. You
consensus clustering. Node
have to make sure the ensemble of community detection algorithms
color represents the commu-
you’re considering is internally consistent. In particular, the methods
nity. (a-c) The results of three
should have a coherent and compatible definition of what a commu-
independent methods. (d) Their
nity is. Limiting ourselves to the small perspective on the problem
majority vote.
from this chapter – ignoring all that is coming next – combining a dy-
namic community discovery with a static local community discovery
434 the atlas for the aspiring network scientist

will probably not help.

Even mashing together superficially similar algorithms might
result in disaster. For instance, the flow-based communities Infomap
returns aren’t extremely compatible with the density optimization
algorithms we’ll see in the next chapter, even if the community
definitions on which they are based don’t look too dissimilar.
So, how do you go about choosing which algorithms to include 72
Michele Coscia. Discovering com-
in your ensemble? In a paper of mine72 , I explore the relationship munities of community discovery. In
between around 70 algorithms, comparing how similar their result- Proceedings of the 2019 IEEE/ACM Inter-
national Conference on Advances in Social
ing partitions are. This results in an algorithm similarity network,
Networks Analysis and Mining, pages 1–8,
which has distinct communities: groups of algorithms returning po- 2019
tentially interchangeable results that are significantly different from
algorithms in a different group.
Figure 31.15 shows the result. Note that, in there, I label each
algorithm with a tag. Chapter 46 contains a map from the tag to a
resource where to recover the specific algorithm.
Now what? Well, we could... aehm... detect... communities... on
this – I love being meta. These communities of community discovery
can drive you in choosing your ensemble set. To find them, I use a
version of Infomap allowing nodes to be part of multiple commu-
nities. This is the so-called overlapping community discovery, the
topic of Chapter 34. Most algorithms allowing overlapping commu-
nities are in the red community in the figure – along with the local
clustering approach I presented in Section 31.5.
The blue community contains Infomap and the label propagation
approaches I explained earlier (Sections 31.2 and 31.3). This shows
the close relationship between the two. The purple community
includes methods departing from the classical “internal density”
definition, to use a “neighbor similarity” approach. These will be
covered in detail in Section 35.4.
A very popular approach is using a quality measure to evalu-
ate how good a partition is, and then finding a smart strategy to
optimize it. Most methods applying this strategy are in the green
community, which is the one we’re exploring in the next chapter.
Before moving to it, let me highlight an important lesson that
we learn from this algorithm similarity network. Its communities
are well-defined. This means that there are different and mutually
exclusive notions of what a community is. This is yet another proof
that the naive definition of community commonly accepted without
criticism must be only one of the many possible. In fact, we can go
deeper than this. The notion that there is a golden partition of the
network is a utopia. As I mentioned at the beginning of the chapter,
community discovery is more useful than that: it decomposes a
network and simplifies it. Since there are innumerable ways – and
 graph partitions 435

collapsed-sbm
birch
ward
agglomerative kmeans
dbscan mixnet bnmtf
crossass pmm

affinity ocg
meanshift kerlin

spectral conclude

kclique code-dense
hrg
vbmod
demon

moses bridgeboundm m s b
fluid
ganxis cme-td oslom leadeig

labelperc
extr
tiles ganet
spinglass
gce conga
agm metis

ilcd moganet
infocentr
edgebetween tabu louvain
bigclam
edgeclust copra
savi cme-bu
svinet lwplocal
fuzzyclust netcarto
slpa

hlc ganet+
linecomms
mlrmcl
rmcl
infomap graclus m s g cliquemod
olc infomap-overlap mcl fastgreedy
bagrowlocal
peacock walktrap
graclus2stage vm

clausetlocal

Figure 31.15: The community

reasons why – to simplify a network, then there are innumerable detection algorithm similarity
approaches to community discovery, and they are all valid even if network. Each node is a com-
they don’t chase the mythical golden pot of “true” communities at munity discovery algorithm.
the end of the rainbow. They are connected if the two
algorithms return similar par-
31.7 Summary titions. I use the node color
to represent the algorithm’s
1. According to the classical definition, communities in complex community. Multicolored nodes
networks are groups of nodes densely connected to each other and belong to multiple communi-
sparsely connected to the rest of the network. They are one of the ties.
most common mesoscale organizations of real world networks.

2. One of the oldest approaches in community detection is to assume

a planted partition of nodes in the network and then finding
the distributions of nodes in communities that has the highest
likelihood of explaining the observed connections.

3. A random walker would tend to be trapped in a community,

because most of the neighbors of a node are part of its same
community. By the same principle, we can detect communities by
letting nodes assume the community label that is most common
among their neighbors.

4. Networks evolve and so do communities. One can track the evo-

436 the atlas for the aspiring network scientist

lution of communities by having a two-part community quality

function. One part tells us how well we’re partitioning the net-
work, the other tells us how compatible the new communities are
with the old ones.

5. Sometimes your input network is too big or you have no interest

in partitioning all of it. In that case, you can perform local commu-
nity detection, detecting communities only in the neighborhoods
of one or more query nodes.

6. There are hundreds of community detection algorithms. To choose

one, you need to know what type of communities it returns. Alter-
natively, you can perform ensemble clustering, averaging out the
results of multiple algorithms.

7. The classical community definition is assortative. Disassortative

communities can exist, where nodes don’t like to connect to mem-
bers of the same group. Temporal communities are also not always
assortative. This shows that there are more types of communities
than the one assumed by the classical definition and they are all
valid objectives you can follow to simplify your network.

31.8 Exercises

1. Find the communities in the network at [Link]

eu/exercises/31/1/[Link] using the label propagation strategy.
Which nodes are in the same community as node 1?

2. Find the local communities in the same network using the same
algorithm, by only looking at the 2-step neighborhood of nodes 1,
21, and 181.

3. Suppose that the network at [Link]

exercises/31/3/[Link] is a second observation of the previ-
ous network. Perform the label propagation community detection
on it and use the Jaccard coefficient to determine how different the
communities containing nodes 1, 21, and 181 are.
32
Community Evaluation

How do you know if you found a good partition of nodes into com-
munities? Or, if you have two competing partitions, how do you
decide which is best? In this chapter, I present to you a battery of
functions you can use to solve this problem. Why a “battery” of func-
tions? Doesn’t “best” imply that there is some sort of ideal partition?
Not really. What’s “best” depends on what you want to use your
communities for. Different functions privilege different applications.
So we need a quality function per application and you need to care-
fully choose your evaluation strategy to match the problem definition
you’re trying to solve with your communities.
Think about “evaluating your communities” more as a data ex-
ploration task than a quest to find the ultimate truth. Since there is
no one True partition – and not even one True definition of commu-
nity as I suggested in the previous chapter –, there also cannot be
one True quality function. You have, instead, multiple ways to see
different kinds of communities, some of which might be more or less
useful given the network you have and the task you want to perform.
In the first two sections, I start by focusing on functions that only
take into account the topological information of your network. In this
case, the only thing that matters are the nodes and edges – at most
we can consider the direction and/or the weight of an edge.
In the latter two sections I move to a different perspective. First,
we consider the network as essentially dynamic and we use commu-
nities as clues as to which links will appear next, under the assump-
tion the communities tend to densify: it is much more likely that a
new link will appear between nodes in the same community. Finally,
we look at metadata that could be attached to nodes, which might be
providing some sort of “ground truth” for the actual communities in
which nodes are grouped into in the real world.
438 the atlas for the aspiring network scientist

32.1 Modularity

As a Quality Measure
When it comes to functions evaluating the goodness of a commu-
nity partition using exclusively topological information, there is one 1
Mark EJ Newman. Modularity and
undisputed queen: modularity1 . You shouldn’t be fooled by its popu- community structure in networks.
larity: modularity has severe known issues that limits its usefulness. Proceedings of the national academy of
sciences, 103(23):8577–8582, 2006b
We’ll get to those in the second half of this section.
Modularity is a measure following closely the classical definition
of community discovery. It is all about the internal density of your
communities. However, you cannot simply maximize internal density,
as the partition with the highest possible density is a degenerate one,
where you simply have one community per edge – two connected
nodes have, by definition, a density of one.

Figure 32.1: (a) A network

with a community structure.
The node colors represent the
community partition. (b) A
configuration model of (a), pre-
serving the number of nodes,
edges, and degree distribution.
The blue outline identifies the
nodes that were grouped in the
(b) blue community in (a).
(a)
Modularity solves this issue by comparing the observed network
with a random expectation. For instance, consider the network in
Figure 32.1(a). If we were to create a randomized version of it, it’d
look like the graph in Figure 32.1(b). In Figure 32.1(b), each node
has the very same degree that it has on the left. However, the edges
are shuffled around. It is clear that this random network has no
community structure. The difference between the two networks is
that the communities of nine nodes have many more links inside
them that any grouping of nine nodes in Figure 32.1(b).
At an abstract level, modularity is the comparison between the
observed number of edges inside a community and the expected
number of edges. The expectation is based on a null model of a ran-
dom graph with the same degree distribution as the observed graph
(i.e. a configuration model, Section 15.1). In Figure 32.1, we see that
this number is positive: there are more edges in the community struc-
ture network than in its randomized version. The blue community
in Figure 32.1(a) contains 36 edges – all communities in the figure
 community evaluation 439

do. Picking those nodes from Figure 32.1(b) results in finding only 17
edges among them.
The domain of the modularity function is thus defined between
+1 and −0.5, as Figure 32.2 shows. A positive modularity happens
when our partition finds nodes whose number of edges exceeds null
expectation. When expectation exactly matches the number of edges
in our community partition, modularity is zero. You can achieve
negative modularity by trying to group nodes together that connect
to each other less than chance. This can be a reasonable scenario: for
instance, if you have disassortative communities (see Section 26.2).
Note that, in the leftmost graph in Figure 32.2, nodes of the same
color do not connect with each other.

Disassortative community structure 0 Strong community structure

-0.5 No community structure +1

Figure 32.2: The domain of
The question is: how do we build this expectation, mathemat-
modularity, with example par-
ically? This boils down to estimating the connection probability
titions returning a given value,
between any two nodes u and v. If this were a true random graph
from −0.5 (disassortative com-
(Gn,p ) it’d be easy: the connection probability is p = 2| E|/|V |. But
munities) to +1 (assortative
we have the constraint of keeping the degree distribution. Each node
communities, the most common
v has a number of connection opportunities equal to its degree. The
case), passing via 0 (random
number of possible wirings we can make in the network is twice
graph with no communities).
the number of edges. In a configuration model, the probability of
connecting u and v is (k u k v )/2| E|.
Now we have all we need to build the modularity formulation:
 
1 kv ku
M=
2| E | ∑ Auv −
2| E |
δ ( c v , c u ),
u,v∈V
where A is our adjacency matrix, and δ is the Kronecker delta: a
function return one if u and v are in the same community (cu = cv ),
zero otherwise.
440 the atlas for the aspiring network scientist

Modularity’s formula is scary looking, but it ought not to be. In

fact, it’s crystal clear. Let me rewrite it to give you further guidance:

 
1 kv ku
M=
2| E | ∑ Auv −
2| E |
δ ( c v , c u ),
u,v∈V

which translates into: for every pair of nodes in the same commu-
nity subtract from their observed relation the expected number of
relations given the degree of the two nodes and the total number of
edges in the network, then normalize so that the maximum is 1.
Modularity and Stochastic Blockmodels are related. Optimizing
the community partition following modularity is proven to be equiv-
2
Mark EJ Newman. Equivalence
between modularity optimization and
alent to a special restricted version of SBM2 . Specifically, you need maximum likelihood methods for
to use the degree-correlated SBM – since it fixes the degree distribu- community detection. Physical Review E,
94(5):052315, 2016a
tion just like the configuration model does (which is the null model
on which modularity is defined). Then, you must fix pin and pout
– the probabilities of connecting to nodes inside and outside their
community – to be the same for all nodes.
In general, you can use both to evaluate the quality of your par-
tition, but there are subtle differences. SBM is by nature generative:
it gives you connection probabilities between your nodes. Modu-
larity doesn’t. On the other hand, modularity has this inherent test
against a null graph which you don’t really have in SBMs. In fact,
you can easily extend modularity in such way that you can talk about
3
Brian Karrer, Elizaveta Levina, and
a statistically significant community partition, one that is sufficiently Mark EJ Newman. Robustness of
different from chance3 . community structure in networks.
Physical review E, 77(4):046119, 2008

Figure 32.3: A network with a

community structure. The node
colors represent the community
partition. (a) Optimal partition.
(b) Sub optimal partition. (c)
Partition grouping all nodes in
the same community.
(a) M = 0.723 (b) M = 0.411 (c) M = 0

Modularity also gives us an intuition about whether a partition

is better than another, without the need of calculating the likeli-
hood, which is a more generic tool that was not developed with
networks in mind. We can compare partitions and see that a higher
modularity implies a better partition, as Figure 32.3 shows. Moving
nodes outside the optimal partition lowers modularity (compare the
scores in the captions of Figures 32.3(a) and 32.3(b)). If we do not do
community discovery and return a single partition (Figure 32.3(c)),
modularity will be equal to zero.
 community evaluation 441

As a Maximization Target
As I mentioned earlier, modularity can be used in two ways. So far,
we’ve seen the use case of evaluating your partitions. You start from
a graph, you try two algorithms (or the same algorithm twice) and
you get two partitions. The one with the highest modularity is the
preferred one – see Figure 32.4(a).

Figure 32.4: (a) The workflow of

using modularity as a quality
criterion for your partitions. (b)
The workflow of using modu-
larity as an optimization target
CD1 CD2 Merge best pair to find the best community
Yes partition.
M increases? 4
Mark EJ Newman. Fast algorithm
No
for detecting community structure in
networks. Physical review E, 69(6):066133,
2004c
5
Aaron Clauset, Mark EJ Newman, and
M(CD1) > M(CD2) Cristopher Moore. Finding community
CD1 is better than CD2 structure in very large networks. Physical
(b) review E, 70(6):066111, 2004
(a) 6
Alex Arenas, Jordi Duch, Alberto
The alternative is to directly optimize it: to modify your partition Fernández, and Sergio Gómez. Size
reduction of complex networks preserv-
in a smart way so that you’ll get the highest possible modularity
ing modularity. New Journal of Physics, 9
score – see Figure 32.4(b). For instance, your algorithm could start (6):176, 2007
with all nodes in a different community. You identify the node pair 7
Azadeh Nematzadeh, Emilio Ferrara,
Alessandro Flammini, and Yong-Yeol
which would contribute the most to modularity and you merge it in
Ahn. Optimal network modularity for
the same community. You repeat the process until you cannot find information diffusion. Physical review
any community pair whose merging would improve modularity4 , 5 . letters, 113(8):088701, 2014
8
Clara Pizzuti. Ga-net: A genetic
Most approaches following this strategy return hierarchical commu-
algorithm for community detection
nities, recursively including low level ones in high level ones, and I in social networks. In International
cover them in detail in Chapter 33. conference on parallel problem solving from
nature, pages 1081–1090. Springer, 2008
But there are other ways to optimize modularity. One strategy 9
Clara Pizzuti. A multiobjective genetic
is to progressively condense your network such that you preserve algorithm to find communities in
its modularity6 . Or using modularity to optimize the encoding of complex networks. IEEE Transactions on
Evolutionary Computation, 16(3):418–430,
information flow in the network, bringing it close to the Infomap 2012
philosophy7 . Another approach is using genetic algorithms8 , 9 or 10
Jordi Duch and Alex Arenas. Com-
munity detection in complex networks
extremal optimization10 : an optimization technique similar to genetic
using extremal optimization. Physical
algorithms, which optimizes a single solution rather than having a review E, 72(2):027104, 2005
pool of potential ones. 11
Bowen Yan and Steve Gregory.
Detecting communities in networks
Other approaches include, but are not limited to:
by merging cliques. In 2009 IEEE
International Conference on Intelligent
• Progressively merging cliques, under the assumption that a clique Computing and Intelligent Systems,
is a structure that has the highest possible modularity11 ; volume 1, pages 832–836. IEEE, 2009
442 the atlas for the aspiring network scientist

• Performing the merging of communities I describe earlier allowing

multiple communities to merge at the same time and then refining 12
Philipp Schuetz and Amedeo Caflisch.
the results by allowing single nodes to move at the end12 ; Efficient modularity optimization by
multistep greedy algorithm and vertex
• Using simualted annealing13 , integrated by using spinglass dy- mover refinement. Physical Review E, 77
namics14 . This technique can also take into account whether your (4):046112, 2008
network is signed15 – i.e. it has positive and negative connections
13
Roger Guimera and Luis A Nunes
Amaral. Functional cartography of
(see Section 4.2), a special and simpler case of multilayer commu- complex metabolic networks. nature, 433
nity discovery, which I’ll cover in details in Chapter 36; (7028):895, 2005
14
Jörg Reichardt and Stefan Bornholdt.
• Using Tabu search, another optimization technique related to simu- Statistical mechanics of community
detection. Physical Review E, 74(1):016110,
lated annealing and working mostly using local information16 ;
2006
15
Vincent A Traag and Jeroen Brugge-
• Even including geospatial terms in the definition, when your
man. Community detection in networks
nodes live into an actual geometric space17 . with positive and negative links. Physical
Review E, 80(3):036115, 2009
16
Alex Arenas, Alberto Fernandez, and
As you can gather from the number of references, we network Sergio Gomez. Analysis of the structure
folks really like to optimize our modularities. of complex networks at different
resolution levels. New journal of physics,
10(5):053039, 2008b
Expanding Modularity
17
Paul Expert, Tim S Evans, Vincent D
Blondel, and Renaud Lambiotte. Uncov-
ering space-independent communities
Unfortunately, modularity is not the end-all be-all of community in spatial networks. Proceedings of the
detection as it initially appeared to be. There are several issues with National Academy of Sciences, 108(19):
it. We start by looking at the less problematic – but still annoying – 7663–7668, 2011

ones. If you go back to the formula, you’ll recognize that it is a little

bit too simple.
The standard definition of modularity works exclusively with
undirected, unweighted, disjoint partitions. We’ll take care about
extending modularity to cover the overlapping case in Chapter 34.
For now, let’s see what we can do when our graphs have directed
edges and/or weighted ones.
The most straightforward way to extend modularity when your 18
Elizabeth A Leicht and Mark EJ
graphs have directed edges is simply modifying your expected Newman. Community structure in
directed networks. Physical review letters,
number of edges between two nodes18 . If in the undirected case we 100(11):118703, 2008
simply used the degree for both nodes u and v, now we have to use
their in- and out-degree alternatively. So the expectation turns from
(k u k v )/2| E| into (kin out
u k v ) / | E | for the v → u edges. Modularity thus
becomes:

kout in 

1 v ku
M=
2| E | ∑ Auv −
| E|
δ ( c v , c u ).
u,v∈V

Since we’re here, why stopping at directed unweighted graphs?

Let’s add weights! Say that the (u, v) edge has weight wuv , and that
wuout is the sum of all edge weights originating from u (with win
u
defined similarly for the opposite direction). Then:
 community evaluation 443

 
1 wvout win
M= ∑ wuv −
u
 δ ( c v , c u ).
 
2| E | u,v∈V ∑ wuv
u,v∈V

Note that this simple move makes optimizing modularity a tad

more complicated – you should check out the original paper to see
why.
This is all well and good, since there aren’t competing definitions
of directed/weighted modularity. What’s that? I’m being told there
are. Oh boy. For instance, an alternative is to look at a directed 19
Roger Guimerà, Marta Sales-Pardo,
network as if it were a bipartite network19 , where each node v can be and Luís A Nunes Amaral. Module
identification in bipartite and directed
seen as two nodes vin and vout .
networks. Physical Review E, 76(3):036102,
It has also being pointed out that, while this generalized modular- 2007
ity gives out different results than the standard modularity, it actually 20
Youngdo Kim, Seung-Woo Son, and
doesn’t really distinguish the u → v and v → u cases very well20 . Hawoong Jeong. Finding communities
in directed networks. Physical Review E,
In this case, the proposed solution is to use the PageRank of nodes
81(1):016103, 2010
u and v as an expectation of their connection strength. And, once
you do that, you open the floodgates of hell, as any directed measure
can be now used to determine your expectation, generating hun-
dreds of different modularity versions, each with its own community
definition.

Known Issues
But the issues raised so far are only child’s play. Let’s take a look
at the real problematic stuff when it comes to modularity. There are
three main grievances with modularity. The first is that random
fluctuations in the graph structure and/or in your partition can 21
Roger Guimera, Marta Sales-Pardo,
and Luís A Nunes Amaral. Modularity
make your modularity increase21 . However, I already mentioned that
from fluctuations in random graphs and
modularity can be extended to take care of statistical significance. complex networks. Physical Review E, 70
A harder beast to tame is the infamous resolution limit of modu- (2):025101, 2004

larity. To put it bluntly, modularity has a preferred community size,

relative to the size of the graph. This means that a partition that a
human would consider the natural partition of the network could
be rejected by modularity maximization as it is not at the preferred 22
Santo Fortunato and Marc Barthelemy.
Resolution limit in community detec-
resolution22 . Empirically, it has been shown that modularity maxi-
p tion. Proceedings of the National Academy
mization approaches tend to find | E| communities in the network of Sciences, 104(1):36–41, 2007
– a number of communities that seems to be common for many other 23
Amir Ghasemian, Homa Hossein-
mardi, and Aaron Clauset. Evaluating
partitioning algorithms23 .
overfit and underfit in models of net-
For instance, consider the network and partition in Figure 32.5 work community structure. TKDE,
(top). Modularity is positive, thus this is a good partition. However, 2019

the partition to the bottom of Figure 32.5 is better, even if a human

would probably disagree. This is because, when we have small
communities relative to the number of the edges of the network,
444 the atlas for the aspiring network scientist

Figure 32.5: The resolution limit

of modularity. For the same
M = 0.532
network, I propose two differ-
ent partitions in communities,
using the node color.

M = 0.535

for modularity it is better to merge them, even if they are clearly

and intuitively distinct. This is the resolution limit of modularity:
it accepts partitions only of a comparable size with the size of the
network.
Mathematically speaking, it’s not too hard to extend modularity
so that it can work at multiple resolutions. The common strategy is 24
Alex Arenas, Alberto Fernandez, and
to add a resolution parameter24 , 25 . This can be interpreted as adding Sergio Gomez. Analysis of the structure
a bunch of self-loops to each node, such that the number of edges in of complex networks at different
resolution levels. New journal of physics,
a small community can still be considerable, due to the presence of
10(5):053039, 2008b
such self loops. However, now you’re not only optimizing modularity, 25
Jianbin Huang, Heli Sun, Yaguang
you also have to search for the optimal value of this parameter. Uff. Liu, Qinbao Song, and Tim Weninger.
Towards online multiresolution commu-
This can get really tricky. Consider Figure 32.6 as another example. nity detection in large-scale networks.
Here we have a ring of cliques, a classical caveman model. How PloS one, 6(8):e23829, 2011a
would you partition this network? It seems natural to just have one
community per clique. Silly human, modularity says, the best partition
is instead merging two neighboring cliques. You look at modularity,
puzzled by this sentence. But she is not done: however, we could also
put random clique pairs in the same community, even if they’re not adjacent.
That’s a good partition as well.
Go home, modularity, you now say, you’re drunk.

Intuitive → M = 0.902
Figure 32.6: A ring of cliques,
showing another side of the
resolution limit problem of
Best → M = 0.904 modularity.

Rnd Pairs → M = 0.888

 community evaluation 445

Joking aside, this is related to another well known problem of 26

Michael T Schaub, Jean-Charles
modularity: the field of view limit26 . Modularity cannot “see” long Delvenne, Sophia N Yaliraki, and
range communities. If your communities are very large and span Mauricio Barahona. Markov dynamics
as a zooming lens for multiscale
across multiple degrees of separation, modularity will overpartition
community detection: non clique-like
them. This means that, if a proper community has nodes whose communities and the field-of-view limit.
shortest connecting path is more than a few edges long, you might PloS one, 7(2):e32210, 2012a

end up splitting them in different groups. This field of view limit is

shared with other community discovery approaches using Markov
processes (random walks). Even vanilla Infomap tends to overparti-
tion such communities. In fact, in the paper I cited in this paragraph,
the authors show how you could interpret the modularity formula as
a one-step Markov process.
All of this is to say that optimizing modularity is NP-hard and the
heuristics have a hard time finding the best partitions because the
space of possible solutions is crowded by high values that look very
different. This is the third grievance with modularity: the degeneracy 27
Benjamin H Good, Yves-Alexandre
of good solutions27 , 28 . It’s easy to get stuck in local maxima even De Montjoye, and Aaron Clauset. Perfor-
when the partition you’re returning makes no sense. A classical mance of modularity maximization in
practical contexts. Physical Review E, 81
solution is to summon consensus clustering29 – we saw it in Section
(4):046106, 2010
31.6. Hopefully, strategies based on perturbation will converge to 28
Andrea Lancichinetti and Santo
different local maxima, and the mistakes will cancel each other out, Fortunato. Limits of modularity
maximization in community detection.
leading you to a global maximum. Physical review E, 84(6):066122, 2011
29
Pan Zhang and Cristopher Moore.
Scalable detection of statistically
32.2 Other Topological Measures significant communities and hierarchies,
using message passing for modularity.
Given these issues, it’s no wonder that researchers have looked Proceedings of the National Academy of
Sciences, 111(51):18144–18149, 2014
elsewhere for alternative quality measures. The ones I’m mentioning
are by no means perfect, and they have been scrutinized less than
modularity, so the absence of known issues should be taken with a
grain of salt.
The likelihood measure introduced for SBMs is an obvious can-
didate as modularity alternative, and I wrote about it in details in
Section 31.1. The obvious downside here is that it needs your com-
munity partition to be a “generative” one: it has to give you a model
of connection probabilities among nodes. Without that, you cannot
estimate how likely your model is to generate the observed network.
There is also a quality measure lurking behind Infomap (Section
31.2). The code length Infomap is trying to minimize is the number
of bits you need to encode the random walks using your partition.
There is, in principle, no issue in generating a community partition
using something else than Infomap, and then testing it with code
length. Thus that is also a valid quality measure. You have to be
careful, because the standard code length is not normalized. Two
networks with different sizes in number of nodes will have a differ-
446 the atlas for the aspiring network scientist

ent expected code length. Thus a better partition in a larger network

could have a worse (higher) code length than a bad partition in a
small network.
A direct evolution of modularity which aims at being a more 30
Renaud Lambiotte, J-C Delvenne, and
general version of it is stability30 , 31 , 32 . In modularity, you see the Mauricio Barahona. Laplacian dynamics
graph as static. Nodes u and v contribute to modularity only insofar and multiscale modular structure in
networks. arXiv preprint arXiv:0812.1770,
as they are directly connected or not. In stability, you see your graph
2008
as a flow. You take into account the amount of time it would take to 31
J-C Delvenne, Sophia N Yaliraki,
reach u from v and vice versa. In practice, modularity is equivalent to and Mauricio Barahona. Stability of
graph communities across time scales.
stability when you only look at immediate diffusion. Proceedings of the national academy of
There’s a battery of other quality measures, conveniently grouped sciences, 107(29):12755–12760, 2010
in a review paper33 . I’m going to cover a few here. In all cases, I use
32
Jean-Charles Delvenne, Michael T
Schaub, Sophia N Yaliraki, and Mauri-
a generic f (C ) to refer to the function taking the community C as an cio Barahona. The stability of a graph
input and returning its evaluation of the quality of that community. partition: A dynamics-based framework
for community detection. In Dynamics
On and Of Complex Networks, Volume 2,
Conductance pages 221–242. Springer, 2013
33
Jure Leskovec, Kevin J Lang, and
The idea behind conductance is that communities should not be Michael Mahoney. Empirical comparison
conductive: whatever falls into, or originates from, them should have of algorithms for network community
detection. In WWW, pages 631–640.
a hard time getting out. In practice, this translates in comparing ACM, 2010b
the volume of edges pointing outside the cluster34 , 35 , 36 , 37 . Here 34
Jianbo Shi and Jitendra Malik. Nor-
we assume that C ⊆ V is a set of nodes grouped in a community. malized cuts and image segmentation.
Departmental Papers (CIS), page 107,
Mathematically speaking, let’s define two sets of edges. The first set 2000
of edges, EC is the number of edges fully inside the community C. 35
Ravi Kannan, Santosh Vempala, and
Adrian Vetta. On clusterings: Good, bad
That means EC = {(u, v) : u ∈ C, v ∈ C }. The second set of edges and spectral. Journal of the ACM (JACM),
is the boundary of C: the edges attached to one node in C and one 51(3):497–515, 2004
node outside C: EB,C = {(u, v) : u ∈ C, v 6∈ C }. We can now define 36
Jure Leskovec, Kevin J Lang, Anirban
Dasgupta, and Michael W Mahoney.
conductance as: Statistical properties of community
structure in large social and informa-
| EB,C | tion networks. In Proceedings of the 17th
f (C ) = . international conference on World Wide
2| EC | + | EB,C |
Web, pages 695–704. ACM, 2008
Note that we want to minimize this function, namely we want 37
Jure Leskovec, Kevin J Lang, Anirban
to find the partition of G such that the average conductance across Dasgupta, and Michael W Mahoney.
all communities is minimal. Figure 32.7 shows two examples of Community structure in large networks:
Natural cluster sizes and the absence
communities with different levels of conductance. of large well-defined clusters. Internet
Figure 32.7(a) (in red) is a low conductance community. It is a Mathematics, 6(1):29–123, 2009
clique of seven nodes, thus we know that | EC | = 7 × 6/2 = 21. It
has four edges in its boundary (| EB,C | = 4), which gives us 4/((2 ×
21) + 4) ∼ 0.087. Figure 32.7(b) (also in red), on the other hand, has a
higher conductance. Being a clique of five nodes, | EC | = 5 × 4/2 = 10.
From the figure we see that | EB,C | = 7, giving us a conductance of
7/((2 × 10) + 7) ∼ 0.26.
Note how conductance doesn’t care too much about internal
density, as one would expect from the classical definition. It cares,
instead, about external sparsity: making sure that the community
 community evaluation 447

Figure 32.7: Two examples of

communities at different con-
ductance levels. I represent the
community as the node color.
In the text, I focus on the red
community.

(a)
(b)

is as isolated as possible from the rest of the network. Here, both

communities are cliques, the denset possible structure. But, since the
one in Figure 32.7(b) also has a lot of connections to the rest of the
network, the resulting conductance is almost three times higher than
the value we get from the community in Figure 32.7(a).
Finally, be aware that you cannot build a community discovery
algorithm that simply minimizes conductance – the same way you’d
try to maximize modularity. That is because there’s a trivial com-
munity with zero conductance: the one including all nodes in your
network.

Internal density
The other side of the conductance coin is the internal density mea-
sure. This is exactly what you’d think it is: how many edges are
inside the community over the total possible number of edges the 38
Filippo Radicchi, Claudio Castellano,
community could host38 . Borrowing EC from the previous section: Federico Cecconi, Vittorio Loreto,
and Domenico Parisi. Defining and
| EC | identifying communities in networks.
f (C ) = . Proceedings of the National Academy of
|C |(|C | − 1)/2 Sciences, 101(9):2658–2663, 2004
So you can see that, in this case, both communities in Figure 32.7
have an internal density of 1, since they’re cliques. Thus, internal
density is unable to distinguish between them, which we would like
since community Figure 32.7(b) is clearly “weaker”, given its high
number of external connections.

Figure 32.8: The best internal

density partition of this core
community. I encode the node’s
community with its color.
448 the atlas for the aspiring network scientist

You can appreciate the paradoxical result of internal density by

looking at Figure 32.8. Here, one might be tempted to merge the red
and blue communities, since their nodes are so densely connected to
each other and to not much else. Yet, the red nodes are a 5-clique and
the blue nodes are a 4-clique, while the red and blue nodes are not a
9-clique. Thus, the best way to maximize internal density is to split
these clearly very related nodes.
Neither conductance nor internal density fully capture the classical
definition of community discovery I provided in the previous chapter.
The definition wants communities to be both internally dense and
externally sparse. Each of the two measures only satisfies one of the
two requirements. Thus, if you’re using either of them to evaluate
your communities, you’re practically having a different definition of
what a community is.
Just like conductance, don’t try to blindly maximize internal
density, as you’re only going to find cliques in your networks.

Cut
Originally, we define the cut ratio as the fraction of all possible edges
leaving the community. The worst case scenario is when every node
in C has a link to a node not in C. There are |C | nodes in C and
(|V | − |C |) nodes outside C, so there can be |C |(|V | − |C |) such links.
Thus:

| EC,B |
f (C ) = .
|C |(|V | − |C |)
This is usually what gets minimized when solving the mincut
problem (Section 8.4). Again, this is a measure easy to game. That is
why we often modify it to be a “normalized” mincut:

| EB,C | | EB,C |
f (C ) = + .
2| EC | + | EB,C | 2(| E| − | EC |) + | EB,C |
The most attentive readers already noticed that the first term in
this equation is conductance. The second term is also a conductance
of sorts. If the first term is the conductance from the community to
the rest of the network, the second term is the conductance from
the rest of the network to the community. The two are not the same,
because the number of edges in C is | EC |, while the number of edges
outside C is | E| − | EC |.

Out Degree Fraction

The out degree fraction (ODF), as the name suggests, looks at the
share of edges pointing outside the cluster. It follows a strategy
 community evaluation 449

similar to conductance. The difference lies in a normalizing factor.

While conductance normalizes with the total number of edges in
the community, in the out degree fraction you normalize node by 39
Gary William Flake, Steve Lawrence,
node. In the original paper39 , the authors present a few variants of C Lee Giles, et al. Efficient identification
the same philosophy. of web communities. In KDD, volume
2000, pages 150–160, 2000
In the Maximum-ODF, you simply pick the node which has the
highest number of edges pointing outside the community (relative to
its degree) as your yardstick:

|(u, v) : v 6∈ C |
f (C ) = max .
u∈C ku
The idea here is that, in a good community partition, there
shouldn’t be any node with a significant number of edges point-
ing outside the community. We can tolerate if a node has a large
number of edges pointing out, only if the node is a gigantic hub with
a humongous degree k u .
Requiring that there is absolutely no node with a large out degree
fraction might be a bit too much. So we also have a relaxed Average-
ODF:

1 |(u, v) : v 6∈ C |
f (C ) =
|C | ∑ ku
.
u∈C
In this case, we’re ok if, on average, nodes tend not to connect
relatively much to neighbors outside the cluster. If there is one
node doing so, the presence of many other nodes without external
connections will overwhelm it.
Finally, Flake et al. in their paper propose a further variant of the
same idea:

1
f (C ) = |{u : u ∈ C, |(u, v) : v 6∈ C | < k u /2}|.
|C |
For each node u in C, we count the number of edges pointing
outside the cluster. If it’s more than half of its edges, we mark the
node as “bad”, because it connects more outside the community than
inside. A node shouldn’t do that! The measure tells you the share of
bad nodes in C, which is something you want to minimize.
Figure 32.9 shows an example community, which we can use to
understand the difference between the various ODF variants. In the
Maximum-ODF, we’re looking for the node with the relative highest
out degree. That is node 1 as its degree is just three, and two of
those edges point outside the community. Thus, the Maximum-ODF
is f (C ) = 2/3. Both nodes 2 and 3 have a higher out-community
degree, but they also have a higher degree and thus they don’t count
at all for the community quality. You can see how Maximum-ODF is
a blunt tool which disregards lots of information.
450 the atlas for the aspiring network scientist

Figure 32.9: An example of

community (in red).

3
2

For Average-ODF we have (clockwise starting from node 1):

f (C ) = (2/3 + 3/5 + 0 + 4/10 + 1/5 + 1/5 + 1/6)/7 ∼ 0.319.
This is awfully close, but not quite, conductance – which is 12/(2 ×
(13) + 12) ∼ 0.316. Finally, we only have two nodes with more links
going outside the community than inside: these are nodes 1 and 3.
Thus, Flake-ODF is f (C ) = 2/7.

32.3 Link Prediction

If you have a temporal network, you gain a new way to test the
quality of your communities. After all, communities are dense areas
in the network, thus they tell you something about where you expect
to find new links. In a strong assortative community partition, there
are more links between nodes in the same community than between
40
Or so the classical definition of
nodes in different communities. Otherwise, you communities would
community says. I already started
be weak – or there won’t be communities at all40 . tearing it apart, and I’ll continue doing
Thus you can use your communities to have a prior about where so, but in this specific test you base
your assumption on this classical
the new links will appear in your dynamic network. This sounds definition. If you have a different
familiar because it is: it is literally the definition of the link prediction definition of community, don’t use this
test.
problem (Part VI). In this approach of community evaluation, you
use the community partition as your input. You use it to estimate the
likelihood of connection between any pair of nodes in the network,
and then you can design the experiment (Chapter 22) and use any
link prediction quality measure as your criterion to decide which
community partition is better. The higher your AUC, the better
looking your ROC curve, the better your partition is.
The classical way to create a score(u, v) is having a simple binary
classifier: 1 if u and v are in the same community, 0 otherwise. This
is a bit clunky, so you usually want to add a bit of information: how
well embedded are the nodes in the network? This also works in the
 community evaluation 451

case of overlapping community discovery (Chapter 34), when nodes

can be part of multiple communities. In that case, score(u, v) can be
the number of communities they have in common. This seemingly
innocuous operation has some rather interesting repercussions,
which we will see dubbed as the “overlap paradox” in Section 34.7.
A method that works naturally well to be evaluated via link pre-
diction is finding communities via SBMs. This is because, in the
general SBM, every pair of nodes receives a pin or a pout connection
probability given the planted partition. Only in the simplest SBM
techniques these values are the same for every pair of nodes. In more
sophisticated approaches they are personalized for each node pair,
and thus serve as a natural score(u, v) function.
Remember that, when looking at communities, you could have
a disassortative community partition, where nodes tend to connect
to other nodes outside their own community. You can still use this
approach, now penalizing in your score(u, v) function nodes part of
the same community. You could even do more fun stuff, by creating
a community-community similarity score, in which nodes that are in
more interconnected communities receive a higher score(u, v) value.
You know from Chapter 22 that you can still evaluate your link
prediction also in presence of static network data, via k-fold cross
validation. This would allow you to evaluate your communities as
input for link prediction even lacking a temporal network, making
this a more general evaluation tool.

32.4 Normalized Mutual Information

Your network might not be temporal, but you could have additional
information about the nodes, besides to which other nodes they
connect (Section 4.5). In this context, node attributes are usually
referred to as “node metadata”. There is a widespread assumption
in community discovery: if you have good node metadata, some of
them have information about the true communities of the network.
Nodes with similar values, following the homophily assumption
(Chapter 26), will tend to connect to each other. Therefore there
should be some sort of agreement between the community partition 41
Jaewon Yang and Jure Leskovec.
of the network and the node metadata41 . Defining and evaluating network
communities based on ground-truth.
For instance, a classical paper42 analyzed a network whose nodes Knowledge and Information Systems, 42(1):
were cellphones, connected together if they made a significant num- 181–213, 2015
ber of calls to each other. The network showed three well-separated
42
Vincent D Blondel, Jean-Loup Guil-
laume, Renaud Lambiotte, and Etienne
communities. Figure 32.10 shows an extreme simplification of that Lefebvre. Fast unfolding of communities
(very large) graph. in large networks. Journal of statistical
mechanics: theory and experiment, 2008
Why was that the case? Why were there gigantic communities? It
(10):P10008, 2008
all becomes clear when I tell you that the country they studied was
452 the atlas for the aspiring network scientist

Figure 32.10: A simplification

Ja?
Oui? of the Belgian cellphone call
graph, highlighting three com-
munities (with the dashed gray
outline).

Belgium, where roughly half of the population is French-speaking

and the other half Dutch-speaking, and so they do not call each other.
The intersection in the middle is the capital Brussels, where the two
populations have to interact. Knowing which language you speak
should have almost a one-to-one correspondence with the network
community in this case.
How would you calculate such agreement? We can re-use a con-
cept we encountered early on: mutual information (Section 2.8). To
recap briefly: you can consider each node in the network as being
an entry in two vectors. In the first vector, the node is associated
with its metadata: the language the person speaks or whether she
lives in Brussels. In the second vector, the node is associated to its
community.

X Y Figure 32.11: An illustration

of what mutual information
If then (66%) means for two vectors. Vector
y has equal occurrences for its
If then (33%) values (there is one third prob-
ability of any colored square).
If then (50%) However, if we know the value
of x we can usually infer the
If then (50%) corresponding y value with a
higher-than-chance confidence.
If then (100%)

Mutual information tells you the number of bits of information

you gather about one vector by knowing the other vector. Figure
32.11 (a reprisal of Figure 2.13) should help you understanding what
 community evaluation 453

mutual information means: having a set of rules that allow you to

infer the values in one vector by knowing the other with better-than-
chance odds.
In Section 20.7 we used mutual information for link prediction,
meaning that we didn’t care much about comparing different net-
works, as everything happened in the same network. However, when
evaluating community partitions, you need a standardized yardstick
to know whether a network has communities more tightly knit than
another – or if it has communities at all! But mutual information is
dependent on the amount of bits you need to encode the vectors in
the first place. A longer vector needs more bits to be encoded. Thus,
the same value of mutual information can mean different things
when you have 100 nodes or 100, 000.
That is why we often use a normalized mutual information (NMI).
This is a simple normalization that forces mutual information to take
a value between zero and one. This is generally achieved by dividing
mutual information by some combination of the entropy of the two 43
Alexander Strehl and Joydeep Ghosh.
vectors (the community partition and the node metadata)43 , 44 . Cluster ensembles—a knowledge reuse
While normalizing mutual information so that it’s comparable framework for combining multiple
partitions. Journal of machine learning
across networks is nice, that is not the full story. Remember that
research, 3(Dec):583–617, 2002
our end here is knowing whether there is a relationship between 44
William H Press, Saul A Teukolsky,
the communities we found and the node attributes. The problem of William T Vetterling, and Brian P
Flannery. Numerical recipes 3rd edition:
mutual information is that it is always non-zero. This means that The art of scientific computing. Cambridge
there will be always a little mutual information between two vectors, university press, 2007
even if they are both completely random!

a NMI ~ 0.09
Figure 32.12: Two random vec-
b AMI ~ -0.22 tors with a positive NMI even if
generated completely indepen-
Consider the vectors in Figure 32.12. I generated them by extract- dently from one another.
ing ten random elements, with three possible values. This is done
uniformly at random and with independent draws – pinky promise!
Yet, if you calculate their NMI values, you’re going to obtain around
0.09: a non-zero mutual information from vectors that literally have
nothing to do with each other. This is not good.
That is why researchers developed a new normalization for mu-
45
Marina Meilă. Comparing cluster-
tual information: Adjusted mutual information (AMI)45 , 46 . In this ings—an information based distance.
case, we subtract from mutual information the amount of bits we Journal of multivariate analysis, 98(5):
would expect to obtain about a vector by pure chance. In this, AMI 873–895, 2007
46
Nguyen Xuan Vinh, Julien Epps, and
is similar to modularity: you’re comparing the observed value with James Bailey. Information theoretic
the one you’d get from some sort of null model. AMI is defined to be measures for clusterings comparison:
equal to zero when you get nothing more than you’d expect by just Variants, properties, normalization and
correction for chance. J. Mach. Learn. Res,
tossing coins. At this point, any positive AMI value starts getting in- 11(Oct):2837–2854, 2010
teresting. AMI can be negative, and it is for the two vectors in Figure
454 the atlas for the aspiring network scientist

32.12. A negative AMI means that your clustering isn’t good.

It can also mean another thing. You see, so far I grounded this sec-
tion on a key assumption: that node metadata go hand in hand with
the network structure. That... is not always the case. Researchers have 47
Darko Hric, Richard K Darst, and
seen how much the two notions can diverge47 . Node metadata and Santo Fortunato. Community detection
structural network communities are rarely the same thing. Nodes can in networks: Structural communities
versus ground truth. Physical Review E,
share attribute values and not being connected to each other due to a
90(6):062805, 2014
variety of reasons.
In fact, the assumption that communities and node metadata go
hand in hand rests on shaky ground. It seems to give some sort of
importance and status to the node metadata because it calls it “meta”
data. But, at the end of the day, in real observed networks metadata
is just data. Real metadata is like the planted community in an LFR
model (Section 15.2), but when you do data gathering there’s no
such a thing as metadata: what you find is often – if not always –
incomplete, irrelevant, or wrong to a certain degree.
We can call this a “data” problem. Which is yet another issue with
the classical definition of communities. Wanting to find a structural
way to group nodes with the same attributes – especially when we
don’t know their values and we want to infer them – is a totally
valid aim on which to base your community definition. It’s just
that it doesn’t correlate well with the notion of communities made
by densely connected nodes. In this scenario, we might even have
disconnected communities, made by multiple components without
paths leading from one node in the community to another node in
the same community. This is absolutely verboten in the classical view
of community discovery.

Figure 32.13: A network with a

community structure made by
conflicting attributes. One node
attribute is the color, while the
other is its horizontal position-
ing. Nodes connect most likely
with nodes of the same color
and in the same “column”.

Column 1 Column 2 Column 3

The assumption that we can infer the “real” communities from

node attributes rests on data availability: we have some metadata
and we assume that the network follows it. But the network could
 community evaluation 455

be wired following multiple different attributes and the interactions

between them. Figure 32.13 shows a simplified example of that. The
communities found by node colors are “good” to approximate one
attribute, but they are terrible for the classical notion of community.
There’s another problem with ground truth and community
discovery, a more theoretical one. As many other facets of life, in 48
Leto Peel, Daniel B Larremore, and
community discovery there is no free lunch48 . This means that com- Aaron Clauset. The ground truth about
munity discovery is a large problem with many different network metadata and community detection in
networks. Science advances, 3(5):e1602548,
types and valid community definitions. There is no single algorithm
2017
who is going to work reliably better than average in all these scenar-
ios.
We are going to see yet more ways in which the classical definition
of communities break. But this is a good moment to have a brief
pause and collect our thoughts. The real definition of community
depends on what the network represents: if you’re looking at a so-
cial network some definitions of communities make sense, but if
you’re looking at an infrastructure network they do not. Commu-
nities depend on what you’re looking for: whether you’re trying to
approximate a real world property or compress your network. And
they also depend on what’s your criterion of success: nobody says
you cannot use modularity, or NMI, or anything else, as long as it is a
motivated choice.
I want you to learn a lesson, my dear reader. You didn’t have to go
and look for a definition of community: the real definition was inside
you all along.

32.5 Summary

1. The most common function used to evaluate community parti-

tions is modularity. Modularity compares the number of edges
inside the communities you detected with the expected number
of edges in a configuration model which has, by definition, no
communities.

2. You can also use modularity for something more than evaluating
the communities you found: it can be an optimization target. Your
algorithm will operate on your communities until it cannot find
any additional move that would increase modularity.

3. Standard modularity is defined for undirected and unweighted

graphs. There are extensions of the measure to deal with directed
and/or weighted graphs, however such extensions are not unique:
there are multiple competing versions.

4. Modularity has been extensively studied and we know it has sev-

456 the atlas for the aspiring network scientist

eral issues. The main one is resolution limit, where communities

have to be of similar size. There is also degeneracy: many parti-
tions are close to optimal, even if they are very different from each
other.

5. Many other quality functions have been defined. Conductance

aims at minimizing edges flowing out of a community. Internal
density aims at maximizing the edges inside the community.

6. One could use communities as the basis of link prediction, since

nodes in the same community are expected to connect to each
other. Thus a better partition is one that would be more accurate in
predicting new links.

7. Normalized mutual information is another way to evaluate your

partition when you have metadata about your nodes – if you
assume that communities should be used to recover latent node
attributes. Be aware, though, that not always nodes with similar
attributes connect to each other.

32.6 Exercises

1. Detect communities of the network at [Link]

eu/exercises/32/1/[Link] using the asynchronous and the
semi-synchronous label propagation algorithms. Which one does
return the highest modularity?

2. Find the communities of the network at [Link]

eu/exercises/32/2/[Link] using label propagation and calcu-
late the modularity. Then manually create a new partition by
moving nodes 25, 26, 27, 31 into their own partition. Recalculate
modularity for this new partition. Did this move increase modular-
ity?

3. Repeat exercise 1, but now evaluate the difference in performance

of the two community discovery algorithms by means of conduc-
tance, cut size, and normalized cut size.

4. Assume that [Link]

[Link] contains the “true” community partition of the nodes
from the network at [Link]
32/1/[Link]. Determine which algorithm between the asyn-
chronous and the semi-synchronous label propagation achieves
higher Normalized Mutual Information with such gold standard.
33
Hierarchical Community Discovery

When talking about the issues of modularity as a quality measure

for network partitions, we touched on an important subject which
deserves to be explored more deeply. When doing community dis-
covery, you might have a situation where the network can be divided
in different ways and they’re all valid partitions. For instance, in Fig-
ure 33.1, the obvious assortative partition (blue outlines) will divide
scientists into their fields, and laymen into their own communities.
However, it is also reasonable to enlarge the definition of a field and
say that there is a scientific community, which incorporates all of its
subfields (purple outline), and a non-scientific community, which
incorporates all non-scientific subcommunities (green outline).

Figure 33.1: Two possible valid

partitions of this network. Gray
lines are the edges. The blue
outlines identify strong, nar-
row communities. The purple
line outlines the scientific com-
munity, opposed to the green
outline: the laymen community.

To generalize the issue, once you find tightly knit communities,

you might realize that some communities are more related to each
other than others. And so there is a second level on the partition you
can impose on the network. This means that, after finding commu-
nities of nodes, you want to find communities of communities. And
458 the atlas for the aspiring network scientist

communities of communities of communities. And so on. This is

the “hierarchical” community discovery problem: how to create a
hierarchy of communities that best describes the structure of your
network.
I’m going to present some general approaches to hierarchical
community discovery. As usual, be aware that there are more than 1
Spiros Papadimitriou, Jimeng Sun,
I can cover here1 , 2 . However, once you know them, it’s easy for you Christos Faloutsos, and S Yu Philip.
to see that you can redefine many standard community discovery Hierarchical, parameter-free community
discovery. In Joint European Conference
algorithms to find hierarchical communities. For instance, the In-
on Machine Learning and Knowledge
fomap algorithm I described in the previous chapter has a natural Discovery in Databases, pages 170–187.
hierarchical version3 . Springer, 2008
2
Jianbin Huang, Heli Sun, Jiawei
Han, Hongbo Deng, Yizhou Sun, and
33.1 Recursive Approaches Yaguang Liu. Shrink: a structural
clustering algorithm for detecting
hierarchical communities in networks. In
You can transform any community discovery algorithm into a hier- Proceedings of the 19th ACM international
archical community discovery algorithm by applying it recursively conference on Information and knowledge
management, pages 219–228. ACM, 2010
to coarsened views of your network. The easiest way to do so is by 3
Martin Rosvall and Carl T Bergstrom.
following this simple meta algorithm: Multilevel compression of random
walks on networks reveals hierarchical
1. Apply your algorithm to G and find the optimal communities; organization in large integrated systems.
PloS one, 6(4):e18209, 2011
2. Condense your graph by collapsing all nodes belonging to com-
munity C into a meta-node C.;

3. Connect all Cs with each other, according to how many edges

there were between the nodes they include;

4. You now have a new graph G 0 , so you can go back to step 1.

Figure 33.2: (a) The optimal

node partition. (b) Collapsing
each community into a meta-
node. (c) Connecting meta-
(a) (b) (c) (d) nodes made by nodes which
were originally connected to
each other. (d) Finding the
Figure 33.2 shows a graphical example of this meta algorithm. You
second level partition.
can modify step 3 in case your algorithm was an overlapping algo-
rithm, which allows communities to share nodes. In that case, you 4
Michele Coscia, Giulio Rossetti, Fosca
can count the number of shared nodes between the communities4 , Giannotti, and Dino Pedreschi. Un-
rather than the number of edges connecting them. covering hierarchical and overlapping
communities with a local-first ap-
However, such approach is just a hack on top of non-hierarchical proach. ACM Transactions on Knowledge
community discovery. In reality, we want to have a hierarchy-aware Discovery from Data (TKDD), 9(1):6, 2014
approach that was built with this feature in mind from the begin-
ning, rather than adding it as an afterthought. There are two meta-
 hierarchical community discovery 459

approaches for baking in hierarchies in your community discovery:

merging and splitting.

Merging
In the merging approach, you start from a condition where all your
nodes are isolated in their own community and you create a criterion
to merge communities. This is a bottom-up approach. It is similar
to the meta-algorithm from earlier, but it’s not really the same. Let’s
take a look at how it works, highlighting where the differences with
the meta-algorithm are.
The template I’m using to describe this approach is the Louvain 5
Vincent D Blondel, Jean-Loup Guil-
algorithm5 . This is one of the many heuristics used to recursively laume, Renaud Lambiotte, and Etienne
Lefebvre. Fast unfolding of communities
merge communities with the aim of maximizing modularity6 , 7 ,
in large networks. Journal of statistical
which happens to be among the fastest and most popular. mechanics: theory and experiment, 2008
The Louvain algorithm starts with each node in its own commu- (10):P10008, 2008
6
Marta Sales-Pardo, Roger Guimera,
nity. It calculates, for each edge, the modularity gain one would get
André A Moreira, and Luís A Nunes
if they were to merge the two nodes in the same community. Then Amaral. Extracting the hierarchical
it merges all edges with a positive modularity gain. Now we have a organization of complex systems.
Proceedings of the National Academy of
different network for which the expensive modularity gains need to Sciences, 104(39):15224–15229, 2007
be recomputed. However, this network is smaller, because of all the 7
Tiago P Peixoto. Hierarchical block
edge merges. You repeat the process until you have all nodes in the structures and high-resolution model
selection in large networks. Physical
same community. Figure 33.3 shows an example of this process. Review X, 4(1):011047, 2014c

High Gain
Figure 33.3: An example of the
first step of the Louvain algo-
rithm. All in-clique edges (like
Low Gain
the representative I highlight
in blue) are merged, while all
out-clique edges (like the repre-
sentative I point to with a gray
arrow) are ignored.

The Louvain algorithm is particularly smart and optimized to find

the best merges by minimizing the amount of computation needed.
Its first step is expensive, because for every edge you have to know
what’s the modularity gain of merging the nodes. However, once you
start merging, it’s fast because you only need to update the gains
of the nodes directly connected to the new partition. Finally, there
is no need to go all the way: we know that putting all nodes in the
same community has modularity zero, so at some point there are no
8
Aaron Clauset, Mark EJ Newman, and
Cristopher Moore. Finding community
moves that can improve modularity, and we can stop. The algorithm structure in very large networks. Physical
inspiring it8 , for instance, only made one merge per modularity gain review E, 70(6):066111, 2004
460 the atlas for the aspiring network scientist

and thus had to perform the expensive modularity gain calculation

more often for larger networks.
What the algorithm does, in practice, is building a dendogram of
communities from the bottom up. Each iteration brings you further
up in the hierarchy. We start with no partition: each node is in its
own community. And then we progressively make larger and larger
communities, until we have only one. Figure 33.4 shows an example
of this approach. This is the crucial difference between the merging
approach and what I discuss previously. In the meta algorithm, you
don’t perform all the merges, you make lots of them at once when
you run your step 1 to find the initial communities.

Figure 33.4: The dendogram

building from the bottom up
typical of a “merging” ap-
proach in hierarchical commu-
nity discovery.

Iterations

Splitting 9
Michelle Girvan and Mark EJ New-
man. Community structure in social
In the splitting approach, you do the opposite of what I described and biological networks. Proceedings of
so far. You start with all nodes in the same community and you use the national academy of sciences, 99(12):
a criterion to split it up in different communities. For instance by 7821–7826, 2002
10
Mark EJ Newman and Michelle
identifying edges to cut. This is a top-down approach. Girvan. Finding and evaluating
Historically speaking, the first algorithm using this approach used community structure in networks.
edge betweenness as its criterion to split communities9 , 10 . That is Physical review E, 69(2):026113, 2004
11
Filippo Radicchi, Claudio Castellano,
not to say there aren’t valid alternatives as your splitting criterion, Federico Cecconi, Vittorio Loreto,
including – but not limiting to – edge clustering11 and information and Domenico Parisi. Defining and
centrality12 . However, given its historical prominence, I’m going to identifying communities in networks.
Proceedings of the National Academy of
allow the edge betweenness Girvan-Newman algorithm to have its Sciences, 101(9):2658–2663, 2004
place under the limelight. 12
Santo Fortunato, Vito Latora, and
The first step of the algorithm is to calculate the edge betweenness Massimo Marchiori. Method to
find community structures based on
of each edge in the network, that is the normalized number of short- information centrality. Physical review E,
est paths passing through it (Section 11.2). The assumption is that 70(5):056104, 2004
 hierarchical community discovery 461

edges between assortative communities will have a systematically

higher edge betweenness value than edges inside the communities.
Figure 33.5 shows an example. All edges inside the communities
have a low value because there are many alternative paths you can
take, since all nodes are connected to everybody else in the com-
munity. On the other hand, if you want to go from one node in one
community to a node in another, there’s only one edge you can use.
As a result, its edge betweenness value skyrockets.

Figure 33.5: Two cliques con-

nected by an edge. I label links
14
14 16 16 with edge betweenness higher
14
84 16 than one with the number of
14 shortest paths passing through
14 16 16
14 them.

The second step of the algorithm is to cut the edge with the high-
est edge betweenness. The final aim is to break the network down
into multiple components. Each component of the network is a com-
munity.
Unfortunately, after each edge deletion you have to recalculate
the betwennesses. Every time you alter the topology of the network
you change the distribution of its shortest paths. This makes edge
betweenness extremely computationally heavy. Calculating the edge
betweenness for all edges takes an operation per node and per edge
(O(|V || E|)) and you have to repeat this for every edge you delete,
resulting in a crazy complexity of O(|V || E|2 ). You cannot apply this
naive algorithm to anything but trivially small networks.
You can now see the parallels with the Louvain method I de-
scribed earlier. The difference is that you are exploring the dendo-
gram of communities from the top down, rather than bottom up.
Each iteration brings you further down in the hierarchy. At the very
top you start with a network with a single connected component. As
you delete edges, you find different connected components. As you
continue, you end up with more and more. At the last iteration, each
node is now isolated.
Differently from the Louvain algorithm, in the Girvan-Newman
method you do not calculate modularity gains as you explore the
dendogram. Thus, the algorithm will normally perform all the
possible splits and returns you the full structure, rather than the
462 the atlas for the aspiring network scientist

cut that maximizes modularity. Thus you will have to calculate the
modularity of each split yourself, something similar to what you see
in Figure 33.6.

Modularity
Figure 33.6: The dendogram
building from the top down
typical of a “splitting” approach
in hierarchical community dis-
covery. The left panel shows
the modularity values of each
possible cut.

Iterations

Higher modularities are better partitions, thus better cuts. The

good cuts will appear as peaks in the modularity profile of the
dendogram. Thus they are the natural points for us to cut it and
get the best partition. Multiple peaks are a clue of a hierarchical
organization, because they identify good partitions with a very
different number of communities.
The aforementioned edge clustering and information centrality
variants use the same algorithm, changing the criterion to determine
which edge to cut. In edge clustering the assumption is that edges
with high clustering are embedded in communities, because all their
neighboring nodes are connected to each other. Note that in this case
we also modify the definition of local clustering coefficient so that it
applies to edges rather than nodes. The guiding principle is to cut
the edges with the lowest clustering first. This is computationally
more efficient than using edge betweenness, because when you cut
an edge you only change the clustering of the neighboring edges: in
edge betweenness all values need to be recomputed. 13
Ulrik Brandes and Daniel Fleischer.
The information centrality variant has no such benefit, because it Centrality measures based on current
flow. In Annual symposium on theoretical
simply uses a different definition of edge betweenness. Specifically it aspects of computer science, pages 533–544.
uses the edge current flow centrality13 . It is still more computation- Springer, 2005
ally efficient, because it uses random walks rather than shortest paths,
which will treat your CPU with more respect.
 hierarchical community discovery 463

33.2 HRG: Part 2

This section is a throwback to Section 20.5, where I introduced the

usage of Hierarchical Random Graphs to solve the problem of link
prediction. If you remember, the idea was to divide the graph into a
hierarchical organization, under the assumption that nodes part of
the same hierarchical branch are more likely to connect to each other.
If we start from this assumption, then the natural consequence is also
that these nodes already are densely connected. Thus they are proper
communities!
Note that, however, HRG is more flexible than that: it can also
uncover a disassortative community structure where nodes are less
likely to connect to their community mates.
In Section 20.5 I simply said that “we create a hierarchical rep-
resentation of the observed connections that fits the data,” which
is a rather mysterious non-explanation of how we actually group
nodes into communities. The way the algorithm works is by creating
a dendrogram to fit the data. In the dendrogram, the |V | leaf nodes
are the nodes of the network. The other nodes of the dendrogram are
so-called “internal nodes”, and we need |V | − 1 of them to properly
build the full structure.

30 29
Figure 33.7: A graph with hi-
31 32
erarchical communities (node
27
color according to the commu-
33
26 nity partition at one level of the
36
28 34
25
hierarchy).
35

10 15
11
13

9 12 14 16

8
3 18 22
1 6 17 23
4 7
19 21
2 5 20 24

Figure 33.7 shows a network with a hierarchical community struc-

ture. Figure 33.8 is a possible HRG representation of Figure 33.7.
Each internal node i has an associated probability pi . This is the
probability of connecting two nodes u and v that are in the branch
attached to i. So our aim is to assign to all internal nodes the proper
pi probabilities and to shuffle the leaf nodes properly so that the
dendrogram we end up with is the one most likely to describe the
real data.
464 the atlas for the aspiring network scientist

0.1

0.15

0.32 0.32 0.32

0.46 0.46 0.46

1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

33 35 34 36 31 30 32 29 26 25 27 28 11 9 10 12 1 2 3 4 7 8 5 6 17 20 18 19 21 22 24 23 14 15 16 13

Figure 33.8: A likely HRG rep-

If this sounds familiar, you’re not wrong. This looks like a special resentation of Figure 33.7. Pur-
hierarchical version of stochastic blockmodels. You are basically ple nodes are internal nodes. I
assigning to each node pair a different pi probability of connecting, label them according to their pi
depending on which is their most proximate common internal node probability, which is the num-
ancestor i. ber of edges between nodes in
At this point, finding the dendrogram that most likely fits the the branches over all possible
data is just a choice of how you want to explore the space of all number of edges between them.
possible dendrograms. In the original paper, authors use a Markov
chain Monte Carlo method, where each dendrogram is sampled
proportionally to its likelihood value.
Note that the dendrogram in Figure 33.8 is not the only possible
good description of Figure 33.7. For instance, I could have grouped
nodes 1 and 3 together, at the lowest level, rather than 1 and 2. The
resulting dendrogram would have been equally likely, and would
generate an equally good hierarchical representation.

33.3 Density vs Hierarchy

Hierarchical community discovery poses some issues with our tradi-

tional community definition based on density. If there is a partition
with maximum internal density in the network, then a partition at
a different hierarchical level must have a lower density – by defini-
tion. Is that still a valid partition of the network? When you perform
hierarchical community discovery you’d say yes, and that would be
totally valid. There are valid analytic scenarios where you can divide
up a social network at multiple levels. For instance the example I
made at the beginning, dividing it up into a layman and scientific
community, and then breaking down the scientific community in
 hierarchical community discovery 465

fields and subfields. But that is in direct contradiction with our

classical community definition.
This is particularly tricky since even modularity, which should be
defined as in direct correspondence with this density-based defini-
tion, actually disagrees with it. In other words, the density profile
changes differently from the modularity profile. When we group
everything in a community, there’s some density even if modularity
is zero (Figure 33.9(a)). At the top hierarchical level we have high
modularity but low internal density (Figure 33.9(b)). At the best
partition we have agreement (Figure 33.9(c)). But density is still high
even with low modularity for a partition that puts together connected
node pairs (Figure 33.9(d)).

Modularity Modularity

Density Density

Iterations
Iterations

(a) (b)

Modularity Modularity

Density Density

Iterations Iterations

(c) (d)
Figure 33.9: The contrast be-
tween modularity and density
at different cuts in the hierarchi-
33.4 Summary
cal community organization: (a)
1. You can find communities at different scales in a network. Mean- every node in the same commu-
ing that there are communities of nodes, communities of com- nity; (b) sub-optimal high-level
munities, communities of communities of communities, and so partition; (c) optimal low-level
on. The process to find such structures is hierarchical community partition; (d) maximal density
discovery. but low modularity partition.
466 the atlas for the aspiring network scientist

2. You can find hierarchical communities with either a top-down or

a bottom-up approach. In the bottom-up or merging approach,
you start with each node in its own community and then you
recursively merge communities optimizing a quality function.

3. In the top-down or splitting approach, you start with the network

encapsulated in a single community and you recursively split it
following some guiding principle, e.g. removing the edges with
the highest betweenness.

4. The third alternative is to model your network as a hierarchical

system, and then find the hierarchical organization that is the most
likely explanation of your observed data.

5. Not all communities at all levels of the hierarchy maximize the

internal density and/or the external sparsity of your communities.
Thus, even if totally valid, hierarchical communities defy our
classical definition of communities based on edge density.

33.5 Exercises

1. Use the edge betweenness algorithm to find hierarchical commu-

nities in the network at [Link]
33/1/[Link]. Since the algorithm has high time complexity,
perform only the first 10 splits. What is the split with the highest
modularity?

2. Change the splitting criterion of the algorithm, using the inverse

edge weight rather than edge betweenness. Since this is much
faster, you can perform the first 20 splits. Do you get higher or
lower modularity relative to the result from exercise 1?

3. Use the maximum edge weight pointing to a community as a

guiding principle to merge nodes into communities using the
bottom-up approach. (An easy way is to just condense the graph
by merging the two nodes with the maximum edge weight, for
every edge in the network)

4. Using the algorithm you made for exercise 3, answer these ques-
tions: What is the latest step for which you have the average
internal community edge density equal to 1? What is the modu-
larity at that step? What is the highest modularity you can obtain?
What is the average internal community edge density at that step?
34
Overlapping Coverage

Let’s go back for a moment to the classical definition of communities

in networks:

Communities are groups of nodes densely connected to each other and

sparsely connected to nodes outside the community.

Figure 34.1: A social network

? with an individual having an

equal number of relationships
distributed among two commu-
nities.

This seems to imply that communities are a clear cut case. Nodes
have a majority of connections to other nodes in their community.
However real world networks do not have to conform to this expecta-
tion, and in fact often they don’t. There are numerous cases in which
nodes belong to multiple communities: to which community does the
center person in Figure 34.1 belong? The red or the blue one?
The classical community definition forces us to make a choice. Re-
gardless of the choice we make – red or blue – it’d not be a satisfying
solution. The more reasonable answer is “she belongs to both”. For
instance, a person can very well be part of one community because
it is composed by the people they went to school with. And she can
be part of a work community too, of people she works with. Some of
these people could be the same, but usually they are not.
468 the atlas for the aspiring network scientist

The problem is that none of the methods seen so far allow for such
a consideration. For instance, the basic stochastic blockmodels only
allows you to plant a node in a community, not multiple. Modularity
also has issues, because of the Kronecker delta: since this is going
to be 1 for multiple communities for a node, there will be double-
counting and the formula breaks down.
This is where the concept of overlapping community discovery
was born. We need to explicitly allow for overlapping communities:
communities that can share nodes. There are many ways to do this, 1
Jierui Xie, Stephen Kelley, and
which have been reviewed in several articles1 , 2 dedicated especially Boleslaw K Szymanski. Overlap-
to this sub problem of community detection (itself a sub problem of ping community detection in networks:
The state-of-the-art and comparative
network analysis: it’s communities all the way down).
study. Acm computing surveys (csur), 45
Here we explore a few of the most popular approaches. (4):43, 2013
2
Alessia Amelio and Clara Pizzuti.
Overlapping community discovery
34.1 Evaluating Overlapping Communities methods: A survey. In Social Networks:
Analysis and Case Studies, pages 105–125.
Springer, 2014
Before we delve deep into overlapping community discovery, let’s
amend Chapter 32 to this new scenario. We can have a few options
when we try to evaluate how well we divided the network into
overlapping communities.
Normalized mutual information expects you to put nodes into
a single category. However, there are ways to make it accept an 3
Andrea Lancichinetti, Santo Fortunato,
overlapping coverage3 , 4 . The obstacle is that NMI wants to compare and János Kertész. Detecting the over-
the vector of metadata with the vector containing the community lapping and hierarchical community
structure in complex networks. New
partition. The vector can only have one value per node but, in an Journal of Physics, 11(3):033015, 2009
overlapping coverage, it can have multiple values. Thus we don’t 4
Aaron F McDaid, Derek Greene, and
compare the vectors directly. We compare two bipartite matrices. Neil Hurley. Normalized mutual
information to evaluate overlapping
Suppose you found C communities, and you have A node at- community finding algorithms. arXiv
tributes. You can describe the overlapping coverage in communities preprint arXiv:1110.2515, 2011
with a |V | × C binary matrix, whose u, C entry is equal to 1 if node
u is part of community C. The node attribute matrix is similarly de-
fined. Figure 34.2 shows an example of this procedure. Now you can
calculate the mutual information between the two matrices by pairing
the columns such that we assign to each column on one side the ones
on the other side that is the most similar to it.
We can normalize this mutual information in different ways. In
fact, the papers I cited earlier propose six alternatives, providing
different motivations for each of those. These overlapping NMIs
share with their original counterpart the issue of non-zero values for
independent vectors – although they try to mitigate the issue with
different strategies. 5
Alexander J Gates, Ian B Wood,
William P Hetrick, and Yong-Yeol Ahn.
Some researchers have pointed out a few biases in the overlapping
Element-centric clustering comparison
extensions of NMI and similar measures5 . They propose a unified unifies overlaps and hierarchy. Scientific
framework that can evaluate disjoint, overlapping, and even hierar- reports, 9(1):8574, 2019
 overlapping coverage 469

2
Figure 34.2: (a) A network with
1
three overlapping communities,
4
encoded by the node’s color. (b)
7
6 Transforming the overlapping
3 5
coverage into a binary affilia-
8 tion matrix, which we can use
10 as input for the overlapping
9 version of NMI
13

11 12
(b)
(a)

chical communities. This is achieved by creating a node-community

bipartite affiliation graph and then project it so that you obtain a 6
Linda M Collins and Clyde W Dent.
new node-node unipartite graph – just like the original network you Omega: A general formulation of the
started with. However, now, the relations between the nodes are rand index of cluster recovery suitable
for non-disjoint solutions. Multivariate
due to their common cluster affiliations. The similarity between two Behavioral Research, 23(2):231–242, 1988
community coverages is estimated by looking at the similarity of the 7
William M Rand. Objective criteria
stationary distributions of the projected affiliation graphs. for the evaluation of clustering meth-
ods. Journal of the American Statistical
An alternative measure, called Omega Index, attempts to be the association, 66(336):846–850, 1971
overlapping equivalent of the adjusted mutual information: the one 8
Lawrence Hubert and Phipps Arabie.
correcting for chance6 . This is an extension of the adjusted Rand in- Comparing partitions. Journal of
classification, 2(1):193–218, 1985
dex7 , 8 for non-disjoint clusters. The Rand index is simply the number 9
Alexander J Gates and Yong-Yeol
of times two partitions agree over all possible pairs of nodes. The Ahn. The impact of random models
adjusted-for-chance version establishes the probability of agreeing on clustering similarity. The Journal
of Machine Learning Research, 18(1):
by chance and uses that to normalize the index. The solution again 3049–3076, 2017
passes through a procedure similar to what we explained for the 10
Tamás Nepusz, Andrea Petróczi,
overlapping version of NMI – but note that there is no universal null László Négyessy, and Fülöp Bazsó.
Fuzzy communities and the concept
model to correct for, and the one you assume has a severe impact on of bridgeness in complex networks.
the results you’ll be seeing9 . Physical Review E, 77(1):016107, 2008
What about modularity? Of course there should be a way to
11
Hua-Wei Shen, Xue-Qi Cheng, and Jia-
Feng Guo. Quantifying and identifying
extend it to work with multiple clusters! How hard can that be? It the overlapping community structure in
isn’t at all. In fact, it is so easy that there are multiple conflicting networks. Journal of Statistical Mechanics:
Theory and Experiment, 2009(07):P07042,
ways to extend modularity for the overlapping case. Because woe to
2009
the scientific community that can agree on something. 12
Shihua Zhang, Rui-Sheng Wang, and
Solutions span from replacing the binary Kronecker delta with Xiang-Sun Zhang. Identification of
overlapping community structure in
a continuous node similarity measure based on the product10 , 11
complex networks using fuzzy c-means
or the average12 of “belonging” coefficients (i.e. how much a node clustering. Physica A: Statistical Mechanics
really belongs to a community); to simply calculating the average and its Applications, 374(1):483–490,
2007b
modularity of all communities13 ; to a version incorporating both 13
Anna Lázár, Dániel Ábel, and Tamás
overlap and directed edges14 . Vicsek. Modularity measure of networks
Mentioning “belonging” coefficients allows me to make a distinc- with overlapping communities. EPL, 90
(1):18001, 2010
470 the atlas for the aspiring network scientist

tion here. You can perform overlapping community discovery in two 14

Vincenzo Nicosia, Giuseppe Man-
different ways. The first is by saying that nodes fully belong to mul- gioni, Vincenza Carchiolo, and Michele
Malgeri. Extending the definition of
tiple communities – i.e. that all the communities they belong to are modularity to directed graphs with
equal for them. There is no way to say that a node is “more” part of overlapping communities. Journal of Sta-
tistical Mechanics: Theory and Experiment,
one community or another. This is in contrast with fuzzy clustering15 , 2009(03):P03024, 2009
in which nodes have such belonging coefficients and thus can tell you 15
James C Bezdek. Pattern recognition
whether they really feel like they’re strongly part of a community or with fuzzy objective function algorithms.
Springer Science & Business Media,
not. 2013

Overlapping Communities:
Figure 34.3: Comparing overlap-
u: ping and fuzzy clustering for
node u: the size of the square is
u Fuzzy Communities:
proportional to u’s “belonging”
u: coefficient, in share of number
(60%) (40%) of u’s edges connected to the
community.

Figure 34.3 depicts this distinction. Fuzzy communities are more

difficult to calculate, but they are also more precise. However, you
should be careful in not relying on the coefficients you get from fuzzy
clustering too much. Is there really a difference in saying that 49% of
a node “belongs” to a community, versus saying that 51% of the node
belongs to it? In some cases it might be, but you need to have trust in
the fact that your data allows you such level of confidence.

34.2 Adapted Approaches

On the basis of having an overlapping version of modularity, over-

lapping community discovery needs not to be a separated problem
with specialized solutions. We already have delineated a procedure
to solve the problem: trying to maximize a target function. So we can
take all the algorithms which maximize modularity, and make them
maximize overlapping modularity instead.
This move can be applied to multiple other algorithms. For in- 16
Alcides Viamontes Esquivel and
stance, we have an adaptation of Infomap16 . If you remember Section Martin Rosvall. Compression of flow can
31.2, in Infomap we’re looking for a smart way to encode random reveal overlapping-module organization
in networks. Physical Review X, 1(2):
walks. We do so by using short binary codes to identify nodes inside 021025, 2011
modules and special codes to indicate when the random walk crosses
between communities.
In such strategy, the communities are disjoint, because if a node is
part of two communities you would have to use the code for crossing
between communities when you visit it. However, there is a way to
make this encoding compatible with overlapping communities. That
is giving to a node part of multiple communities a different code per
 overlapping coverage 471

01 00
100 01 Figure 34.4: The encoding of
two random walks in the over-
00
lapping version of Infomap.
Note that in neither the red nor
11 the blue path we’re crossing
community boundaries, so
101 100
we don’t use the community
11 101 crossing code.
1100100101 010011100

community. Figure 34.4 shows an example.

For instance, if the Reykjavik airport is part of both the North
American and the European community, it will have two codes.
We would use one code if the random walk approaches Reykjavik
from a North American airport, and the other code if it approaches
Reykjavik from a European one. You would then use the community
crossing code only if you actually transition between clusters in the
next step.
Finally, let’s consider label propagation. There is a way to extend
the classical label propagation algorithm to allow for overlapping 17
Steve Gregory. Finding overlapping
communities17 . The idea is that nodes will not just adopt the single communities in networks by label
most common label among their neighbors. They will adopt all propagation. New Journal of Physics, 12
(10):103018, 2010
labels, each weighted by a “belonging coefficient”, which is the
weighted average of the belonging coefficient of that label across all
neighbors.
Labels that are below a specific belonging coefficient threshold
are removed from the node at each step, preventing the nodes to
converge to a single global status where all nodes belong to all
communities at the same level of belonging.
Figure 34.5 shows a small run of this principle. Let’s suppose
that we set our minimum belonging coefficient to 0.5. In the first

Figure 34.5: A simple run of the

overlapping label propagation.
The node’s color is a pie chart
representing the belonging
coefficient of the node to a
label.
(a) (b) (c)
472 the atlas for the aspiring network scientist

step (from Figure 34.5(a) to 34.5(b)), the two fully red nodes stay
fully red, because they both receive 0.5 of the red label, 0.33 of the
blue and 0.16 of the purple label. The only label clearing the 0.5
threshold is the red one, thus they become fully red. The half red half
purple node becomes red because that’s the only label around it. The
central node is also red, receiving 0.5 red, 0.25 blue and 0.25 orange.
From Figure 34.5(b) to 34.5(c) though, the central node will correctly
split between red and blue, because they both contribute half of its
neighborhood.

34.3 Explicit Structural Approaches

In the class of structural approaches we find methods that have a

definition of what an overlapping community should look like, and
try to find such a structure in the network. The idea is different
from modularity maximization, because it is not primarily driven
by the optimization of a function. I add the word “explicit” because
these approaches exclusively look at the structure as it is, and try
to find the communities there. In the next section we’ll see “latent”
structural approaches which assume that the observed structure is
the result of a hidden one driving the connections.
The explicit structural approach is historically the oldest solution
to overlapping community discovery. I can think of fundamentally
two subclasses: the famous clique percolation approach, and node
splitting.

Clique Percolation
Clique percolation starts from the observation that communities
should be dense. What is the densest possible subgraph? The clique.
In a clique, all nodes are connected to all other nodes. So the problem
of community discovery more or less reduces to the problem of
finding all cliques in the network. However, this is a bit too strict:
there are subgraphs in the network that, while being very dense and
close to being a clique, are not fully connected. It would be a pity to
split them into many small substructures.
Thus researchers developed the more sophisticated k-clique perco- 18
Imre Derényi, Gergely Palla, and
Tamás Vicsek. Clique percolation in
lation algorithm18 . Clique percolation says that communities must be
random networks. Physical review letters,
cliques of at least k nodes, with k being a parameter you can freely 94(16):160202, 2005
set. In the first step, the algorithm finds all cliques of size k, whether
they are maximal or not. Then, it attempts to merge two communities
in the same community if the two communities share at least a k − 1
clique.
For instance, consider the example in Figure 34.6, setting the
 overlapping coverage 473

Figure 34.6: An example of

clique percolation. 5-Cliques
are highlighted by outlines.
Green cliques percolate with
purple cliques because they
share k − 1 = 4 nodes.
(a) (b)

(c) (d)

parameter k = 5. The blue and green 5-cliques only share two nodes,
so it cannot be a 4-clique. But the green and purple do share a 4-
clique, so they are merged (top row). And there is another purple
5-clique that can now be merged with the green community (bottom
row). 19
Tim S Evans. Clique graphs and
This is generally implemented via the creation of a clique graph19 . overlapping communities. Journal
The nodes of a clique graph are the cliques in the original graph. of Statistical Mechanics: Theory and
Experiment, 2010(12):P12037, 2010
We connect two cliques if they share nodes. For instance, if we only
connect cliques sharing k − 1 nodes, then we can efficiently find
all communities by finding all connected components in the clique
graph.
This algorithm works well in practice. It has been used to study 20
Marta C González, Hans J Herrmann,
overlapping friendship patterns in school systems20 – due to class- J Kertész, and Tamás Vicsek. Commu-
room being quasi-cliques: pupils have rare but significant friendships nity structure and ethnic preferences in
school friendship networks. Physica A,
across classes –, and in metabolic networks21 . However, it has a
379(1):307–316, 2007
couple of downsides. 21
Shihua Zhang, Xuemei Ning, and
First, finding all cliques in a network is computationally expensive. Xiang-Sun Zhang. Identification of
functional modules in a ppi network
One could fix this problem by setting k to be relatively high. If we by clique percolation clustering. Com-
set k = 5 we know that nodes with degree three or less cannot be putational biology and chemistry, 30(6):
in any community, because they need at least four edges to be part 445–451, 2006

of a 5-clique. Since most networks have broad degree distributions

22
Jussi M Kumpula, Mikko Kivelä,
Kimmo Kaski, and Jari Saramäki.
(Section 6.3), this means that we can safely ignore the vast majority of Sequential algorithm for fast clique
the network, thus reducing the number of operations we need to find percolation. Physical Review E, 78(2):
026109, 2008
communities. This is a suboptimal solution because it implies that 23
Fergal Reid, Aaron McDaid, and Neil
one will not classify most nodes into communities. For this reason, Hurley. Percolation computation in
there are developments of this algorithm22 , 23 that are a bit more complex networks. In 2012 IEEE/ACM
International Conference on Advances in
computationally efficient. Social Networks Analysis and Mining,
Second, it has limited coverage for sparse networks. That means pages 274–281. IEEE, 2012
474 the atlas for the aspiring network scientist

that it might end up being unable to classify nodes in networks

because they are not part of any clique. If you set your k relatively
low, e.g. k = 4, all nodes with degree equal to one cannot be part
of any community. This is because a node with degree equal to one
cannot be part of a 3-clique. Thus it will never be merged into any
4-clique, which are the basis of our communities.

Node Splitting
Another approach is to simply recognize that a node is part of multi-
ple communities if it has different identities. This is extremely similar
to the approach of overlapping Infomap. In that case we represented
the two identities of the node by giving it two different codes: one
per community to which it belongs. Here we literally split it in two.
We modify the structure of the network in such a way that, when we
are done, by performing a normal non-overlapping community dis-
covery we recover the overlapping clusters. In the resulting structure
we have multiple nodes all referring to the same original one.
If we want to split nodes, we need to answer two questions: which
nodes do we split and how. First we identify the nodes most likely
to be in between communities. If you remember the definition of
24
Steve Gregory. An algorithm to find
betweenness, you’ll recollect that nodes between communities are the overlapping community structure in
gatekeepers of all shortest paths from one community to the other. So networks. In European Conference on
they are the best candidates to split. There are many ways to perform Principles of Data Mining and Knowledge
Discovery, pages 91–102. Springer, 2007
the split, but I’ll focus on the one that involves calculating a special 25
Steve Gregory. Finding overlapping
betweenness: pair betweenness24 , 25 . Pair betweenness is a measure communities using disjoint community
for a pair of edges: the number of shortest paths that use both of detection algorithms. In Complex
networks, pages 47–61. Springer, 2009
them.

5 2 5 2 2 Figure 34.7: Attempting to find

2 2 the best node to split in (a). Se-
1 0 0 lecting node 1 as the candidate,
we build a split betweenness
4 3 4 2 3 graph (b). I label each edge in
(b) with its split betweenness.
(a) (b)

For instance, consider the graph in Figure 34.7(a). The most central
node is node 1. To try and split it, we build its split graph. Meaning
that we remove node 1 and we connect all nodes that were connected
by 1. Each edge has a weight: the number of shortest paths in the
original graph that passed through node 1. In this case, there are
two shortest paths using the (4, 1) and (1, 3) edges: the one going
from node 4 to node 3 and the one going from node 3 to node 4. We
can represent the pair betweenness of all neighbors of node 1 with a
 overlapping coverage 475

4-5 2 2 5 2

4 0 4-5 8 2-3 1 1

3 4 3
(b)

(a) (c)
Figure 34.8: (a-b) Merging the
weighted clique (Figure 34.7(b)). nodes connected by the weakest
To find the split we use a simple algorithm: we identify the edges split betweenness edges. (c) The
with the lowest pair betweenness and we merge the nodes connected resulting split in the original
by those edges (Figures 34.8(a-b)). At each merge, we sum up the graph.
pair betweennesses of all edges that got merged together by the merg-
ing of the node. Once we have one remaining edge, the resulting
split is the best one. The reason is that edges with low pair between-
ness are likely to be in the same community. Once you identify the
split (Figure 34.8(c)), it is easy to find disjoint communities and then
merge them into overlapping.

34.4 Latent Structural Approaches

In this section we have a collection of methods that make an as-

sumption: the observed community division of the network is the
result of a latent structure. In this latent structure, we have nodes
assigned to communities. Then, the probability of observing an edge
between two nodes is proportional to the number of communities the
two nodes have in common in the latent structure. The two classes
of approaches in this category I consider are Mixed Membership
Stochastic Blockmodels (MMSB) and the community affiliation graph.

Mixed Membership Stochastic Blockmodels

The description of the latent structural approaches I just wrote
should turn on a light bulb in your head. The idea that the proba-
bility of connecting two nodes is dependent on a latent community
partition is not new: it is exactly the starting point of the stochastic
blockmodel approach. In SBM, we assume there is a partition of
nodes and we assign a higher probability of connection between
nodes in the same partition than the one between nodes in different
partitions. The little problem we need to solve now is how to make 26
Edoardo M Airoldi, David M Blei,
this mathematical machinery work when we want to allow nodes Stephen E Fienberg, and Eric P Xing.
Mixed membership stochastic block-
to be part of multiple communities. That solution constitutes the models. Journal of Machine Learning
Mixed Membership Stochastic Blockmodels26 , an object that I already Research, 9(Sep):1981–2014, 2008
476 the atlas for the aspiring network scientist

mentioned in Section 15.2.

The trick here is that we represent each node’s membership as
a vector. The vector tells us how much the node belongs to a given 27
Wenjie Fu, Le Song, and Eric P
community. Then, we also have a community-community matrix, Xing. Dynamic mixed membership
blockmodel for evolving networks. In
that tells us the probability of a node belonging to community c1 Proceedings of the 26th annual interna-
to connect to a node belonging to a community c2 . These are the tional conference on machine learning,
two ingredients that replace the simple community partition in the pages 329–336. ACM, 2009

regular SBM. From this moment on, you attempt to find the set 28
Eric P Xing, Wenjie Fu, Le Song,
et al. A state-space mixed membership
of community affiliation vectors and the community-community blockmodel for dynamic network
probability matrix that are most likely to reproduce your observed tomography. The Annals of Applied
Statistics, 4(2):535–566, 2010
data, exactly as you do in SBM. 29
Qirong Ho, Le Song, and Eric Xing.
Just like we saw in Section 31.4, we can have dynamic MMSB, Evolving cluster mixed-membership
adding time to the mix27 , 28 , 29 , 30 : the community affiliation vectors blockmodel for time-evolving networks.
In Proceedings of the Fourteenth Interna-
and the community-community matrix can change over time. There tional Conference on Artificial Intelligence
is also a hierarchical (Chapter 33) variant of MMSB, allowing a nested and Statistics, pages 342–350, 2011
community structure31 . 30
Kevin S Xu and Alfred O Hero.
Dynamic stochastic blockmodels:
Statistical models for time-evolving
Community Affiliation Graph networks. In International conference
on social computing, behavioral-cultural
Affiliations graphs have been often used to describe the overlapping modeling, and prediction, pages 201–210.
Springer, 2013
community structure of real world networks32 . In a community 31
Tracy M Sweet, Andrew C Thomas,
affiliation graph you assume that you can describe your observed and Brian W Junker. Hierarchical mixed
network with a latent bipartite network. In this bipartite network, membership stochastic blockmodels for
multiple networks and experimental
the nodes of one type are the nodes of your observed network. The interventions. Handbook on mixed
other type, the latent nodes, represent your communities. Nodes are membership models and their applications,
pages 463–488, 2014
connected to the communities they belong to. This is the community 32
Jae Dong Noh, Hyeong-Chai Jeong,
affiliation graph, because it describes the affiliations to communities Yong-Yeol Ahn, and Hawoong Jeong.
of your nodes. Growing network model for community
with group structure. Physical Review E,
71(3):036131, 2005
2
1 Figure 34.9: (a) A graph with
4
7 overlapping communities indi-
6 1 2 3 4 5 6 7 8 9 10 11 12 13
3
cated by the colored outlines.
5
8
(b) Its corresponding com-
10
9 munity affiliation graph. The
13
community latent nodes are
11 12 (b)
triangular and their color cor-
responds to the color used in
(a) (a).
Figure 34.9 shows a representation of a community affiliation
graph. Of course, you can build such a graph easily once you already 33
Jaewon Yang and Jure Leskovec.
know to which communities the nodes belong. The hard part is Overlapping community detection at
scale: a nonnegative matrix factorization
finding out the best representation. There are a few ways to do so,
approach. In Proceedings of the sixth ACM
usually relying on the expectation maximization algorithm that is international conference on Web search and
also at the basis of the MMSB. One such approach is BigClam33 , data mining, pages 587–596. ACM, 2013
 overlapping coverage 477

which uses non-negative matrix factorization as the guiding principle

(see Section 5.4 for a refresher).

34.5 Clustering Links

An alternative approach is to look at edges. Why looking at edges?

Because people can be in multiple communities, as we saw: work
mates, school mates, etc. However, a link usually is created for one
single reason: you met that person in one situation and that’s the
defining characteristic of your relationship. You originally met u as
a work colleague, and v as a school mate. So one can cluster links
with a non-overlapping method and say that a person is part of all
the communities to which their edges belong.
There can be many similarity measures and link communities
can be found by adapting and optimizing such functions to the link
community case.

Line Graphs
To cluster the edges rather than the nodes we can transform the 34
TS Evans and Renaud Lambiotte. Line
network into its corresponding line graph34 . In a line graph, as we graphs, link partitions, and overlapping
communities. Physical Review E, 80(1):
saw in Section 3.1, the edges become nodes and they are connected
016105, 2009
if they’re incident on the same node. A way to do so is to generate a
weighted line graph.

5 2 4-5 1-4 1-5 1-2 1-3 2-3 Figure 34.10: (a) A simple
1 graph. (b) A bipartite version of
(a) connecting each node to its
4 5 1 2 3
4 3 edges.
(b)
(a)
To create a line graph you first transform the network into bi-
partite connecting the nodes to the edges they are connected to, as
Figure 34.10 shows. Then you project this network over the edges.
The most important thing to define is how to weight the edges in
the line graph. Different weight profiles will steer the community
discovery on the line graph in different directions.
You could use any of the weighting schemes I discussed in Chap-
ter 23, but the researchers proposing this method also have their
suggestions. The reason you might need a special projection is be-
cause you want nodes that are part of an overlap to give their edges
lower weights, because their connections are spread out in different
communities.
At that point, a disjoint community discovery will downplay
478 the atlas for the aspiring network scientist

1-4 1-2 5 2 Figure 34.11: (a) The blue cir-

4-5 2-3 1 cles are disjoint communities
1-5 1-3 4 3
in the line graph. (b) The red
circles are the corresponding
(a) (b) overlapping communities in the
original graph.
the weak edges and find communities with strong edge weights,
as Figure 34.11(a) shows. Once we bring back the communities to
the other side of the projection, as I do in Figure 34.11(b), we have
overlapping ones.
Other approaches in this class exist, including random walks on 35
Xiaoheng Deng, Genghao Li, and
line graphs35 . Mianxiong Dong. Finding overlapping
communities with random walks on
line graph and attraction intensity.
Hierarchical Link Clustering In International Conference on Wireless
Algorithms, Systems, and Applications,
In Hierarchical Link Clustering36 (HLC), the first step is to calculate pages 94–103. Springer, 2015
a measure of similarity between two edges. We only calculate it for
36
Yong-Yeol Ahn, James P Bagrow, and
Sune Lehmann. Link communities reveal
edges sharing one node: edges (u, k) and (v, k) share node k. If two multiscale complexity in networks.
edges share no node, their similarity is zero. If the edges share a nature, 466(7307):761, 2010
node, their similarity is the Jaccard coefficient of the neighborhoods
of the two non-shared nodes:

| Nu ∩ Nv |
S(u,k),(v,k) = .
| Nu ∪ Nv |

1
3 Figure 34.12: A graph and its
8 best link communities. The
2 color of the edge represents its
4 7 community. Nodes are part of
9 all communities of their links.
6 For instance, node 4 belongs to
5 three communities: red, blue,
and purple.
The edges with the highest S value are merged in the same com-
munity. For instance, in Figure 34.12, edges (1, 2) and (1, 3) have a
high S value: the neighborhoods of nodes 2 and 3 are identical, thus
S(1,2),(1,3) = 1. On the other hand, edges (4, 7) and (7, 8) only have
one node in the numerator, thus: S(4,7),(7,8) = 1/6.
Then, the merging happens recursively for lower and lower S
values, building a full dendrogram, as we saw in Chapter 33 for
hierarchical community discovery. We then need a criterion to cut
the dendrogram. We cannot use modularity, because these are link
communities, not node communities.
 overlapping coverage 479

The original authors develop a new quality measure called “parti-

tion density”. For each link community c, we have | Ec | as the number
of edges it contains, and |Vc | as the number of nodes connected to
those edges. Its density Dc is

| Ec | − (|Vc | − 1)
Dc = ,
|Vc |(|Vc | − 1)/2 − (|Vc | − 1)
which is the number of links in c, normalized by the maximum
number of links possible between those nodes (|Vc |(|Vc | − 1)/2), and
its minimum |Vc | − 1, since we assume that the subgraph induced by
c is connected. Note that, if |Vc | = 2, we simply take Dc = 0. All Dc
scores for all cs in your link partition are aggregated to find the final
partition density, which is the average of Dc weighted by how many
1
links are in c: D = ∑ | Ec | Dc .
| E| c

Ego Networks
Assuming that links exists for one primary reason works usually
well, but it is a problematic assumption. Let’s look back at the case
of work and school communities. What would happen if you were to
end up working in the same company and play in the same team of a
former schoolmate? Is it still fair to say that the link between the two
of you exists for only one predominant reason?
Modeling truly overlapping communities can get rid of this prob-
lem. There are many ways to do it, but we’ll focus on one that is
easy to understand. The starting observation is that networks have
large and messy overlaps. However, just like in the assumption of
clustering links, here we realize that the neighbors of a node usually
are easier to analyze. It is easy for a node to look at a neighbor and
say: “I know this other node for this reason (or set of reasons)”. 37
Michele Coscia, Giulio Rossetti, Fosca
The procedure37 works as follows, and I use Figure 34.13 to guide Giannotti, and Dino Pedreschi. Demon:
you. First, we extract the ego network of a node, removing the ego a local-first discovery method for
overlapping communities. In Proceedings
itself. This creates a simpler network to analyze. In the figure, I of the 18th ACM SIGKDD international
start by looking at node 1 on the top right. This is a graph with two conference on Knowledge discovery and
data mining, pages 615–623. ACM, 2012
connected components: one connecting nodes 2 and 3, the other
connecting nodes 4 and 5.
Then, we apply a disjoint community discovery algorithm to the
ego network. The ego network is easier to analyze and often has
easily distinguishable communities. In the example, these are the
blue and red outlines, which find two trivial communities. We repeat
the process for all nodes in the network, extracting all ego networks:
in the figure I show the ego networks of nodes 2 and 3, with the
communities I find in those cases, the green and purple outlines,
respectively. Note that I omit the ego networks for nodes 4 and 5, but
480 the atlas for the aspiring network scientist

Node 1’s ego network

5 2
Figure 34.13: The process of
5 2 community discovery via the
1 breaking down of the network
into ego networks.
4 3
4 3
Apply CD

{2,3}
1 2 {1,2} Merge similar {1,2,3}
{1,3}
3 1
{4,5}
{1,5} Merge similar {1,4,5}
Repeat for all nodes {1,4}

you can hopefully see where this is going.

Once we’re done, we have a set of communities, and we can
merge them according to some criterion. In the case of the figure,
we merge communities if they share at least one node that is not
part of too many communities. So we merge {2, 3} to {1, 2} and
{1, 3} on the basis of them sharing nodes 2 and 3. Same reasoning
for merging {4, 5} to {1, 4} and {1, 5}. We don’t merge {1, 2, 3} and
{1, 4, 5}, because the only node they have in common is 1, which is in
common with all communities in the network.
The original paper uses label propagation as the community
discovery algorithm to apply to each network, but this needs not
to be the case. We can instead apply a naive overlap algorithm.
This move allows us to solve the problem of the assumption we
mentioned before: we’re allowing the links to have multiple origins,
thus we don’t necessarily force each link to be present for exclusively
one reason.

34.6 Other Approaches

An interesting and more general approach to overlapping commu- 38

Andrea Lancichinetti, Filippo Radic-
nity discovery is the Order Statistics Local Optimization Method 38 chi, José J Ramasco, and Santo Fortu-
(OSLOM). In reality, OSLOM is a bit of a Swiss army knife, in the nato. Finding statistically significant
communities in networks. PloS one, 6(4):
sense that it isn’t limited by finding overlapping communities: it can e18961, 2011
deal with edge weights and directions, hierarchical and evolving
communities. Its basic philosophy is extremely similar to modularity:
it builds an expected number of edges in a community by means of a
configuration model.
Differently from modularity, OSLOM attempts to establish the
statistical significance of the partition. That is, it asks how likely it is
to observe the given community subgraphs in a configuration model.
The less likely a vertex is to be included in a community in a null
 overlapping coverage 481

model, the more likely it is that we should add it to the community.

Thus, these p values can be used as a mean of ranking the next move,
a move being adding a node to a community.
One can see how OSLOM is also a hierarchical community discov-
ery method of the merge type: it assumes all nodes being on their
own community at the beginning, and then it progressively merges
them. Differently from classical hierarchical CD, a node is still a
merge candidate even after it has been added to a community, al-
lowing overlap. Moreover, OSLOM can be used as a post-processing
strategy, to refine the communities you already found using another
method. In fact, one could use OSLOM to transform a hard disjoint
partition into an overlapping coverage.

34.7 The Overlap Paradox

Let’s go back to the beginning of this chapter. Let’s reiterate the

definition of a community in a network:

Communities are groups of nodes densely connected to each other and

sparsely connected to nodes outside the community.

As we’ve been seeing, overlapping communities make a para-

dox arise in our definition of community. If we have no overlap
the “denser inside” and “sparser outside” assumption works well.
However, if we add overlap, we cannot have it both ways.

Figure 34.14: A graph (a) and

its stochastic blockmodel (b).

(b)
(a)
If there are nodes in between communities either of two things
will happen. The nodes in the overlap could connect with all nodes
in both communities but not to each other, to maintain the “external
sparsity” condition. But doing so contradicts the “internal density”
part, because the overlap nodes do no connect to each other even
though they belong to the same community. Figure 34.14 provides an
example for this scenario.
In Figure 34.14(a) we have a graph made by four 5-cliques and
four sets of four nodes overlapping between two neighboring cliques.
482 the atlas for the aspiring network scientist

The overlap nodes don’t connect to each other. Figure 34.14(b) shows
how a stochastic blockmodel would interpret such a structure. You
can clearly see that there are “holes” in the communities where the
overlap nodes should be. If the overlap nodes don’t connect to each
other, they have low connection probability, which contradicts the
fact that they are part of the same community.

Figure 34.15: (a) The same

graph as the one in Figure
34.14, but the added edges
are highlighted in blue. (b)
The corresponding stochastic
blockmodel.

(b)
(a)
If, on the other hand, we maintain the “internal density” condi-
tion, since these nodes share not one but two communities, then
they are more likely to connect to each other than nodes sharing
only one community. In doing so, we end up with the opposite prob-
lem: breaking external sparsity. The overlap, which by definition is 39
Jaewon Yang and Jure Leskovec.
between the two communities, is denser than the community itself39 ! Community-affiliation graph model
In Figure 34.15(a) we have such a scenario, with a graph similar for overlapping network community
detection. In 2012 IEEE 12th international
to the one from Figure 34.14(a). But here all the overlap nodes are conference on data mining, pages 1170–
connected to each other, and they are connected more strongly than 1175. IEEE, 2012
non-overlap nodes, given that they share more communities with
each other. The corresponding stochastic blockmodel (Figure 34.15(b))
now shows that the communities themselves look weaker than the
overlap.
This is another reason why our golden rule, the standard defi-
nition of communities in complex networks, isn’t as shiny as we
originally thought.

34.8 Summary

1. In real world networks, communities can overlap, meaning that

nodes can be part of multiple communities at the same time. For
instance, you’re part of both the community of your high school
friends, and of your university colleagues.

2. Many quality measures like modularity or mutual information

cannot deal with overlapping communities. Thus we have several
extensions that allow them to take into account node-sharing
communities.
 overlapping coverage 483

3. Disjoint community discovery algorithms can be adapted to

find overlapping communities, for instance by means of fuzzy
clustering: each node is given a “belonging coefficient” for each
community, and can have multiple coefficients larger than zero.

4. Explicit structural approaches define the structure of an over-

lapping community and attempt to find it in the network. For
instance, by percolating cliques, or splitting nodes so that each of
their copies can belong to different communities.

5. Alternatively, one can divide edges into communities rather than

nodes. In such approaches, the nodes belong to all communities to
which their connections belong.

6. Overlapping communities put our classical community definition

in crisis: if it is true that the more communities two nodes share
the more likely they are connected, then the overlap of multiple
communities is denser than the communities themselves, i.e. there
are more links going outside communities than inside.

34.9 Exercises

1. Use the k-clique algorithm to find overlapping communities in the

network at [Link]
txt. Test how many nodes are part of no community for k equal to
3, 4, and 5.

2. Compare the k-clique results with the coverage in [Link]

[Link]/exercises/34/2/[Link], by using any varia-
tion of overlapping NMI from [Link]
Overlapping-NMI. For which value of k do you get the best perfor-
mance?

3. Implement the ego network algorithm: for each node, extract its
ego minus ego network and apply the label propagation algorithm,
then merge communities with a node Jaccard coefficient higher
than 0.1 (ignoring singletons: communities of a single node). Does
this method return a better NMI than k-clique percolation for
k = 3?
35
Bipartite Community Discovery

So far we have extended the community discovery problem by

adding or discussing features of the output: do we want communities
to share nodes or not? Do we want to have some sort of hierarchi-
cal optimization? In this and in the next chapter we instead discuss
advancements on the other direction: what if our input is special?
Here we deal with the case of bipartite networks, leaving multilayer
networks for the next chapter.
We start by briefly amending modularity to the bipartite case.
The bulk of the chapter is dedicated to alternative and more special-
ized ways to find bipartite communities. As usual, you can find a 1
Taher Alzahrani and Kathy J Horadam.
specialized survey of bipartite community detection methods1 . Community detection in bipartite
networks: Algorithms and case studies.
In Complex systems and networks, pages
25–50. Springer, 2016
35.1 Evaluating Bipartite Communities

Since it has been a looming presence across this entire book part, let’s
start again with modularity, the elephant in the room of community
discovery. Network scientists in the community detection business
love modularity. If there is a scenario in which modularity doesn’t
work, they panic and start amending it to hell, until it works again.
We’ve seen this with directed and overlapping community discovery,
and we’re seeing it again.
There are a couple of alternatives when it comes to define a modu-
larity that works for bipartite networks. If you remember the original
version of the modularity, it hinges on the fact that we want the parti-
tion to divide the network in communities that are denser than what 2
Michael J Barber. Modularity and
we would expect given a null model – the configuration model. Thus, community detection in bipartite
networks. Physical Review E, 76(6):066102,
extending modularity means to find the right formulation of a null 2007
model for bipartite networks2 , 3 . 3
Roger Guimerà, Marta Sales-Pardo,
This is not that difficult, the only thing to keep in mind is that and Luís A Nunes Amaral. Module
identification in bipartite and directed
the expected number of edges in a bipartite network is different networks. Physical Review E, 76(3):036102,
than in a regular network. So, while in the traditional modularity 2007
 bipartite community discovery 485

ku kv
the configuration model connection probability was , here it is
2| E |
ku kv
instead , with the added constraints that u and v needs to be
| E|
nodes of unlike type. The sum of modularity is made only across
pairs of nodes of unlike types, otherwise we would have negative
modularity contributions from nodes that cannot be connected,
which would make the modularity estimation incorrect.
To see why this is the case, suppose that we’re checking u and v
and they are of the same type. Since they are of the same type and
we’re in a bipartite network, they cannot connect to each other, so
Auv = 0. But they are both part of the network, thus k u 6= 0 and
ku kv ku kv
k v 6= 0. Thus > 0, meaning that Auv − < 0. Negative
| E| | E|
modularity contribution.
Once you have a proper bipartite modularity you can use any of
the modularity maximization algorithms to find modules in your 4
Stephen J Beckett. Improved com-
network, or even specialized ones4 . munity detection in weighted bipartite
networks. Royal Society open science, 3(1):
140536, 2016
35.2 Via Projection

As we saw in previous sections, bipartite networks have nodes of

two different types, and edges are established exclusively between
nodes of different types. In Netflix, we have users watching movies.
It’s natural to want to find communities in these networks. You want
to know which movies are similar to each other so you can suggest
them to users that are similar to each other.
In this case there is an easy obvious strategy. You take the bipartite
network, you project it to unipartite using one of the techniques we
saw in Chapter 23 – simple or hyperbolic weighting, random walks,
etc – and then you apply a normal unipartite community discovery
to the result. Then you can project on the other set of nodes and find
the other communities.
There are a couple of issues with this strategy. The first is that
by projecting you’re losing information. You connect movies with
a weighted edge, which carries a quantitative information. But the
bipartite network had qualitative information: a structure of users
watching different things. That information is lost.
This is related to the second issue: once you project your network
on your two types of edges and find their communities, you have
movies grouped together because watched by the same users. But
you don’t know who those users are. Same with communities of
users: which are their common movies? You have to go back to
5
Martin G Everett and Stephen P
Borgatti. The dual-projection approach
the bipartite network to know. A way to solve this issue is to use for two-mode networks. Social Networks,
the dual projection approach5 . In dual projection, you project the 35(2):204–210, 2013
486 the atlas for the aspiring network scientist

bipartite networks into its two unipartite versions and then you
analyze them at the same time with specialized techniques. This dual 6
David Melamed. Community struc-
projection approach has been applied to community discovery6 , with tures in bipartite networks: A dual-
encouraging results. projection approach. PloS one, 9(5):
e97823, 2014

Figure 35.1: Examples showing

different bi-structures project-
ing to the same uni-structures.
Both a 1,4-clique (top) and six
1,2-cliques (bottom) project into
a 4-clique.

There is an additional, more subtle, issue with this strategy.

There are a number of different structures in bipartite networks
that projects into the same unipartite graphs. This means that a
proper bipartite community discovery algorithm and its hypothetical
unipartite version will return different results, even if someone runs
the mirror algorithm on the projection of the bipartite graph. Figure
35.1 shows two different bipartite networks that project to the same
result. As the figure shows, projecting always means losing informa-
tion. If we were to perform such projection we wouldn’t be able to
distinguish between the two cases Figure 35.1 shows, while a proper
bipartite community detection algorithm could.

35.3 Direct Bipartite Module Detection

Bi-Clique Percolation
The solution is to perform the community discovery directly on the
bipartite structure. Here, we use the concept of bi-clique we saw
earlier. Remember that a clique is a set of nodes in which all possible
edges are present. A bi-clique is the same thing, considering that
some edges in a bipartite network are not possible. For instance, a
5-clique in a unipartite network is a graph with five nodes and ten
edges. In a bipartite network, a 2,3-clique has two nodes of type 1,
three nodes of type 2, and all nodes of type 1 are connected to nodes
 bipartite community discovery 487

Two 1,3-cliques Figure 35.2: Examples of bi-

sharing a 0,2-clique clique percolation. The top two
examples will lead to percola-
tion because they satisfy the
n − 1,m − 1 constraint. The last
Two 2,2-cliques example on the bottom does
sharing a 1,1-clique not.

Two 2,4-cliques
sharing a 0,1-clique

of type 2 – six edges in total. This is the starting point of bipartite

community discovery.
Bi-clique percolation works in the same way as the unipartite
k-clique percolation algorithm – described in Section 34.3 –, with
the added headache of having two numbers of nodes to keep track,
because now it’s a biclique. Communities are n,m-cliques (again, n
and m are parameters) and they get merged if they share an (n −
1)(m − 1)-clique. Figure 35.2 shows some examples of bi-clique
percolation. Note that any combination of n nodes of the same type
can be a 0,n-clique, thus allowing percolation.
The bi-clique percolation algorithm inherits from its predecessor
the ability of returning overlapping communities. Thus in this case
we get not only bipartite communities, but these communities can
also share nodes.

Adapting Classical Approaches

As we already got used to see, many of the classical approaches to

community discovery can be adapted to take into account complica-
tions in the network structure. Bipartite community discovery is no
7
Masoumeh Kheirkhahzadeh, Andrea
Lancichinetti, and Martin Rosvall.
exception. So let’s see what we can do to transform random walks, Efficient community detection of
label propagation, and stochastic blockmodels to the bipartite case. network flows for varying markov
times and bipartite networks. Physical
There are a couple of ways to exploit random walks and find Review E, 93(3):032309, 2016
bipartite communities7 , 8 . We already considered one: simply project 8
Taher Alzahrani, Kathy J Horadam,
the network as unipartite and perform the random walks normally. and Serdar Boztas. Community
detection in bipartite networks using
The issue is that the abundance of links will lower the power of random walks. In Complex Networks V,
random walkers, because the network will be too dense and the pages 157–165. Springer, 2014
488 the atlas for the aspiring network scientist

boundaries between communities will be difficult to find. That is

why you might also want to perform network backboning (Chapter
24).
Alternatively, you could perform high-order random walks. We
saw in Chapter 30 that we can add a parameter to a random walker,
telling it how much time it passes between one step and another.
You can effectively model 2-steps random walks this way, which will
allow you to find the communities for nodes of one type – and then
of the other type, by repeating the process with a different starting
point.

1 4 1 4 Figure 35.3: (a) Communities

from a synchronous label prop-
agation at an hypothetical time
t (the node color is the commu-
2 5 2 5
nity label). Each node of circle
type has a blue label majority
in its neighborhood, and each
3 6 3 6 triangle has a red majority. (b)
At time t + 1 the labels oscillate
(a) (b)
according to the label majority
Classical label percolation community discovery can be tricky. The at time t. The system will oscil-
first reason is that synchronous label percolation has one problem late forever between (a) and (b)
known as a label oscillation. Figure 35.3 provides an example. After a without converging.
few iterations, it might be that a community of nodes has a majority
label on one side and a different majority label on the other side. The
algorithm will be then stuck oscillating the labels at each time step.
9
Lovro Šubelj and Marko Bajec. Robust
In such a scenario, it will never converge9 . network community detection using
One could solve the issue in various ways. First, one could simply balanced propagation. The European
use asynchronous updating. However, traditional asynchronous up- Physical Journal B, 81(3):353–362, 2011

dating will update nodes in a random order. This might impact the
stability of the resulting partition, because the order in which nodes
are updated matters. Two subsequent runs of the algorithm could
yield very different results. Moreover, it might impact convergence
time, because there are many node orders which will still result in
label oscillation. The most sure way to prevent label oscillation is to
update first all nodes of one type and then all nodes of the other.
There are a few other ways to prevent oscillation. First, one could
10
Xin Liu and Tsuyoshi Murata. Com-
integrate label propagation with the modularity approach10 . In munity detection in large-scale bipartite
this scenario, one doesn’t run label propagation until convergence, networks. Transactions of the Japanese
but only for a few steps. Then they would refine the communities Society for Artificial Intelligence, 25(1):
16–24, 2010
by maximizing bipartite modularity. Alternatively, one could put
constraints on how we allow labels to propagate. For instance, we
could force communities to be of comparable sizes in number of
 bipartite community discovery 489

11
Michael J Barber and John W Clark.
nodes or edges11 . Detecting network communities by
Finally, we can adapt stochastic blockmodels to the bipartite case, propagating labels under constraints.
Physical Review E, 80(2):026129, 2009
creating a biSBM12 . This has some similarities with the MMSB we 12
Daniel B Larremore, Aaron Clauset,
saw for overlapping community discovery in Section 34.4. First, we and Abigail Z Jacobs. Efficiently
don’t look directly at the |V1 | × |V2 | biadjacency matrix B. It is more inferring community structure in
bipartite networks. Physical Review E, 90
convenient to look at its adjacency matrix equivalent:
(1):012805, 2014
!
0 B
A=
BT 0
The zeros on the main diagonal mean that nodes of the same type
cannot connect to each other, enforcing the bipartite structure (see
Section 5.1).
Then, just like in the overlapping case, we can have a special
community-community matrix that tells us the probability of nodes
in two distinct communities to connect to each other. The special
condition here is that communities grouping nodes of the same type
will have zero probability of connecting to each other, respecting the
bipartite constraint.
Once we have these two special structures in place, one can pro-
ceed finding the most likely blockmodel that explains the observed
data, which is the one with the best community partition, with the
same strategies as in vanilla SBM. Note that this method can be triv-
ially extended to multi-partite networks, modifying the fundamental
structures accordingly.
The biSBM clusters the two modes separately, so you get a mixing
matrix that tells you how the groups in the V1 nodes interact with the
groups in the V2 nodes. In contrast, bipartite modularity and some
other approaches will produce mixed groups, which contain nodes
from both V1 and V2 . This makes biSBM a co-clustering method, like
the ones we’ll see in Section 35.4. The difference with those methods
is that they find communities discovery via neighbor similarity,
which is not the philosophy of biSBM.
There is a related method that works by means of matrix factoriza- 13
Zhong-Yuan Zhang and Yong-Yeol
tion13 . It starts by noticing that zeroes in A have different meanings. Ahn. Community detection in bipartite
The zeroes in the main diagonal block represent impossible connec- networks using weighted symmetric
binary matrix factorization. International
tions, connections that shouldn’t be penalized. The zeroes in the Journal of Modern Physics C, 26(09):
off-diagonal block instead represent edges that could exist. Thus, the 1550096, 2015
authors define a mask matrix M, with the same dimensions as A,
with zeros on the main diagonal blocks and ones in the off diagonal
blocks. By factorizing the product of M and A together with our best
guess at the community organization of A, we obtain a function we
can maximize to find the best community partition, knowing that
we’re only penalizing zeroes corresponding to connections that could
exist.
490 the atlas for the aspiring network scientist

Redefining the Clustering Coefficient

One reasonable way to find communities in unipartite networks is
by exploiting the clustering coefficient. Nodes with high clustering
coefficients are supposedly well embedded in a community, because
all their neighbors are connected to each other. Thus, an algorithm
cutting low local clustering coefficient areas could perform well.
If you recall Section 33.1, one such strategy is to derive an edge
clustering measure from the local node clustering, and then use it
to determine which edges to cut to find a hierarchical community
structure.
Our problem here is that the local clustering coefficient in bipartite
networks is zero for all nodes. This is because in bipartite networks
there cannot be triangles, which are at the basis of the computation of
the local clustering coefficient (Section 9.2). A triangle, by definition,
connects three nodes together. However, in bipartite networks, the
edge closing the triangle cannot exist, because it would connect two
nodes of the same type.

Figure 35.4: (a) The structure at

the basis of the clustering coeffi-
cient of unipartite networks: the
triangle. (b) Its equivalent in
bipartite networks: the square.

(a) (b)
This needs not to worry us. We can redefine the clustering coeffi-
cient to make sense in a bipartite network. In a unipartite network,
the triangle is the smallest non-trivial cycle, the one that does not
backtrack using the same edge, as you can see in Figure 35.4(a). We
can also have a smallest non-trivial cycle in bipartite networks. It
involves four nodes, as Figure 35.4(b) shows. So we can say that the
local clustering coefficient of a node in a bipartite network is the
number of times such cycles appear in its neighborhood, divided by 14
Peng Zhang, Jinliang Wang, Xiaojia
the number of times they could appear given its degree14 . Li, Menghui Li, Zengru Di, and Ying
Let us assume that we want to know the local square clustering Fan. Clustering coefficient and com-
munity structure of bipartite networks.
coefficient of node z. If we say that nodes u, v, and z are involved Physica A: Statistical Mechanics and its
in suvz squares, then contribution of nodes u and v to the square Applications, 387(27):6869–6875, 2008
clustering coefficient of z is:

suvz
C4u,v (z) = ,
suvz + (k u − ηuvz ) + (k v − ηuvz )
with ηuvz = 1 + suvz . In practice, the number of possible squares
 bipartite community discovery 491

(in the denominator) is the number of actual squares plus how many
additional squares you could have given u’s and v’s free edges, edges
not involved in any square. Here, u and v are the nodes of the same
type.

z
Figure 35.5: An example of
bipartite network on which we
u v can calculate the local cluster-
ing coefficient of node z.

b c a d

Consider Figure 35.5. In the figure, suvz = 1, because nodes u, v,

and z are involved in one square. As a consequence, ηuvz = 2. Since
k u = 4 and k v = 3, we know that there could be suvz + (k u − ηuvz ) +
(k v − ηuvz ) = 1 + (4 − 2) + (3 − 2) = 4 squares between the nodes:
uvza (which exists), uvzb, uvzc, and uvzd (which do not exist). Since
z has no additional neighbors, its local clustering coefficient is thus
1/4.
Once you have a properly defined local node clustering measure,
you can use it to derive the corresponding edge clustering measure.
Then, you can apply the same edge splitting algorithm we explored
in the hierarchical community discovery case to find hierarchical
bipartite communities.

35.4 Neighbor Similarity

A wholly different category of approaches tries to look at the adja-

cency matrix of a bipartite graph under a different perspective. The
point of a community in a bipartite network is not that the nodes
15
Simone Daminelli, Josephine Maria
connect densely to each other, but that they connect to the same Thomas, Claudio Durán, and Carlo Vit-
neighbors – nodes of the other type. Thus it’s not much about inter- torio Cannistraci. Common neighbours
nal density as it is about structural similarity (Section 12.2). Some and the local-community-paradigm for
topological link prediction in bipartite
approaches use some sort of common neighbor approach15 . networks. New Journal of Physics, 17(11):
113037, 2015

Figure 35.6: (a) A classical

unipartite community. (b) A
bipartite community.

(a)
(b)
492 the atlas for the aspiring network scientist

Figure 35.6 shows a way to perform such a mental pivot. In a

regular unipartite network – Figure 35.6(a) – we’re looking at a
community as a set of nodes that connect to each other. In a bipartite
network – Figure 35.6(b) – we’re looking at a community as a set of
nodes that connect to the same nodes. Thus, in a bipartite community,
we don’t require two nodes that are part of the same community to
connect to each other.
This is a fully valid definition of community that can be translated
to unipartite networks, which generates another way to look at
the community discovery problem. I have showed this as a valid
“community” of community discovery algorithms in Section 31.6.
The change in perspective might seem small at first, but it opens
up a sea of possibilities. When you look at an adjacency matrix as a
simple set of feature vectors, you can perform data clustering on it.
Meaning that you can see a node as a point in a multidimensional
space, a space with as many dimensions as there are nodes of the
other type in the network. Then you can pick any of your favorite 16
Hugo Steinhaus. Sur la division des
machine learning algorithms, spanning from k-means16 , 17 , 18 to corp materiels en parties. Bull. Acad.
dbscan19 , and interpret their clusters as communities. Polon. Sci, 1(804):801, 1956
You should also be encouraged to look at bi-clustering (or co-
17
James MacQueen et al. Some methods
for classification and analysis of multi-
clustering) techniques20 , 21 , 22 . The advantage of such techniques is variate observations. In Berkeley symp on
that they will cluster the rows and the columns of your matrix at the math statistics and probability, volume 1,
pages 281–297. Oakland, CA, USA, 1967
same time. In this way, you don’t have to manually map the clusters 18
Stuart Lloyd. Least squares quan-
you found by looking at the |V1 | × |V2 | matrix with the ones you tization in pcm. IEEE transactions on
found in the |V2 | × |V1 | matrix. information theory, 28(2):129–137, 1982
The way these methods find clusters is usually by estimating the
19
Martin Ester, Hans-Peter Kriegel, Jörg
Sander, Xiaowei Xu, et al. A density-
pairwise distance (Euclidean or otherwise) between data points. Then based algorithm for discovering clusters
clusters are sets of points that lump together in this complex space. in large spatial databases with noise. In
Kdd, volume 96, pages 226–231, 1996
Hopefully, boundaries between clusters are clear, as few points are 20
Inderjit S Dhillon. Co-clustering
equidistant from multiple cluster centers. In community discovery, documents and words using bipartite
your data points are the nodes. Figure 35.7 shows an example. spectral graph partitioning. In SIGKDD,
pages 269–274, 2001
Besides old data clustering techniques, there is a lot of excitement 21
Inderjit S Dhillon, Subramanyam
for the application of neural network approaches to community Mallela, and Dharmendra S Modha.
discovery23 . However, usually, adjacency matrices are too sparse Information-theoretic co-clustering. In
SIGKDD, pages 89–98, 2003
and constrained to provide a proper input to neural networks. Thus,
22
Yuval Kluger, Ronen Basri, Joseph T
most of the deep learning attacks to community detection use more Chang, and Mark Gerstein. Spectral
sophisticated representations of the relations between nodes in the biclustering of microarray data: coclus-
graph, in the form of graph embeddings, which we’ll explore more in tering genes and conditions. Genome
research, 13(4):703–716, 2003
depth later on (in Chapter 37). 23
Joan Bruna and X Li. Community
To wrap up this chapter, let’s recall again our definition of commu- detection with graph neural networks.
stat, 1050:27, 2017
nities in complex networks:

Communities are groups of nodes densely connected to each other and

sparsely connected to nodes outside the community.
 bipartite community discovery 493

1 2 3 4 5 6 7 8 9 10 11 12

9 8 7 6 5 4 3 2 1

(a) (b)
(c)
Figure 35.7: (a) A bipartite net-
The bipartite community discovery introduces another issue work. (b) Its adjacency matrix.
with the standard definition of community based on density. In (c) A 2D spatial representation
n,m-cliques, we have n + m nodes that cannot be connected to each of the circular nodes, using
other because the graph is bipartite. For instance, the community in their adjacencies to determine
Figure 35.8 has many missing links. To be precise, since n = 4 (the the position. Node color is its
triangles) and m = 7 (the circles), we have 4 × 3/2 + 7/2 = 27 missing cluster, as identified by spatial
connections that the classical definition would want. More than the clustering (dashed line).
connections actually there! Nodes of the same type cannot connect in
a bipartite graph – so they have density of zero –, but they can and
will be part of the same community. So again this criterion of internal
density is a bit flaky.

Figure 35.8: A possible bipartite

community.

35.5 Summary

1. In bipartite networks, nodes of the same type cannot connect to

each other. However, they could still be in the same community,
because they have lots of common neighbors. Thus we need to
adapt modularity and other community quality measures to take
this into account.

2. One way to perform bipartite community discovery is by project-

ing the bipartite network into unipartite form and then perform
community discovery there. However, the resulting network will
be too dense and we will lose information in the projection.

3. We can adapt clique percolation by percolating bi-cliques. We

can perform random walks by making them perform two steps
494 the atlas for the aspiring network scientist

at a time. We can also adapt label propagation by performing it

asynchronously and refining its results.

4. We can also redefine the local clustering coefficient, so that it is

based not on the number of triangles around a node (which cannot
exist in a bipartite network), but on the number of its surrounding
squares.

5. Alternatively, one could simply infer node similarity measures

by looking at their neighbors, or perform simple data clustering.
However, all these strategies show how our classical definition of
communities in networks cannot really capture the bipartite case.

35.6 Exercises

1. The network at [Link]

1/[Link] is bipartite. Project it into unipartite and find five
communities with the Girvan-Newman edge betweenness al-
gorithm (repeat for both node types, so you find a total of ten
communities). What is the NMI with the partition proposed at
[Link]

2. Now perform asynchronous label propagation directly on the

bipartite structure. Calculate the NMI with the ground truth. Since
asynchronous label propagation is randomized, take the average of
ten runs. Do you get a higher NMI?

3. Consider the bi-adjacency matrix as a data table and perform

bi-clustering on it, using any bi-clustering algorithm provided in
the scikit-learn library. Do you get a higher NMI than in the
previous two cases?
36
Multilayer Community Discovery

The last chapter of community discovery, at least for this book, fo-
cuses on multilayer networks. In multilayer networks, nodes can
belong to different layers and thus they can connect for different
reasons. In multilayer networks we want to find communities that
span across layers. For example, we want to figure out communities
of friends even if your friends are spread across multiple social media
platforms.
There are a few review works you can check out to have a more 1
Jungeun Kim and Jae-Gil Lee. Com-
in-depth exploration of the topic1 , 2 . Here, I go over briefly the main munity detection in multi-layer graphs:
approaches and peculiar problems of community discovery in multi- A survey. ACM SIGMOD Record, 44(3):
37–48, 2015
layer networks. 2
Obaida Hanteer, Roberto Interdonato,
Matteo Magnani, Andrea Tagarelli, and
Luca Rossi. Community detection in
36.1 Flattening multiplex networks, 2019

Similar to the bipartite case, there is a relatively simple solution. You 3

Michele Berlingerio, Michele Cos-
can flatten the network by collapsing nodes across layers3 , meaning cia, and Fosca Giannotti. Finding
that you reduce the multilayer network to a network with a single and characterizing communities in
multidimensional networks. In 2011
layer and weighted edges. The weights on the edges depend on the
International Conference on Advances in
multilayer connections. In practice, you’re collapsing a qualitative in- Social Networks Analysis and Mining,
formation – in which layer a connection appears – into a quantitative pages 490–494. IEEE, 2011a

one – an edge weight. This assumes that every edge type is equally
important. Then you can perform a normal mono-layer community
discovery. Figure 36.1 shows an example.
There are a few choices for your edge weights. The simplest
one could be to simply count the number of layers in which the
connection between the nodes appear. However, you might want to
take into account some interplay between the layers. For instance,
you can count the number of common neighbors that two nodes 4
Jungeun Kim, Jae-Gil Lee, and Sungsu
have and use that as the weight of the layer, under the assumption Lim. Differential flattening: A novel
that a layer where two nodes have many common neighbors should framework for community detection in
multi-layer graphs. ACM Transactions on
count for more when discoverying communities. Or you could use Intelligent Systems and Technology (TIST),
“differential flattening4 ”: flatten the multilayer graph into the single 8(2):27, 2017
496 the atlas for the aspiring network scientist

Figure 36.1: (a) A multilayer

network. (b) A weighted flatten-
ing of (a). An edge’s width is
proportional to its weight.

(a) (b)

layer version of it such that its clustering coefficient is maximized.

As with the bipartite case, simplistic solutions create problems.
The issue is the same: we lose information. When we translate a
layer into a weight, we don’t know any more which type of links
we’re looking at. Some types of links might contribute differently
to the communities. Moreover, flattening is not always possible,
or at least not straightforward. If a node in one layer is coupled
with multiple nodes in another layer, how do we represent it in the
flattened network?

36.2 Layer by Layer

There is another solution that is slightly more sophisticated than flat-

tening a multilayer network but that at the same time doesn’t require
you to go fully multidimensional in your analysis. It is performing
community discovery separately on each layer of your network and
then somehow combine your results. This is sort of like the approach
of dynamic community discovery where you perform your detection
on each snapshot of the network and then you aggregate the results
(Section 31.4).
In fact, one could see each snapshot of the network as a layer –
or vice versa: each layer as a snapshot. Thus anything we do on an
evolving network we could also do on a multilayer one. And – why
not? – we could even have evolving multilayer networks, putting 5
Marya Bazzi, Mason A Porter, Stacy
the two together5 . The solution is not ideal, though, because there Williams, Mark McDonald, Daniel J
are many assumptions you have on a dynamic networks that you Fenn, and Sam D Howison. Community
detection in temporal multilayer
might break on a multilayer network – and vice versa. For instance, networks, with an application to
in a dynamic graph you have some sort of continuity assumption: correlation networks. Multiscale Modeling
& Simulation, 14(1):1–41, 2016
one snapshot should be, in principle, similar to the next. There is no
requirement of similarity between layers: in fact, they can even be
strongly anti-correlated.
For these reasons, you need some specialized community discov-
ery approaches. In one approach, one could build a matrix where
 multilayer community discovery 497

each row is a node and each column is the partition assignment for 6
Lei Tang, Xufei Wang, and Huan Liu.
that node in a specific layer6 . This is then a |V | × |C | matrix. Then one Community detection via heteroge-
could perform kMeans on it, finding clusters of nodes that tend to be neous interaction analysis. Data mining
and knowledge discovery, 25(1):1–33, 2012
clustered in the same communities across layers. 7
Michele Berlingerio, Fabio Pinelli, and
A similar approach7 uses frequent pattern mining, a topic we’ll see Francesco Calabrese. Abacus: frequent
more in depth in Section 39.3. For now, suffice to say that we again pattern mining-based community dis-
covery in multidimensional networks.
perform community discovery on each layer separately. Each node
Data Mining and Knowledge Discovery, 27
can then be represented as a simple list of community affiliations. We (3):294–320, 2013c
then look for sets of communities that are frequently together: these
are communities sharing nodes across layers.

Node L1 L2 L3
1 C1L1 C1L2 C1L3
2 C1L1 C1L2 C1L3
3 C1L1 C1L2 C1L3
MLComm SLComms MLComm Nodes
4 C2L1 C1L2 C1L3
MLC1 C1L1, C1L2, C1L3 MLC1 1, 2, 3
5 C2L1 C1L2 C2L3
MLC2 C2L1, C1L2 MLC2 4, 5, 6
6 C2L1 C1L2 C3L3
MLC3 C2L2, C3L3 MLC3 7, 8, 9
7 C1L1 C2L2 C3L3
8 C2L1 C2L2 C3L3 (b) (c)
9 C2L1 C2L2 C3L3
10 C1L1 C2L2 C2L3
(a)
Figure 36.2 shows an example. In Figure 36.2(a) we have the Figure 36.2: (a) The communi-
communities found for each layer for each node. Then we decide that ties found in each layer of each
we want to merge communities if they have at least three nodes in node. (b) The merged multi-
common, i.e. they appear in at least three rows of the table. layer communities. (c) The final
Figure 36.2(b) shows the multilayer communities mapping and node-community affiliation.
there are many interesting things happening. First, we only want
maximal sets, meaning that we aren’t interested in returning C1L1 by
itself if we also find it in a larger set of communities. Second, we are
ok if a community gets merged in different sets – i.e. the multilayer
communities can overlap –: C1L2 is part of two maximal sets, MLC1
and MLC2. Figure 36.2(c) shows the final output: the multilayer com-
munity affiliation. A node is part of a multidimensional community
if it is part of all communities composing it. 8
Arlei Silva, Wagner Meira Jr, and
Mohammed J Zaki. Mining attribute-
Node 10 is an example of a final interesting thing: it is part of structure correlated patterns in large
no multidimensional community because its affiliation is a weird attributed graphs. Proceedings of the
VLDB Endowment, 5(5):466–477, 2012
combination of communities. We can decide to let it be without com- 9
Zhiping Zeng, Jianyong Wang, Lizhu
munity affiliation, or to allow it to be part only of its non-multilayer Zhou, and George Karypis. Coherent
communities. closed quasi-clique discovery from large
dense graph databases. In Proceedings
There are other algorithms solving the same problem and inspired
of the 12th ACM SIGKDD international
by frequent pattern mining8 , 9 . conference on Knowledge discovery and
The last solution for this section is inspired by ensemble cluster- data mining, pages 797–802. ACM, 2006
498 the atlas for the aspiring network scientist

ing. Again, we have a community per layer. Then we use the same
strategy I outlined in Section 31.6: we consider each community
partition in each layer as a valid clustering of the same underlying re- 10
Andrea Tagarelli, Alessia Amelio,
lationship via different datasets10 . We then find the “true” clustering, and Francesco Gullo. Ensemble-based
which is the partition that is the closest one to the combination of all community detection in multilayer
networks. Data Mining and Knowledge
partitions.
Discovery, 31(5):1506–1543, 2017
Aggregating communities across layers has some benefits. For 11
Dane Taylor, Saray Shai, Natalie
instance, it might solve the resolution problem of modularity11 that Stanley, and Peter J Mucha. Enhanced
I discussed in Section 32.1. However, all these methods have the detectability of community structure
in multilayer networks through layer
downside of relying more or less on the same assumption: that the
aggregation. Physical review letters, 116
layers are correlated to each other. While this might not be a bad (22):228301, 2016
assumption to start with12 , disassortative layers exist and might 12
Desislava Hristova, Mirco Musolesi,
and Cecilia Mascolo. Keep your friends
represent a problem. close and your facebook friends closer:
A multiplex network approach to the
analysis of offline and online social ties.
36.3 Multilayer Adaptations In Eighth International AAAI Conference
on Weblogs and Social Media, 2014
Multilayer Modularity
I already mentioned how obsessed networks scientists are with mod- 13
Peter J Mucha, Thomas Richardson,
ularity, so you know what’s coming next: multilayer modularity13 . Kevin Macon, Mason A Porter, and
Suppose we’re using the Louvain method, which grows communities Jukka-Pekka Onnela. Community
structure in time-dependent, multiscale,
node by node. If we found a triangle in a layer, can we extend it by and multiplex networks. science, 328
taking a node in a different layer? Intuitively yes, the edge should (5980):876–878, 2010
count because the node is the same. However, if we were to represent
this as a flat network, the new node is not densely connected to the
rest of the triangle: a node couples only with itself, not with its com-
munity fellows. So the coupling edges have to count in some special
way.
In practice, standard modularity works well in each layer sepa-
rately. Consider Figure 36.3: in modularity, the part testing for the
ku kv
density of the community is Auv − . If we use this same part for
2| E |
the inter-layer coupling, we would end up with a case in which the
community cannot be expanded across layers, because there are only
sparse connections between layers. A node couples only with itself in
a different layer, not connecting to its community members, making a
multi-layer community sparser than it actually is. So we need to add
something that will allow us to count the coupling links, so that we
don’t end up with the trivial result of all mono-layer communities.
The full formulation of multilayer modularity is the following:

  
1 k vs k us
2(| E| + |C |) ∑ Avus − γs
2| Es |
δsr + Cvsr δuv δ(cus , cvr ).
vusr

Let’s break it down – and you can check Figure 36.4 for a graphical
 multilayer community discovery 499

1 kv ku Figure 36.3: The issues in apply-

∑
2|E| vu [
A vu−
2|E| ]
δ (c v , c u ) ing modularity to the multilayer
case. The part looking at the
This applies only inside a layer community density only ap-
plies inside a single layer, thus
we need to add something to it
to consider the special coupling
edges.
Need to add something to
consider special coupling
links

representation and you can always go back to Section 32.1 to read

more about the notation of classical modularity and compare it with
this formulation. E and C are the sets of (intra-layer) edges and (inter-
layer) couplings. Avus is our multilayer adjacency tensor, it is equal to
1 if nodes u and v are connected in layer s, and it is 0 otherwise. k us
and k vs are the degrees of u and v in layer s, respectively. | Es | is the
number of edges in s.

1 k k
∑
2(|E|+|C|) vusr [( 2|E s| )
u,v = Nodes
]
A vus − γ s vs us δ sr +C vsr δuv δ (c us , c vr ) Figure 36.4: The adaptation of
modularity to the multilyer set-
ting. Each part of the formula
s,r = Layers
If s=r → same layer is underlined with a color corre-
Modularity in s sponding to its interpretation.
Importance of s
If u=v → same node
Coupling strength
If node u in layer s is in
the same community as
node v in layer r
Normalized by number
of links and coupling
links

The major complication in multilayer modularity is that we have

many Kronecker deltas (δ). The first is δsr : its role is to make sure
that standard modularity is applied inside a layer (when s = r,
δsr = 1, because s and r are the same layer). The second is δuv and it
checks whether we are looking at two nodes that are coupled across
layers: δuv is 1 if u = v. Note that δsr and δuv are mutually exclusive.
If s = r we are in the same layer and so it must be that u 6= v,
because we don’t have self loops. On the other hand, if u = v then
s 6= r, because by definition coupling connections go across layers,
500 the atlas for the aspiring network scientist

thus s and r must refer to different layers. Both deltas can be zero if
we’re looking at uncoupled nodes in different layers. The final delta,
δ(cus , cvr ) is the same as in standard modularity, it is equal to 1 only
if we are looking at nodes inside the same community, i.e. cus = cvr .
With γs we can regulate how important each layer s is for the
community. In practice, γ is a vector of weights, one per layer s of the
network.
Cvsr is the strength of the coupling link, which is a parameter just
like γ is: you can decide how strong the layer couplings should be.
It matters only when we’re looking at the same node connected by
a coupling link across layer (u = v), and so δuv is 1. In this case,
nothing else matters, because δsr is 0 (because s 6= r), so the standard
modularity part cancels out.
Just like in standard modularity, only nodes in the same commu-
nity contribute to the sum, so when node u in layer s is in the same
community as node v in layer r (meaning that δ(cus , cvr ) = 1). This
is normalized by the number of edges (| E|) across all layers plus all
coupling links (|C |).

Figure 36.5: The effect of the

Cvsr parameter on multilayer
modularity. (a) High values im-
ply pillar communities. (b) Low
values imply flat communities
(right).

(a) (b)
If you decide that your inter layer couplings are very strong,
you’ll end up with “pillar communities” where nodes tend to favor
grouping with themselves across layers: the inter layer couplings
trump any intra-layer regular edge. If your inter layer couplings
are weak (low Cvsr ) then you’ll end up with “flat communities” as
nodes prefer to group with other nodes in the same layer. I show an
example in Figure 36.5.
Instead, γ allows you to indicate some layers as more important
than others, as I show in Figure 36.6. If the purple layer is more
important than the green one, multilayer modularity will group in
the community a node that is not connected with the two nodes in
the green layer. If we flip the γ values to make green more important
than purple, the situation is reversed, and modularity will return
different communities.
An alternative way to adapt modularity maximization to multi-
 multilayer community discovery 501

Figure 36.6: The effect of the γs

parameter on multilayer mod-
ularity. (a) High γ for the top
(purple) community and low
for the mid (green) community.
(b) Low γ for the top (purple)
community and high for the
mid (green) community.

(a) (b)

layer networks is to adapt the Louvain algorithm (see Section 33.1) to

Inderjit S Jutla, Lucas GS Jeub, and
14
handle networks with multiple relation types14 . Peter J Mucha. A generalized louvain
method for community detection imple-
mented in matlab. URL [Link]
Other Approaches amath. unc. edu/GenLouvain, 2011

Modularity is not the only algorithm that can be adapted to multi-

15
Zhana Kuncheva and Giovanni
layer networks. Following the same strategy we applied for bipartite
Montana. Community detection
networks, we can adapt the kitchen sink of community discovery to in multiplex networks using locally
multilayer structures. adaptive random walks. In ASONAM,
pages 1308–1315. ACM, 2015
The random walks approach can be multilayer15 , 16 . In practice, we 16
Manlio De Domenico, Andrea Lan-
have a random walker that normally selects edges in the same layer, cichinetti, Alex Arenas, and Martin
but it has a special rule that sometimes allows it to go through a Rosvall. Identifying modular flows on
multilayer networks reveals highly over-
coupling edge, and then resume its normal random walk. Figure 36.7 lapping organization in interconnected
shows an example of such a move. Many ways have been proposed systems. Physical Review X, 5(1):011027,
2015a
to expand random walks to multilayer networks, and a special set of
algorithms focuses on local-first community discovery. We center our 17
Lucas GS Jeub, Michael W Mahoney,
Peter J Mucha, and Mason A Porter.
focus on a specific (set of) query nodes and we find the communities
A local perspective on community
surrounding them17 , 18 . structure in multilayer networks.
Network Science, 5(2):144–163, 2017

Figure 36.7: In multilayer ran-

dom walk community discov-
ery, the random walker (orange)
has a certain probability to
perform a layer jump (brown).

Roberto Interdonato, Andrea Tagarelli,

18

Dino Ienco, Arnaud Sallaberry, and

Pascal Poncelet. Local community
detection in multilayer networks.
DMKD, 31(5):1444–1479, 2017
Similarly, one could propagate labels across layers with special 19
Oualid Boutemine and Mohamed
rules19 , and thus adapt the fast label propagation algorithm to find Bouguessa. Mining community struc-
tures in multidimensional networks.
multilayer communities. First, we cannot use synchronous label TKDD, 11(4):51, 2017
propagation: like in the bipartite case, also for multilayer networks
502 the atlas for the aspiring network scientist

we could be stuck with label oscillation (Section 35.3), this time across
layers. Second, the authors define a quality function that regulates
the propagation of labels. This is done because there might be layers
that are relevant for a community and layers that are not. We do not
want a community, which is very strong in some layers, to “evaporate
away” just because in most layers the nodes are not related. 20
Nazanin Afsarmanesh and Matteo
Next on the menu is k-clique percolation20 . In this scenario, we Magnani. Finding overlapping com-
need to redefine a couple of concepts, particularly what a clique is munities in multiplex networks. arXiv
preprint arXiv:1602.03746, 2016
in a multilayer network, and how we determine when two multiplex
cliques are adjacent. For the first case, we need to talk about k-l-
cliques: a set of k nodes all connected through a specific set of l
layers. Moreover, there are two ways for nodes to be all connected
via the layers: all pairs of nodes could be connected in all layers at
the same time, or they could be connected in only one layer at a time.
The first type of clique is an k-l-AND-clique, the second type is a
k-l-OR-clique. Figure 36.8 shows an example.

Figure 36.8: (a) A 3-3-AND-

clique. (b) A 3-3-OR-clique.

(a) (b)
It becomes clear that two k-l-cliques might share (k − 1) nodes
across different layers. In such a case, we need some care in defining
a parameter to regulate percolation. We need a minimum number 21
Caterina De Bacco, Eleanor A Power,
m of shared layers to allow the percolation. If the two cliques do not Daniel B Larremore, and Cristopher
Moore. Community detection, link
share at least m layers, even if they share k − 1 nodes they are not prediction, and layer interdependence
considered adjacent. Figure 36.9 shows an example. in multilayer networks. Physical Review
E, 95(4):042317, 2017

1 1
Figure 36.9: Two 2,2-cliques
2 4 2 4 sharing a 1,2-clique. The edge
color represents the edge layer.
3 3 If m = 1 (a) does NOT percolate
(a) (b) because the rightmost clique
does not share a layer with
The final adaptation we consider is the stochastic blockmodels21 , 22 .
the leftmost clique; (b) DOES
Just like we saw for overlapping and bipartite SBMs, we need to
percolate, since the two cliques
add an additional matrix into our expectation maximization frame-
share the blue layer.
work. For overlapping and hierarchical SBMs it was a community- 22
Natalie Stanley, Saray Shai, Dane
community matrix telling us how strongly communities connect to Taylor, and Peter J Mucha. Cluster-
ing network layers with the strata
each other. In this case, instead, we have a layer-layer matrix telling
multilayer stochastic block model.
how likely it is for two layers to have the same edges. IEEE transactions on network science and
This is neat, because it allows us to model assortative, disassorta- engineering, 3(2):95–105, 2016
 multilayer community discovery 503

tive, and non-assortative layer relationships. In the first case, being

connected in a layer increases the chances of being connected in
the other layer: it is more likely to be friends on Facebook if we are
also friends in real life. The second case is the opposite: a relation
in one layer makes it harder to be connected on another: if you at-
tack me in an online game it’s less likely that we’ll be friend. The
non-assortative case covers the scenario in which being connected in
a layer gives us no information on whether we’re connected in the
other.
All these cases – the special layer-layer jump probability in random
walk, the quality function in label propagation, and the layer-layer
probability matrix in SBM – allow us to select the relevant layers for
a multilayer community. Thus, we can keep a node group together
even if in many layers the nodes don’t connect to each other, proba-
bly because those layers were disassortative with the layers in which
the community appears.
Still an adaptation of already existing methods, but more properly 23
Raul J Mondragon, Jacopo Iacovacci,
multilayer, is the approach of multilink similarity23 . This is heavily and Ginestra Bianconi. Multilink
inspired by the hierarchical link clustering we saw in the overlapping communities of multiplex networks.
PloS one, 13(3):e0193821, 2018
case (Section 34.5). The objective is the same: to define a link-link
similarity measure that we can use to the progressively merge link
communities in a hierarchical fashion.
The similarity measure for multilayer networks is:

S(u,k),(v,k) = ez β uk,vk + (1 − e)| Puvk̄,2 |.

Here, e and z are parameters between 0 and 1 you can set. The real
work is made by β uk,vk and P. β uk,vk takes values between 0 and 1,
and it is one minus the share of layers in common connecting uk and
vk. So, for instance, if the node pairs uk and vk connect in mutually
exclusive layers – i.e. no layers in common – then β uk,vk = 1. On the
other hand, if they connect in exactly the same layers and no other
layer, then β uk,vk = 0. Since our parameter z is between 0 and 1, the
whole term tells us how much we weight in the link-link similarity
the absence of layers, because mutually exclusive layer set will just be
z1 = z.
| Puvk̄,2 | is instead the count of paths of length 2 between u and v
that do not pass through k, plus the number of layers in which u and
v connect directly to each other. This is normalized by the lowest
degree between u and v, excluding all connections going to k. Thus,
in practice, the parameter e regulates the weight we want to give to
the number of the shared layers of edges uk and vk, over the local
multilayer clustering of nodes u and v. If e = 1 we only care about
clustering together node pairs that connect through the same sets of
layers.
504 the atlas for the aspiring network scientist

1
Figure 36.10: A multilayer
network, with the edge color
encoding the layer in which it
7 6 2 5 4 appears.

Let us consider Figure 36.10 and attempt to estimate the similarity

of node pairs (1, 2) and (1, 3). The pairs share two out of four pos-
sible layers, thus β uk,vk = 0.5. There are also four other paths going
from 1 to 3 that do not use node 2. Both node 1 and 3 have degree
equal to four – remember we don’t count the connections going
through node 2 –, thus | Puvk̄,2 | = 4/4 = 1. If we were to set z = 0.6
and e = 0.4, then S(u,k),(v,k) = (0.4 × 0.60.5 ) + ((1 − 0.4) × 1) ∼ 0.91.
Other adaptations I’m not going to discuss in details are an exten- 24
Manel Hmimida and Rushed
sion of local community discovery to multilayer network24 . This is Kanawati. Community detection in
based on redefining the simple concepts of degree and neighborhood multiplex networks: A seed-centric
approach. NHM, 10(1):71–85, 2015
for the multilayer case, and then apply a classical local exploration
approach, as the one we saw in Section 31.5.
Another class of solutions include representing multilayer graphs
in a lower dimensional space with a technique known as “Grassmann 25
Xiaowen Dong, Pascal Frossard,
manifold”25 . Pierre Vandergheynst, and Nikolai
Nefedov. Clustering on multi-layer
graphs via subspace analysis on
grassmann manifolds. IEEE Transactions
36.4 Multilayer Density on signal processing, 62(4):905–918, 2013

We have talked about how to find communities in multilayer net-

works. Implicitly, we’re resting on our definition, which is based on
density. But what actually is multilayer density? This turns out to be
an ambiguous concept.
Let’s go back to Figure 36.8 for a moment. Is a group of nodes
“multilayer densely” connected when they are connected in all layers
(Figure 36.8(a))? Or is it that you need to look at all connections
across layers (Figure 36.8(b))? To use a different perspective, let’s
represent multilayer networks with a labeled multigraph. In our
first example, we have a multigraph with connections in all labels.
In the second example we sill have a triangle, so the multigraph is
dense. But the two concepts are not the same. Which of the two are
we looking at?
This is another case when one has to make their own judgment.
The answer depends on the type of analysis, and the type of data,
you are looking at. In some cases, you want connections in all layers.
 multilayer community discovery 505

In some others, you are ok with looking at all layers to find commu-
nities. You cannot rely on a fixed definition of communities based on
density, because it cannot apply to all scenarios.
Thus, you need to have measures to determine when you are in
one case and when you are in another. I proposed a couple in a paper 26
Michele Berlingerio, Michele Coscia,
of mine26 . We decided to call them “redundancy” and “complemen- and Fosca Giannotti. Finding redundant
tarity”. and complementary communities
in multidimensional networks. In
Redundancy is the easiest of the two. To consider a set of nodes
Proceedings of the 20th ACM international
to be densely connected in a multilayer network, we require that conference on Information and knowledge
the edges appear in all the layers we are interested in. If we have a management, pages 2181–2184, 2011b

community c containing a set of nodes, and we test it over the set of

layers L, redundancy is simply the share of actual edges over all the
edges we would need to connect every pair of nodes through every
layer:

|{l : ∃(u, v, l ) ∈ E}|

ρc = ∑ | L| × Pc
,
u,v∈ Pc
where Pc is the set of all node pairs in c.
Complementarity is a little trickier, because it is the intersection
of three concepts: variety, exclusivity, and homogeneity. Variety is
through how many different layers nodes in community c connect to
each other. This is simply the number of layers in c divided by the
| Lc | − 1
number of layers in the network: . Exclusivity counts how
| L| − 1
many pairs of nodes connect in just one layer within c: if Pc,l is the
number of node pairs in c which connect only in layer l, the exclu-
∑ Pc,l
sivity is l ∈ L . Finally, homogeneity estimates how uniformly the
| Pc |
σc
edges in c distribute across layers, which is 1 − max . Here, σc is the
σc
standard deviation of the distribution of the edges in c across layers,
and it is normalized by its theoretical maximum.

1 2 2 Figure 36.11: Two examples of

4 different types of multilayer
density. The edge color encodes
3 3 the layer in which the edge
appears. (a) A high redundancy
5
5 case. (b) A high complementar-
4 1 ity case.

(a)
(b)
Let’s see some examples to put all these Greek letters to good
use. Let’s consider Figure 36.11, assuming that the network has a
total of three layers. In Figure 36.11(a) we have a high redundancy
506 the atlas for the aspiring network scientist

case. Since the community includes all layers of the network, the
numerator of redundancy is simply the count of edges: 18. The
denominator is 3 (the number of layers) times the number of node
pairs in the community, which is 10, since we have 5 nodes. Thus the
redundancy is 18/30 = 0.6.
Figure 36.11(b) is instead a high complementary case. Variety is 1
by definition, since the community contains all layers of the network.
Exclusivity is 9/10, because there is one pair of nodes (nodes 2 and 3)
which is connected in two layers, and thus it is not counted. Finally,
the standard deviation of the distribution of the edges in c across
layers is the standard deviation of the vector [5, 1, 5], since there are
five edges in the red and green layer, and only one in the blue layer.
This is ∼ 1.88, which is exactly two thirds of the maximum possible,
leaving us with a total homogeneity of 0.33. Thus, complementarity
is 1 × 0.9 × 0.33 = 0.297, penalized by the low representation of the
blue layer in the community.

36.5 Mopping Up Community Discovery

We have finally reached the end of this extremely simplified trip

through community discovery. It all started with a simple, nothing-
up-my-sleeve definition of what communities are in complex net-
works:

Communities are groups of nodes densely connected to each other and

sparsely connected to nodes outside the community.

Yet, as we progressed in this journey, we realized that there is a

gazillion ways in which such definition needs to be stretched, it is
not the full story, or simply does not work. To sum up our list of
grievances:

• The standard definition implies assortative communities: nodes

in the same communities tend to connect to each other more than
random. However, disassortative communities are a thing as well:
nodes in a disassortative community tend to connect to nodes in
different communities (Section 31.1).

• If our network is evolving, communities are evolving too. The

information from past communities should be taken somehow
into account, making the communities at time t + 1 a compromise
between their density and their similarity with the communities at
time t (Section 31.4).

• Communities can be local (Section 31.5), meaning that we might

prevent ourselves from discovering all members of a community
 multilayer community discovery 507

and thus leaving out some parts of the network that would make
the community denser.

• Measures defined with the idea of maximizing density and exter-

nal sparsity have counter-intuitive behavior, for instance modular-
ity has resolution limit, degeneracy of good solutions, and limits in
its field of vision (Section 32.1).

• We want communities to be interpretable and/or to tell us some-

thing about the real world properties of the entities we are group-
ing. This can be achieved by finding the communities maximizing
the normalized mutual information of some other data we have
about nodes. While this is a worthwhile task for many real world
applications, it can – and most often does – clash with the in-
ternal density requirement (Section 32.4). This is because node
“meta”data is just data, and it doesn’t necessarily have any rele-
vance to the edge creation process of your network.

• Many networks have a hierarchical community structure, where

we can find communities of communities. However, by definition,
these communities must be more sparsely connected that the
communities forming them. This should not make them any less
valid, as they are a useful tool to explain many natural structures
(Chapter 33).

• It is equally a fact that many real world networks have overlap-

ping communities: nodes can be part of multiple communities
at the same time. But if it is true that the more communities two
nodes have in common the more likely they are to connect, we end
up with networks in which the overlap of communities is denser
than the communities themselves, which contradicts our original
definition (Chapter 34).

• We can find communities in bipartite networks as well. However,

by definition, these communities will be somewhat sparse, given
that we forbid connections between nodes of the same type. We
need to modify our definition of density accordingly (Chapter 35).

• However, there are proper ways to find bipartite communities –

and even regular unipartite communities – by adapting classical
data clustering algorithms from data mining. These algorithms
will simply take as input the adjacency matrix of the graph as if it
was an attribute table. The meaning of the communities found this
way would change, though: these are not any more nodes densely
connected to each other, but rather nodes connected to the same
neighbors (Section 35.4).
508 the atlas for the aspiring network scientist

• Finally, as we saw in this chapter, we need to adapt density to the

multilayer case as well. However, this is necessarily an ambiguous
operation with multiple valid alternatives, as multilayer density
can be intended both in a “redundancy” and in a “complemen-
tary” sense.

The moral of this story is that you can intend “communities”

in complex networks in a thousand different ways. Performing
community discovery is not a neutral operation you would do like
adding two numbers. It is a complex problem that starts from the
question: what is a community for me? Or for my data? What am I
actually looking for? Just picking an algorithm because someone tells
you so, or because it is the most used, is guaranteed to misfire.
This is also why, if you encounter somebody who uncritically tells
you the classical definition or uses it in their paper without at least a
mention of these caveats, you have my authorization to tell them to
just shut the hell up.

36.6 Summary

1. Multilayer community discovery is the adaptation of community

discovery for networks with multiple edge types. The simplest
approach is to flatten the multilayer structure by collapsing all
layers into a weighted simple graph.

2. Alternatively, you can find communities using a non-multilayer

algorithm on each of your layers separately. Then, you would
merge the results.

3. One can adapt modularity to multilayer networks by adding a

term that takes into account the inter-layer connection strength,
binding the nodes in the communities across layers. There are
similar adaptations for random walks and label percolation ap-
proaches.

4. The concept of “multilayer density” is intrinsically ambiguous.

One could intend it as the requirement of all nodes connected
through all layers at the same time, or in a single different layer for
each node pair.

36.7 Exercises

1. Take the multilayer network at [Link]

exercises/36/1/[Link]. The third column tells you in which
layer the edge appears. Flatten it twice: first with unweighted
edges and then with the count of the number of layers in which
 multilayer community discovery 509

the edge appears. Which flattening scheme returns a higher NMI

with the partition in [Link]
1/[Link]? Use the asynchronous label percolation algorithm
(remember to pass the edge weight argument in the second case).

2. Using the same network, perform label percolation on each layer

separately. Build the |V | × |C | table, perform kMeans (setting k = 4)
on it to merge the communities. Does this return a higher NMI
when compared with the ground truth?

3. Calculate the redundancy of each community you found for the

previous exercises.
 Part X

Graph Mining
37
Graph Embeddings

Graph mining is a category of network analysis including the appli-

cation of data mining techniques to the analysis of complex networks.
In turn, data mining is a collection of data analysis techniques that
aim at bottom-up discovery of patterns in data. Bottom-up means
that, rather than testing a theory you came up with on data, you
scout for patterns and correlations in the data that you didn’t neces-
sarily know about.
Of course, we already saw tons of data mining on networks scat-
tered in practically every section of this book. However, they were
not collected under the graph mining umbrella because they are usu-
ally data mining applications in service of an analysis task that is not,
per se, data mining. In this part of the book I instead collect a group
of analyses that are born directly into the data mining community
and live inside it. These are not data mining slash something, these
are the pure breeds.
I start with this chapter on graph embeddings. Early on, this field
was dubbed as “collective classification”: the attempt of classify- 1
Prithviraj Sen, Galileo Namata,
ing nodes by looking at how they relate to the rest of the network1 . Mustafa Bilgic, Lise Getoor, Brian
As usual, most of this chapter is based on a recent survey of meth- Gallagher, and Tina Eliassi-Rad. Collec-
tive classification in network data. AI
ods2 , which I follow closely. I will start by defining what graph
magazine, 29(3):93–93, 2008
embeddings are, a classification of embeddings techniques, and some 2
Palash Goyal and Emilio Ferrara.
application scenarios. Graph embedding techniques, appli-
cations, and performance: A survey.
If you like what you’re reading in this chapter, there is a library
Knowledge-Based Systems, 151:78–94,
implementing most graph embeddings techniques that are discussed 2018
3
here3 . [Link]

37.1 Embedding Definition

A graph embedding is a function that maps each node of your net-

work to a vector of numbers. This vector of numbers should have
a few properties. First, it should be a faithful representation of the
topology of your network. Nodes that are “similar” should be repre-
512 the atlas for the aspiring network scientist

sented by similar values. Second, it should be small, meaning that it

should have a low dimensionality, few entries. If your network has
|V | nodes, then the vector representing each node should have much
fewer than |V | entries.
These two properties should make clear why we call this section
“graph embedding”. The aim of the technique is to embed your
nodes into a low dimensional space. You can think of your nodes
as the points of a scatter plot: points that are spatially close to each
other are similar. If this low dimensional representation is any good,
you can then analyze this scatter plot and discover interesting prop-
erties of your nodes. This is helpful, because usually the scatter plot
is (i) easier to analyze than a graph, and (ii) a more common data
structure than a graph on which you can apply a more diverse set of
algorithms that were not developed with graphs in mind.

Node Embedding
3
2 1 {0.88, 0.69} Figure 37.1: (a) An example
2 {0.88, 0.69} 4
1 graph. (b) One of the possible
4 3 {0.88, 0.69}
4 {1, 1} 5
embeddings of (a), assigning a
5
5 {0.48, 0.66} 1,2,3 two dimensional vector to each
6 6 {0, 0}
7 node. (c) The scatter plot repre-
7 {0.42, 0.1} 6
9 {0.42, 0.1} 7,8,9
8
8 sentation of (a)’s embeddings.
9 {0.42, 0.1}
(a) (c)
(b)

Figure 37.1 shows a stylized example of what a graph embedding

is. We transform the original graph (Figure 37.1(a)) into a set of
two dimensional numerical vectors for each of its nodes (Figure
37.1(b)). These vectors have a spatial relationship reflecting some of
the graph’s properties (Figure 37.1(c)).
You could ask yourself: why are we bothering with graph embed-
dings? We can already represent easily a node with a vector. In fact,
this is something you taught me since Chapter 5! A node is nothing
more than a row in the adjacency matrix of the graph. Thus is it a
vector. Why can’t we use that as our “embedding”?
The reason is two-fold. First, I said and still maintain that a good
graph embedding should have a low dimensionality. If you slice
the adjacency matrix, your nodes will be represented by a vector of
length |V |, which is not great. We haven’t saved any dimension. Sec-
ond, the problem with using the adjacency matrix is that it is binary
and sparse. Most algorithms that you want to apply on your embed-
dings don’t work well with this sort of input data. So you want your
embeddings to be densely packed with non-binary information.
Moreover, the adjacency matrix is always the same, i.e. it will
always give you the same embedding. However, you might want to
 graph embeddings 513

Node Embedding 4
1 {0.01, 0.01} Figure 37.2: (a) A different
6
2 {0.01, 0.01}
3 {0.01, 0.01}
valid embedding of the graph
5
4 {0.01, 1} in Figure 37.1(a), assigning a
5 {0.33, 0.5} two dimensional vector to each
6 {0.05, 0.9}
7 {0.05, 0.05} 7,8,9
node. (b) The scatter plot repre-
8 {0.05, 0.05} sentation of (a)’s embeddings.
1,2,3
9 {0.05, 0.05}

(a) (b)

build your embeddings differently depending on which property

you’re interested in studying. The example I show in Figure 37.1
is one you’d use if you wanted your embeddings to help you with
community discovery or some spreading event on the network.
However, it would be poor when it comes to estimate, for instance,
structural equivalence (Section 12.2). Figure 37.2 shows a different
valid embedding that would help you with such a task. By having
different techniques optimizing different functions to create your
embeddings, you can specialize your low-dimensional representation
to fit different problems you want to solve, rather than relying on the
immutable adjacency matrix.
Note that you have an additional degree of freedom: not only
you can decide which function you use to create the embedding,
you can also decide the shape of the space in which you’re creating
the embedding. For simplicity, in Figures 37.2(a) and 37.2(b) I use
an Euclidean space: each dimension has equal importance and the
distance between two points is determined by the length of the
straight line between the points. This is not the only choice. For 4
Dmitri Krioukov, Fragkiskos Pa-
instance, some researchers use a hyperbolic space4 , 5 , 6 , where the padopoulos, Maksim Kitsak, Amin
distance between two points is not determined by a straight line, but Vahdat, and Marián Boguná. Hyperbolic
geometry of complex networks. Physical
by the branch of a hyperbole.
Review E, 82(3):036106, 2010
5
Ginestra Bianconi and Christoph
Rahmede. Emergent hyperbolic network
37.2 Building Embeddings geometry. Scientific reports, 7:41974, 2017
6
Maksim Kitsak, Ivan Voitalov, and
The survey paper on which I base this chapter divides embeddings Dmitri Krioukov. Link prediction with
hyperbolic geometry. Physical Review
methods in three main categories: spectral, random walk, and deep Research, 2(4):043113, 2020
learning. Their general philosophies are as follows:

1. Spectral: this is the oldest category and uses simple matrix forms
to represent the relationships between nodes. The idea is to take
a matrix, which can be the adjacency matrix, and factorize it
minimizing some objective function.

2. Random Walk: these methods generate the embedding of a node

by performing several random walks starting from the node and
514 the atlas for the aspiring network scientist

noting down which nodes frequently appear in such random

walks. The number of times v appears in random walks originat-
ing from u becomes a feature for u’s vector, with which you’ll
calculate the similarities with other nodes.

3. Deep Learning: in this category you would apply some deep

neural network algorithm to explore the graph structure and to
create the embedding of each node.

Let’s now explore more in depth some of these methods.

Spectral

This is the oldest category of graph embeddings techniques. The

objective of the researchers working in this early definition of the
problem was simple dimensionality reduction. In other words, they
were mainly interested in having small vectors representing the
topology around a node without having to look at the entirety of
its neighborhood. In practice, they were trying to have some sort of
network-aware Principal Component Analysis (Section 5.4).
The idea is that, if nodes u and v are connected to each other
by a strong link Auv , then their embeddings yu and yv should be
similar. This means that we can calculate how bad an embedding is:
 2
∑ yu − ∑ yv Auv . Note that, here, yu and yv are vectors so once
u v
you’re done subtracting and summing all these vectors you’re left
with a final vector that you need to somehow reduce to a scalar that
is your final quality score.
If u and v are connected by a high Auv link strength, we better
have a low yu − yv difference to cancel it out! It turns out that finding
the set Y of vectors for all our nodes can be solved as an eigenvector 7
Sam T Roweis and Lawrence K Saul.
problem. Specifically, you can take the smallest eigenvectors of the Nonlinear dimensionality reduction by
matrix ( I − A) T ( I − A) – but discarding the actual smallest one –, with locally linear embedding. Science, 290
(5500):2323–2326, 2000
I being the identity matrix. This is the Locally Linear Embedding7 . 8
Mikhail Belkin and Partha Niyogi.
Laplacian Eigenmaps8 change the penalty function, which means Laplacian eigenmaps and spectral tech-
the objective is still the eigenvector decomposition of a matrix, but niques for embedding and clustering. In
Advances in neural information processing
the matrix itself is built differently. In this case, the penalty function systems, pages 585–591, 2002
1
is ∑ (yu − yv )2 Auv , which gives more weight to the yu − yv differ-
2 u
ence than before – because it squares it before multiplying it with the
Auv link strength. In this case, you can solve the problem by taking
the smallest eigenvector of the normalized Laplacian D1/2 LD1/2 ,
with L being the Laplacian of graph G and D its degree diagonal
matrix.
 graph embeddings 515

Random Walks

As anticipated, this category is all about performing random walks.

For each node u in the network, you perform a set of random walks
originating from that node. The objective of the random walk is to
note down all the nodes that it explores, representing the origin node
u as a “bag of nodes”.

Complex network analysis is easy

Figure 37.3: Representing two
Network science is hard sentences as a graph, show-
ing the connection between
complex network analysis is easy Word2Vec and network science.

science hard

The reason to do so is that this representation is equivalent to a

“bag of words” representation of documents that has proven to be
extremely useful in natural language processing – it’s the famous 9
Tomas Mikolov, Kai Chen, Greg Cor-
rado, and Jeffrey Dean. Efficient estima-
Word2Vec approach 9 , 10 . In Word2Vec, each word is represented as a tion of word representations in vector
numerical vector based on the words appearing around it in a corpus space. arXiv preprint arXiv:1301.3781,
of text. Since “bag of nodes” and “bag of words” are a practically 2013a
10
Tomas Mikolov, Ilya Sutskever, Kai
equivalent concept, one can use all of the NLP algorithms designed Chen, Greg S Corrado, and Jeff Dean.
to work on Word2Vec representations to work in the network sce- Distributed representations of words
and phrases and their compositionality.
nario as well. In fact, one could see a sentence as a chain graph, and
Advances in neural information processing
a collection of sentences as a collection of chains, which compose a systems, 26:3111–3119, 2013b
complex network, as Figure 37.3 shows.
Similarly to the community discovery case, the idea is that random
walks of two similar nodes will hit the same neighbors. In Deep- 11
Bryan Perozzi, Rami Al-Rfou, and
Steven Skiena. Deepwalk: Online
Walk11 , one simply performs a bunch of random walks. All random learning of social representations. In
walks have the same length: k. Then, you take the middle point of the Proceedings of the 20th ACM SIGKDD
random walk, the node you hit when you were right in the middle. international conference on Knowledge
discovery and data mining, pages 701–710.
Each time you hit in the middle point of the walk the same node v, ACM, 2014
you update its embedding by increasing the probability of having hit
a series of nodes before v (the ones appearing in the random walk
before v) and after.
Figure 37.4 shows a simple representation to help with intuition.
The first random walk, in blue, shows that node 5 is in the middle
of a random walk going 3 → 4 → 5 → 6 → 7. However, when we
perform the green random walk, we need to update the probabilities
of reaching a given node in a random walk including node 5: while
nodes 4 and 6 are still part of the random walk, we have different
starting and ending points. The objective of DeepWalk is to create a
516 the atlas for the aspiring network scientist

3
2 Φ = max(p(3,4|5) & p(6,7|5)) Figure 37.4: A styilized exam-
ple of DeepWalk. Blue and
1 Φ = max(p(3 or 1,4|5) & p(6, 7 or 8|5))
4 green arrows show two random
5
walks of length five. After each
random walk, the function Φ
6 representing node 5 is updated.
7

9
8

function which can summarize these probabilities in vectors smaller

than |V |. Note that, in DeepWalk, you need to skip some nodes in 12
Bryan Perozzi, Vivek Kulkarni, and
your random walk12 . By skipping more or fewer nodes, regulated by Steven Skiena. Walklets: Multiscale
a parameter, one can learn long- or short-range relationships between graph embeddings for interpretable
network classification. arXiv preprint
nodes.
arXiv:1605.02115, 2016
Node2vec13 follows fundamentally the same philosophy. The only 13
Aditya Grover and Jure Leskovec.
difference is that DeepWalk uses uniform random walks, picking node2vec: Scalable feature learning for
networks. In Proceedings of the 22nd
the next step of the walk completely at random. Node2vec, instead, ACM SIGKDD international conference
performs higher order random walks (Chapter 30). Specifically, the on Knowledge discovery and data mining,
next step of a random walk depends on two parameters regulating pages 855–864. ACM, 2016

how much the random walk needs to look like a DFS or a BFS. If you
came to node v from node u, common neighbors between u and v
will have a different exploration probability than neighbors of v that
are not connected to u. This allows you to set parameters to explore
structural equivalence rather than modular structure, as I already 14
Haochen Chen, Bryan Perozzi,
Yifan Hu, and Steven Skiena. Harp:
showed you in Figures 37.1 and 37.2.
Hierarchical representation learning
HARP14 improves over both methods by employing a smarter for networks. In Thirty-Second AAAI
way to initialize the weights of the function summarizing your nodes Conference on Artificial Intelligence, 2018

– before you start performing your random walks. Alternative ap-

15
Shaosheng Cao, Wei Lu, and
Qiongkai Xu. Grarep: Learning graph
proaches include in the random walk the global structure of the representations with global structural
graph by raising the stochastic matrix to several powers and using information. In Proceedings of the 24th
ACM international on conference on infor-
these k step transition matrix to build the embedding15 . You can also mation and knowledge management, pages
integrate node attributes in your representation16 , 17 . 891–900, 2015
One of the general problem of such approaches is that nodes that
16
Zhilin Yang, William W Cohen,
and Ruslan Salakhutdinov. Revisit-
are structurally equivalent might fail to be encoded with a similar ing semi-supervised learning with
vector if the observation window is too narrow – for instance if in graph embeddings. arXiv preprint
arXiv:1603.08861, 2016b
HARP we choose a k that is too low. This is exactly the problem that 17
Shirui Pan, Jia Wu, Xingquan Zhu,
struct2vec tries to solve18 . Chengqi Zhang, and Yang Wang. Tri-
party deep network representation.
Network, 11(9):12, 2016
Deep Learning 18
Leonardo FR Ribeiro, Pedro HP
Saverese, and Daniel R Figueiredo.
struc2vec: Learning node represen-
For this subsection to make most sense, you’d be required to know tations from structural identity. In
a bit more about what deep learning is. This is outside the scope of SIGKDD, pages 385–394, 2017
 graph embeddings 517

this book, as it deserves a book on its own. For this reason, I suggest 19
Li Deng, Dong Yu, et al. Deep
you some readings19 , 20 , 21 , which might help you figuring out better learning: methods and applications.
what is going on. Foundations and Trends® in Signal
Processing, 7(3–4):197–387, 2014
The fundamental difference between the methods in this class 20
Yann LeCun, Yoshua Bengio, and
and the ones in the previous class, is that random walk models can Geoffrey Hinton. Deep learning. nature,
be considered as a sort of “shallow” learning: they encode the walk 521(7553):436–444, 2015
information with a simple function. Here, instead, we use deep
21
Ian Goodfellow, Yoshua Bengio, and
Aaron Courville. Deep learning. MIT
learning techniques. In general, this allows you to use more complex press, 2016
functions to encode information at the same time. For instance, in 22
Daixin Wang, Peng Cui, and Wenwu
SDNE22 you can learn the first and second order relationships be- Zhu. Structural deep network em-
tween nodes at the same time, using an autoencoder. An autoencoder, bedding. In Proceedings of the 22nd
ACM SIGKDD international conference
as Figure 37.5 shows, is a type of deep neural network that has the
on Knowledge discovery and data mining,
same number of nodes in its output layer as in its input layer. pages 1225–1234, 2016a

Figure 37.5: An autoencoder.

Node color determines its type:
red = input, blue = hidden,
green = output.

In practice, what an autoencoder does is trying to re-encode your

input smoothing the noise away and reconstructing the underlying
signal. It achieves this by introducing an information bottleneck
to force the network to learn a low-dimensional encoding of the
input. In Figure 37.5, the autoencoder has the same dimensionality
for the input and output layers – i.e. the same number of nodes –
because it wants to output a vector that is as similar as possible to the 23
Shaosheng Cao, Wei Lu, and
input. Similar approaches23 also rely on autoencoders to de-noise the Qiongkai Xu. Deep neural networks
mutual information between random walks. for learning graph representations. In
Thirtieth AAAI conference on artificial
The problem of the two aforementioned approaches is that they intelligence, 2016
require a large amount of information per node. In the SDNE case,
you need to look at the entire adjacency matrix. To reduce such re- 24
Thomas Kipf and Max Welling. Semi-
quirements, one could adopt a convolutional strategy24 . You can supervised classification with graph
think of convolution as a sliding dot product. For simplicity, picture a convolutional networks. In ICLR, 2017

simple adjacency matrix as a 2D plane. You summarize the matrix by

operating on one small 2D portion of it at a time, and then pooling
the results. In this way, you can reduce the input size. Figure 37.6
shows a depiction of what a simple convolution operation looks like
at an abstract level. Of course, the figure is just an abstract repre-
sentation: we do not actually process the adjacency matrix that way.
518 the atlas for the aspiring network scientist

25
Joan Bruna, Wojciech Zaremba,
Arthur Szlam, and Yann LeCun. Spec-
There are many approaches defining different strategies to implement tral networks and locally connected
convolution on a network, for instance: using the spectrum of the networks on graphs. arXiv preprint
Laplacian25 , 26 , approaches inspired by extended-connectivity circular arXiv:1312.6203, 2013
26
Michaël Defferrard, Xavier Bresson,
fingerprints27 , a maximization of mutual information between local and Pierre Vandergheynst. Convolu-
and global graph properties28 , and GraphSAGE29 . tional neural networks on graphs with
fast localized spectral filtering. In NIPS,
pages 3844–3852, 2016

Figure 37.6: A convolution op-

eration. Color determines the
layer of the neural network: red
= input, blue = hidden, green =
output.
27
David K Duvenaud, Dougal Maclau-
rin, Jorge Iparraguirre, Rafael Bom-
And then – why not? – one could combine the autoencoders and barell, Timothy Hirzel, Alán Aspuru-
the graph convolutional approach in one neat little package30 . Guzik, and Ryan P Adams. Convolu-
tional networks on graphs for learning
molecular fingerprints. In NIPS, pages
2224–2232, 2015
Other Complications 28
Petar Veličković, William Fedus,
William L Hamilton, Pietro Liò, Yoshua
Graph embedding techniques, regardless of their chosen approach,
Bengio, and R Devon Hjelm. Deep graph
need to be adapted to be able to handle heterogeneous31 and mul- infomax. arXiv preprint arXiv:1809.10341,
tilayer networks, networks where nodes and/or edges can be of 2018
29
Will Hamilton, Zhitao Ying, and Jure
multiple different types. For instance, one could adopt the metapath Leskovec. Inductive representation
approach32 we saw in Section 21.2 when talking about link prediction learning on large graphs. In NIPS, pages
1024–1034, 2017
in multilayer networks. The problem with heterogeneous networks 30
Thomas N Kipf and Max Welling.
is that there are some node types that are more dominant – i.e. con- Variational graph auto-encoders. arXiv
nected – than others. Their representations would then be extremely preprint arXiv:1611.07308, 2016
noisy. In metapath2vec, the problem is solved by switching one’s
31
Shiyu Chang, Wei Han, Jiliang Tang,
Guo-Jun Qi, Charu C Aggarwal, and
attention from nodes to metapaths. Thomas S Huang. Heterogeneous net-
work embedding via deep architectures.

H
In SIGKDD, pages 119–128, 2015
Figure 37.7: A troubling set
of connections in an heteroge-
neous network: a paper with
hundreds of co-authors.

32
Yuxiao Dong, Nitesh V Chawla, and
Ananthram Swami. metapath2vec:
Scalable representation learning for
Figure 37.7 shows a stylized depiction of the issue. The paper heterogeneous networks. In SIGKDD,
reporting the discovery of the Higgs boson has 5, 154 co-authors. pages 135–144, 2017
Every pair of co-authors is a valid path in the co-authorship network.
As a result, the embedding of the node representing the paper is
extremely noisy, as it could be visited by 107 walks of length 2, not
even counting the ones that could lead you back to your origin node.
But by instead focusing on each of the metapaths (Section 21.2), we
will have a much cleaner signal.
 graph embeddings 519

Another way to deal with heterogeneous networks is to infer 33

Yu Shi, Qi Zhu, Fang Guo, Chao
different embeddings for different edge types33 . Let’s say you have Zhang, and Jiawei Han. Easing em-
two nodes of different types: a conference venue c and a topic t. They bedding learning by comprehensive
transcription of heterogeneous informa-
are connected to the same node a, an author, who published in that
tion networks. In Proceedings of the 24th
conference and in that topic, but not at the same time. Thus, c and ACM SIGKDD International Conference
t are not connected and not similar to each other. Then a will be on Knowledge Discovery & Data Mining,
pages 2190–2199, 2018
equidistant from them. However, when we focus on the “author-
topic” edge type, a will be closer to t, and when we focus on the
“author-conference” edge type, a will be closer to c.
As you might expect, there is the further complication of networks 34
Jundong Li, Harsh Dani, Xia Hu,
evolving over time, which require their own special embeddings34 . Jiliang Tang, Yi Chang, and Huan Liu.
One further approach one should consider is LINE35 . Just like in Attributed network embedding for
learning in a dynamic environment.
the deep learning category, LINE realizes that one needs to take into
In Proceedings of the 2017 ACM on
account both first and second order relationships between nodes, Conference on Information and Knowledge
a feature that is absent from classical random walk approaches. Management, pages 387–396, 2017a
35
Jian Tang, Meng Qu, Mingzhe Wang,
Differently from the deep learning category, though, LINE looks at
Ming Zhang, Jun Yan, and Qiaozhu
the two transition matrices – the ones representing first and second Mei. Line: Large-scale information
order transitions – as a joint probability problem, without using deep network embedding. In Proceedings of
the 24th international conference on world
learning techniques. LINE simply minimizes the Kullback-Leibler wide web, pages 1067–1077. International
divergence of these two probability distributions. World Wide Web Conferences Steering
Committee, 2015a

37.3 Knowledge Graph Embedding

A specialized portion of the embedding literature deserves a small 36

Maximilian Nickel, Volker Tresp,
aside in a dedicated section: Knowledge Graph Embedding36 . This is and Hans-Peter Kriegel. A three-way
the application of graph embedding techniques on knowledge graphs. model for collective learning on multi-
relational data. In Icml, volume 11, pages
Knowledge graphs are graphs of entities related to each other by a 809–816, 2011
certain semantic. De facto, these are special heterogeneous networks. 37
Jens Lehmann, Robert Isele, Max
Examples of knowledge graphs are, for instance, DBPedia37 , Wiki- Jakob, Anja Jentzsch, Dimitris Kon-
data, and more. What makes them special is their massive size and tokostas, Pablo N Mendes, Sebastian
Hellmann, Mohamed Morsey, Patrick
rich semantics behind the connections. Van Kleef, Sören Auer, et al. Dbpedia–a
In such cases, graph embedding techniques acquire a special mean- large-scale, multilingual knowledge
base extracted from wikipedia. Semantic
ing. We are now establishing relations between concepts, building a Web, 6(2):167–195, 2015
way to determine that, for instance, knives are to cooks what cameras
are to movie directors. This allows us to create automatically new
connections, previously unknown relationships, in the knowledge
graph. Figure 37.8 shows an example. In the figure, I create the new
abstract concept of “knife minus cook”. By adding “movie director”
to such an abstract concept, I discover the camera-director relation-
ship as equivalent to knife-cook. Thus the “knife minus cook” is a
useful concept, which we might use for all professions.
Such operations can be done directly from text using Natural
Language Process techniques. What makes this special, is that the
embeddings were created not by looking at natural text, but at the
520 the atlas for the aspiring network scientist

Camera Camera Camera

+Director

Director Director Knife - Cook Director

Knife - Cook
-Cook
Knife Knife Knife

Cook Cook Cook

(a) (b) (c)

Figure 37.8: (a) Some node
structure of a knowledge graph. To do this, we need specialized embeddings we learned from
tools38 , 39 , 40 . As usual, you can find more details on this problem in a knowledge graph. (b-c) Ma-
dedicated survey papers41 , 42 . nipulating node embeddings by
adding-subtracting their vector
representations can uncover
37.4 Applications new knowledge.
38
Zhen Wang, Jianwen Zhang, Jianlin
Feng, and Zheng Chen. Knowledge
So, what can you do with node embeddings? To cut a story short: graph embedding by translating on
practically everything. We already saw a couple of applications, in hyperplanes. In Twenty-Eighth AAAI
this chapter and in others. Without repeating myself too much, I’ll conference on artificial intelligence, 2014b
39
Yankai Lin, Zhiyuan Liu, Maosong
gloss over data clustering / community discovery and structural Sun, Yang Liu, and Xuan Zhu. Learning
equivalence – which I already presented with Figures 37.1 and 37.2 –, entity and relation embeddings for
link prediction43 (Part VI), and node role detection (Chapter 12). knowledge graph completion. In
Twenty-ninth AAAI conference on artificial
One natural application of embeddings is visualization. We’re intelligence, 2015
going to do a dive deep on network visualization in Part XII, but 40
Michael Schlichtkrull, Thomas N Kipf,
hopefully it is rather easy for you to understand why embeddings Peter Bloem, Rianne Van Den Berg, Ivan
Titov, and Max Welling. Modeling re-
are useful in this case. One could reduce each node into two or three lational data with graph convolutional
dimensions, and then simply use them as the ( x, y, z) position to networks. In European Semantic Web
Conference, pages 593–607. Springer,
display them spatially. After all, this is what is already being done 2018
with traditional dimensionality reduction techniques: it is the whole 41
Maximilian Nickel, Kevin Murphy,
selling point of the quasi-magical t-distributed stochastic neighbor Volker Tresp, and Evgeniy Gabrilovich.
A review of relational machine learning
embedding44 (t-SNE). for knowledge graphs. Proceedings of the
In fact, the standard approach includes two steps45 . First, you IEEE, 104(1):11–33, 2015
use your favorite graph embedding technique to reduce all nodes
42
Quan Wang, Zhendong Mao, Bin
Wang, and Li Guo. Knowledge graph
to vectors of length d. Then, you apply t-SNE itself to reduce those embedding: A survey of approaches
d dimensions to two and you plot the result. Figure 37.9 shows and applications. IEEE Transactions on
Knowledge and Data Engineering, 29(12):
what happens to a relatively simple LFR benchmark with a high 2724–2743, 2017a
mixing parameter – i.e. communities which tend to share lots of 43
Mingdong Ou, Peng Cui, Jian Pei,
inter-community edges. In Figure 37.9(a) you see what a classical Ziwei Zhang, and Wenwu Zhu. Asym-
metric transitivity preserving graph
network layout would do, while Figure 37.9(b) shows you a pure embedding. In SIGKDD, pages 1105–
t-SNE layout applied to the raw adjacency matrix. In the latter case, 1114, 2016
 graph embeddings 521

Figure 37.9: An LFR benchmark.

The node color represents its
community affiliation. (a) Force
directed layout (Section 44.1).
(b) t-SNE layout.

(a) (b)

the community structure is more evident. Imagine what you could 44

Laurens van der Maaten and Geoffrey
do if you first make a reasonable node embedding, rather than being Hinton. Visualizing data using t-sne.
Journal of machine learning research, 9
a simpleton like me and just feed t-SNE the simple adjacency matrix! (Nov):2579–2605, 2008

3
2 1 2
1,2,3
Figure 37.10: (a) An example
1 graph. (b) One of the possible
4 3 4
4
5 5
embeddings of (a). The dashed
5

6
6 circles encapsulate the radius
6 7
7
7,8,9 inside which we consider the
9 9 8 nodes as connected together.
8
(b) (c) (c) The version of graph (a)
(a)
reconstructed through its two
A second natural application of graph embeddings is graph sum- dimensional embedding.
marization. We’re going to examine the problem more in details in 45
Jian Tang, Jingzhou Liu, Ming Zhang,
Chapter 38. For now, suffice to say that, if two nodes have very simi- and Qiaozhu Mei. Visualizing large-
lar embeddings, we could just collapse them into the same node. The scale and high-dimensional data. In
WWW, pages 287–297, 2016
benefit would be to have a smaller structure to analyze – less memory
and time consumption for your algorithm – while still preserving the
general properties of the network as a whole.
One common way is to do the following. First, take your graph
and calculate its node embeddings. These, as we saw, are spatial
representations in d dimensions. Then, calculate the pairwise dis-
tance between all these points. You can use the Euclidean distance
or whatever floats your boat. At this point, you can establish a cer-
tain distance k. Nodes that are closer than k get connected together,
otherwise they remain disconnected. This is a graph reconstructed
from the embeddings. You can compare the reconstructed graph with
the original one. The more similar they are, the better the embedding
worked. Figure 37.10 shows an example of this procedure.
You can consider closer points as the same node and summarize
your graph this way. For instance, nodes 7, 8, 9 have the same embed-
ding and thus they could be considered to be the same node, just like
522 the atlas for the aspiring network scientist

nodes 1, 2, 3.
Other classical applications of graph embeddings are node rank- 46
Namyong Park, Andrey Kan,
ing46 , solving classical combinatorial problems like the traveling Xin Luna Dong, Tong Zhao, and
salesman problem47 , and network alignment48 , the problem of fig- Christos Faloutsos. Estimating node
importance in knowledge graphs using
uring out which nodes from two distinct networks might refer to
graph neural networks. In Proceedings
the same real world entity. This is still limited to the realm of net- of the 25th ACM SIGKDD International
work analysis for network analysis’ sake, but we know we can use Conference on Knowledge Discovery &
Data Mining, pages 596–606, 2019
networks – and, therefore, graph embeddings – to solve many more 47
Elias Khalil, Hanjun Dai, Yuyu Zhang,
problems. Some include natural language processing49 , computer Bistra Dilkina, and Le Song. Learning
vision50 , and bioinformatics51 , to cite a few pointers. combinatorial optimization algorithms
over graphs. In Advances in Neural
Information Processing Systems, pages
6348–6358, 2017
37.5 Summary 48
Mark Heimann, Haoming Shen,
Tara Safavi, and Danai Koutra. Regal:
1. A graph embedding is a low dimensional representation of nodes Representation learning-based graph
alignment. In Proceedings of the 27th
in your network. Most commonly, this means representing a node
ACM International Conference on Informa-
as a vector of length d, with similar nodes being represented by tion and Knowledge Management, pages
spatially close vectors. 117–126, 2018
49
Zhouhan Lin, Minwei Feng, Cicero
Nogueira dos Santos, Mo Yu, Bing
2. Depending on how you build them, embeddings can have multi-
Xiang, Bowen Zhou, and Yoshua
ple meanings and facilitate different analyses. For instance, you Bengio. A structured self-attentive
can use embeddings to determine node communities, or identify sentence embedding. arXiv preprint
arXiv:1703.03130, 2017
structurally equivalent nodes. 50
Sijie Yan, Yuanjun Xiong, and Dahua
Lin. Spatial temporal graph convo-
3. One can build embeddings with different techniques: by factor- lutional networks for skeleton-based
izing the adjacency matrix or its spectrum, by exploring a node’s action recognition. In Thirty-Second
AAAI Conference on Artificial Intelligence,
neighborhood via random walks, or by employing deep learning 2018
strategies. 51
Marinka Zitnik, Monica Agrawal, and
Jure Leskovec. Modeling polypharmacy
4. Embeddings in knowledge graphs are a special case. Knoweldge side effects with graph convolutional
networks. Bioinformatics, 34(13):i457–i466,
graphs are heterogeneous networks expressing relations between 2018
real world concepts. In this scenario, embeddings can help us to
uncover previously unknown meanings – i.e. groups of nodes in
the knowledge graph.

5. Two classical applications of node embeddings are visualization

and summarization. Visualization allows us to place nodes in a 2D
space convenient for drawing the graph. Summarization collapses
structurally equivalent nodes to reduce the size of the graph while
maintaining its most important topological characteristics.

37.6 Exercises

1. Use the [Link] function on the adjacency matrix

to determine the x and y placement of the nodes in the network
at [Link] and
 graph embeddings 523

plot the result. Use [Link]

1/[Link] to determine the node colors.

2. Use the [Link] function to reduce the dimen-

sionality of the adjacency matrix of the previous network to 3
dimensions. Run the kMeans clustering algorithm on the resulting
network. What is the NMI of the kMeans clusters with the ground
truth you can find at [Link]
1/[Link]? (Note: set k = 8 for kMeans)

3. Is the NMI you get from the previous question better or worse
than the one you’d get from a classical community discovery like
label propagation?

4. Use the [Link] function to reduce the dimen-

sionality of the adjacency matrix of the network to 3 dimensions.
Calculate one minus the pairwise Euclidean distance of all node
vectors and use it as your link prediction score. Draw the ROC
curve of your predictions, assuming that the true new edges are
the ones you can find in [Link]
37/3/[Link].

5. Is the AUC you get from the previous question better or worse
than the one you’d get from a classical link prediction like Jaccard,
Resource Allocation, Preferential Attachment, or Adamic-Adar?
38
Graph Summarization

Graph summarization is a data mining class of algorithms that take

an input graph and reduce its size – summarizing it – returning a
smaller graph as an output. The output graph should respect the
salient characteristics of the input graph, so that analyses performed
on the output return results that can be used to reconstruct what the 1
Yike Liu, Tara Safavi, Abhilash Dighe,
whole input graph would return1 . and Danai Koutra. Graph summa-
There are many reasons why one would want to perform graph rization methods and applications: A
survey. ACM Computing Surveys (CSUR),
summarization. The main ones are four: 51(3):1–34, 2018b
2
Tomás Feder and Rajeev Motwani.
• Algorithmic speedup: this is the classic motivation2 . If your orig- Clique partitions, graph compression
and speeding-up algorithms. Journal
inal graph has, like Facebook, ∼ 109 nodes, even the most ele-
of Computer and System Sciences, 51(2):
mentary algorithms will take a long time to run. This might be 261–272, 1995
a problem if, for instance, you want to perform online analysis
and return results in real time. If you manage to reduce the size of
your network to ∼ 106 nodes, this would buy you a lot of time and
reactivity.
• Storage facilitation: hard disks might be cheap nowadays, but
they ain’t free. Again using the Facebook example, it might not
be a problem storing ∼ 109 nodes, but if all those nodes perform
several activities per day, you might start to get into trouble. More-
over, you will have to hit some physical limits: even if you can
create a system with several petabytes of storage capability, if you
want to use those petabytes of data you have to move them around,
and all of a sudden the speed of light seems so slow.
• Noise reduction: noise creeps up inside your data at every twist
and turn. Storing and using your full network as it is measured
might not be a good idea. Graph summarization can help you
to smooth out the noise using information theoretic techniques,
reconstructing the underlying signal.
• Visualization: as we will see in Chapter 44, plotting a network is
hard and takes a lot of time. No one will ever visualize directly
 graph summarization 525

a ∼ 109 node network. If you summarize it so that all its visual

features are respected, you might then be able to have something
meaningful to show.

Before going on the specific methods, I need to clarify what this

chapter is about and what it isn’t about. This chapter focuses on four
main approaches to summarize graphs:

• Aggregation (Section 38.1): collapsing nodes – and the edges

connecting them – into super nodes;

• Compression (Section 38.2): finding a set of rules enabling you to

encode your network by using fewer bits than its adjacency matrix;

• Simplification (Section 38.3): tossing “unimportant” nodes and

edges;

• Influence-based (Section 38.4): finding the smallest possible graph

which is still able to describe propagation events in the same way
as the original one.

They all have one thing in common: they attempt to reduce a

graph by lowering the number of nodes and edges such that the
output is another – smaller – graph which represents the entire
structure, only simplified.
In this sense, graph summarization is not network sampling, and I
will also not use space in this chapter for other similar methods such
as low-rank approximations.
Graph summarization is fundamentally distinct from network
sampling (Chapter 25) even if both branches start from the same
point: networks are too large and we cannot take them all in at once.
However, in network sampling, you explore a part of the network 3
Marc Najork, Sreenivas Gollapudi, and
and you operate on the observed structure directly. Neither is true Rina Panigrahy. Less is more: sampling
the neighborhood graph makes salsa
in graph summarization. In summarization you want to analyze better and faster. In Proceedings of the
and understand the whole structure, and you do so indirectly by Second ACM International Conference
on Web Search and Data Mining, pages
manipulating it. There are methods that attempt summarization-by- 242–251, 2009
sampling3 , 4 , but I’m not going to cover them. 4
Jihoon Ko, Yunbum Kook, and Kijung
Conversely, low-rank approximations seek to reduce the data Shin. Incremental lossless graph
summarization. In Proceedings of the 26th
size with low reconstruction error5 . This is practically a Principal ACM SIGKDD International Conference
Component Analysis technique that is specialized to work on the on Knowledge Discovery & Data Mining,
pages 317–327, 2020
adjacency matrices of a large sparse graph, rather than on generic
5
Jimeng Sun, Yinglian Xie, Hui Zhang,
attribute tables. Thus one could consider this more akin to matrix and Christos Faloutsos. Less is more:
factorization techniques, for which I invite you to refer to Section 5.4. Compact matrix decomposition for
Finally, a word about how to evaluate your graph summary. Many large sparse graphs. In Proceedings of
the 2007 SIAM International Conference
methods come with their own quality measure that they are trying on Data Mining, pages 366–377. SIAM,
to optimize. Thus you could use one of these measures to decide 2007b
526 the atlas for the aspiring network scientist

whether you have obtained a good summary or not. For instance, in

the compression class we’re trying to minimize the number of bits
needed to describe the graph.
However, Part IX of this book – on community discovery – should
have drilled something in your head: the real quality criterion you
want to run is dependent on what you want to do with your graph
summary. Thus the ideal test should be something as closely related
to your final analysis task as possible. For instance, you might want
to preserve some properties of interest – e.g. the clustering coefficient.
The more you depart from the original value, the more poorly your
method is performing.

38.1 Aggregation

In the aggregation approach, the idea is to take the original structure

and start aggregating nodes – or edges – into superstructures that
stand in for the observed ones. This process is normally guided by
some function that determines the quality loss of each aggregation
operation. The function must be designed with some application in
mind (e.g. community discovery, link prediction, or others).

Figure 38.1: (a) An input graph

with highlighted cliques. (b)
Aggregation step: we consider
each clique as a “supernode”
(in blue). (c) The summarized
graph.
(a) (b) (c)
We already saw a flavor of this approach when we discussed
hierarchical community discovery in Section 33.1. Figure 38.1 is
a reprise of Figure 33.2 and is a great example of the aggregation
approach. In it, we use a community discovery technique to highlight
densely connected modules, and we then collapse each into a “super
node”.
Another example from the past comes from Section 29.2, where we
discussed graph condensation, another summarization technique. In
the case of condensation, it involves neither communities nor cliques
– clique-reduction6 is another aggregation method –, but strongly
6
Hiroshi Ishikawa. Higher-order clique
reduction in binary graph cut. In 2009
connected components in directed networks. IEEE Conference on Computer Vision and
There’s also an approach from a future section of this book. When Pattern Recognition, pages 2993–3000.
IEEE, 2009
visualizing networks, one thing you need to decide is the positioning
of the nodes onto a 2D plane. This is the problem of finding a good
layout for your graph, and I’ll talk in depth about this in Chapter
44. In that chapter, you’ll see that one of the biggest problems is
 graph summarization 527

that nodes sometimes snuggle together a bit too closely, overlapping

with each other. One could identify such nodes that tend to occupy
the same position in space and simply aggregate them into a super 7
Emden R Gansner and Yifan Hu.
node7 , and use this information to bundle up edges as well8 – edge Efficient node overlap removal using a
bundling is a classic visualization improvement I’ll discuss in Section proximity stress model. In International
Symposium on Graph Drawing, pages
44.3.
206–217. Springer, 2008
Of course, community discovery, graph condensation, or visual- 8
Emden R Gansner, Yifan Hu, Stephen
ization were not originally developed with summarization in mind. North, and Carlos Scheidegger. Multi-
level agglomerative edge bundling for
Thus it is possible to design node aggregation methods that are
visualizing large graphs. In 2011 IEEE
specialized for summarization, even if inspired by other related Pacific Visualization Symposium, pages
approaches. One is Grass9 , 10 . In Grass one performs the node ag- 187–194. IEEE, 2011
9
Kristen LeFevre and Evimaria Terzi.
gregation in such a way that the errors in reconstructing the original
Grass: Graph structure summarization.
adjacency matrix are minimized. In Proceedings of the 2010 SIAM Interna-
Suppose you condensed the graph in Figure 38.2(a) into the graph tional Conference on Data Mining, pages
454–465. SIAM, 2010
in Figure 38.2(b). Now all you have is Figure 38.2(b), but you might 10
Matteo Riondato, David García-
want to know what is the probability that nodes 1 and 4 are con- Soriano, and Francesco Bonchi. Graph
nected. You can reconstruct Figure 38.2(a)’s adjacency matrix via summarization with quality guarantees.
Data mining and knowledge discovery, 31
Figure 38.2(b)’s – and keeping track of the original number of edges (2):314–349, 2017
inside and between each super node.

5
1 1 2 3 4 5
1 4, 5 1 0 2/3 2/3 1/3 1/3
2 4
2 2/3 0 2/3 1/3 1/3

2 2 3 2/3 2/3 0 1/3 1/3

4 1/3 1/3 1/3 0 1
1,2,3 5 1/3 1/3 1/3 1 0
3
(b) (c)
(a)
For instance, if two nodes u and v are in the same super node a, Figure 38.2: (a) An input graph.
then their expected connection probability is | Ea |/(|Va |(|Va | − 1)), (b) Aggregation of (a). Node la-
namely the number of edges collapsed inside a over all edges a could bels report the nodes collapsed
contain. Vice versa, if u and v are in different super nodes a and b, into the super node. Edge la-
then their connection probability is | Eab |/(|Va ||Vb |), again: number bels record the number of edges
of edges between a and b over all the possible edges that there could inside or between super nodes.
be. Thus, the reconstructed adjacency matrix of the original graph is (c) The adjacency matrix of (a)
the one in Figure 38.2(c). The quality function guiding this process is as reconstructed via (b).
mutual information: the higher the mutual information between the
original and the reconstructed matrix, the better the aggregation is.
Other approaches compress structurally equivalent nodes11 – see
11
Hannu Toivonen, Fang Zhou, Aleksi
Hartikainen, and Atte Hinkka. Compres-
Section 12.2 for a refresher on structural equivalence. sion of weighted graphs. In Proceedings
Respecting the adjacency of nodes is not necessarily the only of the 17th ACM SIGKDD international
conference on Knowledge discovery and
reasonable guiding principle for your aggregation. As I mentioned data mining, pages 965–973, 2011
previously, one might want to perform summarization to aid visual-
528 the atlas for the aspiring network scientist

ization. In this case, one might want to just simplify complex motifs 12
Cody Dunne and Ben Shneiderman.
that would tangle up your visualization12 . Motif simplification: improving network
Since we’re shifting perspectives, we might as well keep shifting visualization readability with fan,
connector, and clique glyphs. In
them. So far, we have assumed that aggregation involves the collapse
Proceedings of the SIGCHI Conference on
of nodes into super nodes. However, we could very well collapse Human Factors in Computing Systems,
edges instead. In this case, the edge is aggregated into what we call pages 3247–3256, 2013

a “compressor”, or virtual node. The idea is that high degree nodes,

especially those embedded in very dense parts of the network, are at
the center of a lot of redundant information.

Figure 38.3: (a) An input graph.

Nodes in red have low degree,
nodes in blue have high degree.
(b) Its summarization via edge
aggregation into a compressor
(in green).

(b)
(a)

Take Figure 38.3(a) as an example: its structure can be summarized

with a very simple formula – all red nodes connect to all blue nodes.
We can compress this information in a node that represent all red-
blue connections. The resulting graph – in Figure 38.3(b) – describes
exactly the same structure, but does so with nine edges instead of
18, at the price of adding a single node. This looks a bit like the com-
munity affiliation graph we saw in Section 34.4, and in fact I’ll have
you notice that – in this case – we’re practically just compressing a
6,3-clique.

38.2 Compression

If you’re familiar with zipped archives and programs like WinRAR,

you already know what is the guiding principle of this branch of
graph summarization. The idea here is to compress the original
structure so that we minimize the number of bits we need to encode
it. Suppose you’re compressing a text in ASCII format. Each letter
will cost you seven bits. However, by looking at the text, you realize
that many ASCII characters never appear. Thus you can re-encode all
characters using shorter codes: if you only use 40 distinct characters,
you only need a bit more than five bits per character, reducing a 1MB
text into ∼ 760kB. You can do better than that, realizing that most of
the times the character “h” follows specific other characters and so
on. In practice, you’re modeling your text with a model M. Encoding
M takes some bits, but it saves many more. This is following the
 graph summarization 529

same philosophy as the Infomap community discovery approach we

saw in Section 31.2.
Translating this into graph-speak, you want to construct a model
M of your graph G so that the length L describing both is minimal, 13
Saket Navlakha, Rajeev Rastogi,
or: min L( G, M ) = L( M ) + L( G | M ). An example13 creates M using a and Nisheeth Shrivastava. Graph
two-step code: (i) each super node in the summary is connected, in summarization with bounded error. In
Proceedings of the 2008 ACM SIGMOD
the original graph, to all nodes in all super nodes adjacent to it in the
international conference on Management of
summary; and (ii) we correct every edge mistake with an additional data, pages 419–432, 2008
instruction.

8
Figure 38.4: (a) An input graph.
2 7 ,8 4,5,6
2,3 1 (b) Its summarization via mini-
5 6
1 mization of description length.
+ (1,5)
3 - (4,8)
I label each super node with the
7 4 list of nodes it contains. On the
(b)
(a) bottom, the additional rules we
need to reconstruct (a).
Figure 38.4 shows the approach. The original graph in Figure
38.4(a) can be compressed in the graph in Figure 38.4(b). However,
the summary is not perfect. It assumes the existence of an edge that
does not really exist, and it misses another edge that exists. We add
these two rules to the model M and now the summary is a perfect
reconstruction of Figure 38.4(a). In Figure 38.4(b) we say that nodes 7
and 8 are connected to nodes 4, 5, and 6. This is mostly accurate, but
not completely correct: we need the additional rule that nodes 4 and
8 do not connect to each other.
The objective now is to find the best combination of summary 14
Paolo Boldi and Sebastiano Vigna. The
and additional rules that uses as little information as possible, given webgraph framework i: compression
techniques. In Proceedings of the 13th
or take a margin of error you can set as parameter. There are many international conference on World Wide
information-theoretic approaches in this category14 , 15 , 16 . Web, pages 595–602, 2004
A natural domain of application for compression-based summa-
15
Sebastian E Ahnert. Power graph
compression reveals dominant relation-
rization is in the description of evolving networks. In practice, one ships in genetic transcription networks.
has many snapshots of the same network and they are trying to re- Molecular BioSystems, 9(11):2681–2685,
2013
construct what the whole network looks like17 . The idea is to find the 16
Danai Koutra, U Kang, Jilles Vreeken,
model that is able to best represent all the snapshots you collected. and Christos Faloutsos. Vog: Summariz-
You might feel like aggregation and compression are basically ing and understanding large graphs. In
Proceedings of the 2014 SIAM international
the same category. After all, if you look at Figure 38.4, what you’re conference on data mining, pages 91–99.
seeing is basically an aggregation strategy. The fundamental differ- SIAM, 2014
ence between the two categories is the existence of the model M. In 17
Neil Shah, Danai Koutra, Tianmin
Zou, Brian Gallagher, and Christos
aggregation, there is no M: we simply brute force our way through Faloutsos. Timecrunch: Interpretable
the graph to save every node or edge we can, regardless whether dynamic graph summarization. In
we’re uncovering common patterns or not. The existence of M in the Proceedings of the 21th ACM SIGKDD
International Conference on Knowledge
compression category, instead, forces us only to perform an aggrega- Discovery and Data Mining, pages
tion if it results in a leaner and more elegant M. As a consequence, 1055–1064, 2015
530 the atlas for the aspiring network scientist

M itself is an important result of the procedure, because it contains

information about the common patterns you can find in your original
structure.

38.3 Simplification

In this class we group solutions that are the lovechildren of network

sampling (Chapter 25) and backboning (Chapter 24). The idea here is
not to create “super nodes” like in the previous two classes. Here, we
look at the original structure. However, we simplify it by removing
nodes and edges that we consider “unimportant”. Sampling and
backboning techniques can be considered simplification strategies
that are purely structural: they only use information coming from the
graph’s topology.

Timur Bekmambetov
Johnny Depp
Figure 38.5: (a) An input graph:
Johnny Depp directors (in blue) and actors
Tim Burton
Helena Bonham (in red) connected if they collab-
Carter
Helena Bonham orated with each other. (b) Its
Eva Green Carter simplification via the selection
Bernardo Bertolucci Eva Green
of only actor-type nodes di-
Daniel Craig (b) rectly connected to Tim Burton.
(a)
Those are not the only valid approaches. If the graph also has
metadata attached to nodes – or edges – we can exploit them. For 18
Zeqian Shen, Kwan-Liu Ma, and
instance, Ontovis18 allows the simplification of the graph via the Tina Eliassi-Rad. Visual analysis of
large heterogeneous social networks
specification of a set of attribute values we’re interested in studying. by semantic and structural abstraction.
For instance, the graph in Figure 38.5(a) can be simplified into the IEEE transactions on visualization and
one in Figure 38.5(b), if we’re only interested in knowing the rela- computer graphics, 12(6):1427–1439, 2006

tionships between actors (node type value) working with Tim Burton
(topology attribute).
Ontovis finds the best way to simplify the graph, primarily fo-
cusing on its visual characteristics when plotted in 2D: it is first
and foremost a visualization-aiding tool. Ontovis focuses on node 19
Cheng-Te Li and Shou-De Lin. Ego-
attributes, but one could also switch their focus to edges19 . centric information abstraction for
heterogeneous social networks. In 2009
Note, also, that another difference with sampling is that, in graph International Conference on Advances
simplification, we are not really interested in preserving any spe- in Social Network Analysis and Mining,
cific property of the original graph. This is, instead, a core focus of pages 255–260. IEEE, 2009

network sampling.
When not necessarily focusing on visualization, the simplifica-
tion approach uses a database metaphor to help the user navigate
20
Surajit Chaudhuri and Umeshwar
Dayal. An overview of data warehousing
between different “views” of the network data. A classical database and olap technology. ACM Sigmod record,
infrastructure is OLAP20 , which stands for OnLine Analytical Pro- 26(1):65–74, 1997
 graph summarization 531

Full Database Slice Dice Drill

Place
Place Place Time
Product Product
Time Time
Product What sold when and
Time and place of
Which products sold where, for a subset Which products
all purchases across
when and where of products, places, a place sold
all products
& times

Figure 38.6: An example of op-

cessing. In an OLAP database you have data that is inherently mul- erations on an OLAP database.
tidimensional, for instance sales can happen in different shops, at
different times, via different product categories, and customer classes,
etc. OLAP allows you to represent this with a “data cube” that you
can slice and dice to aggregate the dimensions. Figure 38.6 shows
you a visual representation of what different operations look like –
and mean. 21
Yuanyuan Tian, Richard A Hankins,
and Jignesh M Patel. Efficient aggre-
Graph OLAP21 , 22 , 23 is fundamentally the same thing applied to gation for graph summarization. In
networks, using node/edge attributes and characteristics to drive the Proceedings of the 2008 ACM SIGMOD
simplification procedure. While traditional graph OLAP works best international conference on Management of
data, pages 567–580, 2008
with categorical node attributes, there are also ways to dice and slice 22
Chen Chen, Xifeng Yan, Feida Zhu,
your graph using numeric attributes24 , allowing you to also consider Jiawei Han, and S Yu Philip. Graph olap:
node properties such as the degree. Towards online analytical processing on
graphs. In ICDM, pages 103–112. IEEE,
Another related category of approaches is “graph sketches”25 , 26 , 27 . 2008
23
Peixiang Zhao, Xiaolei Li, Dong Xin,
and Jiawei Han. Graph cube: on ware-
38.4 Influence Based housing and olap multidimensional
networks. In Proceedings of the 2011 ACM
SIGMOD International Conference on
In influence-based summarization one is interested in having a sum- Management of data, pages 853–864, 2011
marized graph in which influence events follow the same dynamics 24
Ning Zhang, Yuanyuan Tian, and
Jignesh M Patel. Discovery-driven graph
as in the original graph. That is: if something is flowing through the
summarization. In ICDE, pages 880–891.
graph, percolating node to node, the percolation in the summarized IEEE, 2010
graph follows the same dynamics as in the original graph. 25
Kook Jin Ahn, Sudipto Guha, and
Andrew McGregor. Graph sketches:
There are a few ways to wrap your head around this concept. I
sparsification, spanners, and subgraphs.
feel that I can provide two very different approaches for you, that In SIGMOD-SIGACT-SIGAI, pages 5–14,
should serve different types of understanding graph dynamics. The 2012
26
Edo Liberty. Simple and deterministic
first rests on a data-driven approach. Suppose you have a social matrix sketching. In SIGKDD, pages
network, where nodes are people. Some users are early adopters 581–588, 2013
of a new product. As they perform an action, some of their friends
27
Mina Ghashami, Edo Liberty, and
Jeff M Phillips. Efficient frequent
will see it and will imitate them. This means that you can detect directions algorithm for sparse matrices.
“tribes” of people reacting with the same timing to the same stimuli. In SIGKDD, pages 845–854, 2016
532 the atlas for the aspiring network scientist

This is basically like doing community discovery, with the difference

being that you’re not maximizing internal density, but simply the
synchronization of an action. What I described is, for instance, what
GuruMine detects, and you can use Section 18.2 as a refresher.

Figure 38.7: (a) An influence

graph. The edge direction tells
you who influences whom.
The node color tells you the
detected “tribes”. (b) The sum-
mary graph, compressing all
tribes into a node, to preserve
(b) influence dynamics.
(a)

Visually, this would look like Figure 38.7. The central hubs influ-
ence each other, and each is responsible for influencing their branch.
Thus one could summarize the graph as a clique of interacting tribes. 28
Yasir Mehmood, Nicola Barbieri,
Of course, you don’t have to use GuruMine for this. In some cases, re- Francesco Bonchi, and Antti Ukkonen.
searchers have used special adaptations to estimate community-level Csi: Community-level social influence
analysis. In Joint European Conference
influence28 . on Machine Learning and Knowledge
There’s a completely different way to interpret summarization Discovery in Databases, pages 48–63.
Springer, 2013
by influence preservation. We have seen that the spectrum of the
Laplacian can be used to partition a graph – solving the cut problem
(Section 8.4). This is related to diffusion processes: the reason why
the eigenvectors of the Laplacian help you with cutting is because
they are a sort of simulation of a diffusion process, and the edges to
cut are the bottlenecks though which things cannot flow efficiently. 29
Manish Purohit, B Aditya Prakash,
For now, let’s take this for granted, but we’ll see more about this rela- Chanhyun Kang, Yao Zhang, and
tionship in Section 40.2, where we’ll talk about using the Laplacian to VS Subrahmanian. Fast influence-based
coarsening for large networks. In
estimate distances between sets of nodes by releasing a flow from the Proceedings of the 20th ACM SIGKDD
nodes in the origin to the nodes in the destination. international conference on Knowledge
If we want to summarize the graph to preserve these diffusion discovery and data mining, pages 1296–
1305, 2014
properties, we can use the Laplacian to guide our process. What
30
Michael Mathioudakis, Francesco
we’re after, in the simplest possible terms, is a smaller Laplacian ma- Bonchi, Carlos Castillo, Aristides Gio-
trix, with fewer rows and columns, that has the same eigenvectors29 . nis, and Antti Ukkonen. Sparsification
Among other interesting approaches there is SPINE30 . In SPINE, of influence networks. In Proceedings
of the 17th ACM SIGKDD international
one analyzes many influence events in the network. Then, SPINE conference on Knowledge discovery and
only keeps in the network the edges that are able to better explain data mining, pages 529–537, 2011
the paths of influence you observe. You might realize that an edge
is never used to transport information, and thus you could remove
it from the structure without hampering your explanatory power.
Figure 38.8 shows an example of this.
 graph summarization 533

Figure 38.8: (a) An influence

graph. The edge color rep-
resents the path taken by an
hypothetical spreading event.
(b) The summary graph, includ-
ing only the edges used in the
spreading process.

(a) (b)

38.5 Summary

1. Graph summarization is the task of reducing the size of your

graph so that you can facilitate different operations, such as analy-
sis, storage, data cleaning, and/or network visualization.

2. There are many ways to perform summarization. One is to do so

via aggregation: you find coherent modules in your network and
you collapse them into the same node, aggregating all incoming
and outgoing edges.

3. A second category is compression: you similarly try to aggregate

the graph, but this time you record your operation in a model.
The model also has to be encoded, and thus you have to find the
simplest possible model that best represents your original graph.

4. The third approach is simplification. This is especially relevant for

visualization: you want to simplify the graph so that motifs that
would clutter its representation are reduced and do not cross each
other.

5. Finally, you might have influence in mind: some process is spread-

ing on your network and you want to keep only the connections
that are most likely responsible for that process to percolate
through the nodes.

6. Making a summary of a chapter about summarization is delight-

fully meta and I’m having a hell of a good time.

38.6 Exercises

1. Perform label percolation community discovery on the network at

[Link] Use the
detected communities to summarize the graph via aggregation.

2. The table at [Link]

[Link] contains the information of which node (first
534 the atlas for the aspiring network scientist

column) influenced which other node (second column). Use it

to summarize the graph by keeping only the edges used by the
spreading process.

3. Summarize the summary you generated answering question #1

with the data from question #2. Do you still obtain a connected
graph?
39
Frequent Subgraph Mining

Our experience with modeling real world networks tells us that they
are not random: they are an expression of complex dynamics shaping
their topology. Among other things explored in other chapters, this
also means that networks will tend to have overexpressed connection
patterns. Nodes and edges will form different shapes much more
– or less – often than what you’d expect if the connections were
random. For instance, the clustering coefficient analysis tells us that
you’re going to find more triangles than expected given the number
of nodes or edges.
Frequent subgraph mining is the branch of network science that
attempts to find these overexpressed patterns, when they are more
complex than a simple triangle. Want to know whether a square
with a dangling edge appears more often than chance? You have to
perform subgraph mining! In frequent subgraph mining we have a
wealth of techniques to systematically and efficiently enumerate all
possible subgraphs and finding the ones that occur more often in
your networks.

1
Ron Milo, Shai Shen-Orr, Shalev
39.1 Network Motifs Itzkovitz, Nadav Kashtan, Dmitri
Chklovskii, and Uri Alon. Network
We start by defining the building blocks of complex networks. These motifs: simple building blocks of
complex networks. Science, 298(5594):
are network motifs1 , 2 , 3 , 4 . A network motif is a subgraph of your 824–827, 2002
original network with a given topology. A triangle is a motif, a 2
Shai S Shen-Orr, Ron Milo, Shmoolik
square is a motif, the five nodes with the connection pattern in Figure Mangan, and Uri Alon. Network
motifs in the transcriptional regulation
39.1 is a motif. network of escherichia coli. Nature
Generally, one wants to know which motifs are relevant for a genetics, 31(1):64, 2002
network and which ones aren’t. So the standard technique is to
3
Uri Alon. Network motifs: theory and
experimental approaches. Nature Reviews
follow a relatively simple procedure. First, you count how many Genetics, 8(6):450, 2007
times each motif appears in your network. Second, you define a null 4
Jukka-Pekka Onnela, Jari Saramäki,
model of your network, keeping its relevant properties fixed – maybe János Kertész, and Kimmo Kaski.
Intensity and coherence of motifs in
just the degree distribution. Third, you count the expected number weighted complex networks. Physical
of occurrences of the motifs in the null model. Finally, you compare Review E, 71(6):065103, 2005
536 the atlas for the aspiring network scientist

Figure 39.1: Three examples of

motifs in complex networks.

(b) (c)
(a)

with your observation, so that you can build an idea of the statistical 5
Alex Arenas, Alberto Fernandez,
significance of the motif. Santo Fortunato, and Sergio Gomez.
We use network motifs for many different applications. For in- Motif-based communities in complex
networks. Journal of Physics A: Math-
stance, and I won’t get tired of bringing this up, we use them for
ematical and Theoretical, 41(22):224001,
community discovery5 . Of course nobody forces you to have exclu- 2008a
sively the simple motifs I depict in Figure 39.1. One can extend the
6
Lauri Kovanen, Márton Karsai, Kimmo
Kaski, János Kertész, and Jari Saramäki.
concept of network motifs to encompass temporal networks6 , 7 – so Temporal motifs in time-dependent
time-evolving motifs –, and multilayer networks8 , 9 . networks. Journal of Statistical Mechanics:
Theory and Experiment, 2011(11):P11005,
You might have noticed that the examples I show in Figure 39.1
2011
are all very small. They include only a handful of nodes and edges. 7
Ashwin Paranjape, Austin R Benson,
There’s a reason for that. Finding motifs in a large network is a hard and Jure Leskovec. Motifs in temporal
networks. In Proceedings of the Tenth
problem. There are clever techniques to enumerate specific small ACM International Conference on Web
motifs which are reasonably fast10 , 11 . However, in general, one has to Search and Data Mining, pages 601–610.
ACM, 2017
solve the scary graph isomorphism problem, which is the topic of the 8
Federico Battiston, Vincenzo Nicosia,
next section. Mario Chavez, and Vito Latora. Multi-
layer motif analysis of brain networks.
Chaos: An Interdisciplinary Journal of
Nonlinear Science, 27(4):047404, 2017
39.2 Graph Isomorphism 9
Manlio De Domenico, Vincenzo
Nicosia, Alexandre Arenas, and Vito La-
Colloquially speaking, we can state the graph isomorphism problem tora. Structural reducibility of multilayer
as follows: given two graphs, decide whether they are the same networks. Nature communications, 6:6864,
2015b
graph. Two graphs are the “same” if they have the same topology. 10
Ali Pinar, C Seshadhri, and
More formally, graph isomorphism is the search of a function which Vaidyanathan Vishal. Escape: Effi-
maps each node of a graph to each node of the other graph, such that ciently counting all 5-vertex subgraphs.
In Proceedings of the 26th International
they have the same neighbors – identically mapped nodes12 . Conference on World Wide Web, pages
Are the graphs in Figure 39.2(a-b) isomorphic? Table 39.2(c) at- 1431–1440. International World Wide
Web Conferences Steering Committee,
tempts to answer positively: it relabels nodes from Figure 39.2(a)
2017
into nodes from Figure 39.2(b). Since all nodes are connected to 11
Leo Torres, Pablo Suárez-Serrato, and
their identically labeled neighbors, the answer is yes, the graphs are Tina Eliassi-Rad. Non-backtracking
cycles: length spectrum theory and
isomorphic – in fact they’re both 4-cliques.
graph mining applications. Applied
This example is simple enough, but the problem gets very ugly Network Science, 4(1):41, 2019
very soon when we start considering non-trivial graphs. Subgraph 12
Brendan D McKay et al. Practical
graph isomorphism. Department of Com-
isomorphism is an NP-complete problem13 : a type of problem where
puter Science, Vanderbilt University
a correct solution requires you to try all possible combinations of Tennessee, USA, 1981
labeling. This grows exponentially and requires a time longer than 13
Scott Aaronson. P=?np. Electronic
Colloquium on Computational Complexity
the age of the universe even for simple graphs of a few hundreds (ECCC), 24:4, 2017. URL [Link]
nodes. [Link]/report/2017/004
 frequent subgraph mining 537

1 4 a G1 G2 Figure 39.2: (a, b) Two graphs,

c
1 c with their nodes labeled with
b their ids. (c) A function connect-
2 a
3 d ing the node ids from graph (a)

2 3 d 4 b to the node ids of graph (b).

(c)
(a) (b)

That is why we’re looking for efficient algorithms to solve graph

isomorphism. Recently, a claim of a quasi-polinomial algorithm 14
László Babai. Graph isomorphism in
shook the world14 – well, parts of it. However, this is more of a quasipolynomial time. In Proceedings of
theoretical find, which cannot be used in practice. To the best of the forty-eighth annual ACM symposium
on Theory of Computing, pages 684–697.
my knowledge, the current practical state of the art to solve graph
ACM, 2016
isomorphism is the VF2 algorithm15 , 16 . 15
Luigi Pietro Cordella, Pasquale Fog-
The first step is to make all easy checks that don’t really require gia, Carlo Sansone, and Mario Vento.
An improved algorithm for matching
much thought. For instance, two graphs cannot be isomorphic if they large graphs. In 3rd IAPR-TC15 workshop
have a different number of nodes or a different number of edges. If on graph-based representations in pattern
this check fails, you can safely deny the isomorphism. The real core recognition, pages 149–159, 2001
16
Luigi P Cordella, Pasquale Foggia,
of VF2 is the following: Carlo Sansone, and Mario Vento. A
(sub) graph isomorphism algorithm
• Step #1: match one node in G1 with a node in G2; for matching large graphs. IEEE
transactions on pattern analysis and
• Main loop: try to match the n-th node in G1 with the n-th node machine intelligence, 26(10):1367–1372,
in G2. If the match is unsuccessful, recursively step back your 2004

previous matches and try a new match;

• End loop, case 1: you explored all nodes in G1 and G2, then the
graphs are isomorphic;

• End loop, case 2: you have no more candidate match, then the
graphs are not isomorphic.

Most of the heavy lifting is made in the main loop, when checking
whether a match is successful or not. An illustrated example with a
simple graph would probably be helpful. Consider Figure 39.3. VF2
attempts to explore the tree of all possible node matching (Figure
39.3(c)). It starts from the empty match – the root node.
The first attempted match always succeeds, as any node can be
matched to any other node – in this case matching node 1 with node
a. For the second match to succeed, we need that the two matched
nodes are connected to each other. Since node a connects to node b
and node 1 connects to node 2, then the 1 = a and 2 = b match is a
success.
However, attempting to match 3 = c fails, because while node 2
is connected to node 3, node b (matched with 2) isn’t connected to
c (matched to 3). Thus VF2 backtracks: it undoes the last matches
538 the atlas for the aspiring network scientist

*
Figure 39.3: (a, b) Two graphs,
with their nodes labeled with
10 11
1 their ids. (c) The inner data
b 3 1=a 1=b
structure used by the VF2 algo-
rithm to test for isomorphism.
6
a 2 5 2 9 12
I label each node with the at-
tempted match. The node color
c 2=b 2=c 2=a tells the result of the match
1
(green = successful, red = un-
(a) (b) 3 7
4 8 13 successful). I label the edges to
follow the step progression of
3=c 3=b 3=c the algorithm.

(c)

and starts from the last successful match – provided that there are
possible matches to try. In this case there aren’t , so it backtracks
again.
Trying to set 2 = c and 3 = b fails again, for the same reason as
before. So VF2 has to give up also on the 1 = a match and start from
scratch. Luckily, there’s another possible move: 1 = b. When we go
down the tree all matches are successful, until we touched all nodes
in the graph. At that point, we can safely conclude the two graphs
are isomorphic. Note that Figure 39.3(c) doesn’t include the branches
that VF2 never tries in this case, for instance the 1 = c branch.
As expected, multilayer networks provide another level of dif-
ficulty. One can perform graph isomorphism directly on the full 17
Mikko Kivelä and Mason A Porter.
multilayer structure17 , or give up a bit of the complexity and repre- Isomorphisms in multilayer networks.
sent them as labeled multigraphs18 , 19 . IEEE Transactions on Network Science and
Engineering, 5(3):198–211, 2017
18
Vijay Ingalalli, Dino Ienco, and
39.3 Transactional Graph Mining Pascal Poncelet. Sumgra: Querying
multigraphs via efficient indexing. In
International Conference on Database
So far we’ve been dealing with network motifs on a “top-down” and Expert Systems Applications, pages
approach. We have some motifs of interest and we ask ourselves 387–401. Springer, 2016
whether they are overexpressed or underexpressed. This implies that
19
Giovanni Micale, Alfredo Pulvirenti,
Alfredo Ferro, Rosalba Giugno, and
you have to start with your motifs already in mind. This might not be Dennis Shasha. Fast methods for finding
possible. Sometimes, you need a “bottom-up” approach: you want an significant motifs on labelled multi-
relational networks. Journal of Complex
algorithm telling you the frequencies of all possible simple network Networks, 2019
motifs. This is usually the task of frequent subgraph mining.
We split frequent subgraph mining in two: transactional and sim-
ple graph mining. Transactional graph mining was developed first,
because single graph mining introduces some non-trivial problems.
It’s best to start by explaining transactional graph mining, and we’ll
deal with the additional obstacles of single graph mining later (in
 frequent subgraph mining 539

Section 39.4).

Frequent Itemset Mining 20

Jiawei Han, Jian Pei, and Yiwen Yin.
Transactional graph mining is inspired by frequent itemset mining, a Mining frequent patterns without
candidate generation. In ACM sigmod
classical problem in data mining20 , 21 , 22 . In frequent itemset mining, record, volume 29, pages 1–12. ACM,
your input is a collection of sets. Each set includes different objects. 2000
21
Jian Pei, Jiawei Han, Runying Mao,
The objective is to find the subsets that appear more often in your et al. Closet: An efficient algorithm
collection of sets. The number of times each subset appear is called for mining frequent closed itemsets.
support. We increase support by one each time we find a subset In ACM SIGMOD workshop on research
issues in data mining and knowledge
inside a set in the collection. Figure 39.4 provides an example. discovery, volume 4, pages 21–30, 2000
From Figure 39.4 you can see that this problem explodes in com- 22
Jiawei Han, Hong Cheng, Dong Xin,
plexity very soon. Even with just five itemsets and five items, the and Xifeng Yan. Frequent pattern
mining: current status and future
number of possible subsets can get very high. Thus the crucial prob- directions. Data mining and knowledge
lem in frequent itemset mining is how to efficiently explore the discovery, 15(1):55–86, 2007
search space. There are many algorithms to do it, but I’ll focus on 23
Rakesh Agrawal, Ramakrishnan
the old and legendary Apriori23 , given its simplicity and its didactic Srikant, et al. Fast algorithms for mining
association rules. In Proc. 20th int. conf.
potential. very large data bases, VLDB, volume 1215,
pages 487–499, 1994

Data Frequency
Figure 39.4: An example of
4 3 2 frequent itemset mining. The
5 2 1 original data is on the left, one
3 1 line per set of items (itemset).
We calculate the frequency
2 1
of each itemset, including all
1 3 subsets (right).
1
1

The first thing you do is giving up on the idea of finding all sub-
sets. You only want to find the frequent ones. Thus you establish a
support threshold: if a subset fails to occur in that many sets, then
you don’t want to see it. This allows you to prune the search space.
If subset A is not frequent, then none of its extensions can be: they
24
That is, the support function is anti-
monotonic: it can only stay constant or
have to contain it so they can be at most as frequent as A is24 . Thus, shrink as your set grows in size.
once you rule out subset A, none of its possible extensions should
even be considered, since none can be frequent. This usually allows
to perform much fewer tests than the possible ones, and still return
all frequent subsets.
For instance, in Figure 39.4, the orange circle only occurs once.
If the support threshold is 2, we know we don’t need to check the
red-orange, purple-orange, and red-purple-orange subsets. With one
check, we prevented three.
540 the atlas for the aspiring network scientist

Figure 39.5: Apriori’s search

space. Each node represents a
subset and bears the colors of
the items it contains. Given that
the orange item is not frequent,
Apriori marks as red the links
it needs not to follow, because
they lead to a subset containing
the infrequent orange item,
which cannot make the support
threshold.

Actually, we prevented many more. Figure 39.5 shows the entire

search space in a dataset with five different items. As you can see,
if we have an item which does not pass the support threshold, the
search space crumbles. Apriori explores this graph and marks all
nodes with an orange item as infrequent, checking only the itemsets
without red connections. This doesn’t even take into account the
other infrequent subsets from Figure 39.4.
As a small aside, note that, once you know the frequencies of all
sets, you can build what we call “association rules”. What you want
to do is to find all rules in the form of: “If a set contains the objects A,
then it is likely to also contain object b”. Figure 39.6 shows a simple
example of the problem we’re trying to solve.

Some Rules
Figure 39.6: An example of as-
75%
sociation rule mining. Assume
60% the frequencies of each itemset
100% are the ones from Figure 39.4.
66% We generate rules recording the
relative frequency of observing
Suppose that, in your data, you see 100 instances of sets containing two itemsets. Note that these
objects a1 , a2 , and a3 . And let’s say that, among them, 80 also contain frequencies are not symmet-
object b. Then you can say, with 80% confidence, that the following ric! While the green item only
rule applies: { a1 , a2 , a3 } → b. The { a1 , a2 , a3 } part is the antecedent of occurs 60% of the times a blue
the rule, while b is the consequent. item occurs, every time green
You can also correct your confidence for chance, if you know b’s occurs we also have the blue
overall frequency in the data, and the size of the dataset. This is item.
the “lift” measure. Let’s say that, in our example, b appears in 120
sets. Also, our dataset contains a total of 400 sets. The lift of the
rule is the relative frequency (support) of { a1 , a2 , a3 , b} (80/400) over
the product of the support of the antecedent and the consequent
 frequent subgraph mining 541

(100/400 × 120/400). This latter quantity gives us the probability of

the antecedent and the consequent to co-appear randomly. Doing
the math: .2/(.25 × .3) = 2.6̄. This means that the rule appears more
than twice as much as we would expect if there was no relationship
between the antecedent and the consequent. A lift lower than one
indicates items appearing less often than they would do at random: a
sign that the rule is unlikely to be interesting.
Lift is one of those Swiss army knives that has been independently
invented in multiple fields. For instance it is also known as Relative 25
Jun Zhang and F Yu Kai. What’s the
Risk in in statistics25 , and Revealed Comparative Advantage in trade relative risk?: A method of correcting
economics26 . the odds ratio in cohort studies of
common outcomes. Jama, 280(19):
When you replace itemsets with motifs, you obtain the GERM
1690–1691, 1998
algorithm: that is why I introduced it as graph association rule 26
Bela Balassa. Trade liberalisation and
mining in the link prediction chapters (Sections 20.6 and 21.2). How “revealed” comparative advantage 1. The
manchester school, 33(2):99–123, 1965
to go from itemsets to network motifs is the topic of the rest of this
section.

From Itemsets to Network Motifs

Transactional graph mining applies all this machinery to graphs.

Rather than looking at simple sets of items, we look at graph pat-
terns: triangles, 4-cliques, bi-cliques... any possible combination of
nodes and edges. So we have a database of many different graphs
and we ask in how many graphs the motif appears. This is our defini-
tion of support, as Figure 39.7 shows.
The big problem is how to enumerate all possible graphs effi-

Graph DB
Figure 39.7: For each of the
patterns on the left we check
Support
Motif whether a graph in the database
(on top) contains it (green
checkmark) or not (red cross).
3 The number of graphs in the
database containing the motifs
is its support.
3

1
542 the atlas for the aspiring network scientist

ciently. We want to avoid trying to count the occurrences of a graph

pattern G 00 if it contains a pattern G 0 , which we already counted and
found not frequent enough. Since G 00 is an extension of G 0 , we al-
ready know it cannot be frequent enough: a larger graph can at most
be as frequent as the least frequent of its subgraphs.
There are many ways to do this. An incomplete list of approaches 27
Jun Huan, Wei Wang, and Jan Prins.
includes FFSM27 , Gaston28 , Moss29 , etc. You can find relevant lit- Efficient mining of frequent subgraphs
erature for a historic quantitative comparison of these methods30 . in the presence of isomorphism. In Third
IEEE International Conference on Data
Just like for frequent itemset mining, I’ll focus on a specific method, Mining, pages 549–552. IEEE, 2003
gSpan31 , 32 , given its historical and didactic relevance. 28
Siegfried Nijssen and Joost N Kok.
Graphs are more complex structures than itemsets, so building a The gaston tool for frequent subgraph
mining. Electronic Notes in Theoretical
search space like the one Apriori creates (Figure 39.5) is tricky. If you Computer Science, 127(1):77–87, 2005
cannot explore the search space like Apriori does, it’s even harder 29
Christian Borgelt and Michael R
to prune it by avoiding exploring patterns you already know they Berthold. Mining molecular frag-
ments: Finding relevant substructures
are not frequent. gSpan solves the problem by introducing a graph of molecules. In 2002 IEEE Interna-
lexicographic order (which I’m going to dumb down here, the full tional Conference on Data Mining, 2002.
details are in the paper). Proceedings., pages 51–58. IEEE, 2002

a
a Figure 39.8: Three possible DFS
b
a d explorations of the graph on
b c top. Blue arrows show the DFS
b
a exploration and purple dashed
c arrows indicate the backwards
c
edges (pointing to a node we
already explored). Remember
a a Start
a
a a a that DFS backtracks to the last
Start
b b b
a d a d a d explored node, not to where the
b c b c b c backward edges points.
b b b
a a a
c c c Marc Wörlein, Thorsten Meinl,
c c c
30

Start Ingrid Fischer, and Michael Philippsen.

A quantitative comparison of the
subgraph miners mofa, gspan, ffsm,
and gaston. In European Conference on
Suppose you have a graph, as in Figure 39.8 (top). You can explore Principles of Data Mining and Knowledge
Discovery, pages 392–403. Springer, 2005
it using a DFS strategy. Actually, you can have many different DFS 31
Xifeng Yan and Jiawei Han. gspan:
paths: you can start from node a, from node b, ..., then you can move Graph-based substructure pattern min-
through a different edge any time. We can encode each DFS explo- ing. In 2002 IEEE International Conference
on Data Mining, 2002. Proceedings., pages
ration with a DFS code: a sequence of quintuples (id of source node, 721–724. IEEE, 2002
id of target node, label of source node, label of edge, label of target 32
Xifeng Yan and Jiawei Han. Close-
node). Table 39.1 shows the DFS codes for the three explorations in graph: mining closed frequent graph
patterns. In Proceedings of the ninth
Figure 39.8. Note that, every time we explore a node with backward ACM SIGKDD international conference
edges, we insert them in the code immediately, before continuing on Knowledge discovery and data mining,
pages 286–295. ACM, 2003
with the DFS exploration.
Once you have all DFS codes for a graph, you can find the min-
 frequent subgraph mining 543

Order DFS1 DFS2 DFS3

Table 39.1: The DFS codes for
0 (0, 1, a, a, b) (0, 1, b, a, a) (0, 1, a, a, a)
the DFS explorations in Figure
1 (1, 2, b, b, a) (1, 2, a, a, a) (1, 2, a, a, b)
39.8. DFS exploration edges in
2 (2, 0, a, a, a) (2, 0, a, b, b) (2, 0, b, b, a)
blue. For backward edges (pur-
3 (2, 3, a, c, c) (2, 3, a, c, c) (2, 3, b, b, c)
ple) the node id of the source is
4 (3, 1, c, b, b) (3, 0, c, b, b) (3, 0, c, c, a)
higher than the node id of the
5 (1, 4, b, d, c) (0, 4, b, d, c) (2, 4, b, d, c)
target.
imum DFS code, by simply sorting them alphanumerically. The
minimum DFS code – in our example the third one (DFS3 in Table
39.1) – is a canonical representation for a graph. Two graphs with
the same minimum DFS code are isomorphic, and if you encounter a
non-minimum DFS code in your search space you can safely ignore
it.
This is a big deal, because now we reduced the graph mining prob-
lem to a frequent itemset problem. What before I called items a1 , a2 ,
and a3 are now items like (0, 1, a, a, a), (1, 2, a, a, b), and (2, 0, b, b, a). If
you always find them together with the additional (2, 3, b, b, c), you
have built a graph association rule! So, with this canonical graph
representation, you can solve the frequent subgraph mining problem
with any frequent itemset algorithm (like Apriori).

39.4 Single Graph Mining

Transactional graph mining is great because it’s a very similar prob-

lem to frequent itemset mining and has some interesting applications.
For instance, your graph database could contain thousands of differ-
ent chemical compounds, and you want to find the most common
substructures among all those molecules. However, there’s a great
deal of network data that doesn’t fit this mold.
For instance, if you want to mine all frequent patterns in a social
network, you typically have a single large graph. In those cases, you
cannot have as definition of support the “number of graphs in which
the pattern appears” as in the transactional setting. That number is
always going to be either zero – the pattern doesn’t appear – or one.
What you want to do, instead, is to count the number of times the
patterns appear in the single network.
However, such naive support definition won’t work. To see why,
consider the example in Figure 39.9. It’s obvious that the red node
motif appears only once: there’s only one red node. However, naively,
we could say that motif m2 appears twice. This is unacceptable,
because it breaks the anti-monotonicity rule: a motif can only occur
at most as much as the least frequent of its sub-motifs. Since m2
contains m1 , it cannot appear more often than m1 .
544 the atlas for the aspiring network scientist

Motifs Data
Figure 39.9: Two motifs (left)
m1 and our graph data (right).
Motif m1 appears only once.
How many times does motif m2
m2 appears?

If we were to accept it to have higher support than its submotifs,

we could not prune the search space using Apriori’s strategy, mean-
ing that we would have to explore an exponentially growing set of
possibilities. This would make single graph mining impractical in all
but trivial scenarios.
So the main quest for single graph mining is the one for an anti-
monotonic support definition that is sufficiently easy to compute
– otherwise we don’t gain much time – and that hopefully makes
intuitive sense. I’m going to show you three alternatives: using ego
networks (Section 26.1), harmful overlap, and the minimum image
support.

Ego Networks
One option is to bring back the problem into familiar territory. One
can split the single graph in many different subgraphs and then
apply any transactional graph mining technique. For example, one
could take the ego networks of all nodes in the network. The support
definition would then be the number of nodes seeing the pattern
around them in the network.

Harmful Overlap
Another option starts from recognizing that the entire problem of
non-monotonicity is due to the fact that motif m2 appears twice only
because we allow the re-use of parts of the data graph when counting
the motif’s occurrences. In Figure 39.9, we use the red node in the
data graph twice to count the support of m2 . In practice, the two
patterns supporting m2 overlap: they have the single red node in
common. We could forbid such overlap: we don’t allow the re-use
of nodes when counting a motif’s occurrences. With such a rule, m2
would appear only once in the data graph. If we applied the rule, 33
Michihiro Kuramochi and George
we would have an anti-monotone support definition33 : larger motifs Karypis. Finding frequent patterns in
a large sparse graph. Data mining and
would only appear fewer times or as many times as the smaller knowledge discovery, 11(3):243–271, 2005
motifs they contain.
To see how, consider Figure 39.10. The motif appears four times,
but each of these four occurrences share at least one node. We can
 frequent subgraph mining 545

Motif Data
A B Figure 39.10: From left to right:
A 1 D Simple a pattern, the graph dataset,
Overlap and its corresponding simple
2 3 4 C D and harmful overlap graphs.
B C A B
I label each occurrence of the
5 6 7 Harmful motif with a letter, which also
labels the corresponding node
Overlap
8 9 C D in the overlap graph.

create an “overlap graph” in which each node is an occurrence in

the data graph, and we connect occurrences if they share at least
one node. If we forbid overlaps, we only want to count “complete”
and non-overlapping occurrences. This is equivalent of solving the
maximum independent set problem (see Section 9.3) on the overlap
graph: finding the largest set of nodes which are not connected to
any other member of the set. In this case, we have four independent
sets all including a single node – because the overlap graph is a
clique – and thus the pattern occurs only once.
This is the simple overlap rule and it is usually too strict. There
are some overlaps between the occurrences that do not “harm” the 34
Mathias Fiedler and Christian Borgelt.
anti-monotonicity requirement for the support definition34 . To find Support computation for mining
harmful overlaps you need to do two things. First, you look at which frequent subgraphs in a single graph. In
MLG, 2007
nodes are in common between two occurrences. For instance, in
Figure 39.10, A ∩ B = {1, 8}, A and B share nodes 1 and 8. Second,
you need to make sure that these nodes that are in common between
the two occurrences are not required to map the same nodes at the
same time. In the case of A and B they are, because the only way to
map the top red node in the motif is to use node 8 for A and 1 for B.
This is a harmful overlap.
The non-harmful overlaps are the ones in which this doesn’t
happen. For instance A and C are not overlapping harmfully: the
only node in common between A and C is node 1. However, we
do not need node 1 to map the same node in the motif in A and C:
when we use node 1 in A we use node 9 in C, when we use node 1
in C we use node 8 in A. Node 1 is also the only node in common
between A and D, but in this case the overlap is harmful, because we
use it to map the same node in the motif: the bottom red node.
The simple and harmful overlap build different overlap graphs,
but then they count occurrences in the same way, using the maxi-
mum independent set problem. In the example from Figure 39.10 this
leads to different support values: for the harmful support, the motif
546 the atlas for the aspiring network scientist

occurs twice in the network – you have two independent sets of size
two (A, C and B, D).

Minimum Image Support

The problem of simple and harmful overlap is that they have to solve
the maximum independent set problem for every motif we search in
a possibly very large overlap graph, which is a hard problem. Thus,
researchers proposed a new definition which skips this computation: 35
Björn Bringmann and Siegfried
the minimum image support35 . In this definition, what matters is Nijssen. What is frequent in a single
that we do not re-use the same node in the network to play the same graph? In Pacific-Asia Conference on
Knowledge Discovery and Data Mining,
role in the motif. pages 858–863. Springer, 2008

A B C D Count Figure 39.11: (a) The motif (b)

The image table for the mini-
8 1 1 9 3 mum image support definition,
with the motif’s nodes as rows

5 3 3 7 3 and all the occurrences of the

motif as columns. Each cell
records the node id we use for
2 6 6 4 3 the mapping.

1 8 9 1 3
(a)
(b)

In practice, we look at which node in the network we use to map

each node in the motif. To do so, we build an “image” table. Figure
39.11 shows the image table for the example in Figure 39.10. In the
table, we record the node in the network playing the role of a specific
node in the motif. Thus, the support of the motif is the minimum
number of distinct row values – two identical values in a row stand
for an incompatible pair of occurrences.
The aforementioned Moss method is able to deal with multigraphs,
thus it can be used for some multilayer graph mining as well – as-
suming your multilayer network can be represented as a multigraph. 36
Manlio De Domenico, Mason A
Otherwise, Muxviz36 allows for multilayer motif counting, but em- Porter, and Alex Arenas. Muxviz: a tool
for multilayer analysis and visualization
ploys a naive support definition and thus cannot be used for graph
of networks. Journal of Complex Networks,
mining, due to the break of the anti-monotonicity requirement. 3(2):159–176, 2015c

39.5 Summary

1. Network motifs are small simple graphs that you can use to
describe the topology of a larger network. For instance, you can
count the number of times a triangle or a square appears in your
 frequent subgraph mining 547

network.

2. To do so, you need to solve the problem of “graph isomorphism”,

which is the task of determining whether two graphs have the
exact same topology. Graph isomorphism is a computationally
heavy problem to solve.

3. Frequent subgraph mining is the graph equivalent of frequent

itemset mining: to efficiently find all the graph motifs that appear
in your network, avoiding to perform the expensive graph isomor-
phism problem for patterns that you already discovered not being
frequent.

4. Frequent subgraph mining comes in two flavors. Transactional

mining analyzes many small networks and counts the number
of networks containing the motif we’re counting. Single graph
mining analyzes a single large graph and counts the number of
times the motif appears.

5. Unfortunately, simply counting motif appearances in a single

graph cannot support an efficient exploration of the search space,
because larger motifs might appear more often than smaller motifs,
i.e. the counting function is not-monotonic.

6. We have different ways of counting the frequency of a motif in a

single graph that are anti-monotonic. They are all based on the
concept that we should not count twice patterns that are overlap-
ping, for different definitions of what “overlapping” means.

39.6 Exercises

1. Test whether the motifs in [Link]

39/1/[Link], [Link]
1/[Link], [Link]
[Link], and [Link]
[Link] appear in the network at [Link]
eu/exercises/39/1/[Link].

2. How many times do the motifs from the previous question ap-
pear in the network? [Link]
39/1/[Link] is included in [Link]
exercises/39/1/[Link]: is the latter less frequent the former
as we would require in an anti-monotonic counting function?

3. Suppose you define a new type of clustering coefficient that is

closing [Link]
txt with [Link]
548 the atlas for the aspiring network scientist

txt. What would be the value of this special clustering coefficient

in the network?
 Part XI

Network Distances
40
Node Vector Distance

In Chapter 10 we saw a way to determine the distance between two

nodes: the number of edges you need to cross in the graph to go
from one to the other. Alternatively, one could use the hitting time
(Section 8.3): how long it will take for a random walk to hit both
nodes. However, sometimes you don’t want the distance between
a pair of nodes. Sometimes you want to ask: what is the distance
between a group of nodes and another, given that some nodes might
weigh more than others?

Figure 40.1: A graph with dif-

ferent highlighted groups of
nodes. The intensity of the
color is proportional to how
much weight is on the node.

Figure 40.1 provides an intuitive example to understand this

question. Is the total red hue distributed closer to the blue color, or
to the green? How much does it matter that the darker nodes carry 1
Shmuel Peleg, Michael Werman, and
more weight in estimating such distance? Hillel Rom. A unified approach to
We call this problem the Node Vector Distance, and it has many the change of resolution: Space and
gray-level. IEEE Transactions on Pattern
applications: Analysis and Machine Intelligence, 11(7):
739–742, 1989
• In computer vision1 , 2 , we represent an image as a graph of points 2
Yossi Rubner, Carlo Tomasi, and
of interest, with different values, proportional to how much light Leonidas J Guibas. The earth mover’s
distance as a metric for image retrieval.
or color is in them. Two images can then be compared by esti- International journal of computer vision, 40
mating how much “light” we have to transport from the interest (2):99–121, 2000
 node vector distance 551

points of one to the interest points of the other. Small amounts will
indicate that the images are similar.
3
Ricardo Hausmann, César A Hidalgo,
• In economics3 , 4 , you can represent products as nodes, connected if Sebastián Bustos, Michele Coscia,
there is a significant number of countries that are able to co-export Alexander Simoes, and Muhammed A
Yildirim. The atlas of economic complexity:
significant quantities of them. A country occupies the products in
Mapping paths to prosperity. Mit Press,
this network it can export. From one year to another, the country 2014
will change its export basket, by shifting its industries to different 4
César A Hidalgo, Bailey Klinger, A-
L Barabási, and Ricardo Hausmann.
products. How dynamic is the country’s export basket? The product space conditions the
• In epidemics5 , 6 , 7 , a disease occupies the nodes in a social network development of nations. Science, 317
(5837):482–487, 2007
it has infected. Across time, the disease will move from a set 5
Vittoria Colizza, Alain Barrat, Marc
of infected individuals to another. Similarly, in viral marketing, Barthélemy, and Alessandro Vespignani.
product adoption can be modeled as a disease. The role of the airline transporta-
tion network in the prediction and
predictability of global epidemics.
All these cases can be represented by the same problem formula- Proceedings of the National Academy of
tion. You have a network G. Then you have two vectors: an origin Sciences of the United States of America,
103(7):2015–2020, 2006a
vector p and a destination vector q. Both p and q tell you how much 6
Ayalvadi Ganesh, Laurent Massoulié,
value there is in each node. pu tells you how much value there is and Don Towsley. The effect of network
in node u at the origin, and qv tells you how much value there is in topology on the spread of epidemics.
In INFOCOM 2005. 24th Annual Joint
node v at the destination.
Conference of the IEEE Computer and
All you want to do is to define a δ( p, q, G ) function. Given the Communications Societies. Proceedings
graph and the vectors of origin and destination, the function will tell IEEE, volume 2, pages 1455–1466. IEEE,
2005
you how far these vectors are. There are many ways to do so, which 7
Romualdo Pastor-Satorras and
are organized in a survey paper8 , on which this chapter is based. Alessandro Vespignani. Epidemic
Before we jump into the network distances, it is probably wise to dynamics and endemic states in com-
plex networks. Physical Review E, 63(6):
have a refresher on non-network distances, since it will allow us to 066117, 2001a
introduce concepts that will be helpful later. 8
Michele Coscia, Andres Gomez-
Lievano, James McNerney, and Frank
Neffke. The node vector distance
40.1 Non-Network Distances problem in complex networks. ACM
Computing Surveys, 2020
How to estimate node vector distances on networks is a new and
difficult problem. Let’s take it easy and first have a quick refresher
on the many ways we can estimate distances of vectors without a
network. The easiest way to do it is by assuming a vector of numbers
just represents a set of coordinates in space. If you’re on Earth, with
three numbers you can establish your latitude, longitude and altitude.
That is enough to place you on a position in a three dimensional
space. Another person might be at a different latitude, longitude and
altitude than you. What is the distance between you and your friend?
Easy! You throw a straight rope between you and your friend and its
length is the distance between you. This is the Euclidean distance.
In Section 5.2 I did my very best to connect this intuitive idea in
real life with linear algebra operations. The pain you felt back then
should pay off now. To sum up, if p and q are the vectors defining
your two positions in space, the Euclidean distance is (( p − q) T I ( p −
552 the atlas for the aspiring network scientist

p = (4, 5)
Figure 40.2: Euclidean distance
in m = 2 dimensions. We build
the special p − q vector to have,
((p - q)T(p - q))1/2
at its ith entry, the difference
p2 - q 2 = 4
between the ith entries of p and
q.

p1 - q 1 = 3

q = (1, 1)

(p - q) = (p1 - q1, p2 - q2)

q))1/2 , where I is the identity matrix. Figure 40.2 is a reproduction

of Figure 5.9 and should help refreshing your intuition behind the
Euclidean distance.
You can put any matrix in this formulation instead of I, as long
as they are positive semi-definite matrices. Why would you want to
put any other matrix in there? Remember that matrices are nothing
more than spatial transformations: they tell you how to bend and
warp your space. So putting something else than I will make the
( p − q) vector bend in special ways. What are these ways? Well, the
role of I in the Euclidean distance is to tell us that each dimension of
the vector makes the exact same contribution to the distance. So the
vectors (0, 1, 0) and (0, 0, 1) are exactly equidistant from (1, 0, 0).
However, in some cases, we might notice that some dimensions
are correlated: they change together. So, if the vectors differ in a
direction opposite from what we would expect, this change should
count more than the one we would expect. The correlation between
two variables is called covariance, because the variables vary together.
We can store the relations between the dimensions of p and q in their
covariance matrix cov( p, q) and, since we want to count more when
we have a change going in the opposite direction from the expected,
we invert the matrix: (( p − q) T cov−1 ( p, q)( p − q))1/2 . This is the
Mahalanobis distance.

Figure 40.3: An example of

two correlated variables. The
blue concentric circles represent
Euclidean distances centered
on the blue point. The green
concentric ellipses represent
Mahalanobis distances centered
on the green point.
 node vector distance 553

Figure 40.3 shows you an example. The two dimensions of the

scatter plot are clearly correlated. This means that moving in a
direction orthogonal to the correlation line counts for more distance
covered: it is an unexpected move. That is what the Mahalanobis
distance, in green, is capturing. The Euclidean distance is oblivious
to this and, for it, all directions are equally important. Crossing the
same number of lines means covering the same distance: while the
direction you choose in the Euclidean case doesn’t matter, it does in
the Mahalanobis distance.
In the next section we’ll see how we can use a graph’s topology to
weight the dimensions of our vector in such a way that the difference
between p and q is constrained to happen in the space described by
our network. This boils down to find a smart positive semi-definite
matrix to put in the place occupied by I or cov−1 ( p, q).
Are we done with non-network vector distances? Of course not:
we haven’t even started yet. I already mentioned a few other dis-
tances when I talked about network projections in Section 23.2. We
have correlation distances, cosine distances, etc. I need not to go into
details on how each of these measures work. I will only explain the
cosine distance just to make a point: the Euclidean way of estimating
distances is not the only proper way.
What do I mean by this? Consider the rope example I made before.
For some distance measures, the length of the rope between you and
your friend is not the most important thing. You can have two points
requiring a longer rope to connect that could be closer to each other
than points requiring a shorter rope. This is the case of the widely
used cosine distance.

Figure 40.4: The difference be-

tween the Euclidean distance
between two points (in blue)
and their cosine distance (thick
green arc tracing the angle
between the two points).

In cosine distance you look at the angle made by the vectors con-
necting the two points, as Figure 40.4 shows with a thick green line.
The distance between them is one minus the cosine of that angle.
This is useful, because the cosine is 1 for angles of zero degrees and 0
for angles at ninety degrees. Two points on the same straight line will
have a distance of zero, even if they’re infinitely farther apart on such
a line. For instance, the two points at the bottom of Figure 40.4 are at
a considerable Euclidean distance, but practically neighbors when it
554 the atlas for the aspiring network scientist

comes to cosine distance. Sometimes in life it doesn’t matter where

you are, as long as you’re going in the same direction.
The importance of the cosine distance is in reminding us that a
distance measure does not have to necessarily respect the triangle
inequality like the Euclidean distance does. The triangle inequality
says that the distance between a and b is always lower than or equal
to the distance between a and c and b and c. This is how things work
in the real physical world. But it is not the only way things can work.

40.2 Generalized Euclideans

In this section we’re going to generalize the Euclidean distance by

replacing the identity matrix I with some matrix dependent on the
topology of the network we’re working with. In practice, this means
that we are constraining the diffusion process to happen through the
edges of the network. We don’t want the diffusion to jump between
any two pairs of nodes. If the nodes are far apart in the network,
such jump should be considered differently than the one happening
between two nodes that are directly connected. For instance, in
Figure 40.5, I make the case in which the (0, 1, 0) and (0, 0, 1) vectors
should not be considered equidistant from (1, 0, 0), because the
leftmost and rightmost nodes in this network are not connected, and
they are thus farther apart.

(1, 0, 0) Figure 40.5: The graph repre-

sentation of three vectors. I
highlight in red the node corre-
(0, 1, 0) (0, 0, 1) sponding to the entry equal to 1
in the vector.

With its reliance on I, the Euclidean distance considers these cases

as equidistant. So we need to replace I with something else. We’ll
look at three alternatives.

Laplacian
9
Michele Coscia. Generalized euclidean
In the Laplacian version9 , we look at the Laplacian of the adjacency measure to estimate network distances.
matrix. Remember from Section 5.3, that the Laplacian L = D − A, In Proceedings of the International AAAI
Conference on Web and Social Media,
with D being the degree matrix and A the adjacency matrix. Since volume 14, pages 119–129, 2020
the smallest eigenvalue of L is zero, L is positive semi-definite.
We use the graph Laplacian because the Laplace operator de- 10
Lawrence C Evans. Partial differential
scribes mathematically statuses of equilibrium10 . In practice, if you equations and monge-kantorovich
have a liquid on a container making waves – i.e. being out of equi- mass transfer. Current developments in
mathematics, 1997(1):65–126, 1997
librium – the Laplace operator will tell you how the diffusion of the
 node vector distance 555

liquid will behave when transitioning to its equilibrium state, where

there are no more waves.
Just like in the Mahalanobis case, L like cov tells us how close
together nodes are. Thus, we need to invert it. That’s bad news
though, because L is singular and singular matrices – by definition –
cannot be inverted.
Luckily, there is a way to invert singular matrices, if you’re not too
picky about what “inverting” means. We can calculate their Moore-
Penrose pseudoinverse. To get the Moore-Penrose pseudoinverse,
the first step is to perform the singular value decomposition (SVD)
of L. SVD is one of the many ways to perform matrix factorization
(Section 5.4). In SVD, we want to find the elements for which this
equation holds: Q1 ΣQ2T = L. The important part here is Σ, which is a
diagonal matrix containing L’s singular values. We can easily build a
Σ−1 matrix, containing in its diagonal the reciprocals of L’s singular
values. Then Q2 Σ−1 Q1T = L+ is L’s Moore-Penrose pseudoinverse. It
holds that LL+ L = L and that L+ LL+ = L+ .
So, to sum up, the Laplacian’s δ function is:

δ( p, q, G ) = (( p − q) T L+ ( p − q))1/2 ,
with L+ being the pseudoinverse of the graph Laplacian of G.

Markov Chain
Random walks are helpful to estimate node-node distances (Section
11.4). If we are in the situation of Figure 40.6, we could estimate the
distance between the red and blue node by simply asking how long it
will take for a random walker to go from one node to the other. Here,
we generalize this idea to groups of nodes.
In the Markov Chain distance we start from the assumption that,
given a starting point p, by looking at G we can construct an ex-

Figure 40.6: To estimate the

distance between the red and
the blue node, we could release
random walkers (in green) from
them and calculate how much
time it will take for them to
arrive at the other node.
556 the atlas for the aspiring network scientist

pected “next step”, which is E(q) = Ap. This expected behavior

follows a simple random walk (which is a Markov process, see Sec-
tion 2.6). In other words, we expect that q should be the result of a
one step diffusion of p via random walks. For this to be the case, A
needs to be the stochastic adjacency matrix. Here, we also set the
diagonal of the adjacency matrix to be equal to one before we trans-
form it in its stochastic version. This is equivalent to add a self loop
to all nodes in the network: we want to allow the diffusion process to
stand still in the nodes it already occupies.
For each node u in the network, we can calculate the expected
occupancy intensity in the next time step by unrolling the previous
formula: E(qu ) = ∑ Au,v pv . This is helpful, because it allows us
v ∈V
to calculate the standard deviation of this expectation, σu,v , which
we can do by making a few assumptions on the distribution of this 11
Michele Coscia, Andres Gomez-
expectation (which are spelled out in the original paper11 ). For now, Lievano, James McNerney, and Frank
suffice to say that such deviation is σu,v = ( pv Au,v (1 − Au,v )). Neffke. The node vector distance
problem in complex networks. ACM
Since we now have an expectation and a deviation, we can calcu-
Computing Surveys, 2020
late the z-score of the observation, which is a measure of how many
standard deviations your observation is distant from the expectation.
We calculate a z-score for each node in the network and place it in its
corresponding spot in a diagonal matrix:

[ Z ]u,u = ∑ σu,v
2
.
v ∈V

Now, the problem of this formulation is that it’d make this dis-
tance not symmetric, because to build σu,v we only used p as the
origin of the diffusion. So, if σu,v is the deviation of the diffusion
from v to u, we can also calculate a deviation of the diffusion from u
to v: σv,u . This is done as above, switching p’s and q’s places. Then
2 and σ2 .
the u, u entry in Z’s diagonal is the sum of σu,v v,u
Finally we can write our distance as:

δp,q,G = (( p − q) T Z −1 ( p − q))1/2 .

In this distance measure you can tune the propagation duration.

Maybe you don’t want to make one-step random walks by using
simply the stochastic matrix A. Maybe you want to have two steps
random walks. In this case, you would use A2 . To ensure that the
two farther apart nodes can still reach each other, you could consider
to use A` , with ` being the diameter of the network (Section 10.3). If
you were to take A∞ , then you’d be using the stationary distribution
(Section 8.1). In that case, the topology of G doesn’t matter any more:
the only thing that makes two nodes closer or farther is their degree.
 node vector distance 557

Annihilation
Let’s take that last thought a bit further. If you let your p and q vec-
tors to diffuse via random walks for an infinite amount of time, they
will distribute themselves to all nodes of G proportionally to their
degree, because they will both tend to approximate the stationary
distribution. In the wavy water basin I mentioned before, p and q are
simply two different waves conditions, while the stationary distribu-
tion is... well ... the stationary distribution: a waveless basin where
the water is at the same level everywhere. Mathematically, this means
∞ ∞ ∞
that ∑ Ak q and ∑ Ak p are the same thing, or ∑ Ak ( p − q) = 0.
k =0 k =0 k =0
Now, the interesting bit is for which value of k this is true or, put
in another words, how fast will p and q cancel out. If they cancel
each other out quickly, it means that they were already pretty similar
to begin with. In fact, that equation would be true at k = 0 if p = q.
So we’re interested in the speed of that equation. This is given us by
the following formula:
∞
δp,q,G = (( p − q) T ∑ Ak ( p − q))1/2 .
k =0
∞
An efficient way to approximate ∑ Ak is by calculating ( I − ( P −
k =0
P∞ ))−1 .

40.3 Shortest Path Based

The solutions based on shortest paths start from the assumption that
the problem of establishing distances between sets of nodes can be
generalized from solving the problem of finding the distance between
pairs of nodes. This is a well understood and solved problem: using a
12
Edsger W Dijkstra. A note on two
shortest path algorithm – for instance Dijkstra’s12 – one can count the problems in connexion with graphs.
number of edges separating node u to v. Numerische mathematik, 1(1):269–271,
1959

2 Figure 40.7: A network where

8 the red nodes represent the
4 9 origins and the green nodes
6 represent the destinations.
5 7
1
3

If we take Figure 40.7 as an example, we’d start by collecting a

bunch of distances: [2, 3, 3] when starting from node 5 or node 8, and
[1, 2, 2] when starting from node 7. We then want to aggregate these
558 the atlas for the aspiring network scientist

distances with a given strategy, to define several functions solving the

problem.
Since this section deals with shortest paths, a useful convention
is to refer to all possible paths between origins and destinations as
Pp,q . This is to avoid to calculate all shortest paths between all pairs
of nodes in the network, which is computationally expensive and not
necessary, since we don’t need the distances between nodes that have
a zero value in both p and q. A path length | Pu,v | ∈ P is the minimum
number of edges required to cross to move from node u to node v.
There are two subcategories in this group: methods which try to
optimize the paths from p to q, and methods which do not. We start
from the latter.

Non-Optimized
Here we show a set of possible aggregations of shortest path dis-
tances between the nodes in p and q, by taking hierarchical clustering
as an inspiration. There are of course more strategies than the ones
listed here, but I can’t really list them all – and most haven’t really
been researched yet.
When performing hierarchical clustering, there are three common 13
Gabor J Szekely and Maria L Rizzo.
ways to merge clusters according to their distance13 : single, complete, Hierarchical clustering via joint
and average linkage. Single linkage (green in Figure 40.8) means that between-within distances: Extend-
ing ward’s minimum variance method.
the distance between two clusters is the distance between their two
Journal of classification, 22(2):151–183,
closest points. On the other hand, complete linkage (purple in Figure 2005
40.8) considers the distance of the two farthest points as the cluster
distance. In average linkage (orange in Figure 40.8), one calculates
the average distance between all pairs of points in the two clusters as
the distance between the clusters.

Figure 40.8: Different link-

age strategies to estimate the
distances between clouds of
points: single (green), complete
(purple), and average (orange).

Similarly, our aim is to reach the destination from the origin in the
minimum distance possible. In the single linkage strategy, the “cost”
of reaching a destination node is the distance of it from the closest
possible origin node. First, we need to make sure that ∑ p = ∑ q. If
that isn’t the case, we rescale up the vector with the smallest sum so
that this equation is satisfied. For instance, if q had a lower sum, we
transform it: q0 = (∑ p/ ∑ q)q.
 node vector distance 559

Then we start a loop. At each iteration, we want to find the pair

of closest nodes (arg min | Pu,v |) that can exchange the largest possible
u,v
value. How much value can two nodes exchange? A node can only
give what they have, so we will exchange the minimum of the two
values: min( pu , qv ). The contribution of this move to the distance
is | Pu,v | min( pu , qv ): we move min( pu , qv ) across | Pu,v | edges. Once
the value is exchanged, we update pu and qv to reflect the successful
transaction: pu = pu − min( pu , qv ) and qv = qv − min( pu , qv ). Eventu-
ally, all values would have been transferred, because we ensured that
the two vectors sum to the same value, and the iterations will stop.
The complete linkage uses the very same operation: the only
difference is using at each step arg max | Pu,v | instead of arg min | Pu,v |.
u,v u,v
This means that we preferentially exchange value between the node
pairs that are farthest, not closest.
The average linkage is conceptually simpler: it is the weighted
average path distance between all u ∈ p and all v ∈ q:

∑ ∑ pu qv | Pu,v |
∀v∈q ∀u∈ p
δp,q,G = .
∑p
Here it doesn’t matter what we put in the denominator, since we
already ensured that p and q sum to the same value.

9 Figure 40.9: A network where

8 the red nodes represent the
4 2 origins and the green nodes
6 represent the destinations.
5 7
1
3

Looking at Figure 40.9, we can now estimate the different dis-

tances between red and green nodes. In single linkage, we try to find
the shortest path to the closest destination from each origin. Origin
8 goes to destination 9 because they are directly connected, and so
does origin 7 with destination 4. Origin 5 has to take a path of length
3 to reach destination 1: 5 → 7 → 6 → 1. Thus, for single linkage,
δp,q,G = 1 + 1 + 3 = 5. If we normalize the vectors beforehand, each
step counts for 1/3, and thus the distance would be 5/3 = 1.6̄.
The average linkage looks at all nine shortest paths, and calcu-
lates an average. The total length of all shortest paths is 18. It then
normalizes with the total moved weight: ∑ p = 3. Thus the average
linkage estimates the distance as 18/3 = 6. If we normalized the
560 the atlas for the aspiring network scientist

vectors, we would again count each path as contributing one third, i.e.
(18/3)/3 = 2.
In complete linkage, we perform a similar operation as in single
linkage, but looking at the farthest destination for each origin. The
farthest destination is 5 → 1, at three steps; then 8 → 4 and 7 → 9 at 14
Andrew McGregor and Daniel Stubbs.
two steps each. Thus, complete linkage will return 3 + 2 + 2 = 7 as Sketching earth-mover distance on
graph metrics. In Approximation, Random-
distance. If we normalized the vectors, we would again count each ization, and Combinatorial Optimization.
path as contributing one third, i.e. 7/3 = 2.3̄. Algorithms and Techniques, pages 274–
286. Springer, 2013
15
Gaspard Monge. Mémoire sur la
théorie des déblais et des remblais.
Optimized Histoire de l’Académie Royale des Sciences
de Paris, 1781
Here we try to be a bit smarter than the aggregation strategies we 16
Frank L Hitchcock. The distribution
saw so far. In this branch of approaches, we try to optimize this of a product from several sources to
numerous localities. Studies in Applied
aggregation such that the number of edge crossing is minimized. Mathematics, 20(1-4):224–230, 1941
If there are no further constraints in this optimization problem, 17
Ira Assent, Andrea Wenning, and
we are in the realm of the Optimal Transportation Problem (OTP) Thomas Seidl. Approximation tech-
niques for indexing the earth mover’s
on graphs14 . In its original formulation15 , OTP focuses on the dis- distance in multimedia databases. In
tance between two probability distributions without an underlying Data Engineering, 2006. ICDE’06. Proceed-
ings of the 22nd International Conference
network. However, it has been observed how this problem can be on, pages 11–11. IEEE, 2006
applied to transportation through an infrastructure, known as the 18
Matthias Erbar, Martin Rumpf,
multi-commodity network flow16 . Specifically, one has to simply Bernhard Schmitzer, and Stefan Simon.
Computation of optimal transport on
specify how distant two dimensions in the vector are. The distance
discrete metric measure spaces. arXiv
needs to be a metric, and the number of edges in the shortest path preprint arXiv:1707.06859, 2017
between two nodes satisfies the requirement. 19
Montacer Essid and Justin Solomon.
Quadratically-regularized optimal
In its most general form, the assumption is that we have a distri-
transport on graphs. arXiv preprint
bution of weights on the network’s nodes, and we want to estimate arXiv:1704.08200, 2017
the minimal number of edge crossings we have to perform to trans- 20
George Karakostas. Faster ap-
proximation schemes for fractional
form the origin distribution into the destination one. This is a high
multicommodity flow problems. ACM
complexity problem, which has lead to an extensive search for effi- Transactions on Algorithms (TALG), 4(1):
cient approximations17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 . For what concerns us, all 13, 2008
21
Jan Maas. Gradient flows of the
these methods are equivalent: they all solve OTP and the difference entropy for finite markov chains. Journal
between them is how they perform the expensive optimization step. of Functional Analysis, 261(8):2250–2292,
Thus, they all return a very similar distance given p, q and G – plus 2011
22
Ofir Pele and Michael Werman.
or minus some approximation due to their optimization strategy –, A linear time histogram metric for
and fall in the same category. improved sift matching. In European
conference on computer vision, pages
More formally, in OTP we want to find a set of movements M such
495–508. Springer, 2008
that: 23
Ofir Pele and Michael Werman. Fast
and robust earth mover’s distances.
In Computer vision, 2009 IEEE 12th
M = arg min ∑ ∑ m pu ,qv du,v , international conference on, pages 460–467.
m pu ,qv pu qv IEEE, 2009
24
Justin Solomon, Raif Rustamov,
where pu and qv are the weighted entries of p and q, respectively; Leonidas Guibas, and Adrian Butscher.
Continuous-flow graph transportation
m pu ,qv is the amount of weights from pu that we transport into qv ; distances. arXiv preprint arXiv:1603.06927,
and d pu ,qv is the distance between them. Then: 2016
 node vector distance 561

∑ ∑ m pu ,qv du,v
pu qv
δp,q,G = ,
∑ ∑ m pu ,qv
pu qv

where the m pu ,qv movements come from the M we found at the

previous step. The differences between the methods cited before
almost exclusively lie in the strategy to find the optimal M. The
thing left to determine in the δ formula is the distance function
du,v between pairs of nodes. As mentioned previously, we choose
this to be the length of the shortest path in G between u and v, or:
du,v = | Pu,v |. This is zero if u = v.
There could be additional constraints to this optimized many-to-
many distance. For instance, we could have the constraint that, while
we are moving something from node u to node v via the edge that
connects them, nothing else can pass through that edge. In practice, 25
Oded Goldreich. Finding the
we are simulating an actual physical transportation system, in which shortest move-sequence in the graph-
generalized 15-puzzle is np-hard.,
edges and nodes have capacities. If we want to move two values at
2011
the same time through the same edge, we need to re-route one of
them, because the edge – or the node – is occupied. How to find the 26
Jingjin Yu and Daniela Rus. Pebble
motion on graphs with rotations:
optimal way to solve this problem is the realm of Multi-Agent Path Efficient feasibility tests and planning
Finding (MAPF)25 , 26 . algorithms. In Algorithmic Foundations
of Robotics XI, pages 729–746. Springer,
2015
8
4
Figure 40.10: Attempting to
2
find a MAPF solution from
6 red nodes to green nodes. Blue
5 X 7 arrows show two attempted
1
3 moves that cannot be executed
at the same time.

Figure 40.10 shows an example of this problem: we want to go

from node 8 to node 2 and from node 5 to node 1. Unfortunately, the
shortest paths for these two objectives both involve passing through
node 7 at the same time. This cannot happen, since node 7 can host
only a single walker at a time. Thus, either the walker in 8 or the
walker in 5 need to wait for a bit until node 7 is clear again.
In general MAPF, you have multiple robots occupying one node at
a time and they each have a specific node as their intended destina- 27
Klaus-Tycho Foerster, Linus Groner,
tion27 . This is slightly different from our problem, where each weight Torsten Hoefler, Michael Koenig, Sascha
in p can potentially reach any other destination in q. So one has to Schmid, and Roger Wattenhofer. Multi-
agent pathfinding with n agents on
determine, before running MAPF, which u ∈ p should go to which graphs with n vertices: Combinatorial
v ∈ q. One solution is to simply use the same strategy we used for classification and tight algorithmic
bounds. In International Conference
single linkage in the non-optimized shortest path category: we look
on Algorithms and Complexity, pages
for the shortest path length | Pu,v | carrying the largest possible weight 247–259. Springer, 2017
min( pu , qv ).
562 the atlas for the aspiring network scientist

28
Julio E Godoy, Ioannis Karamouzas,
Stephen J Guy, and Maria Gini. Adap-
Moreover, since in MAPF robots cannot be in the same node at
tive learning for multi-agent navigation.
the same time, you still have a problem. Say that we assigned a In Int Conf on Autonomous Agents and
robot to go from u to v in our preprocessing. If pu 6= qv , then either Multiagent Systems, pages 1577–1585. In-
ternational Foundation for Autonomous
u or v has some unallocated weight. Thus we would need to add Agents and Multiagent Systems, 2015
at least a second robot that can either start in u or terminate in v. 29
Jamie Snape, Jur Van Den Berg,
But this violates MAPF. The way we solve the issue is by running Stephen J Guy, and Dinesh Manocha.
The hybrid reciprocal velocity obstacle.
a sequence of MAPF sessions. In each session, we attempt to move IEEE Transactions on Robotics, 27(4):
all the weights that were left over during the previous session. We 696–706, 2011
keep running smaller and smaller sessions until all weights have
30
Glenn Wagner and Howie Choset. Sub-
dimensional expansion for multirobot
been allocated – which we can guarantee by normalizing either p or path planning. Artificial Intelligence, 219:
q so that they sum to the same value, as we did in the non-optimized 1–24, 2015
solutions.
31
Andrew Dobson, Kiril Solovey, Rahul
Shome, Dan Halperin, and Kostas E
There are many algorithms to solve MAPF28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , Bekris. Scalable asymptotically-optimal
each of them providing a different solution to NVD with our prepro- multi-robot motion planning. In 2017
International Symposium on Multi-Robot
cessing strategy.
and Multi-Agent Systems (MRS), pages
120–127. IEEE, 2017

8
4
Figure 40.11: Attempting to
2
find a pursue solution from
6 red nodes to green nodes. Blue
5 7 arrows show attempted moves.
1
3 Hang Ma, TK Satish Kumar, and Sven
32

Koenig. Multi-agent path finding with

delay probabilities. In Thirty-First AAAI
Conference on Artificial Intelligence, 2017
Another variant to OTP is pursuit-evasion games. In these games,
33
Thayne T Walker, David M Chan, and
Nathan R Sturtevant. Using hierarchical
we populate a space with a set of robots. Some robots are pursuers constraints to avoid conflicts in multi-
and they aim at capturing the other robots, the evaders. In discrete agent pathfinding. In Int Conf on
Automated Planning and Scheduling, 2017
pursuit-evasion (DPE) we force the robots to move through the nodes 34
Konstantin Yakovlev and Anton
and edges of a graph, rather than in a Euclidean space39 . Many Andreychuk. Any-angle pathfinding
algorithms have been proposed to model different strategies and for multiple agents based on sipp
algorithm. In Twenty-Seventh International
constraints both on the pursuer and on the evader side.
Conference on Automated Planning and
One can see how it is possible to adapt DPE to solve our distance Scheduling, 2017
problem. First, we set the pursuers as p and the evaders as q. Then 35
Thayne T Walker, Nathan R Sturte-
vant, and Ariel Felner. Extended
we run any DPE solving algorithm. Alternatively, both p and q
increasing cost tree search for non-unit
are sets of pursuers and try to capture each other, with no evasion. cost domains. In IJCAI, pages 534–540,
Figure 40.11 is an example: here nodes 5 and 1 are trying to capture 2018
36
Jiaoyang Li, Pavel Surynek, Ariel
each other. Every time pursuers capture each other, the one carrying Felner, Hang Ma, TK Satish Kumar, and
the min( pu , qv ) weight disappears and the other one carries its own Sven Koenig. Multi-agent path finding
for large agents. In Proceedings of the
weight minus min( pu , qv ). The amount of time/moves it takes for all
AAAI Conference on Artificial Intelligence,
weights to disappear is the distance between p and q. Since p and volume 33, pages 7627–7634, 2019
q sum to the same value – either by normalization or by the usual 37
Anton Andreychuk, Konstantin
expansion strategy –, the system will terminate. Yakovlev, Dor Atzmon, and Roni
Sternr. Multi-agent pathfinding with
There are a number of solutions to DPE, satisfying a vast number continuous time. In IJCAI, volume 19,
of different constraints40 , 41 , 42 , 43 , 44 , 45 . 2019
 node vector distance 563

40.4 Graph Fourier Transform 38

Minghua Liu, Hang Ma, Jiaoyang
Li, and Sven Koenig. Task and path
planning for multi-agent pickup and
There are other ways to solve the node vector distance problem in delivery. In Int Conf on Autonomous
networks that do not start either from shortest paths nor from a Agents and MultiAgent Systems, pages
1152–1160. IFAAMAS, 2019
generalization of Euclidean distances. They are generally linked to 39
Torrence D Parsons. Pursuit-evasion
the spectrum of the graph, since the spectrum can be used to describe in a graph. In Theory and applications of
diffusion processes on the network, and the node vector distance is a graphs, pages 426–441. Springer, 1978
type of diffusion process.
40
Saeed Akhoondian Amiri, Lukasz
Kaiser, Stephan Kreutzer, Roman Rabi-
In the signal processing literature, a common scenario is one novich, and Sebastian Siebertz. Graph
where the analyst has a battery of sensors, whose readings are corre- searching games and width measures
for directed graphs. In LIPIcs-Leibniz
lated with each other. In order to extract the actual signal ŝ from the
International Proceedings in Informatics,
noisy and correlated signal data s, these relationships between sensor volume 30. Schloss Dagstuhl-Leibniz-
outputs have to be taken into account. The relationships can be mod- Zentrum fuer Informatik, 2015
41
Brian Alspach. Searching and
eled with a network G connecting related sensors. Then, the outputs
sweeping graphs: a brief survey. Le
are smoothed using the Graph Fourier Transform ŝ = Φ T s46 , 47 . matematiche, 59(1, 2):5–37, 2006
The Graph Fourier Transform Φ of G is computed in the following 42
Fedor V Fomin and Dimitrios M
Thilikos. An annotated bibliography on
way. First, we take the Laplacian of G, i.e. L = D − E. Then, we
guaranteed graph searching. Theoretical
calculate the eigenvectors of L (φ), whose eigenvalues λ ∈ R satisfy computer science, 399(3):236–245, 2008
0 = λ0 < λ1 ≤ λ2 ≤ · · · ≤ λ|V |−1 (as usual, we assume G is 43
Flaminia L Luccio. Intruder capture
in sierpinski graphs. In FUN, pages
connected). With Φ we refer to the matrix whose columns are the
249–261. Springer, 2007
eigenvectors, in increasing order of their corresponding eigenvalues: 44
Victor Gabriel Lopez Mejia, Frank L
Φ = (l0 , l1 , . . . , ln ). We call Φ the “spectrum” of E, as this procedure Lewis, Yan Wan, Edgar N Sanchez,
and Lingling Fan. Solutions for
can be seen as a graph spectroscopy.
multiagent pursuit-evasion games on
Suppose that our signal is p, then the corrected signal is equal to: communication graphs: Finite-time
pb = Φ T p. With this transformation, pb tells us how much each of the capture and asymptotic behaviors. IEEE
Transactions on Automatic Control, 2019
eigenvectors contributes to p. In other words, we are changing our 45
Nicholas M Stiffler and Jason M
representation from the “spatial nodes” (p) to the “frequency modes” O’Kane. Pursuit-evasion with fixed
(bp). We can now weight the modes so that we take into account the beams. In 2016 IEEE International
Conference on Robotics and Automation
topology of the graph. This is usually achieved by filtering the signal (ICRA), pages 4251–4258. IEEE, 2016
in the spectral domain, multiplying it with the diagonal matrix of the 46
David K Hammond, Pierre Van-
Laplacian’s eigenvectors Λ. This is the Laplace operator: dergheynst, and Rémi Gribonval.
Wavelets on graphs via spectral graph
  theory. Applied and Computational
λ0 ... 0 Harmonic Analysis, 30(2):129–150, 2011
Λ=
 ..  47
David I Shuman, Benjamin Ricaud,
0 . 0

and Pierre Vandergheynst. Vertex-
0 ... λn frequency analysis on graphs. Applied
and Computational Harmonic Analysis, 40
Once we apply this transformation to both p and q, we have (2):260–291, 2016
encoded G’s topology in the vectors. The Euclidean distance between
them is the node vector distance that we are looking for:

δp,q,G = Euclidean( pΛΦ T , qΛΦ T ).

Note that this is not one, but a family of measures. One could
replace the Euclidean distance with any other off-the-shelf measure
(cosine, correlation, etc) to estimate the distance between the filtered
564 the atlas for the aspiring network scientist

p and q, because they already contain G’s topology in their values.

These approaches can be used to establish the distance between
two different signals on a graph. However, this is but one of the ap-
plications of graph signal processing. Other scenarios include signal 48
Patric Hagmann, Leila Cammoun,
cleaning48 , frequency analysis49 , sampling50 , interpolation51 , and Xavier Gigandet, Reto Meuli, Christo-
trend filtering52 , to cite a few. This also means that the transforma- pher J Honey, Van J Wedeen, and Olaf
Sporns. Mapping the structural core of
tion proposed here might not be the optimal one, and it is for sure
human cerebral cortex. PLoS biology, 6(7):
not the only one. e159, 2008
49
Aliaksei Sandryhaila and Jose MF
Moura. Discrete signal processing on
40.5 Summary graphs: Frequency analysis. IEEE Trans.
Signal Processing, 62(12):3042–3054, 2014
50
Aamir Anis, Akshay Gadde, and
1. The node vector distance problem is the quest for finding a way
Antonio Ortega. Towards a sampling
to estimate a network distance between two vectors describing theorem for signals on arbitrary graphs.
the degree of occupancy of the nodes in the network. If at time t In Acoustics, Speech and Signal Processing
(ICASSP), 2014 IEEE International
I occupy nodes 1, 2, and at time t + 1 I occupy nodes 3, 4, 5, how Conference on, pages 3864–3868. IEEE,
much did I move in the network? 2014
51
Sunil K Narang, Akshay Gadde,
2. The Euclidean distance can be used to estimate distances between Eduard Sanou, and Antonio Ortega.
Localized iterative methods for interpo-
vectors in a homogeneous space. Thus, a family of solution focuses lation in graph structured data. In Global
on “warping” the space so that it is described by the topology of Conference on Signal and Information
Processing (GlobalSIP), 2013 IEEE, pages
the network. At that point, you can use the Euclidean distance on
491–494. IEEE, 2013
such a warped space. 52
Yu-Xiang Wang, James Sharpnack,
Alex Smola, and Ryan J Tibshirani.
3. Another family of solutions uses shortest paths: you calculate all Trend filtering on graphs. Journal of
Machine Learning Research, 17(105):1–41,
shortest paths between all nodes of origin and destination, and
2016b
you aggregate the results somehow. Alternatively, you can try to
find only those shortest paths minimizing the resulting distance.

4. You can add several constraints to the optimized shortest path

strategy. For instance, you could model a real infrastructure net-
work: nodes and edges have finite capacity.

5. Finally, you can use signal cleaning techniques. You can see
your network as describing sets of sensors that return correlated
results. Thus, two “signals” are far apart if they are reported by
uncorrelated sensors, which are not connected to each other.

40.6 Exercises

1. Calculate the distance between the node vectors in [Link]

[Link]/exercises/40/1/[Link] and http://
[Link]/exercises/40/1/[Link] over the
network in [Link]
txt, using the Laplacian approach. The vector files have two
columns: the first column is the id of the node, the second column
 node vector distance 565

is the corresponding value in the vector. Normalize the vectors so

that they both sum to one.

2. Calculate the distance using the same data as the previous ques-
tion, this time with the average linkage shortest path approach.
Normalize the vectors so that they both sum to one.

3. Calculate the distance using the same vectors as the previous ques-
tions, this time on the [Link]
40/3/[Link] network, with both the average linkage shortest
path and the Laplacian approaches. Are these vectors closer or
farther in this network than in the previous one?
41
Topological Distances

In the previous chapter we learned how to estimate the distance

between two vectors describing the occupancy of sets of nodes in the
same network. In that problem, you get two vectors with |V | entries,
and you calculate their distance on the same network topology. We
called this “node vector distance”, a certain type of “network dis-
tance”. There are other ways one could interpret the term “network
distance”. I group them all in this chapter.
Specifically I talk about:

• Network similarity (Section 41.1): how to tell if two graphs G1 and

G2 have a similar topology;

• Network alignment (Section 41.2): finding nodes in G1 that are

similar to nodes in G2 , so that we could couple them and consider
G1 and G2 as two layers of a multilayer network;

• Network fusion (Section 41.3): given multiple observations of

a network, combine them to create a summary that is the most
similar to all observations.

41.1 Network Similarity

By far, the most common and popular way to intend the term “net-
work distance” is as the opposite of the similarity between two
networks. The term “network similarity” is, unfortunately, rather
ambiguous, and you might find papers dealing with very different
problems but using the same terminology. For instance, one could
intend “network similarity” as a measure of how similar two nodes
are (see Section 12.2). Or one could be talking about “similarity net-
works”, which are ways to express the similarities between different
entities by connecting the ones that are the most similar to each other
– something you might do via bipartite projections (Chapter 23).
 topological distances 567

Here, we focus on a different problem. The idea here is simple:

we have two networks G1 and G2 and we want to know how similar
the two are. Namely, how easy it is to mistake G1 as G2 by looking
at their edges. There are many ways to do this, and I’ll try to give a
general overview.
Most of the applications of these techniques are in biology and
chemistry. The idea is to compare networks describing specific
pathways. However, there are also more peculiar applications, for 1
Neha Runwal, Richard M Low, and
instance in malware detection1 and image recognition2 . I am going Mark Stamp. Opcode graph similarity
to include the approaches used mostly for practical problems in and metamorphic detection. Journal in
computer virology, 8(1-2):37–52, 2012
computer science. However, there are many more distance measures 2
Sébastien Sorlin and Christine Solnon.
that have a more distinctively “mathy” flavor. The bible for this kind Reactive tabu search for measuring
of things is for sure the Encyclopedia of Distances3 . Some examples graph similarity. In International
Workshop on Graph-Based Representations
of distance measures you can find there are the Chartrand-Kubicki-
in Pattern Recognition, pages 172–182.
Schultz distance4 , the rectangle distance5 , and many more others. Springer, 2005
At a practical level, all the methods that follow have one thing 3
Michel Marie Deza and Elena Deza. En-
cyclopedia of distances. In Encyclopedia
in common. Comparing two networks using a handful of summary
of distances, pages 1–583. Springer, 2009
statistics means to define a low-dimensional space in which every 4
Gary Chartrand, Grzegorz Kubicki,
network is a point. You can visualize that as a scatter plot: Figure and Michelle Schultz. Graph similarity
and distance in graphs. Aequationes
41.1 makes a super simple one where I decide to classify networks
Mathematicae, 55(1-2):129–145, 1998
by their number of nodes and edge density. However, networks are 5
Christian Borgs, Jennifer Chayes,
a high-dimensional object, and the summary statistics commonly László Lovász, Vera T Sós, Balázs
Szegedy, and Katalin Vesztergombi.
used in network science are usually non orthogonal – differently
Graph limits and parameter testing. In
from what you’d get from, for instance, Principal Component Anal- Proceedings of the thirty-eighth annual
ysis (Section 5.4) –: in my case, from Section 9.1 you know that the ACM symposium on Theory of computing,
pages 261–270, 2006
number of nodes is usually negatively correlated with edge density.

# Nodes
Figure 41.1: On the left we have
four graphs, each identified by
the color of its nodes. On the
right, I make a two dimensional
projection by recording each
graph’s node count (y axis)
Edge Density and edge density (x axis). The
similarity between two graphs
The main issue is that we still don’t know which set of network is the inverse of their distance
statistics is sufficient to cover the space of all possible networks. in this space.
Whatever dimensions you use to organize your networks will col-
lapse many – possibly dissimilar – networks into the same place in
your scatter plot. This happens in Figure 41.1, where a star (in red)
is confused with a set of unconnected cliques (in blue). This is not
necessarily a bad thing! If the summary statistics you chose are mean-
ingful to you in some fundamental way, this is a feature. However, if
you’re hunting for “universal” patterns, this approach could mislead
568 the atlas for the aspiring network scientist

you.

Global Property Comparison

6
Geng Li, Murat Semerci, Bulent
The most basic way to tell whether two networks are similar is by Yener, and Mohammed J Zaki. Graph
looking at their global properties6 . If two networks have the same de- classification via topological and label
gree distribution, the same average path length, the same clustering attributes. In Proceedings of the 9th
international workshop on mining and
coefficient, the same average degree, and so on... Well, doesn’t that learning with graphs (MLG), San Diego,
mean that these two networks are.... the same? USA, volume 2, 2011
This is a seducing option because, as we’ll see, estimating the
similarity between two networks by looking at their topology is
computationally very hard. It is related to the graph isomorphism
problem, and we saw that graph isomorphism is a though nut to
crack in Section 39.2. On the other hand, estimating many global
properties is trivial and instantaneous in many cases, and well stud-
ied and optimized in others.

Figure 41.2: Two graphs of

which we want to estimate the
similarity.

(a) (b)

Of course, you need to be extremely careful in considering two

things. First, what are the global properties you’re looking at? Sec-
ond, how do you aggregate the differences between these properties
to end up with a single measure of similarity? These are important
questions, because you might end up considering as similar two
networks that are very different. Consider Figures 41.2(a) and 41.2(b).
The two networks have a lot in common: same number of nodes and
edges (thus the average degree and density are the same as well).
They have almost identical degree distributions, approximated by a
Gaussian. They have the same diameter and a very similar average
path length (2.1 vs 2.4). Up until now, you’d consider them practi-
cally equivalent. And yet, they’re still relatively different, as they
were generated using two very different processes. Figure 41.2(a) is a
Gn,m random graph, while Figure 41.2(b) is a small-world graph. The
crucial factor I forgot to check is the clustering coefficient, which is
low for Gn,m graphs (0.17 in this instance) and high for small-world
networks (0.41 here).
 topological distances 569

Pairwise Node Similarity

A common approach is the estimation of all possible combinations

of node similarities. This is a relatively popular way to attack the
problem, which underlies many other techniques. The reason is that
it is a natural way to think about network similarity: two networks
are similar if they have the same nodes and these nodes connect to
the same neighbors. Estimating all the pairwise node similarities is
the first step to tell which nodes are the same. More often than not,
that is the end goal of estimating network similarity: we might be
less interested in how similar two networks are and more in which
nodes from one network are the same nodes in the other. This is
the problem of network alignment and we’ll see it more in depth in
Section 41.2.
We have seen dozens of ways to tell how similar two nodes are,
both in Section 12.2 and in Chapter 20. The general idea is to try and
estimate the structural equivalence of all nodes in the two graphs.
Then you can either average all the node-node similarities you cal-
culated, or find the best way to map nodes: for each u1 in G1 you
find the best corresponding u2 in G2 such that, when you mapped all
nodes, the average similarity is maximized.

1 a Figure 41.3: Two graphs of

which we want to estimate the
similarity. Edge color represents
its type.
2 3 b c

(a) (b)

Just to get a better intuition on how this might work, consider

Figure 41.3. We can estimate the networks’ similarity by looking at
the structural equivalence of their nodes. Nodes 1 and a are very
similar: they both have outdegree of two and they point to the same
neighborhood – two nodes connected by a single green edge. The
only difference is in the label of one of their edges. Nodes 2 and c
are also of relatively high similarity, given their equal in- and out-
7
Sergey Melnik, Hector Garcia-Molina,
and Erhard Rahm. Similarity flooding: A
degree with again the sole difference of the edge color. Nodes 3 and versatile graph matching algorithm and
b are, on the other hand, almost structurally identical, with the sole its application to schema matching. In
Proceedings 18th International Conference
difference being not between them, but between their neighbors. We on Data Engineering, pages 117–128.
can conclude, then, that the two graphs are extremely similar, since IEEE, 2002
we just made a node mapping among very similar nodes.
8
Laura A Zager and George C Verghese.
Graph similarity scoring and matching.
This is a simplification of real approaches7 , 8 . The hard part is Applied mathematics letters, 21(1):86–94,
defining an efficient technique to find such mappings. 2008
570 the atlas for the aspiring network scientist

Graph Edit Distance

The strictest possible criterion to establish the similarity between
two networks is by solving the graph isomorphism problem. If two
graphs are literally the same, their similarity is equal to one. Of
course, the graph isomorphism test is binary, thus it is too strict.
A single edge difference would net you a zero similarity. We can
transform this test into something more useful by counting the
number of edge differences between the two graphs. This is akin to
define a “graph edit distance”.
The edit distance between objects a and b is an estimation of
the number of edits you need to make on a in order to transform
it into b. Perhaps the most known and used edit distance is the
string edit distance, of which the most famous is the Levenshtein 9
Vladimir I Levenshtein. Binary
distance9 : this tells you how far apart two strings are. Variants of it codes capable of correcting deletions,
are widely used, for instance, by search engines and word processors: insertions, and reversals. In Soviet physics
doklady, volume 10, pages 707–710, 1966
when you mistype a word, the software will look up what are the
properly spelled words that are at the smallest edit distance from
what you typed, and it will suggest them to you. This works well
because, usually, you won’t make more than one or two mistakes
in typing something – unless you’re me and you’re trying to retype
“Levenshtein” from memory. 10
Xinbo Gao, Bing Xiao, Dacheng Tao,
String and graph edit distances work with the same principles10 , 11 . and Xuelong Li. A survey of graph edit
In strings you are allowed to perform three operations: character distance. Pattern Analysis and applications,
13(1):113–129, 2010
insertion, deletion, and replacement. In graphs you have the same 11
Kaspar Riesen and Horst Bunke.
three operations, but you can apply them to either nodes or edges, Approximate graph edit distance
for a total of six operations. Your nodes and edges might have labels, computation by means of bipartite
graph matching. Image and Vision
so you want to be able to flip the label values as well. computing, 27(7):950–959, 2009

6 6 Figure 41.4: Two graphs of

2
1 which we want to estimate the
4
similarity. Node and edge color
4
3 3
represents their type.
2
1

5
5
(a)
(b)
Figure 41.4 can help you to visualize the process. Here, we want to
know how many operations we need to go from the graph in Figure
41.4(a) to the graph in Figure 41.4(b). Starting from node 1, we need
to change its label (from red to blue) and to add the edge connecting
it to node 6. Node 2 is fine, but node 3 needs to replace its edge to
node 5 with one labeled in green. There are no more edits we need to
 topological distances 571

do, so the distance between the two graphs is three.

Of course, the hard part of graph edit distance is finding the
minimum set of edits, so there are a bunch of ways to go about it,
ranging from Expectation Maximization to Self-Organizing Maps, 12
Horst Bunke. On a relation between
to subgraph isomorphism12 . Special network types deserve special graph edit distance and maximum
approaches, for instance in the case of Bayesian networks13 (see common subgraph. Pattern Recognition
Letters, 18(8):689–694, 1997
Section 4.6) and trees14 , 15 . 13
Richard Myers, RC Wison, and
The prototypical graph edit distance metric16 is relatively sim- Edwin R Hancock. Bayesian graph edit
ple to understand. It is based on the maximum common subgraph. distance. IEEE Transactions on Pattern
Analysis and Machine Intelligence, 22(6):
Given two graphs G1 and G2 , first you find the largest common sub- 628–635, 2000
graph Gs : the largest collection of nodes and edges that is isomorphic 14
Davi de Castro Reis, Paulo Braz
in both graphs. Then, the distance between G1 and G2 is simply the Golgher, Altigran Soares Silva, and
AlbertoF Laender. Automatic web
number of nodes and edges that remain outside Gs : news extraction using tree edit distance.
In Proceedings of the 13th international
δG1 ,G2 = | E1 − Es | + | E2 − Es | + ||V1 | − |V2 ||, conference on World Wide Web, pages
502–511, 2004
with Vx and Ex being the set of nodes and edges of graph Gx . 15
Mateusz Pawlik and Nikolaus Aug-
sten. Rted: a robust algorithm for
This is actually a metric, as it respects the triangle inequality. An
the tree edit distance. arXiv preprint
evolution of this approach tries to find the maximum common edge arXiv:1201.0230, 2011
subgraph17 , which is found in the line graph representation of G. 16
Vladimír Baláž, Jaroslav Koča,
Vladimír Kvasnička, and Milan Sekan-
The problem gets significantly easier when the networks are ina. A metric for graphs. Časopis pro
aligned. Two networks are aligned if we have a known node corre- pěstování matematiky, 111(4):431–433,
spondence between the two. This means that we know that node u 1986
17
John W Raymond, Eleanor J Gardiner,
in one network is the same as node v in the other. How to align two and Peter Willett. Rascal: Calculation
networks is an interesting problem in and of itself, and we’re going of graph similarity using maximum
to look at it in Section 41.2. For now, we just take for granted that the common edge subgraphs. The Computer
Journal, 45(6):631–644, 2002
two networks we’re comparing are already aligned.

Figure 41.5: Three graphs of

which we want to estimate the
similarity. Note how (b) misses
the edge connecting the cliques,
and (c) misses an edge inside
the top clique.

(a) (b) (c)

In this case, we don’t have to go looking for maximum subgraphs.
We can just iterate over all the nodes and edges in the two networks
and note down every time we find an inconsistency: a node or an
edge that is present in one network and absent in the other. Simply
counting won’t do much good, though, because some differences
should count more than others, if they are significantly affecting
the local or global properties of the network. Consider Figure 41.5:
both Figure 41.5(b) and Figure 41.5(c) are just one edge away from
572 the atlas for the aspiring network scientist

Figure 41.5(a). However, since Figure 41.5(b) breaks down in multiple

connected components, its difference should be counted as higher.
There are a few strategies to estimate these differences. Delta- 18
Danai Koutra, Joshua T Vogelstein,
con is based on some sort of node affinity estimation18 . One could and Christos Faloutsos. Deltacon: A
also do vertex rank comparison19 : if the most important nodes in principled massive-graph similarity
function. In Proceedings of the 2013 SIAM
two networks are the same, then the networks must be similar, to
International Conference on Data Mining,
some extent. The same authors propose other ways to estimate net- pages 162–170. SIAM, 2013
work similarity, for instance via shingling: reducing the networks 19
Panagiotis Papadimitriou, Ali Dasdan,
and Hector Garcia-Molina. Web graph
to sequences and then applying a sequence comparing algorithm.
similarity for anomaly detection. Journal
These latter approaches are specialized to find changes in the same of Internet Services and Applications, 1(1):
time-evolving network. 19–30, 2010

The big caveat for using graph edit distances is that they only
work for specific data generating processes. For instance, remember
the Gn,p uniform random graphs from Chapter 13? Two Gn,p graphs
with the same n and (low) p are similar, in the sense that they are re-
alizations of the same process. However, since edges are independent
and the graphs are sparse, they will have almost no edge in com-
mon. As a consequence, their edit distance is large! So what you’re
looking for when using edit distances is for a generating process that
has strong dependencies between edges: the fact that two nodes are
connected implies the presence/absence of other edges in their neigh-
borhood. You should discard graph edit distance measures as soon as
you think that the edges inside your networks are independent from
each other.

Substructure Comparison
20
Xifeng Yan, Philip S Yu, and Jiawei
Substructure comparison20 , 21 is similar to graph edit distance. In Han. Substructure similarity search
this class of methods, you describe the network as a dictionary of in graph databases. In Proceedings of
the 2005 ACM SIGMOD international
motifs and how they connect to each other. Usually, you’d find
conference on Management of data, pages
the motifs by applying frequent subgraph mining (Chapter 39). In 766–777, 2005
practice, graph edit distance is equivalent to a simple substructure 21
Haichuan Shang, Xuemin Lin, Ying
Zhang, Jeffrey Xu Yu, and Wei Wang.
comparison, where the only substructure you’re focusing on is the
Connected substructure similarity
edge. There is not much to say about this class, given its similarity search. In Proceedings of the 2010 ACM
with the previous one: the same considerations and warnings that SIGMOD International Conference on
Management of data, pages 903–914, 2010
applied there also apply here. 22
Thomas R Hagadone. Molecular
When it comes to applications of substructure similarity, the clas- substructure similarity searching:
sical scenario is estimating compound similarities at the molecular efficient retrieval in two-dimensional
structure databases. Journal of chemical
level in a biological database22 . But there are more fun scenarios, information and computer sciences, 32(5):
such as an analysis of Chinese recipes23 . 515–521, 1992
23
Liping Wang, Qing Li, Na Li, Guozhu
Dong, and Yu Yang. Substructure
Holistic Approaches similarity measurement in chinese
recipes. In Proceedings of the 17th
In the holistic category I group a series of approaches that are a international conference on World Wide
mixture of the four previous strategies. Meaning that they use parts Web, pages 979–988, 2008a
 topological distances 573

of all global properties, node similarities, and edit distance, to build

a more general similarity measure for networks. The idea is to build
a “signature vector”: a numerical vector that describes the relevant
aspects of the topology of G. Then, the similarity between two graphs
is simply the similarity between their signature vectors. In practice,
one could think this class to include a sort of “graph embeddings”
(Chapter 37).

(4, 0.5, ...)

Degree Average = 4
(4, 0.5, ...) Degree Median = 4
Figure 41.6: The workflow
Degree StDev = 0.46
of NetSimile. The rightmost
Degree Skewness = 0
column of number is the begin-
(4, 0.33, ...) ning of the graph’s signature
Degree Kurtosis = 2.98
(4, 0.66, ...) vector.
CluCoef Average = 0.41

CluCoef Median = 0.5

(4, 0.66, ...)
CluCoef StDev = 0.19

... ...
24
Michele Berlingerio, Danai Koutra,
Tina Eliassi-Rad, and Christos Faloutsos.
Netsimile: A scalable approach to size-
NetSimile24 is one of the many algorithms in this class. I represent independent network similarity. arXiv
preprint arXiv:1209.2684, 2012
its workflow in Figure 41.6. First, NetSimile calculates seven features 25
Godfrey N Lance and William T
for each node of the graph: degree, local clustering coefficient, aver- Williams. Computer programs for
age neighbor degree, average neighbor local clustering coefficient, hierarchical polythetic classification
(“similarity analyses”). The Computer
number of edges among neighbors, etc. Then, these features are Journal, 9(1):60–64, 1966
aggregated across nodes, i.e. NetSimile calculates their summary 26
Xiaohong Wang, Jun Huan, Aaron
statistics like average, standard deviation, etc. This is the signature Smalter, and Gerald H Lushington.
G-hash: towards fast kernel-based simi-
vector of the graph, which can now be used to compare G with any larity search in large graph databases. In
other graph. Any distance measure discussed so far in the book – Graph Data Management: Techniques and
Applications, pages 176–213. IGI Global,
cosine, Euclidean, ... – can be used to perform the comparison. The
2012c
authors focus specifically on the Camberra distance25 . 27
Michele Berlingerio, Danai Koutra,
Similar approaches are graph hashes26 , designed for optimizing Tina Eliassi-Rad, and Christos Faloutsos.
Network similarity via multiple social
graph similarity searches in a graph database possibly containing theories. In Proceedings of the 2013
thousands of graphs; and approaches that are more rooted in social IEEE/ACM International Conference on
Advances in Social Networks Analysis and
theories27 . The latter case is an evolution of NetSimile. Rather than
Mining, pages 1439–1440, 2013b
including a laundry list of all the measures we think we can use to 28
Thomas Gärtner, Peter Flach, and
compare graphs, we pick the ones that are theoretically motivated. Stefan Wrobel. On graph kernels: Hard-
We define which are the criteria of similarity based on different ness results and efficient alternatives.
In Learning theory and kernel machines,
theories, and we discard the rest. The objective is to be able to better pages 129–143. Springer, 2003
interpret the similarity scores. 29
SVN Vishwanathan, Karsten M
A close cousin of holistic approaches is the one of graph ker- Borgwardt, Nicol N Schraudolph, et al.
Fast computation of graph kernels. In
nels28 , 29 , 30 . Just like in NetSimile and in graph embeddings, a graph NIPS, volume 19, pages 131–138, 2006
kernel is the reduction of a complex high-dimensional graph into 30
U Kang, Hanghang Tong, and Jimeng
a vector of numbers. These vectors are then fed to a machine learn- Sun. Fast random walk graph kernel. In
Proceedings of the 2012 SIAM international
ing algorithm that is able to learn the shape of the space in which conference on data mining, pages 828–838.
these vectors live and thus the similarity between them. Just like with SIAM, 2012
574 the atlas for the aspiring network scientist

many graph embeddings techniques, these kernel are usually created

by means of some sort of random walk process.

Information Theory
A radically different approach works directly with the adjacency
matrix of a graph. The idea here is to generalize the Kullback-Leibler
divergence (KL-divergence) so that it can be applied to determining
the distance between two graphs. The KL-divergence is a cornerstone
of information theory and linked with the concept of information
entropy – see Section 2.8 for a refresher.
The KL-divergence is also known as “relative entropy”. From
Section 2.8, you learned that the information entropy of a vector X is
the number of bits per element you need to encode it. Now, of course
when you try to encode a vector, you try to be as smart as possible.
You create a codebook that is specialized to encode that particular
vector. If there is an element that appears much more often than the
others, you will give it a short code: you will have to use it more
often and, if it is shorter, every time you use it you will save bits. This
is the strategy used by Infomap to solve community discovery – see
Section 31.2.
Now suppose you have another vector, Y. You want to know how
similar Y is to X. One thing you could do is to encode Y using the
code book you optimized to encode X. If X = Y, then the codebook
is as good encoding X as it is encoding Y: you need no extra bits. As
soon as there are differences between X and Y, you will start needing
extra bits to encode Y, because X’s codebook is not perfect for Y any
more. The KL-divergence boils down to the number of extra bits you
need to encode Y using X’s codebook.

X X Y Y
=0 0 =0 0 Figure 41.7: An example of the
9 bits 11 bits
0 10
0
6 values
10
6 values spirit of KL-divergence. The
= 10 10 =9/6 = 10 10 = 11 / 6 code we use for X (a) requires
10 = 1.5 bits/value 10 = 1.83 bits/value additional bits to encode Y (b).
= 11 11 = 11 11

(a) (b)
Figure 41.7 presents a rough outline of the idea behind the KL-
divergence – simplified to help intuition. The X vector in Figure
41.7(a) requires 1.5 bits per element. Using its codebook to encode Y
in Figure 41.7(b) increases the requirement to 11 total bits instead of
the original 9. 31
David J Galas, Gregory Dewey,
In its original formulation, the KL-divergence is defined for pairs James Kunert-Graf, and Nikita A
Sakhanenko. Expansion of the kullback-
of vectors. However, one can expand it to allow it to consider differ- leibler divergence, and a new class of
ent inputs31 . One can say that entries in the vectors are dependent on information metrics. Axioms, 6(2):8, 2017
 topological distances 575

each other. Thus, if you consider a graph as a series of |V | variables,

one per node, you can express the pairwise dependencies as the 32
Christopher L McClendon, Lan Hua,
edges of the graph. This approach has applications in chemistry32 . Gabriela Barreiro, and Matthew P
Another way of comparing networks by means of analyzing their Jacobson. Comparing conformational
ensembles using the kullback–leibler
adjacency matrices comes from comparing the eigenvectors of their
divergence expansion. Journal of chemical
Laplacians33 . Similar networks will experience similar spreading theory and computation, 8(6):2115–2126,
patterns, which are reflected in their spectra. 2012
33
Anirban Banerjee. Structural distance
and evolutionary relationship of
networks. Biosystems, 107(3):186–196,
41.2 Network Alignment 2012

Earlier in the chapter, I mentioned what aligned networks are: two

networks are aligned if we have a node to node mapping, i.e. for each
node in network G1 we have a corresponding node in G2 represent-
ing the same entity. Many networks are naturally aligned. The most
typical case of aligned networks are time-evolving networks. Two
snapshots of a structure are simply two different networks: since we
have the same ids on the nodes, we have the alignment for free. An-
other example could be networks describing brain scans: we divide
the brain in different areas for all individuals, and these areas might
interact differently between individuals. The areas are the nodes,
thus their identities are known, while the interactions are the edges,
which might change. In general, any multilayer network (Section
4.2) could be seen as a collection of aligned networks: each layer is
a network and the inter layer couplings are the mappings from one
layer to another.
However, you might not be as lucky as in the case of evolving
networks: sometimes you have two observations that you think
you should be able to align, but you actually do not have neither
consistent node ids, nor a reliable node mapping. For instance, you
might have collected a bunch of data from different social media.
You know that people have profiles in different platforms, but these
platforms will use different and mutually incompatible identifiers.
Thus, you will need to figure out who is who in all the networks 34
Huynh Thanh Trung, Nguyen Thanh
you collected. This is the network alignment problem34 . A classical Toan, Tong Van Vinh, Hoang Thanh
application of network alignment is the attempt to map the protein- Dat, Duong Chi Thang, Nguyen
Quoc Viet Hung, and Abdul Sattar.
protein interaction of different organisms35 , discovering that many A comparative study on network
biological pathways are preserved across species. alignment techniques. Expert Systems
with Applications, 140:112883, 2020
Figure 41.8 shows an example. Given two graphs as input with V1 35
Brian P Kelley, Roded Sharan,
and V2 as their node sets, you want to produce a |V1 | × |V2 | matrix Richard M Karp, Taylor Sittler, David E
telling you the probability of each node from the first graph to be Root, Brent R Stockwell, and Trey
Ideker. Conserved pathways within
the node in the second graph. The way this matrix is built can rely bacteria and yeast as revealed by global
on any structural similarity measure, we saw a few in different parts protein network alignment. Proceedings
of this book. Then, the idea is to pick the cells in this matrix so that of the National Academy of Sciences, 100
(20):11394–11399, 2003
the sum of the scores is maximized and, at the same time, we match
576 the atlas for the aspiring network scientist

36
Oleksii Kuchaiev and Nataša Pržulj.
as many nodes as possible36 . If |V1 | 6= |V2 | you will have to face a Integrative network alignment reveals
choice: either you do not map some nodes or you allow nodes from large regions of global network similar-
ity in yeast and human. Bioinformatics,
one network to map to multiple nodes in the other. This common ap-
27(10):1390–1396, 2011
proach can be extended, for instance, by calculating multiple versions
of this matrix using different measures and then seeking a consensus
matrix which is a combination of all the similarity measures.

6 e
b Figure 41.8: Two graphs of
1
4 which we want to discover the
c d
3 alignment. (c) assigns to each
2 a
node pair from (a) and (b) an
5 f
alignment probability.
(a) (c)
(b)
One could also find just a few node mappings with extremely 37
Giorgos Kollias, Shahin Mohammadi,
high confidence and then expand from that seed37 , assuming that and Ananth Grama. Network similarity
the neighborhoods around these high confidence nodes should look decomposition (nsd): A fast and
scalable approach to network alignment.
alike. Of course, a large portion of network alignment solutions rely
IEEE Transactions on Knowledge and Data
on solving the maximum common subgraph problem: if you find Engineering, 24(12):2232–2243, 2011
isomorphic subgraphs in both networks, chances are that the nodes
inside these subgraphs are the same, and thus should be aligned 38
Gunnar W Klau. A new graph-based
to each other38 . Other approaches rely on the fact that isomorphic method for pairwise global network
graphs have the same spectrum, thus similar values in the eigenvec- alignment. BMC bioinformatics, 10(1):S59,
2009
tors of the Laplacian imply that the nodes are relatively similar39 . 39
Rob Patro and Carl Kingsford. Global
Another approach uses a dictionary of networks motifs. Each network alignment using multiscale
node is described by counting the number of motifs it is part of. We spectral signatures. Bioinformatics, 28(23):
3105–3114, 2012
can then describe the node as a numerical count vector. Two nodes 40
Tijana Milenković, Weng Leong Ng,
with similar vectors are similar40 . This approach has to solve the Wayne Hayes, and Nataša Pržulj. Opti-
graph isomorphism problem as well, but it needs to do so only for mal network alignment with graphlet
degree vectors. Cancer informatics, 9:
small graph motifs rather than for – supposedly – large common
CIN–S4744, 2010
subgraphs. This way, it can be more efficient.

6 Node Figure 41.9: (a) A graph. (b) A

1
4 node-motif table, counting how
1 2 1 1 0
3 many times each node is part of
2 2 2 1 1 0
a given motif.
3 4 2 1 1
5
... ... ... ... ...
(a)
(b)
Figure 41.9 shows an example: here I choose a relatively small
set of motifs, which generate a short vector. However, one could
define as many motifs as they are relevant for a specific application,
and obtain much more precise vectors describing the nodes. A final
 topological distances 577

approach I mention is MAGNA, which uses a genetic algorithm ap-

proach: it tries aligning by exploring the search space of all possible
node mappings, allowing the best matches to survive and evolve, and 41
Vikram Saraph and Tijana Milenković.
dropping the worst matches41 . Magna: maximizing accuracy in global
network alignment. Bioinformatics, 30(20):
2931–2940, 2014
41.3 Network Fusion

The final problem related to network distance/similarity is network

fusion. Network fusion is a relatively old term and branch of com-
puter science that up until recently had little to do with network
science proper. It was introduced a few decades ago in the field of 42
Lars Kai Hansen and Peter Salamon.
neural networks42 , 43 . The idea was that you trained a neural network Neural network ensembles. IEEE
transactions on pattern analysis and
on some data. The network has grown to be able to capture as much
machine intelligence, 12(10):993–1001,
of the variation as possible. If you use the same algorithm to train 1990
on different data, you might end up with a similar, but not identical, 43
Sung-Bae Cho and Jin H Kim. Mul-
tiple network fusion using fuzzy logic.
configuration in your neural network. Some connections are stronger, IEEE Transactions on Neural Networks, 6
others are weaker. Most of the variation between the same neural (2):497–501, 1995
networks trained on different data is due to overfitting. Thus you
want to build yet another neural topology, smoothing out all the
noise. This is network fusion, because effectively you want to fuse
together all the neural networks that you have trained. This might be 44
Xianzhi Du, Mostafa El-Khamy,
an old idea, but is still an area of active research44 . Jungwon Lee, and Larry Davis. Fused
dnn: A deep neural network fusion
Figure 41.10 is the easiest mental picture you need to understand approach to fast and robust pedestrian
the principle of network fusion. We have two aligned networks in detection. In 2017 IEEE winter conference
on applications of computer vision (WACV),
Figure 41.10(a) and Figure 41.10(b). We decide that we want to fuse
pages 953–961. IEEE, 2017
them together by calculating the average edge weight. We also decide
that we keep a connection in the fused network only if its resulting
average weight is higher than 2. Figure 41.10(c) is the result. Of
course, real network fusion algorithms are much smarter and more
sophisticated than this.

3 2
9
1
1 2 3 3
9 1 9 1 Figure 41.10: An example of
2
3 3
2
3 2 2 4 3 .0
2.0
2 . 5 3. 0
8 2 network fusion: (c) is the re-
4 4 8 4 2 8
2
2
1 1
2 3 2.0
sult of the fusion of (a) and (b).
3
4 7 7
2.0
Edge thickness proportional to
7 2
1
5 5 5 its weight.
2 3 2. 5
3 1 2.0
6 6
6

(b) (c)
(a)
Slowly but surely, network fusion crept into network science and
found applications that go beyond increasing the performance and
applicability of neural networks. For instance, consider genomic data.
You can collect samples of interactions from many individuals. These
578 the atlas for the aspiring network scientist

are similar, but not always the same. You might want to combine 45
Bo Wang, Aziz M Mezlini, Feyyaz
them to create a prototypical interaction network45 . Alternatively, it Demir, Marc Fiume, Zhuowen Tu,
could be that some of these samples are incomplete, and you can use Michael Brudno, Benjamin Haibe-Kains,
and Anna Goldenberg. Similarity
their fusion as the complete genomic data.
network fusion for aggregating data
types on a genomic scale. Nature methods,
11(3):333, 2014a
41.4 Summary

1. In this chapter we explore different ways to estimate the similar-

ity/distance between the topologies of two networks: given two
graphs, quantitatively estimate how much of their topologies are
the same. Related applications are network alignment and fusion.

2. Network similarity can be done in many ways. One could com-

pare the global properties of the network such as degree distribu-
tion and clustering; or aggregate all the node pairwise similarities;
or estimate the number of edits needed to go from one network to
the other.

3. One can combine all those approaches in a holistic one, determin-

ing which elements of the similarities between networks are more
relevant, based on what the networks represent. Additionally, one
could see adjacency matrices as signals and calculate the mutual
information entropy between them.

4. Network alignment is the problem of finding a node-to-node

mapping between two networks. We hypothesize that the two
networks represent the connections between the same real world
entities and we need to re-identify them by looking exclusively at
the network topologies.

5. Network fusion is the process of taking multiple versions of the

same network and reconstructing the underlying structure. The
idea is that each observation might be noisy or incomplete, while
their combination should represent the ideal structure.

41.5 Exercises

1. Estimate the similarity between the networks at [Link]

[Link]/exercises/41/1/[Link], [Link]
eu/exercises/41/1/[Link], and [Link]
eu/exercises/41/1/[Link], by comparing their average degree,
average clustering coefficient, and density (average their absolute
differences). Which pair of networks are more similar to each
other?
 topological distances 579

2. Calculate the structural similarities of all pairs of nodes for all

pairs of networks used in the previous question. Derive a network
similarity value by averaging the node-node similarities. Since
the networks are aligned, the node-node similarity is the Jaccard
coefficient of their neighbor sets, and you should only calculate
them for pairs of nodes with the same id. Which pair of networks
are more similar to each other?

3. Calculate the graph edit distances between the networks used in

the previous questions. Remember that the networks are aligned,
thus you just need to iterate over nodes and compare their neigh-
borhoods. Which pair of networks are more similar to each other?

4. Fuse the three networks together to produce a consensus network.

You can keep an edge in the consensus network only if it appears
in two out of three networks – assume that their are aligned and
that nodes with the same id are the same node.
 Part XII

Visualization
42
Node Visual Attributes

Data visualization is at the core of any data analysis undertaking –

call it statistics, data science, or whatever else. There are two reasons
why: gathering insights in exploratory data analysis, and presenting
your results.
Let’s start from gathering insights. It is sometimes – not always! –
easier to spot patterns when you look at them with a proper visual-
ization, rather than relying on summary statistics. The classical case
1
Francis J Anscombe. Graphs in
for this position is the Anscombe quartet1 . In Figure 42.1(a) you have statistical analysis. The american
four datasets of two variables. statistician, 27(1):17–21, 1973

12 12

10 10
I II III IV
x y x y x y x y 8 8
y

y
10 8.04 10 9.14 10 7.46 8 6.58
I II
6 6
8 6.95 8 8.14 8 6.77 8 5.76
4 4
13 7.58 13 8.74 13 12.74 8 7.71
4 6 8 10 12 14 16 18 4 6 8 10 12 14 16 18
9 8.81 9 8.77 9 7.11 8 8.84 x x
11 8.33 11 9.26 11 7.81 8 8.47
14 9.96 14 8.1 14 8.84 8 7.04 12 12

6 7.24 6 6.13 6 6.08 8 5.25 10 10

4 4.26 4 3.1 4 5.39 19 12.5
8 8
y

12 10.84 12 9.13 12 8.15 8 5.56

III IV
7 4.82 7 7.26 7 6.42 8 7.91 6 6

5 5.68 5 4.74 5 5.73 8 6.89 4 4

4 6 8 10 12 14 16 18 4 6 8 10 12 14 16 18
(a) x x

(b)

Figure 42.1: (a) Four datasets

In all datasets, these variables have the same mean and standard
with x and y coordinates. (b)
deviation, the same correlation and even drawing a regression line
Data visualization of (a) in the
between the two variables leads to the same result. You’d think that
form of a scatter plot.
the four datasets are identical and no clear patterns distinguish them.
However, simply seeing how these datasets look like – as I show in
Figure 42.1(b) with a humble scatter plot – immediately tells you that
there’s something interesting going on.
Then there is result communication. When you write a paper, you
582 the atlas for the aspiring network scientist

must state your results clearly and in an intuitive matter. In many

cases, it is true that showing a picture of them is necessary. Thus, you
need to be proficient in data visualization techniques, so that you
won’t accidentally trick your reader – or you won’t tricked yourself
while reading a paper from a malicious miscommunicator.
The classical building blocks of network visualization are nodes
and edges. Traditionally, we represent nodes as circles or dots, and
edges as lines connecting dots. Then, nodes are scattered around
so that the ones not connected to each other tend to be far apart.
Instead, edges tend to be as short as possible, so connected nodes
appear in close spatial proximity. This is so ingrained in network sci-
ence visualization that you saw me using this approach throughout
most of the examples I presented so far.
There are reasons why rules and best practices exist. They work
in most scenarios. They also build familiarity: if you’re exposed to
the same strategies over and over again, you become literate and
know immediately what’s going on. In the majority of what follows
I will align myself with these conventions. However, the first thing
that needs to be highlighted is something that might appear obvious.
Even if we represent them that way, nodes aren’t dots, and edges
aren’t lines. The dot-line diagram is a map, not the territory. The
reality that lurks behind a graph can take many forms and some will
communicate better your intentions than others.

Figure 42.2: Hello hairball,

my old friend. I’ve come to
talk with you again, because
a vision softly creeping left its
seeds while I was sleeping and
the vision that was planted in
my brain still remains.

In practice, my aim is to give you the best practices and then

empower you to break them when you feel they get in the way
of your network visualization. Our journey is a fight against the
nemesis of every network scientist who dares visualizing her own
networks: the hairball. An hairball, or spaghettigraph, is something
that looks like the example in Figure 42.2.
We already saw how the hairball can get in your way when you’re
analyzing network data in Part VII. Here, we’re trying to defeat it in
 node visual attributes 583

the realm of communicating to others what your network contains.

If we trust the node-link diagram convention too much, we end up
with visualizations that are cluttered like in Figure 42.2 and do not
communicate much besides “it’s complicated”.
So, if there’s something you will take away from this part, it is
how to make hairballs less hairbally. We start in this chapter with
node visual attributes, then we move on to edge visual attributes
(Chapter 43) and network layouts (Chapter 44), with a small carousel
of peculiar examples. 2
Paul Shannon, Andrew Markiel,
My software of choice is usually Cytoscape2 , 3 , which is the one Owen Ozier, Nitin S Baliga, Jonathan T
I’m most proficient with. Most of the examples in this part will be Wang, Daniel Ramage, Nada Amin,
Benno Schwikowski, and Trey Ideker.
based on Cytoscape and can be achieved by using it without any real
Cytoscape: a software environment
programming skill. A popular alternative would be Gephi4 , 5 . for integrated models of biomolecular
Finally, I should say that what follows is all practical knowledge interaction networks. Genome research, 13
(11):2498–2504, 2003
of me messing up with Cytoscape for ten years and learning my- 3
[Link]
self what looks good and what doesn’t. I’m not an expert on data 4
Mathieu Bastian, Sebastien Heymann,
visualization and visual communication in general. If you want a Mathieu Jacomy, et al. Gephi: an open
source software for exploring and
more in-depth dive into proper visualization techniques – which manipulating networks. Icwsm, 8(2009):
go beyond simple network visualization – you’re best served with 361–362, 2009
one of the many awesome books and papers out there6 , 7 , 8 , 9 , 10 . You 5
[Link]
6
Ben Shneiderman. The eyes have
should also consider keeping an eye on some conferences on data
it: A task by data type taxonomy for
visualizations such as IEEE Visualization Conference and the ACM information visualizations. In Proceedings
Computer-Human Interaction conference. 1996 IEEE symposium on visual languages,
pages 336–343. IEEE, 1996
7
Alberto Cairo. The Functional Art: An
introduction to information graphics and
42.1 Size visualization. New Riders, 2012
8
Isabel Meirelles. Design for information:
The first thing you might want to modify about a node is its size. an introduction to the histories, theories,
This should be used for quantitative attributes, measuring some sort and best practices behind effective informa-
tion visualizations. Rockport publishers,
of importance of the node. They can be directly calculated from the 2013
graph properties (such as number of connections, PageRank, etc) or 9
Edward Tufte and P Graves-Morris.
they can be provided as quantitative metadata. For instance, in a The visual display of quantitative
information.; 1983, 2014
network where nodes are traffic junctions, it could be the number of 10
Tamara Munzner. Visualization analysis
cars that can pass through a street crossing per unit of time. It seems and design. AK Peters/CRC Press, 2014
natural to encode the node’s importance directly on its size, as I show
in Figure 42.3.
The reason for using sizes – and other visual features as we will
see – is to facilitate perception of the quantities and hence facilitate
inference and insight. You want to make quantitative distinctions so
that the variables you’re visualizing stand out. If you cannot tell the
difference between two different node sizes because the variations are
too subtle, you’re not communicating anything to your viewer. Thus,
you have to have enough diversity in your visual features: just the
amount that the human eye can perceive. This is one of the reasons
why it is so hard to create visualizations. Finding the right scale to
584 the atlas for the aspiring network scientist

Figure 42.3: The natural scale

with which we can use node
size – meaning: its area – to
confer the idea of its impor-
tance.

Less Important More Important

encode a quantity into a size is hard, especially when you’re dealing

with continuous variables and you have to bin them yourself.
Taking Figure 42.4 as an example, in Figure 42.4(b) you cannot
really tell who is boss by simply looking at the node sizes. As soon as
we exaggerate the size difference – Figure 42.4(c) –, it becomes clearer
and the visualization becomes more informative and, arguably,
visually more pleasing. Differences have to jump to the eye: making
subtle changes is not going to communicate much to the viewer. In
other words, to facilitate the viewer’s perception of the differences
in the quantities mapped, your visualization has to have enough
“action”, differences, it has to say something.

Figure 42.4: (a) A graph in

which all nodes have the same
size. (b) A graph in which the
node’s degree determines its
size, with subtle variations. (c)
Same as (b), but exaggerating
the node size variation.

(a) (b) (c)

You cannot simply take away the message that any quantitative
measure of node importance is an equally good choice for your node
size. Some of those measures will not highlight what you want to
highlight. For instance, the degree is not always the right choice.
Consider Figure 42.5(a): would you think to use the node’s degree
as a measure of its size? If you do, you end up with Figure 42.5(b)
where the node playing arguably the strongest role in keeping the
network together almost disappears. A much better choice, in this
case, is betweenness centrality (Figure 42.5(c)).
It shouldn’t surprise you – after all the network analysis we’ve
 node visual attributes 585

(a) (b) (c)

Figure 42.5: (a) A graph in
done – to hear that many variables of interest in a network have which all nodes have the same
very broad distributions. Degree and betweenness centrality, the two size. (b) A graph in which the
examples cited so far, follow (quasi) power laws, with few gigantic node’s degree determines its
hubs and many nodes with minimum values. This means that linear size. (c) Same as (b), but using
size scales are not going to work very well: everything is going to be betweenness centrality instead
tiny and then BAM! One huge node, the hub. of the degree for the nodes’
Consider Figure 42.6. Here, we use the degree to determine the size.
node size and we use a linear scale. Since this is an example of comic
book characters, we expect the ones appearing with many other
characters in the same comic book to be the most important. And
they are: the largest nodes are the ones you would expect. But... they
are too much the ones you expect. Their size swamps everything else.
As a result, the visualization might be truthful, but it’s not informative.
It doesn’t show you any new information. You already knew all you
can gather from it.

100

10-1

10-2
p(k)

10-3

10-4
100 101 102 103 104
k

(a)

(b)
Figure 42.6: (a) Degree distri-
To counteract this, you need to apply a quasi-logarithmic scaling.
bution of the Marvel social
If you’re creating your visualizations programmatically you can have
network example. (b) Visualiz-
an actual log scale, although you probably will still have to manually
ing the network with a linear
tweak it a bit to make the result more pleasing. The idea is to have
node size map, where the de-
diminishing returns to the contribution of the degree to the node
gree directly determines the
size. The differences in size from the minimum degree, to the average
node size.
586 the atlas for the aspiring network scientist

degree – which can be quite low – are big but, from that point on, the
contribution to the node’s size plateaus.

Figure 42.7: (a) The comic

book social network using de-
gree directly for node size. (b)
Same network using a quasi-
logarithmic scaling for the node
size.

(a) (b)

If you do so, you can find new clusters that were previously clut-
tered by the huge nodes, or that had a low degree and so they did
not pop up. You can compare the two hairballs in Figures 42.7(a) and
42.7(b). Note that the visualization is still truthful: we’re never going
to make nodes with lower degree larger than nodes with higher de-
gree. That would be bad and land you in a corner. We’re just making
the visualization more useful.
This is probably a good place to stop and make a disclaimer. Even
if eyes are the highest bandwidth sensors we have, it doesn’t mean
they are flawless. Nor that our monkey brain is able to use the in-
formation they gather in a perfect way. Human perception is flawed
and you cannot expect that something a computer understands will
appear obvious to your viewers as well. In the case of node size this
takes the form of the confusion between radii and areas.
Unless otherwise specified by the software/program of choice,
you are going to decide the radius of the node when determining its
size. This can be trouble if you don’t handle this choice properly. The
reason is that, when you increase the radius, you are substantially
performing a linear increase: you think that, if the degree increases
by one unit, you should increase the radius by one unit. Unfortu-
nately, what a viewer will perceive is you changing the area of the
circle. The crux of the problem is that a radius is a one dimensional
quantity, and it should never be used for controlling a two dimen-
sional one such as an area – which is what your readers perceive.
You think you’re increasing something linearly, but you’re actually
raising that increase by the power of two.
Figure 42.8 shows you why you need to be well aware of the
difference. What you think is a small increase can seem humongous
to your reader.
 node visual attributes 587

Figure 42.8: A human perceives

a node with radius one as being
of size π, its area. So she will
also perceive a node of radius
two as being of size 4π: double
degree, but four times as large!
Degree = 1 Degree = 2
Radius = 1 Radius = 2
Area = π Area = 4π (!!)

42.2 Color

The second obvious feature to manage for your nodes is their color. If
we routinely use node sizes for quantitative attributes, we primarily
use node color for qualitative ones. The reason is that, while humans
perceive size as quantitative, color hue is not perceived in the same 11
William S Cleveland and Robert
way. Cleveland and McGill11 distinguish between different data types McGill. Graphical perception: Theory,
and how much different graphical features are effective for each data experimentation, and application to the
development of graphical methods. Jour-
type. Color is good for nominal attributes – categories that cannot nal of the American statistical association,
be compared/sorted, like “apple” vs “orange”. Color could be used 79(387):531–554, 1984
for ordinal attributes, that are still categories but can be compared –
for instance days of the week, Monday comes before Tuesday. Color
is terrible for quantitative attributes, for which areas are a more
effective tool. As always, what follows is based on my experience
and, if you want or need more in-depth explanations, you should
check out the paper.
When it comes to network visualization, this implies that we put
nodes into classes and we use colors to emphasize that different
nodes are in different classes. Classical examples can be the node’s
community – Part IX –, or its role – Section 12. I already mentioned
nodes can have metadata, and these metadata could be categorical.
For instance, in a network connecting online shopping products
because they are co-purchased together you could use the color to
determine their category (outdoors, rather than kitchen, rather than
electrical appliances).
You could still use node color for ordinal attributes, and maybe Figure 42.9: (top) A gradient
palette for diverging quantities
and a meaningful middle point
of the spectrum. (bottom) An
intensity gradient, useful to go
from zero to a maximum value
without a meaningful middle
point.
588 the atlas for the aspiring network scientist

for quantities as well, provided that you have clear and intuitive bins.
The way one would use colors for quantities is by implementing a
gradient. A classical one is a blue-red spectrum for temperatures:
this is a diverging scale that can be useful, e.g., if you have some
sort of correlation data. You have a very precise and semantically
meaningful middle point, and nodes can diverge in either of two
directions, as I show in Figure 42.9 (top). Otherwise, if we’re talking
of a more classical intensity – say how much money a customer spent
in your online shop – you want a simple sequential gradient, just like
the one in Figure 42.9 (bottom).
There are many things you need to take into account when using 12
Samuel Silva, Beatriz Sousa Santos,
colors. One of the trickiest ones is cultural associations12 . When you and Joaquim Madeira. Using color in
visualize something, your visualizations come after centuries – if not visualization: A survey. Computers &
Graphics, 35(2):320–333, 2011
millennia – of other people using colors for different tasks. These
usages ingrained in our mind a quick way to decode information.
For instance, we associate red with danger, yellow with caution,
green with “good to go”. Black is death, and – stereotypical – blue
is for boys and pink is for girls. But blue is also Democrat against
red Republican if we’re talking about elections in the US – which,
interestingly, is the opposite of the left-right wing spectrum for
other countries in which red is communism. It all depends on the
context in which you’re visualizing. Color can aid you in making
your visualization quicker to decode, but if you’re instead using it
differently from a convention it can make things harder.
This doesn’t even take into consideration the deficits in the phys-
ical perception by humans. Just to repeat myself – we are very lim-
ited when it comes to distinguish colors. If you ask your laptop
how many colors there are out there, a popular reaction would be
counting the number of possible RGB combinations and to reply: 16
millions! That would be very wrong for any human with a hint of
common sense. In fact, what I would say is that you should never
use more than nine colors in your visualization, and I’m sure that a
few of my data designer friends are already gasping in horror to the
extent of my liberalism. Nine, for them, is already way too much.
It’s not just about the quantity of colors, though, it is also about
how to choose and use them. How many colors would you say I
used for the nodes in Figure 42.10(a)? If you guessed 16 – which is
the correct answer – you’re very lucky, or you have some Truman
Capote levels of pattern recognition. In the network I highlighted
three groups of nodes. These have different colors, believe me or
not. Few – if any – people would be able to tell without scanning
the figure for more than a handful of seconds. Requiring this level of
effort from your viewer means to lose them.
Why does Figure 42.10(a) fail? Because it assumes that RGB is a
 node visual attributes 589

Figure 42.10: (a) The comic

book social network using com-
munities for the node’s color.
(b) An example of the RGB
color space. The black arrows
indicate equal distance move-
ments in this space, connecting
colors at different human per-
(b) ceptive distances.

(a)

perceptive color space. In a perceptive color space, if you move by

a given amount, you get to a color that will be perceived differently.
This is wrong, as Figure 42.10(b) shows: the two black arrows in
it make two movements in the space and show that the same RGB
space distance can connect either two virtually identical colors –
virtually identical for our monkey brains – or two very distinct ones.

Figure 42.11: A version of Fig-

ure 42.10(a) with fewer colors
and based on a more sane color
space than RGB.

If there is one message I wish I could ingrain in you after read-

ing this material is this one: RGB is a terrible terrible terrible color
space for information visualization and nobody should use it for
anything related to data design ever. There is some research back-
13
Cynthia A Brewer. Color use guide-
ing color palettes that align better with human perception – also lines for mapping. Visualization in
including the case of people with color blindness13 . A good resource modern cartography, 1994:123–148, 1994
590 the atlas for the aspiring network scientist

14
Mark Harrower and Cynthia A
you can use is the Color Brewer interactive tool14 , which will gen- Brewer. Colorbrewer. org: an online tool
erate the palettes for you15 . Color Brewer is embedded in many for selecting colour schemes for maps.
The Cartographic Journal, 40(1):27–37,
software/programming packages that you might already use for
2003
your visualizations, including R16 , Cytoscape (since version 3.7.1, 15
[Link]
for earlier version you need the Color Cast plugin), QGis, Python 16
Erich Neuwirth and R Color Brewer.
(Matplotlib and Seaborn, for instance), and Matlab. Colorbrewer palettes. R package version,
pages 1–1, 2014
Figure 42.11 uses the Color Brewer space and fixes one of the
many problems of Figure 42.10(a). In Figure 42.11 we use also fewer
colors – just nine – which is always good.
Color Brewer and RGB are not the only possible color spaces you
could use. If you are creating visualization for printing, you should
use a CMYK color space. This is similar to RGB, but RGB is an
additive color space, while CMYK is subtractive. Additive color spaces
describe how different wavelengths of light add to each other, which
is how computer screens work. Subtractive color spaces, instead,
describe how ink combines on the page, which is why it’ll show
better how things will look in print. HSV and HSL are alternative
color spaces which transform RGB to be more perceptually-relevant:
we as humans don’t really perceive colors as combinations of red-
blue-green, but as variation in hue, saturation and lightness, which is
what HSL stands for.

Figure 42.12: Some linear color

palettes you’ll find available
in different software. From
top to bottom: viridis (Mat-
There are other options for linear color gradients – which I show plotlib), Gnuplot default, and
in Figure 42.12. Some of these palettes were systematically compared jet (Matlab).
across a series of tasks viewer might want to perform17 , as well as 17
Yang Liu and Jeffrey Heer. Some-
different issues your readers might have with perceiving colors. where over the rainbow: An empirical
assessment of quantitative colormaps.
For instance, to tackle the aforementioned issue of color blindness,
In Proceedings of the 2018 CHI Conference
you could transform these palettes into their correspondent black on Human Factors in Computing Systems,
and white version and see how they look like to a person unable to pages 1–12, 2018

distinguish colors but only relying on lightness. I do exactly this in

Figure 42.13 and show that, for instance, the jet palette in Matlab
performs poorly because the two ends of the spectrum become
indistinguishable. Of course, testing for color blindness and other
human vision deficiencies is much more complex than this, and you
should delve deeper in the literature I cited at the beginning of the
chapter.
Note that here I used node colors for communities. It seems that
I’m suggesting that you shouldn’t find more than nine communities
in your networks. That is not exactly what I’m saying. Of course,
when it comes to the analysis, you will find the number of communi-
 node visual attributes 591

Figure 42.13: The linear color

palettes from Figure 42.12,
transformed in a grayscale.
From top to bottom: viridis
ties that you will find. The sky is the limit there. It’s when it comes to (Matplotlib), Gnuplot default,
visualizing them that you should never show more than nine. If you and jet (Matlab).
try to break that limit, you may as well not visualize anything. A fa-
mous motto in data visualization is: “emphasizing everything means
to emphasize nothing”. So you need to find a different solution,
maybe showing smaller extracts of your data.
As with node sizes, also in node colors – if you’re applying a
gradient – you’re best served using a (quasi)logarithmic scale to high-
light differences better and make color variance more meaningful.

42.3 Other Features

To wrap up this chapter, let’s see a few more things you can do
to your nodes. They both stem from the same idea: your nodes
represent something, and so you want to communicate this to your
viewers.
The first strategy involves node labels. If you want the audience to
know something, you simply tell them. You plaster some text on top
of your nodes and you call it a day. In my opinion, this is a desperate
move and it should be avoided if possible. Just as in movies, also
in data visualizations it’s better to “show, not tell”. In other words,
nobody wants to read your network. They want it to speak to them.
That is not to say that sometimes a good choice of node labels
can enhance your visualization. You can practically transform your
network into a glorified word cloud. I don’t love it, but I grudgingly
admit that sometimes it works. An example could be the one in Fig-
ure 42.14 – although in this case one should choose a less saturated
color for the nodes, because the current red goes in the way of the
readability of the label. My rule of the thumb is that the node label
font size should have a one-to-one correspondence to the node size. It
would look weird to have a gigantic label on top of a tiny node, and
vice versa.
The second visual attribute you could play with is the node’s
border. This is an interesting one, because it could be used for quanti-
tative and qualitative attributes at the same time. For the node border,
you can both decide the color and the thickness. Again, you should
really ask yourself whether you really need to do it. Personally I
almost never touch node borders – I’d say that in 99% of my visu-
alization the border is invisible. If you already have node sizes and
592 the atlas for the aspiring network scientist

RWA
BHR SWZ
NPL UGA
KHM BTN MDV

THA
HKG
YEM
SOM
BDI Figure 42.14: A network with
PHL OMN SDN KEN
KWT LKA TZA MWI

PNG
LAO
MMR MYS
DJI
JOR
PAK
node labels conveying informa-
IND
NZL BGD ZMB
BHS IDN LBN
tion about a node’s importance.
AUS
FJI
SAU SYR
TGO
ZAF ZWE
SLB MLI
MNG BEN
SGP KOR KGZ
TJK
AFG NAM
In this trade network, it is
GHA

QAT EGY IRN UZB

SLE VNM

JPN CHN ARE MDG

COM
MLT UKR GEO
the country’s Gross Domestic
GMB
TKM
AGO LBR

GIN SEN
CIV GNB BFA
BRN
IRQ
BWA
MOZ
TUR
BLR ARM Product.
BGR
COG KAZ
MRT MDA
TCD LSO LBY MUS
NER
CHE AZE SVK
ROM
ALB
GRC
CHL ARG GAB
CYP
LTU
TUN
NGA POL
BRA
ERI

PER
USA ISR
GBR RUS
CZE LVA

PRY
BOL
CMR ITA HUN MKD
PRT
VEN SUR AUT
IRL
FRA
CAF

DEU
URY PAN HRV EST
ECU
MEX MAR CPV
HTI
COL SWE
BIH
DZA SVN
GUY
ESP NLD
CAN
DOM TTO BLZ
GNQ
BEL NOR
NIC GTM
JAM FIN
ISL DNK
CRI LUX
CUB
HND
SLV

colors, adding a border of a different size and color would just cause
information overload in your reader’s brain. You should only do it if
there are extremely clear patterns in your network, which involve no
more than a handful distinct values, and that can be easily parsed.
For instance, in Figure 42.15, we could have two nodes of same
size and color – perhaps these are two plants in the same country
(color) and employing the same number of people (size). However,
they process different products (border color) and they have differ-
ent throughput in number of products processed per day (border
thickness).

Figure 42.15: Two nodes with

same color and size, but with
borders of different thickness
and color.

Another strategy is more creative – and for this reason you should
apply tons of caution if you want to go this way. It involves xeno- 18
[Link]
graphic18 . This translates to “weird visualizations”, stuff that has
very specific and almost unique use cases, and thus it’s likely to
choose a style that people haven’t seen before. You can be creative
with what you put on your nodes, as long as you don’t abuse it and
it has a meaningful relationship with your message.
One obvious way you can communicate differences in kind when
it comes to a node would be to represent it not as a dot, but as a fig-
ure. The classical case is by transforming the node’s shape. I already
used this approach in this book for bipartite networks. A classic way
to visualize them is to use one node shape for V1 nodes, and another
 node visual attributes 593

for V2 nodes. For instance they can be circles vs squares. Another

way is to use symbols. For instance, you could have a network of
dogs and use a silhouette of the dog’s appearance to encode its breed 19
[Link]
– this is inspired by the beautiful “Top Dog” visualization19 . informationisbeautiful.
My favorite use case, instead, keeps the node’s shape constant, but net/visualizations/
best-in-show-whats-the-top-data-dog/
transforms it into a chart in itself. This involves the often-maligned
pie charts. Pie charts get a bad rep, often deservedly so, but can
be rather useful in specific instances. You can use them both in the
qualitative and in the quantitative use case, making them more
versatile than either node color or size.

Figure 42.16: (a) Using pie

charts on nodes to signify their
allegiance to multiple commu-
nities. (b) Using pie charts to
represent the relative centrality
(a)
(b) of each node.

We know that communities in networks can share nodes (Chapter

34). If you’re using the node color to encode the community, what
do you do if a node belongs to more than one of them? You can
use a pie chart for that! Figure 42.16(a) shows an example. This
assumes that the node is not part of too many communities but, if
you have enough communities in your network to break a pie chart,
you shouldn’t use colors to encode them to begin with.
In the quantitative case, pie charts are more limited, but still work
in case you have “quantitative classes”. With that, I mean that you
have a quantitative attribute that can take very specific and very
different values, such as a centrality. In that case, distinguishing
between few very different pie charts is possible even for the human
brain, as you can see in Figure 42.16(b).
A final xenographic touch concerns playing with the alpha chan-
nel – i.e, the opacity of the node. In this paragraph, I consider the
extreme case of not showing the nodes at all. Normally, there’s no
point in having invisible nodes. After all, the nodes are what you
want to see in a network, so why making it impossible to look at
them? However, there are some use cases in which this rule can be
broken. I present one in Figure 42.17. I’m not arguing that the figure
is a good visualization: what I’m saying is that being able to see the
nodes would not make much of a difference, especially since they do
not have attributes of interest. Rather, the visualization allows you to
see where different types of edges create red and green clumps, and
which edge type keeps the network together in which branches. And
how to deal with edge visual attributes is exactly the topic of the next
594 the atlas for the aspiring network scientist

Figure 42.17: A network with

fully transparent nodes, where
all the topological information
is conveyed by the edge colors.

chapter.

42.4 Summary

1. The first visual attribute of nodes is their size. Usually, you want
to show quantitative attributes via size – the degree, the capacity,
etc. Be aware that you should always manipulate the area of the
node, which is what your viewer perceives. If your software only
allows you to control a node’s radius, keep in mind that your area
will change quadratically for each linear change of the radius.

2. Second, you can control a node’s color. Usually, this is for qualita-
tive attribute, e.g. community affiliation. Use no more than nine
distinct colors, from a perceptual-aware space (not RGB rainbows!).

3. Gradients can be used for quantitative attribute: diverging ones

for quantities with a clear midpoint – e.g. correlations –, otherwise
sequential ones for quantities going from zero to an arbitrary
maximum.

4. You can augment your nodes with additional visual elements.

Labels could be used – sparingly – and their size should be locked
with the node’s area size. You can use pie charts and icons to
embed additional information on the nodes.

42.5 Exercises

1. Import the network at [Link]

42/1/[Link], calculate the nodes’ degrees and use them to set
 node visual attributes 595

the node size. Make sure you scale it logarithmically. This can be
performed entirely via Cytoscape. (The solution will be provided
as a Cytoscape session file)

2. Import the community information from [Link]

eu/exercises/42/2/[Link] and use it to set the node color.
(The solution will be provided as a Cytoscape session file)
43
Edge Visual Attributes

When it comes to edge visual attributes, most of the things already

mentioned for nodes in Chapter 42 still apply. So this is going to be
mostly a recap, with a few additional warnings.

43.1 Classical Visual Elements

Size
The equivalent for edges of node size is the thickness. As in the
previous case, this is mostly for quantitative attributes on edges. The
most trivial one is the edge’s weight: heavy edges usually appear to
be more thick. Another common use case is to put edge betweenness
as the determinant of the edge thickness. This works well when used
in conjunction with nodes sizes following the same semantics. It
gives a sense of balance to the visualization, so you can see which
edges are contributing to the node’s centrality. Figure 43.1 shows an
example.

Figure 43.1: In this network the

edge thickness is proportional
to its edge betweenness value.
The node size is proportional to
the node betweenness.
Just like node betwenness, also edge betweenness is unevenly
distributed across edges. And, as you already saw, typically edge
weights distribute equally broadly – see Chapter 24 for a refresher.
So you have to apply the same pseudo log scaling for edge thickness
as you did for node size. Lines are considered one dimensional,
so you shouldn’t worry too much about the square area problem I
mentioned for node sizes. It will start to be a problem only if your
edges are so large that your eyes start interpreting lines as rectangles,
at which point they’re probably already too large!
 edge visual attributes 597

Color

When it comes to colors, there are more differences between edges 1

Danielle Albers Szafir. Modeling color
and nodes than we just saw for sizes1 . The fundamental difference difference for visualization design. IEEE
between edges and nodes is that there are so many more of the transactions on visualization and computer
graphics, 24(1):392–401, 2017
former than of the latter: typically twice or three times as many. Also,
dots and circles are much easier to see than lines, especially thin
lines. Since most edges will have low weights, most of them will be
relatively thin, as we just discussed. Thus, seeing the edges is trickier.
There is another reason, which is even more practical. Very rarely,
if at all, you will have good qualitative information about your edges.
By their very nature of being the glue connecting things, edges are
much more likely to have quantitative information attached to them.
We usually classify things, not the glue between things.

Figure 43.2: (a) Using edge

colors to represent the edge’s
u layer. (b) Using edge colors to
(a)
represent the link community
(b)
to which they belong.
The most obvious exceptions are two. You can have qualitative
information telling you to which layer and to which community an
edge belongs, if you have multilayer networks (Section 4.2) and/or
link communities (Section 34.5). For multilayer networks you can use
edge colors to represent the layer if you adopt a multigraph visual-
ization – as I do in Figure 43.2(a). However, this will get unwieldy
pretty soon, as the number of nodes, edges and layers grows beyond
an elementary size. In fact, it’s usually best to use dedicated tools 2
Manlio De Domenico, Mason A Porter,
for the visualization of multilayer networks2 – although the field of and Alex Arenas. Muxviz: a tool for
multilayer analysis and visualization of
visualizing multilayer networks is still in its infancy. networks. Journal of Complex Networks, 3
For link communities, keep in mind the same warning I made (2):159–176, 2015c
regarding node communities. It is pointless to try and visualize more
than a handful – nine – communities, even more so when it comes
to links. Figure 43.2(b) shows an example. Again, it’s not that you
should always find fewer than nine communities in your analysis.
You can and should find however many there are in the network. It’s
visualizing them that is a problem.
As said, most often you will have quantitative information on your
edges. So it is much more common to use gradients on the edges
than it is on the nodes. You have gathered that I’m not a great fan
of gradients from the previous chapter, but that’s what we have to
work with. The way I usually fix the problem is to use the same
quantitative attribute for both thickness and color, so that the two
can work in concert and reinforce each other. Two imperfect visual
598 the atlas for the aspiring network scientist

clues can sum and make each other clearer. This is the case for edge
betweenness, determining both color and thickness of the edges in
Figure 43.3(a).

Figure 43.3: (a) Using edge

colors to reinforce the message
conveyed by the thickness: the
edge’s betweenness centrality.
(b) Using edge colors for an
(a)
(b) orthogonal quantitative infor-
That is not to say that there aren’t good reasons to break this rule. mation: edge thickness is its
One dimension I play with is usually the one of the edge weight’s weight, while the color repre-
significance, for instance when doing network backboning – see sent the weight’s significance.
Chapter 24. In that case, the color can work in contrast with the
thickness. I find more natural to use thickness to represent how
heavy an edge is, so to assign it to represent the weight. This is under
the metaphor that large things generally weigh more. On the other
hand, there is no inherent connection between the color of a thing
and its weight. So I assign the color to represent the significance.
Usually, larger weights tend to connect nodes which have higher
average connection weights, so large links will tend to have paler
colors than smaller links, which creates a nice contrast in Figure
43.3(b).
A final word of caution about the number of colors in your vi-
sualization. You might have noticed that Figures 43.2 and 43.3 use
edge colors but choose a single hue for the nodes. This is because
in a network you have two visual elements – circles and lines – and
allowing both of them to be colored differently effectively doubles
the number of visual elements in your visualizations. Already Figure
43.2(a) feels way too busy. If in the previous chapter I told you not to
use more than nine colors in the visualization, here I’m giving you
a more nuanced guideline: the sum of the distinct number of colors
for edges and nodes must be nine or lower. Meaning that, if you have
five colors for nodes, you shouldn’t have more than four for edges.

Transparency
Transparency is another aspect in which edges diverge from nodes.
In the previous chapter I mentioned that nodes should be fully
visible, and provided only a single use case in which I believe trans-
parency can add something to the visualization by removing the
nodes from sight. When it comes to edges, I usually abuse trans-
parency lavishly. Most commonly, I make transparencies work to-
gether with colors, to reinforce them. Significant links have darker
 edge visual attributes 599

colors and are more opaque. The objective of playing with the alpha
channel for edges is to create a visual hierarchy, where nodes come to
the forefront and edges go to the background.
However, sometimes you can play with edge transparencies even if
you don’t have any attribute to attach to them at all! This is because
of the sheer number of edges: in most real world networks, they
are going to overlap to each other, no matter what. Thus edge trans-
parency, even a fixed value, can highlight structure, because there
are going to be more overlaps in dense areas of the network than in
sparser ones. Thus, you can highlight such clusters even without any
edge metadata.

Figure 43.4: (a) A network with

solid edges with 100% opacity.
(b) The same network, but this
time all edges have a 37.5%
opacity, no matter their weight.

(a) (b)

I do exactly that in Figure 43.4(b). Compared to Figure 43.4(a),

the version with edge transparency looks less like a random smudge
on the paper and shows a few structures of interest, even if I added
literally zero bits of information by removing some edge opacity.

Labels

Moving on to edge labels: if I said that node labels have to be rarely

used, then use edge labels even less than that. This is an extremely
rare use case, you should avoid edge labels at (almost) all costs. Prac-
tically, they’re only useful for scholastic examples, when explaining
simple dynamics of super simple graphs. For instance, I used edge
labels in this book for examples of weighted edges in weighted net-
works, with a grand total of five nodes and five edges. That’s more or
less stretching the use case of edge labels to the limit.
600 the atlas for the aspiring network scientist

43.2 Xenographic Elements

Just like with nodes, also with edges you can be... edgy in how you
visualize them. There are two fundamental aspects I’m going to
mention here: shapes and bends.
The classical edge visualization is as a straight, solid line. This is
what you should do in 99% of the cases. However, in many cases,
you might want to slightly change this shape. The most common
shape change for edges is when you are working with directed con-
nections. In this case, the convention is to add an arrow that indicates
the direction of the edge. The arrow points from the originator of the
edge to the target.
Directed networks are more challenging to visualize than you
might think. The reason is not only that you’re doubling the possible
number of edges, which is true and it is an issue. But the real trouble
is that now you might have a significant number of double edges be-
tween the same two nodes: u → v and u ← v. This might make your
visualization a real mess. One convention you can implement is not
to actually draw the two edges. What you can do is to draw a single
edge and add to it a second arrow pointing in the opposite direction
if that edge is reciprocal. Figure 43.5 shows how this strategy looks
like.

Figure 43.5: (a) The classical re-

ciprocal edge visualization with
(b) two bent edges. (b) Merging the
(a)
two edges in a single reciprocal
Unfortunately, this is not an immediately obvious thing to do edge.
with standard network plotting software, so it might take some
effort. Moreover, this visualization technique gets significantly more
complicated in the case of weighted directed networks. If the two
edges, u → v and u ← v, have two different weights, it is even less
clear how you should handle visualizing them in a single line.
Line shapes can be manipulated in other ways, specifically by
altering the line style. One can have dashed, dotted, wavy lines.
These are clearly differences in qualities of the connection, and thus
should only be used for qualitative attributes. My advice would be
to rely on changing the edge line style exclusively for (i) very small
networks, and (ii) just in those rare cases you need to print your
visualization in black and white and thus cannot use color instead.
It is clear that, once you have a lot of connections, they are going to
inevitably be drawn one on top of the other. Distinguishing between
a dashed and a dotted line if they overlap is nigh impossible.
The final odd thing you could do to your edges is to bend them,
 edge visual attributes 601

meaning that the (u, v) edge is not a straight line from u to v any
more, but it takes a “detour”. Why would you want to do this?
There are fundamentally two reasons. Figure 43.5(a) provides an
example: since there are two edges between the nodes, we want the
visualization to be more symmetric and pleasant, and thus we bend
the two edges.
More often, edge bends are used to make your network layout
more clear. You bend edges to bundle together the ones going from
nearby nodes to other nearby nodes. Since this is done mostly to
clean up the visualization after you already decided where the nodes
should be placed, I will deal with this topic in the network layout
chapter (Chapter 44).

43.3 Network Lifting

Let’s recap all the advice I gave you on node and edge visual at-
tributes and see a case of applying each feature one by one to go
from a meaningless hairball to something that conveys at least a little
bit of information. Our starting point is the smudge of edges you al-
ready saw a couple of times: that’s Figure 43.4(a). Note that this isn’t
really the starting point, because we already settled on a network
layout, but that will be the topic of the next chapter.
The usual order I apply to my networks after I settled on a layout
is the following:

1. Edge transparencies;

2. Edge sizes;

3. Edge colors;

4. Node sizes;

5. Node colors.

So let’s do this.
Edge transparencies. In this network, I do have quantitative infor-
mation, that is the edge betweenness of each connection. However,
I think that it’s better if I limit that to the other edge visual features,
so I fix the same edge transparency to all links. The result is Figure
43.4(b).
Edge sizes & colors. We now move on to use edge betweenness. I
merge the two steps of edge size and color into one, because using
simply the thickness does not make a significant difference with the
previous visualization. Compare Figure 43.4(b) with Figure 43.6(a)
and see that not much has changed. So I apply a Color Brewer color
602 the atlas for the aspiring network scientist

Figure 43.6: (a) Using edge

betweenness for the link’s thick-
ness of Figure 43.4. (b) Using it
for the link’s color as well.

(a) (b)

gradient to the edges, resulting in Figure 43.6(b). Hopefully now you

can see that there are a few very important long connections keeping
the network together, connecting very central comic book characters
to a sub universe they’re almost exclusively part of.
Node sizes. It’s time to deal with nodes. There’s something to be
said for keeping them almost invisible, but that’s not what we want
to do here. This is a comic book network and so we want to know
which characters are tightly connected to the universe of which other
characters. So first we need to know who the important fellows are.
We use node size for that. As outlined in the previous chapter, this
is a job for the degree. The more connections a node has, the more
important it is, the larger it should be. And that’s what Figure 43.7(a)
does. Note the pseudo-logarithmic node size scaling.

Figure 43.7: (a) Using node

degree for the node’s size of
Figure 43.6. (b) Using communi-
ties for the node’s color.

(a) (b)

Node colors. Finally, we discover groups of characters using a

community discovery algorithm – see Part IX. I tweak it so that it will
return no more than nine communities. Again, I trust the qualitative
 edge visual attributes 603

color palette that Color Brewer provides, I’m partial to Set1 – even if
it is not exactly color blind friendly. See Figure 43.7(b) for the final
result.
Figure 43.7(b) still has a long way to go before we can call it a
good network visualization. But, compared to the starting point in
Figure 43.4(a) we can definitely say more things about its structure.
Which is exactly what network visualization is for.
A way to improve this picture would be to choose a better layout,
and to apply some tweaks to it. That is the topic of next chapter.

43.4 Summary

1. Edge visual elements should be paired, whenever possible, with

the same semantics as node visual elements. The thickness of an
edge should be proportional to the same – or a similar – variable
determining node size. If node size is its betweenness centrality,
edge size should be its edge betweenness.

2. Differently from node colors, edge colors are used mostly for
quantitative attributes. Usually, there are more edges than nodes
in a network, thus it is harder to limit the number of edge colors.
Anyhow, you should not have more than nine different colors in
total, whether they are node or edge colors. Classical qualitative
edge colors choices are layers or link communities.

3. You should try to collapse reciprocal edges in a directed network

into a single edge with arrows pointing in both directions. You
should use line style very sparingly, only for specific scenarios like
black and white printing.

4. A classical workflow to improve your network visualization is

to determine edge and node visual attributes in this order: edge
transparency, edge width, edge color, node size, and node color.

43.5 Exercises

1. Build on top of you visualization from exercise #2 in Chapter 42.

Assign to edges a sequential color gradient and a transparency
proportional to the logarithm of their edge betweenness (the
higher the edge betweenness the more opaque the edge).

2. Import edge data from [Link]

43/2/[Link] and use the attribute to determine the edge’s
width.
44
Network Layouts

The network layout is the algorithm that decides where to place

each node. This is usually a result of the other nodes to which it is
connected. That is why I dub this process “the art of scattering nodes 1
Mary L Northway. A method for
around”. Such art has a queen: Mary Northway1 , the first person depicting social relationships obtained
who realized – in 1940! – that displaying nodes accurately is a must if by sociometric testing. Sociometry, pages
144–150, 1940
one wants to parse social networks.

100

10-1

10-2
p(k)

10-3

10-4
100 101 102 103 104
k

(a)

(b)
Figure 44.1: (a) In a scatter plot,
Changing graphical elements as we saw in the previous chapters is
changing the x-y coordinates
good, but network data has a peculiarity that other data types don’t
of a point is forbidden, because
have. Networks are a particular data type that allow our representa-
that will change the data. (b) In
tions an additional degree of freedom. This influences the network
a network, you can move nodes
visualization. To see what I mean consider that, in a scatter plot, you
around, as long as you do it
cannot move the points around because that would change the data.
with a consistent set of rules to
But in a network you have connections. What count is not the “abso-
all nodes.
lute” position of a node, but its “relative” one. Figure 44.1 shows an
example of what I mean.
 network layouts 605

The aim of this chapter is to present a few of the classical network

layouts, trying to provide a rough guide on what should be used
when – although this is sometimes subtle and a matter of personal
preferences.
There is a general issue you should be aware of: the vast majority
of network layouts were developed with single layer networks in
mind. However, they are commonly used for tasks that go beyond
these types of networks. For instance, we will see that one common
layout principle is the force directed one. People have been applying 2
Erkki Mäkinen. How to draw a
it to more complex structures such as hypergraphs2 , 3 , or multilayer hypergraph. International Journal of
networks4 . Computer Mathematics, 34(3-4):177–185,
1990
Unfortunately, there is not much visual research that I am aware 3
Naheed Anjum Arafat and Stéphane
of in these subfields. How to visualize a hypergraph or a multilayer Bressan. Hypergraph drawing by
network is a problem with its own challenges and they are both in force-directed placement. In International
Conference on Database and Expert
need of finding their own conventions. The conventions I used when Systems Applications, pages 387–394.
talking about these structures, specifically in Sections 4.2 and 4.3, are Springer, 2017
a good starting point. Otherwise, you should check out some recent 4
Manlio De Domenico, Mason A Porter,
and Alex Arenas. Muxviz: a tool for
surveys5 .
multilayer analysis and visualization of
Another entry in the “weird network types that we don’t really networks. Journal of Complex Networks, 3
know how to visualize” is high order networks – see Chapter 30. As (2):159–176, 2015c
5
Fintan McGee, Mohammad Ghoniem,
far as I know, HoNVis6 is the only technique occupying the high Guy Melançon, Benoît Otjacques, and
order network visualization niche, so you might as well just read that Bruno Pinaud. The state of the art in
paper. multilayer network visualization. In
Computer Graphics Forum, volume 38,
pages 125–149. Wiley Online Library,
2019
44.1 Force-Directed 6
Jun Tao, Jian Xu, Chaoli Wang, and
Nitesh V Chawla. Honvis: Visualizing
By far, the layout you will see in network science papers most often and exploring higher-order networks.
In 2017 IEEE Pacific Visualization Sym-
is the force directed layout. There are many variants of the same posium (PacificVis), pages 1–10. IEEE,
basic principle, but here I’ll limit myself to explain the mechanics 2017
underlying most of them, and provide four examples you can find
implemented in Cytoscape.

+ + Figure 44.2: (a) Nodes behave

like particles of the same charge
repelling each other. (b) Edges
+ + behave like springs, bringing
connected nodes together.
(a) (b)
The basic principle underlying any force directed method is that
nodes should try to repel each other so that they do not overlap, as
you can see in Figure 44.2(a). However, connected nodes should be
closer to each other, to represent relatedness using spatial proximity.
The way this is usually implemented is to consider nodes to have the
606 the atlas for the aspiring network scientist

same magnetic sign – creating the repulsive force. To bring connected

nodes together, edges are – rather than strings – springs. The spring
wants to be as short as possible and so it will pull connected nodes
close together – see Figure 44.2(b). Usually, springs are stronger than
the repelling charge force at long distances, but have a minimum
length, so that when the nodes are very close together they will
not overlap. Of course, as soon as you add more nodes to the mix 7
Jeffrey Heer, Stuart K Card, and
James A Landay. Prefuse: a toolkit for
this gets pretty complicated, as nodes pull other nodes in different interactive information visualization.
directions and so also springs overstretch, even if they’re stronger In Proceedings of the SIGCHI conference
than charges. on Human factors in computing systems,
pages 421–430. ACM, 2005
Different force-directed network layouts include Prefuse7 , yFiles 8
Roland Wiese, Markus Ei-
organic8 , regular9 and compound spring embedded10 , Fruchterman- glsperger, and Michael Kaufmann.
Reingold11 , simulated annealing12 , GME13 and a bunch that I’m yfiles—visualization and automatic
layout of graphs. In Graph Drawing
probably forgetting. They’re all implemented in either Cytoscape
Software, pages 173–191. Springer, 2004
or igraph. They usually differ in the strength and length they give 9
Tomihisa Kamada and Satoru Kawai.
to charges and springs, or in the strategy they apply to find the A simple method for computing
general position in displaying three-
configuration in which charges and springs are the least stressed, i.e.
dimensional objects. Computer Vision,
the system has the minimal possible residual energy. Graphics, and Image Processing, 41(1):
43–56, 1988

Figure 44.3: The same network

displayed with different flavors
of force directed layout. Node
colors are their communities,
node sizes are their degree.
Edge colors and sizes are pro-
portional to their betweenness
centrality.
(b) Prefuse 10
Ugur Dogrusoz, Erhan Giral, Ahmet
(a) Spring Embedded Cetintas, Ali Civril, and Emek Demir.
A layout algorithm for undirected
compound graphs. Information Sciences,
179(7):980–994, 2009
11
Thomas MJ Fruchterman and Ed-
ward M Reingold. Graph drawing by
force-directed placement. Software:
Practice and experience, 21(11):1129–1164,
1991
12
Ron Davidson and David Harel.
Drawing graphs nicely using simulated
annealing. TOG, 15(4):301–331, 1996
(c) yFiles organic (d) Compound spring embedded 13
Arne Frick, Andreas Ludwig, and
Heiko Mehldau. A fast adaptive lay-
Of these, I provide a few visualizations of the typical results you’d out algorithm for undirected graphs
get in Cytoscape with its default parameters choices. They appear (extended abstract and system demon-
stration). In International Symposium on
to be quite similar, with a few differences. From the strongest to
Graph Drawing, pages 388–403. Springer,
the weakest charges: spring embedded (Figure 44.3(a)), Prefuse 1994
(Figure 44.3(b)), yFiles organic (Figure 44.3(c)), and compound spring
embedded (Figure 44.3(d)).
 network layouts 607

Spring embedded (Figure 44.3(a)) tends to shoot nodes in the

stratosphere. It highlights clusters and central backbones better, but
nodes tend to overlap. Note how the purple community here looks
central. We’ll see it changing its relative position in the other layouts,
a sign that even different flavors of force directed can communicate
different messages about the centrality of nodes and/or communities.
Prefuse (Figure 44.3(b)) is the bread and butter of network visu-
alization. It isn’t particularly good, but works ok in most cases, and
tends to be computationally less expensive than the other force di-
rected alternatives. You will find Prefuse to be the default choice in
many cases, in Cytoscape for instance. It deploys a classical balance
between charges and springs. Note how the purple community isn’t
as central here as it was in Figure 44.3(a).
yFiles organic (Figure 44.3(c)) is very similar to Prefuse force
directed. It tends to clusters nodes a bit more snuggingly, usually
works a bit better for more complex networks than the one in Figure
44.3. It is my default choice, unless the network is too big, since the
yFiles organic algorithm has a higher computational complexity.
Compound spring embedded (Figure 44.3(d)) is at the opposite
end of the spectrum when compared to spring embedded. It forces
nodes to be more or less equidistant from each other. It is a good op-
tion if the nodes are all more or less equivalent, but plays badly once
you have diverse node sizes. You can see a lot of nodes overlapping
in Figure 44.3(d).
My rule of thumb here is that, the more complex interconnections
you have the more you want your clusters – if you have any – to be
separated. So you would choose the spring embedded layout. On the
other hand, well balanced and separated clusters, with nodes more
or less on equal footing, will mean going to the opposite end of the
spectrum, to the compound spring embedded.
Note that Cytoscape’s implementation of layout algorithms is not
the most efficient. However, as most network plotting programs,
it will accept you to pass x and y coordinates as attributes of your
nodes, which you can use to display them. One trick is to calculate
the layout using a more efficient script – for instance igraph in R or
C – and then import the result in Cytoscape. Alternatively, remember
that you can have a force directed style layout even without applying
the force directed algorithm. Section 37.4 taught you how to use node
embeddings to determine the placement of nodes on a 2D space.
Force directed is a good choice for sparse to medium-sparse
networks. It works well when you have well defined clusters: any
additional density coming from quasi-cliques is well handled because
the nodes will bunch up in a ball together no matter what. Problems
arise when the additional density comes from interconnections
608 the atlas for the aspiring network scientist

spanning across clusters. Also, it tends to fit your network onto a

virtual sphere: its layouts tend to be circular. This is not always the
best choice, as some networks are going to fit different shapes better.

44.2 Other Node-Link Layouts

Hierarchical
If force directed and its variants are a good default choice, they are
not the only way to display your networks. As I concluded in the
previous section, they have some pretty limited use cases. What
happens when we go out of those use cases?

(a) yFiles organic (b) yFiles tree

(c) yFiles radial

Figure 44.4: The same network
The first scenario we consider is the one of extremely sparse
displayed with different layouts,
networks. In that case you can obviously use force directed. However,
force directed and hierarchical.
it can arguably be not the better choice. Consider Figure 44.4(a): even
for this extremely sparse graph, the layout still manages to have
many awkward node-edge overlaps. This is because very sparse
networks are similar to trees, and a tree hardly fits the assumption
of a circular layout. A tree has a root and leaves, thus a inherent top-
down flow, which doesn’t fit well the “in-out” flow of a circle (from
the center to the circumference).
In this case you want to use a tree layout as I show in Figure
44.4(b). A popular variant would be a radial layout – Figure 44.4(c).
The radial layout is a compromise that still respects the tree-like
structure of sparse graphs and, at the same time, has a force-directed
“in-to-out” flavor to it.
As soon as your network becomes more dense than a quasi-tree,
you should probably stop considering the hierarchical layouts. In
some extreme cases they can work, if you want to highlight some
specific messages. For instance, I would not use it for the network
in Figure 44.5(a), but I can see how it can communicate something
about the clusters of the network. The layout manages to put nodes
 network layouts 609

Figure 44.5: (a) A dense net-

work still manageable with
a hierarchical layout. (b) An
example of a network too com-
plex to be displayed with a
hierarchical layout

(a)
(b)

in the same community in the same column, showing how some

communities have stronger – darker, thicker – connections than
others. However, a non-trivial number of nodes and edges would
make you network visualization completely unintelligible, as it
happens in Figure 44.5(b).

Circular
A second scenario to consider is the case of an extremely dense
network. The network could be so dense, that the force directed is
not able to pull nodes apart and show structure. In this case, the first
step of the solution involves considering a layout that might not seem
the best for the job, but has a few tricks up its sleeve: the circular
layout. Which is exactly what it sounds: it places nodes on a circle,
equidistant from one another.
The first part of the trick in using circular layouts is not to display
the nodes in a random order, but choosing an appropriate one. Ide-
ally, you want to place nodes in bunches such that most connections
happen across neighbors. Usually this is achieved by identifying the
nodes’ attribute which groups them best. You can also run a custom
algorithm deciding the order and then provide that as the attribute
for the circular layout.
Figure 44.6 shows an example. In the figure you can see that the
layout still works: it shows how most connections remain within
the communities, and clearly points at how many and where the
inter-community connections are. In a force directed layout, these
connections would stretch long and be forced in the background of
denser areas, with the effect of being difficult to appreciate. However,
the real kicker for circular layouts happens when you consider the
610 the atlas for the aspiring network scientist

Figure 44.6: A circular layout.

use of an additional visual feature: edge bends.

44.3 Edge Bends

The default choice in network layouts up until recently was to show

edges as straight lines. However this is usually not pleasing visually.
Go back to Figure 44.3(a) and you will feel how clunky the long thick
edges feel. People started playing with edge bends and discovered
that they can solve a series of other problems, rather than being a
simple cosmetic enhancement.
The first strategy to bend your edges is to bundle together the
ones coming/going from/to more or less the same part of the net- 14
Danny Holten. Hierarchical edge
bundles: Visualization of adjacency
work14 , 15 . This is extremely useful for the very dense networks in
relations in hierarchical data. IEEE
circular layouts I teased at the end of the previous section16 . Con- Transactions on visualization and computer
sider Figure 44.7(a): here the circular layout isn’t helping us much graphics, 12(5):741–748, 2006
15
Danny Holten and Jarke J Van Wijk.
in making sense of this extreme density. However, once we bundle
Force-directed edge bundling for graph
edges in Figure 44.7(b), we obtain a much clearer idea of what is go- visualization. In Computer graphics forum,
ing where. Sure, the visualization is still complex and it is still hard volume 28, pages 983–990. Wiley Online
Library, 2009
to tell which nodes are connected to which other. However, that is a 16
Emden R Gansner and Yehuda
much better situation than the alternative visualization you would Koren. Improved circular layouts.
get from a force directed without edge bends. Figure 44.7(c) is my In International Symposium on Graph
Drawing, pages 386–398. Springer, 2006
proof.
You can use edge bundling in other layouts too. Usually this will
reduce clutter and make your visualization clearer and easier to
parse. For instance, in a force directed layout communities will re-
duce to flower bouquets where all connections collapse in the same
point. See Figure 44.8(a) for an example. This is as if the community
 network layouts 611

(a) (b) (c)

Figure 44.7: The same network
is a hyperedge connecting all the nodes that it groups. This is not ex- displayed with different layouts:
actly the full topological information in the network, but oftentimes it (a) circular with no bends, (b)
is an acceptable approximation. circular with edge bundling, (c)
A hierarchical network layout benefits from edge bends as well – Prefuse force directed.
see Figure 44.8(b). This is because it is difficult to display hierarchies
in a tight space, they tend to grow horizontally and vertically. With
edge bends you can make your edges take a slightly longer route
which increases the picture’s readability. Of course, a bend can
decrease the readability if you are then unable to clearly distinguish
which edge goes where in a bundle. So there are techniques to make 17
Benjamin Bach, Nathalie Henry Riche,
this distinction17 . Christophe Hurter, Kim Marriott, and
Bending edges helps in a second scenario. When squeezing com- Tim Dwyer. Towards unambiguous
edge bundling: Investigating confluent
plex multidimensional structures, such as dense networks, onto a drawings for network visualization.
two dimensional plane, you’ll often have no good placement choice IEEE transactions on visualization and
computer graphics, 23(1):541–550, 2016
for your nodes and edges. In many cases this is just awkward and
displeasing to the eye, but in many others it can create problems and
untruthful visualizations. The most dreaded case you should avoid at
all cost is ghost edges.

Figure 44.8: Edge bundles

in (a) force directed and (b)
hierarchical network layouts.

(a) (b)

Consider Figure 44.9. Suppose we know that the network’s topol-

ogy involving those three nodes is on the left. Node 1 is connected
to node 2 which is connected to node 3 in a chain. However, these
612 the atlas for the aspiring network scientist

three nodes don’t live in a vacuum. There are thousands of other

nodes and edges around them, pulling and pushing them in different
directions. After a few layouts, you might not notice that your three
nodes ended up in the configuration to the right.

1 2 2 Figure 44.9: A scenario in

which you might create ghost
1 edges.

3 3
(a) (b)
Seeing that configuration, a viewer would instantly assume that
node 1 is connected to both node 2 and 3, without a connection
between the latter two. This, as we know, is wrong. Worse still,
there’s no way by looking at the configuration to the right to know
what really is going on between these three nodes. It might be that
node 1 is not connected to either of those, and ended up there by
accidents of your layout. Or that could be a squeezed triangle. Or it
might be even true that the three nodes are not connected at all to
each other, and that’s just a long edge connecting two other nodes
out of sight. There are so many ways to lie in a 2D network layout –
whether you do it accidentally or on purpose.

Figure 44.10: A force directed

layout with organic edge rout-
ing.

Edge bends can partially save you. In particular, the trick is to use
organic or orthogonal edge routing18 , implemented by yFiles. To
18
Tim Dwyer, Kim Marriott, and
Michael Wybrow. Integrating edge
know what it looks like, consider Figure 44.10. In the figure, there routing into force-directed layout.
are many cases in which the vanilla force directed layout would pass In International Symposium on Graph
Drawing, pages 8–19. Springer, 2006
a straight edge across the nodes. As it happens, some of those cases
were actually two edges with a node in between. The organic edge
router would not change the edge shape in that case. But some edges
 network layouts 613

get a dramatically evident bend, because the vanilla force directed

would make you believe they connected nodes while, in reality, they
did not.

44.4 Alternative Layouts

In this section, we break the assumption that nodes are circles and
edges are lines. We try to find weird ways to summarize the network
topology in a way that is more compact and compelling.

Matrix Layouts
What’s the last resource for networks in which the density is too high
even for a circular layout plus edge bundles? If your network is so
dense, then you don’t have a network: you have a matrix and you
should visualize it as such. In these cases, what matters more is not
really which area is denser than which other, but which blocks of If your network is this dense and also
19

nodes have connections with higher and lower weights.19 unweighted, consider changing job.

Figure 44.11: (a) A matrix view

of a network with progressive
edge weights. (b) A matrix
view of a network with diver-
gent edge weights, such as
correlations.

(a) (b)
Color scales should be chosen depending whether your edge
weights are defined as progressive or divergent. The first case, in
Figure 44.11(a), is the classical case of an edge indicating the intensity
of the connection between two nodes, or the cost of edge traversal.
The second case, in Figure 44.11(b), is a classical correlation network.
This is a default visualization scenario, as correlations are always
defined between any pair of nodes, and thus the network will be
complete.
That is not to say that this is the only use case of a matrix visual-
ization. Even for sparser networks, sometimes a matrix is worth a
thousand nodes. Consider the case of nestedness (Section 28.4), a par-
ticular core-periphery structure for bipartite networks where you can
sort nodes from most to least connected. The most connected node
connects to every node in the network, while the least connected
nodes only connects to the nodes that everyone connects to. This
sort of linear ordering naturally lends itself to a matrix visualization.
614 the atlas for the aspiring network scientist

Figure 44.12: A matrix view of

a nested network.

While a node-link diagram would make a mess of such a core, ren-

dering the message difficult to perceive, a matrix view is deceptively
simple, as I show in Figure 44.12.
This example is a good example of the main problem in visu-
alizing networks as matrices: the order of the rows/columns you
choose is the most important thing. You should put your nodes in
the sequence that highlights the crucial structural characteristics the
best. In the nestedness case, nodes are sorted by degree (or total in-
coming weight sum). If your network has communities, you want to
have nodes next to their community mates. This creates the classical
block diagonal matrices. There are other criteria you might want to 20
Michael Behrisch, Benjamin Bach,
consider20 . Nathalie Henry Riche, Tobias Schreck,
and Jean-Daniel Fekete. Matrix reorder-
ing methods for table and network
visualization. In Computer Graphics
Hive Plots Forum, volume 35, pages 693–716. Wiley
Online Library, 2016
The issue with all traditional node-link diagrams is that the posi-
tion in space of a node is arbitrary: it does not reflect its properties,
but it is just relative to its connections. As such, layouts are not re-
producible, because a small change in the initial conditions in the
placement of a single node will result in a completely different lay- 21
Martin Krzywinski, Inanc Birol,
Steven JM Jones, and Marco A Marra.
out. Hive plots21 try to fix these issues by providing a way to have a
Hive plots – rational approach to
deterministic node layout placement. visualizing networks. Briefings in
The idea is the following: first, the user determines a set of rules. bioinformatics, 13(5):627–644, 2011

The aim of these rules is to divide nodes into classes. Nodes in the
same class will be grouped together on the same axis. Then, the user
selects a specific measure, which determines the position of a node
on that axis. The hive plot will then attempt to place the axes in such
a way to minimize edge crossings. If there are connections between
the nodes on the same axis, the axis will be duplicated in order to
avoid confusing loops.
 network layouts 615

Figure 44.13: (a) A graph where

I encode with colors the node
and edge types. (b) The hive
plot version of (a).

(a)

(b)

Figure 44.13 shows a simple example. In Figure 44.13(a) we have

three node types, which result in three axes in Figure 44.13(b). How-
ever, the blue nodes also connect to each other, and have more com-
plex connecting patterns with the other groups. Thus, we duplicate
the blue axis, meaning that we have a copy of it at slightly different
angles. The blue-blue links flow between the copies, and we divided
the connections between blue and other nodes as to minimize the
number of crossings.
The position of nodes on the axes is determined by the degree.
The nodes with high degree on each axis are the ones on top, sepa-
rated by the rest. The other nodes have all the same degree, so they
are grouped, although we separate them so that the nodes don’t over-
lap with each other. You can choose which measure to use for the
node positioning on the axis, you are not forced to use the degree.

Graph Thumbnails 22
Vahan Yoghourdjian, Tim Dwyer,
Karsten Klein, Kim Marriott, and
Just like hive plots, also graph thumbnails22 aim to provide a de- Michael Wybrow. Graph thumbnails:
terministic layout, where two isomorphic graphs result in the same Identifying and comparing multiple
graphs at a glance. IEEE Transactions on
visualization. In graph thumbnails, we decide to give up the ability
Visualization and Computer Graphics, 24
of analyzing local structures. We are not seeing each individual node: (12):3081–3095, 2018
the visualization is a summary of the graph’s global structure. The
idea is to dissect a graph into its main core components in a hierarchi-
cal fashion. Each core is then visualized as a circle, whose color tells
us its core level. You should use graph thumbnails when you need
to compare a large number of graphs in a compact way and you care
about the high-level organization of the graph as a whole, rather than
the meso-level communities – or the individual nodes.
The decomposition is done via the classical k-core detection – see
Section 11.7. Each connected component of a network is part of a 1-
core. Then, there could be multiple k-cores around the network. Each
k-core is represented as a circle, and it is nestled inside the (k − 1)-core
that contains it.
616 the atlas for the aspiring network scientist

Figure 44.14: (a) A graph and

its (b) graph thumbnail visual-
ization. The color of the circle
encodes the k value of the k-
core, while its size is (loosely)
proportional to the number of
nodes in that particular core.

(b)
(a)

Figure 44.14 shows an example. The graph in Figure 44.14(a)

has two communities. All nodes in the graph are part of the 1-core
because it’s a single connected component. Some nodes are not part
of the 2-core, but they are only the peripheral dangling ones: both
communities are part of the same 2-core. The communities split
when we consider the 3-core, that is why the second circle in the
graph thumbnail in Figure 44.14(b) contains two subcircles. The
smallest community only contains a 4-core, while the largest goes up
to a 6-core, explaining why the second circle goes to darker hues.
A disadvantage of the graph thumbnail visualization is that it 23
Charles R Collins and Kenneth
needs to apply a circle packing algorithm23 . It is impossible to pack Stephenson. A circle packing algorithm.
circles efficiently inside other circles. In this specific example, both Computational Geometry, 25(3):233–256,
2003
the 1- and the 2-core circles are larger than they should be given the
number of nodes they contain. I needed to enlarge them, because
otherwise they could not contain properly the two 3-cores of the
network.

Probabilistic Layout
Following the same “we can’t visualize all nodes” philosophy of 24
Christoph Schulz, Arlind Nocaj,
graph thumbnails, we have probabilistic layouts24 . This technique is Jochen Goertler, Oliver Deussen,
handy when you have a generic guess of where the nodes should be, Ulrik Brandes, and Daniel Weiskopf.
Probabilistic graph layout for uncertain
but you cannot draw them all. You should use such layouts especially network visualization. IEEE transactions
for very large graphs that you couldn’t visualize otherwise, because on visualization and computer graphics, 23
(1):531–540, 2016
they have too many nodes and/or edges.
The idea is as follows. First, you sample the nodes in your net-
work, taking only a few of them. Then you calculate their positions
using a deterministic force directed layout. You repeat the procedure
multiple times, obtaining, for each node, a good approximation of
where it should be. If you have nodes that you never sampled, you
can reasonably assume that they are going to be in the area surround-
 network layouts 617

ing their neighbors. Since you’re applying the algorithm to a sample,

this won’t take much time even if the original network was too large
to be analyzed in its entirety.
Now each node is associated to a spatial probability distribution,
much like elementary particles in quantum physics. You can assume
that, if the node is anywhere, it’ll be somewhere in the area where
its probability is nonzero. At this point, you can merge nodes whose
spatial probabilities overlap, by detecting and drawing a contour
containing them. You should then bend edges and smudge them as
well, to reflect the uncertainty of where their endpoints are.

Figure 44.15: A graph and

its probabilistic layouts, for
different levels of α.

(a) Original (b) α = 0.01 (c) α = 0.7

You can specify a parameter α regulating how tight your smudges
should be. For low values of α, you get the quickest results at the
price of large uncertainties. When α ∼ 1, your smudges become
points, tightening up all nodes belonging to a smudge in the same
area. Figure 44.15 shows a toy example, for two levels of α.

Revealing Matrices
One key visualization technique is scatterplot matrices or SPLOMs.
When you have multiple variables in your dataset, you might be
interested in knowing which one correlates with which other. So you
can create a matrix where each row/column is a variable, and each
cell contains the scatter plot of the row variable against the column
variable. Figure 44.16 shows an example.
The same visualization technique can be applied to networks. In 25
Maximilian Schich. Revealing matrices.
revealing matrices25 , each row/column of your matrix is an entity. 2010
Then, each cell of the matrix contains a bipartite network, where the
nodes of one type are the row entity and the nodes of the other type
are the column entity.
One defect of SPLOMs is that the main diagonal of the matrix is a
bit awkward. In it, the row variable and the column variable are the
same. Thus the scatter plot is meaningless, as it is the same variable
on the x and y axes: a straight line. One could modify it by showing
some sort of statistical distribution of the variable, but that would
mean breaking the axis consistency of the SPLOM. For this reason,
the main diagonal of a SPLOM is often omitted.
618 the atlas for the aspiring network scientist

Figure 44.16: An example of

SPLOM visualization.

This defect does not apply to the revealing matrices visualization.

The main diagonal in this case is well defined: it is simply the direct
relationship between entities of the same type. Thus, it contains a
unipartite network per node type in your database.

44.5 Case Studies

As it often happens in data visualization, what you want to say with

your network visualization might not be supported by any standard
visualization technique out there. Sometimes, you need to craft a
custom visualization, bending and breaking rules along the way. No
one should really follow your workflow, because it applies only to
your specific aim with your specific data. However, seeing some of
these examples could be helpful in making you realize that you are
not mad: sometimes you really do know better than everybody else.
The aim of this section is to empower you in being daring: try to look
at your data and your communication objective, and create your way
to bringing them together.
I touch on two examples I worked on. These are custom ways of
displaying a node-link diagram that I found useful. These node-link
diagrams have special configurations given the need to highlight
specific features of the networks they represent. Of course, there’s
much more out there, but these are two cases I’m familiar with.
 network layouts 619

Product Space
26
César A Hidalgo, Bailey Klinger,
The Product Space26 , 27 is a popular example. The Product Space A-L Barabási, and Ricardo Hausmann.
The product space conditions the
is a network in which each node is a product that is traded among
development of nations. Science, 317
countries in the global market. Two products are connected if the sets (5837):482–487, 2007
of countries exporting them have a large overlap. The idea of this 27
Ricardo Hausmann, César A Hidalgo,
Sebastián Bustos, Michele Coscia,
visualization is to show you which products are similar to each other,
Alexander Simoes, and Muhammed A
because if your country can make a given set of products, via the Yildirim. The atlas of economic complexity:
Product Space it can figure out which are the most similar products it Mapping paths to prosperity. Mit Press,
2014
should consider trying to export.

(b)
(a)
Figure 44.17: The Product
The original way to try and visualize the Product Space was a Space. (a) Classical force di-
simple force directed layout, as I show in Figure 44.17(a). However, as rected layout. (b) Manually
I mentioned previously, the force directed layouts have this tendency adjusted linear force directed.
of placing your node in a circle. This happens to work really poorly The node color is a product’s
in the case of the Product Space. The reason is that not all prod- Leamer category.
ucts are the same. Some products are harder to export than others.
This is a key concept in the original research, known as Economic
Complexity.
This means that the Product Space has an inherent “direction”.
Countries want to move from simple to more complex products, as
the latter is a more rewarding category to be able to export. However,
the circle has no direction. It is a loop: you always get back to where
you started. The shape of the Product Space in Figure 44.17(a) does
not allow us to perceive the development path. That is why it is 28
Eerily looking like an angel from
necessary to stretch out the visualization as I do in Figure 44.17(b): Neon Genesis Evangelion, with that
creepy head with multiple green eyes...
now the Product Space is a (complex, multidimensional) line28 Am I the only one seeing it?
and you can see that there is a clear direction going from right to
left, from less to more complex products. It is still a type of force
directed layout, but it needed to be customized to remove its inherent
circularity.
620 the atlas for the aspiring network scientist

Cathedral
29
Stephen Kosack, Michele Coscia,
In another paper of mine, I analyze government networks29 . My Evann Smith, Kim Albrecht, Albert-
nodes are government agencies and I establish edges between them László Barabási, and Ricardo Haus-
mann. Functional structures of us state
if the website of an agency has an hyperlink pointing to the website governments. Proceedings of the National
of another agency. One key question is verifying if this network has Academy of Sciences, 115(46):11748–11753,
2018
a hierarchical organization – see Chapter 29. One obvious way to
explore this question is visualizing the network and see if it looks like
a hierarchy. Unfortunately, the network is relatively large and dense.
So I need to come up with a custom layout. Such layout is useful to
visualize dense hierrchical networks, and thus can be considered as
an enhancement of the classical hierarchical layout presented earlier,
that works only for tree-like structures.
The first step is to group nodes into a 2-level functional classifica-
tion. This means to assign to each agency the function it performs
in the government. For instance, a school is part of the education
system (level 1 function) and of the primary & secondary education
(level 2 function). Or: a city government is part of general adminis-
tration (level 1 function) and of the municipal administration (level 2
function).
There are not many level 2 functions so I can collapse all agencies
into their level 2 function. Then I display these functions in a scatter

Figure 44.18: The triangular

two-level centrality plot I de-
scribe in the main text – image
by Kim Albrecht.
 network layouts 621

plot. On the x axis, I report the centrality of the level 1 function.

The most central level 1 function is in the middle and, as we get to
the edges of the visualization, we get to progressively less central
level 1 functions. Now, in this plot, each column contains all level
2 functions belonging to a specific level 1 function. On the y axis I
report the centrality of the level 2 function. Functions at the top are
more central than functions at the bottom. The result is in Figure
44.18. It looks like a nice hierarchy!
The attack you could do to this visualization is that it might
make any network look like a hierarchical network, no matter if
it is actually hierarchical or not. That is why I embed a small inset
in the top left corner. The network in the inset is the result of a
configuration model version of the original network: it has the same
number of nodes, edges, and the same degree distribution. The
connections are rewired randomly, destroying the hierarchy – if
any is present. When I apply the same layout strategy, I obtain the
visualization in the inset: there is not a trace of hierarchy any more!
This proves that the layout is not showing hierarchies where there are
none.

44.6 Summary

1. A network layout is an algorithm that determines where the

nodes of your network visualization should be in a 2D space. The
positions of the nodes are determined by the connections between
them.

2. The most common principle is the one of the force directed layout.
Nodes are charges of the same sign repelling each other and edges
are springs trying to keep connected nodes together. This layout
works for sparse networks with communities, whose topology fits
on a circle.

3. Specialized layouts exists for even sparser networks with a hierar-

chical organization. Circular layouts can work for denser networks,
provided that you use edge bends, bundling edges between nodes
located in the same regions of the circle.

4. Edge bends help in many layouts, particularly avoiding the cre-

ation of “ghost edges”. These happen when your network layout
places a node on top of an edge that is not connected to it, giving
the appearance that the node is part of a chain.

5. Node-link diagrams representing nodes as circles and edges as

lines are not the only solution. For very dense networks you can
show the network as a matrix, as a graph thumbnail via k-core
622 the atlas for the aspiring network scientist

decomposition, or with a probabilistic layout associating a node to

a cloud of probability in space.

6. In many cases, your network will have a clear and unique message
that has never been visualized before. In those cases, you need to
bend rules and create a unique visualization serving your specific
communication objective.

44.7 Exercises

1. Which network layout is more suitable to visualize the network at

[Link] Choose
between hierarchical, force directed, and circular. Visualize it using
all three alternatives and motivate your answer based on the result
and the characteristics of the network.

2. Which network layout is more suitable to visualize the network at

[Link] Choose
between hierarchical, force directed, and circular. You might
want to use the node attributes at [Link]
eu/exercises/44/2/[Link] to enhance your visualization.
Visualize it using all three alternatives and motivate your answer
based on the result and the characteristics of the network.

3. Which network layout is more suitable to visualize the network at

[Link] Choose
between hierarchical, force directed, and circular. Visualize it using
all three alternatives and motivate your answer based on the result
and the characteristics of the network.
 Part XIII

Useful Resources
45
Network Science Applications

Network science is a vast field, exponentially expanding since the late

nineties. There has been so much work on it. This book so far cites
more than 1, 000 papers, and yet there still an incredible wealth of
produced knowledge that did not fit in here. In this chapter I want to
give you a taste of what network science can do. The idea is to briefly
discuss the main contributions of a handful of classic papers that, for
one reason or another, did not find space in the more pedagogical
chapters that preceded this one.
Of course, the set of papers discussed here is subjective: it is my
own perspective of the field, the papers and contributions on which
I stumbled most often while working. Specifically, I am partial to the 1
David Lazer, Alex Sandy Pentland,
field of computational social science1 : the use of computer science Lada Adamic, Sinan Aral, Albert Laszlo
techniques to study social systems – and humanities in general. I Barabasi, Devon Brewer, Nicholas
Christakis, Noshir Contractor, James
am still deeply embedded in the digital humanities tribe. It is also,
Fowler, Myron Gutmann, et al. Life
ironically, a largely incomplete set. No matter how much effort I pour in the network: the coming age of
into this book to make it more exhaustive, it seems that each paper I computational social science. Science
(New York, NY), 323(5915):721, 2009
add simply increases its surface area and makes it less thorough, not
more. It’s the fractal nature of complex systems, and it’s something I
will have to live with.

45.1 Network Effects of Innovation

Why is society nudging us to live in cities? Is there an invisible

force gluing humans in larger and larger settlements? As a matter 2
Luís MA Bettencourt, José Lobo,
of fact, this might very well be true. A research collaboration2 , 3 Dirk Helbing, Christian Kühnert, and
started investigating these questions by performing a deceptively Geoffrey B West. Growth, innovation,
scaling, and the pace of life in cities.
simple analysis. They took data about as many cities in the world
Proceedings of the national academy of
as possible. Then, they made a straightforward plot. They placed sciences, 104(17):7301–7306, 2007
the population of the city on the x-axis, and plotted a bunch of other 3
Andres Gomez-Lievano, HyeJin Youn,
and Luis MA Bettencourt. The statistics
variables in the y-axis.
of urban scaling and their connection to
Then they found a power relation between a city’s population and zipf’s law. PloS one, 7(7), 2012
its outcomes. Their plots looked like the ones I show in Figure 45.1.
 network science applications 625

1012 103
Figure 45.1: (a) Total wage sum
1011
(y axis) as a function of a city’s

# Gas Stations
2
10
Total Wage

1010 population (x-axis). (b) Num-

101 ber of gas stations (y axis) as a
109
function of a city’s population
108 4 100 4 (x-axis).
10 105 106 10 105 106
Population Population

(a) α ∼ 1.12 (b) α ∼ 0.77

When they looked at their α exponents, they discovered something

remarkable. The two plots in Figure 45.1 might seem identical, but
they differ in a crucial aspect: the value of the α exponent. Figure
45.1(a) has an α > 1, while Figure 45.1(b) has an α < 1. This is a
much bigger deal than you might think.
The authors found a consistent higher-than-one α for all wealth
creation and innovation activities in a city and, at the same time, a
consistent lower-than-one α for all activities accounting for infras-
tructure management. α > 1 means that each added individual to
the city contributes more than its fair share to the total. If you have a
city where each individual publishes a patent per year and you add
a new inhabitant to the city, you don’t get an additional patent that
year: you get that now each individual publishes 1.12 patents! Vice
versa, α < 1 is a classic “economies of scale” scenario: once you serve
100 people, you can serve an additional person without increasing
your effort by 1%. 4
Luís MA Bettencourt. The origins
of scaling in cities. science, 340(6139):
So far, this is not a network paper: it’s just a purely statistical 1438–1441, 2013
observation. It is when trying to explain such phenomena that you 5
Elsa Arcaute, Erez Hatna, Peter Fer-
find networks everywhere4 , 5 , 6 . Networks are a necessary ingredient guson, Hyejin Youn, Anders Johansson,
to explain why an additional node enriches the network in a non and Michael Batty. Constructing cities,
deconstructing scaling laws. Journal
linear way. Humans have limited resources, so they can only interact
of The Royal Society Interface, 12(102):
with what they can access. In network terms, these are the other 20140745, 2015
nodes at a maximum distance l from them. Every time you add a 6
Andres Gomez-Lievano, Oscar
Patterson-Lomba, and Ricardo Haus-
neighbor with new connections, lots of new nodes will get closer to mann. Explaining the prevalence,
you, some closer than l. scaling and variance of urban phe-
Consider Figure 45.2 as an example. Let’s assume that node u is nomena. Nature Energy, pages 1–9,
2018
selling something, and it has a range: it can only serve up until its
neighbors’ neighbors. Its original productivity is then 10. Then, node
v appears, and it connects to u. If v’s contribution were to be linear,
u’s productivity would go up to 11. But v has other neighbors of its
own, neighbors that were previously unreachable by u, as they were
at distance l = 3. Thus, u’s productivity jumps to 15!
This is a sort of combinatorial effect, where each node adds a
new factor you can use and recombine with all the factors that were
626 the atlas for the aspiring network scientist

Figure 45.2: An explanation

of non-linear node addition
contribution. The node color
u u
encodes the reachability of a
v
node from u: red = unreachable,
green = reachable.

(a) (b)
7
Hyejin Youn, Deborah Strumsky,
already present so far. This is easy to see especially in patents data7 . Luis MA Bettencourt, and José Lobo.
Every time someone makes a new invention, that new invention Invention as a combinatorial process:
evidence from us patents. Journal of The
can be combined with all the previous inventions to create a new
Royal Society Interface, 12(106):20150272,
one, and so on at infinity. Thus, the knowledge added by a new 2015
invention potentially multiplies itself with the previously accumulated
knowledge, rather than just adding to it.

45.2 Anonymity in the Age of Social Networks

We live in troubling times when it comes to our privacy. Large or-

ganizations have an interest in gathering information about each 8
Lars Backstrom, Cynthia Dwork,
individual, whether they do it for surveillance – as highlighted by and Jon Kleinberg. Wherefore art
thou r3579x?: anonymized social net-
Snowden’s leaks –, or for profit – Facebook tracking is ubiquitous, works, hidden patterns, and structural
but by no mean the exception in the private sector. The problem steganography. In Proceedings of the 16th
is that the simple usage of technology spreads an uncontrollable international conference on World Wide
Web, pages 181–190. ACM, 2007
amount of information about us: the simple sequences of queries 9
Latanya Sweeney. k-anonymity: A
you ask a search engine might be enough to identify you8 , and the model for protecting privacy. Interna-
combination of few elementary demographic pieces of data can tional Journal of Uncertainty, Fuzziness
and Knowledge-Based Systems, 10(05):
de-anonymize 80% of people9 .
557–570, 2002

Figure 45.3: A network in

which I label the identified
2 3
nodes.
1

4 5

If that worries you, consider that queries and demographics are

simple unconnected data. When you talk about interconnected
information, the problem is much worse. Consider what you see
 network science applications 627

10
Shirin Nilizadeh, Apu Kapadia, and
in Figure 45.3. The node 1 in the center of this network might be Yong-Yeol Ahn. Community-enhanced
you. If an attacker has identified some of your connections, they de-anonymization of online social
can say a lot about you10 : the only node in this network that has networks. In Proceedings of the 2014
acm sigsac conference on computer and
exactly {2, 3, 4, 5} as the set of their friends. They might not be able to communications security, pages 537–548,
know your name, but under the assumption of homophily (Chapter 2014
26) they could infer what you like, your sexual orientation, and
11
Elena Zheleva and Lise Getoor. To join
or not to join: the illusion of privacy in
maybe even health issues. This is not solved by making your profile social networks with mixed public and
private11 . In fact, even not having a profile at all on a social media private user profiles. In Proceedings of
the 18th international conference on World
won’t make you safe: the platform can always create a shadow profile wide web, pages 531–540, 2009
of you12 . 12
David Garcia. Leaking privacy
De-anonymizing social networks13 is feasible, under a wide array and shadow profiles in online social
networks. Science advances, 3(8):e1701172,
of different scenarios – whether the attacker is a government agency, 2017
a marketing campaign, or an individual stalker. This is usually done 13
Arvind Narayanan and Vitaly
by creating a certain amount of auxiliary information that can then Shmatikov. De-anonymizing social
networks. In 2009 30th IEEE symposium
be used to recursively de-anonymize more and more nodes in the on security and privacy, pages 173–187.
network. Counter-measures usually adopt the k-anonymity style: IEEE, 2009
making sure that no individual can be identified by obfuscating 14
Kun Liu and Evimaria Terzi. Towards
identity anonymization on graphs. In
enough data to make at least k − 1 other individuals identical to her in Proceedings of the 2008 ACM SIGMOD
some respect. For instance, a network is k-degree anonymous if there international conference on Management of
data, pages 93–106, 2008
are at least k nodes with any given degree value14 .
15
Elena Zheleva and Lise Getoor.
Sometimes, the focus is preventing the disclosure of information Preserving the privacy of sensitive rela-
about a relationship, i.e. to combat link re-identification15 . You might tionships in graph data. In International
not want Facebook to know you are friend with someone, which Workshop on Privacy, Security, and Trust
in KDD, pages 153–171. Springer, 2007
they could do by performing some relatively trivial link prediction 16
JW Scannell, GAPC Burns, CC Hilge-
– see Part VI. In those cases, you might want to hide some of your tag, MA O’Neil, and Malcolm P Young.
relationships, and/or add a few fake connections, to throw off the The connectional organization of the
cortico-thalamic system of the cat.
score function of the link you want to hide. Cerebral Cortex, 9(3):277–299, 1999
17
Quanxin Wang, Olaf Sporns, and
Andreas Burkhalter. Network analysis
45.3 Human Connectome of corticocortical connections reveals
ventral and dorsal processing streams
in mouse visual cortex. Journal of
Quite likely, the most famous and studied network in human history Neuroscience, 32(13):4386–4399, 2012b
is the brain. We have been studying neural networks of many ani- 18
Siming Li, Christopher M Armstrong,
mals, due to their limited size and ease of analysis: cats16 , mices17 , Nicolas Bertin, Hui Ge, Stuart Milstein,
Mike Boxem, Pierre-Olivier Vidalain,
and, of course, the superstar C. Elegans worm18 . However, most of Jing-Dong J Han, Alban Chesneau,
this is done with the big prize as the ultimate objective: the human Tong Hao, et al. A map of the inter-
brain. You might have heard of the Human Connectome Project. Pro- actome network of the metazoan c.
elegans. Science, 303(5657):540–543, 2004
posed in 200519 , its objective was to create a low-level network map 19
Olaf Sporns, Giulio Tononi, and Rolf
of the human brain: a network where nodes are individual neurons Kötter. The human connectome: a
and connections are the synapses between them. structural description of the human
brain. PLoS computational biology, 1(4),
The idea was that applying all the network science artillery to 2005
such a network would help us understanding better how our brains 20
Ed Bullmore and Olaf Sporns. Com-
work20 – or don’t, sometimes. In fact, one of the major lines of re- plex brain networks: graph theoretical
analysis of structural and functional
search is comparing the brain connection patterns between healthy systems. Nature reviews neuroscience, 10
and unhealthy individuals, because network analysis should be (3):186–198, 2009
628 the atlas for the aspiring network scientist

able to easily allow the identification of significant differences in 21

Danielle S Bassett and Edward T
the structures21 . For instance, as Figure 45.4 shows, a simple edge Bullmore. Human brain networks in
betweenness centrality analysis could identify the overload on some health and disease. Current opinion in
neurology, 22(4):340, 2009
synapses caused by structural differences.

Figure 45.4: The comparison

between (a) a healthy brain and
(b) an unhealthy brain. Nodes
are neurons, edges are synapses
and their thickness is propor-
(a) (b) tional to their edge betweenness
There have been many studies of the human brain before the centrality.
Human Connectome started. For instance, researchers have studied 22
Danielle S Bassett, Nicholas F Wymbs,
the effect of learning on the connections between brain areas22 . Mason A Porter, Peter J Mucha, Jean M
The reason why the Human Connectome is so revolutionary is the Carlson, and Scott T Grafton. Dynamic
reconfiguration of human brain net-
granularity of the data. Most brain network studies look at brain works during learning. Proceedings of
activity patterns: the high level difference in electrical potential of the National Academy of Sciences, 108(18):
7641–7646, 2011
brain areas. In this case, the nodes are not individual neurons, but
larger modules of the brain.
To be clear, one does not exclude the value of the other. In fact, the
brain is an extremely complex organ: it is the most important organ 23
According to the brain.
of your body23 . This means that it operates at multiple scales24 , in Richard F Betzel and Danielle S
24

a hierarchical fashion: neurons are part of modules25 , and there are Bassett. Multi-scale brain networks.
Neuroimage, 160:73–83, 2017
modules of modules, and so on – check out Chapters 29 and 33 for 25
Paolo Bonifazi, Miri Goldin, Michel A
a few refreshers on hierarchies. In fact, one of the most appropriate Picardo, Isabel Jorquera, A Cattani,
models of the brain is multilayer networks26 . Gregory Bianconi, Alfonso Represa,
Yehezkel Ben-Ari, and Rosa Cossart.
Gabaergic hub neurons orchestrate
synchrony in developing hippocampal
45.4 Science of Science and of Success networks. Science, 326(5958):1419–1424,
2009
Unsurprisingly, one of the things that interests scientists the most 26
Manlio De Domenico. Multilayer
modeling and analysis of human brain
is... scientists. Network scientists are no exception to this rule. There networks. Giga Science, 6(5):gix004, 2017
is a large and healthy literature in analyzing networks of scientists. 27
According to scientists.
We already saw many examples of two types of science networks: 28
Albert-László Barabási, Chaoming
co-authorship networks, where scientists are connected to each other Song, and Dashun Wang. Publishing:
if they collaborate on the same paper/project; and citation networks, Handful of papers dominates citation.
Nature, 491(7422):40, 2012
connecting papers if one cites another. 29
Dashun Wang, Chaoming Song, and
The two can be combined to try and gather a general picture of Albert-László Barabási. Quantifying
how science gets done. Science is one of the most important human long-term scientific impact. Science, 342
(6154):127–132, 2013
activities27 , because we rely on it to develop new and better ways to 30
Santo Fortunato, Carl T Bergstrom,
improve our everyday life. It’s better to understand how it works, so Katy Börner, James A Evans, Dirk
that we can do it better. This is fundamentally the mission statement Helbing, Staša Milojević, Alexander M
Petersen, Filippo Radicchi, Roberta
of the science of science field28 , 29 , 30 , kickstarted by network scientists Sinatra, Brian Uzzi, et al. Science of
and making extensive use of network analysis tools. science. Science, 359(6379):eaao0185, 2018
 network science applications 629

One of the most peculiar findings is that the occurrence of the

highest impact work of a scientist’s career will happen at a random 31
Roberta Sinatra, Dashun Wang,
point of it31 . In other words, there is no way to predict which of your Pierre Deville, Chaoming Song, and
papers will earn you a Nobel prize: it could be your first, it could be Albert-László Barabási. Quantifying the
evolution of individual scientific impact.
your last, or any in between. This is bad news if we want to predict
Science, 354(6312):aaf5239, 2016
the success of some research, but it’s great news for me. The fact that
I haven’t come even close to making a groundbreaking discovery
doesn’t mean it won’t happen eventually. I simply won’t see coming
if it does (it won’t).
Figure 45.5 shows an example of this concept. In both cases, the
breakout paper arrived early, but there is no pattern in how citations
come. Moreover, the red scientist (Figure 45.5(a)) is a better scientist
on average than the blue one (Figure 45.5(b)), having 23.5 citations
per paper against blue’s 16.2. They’re also more productive (24
vs 18 papers). And yet, it is the blue scientists who published the
best paper – with 223 citations, while red’s best paper only has 115
citations. Life is unfair this way.

103 103
Figure 45.5: Two examples of
10
2
10
2 career paths of scientists, show-
# Citations

# Citations

ing the number of citations

101 101 (y axis) gathered from papers
published in a given year (x
0 0
10
0 5 10 15 20 25 30 35
10
0 5 10 15 20 25 30 35
axis).
Year Year

(a) (b)

Science of science ended up being a specialized niche of the more

broad field of the science of success: the systematic investigation 32
Samuel P Fraiberger, Roberta Sinatra,
of the gap between one’s performance and their success32 , 33 . It is Magnus Resch, Christoph Riedl, and
Albert-László Barabási. Quantifying
not always the best work of a person that ends up being the most reputation and success in art. Science,
successful. For instance, the Mona Lisa, the most famous painting 362(6416):825–829, 2018
of the world, is a masterpiece from one of the greatest intellectuals
33
Lu Liu, Yang Wang, Roberta Sinatra,
C Lee Giles, Chaoming Song, and
of all time – Leonardo da Vinci – but, among its other breathtaking Dashun Wang. Hot streaks in artistic,
creations, it is rather unremarkable. So unremarkable, in fact, that cultural, and scientific careers. Nature,
559(7714):396, 2018a
it was completely ignored and not even exposed until interest in it
exploded after its theft.
This disconnect between performance and success is not an ex-
clusive domain of art. It is a much more universal phenomenon. 34
Burcu Yucesoy and Albert-László
Another studied example is tennis34 : it is not necessarily the tennis Barabási. Untangling performance from
success. EPJ Data Science, 5(1):17, 2016
player at the top of the world ranking the one gathering the most
Wikipedia page views – or news articles about them, for that matter.
In fact, the performance-success disconnect can and should be
630 the atlas for the aspiring network scientist

applied to science as well. In this section, I equated “success” with

citations: a successful paper gathers tons of citations. But is it the
best (read: highest performing) paper? Not at all! Citations and
grant awards correlate with things that are independent of the sci- 35
Jonathan R Cole. Fair science: Women
ence/performance itself (e.g., gender35 , race36 and how junior a in the scientific community. 1979
person is37 ). The world isn’t a perfect meritocracy. Cumulative 36
Donna K Ginther, Walter T Schaffer,
advantage is not just the pretty story of how you model broad de- Joshua Schnell, Beth Masimore, Faye
Liu, Laurel L Haak, and Raynard
gree distributions in networks (Section 14.3): it is the real unfair- Kington. Race, ethnicity, and nih
ness in front of everybody who does not start in the advantaged research awards. Science, 333(6045):
1015–1019, 2011
place/time/gender/race. We should investigate the performance- 37
Robert T Blackburn, Charles E
success disconnect in order to make the world suck a little less. One Behymer, and David E Hall. Research
way to do it is to model science as the interaction between individual note: Correlates of faculty publications.
Sociology of Education, pages 132–141,
characteristics and systemic structures38 .
1978
38
Samuel F Way, Allison C Morgan,
Daniel B Larremore, and Aaron Clauset.
45.5 Human Mobility Productivity, prominence, and the
effects of academic environment.
A significant portion of network scientists have also worked on issues Proceedings of the National Academy of
Sciences, 116(22):10729–10733, 2019
of human mobility: describing and predicting how individuals and 39
Marta C Gonzalez, Cesar A Hidalgo,
collectives move in the urban and global landscape39 , 40 . There are and Albert-Laszlo Barabasi. Understand-
a few reasons for this. First, there is a strong connection between ing individual human mobility patterns.
nature, 453(7196):779–782, 2008
human mobility and many networked phenomena that network 40
Julián Candia, Marta C González,
scientists investigate. Just to highlight the example from the previous Pu Wang, Timothy Schoenharl, Greg
sections: one can use the “mobility” of scientists between affiliations Madey, and Albert-László Barabási.
Uncovering individual and collective
to predict their success41 . Alternatively, one can use mobility data to human dynamics from mobile phone
augment the de-anonymization process of people in social settings42 , records. Journal of physics A: mathematical
and theoretical, 41(22):224015, 2008
or to better predict the spread of infectious diseases43 , 44 , 45 . 41
Pierre Deville, Dashun Wang, Roberta
Second, complex networks are themselves useful tools to model Sinatra, Chaoming Song, Vincent D
and analyze mobility patterns. For instance, one can create a better Blondel, and Albert-László Barabási.
Career on the move: Geography,
synthetic model of human mobility by using an underlying social stratification, and scientific impact.
network to create realistic motivations for the simulated agents to Scientific reports, 4:4770, 2014
move in space46 . 42
Yves-Alexandre De Montjoye, César A
Hidalgo, Michel Verleysen, and Vin-
Classically, to predict the number of people moving from area cent D Blondel. Unique in the crowd:
A to area B, one would use a “gravity model”. This works just like The privacy bounds of human mobility.
Newton’s gravity law: the mobility relation between two areas is Scientific reports, 3:1376, 2013
43
Vittoria Colizza, Alain Barrat, Marc
directly proportional to how many people live in them (their “mass”) Barthelemy, Alain-Jacques Valleron,
and inversely proportional to their distance47 . In other words, there and Alessandro Vespignani. Modeling
can be many people moving between New York and Chicago because the worldwide spread of pandemic in-
fluenza: baseline case and containment
they are huge cities, but Boston might attract more Newyorkers interventions. PLoS medicine, 4(1), 2007
despite being less populous, simply because it’s closer. The gravity 44
Duygu Balcan, Vittoria Colizza, Bruno
model is overly simplistic: it’s deterministic, it requires previous Gonçalves, Hao Hu, José J Ramasco,
and Alessandro Vespignani. Multiscale
mobility data to fit parameters, it lacks theoretical grounding, and mobility networks and the spatial
it simply doesn’t predict observations that well. Network scientists spreading of infectious diseases. PNAS,
106(51):21484–21489, 2009
have then developed a radiance model to fix these shortcomings48 .
What do we find? At a collective level, human mobility patterns
 network science applications 631

Figure 45.6: (a) Probability of

a trip (y axis) as a function of
PDF

PDF
the distance to the destination
(x axis). (b) Probability of a
trip (y axis) as a function of
the destination’s rank (x axis).
Distance Rank In both cases, different colors
(a) (b) report data from different cities.

are surprisingly universal, but not when it comes to the covered 45

Michele Tizzoni, Paolo Bajardi,
distance49 . Figure 45.6(a) shows that the probability of making a Adeline Decuyper, Guillaume Kon Kam
King, Christian M Schneider, Vincent
trip is only mildly related to distance: there is no function properly Blondel, Zbigniew Smoreda, Marta C
approximating the likelihood of you visiting a place given its distance González, and Vittoria Colizza. On
the use of human mobility proxies for
to you, and different cities have different scaling and cutoffs. In other modeling epidemics. PLoS computational
words, it is not true that the farther apart a pizza place is, the least biology, 10(7), 2014
you go to eat there. 46
Mirco Musolesi and Cecilia Mascolo.
A community based mobility model for
It is rather that how frequently you go to eat there is connected to ad hoc network research. In Proceedings
the number – and quality – of the alternatives in between you and of the 2nd international workshop on Multi-
the pizza place. This is only correlated with, rather than being caused hop ad hoc networks: from theory to reality,
pages 31–38, 2006
by, distance. What matters most, is the rank of the place in your 47
Dirk Brockmann, Lars Hufnagel, and
preferences. Figure 45.6(b) shows that there is a clear and universal Theo Geisel. The scaling laws of human
function predicting a trip’s probability given the popularity rank of travel. Nature, 439(7075):462–465, 2006
48
Filippo Simini, Marta C González,
the destination – e.g. no matter the city, 20% of trips go to the most Amos Maritan, and Albert-László
popular destination. Barabási. A universal model for mobility
The predictability of the collective dynamics, however, does not and migration patterns. Nature, 484
(7392):96–100, 2012
trickle down to predictability of individuals. Yes, we’re animals of 49
Anastasios Noulas, Salvatore Scellato,
habit: we often commute between the same two places – a property Renaud Lambiotte, Massimiliano Pontil,
one can exploit to infer home and work locations by looking at in- and Cecilia Mascolo. A tale of many
cities: universal patterns in human
complete mobility data50 , 51 . On average, one can confidently predict urban mobility. PloS one, 7(5), 2012
93% of individual mobility52 . However, there is a large variation be- 50
Md Shahadat Iqbal, Charisma F
tween individuals: for some you could predict even better than that, Choudhury, Pu Wang, and Marta C
González. Development of origin–
while others are fundamentally unpredictable53 . destination matrices using mobile
phone call data. Transportation Research
Part C: Emerging Technologies, 40:63–74,
2014
45.6 Memetics 51
Lauren Alexander, Shan Jiang, Mikel
Murga, and Marta C González. Origin–
Who around here doesn’t like Internet memes? Cute and funny little destination trips by purpose and time
of day inferred from mobile phone data.
pictures, perfect to waste time at work. Of course network scientists Transportation research part c: emerging
love them. However, we need to maintain appearances and pretend technologies, 58:240–250, 2015
that the penguin image we have on our screens is there really for 52
Chaoming Song, Zehui Qu, Nicholas
Blumm, and Albert-László Barabási.
work. We’re studying memes, you know? This is for science. There Limits of predictability in human
are a few angles with which network scientists attack the study of mobility. Science, 327(5968):1018–1021,
memes. Some of those already found space elsewhere in the book – 2010

e.g. in Section 18.2.

632 the atlas for the aspiring network scientist

The first is the relationship between the network structure and 53

Luca Pappalardo, Filippo Simini,
the probability of a rumor to spread – or the fraction of nodes who Salvatore Rinzivillo, Dino Pedreschi,
Fosca Giannotti, and Albert-László
will end up hearing a rumor. Theoretical calculations54 show the Barabási. Returners and explorers
impact of the network’s topology: in a random graph, initial spread dichotomy in human mobility. Nature
communications, 6:8166, 2015
is slow but it will relentlessly cover the entire network; while for 54
Maziar Nekovee, Yamir Moreno,
scale free networks the initial speed is fast but, in presence of degree Ginestra Bianconi, and Matteo Marsili.
correlations, it might fail to cover the entire network. All of this is Theory of rumour spreading in complex
social networks. Physica A: Statistical
very similar to simulations of diseases spreading on a network (see Mechanics and its Applications, 374(1):
Chapter 17). 457–470, 2007
Other studies show how the large diversity in the meme success 55
Nathan Oken Hodas and Kristina
distribution – few memes spread globally while most are immedi- Lerman. How visibility and divided
ately forgot – are due to the limited capacity of brains to process attention constrain social contagion. In
SocialCom, pages 249–257. IEEE, 2012
information55 , 56 , 57 . Another key question is whether memes spread 56
Lilian Weng, Alessandro Flammini,
following simple or complex contagion: is a single exposure sufficient Alessandro Vespignani, and Fillipo
or does reinforcement play a significant role? It seems that memes Menczer. Competition among memes in
a world with limited attention. Scientific
indeed obey the complex contagion rules58 . For a refresher on the
reports, 2:335, 2012
concepts, see Chapter 18.

Figure 45.7: A social network.

The red and blue nodes are the
origin points of two memes.

57
James P Gleeson, Jonathan A Ward,
Kevin P O’sullivan, and William T Lee.
Competition-induced criticality in a
model of meme popularity. Physical
review letters, 112(4):048701, 2014
More complex topological features, such as communities, are 58
Bjarke Mønsted, Piotr Sapieżyński,
difficult to treat mathematically, but their impact can be studied Emilio Ferrara, and Sune Lehmann.
using real world data. Figure 45.7 shows a toy example of the role of Evidence of complex contagion of infor-
mation in social media: An experiment
communities in meme propagation. Memes originating in the overlap using twitter bots. PloS one, 12(9), 2017
between different communities – in red in Figure 45.7 – have a better 59
Lilian Weng, Filippo Menczer, and
chance to go viral59 , 60 . Being born well embedded in a community Yong-Yeol Ahn. Virality prediction and
community structure in social networks.
– blue in Figure 45.7 – is bad for propagation, because there are not Scientific reports, 3:2522, 2013
many paths leading the meme outside of the community. 60
Lilian Weng, Filippo Menczer, and
In general, there are many empirical studies investigating how Yong-Yeol Ahn. Predicting successful
memes using network and community
information propagates through a social network61 , be it memes, structure. In ICWSM, 2014
rumors, news62 , videos63 , 64 , or photographs65 , 66 . 61
Kristina Lerman and Rumi Ghosh.
Specifically, one study focuses on the dynamics of “following” a Information contagion: An empirical
study of the spread of news on digg
content creator on social media67 . The common sense thing is that and twitter social networks. In ICWSM,
the more people are following you – say on Twitter – the better it is. 2010
You can be more influential if more people listen to you. However,
62
Soroush Vosoughi, Deb Roy, and
Sinan Aral. The spread of true and
experiments show that this is true only to a certain point. What false news online. Science, 359(6380):
matter most is the engagement of the followers. Just increasing the 1146–1151, 2018
 network science applications 633

count is doing you no good if the people clicking on the following 63

Meeyoung Cha, Haewoon Kwak,
button actually don’t read what you produce. Your voice is diluted Pablo Rodriguez, Yong-Yeol Ahn, and
Sue Moon. I tube, you tube, everybody
on the platform and you have less reach if you inflate those numbers. tubes: analyzing the world’s largest
The structure of the social network is not, however, the only thing user generated content video system. In
SIGCOMM, pages 1–14, 2007
that matters. I already mentioned elsewhere in the book (in Section 64
Meeyoung Cha, Haewoon Kwak,
18.2) that an important factor is also timing: when and how fast you Pablo Rodriguez, Yong-Yeol Ahn,
get your appreciation matters a lot in determining whether you are and Sue Moon. Analyzing the video
popularity characteristics of large-
going viral. Alternatively, one could look at the content itself of the scale user generated content systems.
meme: the specific image or text associated to it. Studies show how IEEE/ACM Transactions on networking, 17
positive valence – a happy meme – are useful for propagation68 . In (5):1357–1370, 2009a
65
Meeyoung Cha, Alan Mislove, Ben
my own research, I instead show how innovation is the key: you Adams, and Krishna P Gummadi.
want to do something that is dissimilar from everything that has been Characterizing social cascades in flickr.
done before69 , 70 . In Proceedings of the first workshop on
Online social networks, pages 13–18, 2008
66
Meeyoung Cha, Alan Mislove, and
Krishna P Gummadi. A measurement-
45.7 Digital Humanities driven analysis of information propa-
gation in the flickr social network. In
WWW, pages 721–730, 2009b
Digital humanities is an umbrella term, covering a vast set of appli- 67
Meeyoung Cha, Hamed Haddadi,
cations of computational tools to disciplines in the humanities. As Fabricio Benevenuto, and Krishna P
a trained digital humanist myself, I cannot end this book without Gummadi. Measuring user influence in
twitter: The million follower fallacy. In
taking a closer look at this field. And, in a sense, I wasn’t, because ICWSM, 2010
one could argue that the entire network analysis field is one of the 68
Jonah Berger and Katherine L Milk-
largest subfield of digital humanities. In network analysis we have man. What makes online content
viral? Journal of marketing research, 49(2):
mathematical and computational models – graphs and networks – 192–205, 2012
which are primarily applied to understand social systems. However, 69
Michele Coscia. Average is boring:
among the gigantic bazaar of network science applications, some How similarity kills a meme’s success.
Scientific reports, 4:6477, 2014
stand out as poster children of digital humanities. 70
Michele Coscia. Popularity spikes hurt
One is for sure the study of the birth-death network across cultural future chances for viral propagation of
history71 . Figure 45.8 shows an interesting historical pattern: each protomemes. Communications of the ACM,
61(1):70–77, 2017
point is a city, and for each city we count the number of famous peo- 71
Maximilian Schich, Chaoming Song,
ple who were born and who died in that city. In red we can see death Yong-Yeol Ahn, Alexander Mirsky,
attractors: cities who had more famous deaths than births. In blue Mauro Martino, Albert-László Barabási,
and Dirk Helbing. A network framework
we have the emitting cities. By analyzing the historical trajectories of of cultural history. science, 345(6196):
cities, we can see how the cultural center of the world moved from 558–562, 2014

104

103
Total Deaths

Figure 45.8: The number of

102
famous people who was born (x
101 axis) and died (y axis) in a city.
The identity line is in gray. Red
100 1
10 10
2 3
10 points above the identity line
Total Births and blue points below.
634 the atlas for the aspiring network scientist

Rome to Paris and then to New York, because more and more people
die in the city where they work – and notable people work where
most notable people are. There are other interesting patterns, for in-
stance the fact that the median distance between the birth and death
place is increasing, reflecting technological advancements.
With a similar dataset, researchers built the “notable people portfo- 72
Amy Zhao Yu, Shahar Ronen, Kevin
lio” of cities and nations72 . The idea is to classify all famous people Hu, Tiffany Lu, and César A Hidalgo.
in the area they contributed the most to humanity. Then, one can Pantheon 1.0, a manually verified
dataset of globally famous biographies.
visualize in which areas places specialize73 . For instance, the largest
Scientific data, 3:150075, 2016
profession represented in the United States is actors, while it is 73
[Link]
politicians for Greece. But one could explore other dimensions. For
instance, professions that are over-expressed in a country against the
rest of the world, like chess players in Armenia (6% of all famous
people!). Or explore gender divide: in Canada 27.4% of male famous
people were actors against 55.4% female famous people. Finally, you
can explore time as well. Before 1700 AD, the most common way
to become famous in Italy was to have a career in politics (30.7% of 74
Tom Brughmans. Thinking through
networks: a review of formal network
famous people did). Afterward? You’re better off trying as a soccer methods in archaeology. Journal of
player (21.6%). Archaeological Method and Theory, 20(4):
Other digital humanities applications of network science involve 623–662, 2013
75
Tom Brughmans. Connecting the
archaeology. This mostly involves the use of network visualization dots: towards archaeological network
techniques to make sense of a complex, interconnected, and often analysis. Oxford Journal of Archaeology, 29
largely incomplete set of evidence74 . However, it is not necessary (3):277–303, 2010
76
Barbara J Mills, Jeffery J Clark,
to limit ourselves to this: network analysis can be used as a tool to Matthew A Peeples, W Randall Haas,
explore evidence. For instance, there are studies of social networks John M Roberts, J Brett Hill, Deborah L
in classical Rome75 . Departing from archaeology, the field of social Huntley, Lewis Borck, Ronald L Breiger,
Aaron Clauset, et al. Transformation of
network analysis in a historic76 , religious77 , or anthropological78 social networks in the late pre-hispanic
setting is alive and well. us southwest. Proceedings of the National
Academy of Sciences, 110(15):5785–5790,
And since network analysis endows us with powerful tools to 2013
study hidden preferences – such as homophily and segregation, see 77
Eleanor A Power. Discerning devotion:
Chapter 26 – it is a natural instrument to use in other humanities Testing the signaling theory of religion.
Evolution and Human Behavior, 38(1):
fields, such as gender studies. In particular, there are studies showing 82–91, 2017
unequal gender dynamics when it comes to power relations in online 78
Jessica C Flack, Michelle Girvan,
collaborative tools such as, e.g., Wikipedia. As you might expect, the Frans BM De Waal, and David C
Krakauer. Policing stabilizes construc-
majority of Wikipedia contributors are white men. tion of social niches in primates. Nature,
When it comes to female representation in the content79 , 80 , this 439(7075):426–429, 2006
gender gap shows. The researchers find that women are equally 79
Claudia Wagner, David Garcia,
Mohsen Jadidi, and Markus Strohmaier.
represented in article numbers – at least in the main six language It’s a man’s wikipedia? assessing
editions of Wikipedia. However, it is the way women are portrayed gender inequality in an online ency-
clopedia. In Ninth international AAAI
that is the problem. Women on Wikipedia tend to be more linked to
conference on web and social media, 2015
men than vice versa. Moreover, romantic relationships and family- 80
Claudia Wagner, Eduardo Graells-
related issues are much more frequently discussed on Wikipedia Garrido, David Garcia, and Filippo
articles about women than men. Menczer. Women through the glass ceil-
ing: gender asymmetries in wikipedia.
EPJ Data Science, 5(1):5, 2016
46
Data & Tools

A professional worker is only as good as the tools they have and

their mastery of them. Thus, knowing where to find the best tools is
the first necessary step for being a good network scientist. The tools
aren’t going to do the job for you, but without them you’re just a
person armed with lots of good intentions. The aim of this chapter is
to kickstart you to your career. I will give you a brief overview of the
software libraries and programs one can use to analyze and visualize
networks, point you to useful online resources – especially to find
new data sources –, and briefly discuss some of the most famous
graph data you will find in the literature.
You might have already seen some of these resources here and
there mentioned throughout the book. For instance, the vast majority
of the exercises rely on you using Networkx, while in Part XII I
heavily relied on the knowledge of what Cytoscape and Gephi can do
for network visualization.

46.1 Libraries

It might be my bias as a computer scientist showing, but my opin-

ion is that, if you want to have a career in network science, the first
thing you have to look at is libraries that help you programming your
custom network analyses. There are many fully fledged software pro-
grams, with their graphical interfaces and ready-to-use implemented
analyses, but I don’t think you’re really going to be a complete net-
work scientist if you only rely on them. At some point, you will find
out something you cannot do with them, and you’ll need to roll up
your sleeves and get your hands dirty with programming. If you start
by learning the libraries, instead, you can always have the option of
lazily use another software for all the trivial tasks that don’t need any
specific customized contribution.
636 the atlas for the aspiring network scientist

Networkx
1
Aric Hagberg, Pieter Swart, and Daniel
I start by dealing with Networkx1 , 2 . Networkx is a Python library S Chult. Exploring network structure,
implementing a vast array of network algorithms and analyses. Net- dynamics, and function using networkx.
Technical report, Los Alamos National
workx is – as far as I can tell – the most popular choice for students Lab.(LANL), Los Alamos, NM (United
approaching network analysis tasks. It isn’t particularly good at States), 2008
2
anything, and it’s by far the library struggling with computational [Link]

efficiency the most, but it is the most complete and popular. If this
were a chapter about cinema, Networkx would be Steven Spielberg:
everybody knows him, every movie he makes is good but not really
great, and it’s the most boring possible choice as a favorite director.

3
10
iGraph Figure 46.1: The running times
Networkx
2 for different implementations
Running Time (s)

10
of community discovery algo-
101 rithms in Networkx (red) and
iGraph (blue).
100

10-1
LPA FG
Algorithm

As I mentioned, the biggest issue of Networkx is efficiency. Net-

workx is mostly implemented in Python, and it’s so large it is impos-
sible for the maintainers to guarantee high code standards through-
out the library. Thus you might end up waiting for hours – or days! –
for an operation that would take other libraries few seconds to com-
plete. For instance, Figure 46.1 compares the running times – on the
same network with ∼ 100k nodes and on the same machine – of the
label propagation and fast greedy modularity community discovery
algorithms between Networkx and iGraph. Note the logarithmic
scale on the y axis, and despair. Thus, for any analysis with a high
time complexity – for instance frequent pattern mining – you should
definitely look somewhere else.
Another downside I stumbled upon is buggyness. You will often
find that the most obscure network functions are sometimes not
implemented correctly. I found myself having to switch library
because the graph isomorphism functions on labeled multigraphs
were just returning the wrong results. This is to be expected for a
vast library like this: bugs take time to be noticed, and their fixes to
be incorporated.
That is not to say that things aren’t improving. I hope that the
bug I stumbled upon is now corrected. And other examples of slow
implementations are actually on par with state of the art implemen-
 data & tools 637

tation. It used to take impossibly long to generate LFR benchmarks

on Networkx, but when I checked on the current version at the time
of writing this chapter, I noticed no runtime difference with the
C binary provided by the original authors of the paper. Kudos to
Networkx on this!
Networkx comes with big strengths as well. First, it is very
pythonic, which means intuitive and easy to use – well, at least to
me and to all who are comfortable with Python’s style of doing stuff.
Second, as mentioned, it has really a broad coverage. You can tell
this was a tool made by network scientists. The array of functions
included in Networkx has no peers in any other library that I know.
Finally, it has a relatively decent ecosystem. Of course, not everything
can be implemented in Networkx. The developers need to choose
what to focus on. But it is a tool on which it is relatively easy to build.
Thus you can easily find packages expanding Networkx’s capabilities.
For instance, Networkx doesn’t include the popular Louvain commu-
nity discovery method (Section 33.1), but you can easily find it and 3
[Link]
plug it in3 . [Link]/

Graph Tool

If you want to stay in the domain of Python, the obvious alternative 4

Tiago P Peixoto. The graph-tool python
library. figshare, 2014b
to Networkx is Graph Tool4 , 5 by Tiago Peixoto. Graph Tool is, to 5
[Link]
some extent, the opposite of Networkx in almost every respect. For
this reason, it represents a perfect complementary tool.
Graph Tool has several weaknesses. Many are connected to its
strengths. For instance, one major strength of Graph Tool is its effi-
ciency: it is really fast in computing almost anything. This is due to
the fact that it is one of the very few libraries I know that actually
has parallel implementations of the network algorithms. This means
that, if you are on a machine with multiple cores – which is to say,
your computer is not older than ten years –, your analyses are going
to run much faster because each of your cores will be involved in
the computation. This comes at the downside of making Graph Tool
a nightmare to install. I have had students who had to give up on
some parts of their projects because they could not install Graph Tool.
Running the command sudo pip install graph-tool might results
in unpleasant surprises.
The other weakness is its incompleteness. Graph Tool is nowhere
near Networkx when it comes to offering network analysis tools.
This is due to the fact that Graph Tool is practically a one man show.
Tiago has made a godly amount of work for one person, but he is
still one human. On the other hand, this is linked to a strength as
well. There might not be many functions implemented in Graph Tool,
638 the atlas for the aspiring network scientist

but the ones that are there benefit from having a single mind behind
them. The aforementioned issue I had with the bugs in labeled
multigraph isomorphism was solved by simply using Graph Tool.
Moreover, getting into Tiago’s frame of mind is necessary to use
Graph Tool. You need to understand the way he does things in order
to be able to do them as well. Things like function naming, object
types, parameter passing – what you call the interface of the library –
are not as pythonic and intuitive as in Networkx.

Figure 46.2: A network visual-

ization generated with Graph
Tool.

The final strength of Graph Tool is in visualization. I typically use

other programs to visualize graphs, but it is undeniable that Graph
Tool is lightyears ahead any other library you can think of. Figure
46.2 is an example of what you can do with only minimal effort.
If you want to stay in Python and have performance and a bundle
of graph utilities – rather than a hand-holding library who thinks for
you, but also limits you – Graph Tool is the way to go. Graph Tool is,
in my movie director analogy, Werner Herzog. Extremely prolific and
productive, but you already know that everything you’re going to see
will be heavily influenced by his charming accent.

iGraph
6
Gabor Csardi and Tamas Nepusz. The
Among all the alternatives, iGraph6 , 7 is certainly the most versatile igraph software package for complex
tool. It combines the strengths – and weaknesses – of Networkx and network research. InterJournal, Complex
Systems, 1695(5):1–9, 2006
Graph Tool. On the one hand, it is a surprisingly complete tool with 7
[Link]
lots of implemented functions – just like Networkx –, and it is pretty
efficiently written – like Graph Tool. Other advantages reside in the
fact that the library is available on a vast array of platforms: you can
use it both in Python and in R. You can even import it directly as a
C library. Thus, if you are capable of writing in C, you can probably
cook up a customized analysis using the power of iGraph that cannot
 data & tools 639

be beaten in terms of running time.

That said, iGraph is not the be all end all of network analysis. As
I mentioned, it is available on R. In fact, I’d venture the guess that it
was developed primarily for R. And I am personally incompatible
with R – some people love it, others like me cannot really understand
it. To me, the interface of the library makes no sense. Function names,
parameter passing, how things are stored and retrieved from objects:
it is all in R style, which my brain unfortunately translates to “in-
comprehensible randomness”. The fact that it is possible to import
iGraph in Python should not fool you: it is not pythonic at all, and
the Python code you end up writing while using iGraph doesn’t even
look like Python (for some it is a plus, but not for me). You’ve been
warned.
A note of merit to the documentation. All documentations are
bad. The relationship between what’s written in Graph Tool’s docu-
mentation and how you actually do things in Graph Tool is flimsy in
the best of days. But only iGraph’s documentation contains timeless
comedic gems such as:

layout_on_sphere places the vertices (approximately) uniformly on

the surface of a sphere, this is thus a 3d layout. It is not clear however
what “uniformly on a sphere” means.

Gee, thank you, it’s refreshing to see that not even who developed
this function knows what the function is doing. Continuing my
movie directors analogy, iGraph is David Lynch: probably the only
one able to do what he is doing, but good luck knowing what’s going
on when you look at something made by him.
The fact that I’m badmouthing iGraph so hard should really
convince you that it is a fundamental tool. I hate it with a passion
and yet I am including it in the book and I use it. If I could live
without it – trust me – I would. But I can’t, because sometimes it is
the only thing that will save you.

Other
The libraries I presented in this section are the three horsemen of
network analysis: most of the times, if you need a library, one of
them will cover you. That is not to say they are the only things you
should know. Here I group a bunch of miscellanea that could come
in handy sooner of later. 8
Jure Leskovec and Rok Sosič. Snap: A
There are a few competing libraries for doing network analysis. general-purpose network analysis and
graph-mining library. ACM Transactions
Stanford Network Analysis Project8 , 9 (SNAP) comes to mind. This on Intelligent Systems and Technology
is another C++ library, thus it competes more directly with iGraph. (TIST), 8(1):1–20, 2016
It is owned by a research group at Stanford, which is both a blessing 9
[Link]
640 the atlas for the aspiring network scientist

and a curse. It contains mostly implementations of the analyses de-

veloped in that research group, which are great and extremely useful.
This, however, comes to the cost of completeness. It is excessively
focused.
Then there are packages that provide specialized utilities. There
are probably more out there than I can count, so I’m making only
one example to make you aware of the wealth of useful things that 10
Jeff Alstott and Dietmar Plenz Bull-
you could find. The powerlaw package10 , 11 allows you to perform more. powerlaw: a python package for
statistical testing to verify whether your degree distribution is really analysis of heavy-tailed distributions.
PloS one, 9(1), 2014
a power law and cannot be explained by more mundane generating 11
[Link]
processes. It is what we rely on in Section 6.4. powerlaw
Other notable libraries include:
12
Giulio Rossetti, Letizia Milli, and
• CDlib12 , 13 , specializing on implementing community discovery Rémy Cazabet. Cdlib: a python library
algorithms. It includes more than 50 of them, plus around 30 to extract, compare and evaluate
communities from complex networks.
quality functions you can use to evaluate them (Chapter 32); Applied Network Science, 4(1):52, 2019
• NDlib14 , 15 , focusing on models of epidemics/spreading events on 13
[Link]
14
Giulio Rossetti, Letizia Milli, Salvatore
networks (Part V);
Rinzivillo, Alina Sîrbu, Dino Pedreschi,
• DyNetX16 , which extends networkx adding to it the ability of and Fosca Giannotti. Ndlib: a python
library to model and analyze diffusion
dealing with temporal/dynamic networks. processes over complex networks.
International Journal of Data Science and
The final piece of advice I give you is to remember that networks Analytics, 5(1):61–79, 2018
15
and graphs are, at the end of the day, matrices. I try to ignore this [Link]
16
[Link]
fact as much as I can, but it is an undeniable truth. Sometimes, the
best thing you can do is to treat them as such, and to start doing
some good old linear algebra. Thus, if you are using Python, you can- 17
Pauli Virtanen, Ralf Gommers,
not live without learning at least the basics of Numpy and Scipy17 , 18 , Travis E Oliphant, Matt Haberland,
especially when it comes to use sparse matrices. Pandas19 is a good Tyler Reddy, David Cournapeau, Evgeni
Burovski, Pearu Peterson, Warren
tool as well, because you can use it to pivot effortlessly between Weckesser, Jonathan Bright, et al.
dealing with networks as edgelists and as matrices. Networkx can Scipy 1.0: fundamental algorithms for
scientific computing in python. Nature
convert to and from its data structures into Numpy, Scipy, and Pan-
methods, pages 1–12, 2020
das. 18
[Link]
Which means that your toolbox can include specialized software 19
[Link]
like Matlab or Octave. There are in fact, network scientists who are
able to do everything they need to do exclusively in these program-
ming environments. I always look at them in awe, not knowing if I
do so out of being fascinated or terrified by them.

46.2 Software

So far, we’ve seen software libraries: additional packages for pro-

graming languages that you can use to code your own solutions to
network analysis problems. Sometimes, you instead want a fully-
fledged software, possibly with a graphical user interface, to operate
 data & tools 641

a set of standard operations. This is what we deal with in this sec-

tion.

For Visualization 20
Paul Shannon, Andrew Markiel,
Owen Ozier, Nitin S Baliga, Jonathan T
By far, the software I use the most is Cytoscape20 , 21 . Mostly, I use it Wang, Daniel Ramage, Nada Amin,
for network visualization. The visual style of Cytoscape is based on Benno Schwikowski, and Trey Ideker.
the Protovis Java library22 , which is one of the ancestors of D323 , 24 – Cytoscape: a software environment
for integrated models of biomolecular
and it shows. The visual style of Cytoscape is really good, and you interaction networks. Genome research, 13
can customize a large quantity of visual attributes relatively easily. (11):2498–2504, 2003
21
[Link]
Cytoscape supports some basic network analysis. You can calcu- 22
Michael Bostock and Jeffrey Heer.
late a bunch of node, edge, and network statistics, the ones you’d Protovis: A graphical toolkit for visual-
come up first in your exploratory data analysis phase – nothing too ization. IEEE transactions on visualization
and computer graphics, 15(6):1121–1128,
fancy. This analytic capability is mostly there only to allow you to use
2009
node and edge statistical properties to augment your visualization. 23
Michael Bostock, Vadim Ogievetsky,
Since version 3.8, Cytoscape shows fewer plots. For instance you and Jeffrey Heer. D3 data-driven
documents. IEEE transactions on
cannot see any more the distribution of shortest path lengths. More-
visualization and computer graphics, 17
over, I can’t seem to be able to show them in a log-log scale – it was (12):2301–2309, 2011
possible before. However, the graphical quality of the plots greatly 24
[Link]

improved, and now they are interactive, allowing you to manipulate

them to select nodes in the networks.
Another major strength of Cytoscape is its good selection of plug-
ins. Just like Networkx, it supports a healthy ecosystems of contrib-
utors. The Cytoscape community skews heavily on the biological
network crowd: protein-protein networks, gene interactions, and
the like. Thus, if you’re part of that community, you will likely find
everything you need for Cytoscape. If you’re not part of that commu-
nity, the coverage is spotty at best.
Cytoscape comes with an API interface and a console. This means
that you can write your customized piece of code that can use Cy-
toscape as a utility to augment your visualizations. I haven’t tested
it myself, but it is a nice option to have, if you’re the tinkerer kind
of analyst. Moreover, Cytoscape understands relatively advanced 25
Ulrik Brandes, Markus Eiglsperger,
Jürgen Lerner, and Christian Pich. Graph
graph file formats such as GraphML25 and XGMML26 . These are markup language (GraphML). 2013
XML dialects specifically developed for graphs. They allow you to 26
John Punin and Mukkai Krishnamoor-
store a comprehensive list of node and edge attributes, which you thy. Xgmml (extensible graph markup
and modeling language), 2001
can use to directly encode how your graph is supposed to look like.
This means that you can transport your Cytoscape visualizations to
any other software which understands them.
Finally, on the downsides, two more quick notes. First, it is a
bit annoying to install, because at any given time it will rest on the
previous long term support Java version (at the time of writing
it is Java 11, but for most of 2020 you needed Java 8, which came
out in 2014 – i.e. you were running 6 year old code). Second, I find
642 the atlas for the aspiring network scientist

Cytoscape to be a rather buggy piece of software, with random

mouse focus fails when selecting/editing text/nodes. This might
be my personal experience using it on Linux, which is probably not
as well supported as the versions for other major OS platforms. In
any case, things are improving. Cytoscape is, in other words, Peter
Jackson: he might not be perfect, he might have defects, but boy are
his movies nice to look at! 27
Mathieu Bastian, Sebastien Heymann,
The main alternative to Cytoscape is Gephi27 , 28 . In fact, calling Mathieu Jacomy, et al. Gephi: an open
it an “alternative” might even be unfair: my sense is that Gephi is source software for exploring and
manipulating networks. Icwsm, 8(2009):
actually more popular than Cytoscape among network scientists.
361–362, 2009
However, that is not what I started using during my PhD and so 28
[Link]
I never ended up installing it. So I don’t have any specific way to
compare their relative strengths and weaknesses. Chances are that,
for 99% of visualization tasks you will find yourself doing, the two
programs can be considered equivalent. Gephi is like Guillermo Del
Toro: I am unable to tell him apart from Peter Jackson. Regarding the
topic of file formats, Networkx can read the GEXF file format, which
is the one Gephi uses to save your network visualizations. 29
Manlio De Domenico, Mason A
Muxviz29 , 30 is another great piece of software. Muxviz covers Porter, and Alex Arenas. Muxviz: a tool
a slightly different angle from Cytoscape or Gephi. First, even if I for multilayer analysis and visualization
of networks. Journal of Complex Networks,
classify it in the visualization subsection, it is much more analysis-
3(2):159–176, 2015c
oriented. In fact, it requires a lot of analytical power installed on your 30
[Link]
machine (Octave and R for instance). And you might find yourself
using the command line interface more than the graphical interface.
In this sense, it could have been listed as a library in the previous
section.
More importantly, Muxviz is much more specialized. It has a
specific focus on multilayer networks (Section 4.2). Which is a good
thing, because Cytoscape is not very good for visualizing them.
As far as I know, with Cytoscape the only choice you have is to ei-
ther visualize them with a multigraph, or visualizing one layer at a
time and then use a lot of elbow grease to piece the layers together.
Muxviz, instead, supports them natively, and it is thus a very comple-
mentary choice if you want to be prepared for all the layers life will
throw at you. 31
Seth Tisue and Uri Wilensky. Netlogo:
Finally, there’s NetLogo31 , 32 . I frankly don’t know where to clas- A simple environment for modeling
sify it, because it is a weird mix of everything. First and foremost, complexity. In International conference on
complex systems, volume 21, pages 16–21.
NetLogo is a programming language. It is explicitly designed to
Boston, MA, 2004
facilitate the simulation of agent-based models. This includes all sorts 32
[Link]
of models, not necessarily the ones involving a network. Thus it is a netlogo/

more general tool, which allows you to do more than what this book
focuses on.
Secondarily, you can use NetLogo for visualizing the effects of
specific network processes. If you follow the link I provide, you can
 data & tools 643

access NetLogo Web, a collection of simulations programmed in

NetLogo that allows you to play with a bunch of different models.
For instance, you can run a SIR model (Section 17.3), modifying its
parameters and tracking the effects of your actions. NetLogo is a
godsend for all visual thinkers who need to see things happening in
front of them to really understand them.

For Analysis

There are many pieces of software out there that will allow you
to perform network analysis and are commonly used by network
professionals. They are far more than I can include here. So I will
limit myself to those with which I had some personal experience.
The programs I talk about here are the ones that primarily pro-
vide analytic power. You can visualize networks with them, but
you should not do that. Their visualization capabilities are not the
main focus of the software, and are there mostly for you to get a
quick sense of what sort of analyses you should ask the program to
perform.
I think the program for network analysis I stumble the most upon 33
Wouter De Nooy, Andrej Mrvar, and
Vladimir Batagelj. Exploratory social
in the literature is Pajek33 , 34 . Pajek allows you to perform a vast array network analysis with Pajek. Cambridge
of network analysis, ranging from classical social science ones, to University Press, 2018
more computer science-y ones – like community discovery. Pajek 34
[Link]
networks/pajek/
comes in different versions: Pajek, Pajek XXL, and Pajek 3XL. The
main difference between the versions is the capability of handling
larger and larger networks. The idea is that you would perform
the memory-intense analyses on the XL versions of Pajek and then
import the results for further investigation in the standard version of
the program.
Pajek is such a popular program that its own specific file format
is compatible with most of the software libraries I mentioned earlier.
Both Networkx and iGraph have functions that will allow you to
import networks saved in Pajek’s file format. Pajek is Lars von Trier:
perfect for geeking out every possible detail, but not the prettiest
thing to look at. 35
Stephen P Borgatti, Martin G Everett,
and Linton C Freeman. Ucinet for
A popular alternative from Pajek is UCINET35 , 36 . UCINET’s windows: Software for social network
strength is in its deep dive into the social branch of social network analysis. 2002
analysis. It is possibly the most comprehensive tool for social scien- 36
[Link]
ucinetsoftware/home
tists to use.
As a result, its coverage of the more computer science and physics
branches is less than optimal. UCINET works best with small net-
works, it is not particularly well optimized for large scale analysis,
and will lack some of the typical algorithms you might expect to find
after reading this book. However, my biggest gripe with it is prob-
644 the atlas for the aspiring network scientist

ably the fact that – differently from almost everything I mentioned

so far – UCINET is not a free program. If you’re a full time student,
it will cost $40. UCINET is George Méliès: an immortal classic, but
probably not the style you want to adopt in 2021. 37
Derek L Hansen, Ben Shneiderman,
Finally, I should mention NodeXL37 , 38 . NodeXL is a weird animal, and Marc A Smith. Analyzing social media
which lives on the border between being a software analysis tool networks with NodeXL: Insights from a
connected world. Morgan Kaufmann, 2010
with a graphical user interace like Pajek, and a library like NetworkX. 38
[Link]
NodeXL is a graphical front-end that integrates network analysis
into Microsoft Excel. Excel is a phenomenal tool, easily the best and
most used software Microsoft has ever written. Excel allows you
to perform powerful and sophisticated analysis tasks. There are
people whose entire careers could be summed up by a handful of
painstakingly crafted Excel spreadsheets. So it is no wonder that
there is demand for integrating network analysis in Excel. NodeXL
fills that niche.

Community Discovery
I create this special subsection to focus exclusively of implementa-
tions of algorithms solving the community discovery problem. This is
easily the largest subfield of network analysis. Thus this subsection
satisfies two needs. First, it gives you an idea about the immense
wealth of code that cannot find space in generic libraries/software.
Second, it contains the necessary references to the algorithms I con-
sider in my algorithm similarity network that was included in Section
31.6.
The way this subsection works is as follows. Now I will list a
bunch of labels that are consistent with Figure 31.15. For each label,
I tell you where to find the implementation I used to build that
figure. The general disclaimer is that, of course, some of these links
are bound to break in the future. I accessed them last time around
November 2018, so the Internet Archive could help.

• edgebetween, fastgreedy, hrg, labelperc, leadeig, louvain, spinglass,

walktrap: igraph implementation in R ([Link]

• mcl: [Link]

• tabu, extr: options #5 and #6 in [Link]

gomez/[Link].

• ganet, ganet+, moganet: [Link]

[Link].

• ganxis: [Link]

• conclude: [Link]
 data & tools 645

• conga, copra, cliquemod, peacock: [Link]

networks/.

• mlrmcl: [Link]

• metis: [Link]

• slpa, fluid, kerlin, kclique: networkx implementation in python

([Link]

• pmm: [Link]

• crossass: [Link]
[Link].

• demon: [Link]

• bigclam, agm: part of the SNAP library ([Link]

edu/).

• hlc: [Link]

• tiles: [Link]

• oslom: [Link]
software.

• kmeans, dbscan, ward, agglomerative, spectral, meanshift, affinity,

birch: [Link]
[Link].

• code-dense: [Link]
s10618-014-0373-y.

• moses, collapsed-sbm: [Link]

downloads.

• gce: [Link]

• ilcd: [Link]

• svinet, mmsb: [Link]

• bnmtf: [Link]

• rmcl: [Link]
[Link].

• OLC: [Link]
zip.

• cme-td, cme-bu: [Link]

646 the atlas for the aspiring network scientist

39
Santo Fortunato, Vito Latora, and
• edgeclust: [Link] Massimo Marchiori. Method to
radetal_algorithm.tgz. find community structures based on
information centrality. Physical review E,
• infocentr: my own implementation of the algorithm described in 70(5):056104, 2004
40
Anand Narasimhamurthy, Derek
the original paper39 .
Greene, Neil Hurley, and Pádraig
• msg, vm: [Link] Cunningham. Community finding in
large social networks through problem
• ocg: [Link] decomposition. In Proc. 19th Irish
Conference on Artificial Intelligence and
• savi: [Link] Cognitive Science, AICS, volume 8, 2008
• mixnet: [Link]
mixnet. 41
E Gabasova. The star wars so-
• vbmod: [Link] cial network. Evelina Gabasova’s
Blog. Data available at: [Link]
• bridgebound, bagrowLocal, clausetLocal, lwplocal: https:// com/evelinag/StarWars-social-
network/tree/master/networks, 2015
[Link]/kleinmind/bridge-bounding.
42
Tom AB Snijders, Gerhard G Van de
• netcarto: [Link] Bunt, and Christian EG Steglich. In-
troduction to stochastic actor-based
network-cartography-netcarto/.
models for network dynamics. Social
• infomap, infomap-overlap: [Link] networks, 32(1):44–60, 2010
html#Download-and-compile.
43
Gerhard G Van de Bunt, Marijtje AJ
Van Duijn, and Tom AB Snijders.
• graclus: [Link] Friendship networks through time:
An actor-oriented dynamic statistical
[Link].
network model. Computational &
• graclus2stage: my own implementation of the algorithm described Mathematical Organization Theory, 5(2):
167–192, 1999
in the original paper40 . 44
CJ Rhodes and P Jones. Inferring
• fuzzyclust: [Link] missing links in partially observed
social networks. In OR, Defence and
• linecomms: [Link] (to Security, pages 256–271. Springer, 2015
generate the line graph + igraph’s implementation of Louvain).
45
Karine Descormiers and Carlo
Morselli. Alliances, conflicts, and
contradictions in montreal’s street gang
It’s now time to move to a section dedicated not to code, but to
landscape. International Criminal Justice
data. However, before doing so, I’ll give you a small preview. In Review, 21(3):297–314, 2011
the paper building the algorithm similarity network, I test all these 46
Siva R Sundaresan, Ilya R Fischhoff,
Jonathan Dushoff, and Daniel I Ruben-
algorithms on 819 real world networks that I use as a benchmark.
stein. Network metrics reveal differences
These networks are taken from data kindly shared by the authors in social organization between two
of a bunch of papers41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 and online web- fission–fusion species, grevy’s zebra
and onager. Oecologia, 151(1):140–149,
pages53 , 54 . 2007
47
Martin W Schein and Milton H
Fohrman. Social dominance relation-
46.3 Data ships in a herd of dairy cattle. The British
Journal of Animal Behaviour, 3(2):45–55,
There’s more to life than just the software running on your computer 1955
48
A Gimenez-Salinas Framis. Illegal
– or so I’m told. Many great resources that can make you a better
networks or criminal organizations:
network analyst – or even just a better person overall – can be found Power, roles and facilitators in four
online. Specifically, here I focus on online network resources concern- cocaine trafficking structures. In Third
Annual Illicit Networks Workshop, 2011
ing the first ingredient of every network paper: network data. You 49
Andrew Beveridge and Jie Shan.
need data to test your models, to run your algorithm, to make a com- Network of thrones. Math Horizons, 23
pelling case of why your paper is important. Often, your study will (4):18–22, 2016
 data & tools 647

start from a dataset you already have, or you will collect one specially 50
Wouter De Nooy. A literary play-
tailored for your purposes. In many other cases, you simply need ground: Literary criticism and balance
theory. Poetics, 26(5-6):385–404, 1999
any network you can put your hands on that fulfills some specific 51
Dale F Lott. Dominance relations and
constraints. This section should help you with this task. breeding rate in mature male american
There are many places where you can find networks directly bison. Zeitschrift für Tierpsychologie, 49(4):
418–432, 1979
available for download, but I start with an index: the Colorado 52
Jermain Kaminski, Michael Schober,
Index of Complex Networks55 , 56 (ICON). This is quite possibly the Raymond Albaladejo, Oleksandr Zastu-
most comprehensive index of network datasets from all domains of pailo, and Cesar Hidalgo. Moviegalaxies-
social networks in movies. 2018
network science. Chances are that, if the network data is available 53
[Link]
somewhere, you can find it via ICON. networks/data/bio/foodweb/foodweb.
htm
However, this is an index of network datasets, not a dataset repos- 54
[Link]
itory, like the ones that will follow. This means that ICON is not [Link]

hosting any network data itself. It rather contains the links to those
55
A Clauset, E Tucker, and M Sainz. The
colorado index of complex networks,
datasets. This has advantages and disadvantages. The advantage 2016
is completeness: not all datasets can be moved from their original 56
[Link]

source and hosted somewhere else. ICON can include those datasets,
while the other repositories cannot. The other side of the coin is the
dynamism of the Internet. Resources get moved all the time, and not
everybody does it properly via HTTP redirects – actually almost no
one does it. Thus it is possible to find dead links in ICON, because
the managers of the website cannot possibly constantly check that all
links are working.
ICON will point you to tons of resources from which you can
actually download your data, for instance Pajek’s and UCINET’s
websites. There you can find a collection of network datasets you can
download, which is a nice additional resource to the software. One
issue you might have with this solution is that they distribute data in
their own file formats, so you might need to convert them before you
can use them with another software.
Also SNAP provides network data. While there is a large overlap
between what you can find in Pajek and UCINET, SNAP’s focus
goes decisively more towards computer science. You will find very
large datasets there, sometimes larger than what you can handle –
at the time of writing this chapter, I believe the largest network is
from Friendster, which contains more than 1.8 billion edges. Just like
with the implemented functions in SNAP, also the datasets are very
much focused on the ones the Stanford research group used for their
publications. 57
Jérôme Kunegis. Konect: the koblenz
Another interesting resource is Konect57 , 58 . Konect is also a Mat- network collection. In Proceedings of the
lab package for network analysis. Since I do not use Matlab unless 22nd International Conference on World
Wide Web, pages 1343–1350, 2013
someone is pointing a gun at me, I have no experience with it as 58
[Link]
an analysis tool. However, I used to browse Konect daily to find
and download some interesting network data. The list of available
datasets, as far as I can tell, is a superset of what you can find in the
648 the atlas for the aspiring network scientist

websites of Pajek and UCINET, and more. The web interface is also
well done, and you will be able to tell what are the main character-
istics of a network before downloading it: if it’s bipartite, what its
degree distribution is, etc. I said “used to browse” because recently
Konect moved to a subscription system – it’s not free any more. The
price tag is not something than an individual can afford without an
organization with deep pockets backing them, so it’s a non-starter for
independents. 59
[Link]
Network Repository59 is probably the largest repository with di-
rect network data download – thus excluding ICON. The interface
isn’t as good as Konect, but it’s free and it includes more network
data. Tiago Peixoto of Graph Tool fame also launched his own net- 60
[Link]
work data resource: Netzschleuder60 , which rivals Network Repos-
itory in size. It mostly takes all the network data indexed by ICON
or Konect and provides a direct download in different formats. The
networks can also be directly imported in Graph Tool via a function,
without worrying about having the network file saved on your hard
disk. The interface is snappier although it could do a better job to
highlight where the direct download buttons are.
If you’re specifically interested in multilayer network data, one 61
[Link]
cool resource is the Comune Lab61 . Comune is a project owned by
the same people behind Muxviz and the two can be considered as
closely integrated.

46.4 Legendary Graphs

There are some graphs that are so widely used that you don’t really
need to look for them in an online repository. These are the pillars
on which the entire cathedral of network science is founded. They
are often directly included in software and libraries, and all online
self-respecting network data repositories have one or multiple copies
of them. I include a few here.
The first – and by far most popular – of these legendary graphs is 62
Wayne W Zachary. An information
the Zachary Karate Club62 . This is a network of members of a karate flow model for conflict and fission in
club, connecting two members if they sparred against each other. It small groups. Journal of anthropological
research, 33(4):452–473, 1977
is often used because the network focuses on two main nodes: the
coach and the president of the club. The club eventually split due to
a disagreement between the two, and one can reconstruct on which
side each member went by analyzing with whom they sparred. It is
a classical example of community discovery. Figure 46.3 shows this
beauty in all of its glory.
Aaron Clauset told me a fun fact about this network. Zachary’s
original paper contains a figure showing the undirected adjacency
matrix of the Karate Club network, except that it’s not fully undi-
 data & tools 649

Figure 46.3: The Zachary Karate

Club network. I label the nodes
President
representing the coach and
the president. The node color
indicates whom the member
followed after the club split up.
Coach

rected! One edge appears in one direction, but not in the other. This
means that there are technically two Karate Club graphs, depending
on whether this edge is a typo or not, one with | E| = 77 edges and
one with | E| = 78 edges. The latter is the most common you’ll find
around, because it is the one that Mark Newman and Michelle Gir-
van used for their paper, which arguably launched the Karate Club
network in the Olympus of network science.
Network scientists are obsessed with this network. It has its own 63
[Link]
t-shirt63 . They even created the Zachary Karate Club Club64 : the club zachary_karate_club_with_label_
of network scientists who are the first using the Zachary network as t_shirt-235415254499870147, the label
says: “If your method doesn’t work on
an example in their presentation at a network science conference. If
this network, then go home”.
you do so, you become the current holder of the Zachary Karate Club 64
[Link]
Trophy and you are responsible for handing it at the next conference
you attend. This is fiercely competitive, and often you’ll see this prize
awarded at satellites events happening before the conference itself,
because people will use the network as an example as soon as they
can, to get their hands on the trophy.
The Network Science Society hands many prestigious awards: the 65
[Link]
Erdős-Rényi prize65 , to the career of the most outstanding network award-prizes/er-prize
scientist under the age of forty; or the Euler award66 , to the authors 66
[Link]
award-prizes/euler-award
of paradigm-changing publications in network science. But don’t get
fooled. The Zachary Karate Club Trophy is where it’s at.
Another commonly used network is the one obtained from Victor 67
Donald Ervin Knuth. The Stanford
Hugo’s novel Les Miserables67 . In the network, each node is a char- GraphBase: a platform for combinatorial
acter, and two characters are connected together if they appear in computing. AcM Press New York, 1993
the same chapter. Also in this case the classical application is for
community discovery, given that there are sets of characters closely
interacting with each other that never appear in chapters with other
groups of characters. Figure 46.4 shows an example. This is one of
those graphs that even non-network scientist would use for examples
650 the atlas for the aspiring network scientist

Figure 46.4: The Les Miser-

ables network. The node color
follows the community assign-
ment from a label percolation
community discovery

68
[Link]
related to other fields, for instance data visualization68 . The likely miserables/
reason is the inclusion of this network in Knuth’s popular book. 69
[Link]
The college football network69 is another network commonly American_College_Football_Network_
used for community discovery – I’m sensing a pattern here. Figure Files/93179

46.5 shows it. The reason it works well is due to the way sports
are organized in the United States. Usually, teams are divided in
conferences and divisions. A team will play with all other teams
in their division, but only with a selected number of teams in the
same conference and almost no team from the other conference. This
creates a nice hierarchical community structure. There is also an
overlap, as the most successful teams will then access to the finals
and thus play a significant number of matches with teams from the
other conference.

Figure 46.5: The Football net-

work. The node color follows
the community assignment
from a label percolation com-
munity discovery

Other examples of networks I’m not going to discuss in details are:

70
Ronald L Breiger and Philippa E
Pattison. Cumulated social roles: The
• Florentine families70 : each node is a family from Renaissance duality of persons and their algebras.
Florence, and families are connected if there is a marriage tie Social networks, 8(3):215–256, 1986
 data & tools 651

between them (I used this network in Section 16.2 when talking

about ERGMs);
71
Allison Davis, Burleigh Bradford
• Davis Southern women social network71 : a bipartite network, Gardner, and Mary R Gardner. Deep
connecting 18 women to 14 informal social events they attended; South: A social anthropological study of
caste and class. Univ of South Carolina
• C. Elegans: this is not a single graph, it is actually multiple. We Press, 1941
have extracted all possible ways to represent this poor little worm
in networks forms, from a neural network to protein-protein
interaction networks.
47
Glossary

Actor: The entity to which nodes in different layers in a multilayer

network refer to. It can be considered as the connected component
formed when using exclusively inter-layer couplings.

Acyclic Graph: See Tree.

Adjacency Matrix: A matrix where each row and column correspond

to a node in the graph. The Auv entry of the matrix is one if nodes u
and v are connected, zero otherwise.

Adjusted Mutual Information: A uniform random variable will have

some mutual information with a non-random variable. In AMI, by
definition, if there is no relation between the two variables the result
will be zero. Thus, AMI is mutual information adjusted for chance.

Arborescence: A directed tree in which all nodes have in-degree of

one, except the root, which has in-degree of zero.

Arborescence Forest: A graph with multiple weakly connected com-

ponents, each one of them being an arborescence.

Assortativity: The tendency of nodes to connect with other nodes

carrying similar attributes. Synonym of homophily.

Arc: See Edge.

Average Path Length: The sum of the lengths of all shortest paths in a
network over the total number of such paths.
 glossary 653

Balanced Graph: A directed graph whose in- and out-degree se-

quences are the same.

Betweenness Centrality: Normalized number of shortest paths pass-

ing through the node.

Biclique: A clique in a bipartite network.

Bipartite Network: A network with two types of nodes and whose

edges can only connect two nodes of different type.

Breadth First Search: The exploration of a graph by exploring all

neighbors of a node before moving on the the neighbors of the next
node.

Chain: A set of nodes that can be ordered, and each node is con-
nected only to its predecessor – except the first node – and its succes-
sor – except the last node.

Clique: A set of nodes where all possible edges are present, i.e. each
node in a clique is connected with each other node in the same clique.

Closeness Centrality: Normalized inverse of the average shortest path

length from a node to all other nodes in the network (or connected
component).

Complement Graph: given a graph G, its complement is a graph

where we remove all edges from G and we connect all pairs of nodes
that were not connected in G.

Complement of the Cumulative Distribution: A plot telling you the

fraction of points with value equal to or greater than x.

Connected Component: The maximal (sub)set of nodes in a network

that can all reach each other through walks.

Connected Network: A network composed by a single connected

component.

Connection: see Edge.

654 the atlas for the aspiring network scientist

Convex Network: A network whose all connected subgraphs are

convex, i.e. a tree of cliques.

Convex Subgraph: A subgraphs that contains all shortest paths

existing in the main network between its nodes.

Coupling Strategy: The way nodes belonging to the same actor

connects to each other across layers in a multilayer network. Example:
clique, chain, star.

Cumulative Distribution: A plot telling you the fraction of points

with value lower than x.

Cycle: A path in which the starting and ending node is the same.

Cyclic Graph: A graph containing at least a cycle.

Degree: The number of edges a node has.

Degree Matrix: A matrix whose diagonal entries are the degrees

of the corresponding nodes and the rest of the matrix is filled with
zeros.

Depth First Search: The exploration of a graph by exploring as far as

possible along a branch before backtracking.

Diameter: The length of the longest shortest path in a network.

Digraph: See Directed Graph.

Directed Acyclic Graph: A directed graph which does not contain a

cycle.

Directed Cyclic Graph: A directed graph containing a cycle.

Directed Edge: A non-reciprocal edge, which implies a relationship

that is not symmetric.

Directed Graph: A graph containing directed edges.

Directed Tree: A directed graph which would not contain a cycle

even if we were to ignore edge directions.

Disassortativity: The tendency of nodes to connect with nodes with

unlike attributes. Opposite of homophily.

Dynamic Network: A network whose edges can become active

and/or inactive at different moments in time, usually represented as
edge attributes.
 glossary 655

Edge: The interaction between two nodes, usually represented as a

pair of nodes.

Ego Network: A network focused on a node (ego). It contains the ego

node, all his neighbors, and all the connections between these nodes.

Eigenvalue: Given a matrix A, an eigenvalue of A is the scaling factor

of one of its eigenvectors, i.e. if Av = λv for some vector v, then λ is
an eigenvalue of A.

Eigenvector: Given a matrix A, an eigenvector of A is a special vector

that only changes its length – but not its direction – when multiplied
to A, i.e. if Av = λv for some value λ, then v is an eigenvector of A.

Fiedler Vector: The second smallest eigenvector of the Laplacian.

Forest: A network composed by more than one connected compo-

nent, each one of them being a tree.

Giant Connected Component: In real world networks, the largest

component which holds the majority of the nodes of a network.

Graph: A set of nodes connected by a set of edges.

Hairball: Incoherent ball of nodes and edges, typical result of a

naive visualization of a network too large and dense to be spread
out in a two dimensional plane. Also known as ridiculogram or
spaghettigraph.

Heterogeneous Network: A network with multiple node and edge

types.

Heterophily: The tendency of nodes to connect to nodes with unlike

attributes. Synonym of disassortativity.

Homophily: The tendency of nodes to connect to other nodes with

the same or similar attributes. Synonym of assortativity.
656 the atlas for the aspiring network scientist

Hub: A central node with many connections.

Hyperedges: Edges that can connect more than two nodes at the
same time.

Hypergraph: A graph containing hyperedges.

Identity Matrix: A matrix with ones on the diagonal and zeros every-
where else.

In-Component: A weakly connected component in a directed graph

whose paths can reach a strongly connected component but will
never reach back.

Incidence Matrix: A matrix with nodes on the rows, edges on the

columns, and whose non-zero entries report to which edges a node is
connected.

Induced Subgraph: a subgraph of an original graph formed from a

subset of the vertices of the graph and all of the edges connecting
pairs of vertices in that subset.

Interlayer Coupling: In a multilayer network, the special connections

connecting the nodes belonging to the same actor.

Isolated Node: a node with zero degree.

k-Clique: A clique of k nodes.

k-core: Set of nodes that have a minimum degree of k, once you recur-
sively remove from the network all nodes that have k − 1 connections
or fewer.

Laplacian: The matrix obtained subtracting the adjacency matrix

from the degree matrix.

Lattice: A simple graph in which nodes are uniformly distributed in

a n-dimensional space and they connect with a given number of their
nearest neighbors.
 glossary 657

Leaf Node: A node with degree equal to one.

Left Eigenvector: An eigenvector obtained multiplying the matrix

from the left. If vA = vλ, the v is a left eigenvector of A, in contrast
with right eigenvectors.

Line Graph: The graph that represents the adjacencies between edges
of an undirected graph: each edge of the original graph is a node in
the line graph, and two nodes in the line graph connect if they have a
node in common in the original graph.

Link: See Edge.

Maximal Clique: A clique in a network to which you cannot add any

nodes and still obtain a clique.

Maximum Spanning Tree: A spanning tree of a weighted graph

which has the highest edge weight sum of all spanning trees for that
graph.

Metapath: A path in a heterogeneous network, including nodes of

different types.

Minimum Spanning Tree: A spanning tree of a weighted graph

which has the lowest edge weight sum of all spanning trees for that
graph.

Multidimensional Network: A network with multiple edge types. A

subtype of multilayer networks.

Multigraph: A graph in which there can be multiple parallel edges

between the same two nodes.

Multilayer Network: A network in which nodes can connect to each

other with different types edges, and can have multiple identities.

Multipartite Network: A network with two or more node types, and

whose edges can only be established between nodes of unlike type.

Multiplex Network: A network with multiple edge types. A subtype

of multilayer networks.

Mutual information: A relatedness measure between two random

variables, namely the number of bits of information you obtain about
a random variable if you know the other one.
658 the atlas for the aspiring network scientist

n, m-Clique: A biclique with n nodes of type 1 and m nodes of type 2.

Neighbor: A node directly connected to your focus node by an edge.

Node: The fundamental interacting unit of a graph. In a social net-

work, it will be a person. In the Internet network, it will be a router.

Normalized Mutual information: Equivalent to Mutual Information,

normalized so that it takes values between zero and one.

Out-Component: A weakly connected component in a directed graph

which can receive paths from a strongly connected component but
cannot reach it back.

Parallel edges: Two (or more) edges established between the same
pair of nodes.

Path: A walk with no repeating nodes.

Planar Graph: A graph you can draw on a 2D plane without inter-

secting any edges.

Reverse Graph: the reverse graph of directed graph G is another

directed graph where we flip all edge directions.

Ridiculogram: see Hairball.

Right Eigenvector: An eigenvector obtained multiplying the matrix

from the right. If Av = λv, the v is a right eigenvector of A, in
contrast with left eigenvectors.
 glossary 659

Self-loop: An edge connecting a node with itself.

Simple Path: See Path.

Simplicial Complex: A set of nodes whose connections to each other

are part of a single high-order structure. Similar to hyperedges, with
the constraint of being embedded in a geometric space.

Singleton: see Isolated Node.

Spaghettigraph: See Hairball.

Spanning Tree: A subgraph that is a tree and includes all of the

nodes of its parent graph.

Square: A cycle of four nodes and four edges.

Star: A set of nodes with one acting as a center connected to all other
nodes in the star. All other nodes have only one connection, to the
star’s center.

Stationary Distribution: The probability of ending in a node after a

random walk of infinite length, equivalent to the degree for undi-
rected networks.

Stochastic Adjacency: A normalized adjacency matrix, whose rows

have been divided by their sum.

Strongly Connected Component: A component in a directed graph

that contains paths from any node of the component to any other
node of the component, respecting edge directions.

Subgraph: A graph whose sets of nodes and edges are completely

included in the node and edge sets of another graph.

Temporal Network: See Dynamic Network.

Tree: A graph containing no cycles.

Triad: A connected graph with three nodes and two edges.

Triangle: A connected graph with three nodes and three edges.

Tripartite Network: A network with three node types and whose

edges can only be established between nodes of unlike type.
660 the atlas for the aspiring network scientist

Undirected Network: A network whose edges are all without direc-

tion, i.e. all connections are symmetric.

Uniform Hypergraph: A hypergraph containing hyperedges with the

same cardinality – i.e. each hyperedge contains the same number of
nodes.

Unipartite Network: A network with only one node type, without

restrictions on how nodes connect – in direct contrast with a bipartite
network.

Unweighted Network: A network whose edges have no weight – or

where all weights are equal to one. In direct contrast with a weighted
network.

Vertex: See Node.

Walk: A sequence of nodes. Two consecutive nodes in the sequence

must be adjacent.

Weakly Connected Component: A component in a directed graph

that contains paths from any node of the component to any other
node of the component, but only if we ignore edge directions.

Weighted Adjacency: An adjacency matrix which is not binary. Each

cell contains the weight of its corresponding edge.

Weighted Edge: An edge with quantitative information, determining

its strength.

Weighted Network: A network containing weighted edges.

48
Most Common Abbreviations

A: Adjacency matrix.

AMI: Adjusted Mutual Information.

APL: Average Path Length.

APLv : The Average Path Length of paths starting from node v.

AUC: Area Under the (ROC) Curve, a common way to estimate

prediction performance.

BFS: Breadth First Search.

CC: Global Clustering Coefficient.

CCavg : Average Clustering Coefficient of the network, the average of

CCv for all vs in the network.

CCv : Local Clustering Coefficient of node v.

CCDF: Complement of the Cumulative Distribution Function.

CDF: Cumulative Distribution Function.

DFS: Depth First Search.

662 the atlas for the aspiring network scientist

E: Set of edges.

ERGM: Exponential Random Graph Model, a technique to generate

synthetic graphs for statistical testing.

FN: False Negative.

FP: False Positive.

FPR: False Positive Rate, equal to FP/( FP + TN ).

GCC: Giant Connected Component.

GERM: Graph Evolution Rule Mining, a way to predict links in

networks.

H: The hitting time matrix, telling you how long it’ll take for a ran-
dom walker to visit one node when starting from another.

I: The identity matrix.

k̄: The average degree of the network.

k v : The degree of node v.

L: Laplacian matrix.

MI: Mutual Information.

 most common abbreviations 663

NMF: Non-negative Matrix Factorization.

NMI: Normalized Mutual Information.

Nu : The set of neighbors of node u.

Nu,l : In a multilayer network, the set of neighbors of node u in layer l.

Puv : A path going from node u to node v.

PCA: Principal Component Analysis.

ROC: Receiver Operating Characteristic, a common way to estimate

prediction performance.

SBM: Stochastic Block Model, a network generative model.

SCC: Strongly Connected Component.

SI: Susceptible-Infected, an compartmental epidemiology model with

two states and one transition.

SIR: Susceptible-Infected-Removed, an compartmental epidemiology

model with two states and two irreversible transitions.

SIS: Susceptible-Infected-Susceptible, an compartmental epidemiol-

ogy model with two states and one reversible transition.

SVD: Singular Value Decomposition, an operation to decompose an

arbitrary matrix into a diagonal one.

SVM: Support Vector Machine, a machine learning technique.

TN: True Negative.

TP: True Positive.

TPR: True Positive Rate, equal to TP/( TP + FN ).

664 the atlas for the aspiring network scientist

V: Set of nodes.

W: Set of possible weights in a weighted network.

WCC: Weakly Connected Component.

Bibliography

Scott Aaronson. P=?np. Electronic Colloquium on Computational

Complexity (ECCC), 24:4, 2017. URL [Link]
report/2017/004.

Dimitris Achlioptas, Raissa M D’souza, and Joel Spencer. Explosive

percolation in random networks. Science, 323(5920):1453–1455, 2009.

Lada A Adamic and Eytan Adar. Friends and neighbors on the web.
Social networks, 25(3):211–230, 2003.

Lada A Adamic and Natalie Glance. The political blogosphere and

the 2004 us election: divided they blog. In Proceedings of the 3rd
international workshop on Link discovery, pages 36–43. ACM, 2005.

Lada A Adamic and Bernardo A Huberman. Power-law distribution

of the world wide web. science, 287(5461):2115–2115, 2000.

Lada A Adamic, Rajan M Lukose, Amit R Puniyani, and Bernardo A

Huberman. Search in power-law networks. Physical review E, 64(4):
046135, 2001.

Lada A Adamic, Jun Zhang, Eytan Bakshy, and Mark S Ackerman.

Knowledge sharing and yahoo answers: everyone knows something.
In Proceedings of the 17th international conference on World Wide Web,
pages 665–674, 2008.

Arun Advani and Bansi Malde. Empirical methods for networks data:
Social effects, network formation and measurement error. Technical
report, IFS Working Papers, 2014.

Nazanin Afsarmanesh and Matteo Magnani. Finding overlapping

communities in multiplex networks. arXiv preprint arXiv:1602.03746,
2016.

Rakesh Agrawal, Ramakrishnan Srikant, et al. Fast algorithms for

mining association rules. In Proc. 20th int. conf. very large data bases,
VLDB, volume 1215, pages 487–499, 1994.
666 the atlas for the aspiring network scientist

Nesreen Ahmed, Jennifer Neville, and Ramana Rao Kompella. Net-

work sampling via edge-based node selection with graph induction.
2011.

Nesreen K Ahmed, Jennifer Neville, and Ramana Kompella. Space-

efficient sampling from social activity streams. In Proceedings of
the 1st international workshop on big data, streams and heterogeneous
source mining: algorithms, systems, programming models and applications,
pages 53–60. ACM, 2012.

Nesreen K Ahmed, Jennifer Neville, and Ramana Kompella. Network

sampling: From static to streaming graphs. ACM Transactions on
Knowledge Discovery from Data (TKDD), 8(2):7, 2014.

Kook Jin Ahn, Sudipto Guha, and Andrew McGregor. Graph

sketches: sparsification, spanners, and subgraphs. In SIGMOD-
SIGACT-SIGAI, pages 5–14, 2012.

Yong-Yeol Ahn, James P Bagrow, and Sune Lehmann. Link communi-

ties reveal multiscale complexity in networks. nature, 466(7307):761,
2010.

Yong-Yeol Ahn, Sebastian E Ahnert, James P Bagrow, and Albert-

László Barabási. Flavor network and the principles of food pairing.
Scientific reports, 1:196, 2011.

Sebastian E Ahnert. Power graph compression reveals dominant

relationships in genetic transcription networks. Molecular BioSystems,
9(11):2681–2685, 2013.

Ravindra K Ahuja, Kurt Mehlhorn, James Orlin, and Robert E

Tarjan. Faster algorithms for the shortest path problem. Journal of the
ACM (JACM), 37(2):213–223, 1990.

William Aiello, Fan Chung, and Linyuan Lu. A random graph model
for massive graphs. In Proceedings of the thirty-second annual ACM
symposium on Theory of computing, pages 171–180. Acm, 2000.

Edoardo M Airoldi, David M Blei, Stephen E Fienberg, and Eric P

Xing. Mixed membership stochastic blockmodels. Journal of Machine
Learning Research, 9(Sep):1981–2014, 2008.

Saeed Akhoondian Amiri, Lukasz Kaiser, Stephan Kreutzer, Roman

Rabinovich, and Sebastian Siebertz. Graph searching games and
width measures for directed graphs. In LIPIcs-Leibniz International
Proceedings in Informatics, volume 30. Schloss Dagstuhl-Leibniz-
Zentrum fuer Informatik, 2015.
 bibliography 667

Ricardo Alberich, Joe Miro-Julia, and Francesc Rosselló. Marvel

universe looks almost like a real social network. arXiv preprint
cond-mat/0202174, 2002.

Reka Albert. Scale-free networks in cell biology. Journal of cell science,

118(21):4947–4957, 2005.

Réka Albert, Hawoong Jeong, and Albert-László Barabási. Error and

attack tolerance of complex networks. nature, 406(6794):378, 2000.

Lauren Alexander, Shan Jiang, Mikel Murga, and Marta C González.

Origin–destination trips by purpose and time of day inferred from
mobile phone data. Transportation research part c: emerging technologies,
58:240–250, 2015.

Noga Alon, Erik D Demaine, Mohammad T Hajiaghayi, and Tom

Leighton. Basic network creation games. SIAM Journal on Discrete
Mathematics, 27(2):656–668, 2013.

Uri Alon. Network motifs: theory and experimental approaches.

Nature Reviews Genetics, 8(6):450, 2007.

Brian Alspach. Searching and sweeping graphs: a brief survey. Le

matematiche, 59(1, 2):5–37, 2006.

Jeff Alstott and Dietmar Plenz Bullmore. powerlaw: a python

package for analysis of heavy-tailed distributions. PloS one, 9(1), 2014.

Taher Alzahrani and Kathy J Horadam. Community detection in

bipartite networks: Algorithms and case studies. In Complex systems
and networks, pages 25–50. Springer, 2016.

Taher Alzahrani, Kathy J Horadam, and Serdar Boztas. Community

detection in bipartite networks using random walks. In Complex
Networks V, pages 157–165. Springer, 2014.

Alessia Amelio and Clara Pizzuti. Overlapping community discovery

methods: A survey. In Social Networks: Analysis and Case Studies,
pages 105–125. Springer, 2014.

Carolyn J Anderson, Stanley Wasserman, and Bradley Crouch. A p*

primer: Logit models for social networks. Social networks, 21(1):37–66,
1999.

Philip W Anderson. More is different. Science, 177(4047):393–396,

1972.

Anton Andreychuk, Konstantin Yakovlev, Dor Atzmon, and Roni

Sternr. Multi-agent pathfinding with continuous time. In IJCAI,
volume 19, 2019.
668 the atlas for the aspiring network scientist

Aamir Anis, Akshay Gadde, and Antonio Ortega. Towards a

sampling theorem for signals on arbitrary graphs. In Acoustics, Speech
and Signal Processing (ICASSP), 2014 IEEE International Conference on,
pages 3864–3868. IEEE, 2014.

Francis J Anscombe. Graphs in statistical analysis. The american

statistician, 27(1):17–21, 1973.

Tibor Antal, Pavel L Krapivsky, and Sidney Redner. Dynamics of

social balance on networks. Physical Review E, 72(3):036121, 2005.

Jac M Anthonisse. The rush in a directed graph. Stichting Mathema-

tisch Centrum. Mathematische Besliskunde, (BN 9/71), 1971.

Naheed Anjum Arafat and Stéphane Bressan. Hypergraph drawing

by force-directed placement. In International Conference on Database
and Expert Systems Applications, pages 387–394. Springer, 2017.

Elsa Arcaute, Erez Hatna, Peter Ferguson, Hyejin Youn, Anders

Johansson, and Michael Batty. Constructing cities, deconstructing
scaling laws. Journal of The Royal Society Interface, 12(102):20140745,
2015.

Samin Aref and Mark C Wilson. Balance and frustration in signed

networks. Journal of Complex Networks, 7(2):163–189, 2019.

Alex Arenas, Jordi Duch, Alberto Fernández, and Sergio Gómez. Size
reduction of complex networks preserving modularity. New Journal
of Physics, 9(6):176, 2007.

Alex Arenas, Alberto Fernandez, Santo Fortunato, and Sergio

Gomez. Motif-based communities in complex networks. Journal of
Physics A: Mathematical and Theoretical, 41(22):224001, 2008a.

Alex Arenas, Alberto Fernandez, and Sergio Gomez. Analysis of the

structure of complex networks at different resolution levels. New
journal of physics, 10(5):053039, 2008b.

Armen S Asratian, Tristan MJ Denley, and Roland Häggkvist. Bipar-

tite graphs and their applications, volume 131. Cambridge University
Press, 1998.

Ira Assent, Andrea Wenning, and Thomas Seidl. Approximation

techniques for indexing the earth mover’s distance in multimedia
databases. In Data Engineering, 2006. ICDE’06. Proceedings of the 22nd
International Conference on, pages 11–11. IEEE, 2006.

Sitaram Asur, Srinivasan Parthasarathy, and Duygu Ucar. An event-

based framework for characterizing the evolutionary behavior of
 bibliography 669

interaction graphs. ACM Transactions on Knowledge Discovery from

Data (TKDD), 3(4):16, 2009.

Wirt Atmar and Bruce D Patterson. The measure of order and disor-
der in the distribution of species in fragmented habitat. Oecologia, 96
(3):373–382, 1993.

James Atwood and Don Towsley. Diffusion-convolutional neural

networks. In NIPS, pages 1993–2001, 2016.

Felix Auerbach. Das gesetz der bevölkerungskonzentration. Peter-

manns Geographische Mitteilungen, 59:74–76, 1913.

László Babai. Graph isomorphism in quasipolynomial time. In

Proceedings of the forty-eighth annual ACM symposium on Theory of
Computing, pages 684–697. ACM, 2016.

Benjamin Bach, Nathalie Henry Riche, Christophe Hurter, Kim

Marriott, and Tim Dwyer. Towards unambiguous edge bundling:
Investigating confluent drawings for network visualization. IEEE
transactions on visualization and computer graphics, 23(1):541–550, 2016.

Louis Bachelier. Théorie de la spéculation. In Annales scientifiques de

l’École normale supérieure, volume 17, pages 21–86, 1900.

Giacomo Bachi, Michele Coscia, Anna Monreale, and Fosca Gi-

annotti. Classifying trust/distrust relationships in online social
networks. In 2012 International Conference on Privacy, Security, Risk
and Trust and 2012 International Confernece on Social Computing, pages
552–557. IEEE, 2012.

Lars Backstrom, Cynthia Dwork, and Jon Kleinberg. Wherefore art

thou r3579x?: anonymized social networks, hidden patterns, and
structural steganography. In Proceedings of the 16th international
conference on World Wide Web, pages 181–190. ACM, 2007.

Lars Backstrom, Paolo Boldi, Marco Rosa, Johan Ugander, and

Sebastiano Vigna. Four degrees of separation. In Proceedings of the 4th
Annual ACM Web Science Conference, pages 33–42. ACM, 2012.

James P Bagrow. Evaluating local community methods in networks.

Journal of Statistical Mechanics: Theory and Experiment, 2008(05):
P05001, 2008.

Eytan Bakshy, Jake M Hofman, Winter A Mason, and Duncan J

Watts. Everyone’s an influencer: quantifying influence on twitter. In
Proceedings of the fourth ACM international conference on Web search and
data mining, pages 65–74. ACM, 2011.
670 the atlas for the aspiring network scientist

Eytan Bakshy, Solomon Messing, and Lada A Adamic. Exposure to

ideologically diverse news and opinion on facebook. Science, 348
(6239):1130–1132, 2015.

Bela Balassa. Trade liberalisation and “revealed” comparative

advantage 1. The manchester school, 33(2):99–123, 1965.

Vladimír Baláž, Jaroslav Koča, Vladimír Kvasnička, and Milan

Sekanina. A metric for graphs. Časopis pro pěstování matematiky, 111
(4):431–433, 1986.

Duygu Balcan, Vittoria Colizza, Bruno Gonçalves, Hao Hu, José J

Ramasco, and Alessandro Vespignani. Multiscale mobility networks
and the spatial spreading of infectious diseases. PNAS, 106(51):
21484–21489, 2009.

Pierre-Alexandre Balland, José Antonio Belso-Martínez, and Andrea

Morrison. The dynamics of technical and business knowledge
networks in industrial clusters: Embeddedness, status, or proximity?
Economic Geography, 92(1):35–60, 2016.

Anirban Banerjee. Structural distance and evolutionary relationship

of networks. Biosystems, 107(3):186–196, 2012.

Jørgen Bang-Jensen and Gregory Z Gutin. Digraphs: theory, algorithms

and applications. Springer Science & Business Media, 2008.

Albert-László Barabási. Linked: The new science of networks, 2003.

Albert-Laszlo Barabasi. The origin of bursts and heavy tails in human

dynamics. Nature, 435(7039):207, 2005.

Albert-László Barabási. Scale-free networks: a decade and beyond.

science, 325(5939):412–413, 2009.

Albert-László Barabási and Réka Albert. Emergence of scaling in

random networks. science, 286(5439):509–512, 1999.

Albert-László Barabási and Eric Bonabeau. Scale-free networks.

Scientific american, 288(5):60–69, 2003.

Albert-Laszlo Barabâsi, Hawoong Jeong, Zoltan Néda, Erzsebet

Ravasz, Andras Schubert, and Tamas Vicsek. Evolution of the social
network of scientific collaborations. Physica A: Statistical mechanics and
its applications, 311(3-4):590–614, 2002.

Albert-László Barabási, Chaoming Song, and Dashun Wang. Pub-

lishing: Handful of papers dominates citation. Nature, 491(7422):40,
2012.
 bibliography 671

Albert-László Barabási et al. Network science. Cambridge university

press, 2016.

Paul Baran. Introduction to distributed communications networks.

Technical report, Memorandum RM-3420-PR, Rand Corporation,
1964.

Michael J Barber. Modularity and community detection in bipartite

networks. Physical Review E, 76(6):066102, 2007.

Michael J Barber and John W Clark. Detecting network communities

by propagating labels under constraints. Physical Review E, 80(2):
026129, 2009.

Mauro Barone and Michele Coscia. Birds of a feather scam together:

Trustworthiness homophily in a business network. Social Networks,
54:228 – 237, 2018. ISSN 0378-8733.

Alain Barrat, Marc Barthelemy, Romualdo Pastor-Satorras, and

Alessandro Vespignani. The architecture of complex weighted
networks. Proceedings of the national academy of sciences, 101(11):
3747–3752, 2004a.

Alain Barrat, Marc Barthélemy, and Alessandro Vespignani. Weighted

evolving networks: coupling topology and weight dynamics. Physical
review letters, 92(22):228701, 2004b.

Baruch Barzel and Albert-László Barabási. Network link prediction

by global silencing of indirect correlations. Nature biotechnology, 31
(8):720–725, 2013a.

Baruch Barzel and Albert-László Barabási. Universality in network

dynamics. Nature physics, 9(10):673, 2013b.

Jordi Bascompte, Pedro Jordano, Carlos J Melián, and Jens M Ole-

sen. The nested assembly of plant–animal mutualistic networks.
Proceedings of the National Academy of Sciences, 100(16):9383–9387,
2003.

Danielle S Bassett and Edward T Bullmore. Human brain networks

in health and disease. Current opinion in neurology, 22(4):340, 2009.

Danielle S Bassett, Nicholas F Wymbs, Mason A Porter, Peter J

Mucha, Jean M Carlson, and Scott T Grafton. Dynamic reconfigu-
ration of human brain networks during learning. Proceedings of the
National Academy of Sciences, 108(18):7641–7646, 2011.

Danielle S Bassett, Mason A Porter, Nicholas F Wymbs, Scott T

Grafton, Jean M Carlson, and Peter J Mucha. Robust detection of
672 the atlas for the aspiring network scientist

dynamic community structure in networks. Chaos Journal, 23(1):

013142, 2013.

Mathieu Bastian, Sebastien Heymann, Mathieu Jacomy, et al. Gephi:

an open source software for exploring and manipulating networks.
Icwsm, 8(2009):361–362, 2009.

Ugo Bastolla, Miguel A Fortuna, Alberto Pascual-García, Antonio

Ferrera, Bartolo Luque, and Jordi Bascompte. The architecture
of mutualistic networks minimizes competition and increases
biodiversity. Nature, 458(7241):1018, 2009.

Vladimir Batagelj and Ulrik Brandes. Efficient generation of large

random networks. Physical Review E, 71(3):036113, 2005.

Vladimir Batagelj and Matjaz Zaversnik. An o (m) algorithm for

cores decomposition of networks. arXiv preprint cs/0310049, 2003.

Federico Battiston, Vincenzo Nicosia, and Vito Latora. Structural

measures for multiplex networks. Physical Review E, 89(3):032804,
2014.

Federico Battiston, Vincenzo Nicosia, and Vito Latora. Efficient

exploration of multiplex networks. New Journal of Physics, 18(4):
043035, 2016.

Federico Battiston, Vincenzo Nicosia, Mario Chavez, and Vito

Latora. Multilayer motif analysis of brain networks. Chaos: An
Interdisciplinary Journal of Nonlinear Science, 27(4):047404, 2017.

Federico Battiston, Jeremy Guillon, Mario Chavez, Vito Latora, and

Fabrizio De Vico Fallani. Multiplex core–periphery organization of
the human connectome. Journal of the Royal Society Interface, 15(146):
20180514, 2018.

Heiko Bauke. Parameter estimation for power-law distributions by

maximum likelihood methods. The European Physical Journal B, 58(2):
167–173, 2007.

Alex Bavelas. A mathematical model for group structures. Human

organization, 7(3):16, 1948.

Marya Bazzi, Mason A Porter, Stacy Williams, Mark McDonald,

Daniel J Fenn, and Sam D Howison. Community detection in
temporal multilayer networks, with an application to correlation
networks. Multiscale Modeling & Simulation, 14(1):1–41, 2016.

Stephen J Beckett. Improved community detection in weighted

bipartite networks. Royal Society open science, 3(1):140536, 2016.
 bibliography 673

Michael Behrisch, Benjamin Bach, Nathalie Henry Riche, Tobias

Schreck, and Jean-Daniel Fekete. Matrix reordering methods for table
and network visualization. In Computer Graphics Forum, volume 35,
pages 693–716. Wiley Online Library, 2016.

Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps and spectral

techniques for embedding and clustering. In Advances in neural
information processing systems, pages 585–591, 2002.

David C Bell, Elizabeth B Erbaugh, Tabitha Serrano, Cheryl A

Dayton-Shotts, and Isaac D Montoya. A comparison of network
sampling designs for a hidden population of drug users: Random
walk vs. respondent-driven sampling. Social science research, 62:
350–361, 2017.

Austin R Benson, David F Gleich, and Jure Leskovec. Tensor spectral

clustering for partitioning higher-order network structures. In
Proceedings of the 2015 SIAM International Conference on Data Mining,
pages 118–126. SIAM, 2015.

Austin R Benson, David F Gleich, and Jure Leskovec. Higher-order

organization of complex networks. Science, 353(6295):163–166, 2016.

Jonah Berger and Katherine L Milkman. What makes online content

viral? Journal of marketing research, 49(2):192–205, 2012.

Michele Berlingerio, Francesco Bonchi, Björn Bringmann, and

Aristides Gionis. Mining graph evolution rules. In joint European
conference on machine learning and knowledge discovery in databases,
pages 115–130. Springer, 2009.

Michele Berlingerio, Michele Coscia, and Fosca Giannotti. Finding

and characterizing communities in multidimensional networks. In
2011 International Conference on Advances in Social Networks Analysis
and Mining, pages 490–494. IEEE, 2011a.

Michele Berlingerio, Michele Coscia, and Fosca Giannotti. Finding

redundant and complementary communities in multidimensional
networks. In Proceedings of the 20th ACM international conference on
Information and knowledge management, pages 2181–2184, 2011b.

Michele Berlingerio, Danai Koutra, Tina Eliassi-Rad, and Christos

Faloutsos. Netsimile: A scalable approach to size-independent
network similarity. arXiv preprint arXiv:1209.2684, 2012.

Michele Berlingerio, Michele Coscia, Fosca Giannotti, Anna Mon-

reale, and Dino Pedreschi. Multidimensional networks: foundations
of structural analysis. WWW, 16(5-6):567–593, 2013a.
674 the atlas for the aspiring network scientist

Michele Berlingerio, Danai Koutra, Tina Eliassi-Rad, and Christos

Faloutsos. Network similarity via multiple social theories. In
Proceedings of the 2013 IEEE/ACM International Conference on Advances
in Social Networks Analysis and Mining, pages 1439–1440, 2013b.

Michele Berlingerio, Fabio Pinelli, and Francesco Calabrese. Abacus:

frequent pattern mining-based community discovery in multidimen-
sional networks. Data Mining and Knowledge Discovery, 27(3):294–320,
2013c.

H Russell Bernard and Harvey Russell Bernard. Social research

methods: Qualitative and quantitative approaches. Sage, 2013.

Timothy J Berners-Lee. Information management: A proposal.

Technical report, 1989.

Boris C Bernhardt, Zhang Chen, Yong He, Alan C Evans, and Neda
Bernasconi. Graph-theoretical analysis reveals disrupted small-world
organization of cortical thickness correlation networks in temporal
lobe epilepsy. Cerebral cortex, 21(9):2147–2157, 2011.

Alessandro Bessi and Emilio Ferrara. Social bots distort the 2016 us
presidential election online discussion. 2016.

Luís MA Bettencourt. The origins of scaling in cities. science, 340

(6139):1438–1441, 2013.

Luís MA Bettencourt, José Lobo, Dirk Helbing, Christian Kühnert,

and Geoffrey B West. Growth, innovation, scaling, and the pace of
life in cities. Proceedings of the national academy of sciences, 104(17):
7301–7306, 2007.

Richard F Betzel and Danielle S Bassett. Multi-scale brain networks.

Neuroimage, 160:73–83, 2017.

Andrew Beveridge and Jie Shan. Network of thrones. Math Horizons,

23(4):18–22, 2016.

James C Bezdek. Pattern recognition with fuzzy objective function

algorithms. Springer Science & Business Media, 2013.

Shankar Bhamidi, Guy Bresler, and Allan Sly. Mixing time of

exponential random graphs. In 2008 49th Annual IEEE Symposium on
Foundations of Computer Science, pages 803–812. IEEE, 2008.

Ginestra Bianconi. Multilayer Networks: Structure and Function. Oxford

University Press, 2018.

Ginestra Bianconi and Christoph Rahmede. Emergent hyperbolic

network geometry. Scientific reports, 7:41974, 2017.
 bibliography 675

Patrick Biernacki and Dan Waldorf. Snowball sampling: Problems

and techniques of chain referral sampling. Sociological methods &
research, 10(2):141–163, 1981.

Christian Bird, David Pattison, Raissa D’Souza, Vladimir Filkov, and

Premkumar Devanbu. Latent social structure in open source projects.
In Proceedings of the 16th ACM SIGSOFT International Symposium on
Foundations of software engineering, pages 24–35, 2008.

Robert T Blackburn, Charles E Behymer, and David E Hall. Research

note: Correlates of faculty publications. Sociology of Education, pages
132–141, 1978.

Neli Blagus, Lovro Šubelj, and Marko Bajec. Empirical comparison of

network sampling techniques. arXiv preprint arXiv:1506.02449, 2015.

Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and

Etienne Lefebvre. Fast unfolding of communities in large networks.
Journal of statistical mechanics: theory and experiment, 2008(10):P10008,
2008.

Stefano Boccaletti, Ginestra Bianconi, Regino Criado, Charo I

Del Genio, Jesús Gómez-Gardenes, Miguel Romance, Irene Sendina-
Nadal, Zhen Wang, and Massimiliano Zanin. The structure and
dynamics of multilayer networks. Physics Reports, 544(1):1–122, 2014.

Marián Boguñá and Romualdo Pastor-Satorras. Class of correlated

random networks with hidden variables. Phys. Rev. E, 68:036112, Sep
2003. doi: 10.1103/PhysRevE.68.036112.

Marián Boguñá, Romualdo Pastor-Satorras, and Alessandro Vespig-

nani. Absence of epidemic threshold in scale-free networks with
degree correlations. Phys. Rev. Lett., 90:028701, Jan 2003. doi:
10.1103/PhysRevLett.90.028701.

Marián Boguñá, Romualdo Pastor-Satorras, Albert Díaz-Guilera,

and Alex Arenas. Models of social networks based on social distance
attachment. Physical review E, 70(5):056122, 2004.

Marián Boguná, Romualdo Pastor-Satorras, and Alessandro Vespig-

nani. Cut-offs and finite size effects in scale-free networks. The
European Physical Journal B, 38(2):205–209, 2004.

Ludvig Bohlin, Daniel Edler, Andrea Lancichinetti, and Martin

Rosvall. Community detection and visualization of networks with
the map equation framework. In Measuring Scholarly Impact, pages
3–34. Springer, 2014.
676 the atlas for the aspiring network scientist

Aleksandar Bojchevski, Oleksandr Shchur, Daniel Zügner, and

Stephan Günnemann. Netgan: Generating graphs via random walks.
In International Conference on Machine Learning, pages 609–618, 2018.

Paolo Boldi and Sebastiano Vigna. The webgraph framework i:

compression techniques. In Proceedings of the 13th international
conference on World Wide Web, pages 595–602, 2004.

Paolo Boldi and Sebastiano Vigna. Axioms for centrality. Internet

Mathematics, 10(3-4):222–262, 2014.

Johan Bollen and Bruno Gonçalves. Network happiness: How online

social interactions relate to our well being. In Complex Spreading
Phenomena in Social Systems, pages 257–268. Springer, 2018.

Johan Bollen, Bruno Gonçalves, Guangchen Ruan, and Huina Mao.

Happiness is assortative in online social networks. Artificial life, 17(3):
237–251, 2011.

Johan Bollen, Bruno Gonçalves, Ingrid van de Leemput, and

Guangchen Ruan. The happiness paradox: your friends are happier
than you. EPJ Data Science, 6(1):4, 2017.

Béla Bollobás. Random graphs. In Modern graph theory, pages 215–252.

Springer, 1998.

Béla Bollobás and Oliver Riordan. Robustness and vulnerability of

scale-free random graphs. Internet Mathematics, 1(1):1–35, 2004.

Béla Bollobás, Oliver Riordan, Joel Spencer, and Gábor Tusnády.

The degree sequence of a scale-free random graph process. Random
Structures & Algorithms, 18(3):279–290, 2001.

Phillip Bonacich and Paulette Lloyd. Eigenvector-like measures of

centrality for asymmetric relations. Social networks, 23(3):191–201,
2001.

John Adrian Bondy, Uppaluri Siva Ramachandra Murty, et al. Graph

theory with applications, volume 290. Citeseer, 1976.

Douglas G Bonett and Thomas A Wright. Sample size require-

ments for estimating pearson, kendall and spearman correlations.
Psychometrika, 65(1):23–28, 2000.

Paolo Bonifazi, Miri Goldin, Michel A Picardo, Isabel Jorquera,

A Cattani, Gregory Bianconi, Alfonso Represa, Yehezkel Ben-Ari,
and Rosa Cossart. Gabaergic hub neurons orchestrate synchrony
in developing hippocampal networks. Science, 326(5958):1419–1424,
2009.
 bibliography 677

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason

Weston, and Oksana Yakhnenko. Translating embeddings for
modeling multi-relational data. In Advances in neural information
processing systems, pages 2787–2795, 2013.

Stephen P Borgatti and Martin G Everett. Models of core/periphery

structures. Social Networks, 21(4):375 – 395, 2000. ISSN 0378-8733.
doi: [Link]

Stephen P Borgatti and Martin G Everett. A graph-theoretic perspec-

tive on centrality. Social networks, 28(4):466–484, 2006.

Stephen P Borgatti, Candace Jones, and Martin G Everett. Network

measures of social capital. Connections, 21(2):27–36, 1998.

Stephen P Borgatti, Martin G Everett, and Linton C Freeman. Ucinet

for windows: Software for social network analysis. 2002.

Stephen P Borgatti, Ajay Mehra, Daniel J Brass, and Giuseppe

Labianca. Network analysis in the social sciences. science, 323(5916):
892–895, 2009.

Christian Borgelt and Michael R Berthold. Mining molecular

fragments: Finding relevant substructures of molecules. In 2002 IEEE
International Conference on Data Mining, 2002. Proceedings., pages
51–58. IEEE, 2002.

Christian Borgs, Jennifer Chayes, László Lovász, Vera T Sós, Balázs

Szegedy, and Katalin Vesztergombi. Graph limits and parameter
testing. In Proceedings of the thirty-eighth annual ACM symposium on
Theory of computing, pages 261–270, 2006.

Otakar Borůvka. O jistém problému minimálním. 1926.

Edward Bortnikov, Maxim Gurevich, Idit Keidar, Gabriel Kliot, and

Alexander Shraer. Brahms: Byzantine resilient random membership
sampling. Computer Networks, 53(13):2340–2359, 2009.

Michael Bostock and Jeffrey Heer. Protovis: A graphical toolkit for

visualization. IEEE transactions on visualization and computer graphics,
15(6):1121–1128, 2009.

Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. D3 data-driven

documents. IEEE transactions on visualization and computer graphics, 17
(12):2301–2309, 2011.

Giulio Bottazzi and Davide Pirino. Measuring industry relatedness

and corporate coherence. 2010.
678 the atlas for the aspiring network scientist

Oualid Boutemine and Mohamed Bouguessa. Mining community

structures in multidimensional networks. TKDD, 11(4):51, 2017.

Ulrik Brandes and Daniel Fleischer. Centrality measures based on

current flow. In Annual symposium on theoretical aspects of computer
science, pages 533–544. Springer, 2005.

Ulrik Brandes, Markus Eiglsperger, Jürgen Lerner, and Christian

Pich. Graph markup language (GraphML). 2013.

Ronald L Breiger and Philippa E Pattison. Cumulated social roles:

The duality of persons and their algebras. Social networks, 8(3):
215–256, 1986.

Alain Bretto. Hypergraph theory: An introduction. Mathematical

Engineering. Cham: Springer, 2013.

Cynthia A Brewer. Color use guidelines for mapping. Visualization in

modern cartography, 1994:123–148, 1994.

Graham Brightwell and Peter Winkler. Maximum hitting time for

random walks on graphs. Random Structures & Algorithms, 1(3):
263–276, 1990.

Björn Bringmann and Siegfried Nijssen. What is frequent in a single

graph? In Pacific-Asia Conference on Knowledge Discovery and Data
Mining, pages 858–863. Springer, 2008.

Björn Bringmann, Michele Berlingerio, Francesco Bonchi, and

Arisitdes Gionis. Learning and predicting the evolution of social
networks. IEEE Intelligent Systems, 25(4):26–35, 2010.

Dirk Brockmann, Lars Hufnagel, and Theo Geisel. The scaling laws
of human travel. Nature, 439(7075):462–465, 2006.

Tom Broekel and Matté Hartog. Explaining the structure of inter-

organizational networks using exponential random graph models.
Industry and Innovation, 20(3):277–295, 2013.

Tom Broekel, Pierre-Alexandre Balland, Martijn Burger, and Frank

van Oort. Modeling knowledge networks in economic geography:
a discussion of four methods. The annals of regional science, 53(2):
423–452, 2014.

Anna D Broido and Aaron Clauset. Scale-free networks are rare.

Nature communications, 10(1):1017, 2019.

Tom Brughmans. Connecting the dots: towards archaeological

network analysis. Oxford Journal of Archaeology, 29(3):277–303, 2010.
 bibliography 679

Tom Brughmans. Thinking through networks: a review of formal

network methods in archaeology. Journal of Archaeological Method and
Theory, 20(4):623–662, 2013.

Charles D Brummitt, Raissa M D’Souza, and Elizabeth A Leicht.

Suppressing cascades of load in interdependent networks. Proceedings
of the National Academy of Sciences, 109(12):E680–E689, 2012.

Joan Bruna and X Li. Community detection with graph neural

networks. stat, 1050:27, 2017.

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun.

Spectral networks and locally connected networks on graphs. arXiv
preprint arXiv:1312.6203, 2013.

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann Lecun.

Spectral networks and locally connected networks on graphs. In
ICLR, 2014.

Sergey V Buldyrev, Roni Parshani, Gerald Paul, H Eugene Stanley,

and Shlomo Havlin. Catastrophic cascade of failures in interdepen-
dent networks. Nature, 464(7291):1025, 2010.

Ed Bullmore and Olaf Sporns. Complex brain networks: graph

theoretical analysis of structural and functional systems. Nature
reviews neuroscience, 10(3):186–198, 2009.

Horst Bunke. On a relation between graph edit distance and maxi-

mum common subgraph. Pattern Recognition Letters, 18(8):689–694,
1997.

Rebekka Burkholz, Antonios Garas, and Frank Schweitzer. How

damage diversification can reduce systemic risk. Physical Review E,
93(4):042313, 2016a.

Rebekka Burkholz, Matt V Leduc, Antonios Garas, and Frank

Schweitzer. Systemic risk in multiplex networks with asymmetric
coupling and threshold feedback. Physica D: Nonlinear Phenomena,
323:64–72, 2016b.

Roberto Busa. Index thomisticus sancti thomae aquinatis operum

omnium indices et concordantiae in quibus verborum omnium et
singulorum formae et lemmata cum suis frequentiis et contextibus
variis modis referuntur. 1974.

Vannevar Bush et al. As we may think. The atlantic monthly, 176(1):

101–108, 1945.
680 the atlas for the aspiring network scientist

Sebastián Bustos, Charles Gomez, Ricardo Hausmann, and César A

Hidalgo. The dynamics of nestedness predicts the evolution of
industrial ecosystems. PloS one, 7(11):e49393, 2012.

Qing Cai, Lijia Ma, Maoguo Gong, and Dayong Tian. A survey on
network community detection based on evolutionary computation.
IJBIC, 8(2):84–98, 2016.

Alberto Caimo and Isabella Gollini. A multilayer exponential random

graph modelling approach for weighted networks. Computational
Statistics & Data Analysis, 142:106825, 2020.

Alberto Cairo. The Functional Art: An introduction to information

graphics and visualization. New Riders, 2012.

Guido Caldarelli. Scale-free networks: complex webs in nature and

technology. Oxford University Press, 2007.

Guido Caldarelli and Michele Catanzaro. Networks: A very short

introduction, volume 335. Oxford University Press, 2012.

Duncan S Callaway, Mark EJ Newman, Steven H Strogatz, and

Duncan J Watts. Network robustness and fragility: Percolation on
random graphs. Physical review letters, 85(25):5468, 2000.

Colin Campbell, Suann Yang, Réka Albert, and Katriona Shea. A

network model for plant–pollinator community assembly. Proceedings
of the National Academy of Sciences, 108(1):197–202, 2011.

Julián Candia, Marta C González, Pu Wang, Timothy Schoenharl,

Greg Madey, and Albert-László Barabási. Uncovering individual and
collective human dynamics from mobile phone records. Journal of
physics A: mathematical and theoretical, 41(22):224015, 2008.

Carlo Vittorio Cannistraci, Gregorio Alanis-Lobato, and Timothy

Ravasi. From link-prediction in brain connectomes and protein in-
teractomes to the local-community-paradigm in complex networks.
Scientific reports, 3:1613, 2013.

Shaosheng Cao, Wei Lu, and Qiongkai Xu. Grarep: Learning graph
representations with global structural information. In Proceedings of
the 24th ACM international on conference on information and knowledge
management, pages 891–900, 2015.

Shaosheng Cao, Wei Lu, and Qiongkai Xu. Deep neural networks
for learning graph representations. In Thirtieth AAAI conference on
artificial intelligence, 2016.
 bibliography 681

Amedeo Cappelli, Michele Coscia, Fosca Giannotti, Dino Pedreschi,

and Salvo Rinzivillo. The social network of dante’s inferno. Leonardo,
44(3):246–247, 2011.

Alessio Cardillo, Jesús Gómez-Gardenes, Massimiliano Zanin,

Miguel Romance, David Papo, Francisco Del Pozo, and Stefano
Boccaletti. Emergence of network features from multiplexity. Scientific
reports, 3:1344, 2013.

Shai Carmi, Shlomo Havlin, Scott Kirkpatrick, Yuval Shavitt, and

Eran Shir. A model of internet topology using k-shell decomposition.
Proceedings of the National Academy of Sciences, 104(27):11150–11154,
2007.

J Douglas Carroll and Jih-Jie Chang. Analysis of individual differ-

ences in multidimensional scaling via an n-way generalization of
“eckart-young” decomposition. Psychometrika, 35(3):283–319, 1970.

Claudio Castellano and Romualdo Pastor-Satorras. Thresholds for

epidemic spreading in networks. Physical review letters, 105(21):218701,
2010.

Claudio Castellano, Santo Fortunato, and Vittorio Loreto. Statistical

physics of social dynamics. Reviews of modern physics, 81(2):591, 2009.

Ciro Cattuto, Wouter Van den Broeck, Alain Barrat, Vittoria Colizza,
Jean-François Pinton, and Alessandro Vespignani. Dynamics of
person-to-person interactions from distributed rfid sensor networks.
PloS one, 5(7), 2010.

Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn,

and Sue Moon. I tube, you tube, everybody tubes: analyzing the
world’s largest user generated content video system. In SIGCOMM,
pages 1–14, 2007.

Meeyoung Cha, Alan Mislove, Ben Adams, and Krishna P Gum-

madi. Characterizing social cascades in flickr. In Proceedings of the first
workshop on Online social networks, pages 13–18, 2008.

Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn,

and Sue Moon. Analyzing the video popularity characteristics of
large-scale user generated content systems. IEEE/ACM Transactions
on networking, 17(5):1357–1370, 2009a.

Meeyoung Cha, Alan Mislove, and Krishna P Gummadi. A

measurement-driven analysis of information propagation in the
flickr social network. In WWW, pages 721–730, 2009b.
682 the atlas for the aspiring network scientist

Meeyoung Cha, Hamed Haddadi, Fabricio Benevenuto, and Kr-

ishna P Gummadi. Measuring user influence in twitter: The million
follower fallacy. In ICWSM, 2010.

Deepayan Chakrabarti, Ravi Kumar, and Andrew Tomkins. Evo-

lutionary clustering. In Proceedings of the 12th ACM SIGKDD in-
ternational conference on Knowledge discovery and data mining, pages
554–560. ACM, 2006.

Arun G Chandrasekhar and Matthew O Jackson. Tractable and

consistent random graph models. Technical report, National Bureau
of Economic Research, 2014.

Shiyu Chang, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C Aggar-
wal, and Thomas S Huang. Heterogeneous network embedding via
deep architectures. In SIGKDD, pages 119–128, 2015.

Gary Chartrand, Grzegorz Kubicki, and Michelle Schultz. Graph

similarity and distance in graphs. Aequationes Mathematicae, 55(1-2):
129–145, 1998.

Surajit Chaudhuri and Umeshwar Dayal. An overview of data

warehousing and olap technology. ACM Sigmod record, 26(1):65–74,
1997.

Bernard Chazelle. A minimum spanning tree algorithm with inverse-

ackermann type complexity. Journal of the ACM (JACM), 47(6):
1028–1047, 2000.

Chen Chen, Xifeng Yan, Feida Zhu, Jiawei Han, and S Yu Philip.
Graph olap: Towards online analytical processing on graphs. In
ICDM, pages 103–112. IEEE, 2008.

Chen Chen, Hanghang Tong, B Aditya Prakash, Charalampos E

Tsourakakis, Tina Eliassi-Rad, Christos Faloutsos, and Duen Horng
Chau. Node immunization on large graphs: Theory and algorithms.
IEEE Transactions on Knowledge and Data Engineering, 28(1):113–126,
2015.

Haochen Chen, Bryan Perozzi, Yifan Hu, and Steven Skiena. Harp:
Hierarchical representation learning for networks. In Thirty-Second
AAAI Conference on Artificial Intelligence, 2018.

Zhengzhang Chen, Kevin A Wilson, Ye Jin, William Hendrix, and

Nagiza F Samatova. Detecting and tracking community dynamics in
evolutionary networks. In ICDMW, pages 318–327. IEEE, 2010.

Justin Cheng, Lada Adamic, P Alex Dow, Jon Michael Kleinberg,

and Jure Leskovec. Can cascades be predicted? In WWW, pages
925–936. ACM, 2014.
 bibliography 683

Justin Cheng, Lada A Adamic, Jon M Kleinberg, and Jure Leskovec.

Do cascades recur? In WWW, pages 671–681. ACM, 2016.

Michael Chertok and Yosi Keller. Efficient high order matching.

IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(12):
2205–2215, 2010.

Kai-Yang Chiang, Nagarajan Natarajan, Ambuj Tewari, and Inder-

jit S Dhillon. Exploiting longer cycles for link prediction in signed
networks. In Proceedings of the 20th ACM international conference on
Information and knowledge management, pages 1157–1162. ACM, 2011.

Eunjoon Cho, Seth A Myers, and Jure Leskovec. Friendship and

mobility: user movement in location-based social networks. In
SIGKDD, pages 1082–1090. ACM, 2011.

Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry

Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio.
Learning phrase representations using rnn encoder–decoder for
statistical machine translation. In EMNLP, pages 1724–1734, 2014.

Sung-Bae Cho and Jin H Kim. Multiple network fusion using fuzzy
logic. IEEE Transactions on Neural Networks, 6(2):497–501, 1995.

Nicholas A Christakis and James H Fowler. The spread of obesity in

a large social network over 32 years. New England journal of medicine,
357(4):370–379, 2007.

Nicholas A Christakis and James H Fowler. The collective dynamics

of smoking in a large social network. New England journal of medicine,
358(21):2249–2258, 2008.

Walter Christaller. Central places in southern Germany. Prentice Hall,

1966.

Fan Chung and Linyuan Lu. The average distances in random graphs
with given expected degrees. Proceedings of the National Academy of
Sciences, 99(25):15879–15882, 2002a.

Fan Chung and Linyuan Lu. Connected components in random

graphs with given expected degree sequences. Annals of combinatorics,
6(2):125–145, 2002b.

A Clauset, E Tucker, and M Sainz. The colorado index of complex

networks, 2016.

Aaron Clauset. Finding local community structure in networks.

Physical review E, 72(2):026132, 2005.
684 the atlas for the aspiring network scientist

Aaron Clauset, Mark EJ Newman, and Cristopher Moore. Finding

community structure in very large networks. Physical review E, 70(6):
066111, 2004.

Aaron Clauset, Cristopher Moore, and Mark EJ Newman. Hierarchi-

cal structure and the prediction of missing links in networks. Nature,
453(7191):98, 2008.

Aaron Clauset, Cosma Rohilla Shalizi, and Mark EJ Newman. Power-

law distributions in empirical data. SIAM review, 51(4):661–703,
2009.

Aaron Clauset, Samuel Arbesman, and Daniel B Larremore. Sys-

tematic inequality and hierarchy in faculty hiring networks. Science
advances, 1(1):e1400005, 2015.

David Clayton, Michael Hills, and A Pickles. Statistical models in

epidemiology, volume 161. Oxford university press Oxford, 1993.

William S Cleveland and Robert McGill. Graphical perception:

Theory, experimentation, and application to the development of
graphical methods. Journal of the American statistical association, 79
(387):531–554, 1984.

Edith Cohen and Haim Kaplan. Spatially-decaying aggregation over

a network. Journal of Computer and System Sciences, 73(3):265–288,
2007.

Reuven Cohen, Keren Erez, Daniel Ben-Avraham, and Shlomo

Havlin. Resilience of the internet to random breakdowns. Physical
review letters, 85(21):4626, 2000.

Reuven Cohen, Keren Erez, Daniel Ben-Avraham, and Shlomo

Havlin. Breakdown of the internet under intentional attack. Physical
review letters, 86(16):3682, 2001.

Reuven Cohen, Shlomo Havlin, and Daniel Ben-Avraham. Efficient

immunization strategies for computer networks and populations.
Physical review letters, 91(24):247901, 2003.

Jonathan R Cole. Fair science: Women in the scientific community.

1979.

Jonathan R Cole and Stephen Cole. Social stratification in science.

1974.

Vittoria Colizza, Alain Barrat, Marc Barthélemy, and Alessandro

Vespignani. The role of the airline transportation network in the
prediction and predictability of global epidemics. Proceedings of the
 bibliography 685

National Academy of Sciences of the United States of America, 103(7):

2015–2020, 2006a.

Vittoria Colizza, Alessandro Flammini, M Angeles Serrano, and

Alessandro Vespignani. Detecting rich-club ordering in complex
networks. Nature physics, 2(2):110–115, 2006b.

Vittoria Colizza, Alain Barrat, Marc Barthelemy, Alain-Jacques

Valleron, and Alessandro Vespignani. Modeling the worldwide
spread of pandemic influenza: baseline case and containment
interventions. PLoS medicine, 4(1), 2007.

Charles R Collins and Kenneth Stephenson. A circle packing

algorithm. Computational Geometry, 25(3):233–256, 2003.

Linda M Collins and Clyde W Dent. Omega: A general formula-

tion of the rand index of cluster recovery suitable for non-disjoint
solutions. Multivariate Behavioral Research, 23(2):231–242, 1988.

Kathryn Cooper and Mauricio Barahona. Role-based similarity in

directed networks. arXiv preprint arXiv:1012.2726, 2010.

Gennaro Cordasco and Luisa Gargano. Community detection via

semi-synchronous label propagation algorithms. In 2010 IEEE
International Workshop on: Business Applications of Social Network
Analysis (BASNA), pages 1–8. IEEE, 2010.

Luigi P Cordella, Pasquale Foggia, Carlo Sansone, and Mario Vento.

A (sub) graph isomorphism algorithm for matching large graphs.
IEEE transactions on pattern analysis and machine intelligence, 26(10):
1367–1372, 2004.

Luigi Pietro Cordella, Pasquale Foggia, Carlo Sansone, and Mario

Vento. An improved algorithm for matching large graphs. In
3rd IAPR-TC15 workshop on graph-based representations in pattern
recognition, pages 149–159, 2001.

Michele Coscia. Average is boring: How similarity kills a meme’s

success. Scientific reports, 4:6477, 2014.

Michele Coscia. Popularity spikes hurt future chances for viral

propagation of protomemes. Communications of the ACM, 61(1):70–77,
2017.

Michele Coscia. Using arborescences to estimate hierarchicalness in

directed complex networks. PloS one, 13(1):e0190825, 2018.

Michele Coscia. Discovering communities of community discovery. In

Proceedings of the 2019 IEEE/ACM International Conference on Advances
in Social Networks Analysis and Mining, pages 1–8, 2019.
686 the atlas for the aspiring network scientist

Michele Coscia. Generalized euclidean measure to estimate network

distances. In Proceedings of the International AAAI Conference on Web
and Social Media, volume 14, pages 119–129, 2020.

Michele Coscia and Ricardo Hausmann. Evidence that calls-based

and mobility networks are isomorphic. PloS one, 10(12):e0145091,
2015.

Michele Coscia and Frank MH Neffke. Network backboning with

noisy data. In 2017 IEEE 33rd International Conference on Data Engineer-
ing (ICDE), pages 425–436. IEEE, 2017.

Michele Coscia and Viridiana Rios. Knowing where and how

criminal organizations operate using web content. In Proceedings
of the 21st ACM international conference on Information and knowledge
management, pages 1412–1421, 2012.

Michele Coscia and Luca Rossi. Benchmarking api costs of network

sampling strategies. In 2018 IEEE International Conference on Big Data
(Big Data), pages 663–672. IEEE, 2018.

Michele Coscia and Michael Szell. Multiplex graph association rules

for link prediction. arXiv preprint arXiv:2008.08351, 2020.

Michele Coscia, Fosca Giannotti, and Dino Pedreschi. A classification

for community discovery methods in complex networks. SADM, 4
(5):512–546, 2011.

Michele Coscia, Giulio Rossetti, Fosca Giannotti, and Dino Pe-

dreschi. Demon: a local-first discovery method for overlapping
communities. In Proceedings of the 18th ACM SIGKDD international
conference on Knowledge discovery and data mining, pages 615–623.
ACM, 2012.

Michele Coscia, Ricardo Hausmann, and César A Hidalgo. The

structure and dynamics of international development assistance.
Journal of Globalization and Development, 3(2):1–42, 2013a.

Michele Coscia, Giulio Rossetti, Diego Pennacchioli, Damiano

Ceccarelli, and Fosca Giannotti. You know because i know: a
multidimensional network approach to human resources problem. In
Proceedings of the 2013 IEEE/ACM International Conference on Advances
in Social Networks Analysis and Mining, pages 434–441. ACM, 2013b.

Michele Coscia, Giulio Rossetti, Fosca Giannotti, and Dino Pe-

dreschi. Uncovering hierarchical and overlapping communities with
a local-first approach. ACM Transactions on Knowledge Discovery from
Data (TKDD), 9(1):6, 2014.
 bibliography 687

Michele Coscia, Andres Gomez-Lievano, James McNerney, and

Frank Neffke. The node vector distance problem in complex net-
works. ACM Computing Surveys, 2020.

Owen T Courtney and Ginestra Bianconi. Generalized network

structures: The configuration model and the canonical ensemble of
simplicial complexes. Physical Review E, 93(6):062311, 2016.

Skyler J Cranmer and Bruce A Desmarais. Inferential network

analysis with exponential random graph models. Political analysis, 19
(1):66–86, 2011.

Matthieu Cristelli, Andrea Tacchella, and Luciano Pietronero. The

heterogeneous dynamics of economic complexity. PloS one, 10(2):
e0117174, 2015.

Nick Crossley, Elisa Bellotti, Gemma Edwards, Martin G Everett,

Johan Koskinen, and Mark Tranmer. Social network analysis for
ego-nets: Social network analysis for actor-centred networks. Sage, 2015.

Paolo Crucitti, Vito Latora, Massimo Marchiori, and Andrea Rapis-

arda. Error and attack tolerance of complex networks. Physica A:
Statistical mechanics and its applications, 340(1-3):388–394, 2004.

Gabor Csardi and Tamas Nepusz. The igraph software package for
complex network research. InterJournal, Complex Systems, 1695(5):1–9,
2006.

Peter Csermely, Tamás Korcsmáros, Huba JM Kiss, Gabor London,

and Ruth Nussinov. Structure and dynamics of molecular networks:
a novel paradigm of drug discovery: a comprehensive review.
Pharmacology & therapeutics, 138(3):333–408, 2013a.

Peter Csermely, András London, Ling-Yun Wu, and Brian Uzzi.

Structure and dynamics of core/periphery networks. Journal of
Complex Networks, 1(2):93–123, 2013b.

Gregorio D’Agostino and Antonio Scala. Networks of networks: the last

frontier of complexity, volume 340. Springer, 2014.

Jesper Dall and Michael Christensen. Random geometric graphs.

Physical review E, 66(1):016121, 2002.

Simone Daminelli, Josephine Maria Thomas, Claudio Durán, and

Carlo Vittorio Cannistraci. Common neighbours and the local-
community-paradigm for topological link prediction in bipartite
networks. New Journal of Physics, 17(11):113037, 2015.
688 the atlas for the aspiring network scientist

Leon Danon, Albert Diaz-Guilera, Jordi Duch, and Alex Arenas.

Comparing community structure identification. Journal of Statistical
Mechanics: Theory and Experiment, 2005(09):P09008, 2005.

Vinh-Loc Dao, Cécile Bothorel, and Philippe Lenca. Community

structure: A comparative evaluation of community detection meth-
ods. arXiv preprint arXiv:1812.06598, 2018a.

Vinh-Loc Dao, Cécile Bothorel, and Philippe Lenca. Estimating the

similarity of community detection methods based on cluster size
distribution. In International Workshop on Complex Networks and their
Applications, pages 183–194. Springer, 2018b.

Charles Darwin. On the origin of species. 1859.

Anirban Dasgupta, Ravi Kumar, and D Sivakumar. Social sampling.

In Proceedings of the 18th ACM SIGKDD international conference on
Knowledge discovery and data mining, pages 235–243. ACM, 2012.

Ron Davidson and David Harel. Drawing graphs nicely using

simulated annealing. TOG, 15(4):301–331, 1996.

Allison Davis, Burleigh Bradford Gardner, and Mary R Gardner. Deep

South: A social anthropological study of caste and class. Univ of South
Carolina Press, 1941.

Darcy Davis, Ryan Lichtenwalter, and Nitesh V Chawla. Multi-

relational link prediction in heterogeneous information networks. In
ASONAM, pages 281–288. IEEE, 2011.

Caterina De Bacco, Eleanor A Power, Daniel B Larremore, and

Cristopher Moore. Community detection, link prediction, and layer
interdependence in multilayer networks. Physical Review E, 95(4):
042317, 2017.

Nicola De Cao and Thomas Kipf. Molgan: An implicit generative

model for small molecular graphs. arXiv preprint arXiv:1805.11973,
2018.

Manlio De Domenico. Multilayer modeling and analysis of human

brain networks. Giga Science, 6(5):gix004, 2017.

Manlio De Domenico, Albert Solé-Ribalta, Emanuele Cozzo, Mikko

Kivelä, Yamir Moreno, Mason A Porter, Sergio Gómez, and Alex
Arenas. Mathematical formulation of multilayer networks. Physical
Review X, 3(4):041022, 2013.

Manlio De Domenico, Albert Solé-Ribalta, Sergio Gómez, and Alex

Arenas. Navigability of interconnected networks under random
 bibliography 689

failures. Proceedings of the National Academy of Sciences, 111(23):

8351–8356, 2014.

Manlio De Domenico, Andrea Lancichinetti, Alex Arenas, and

Martin Rosvall. Identifying modular flows on multilayer networks
reveals highly overlapping organization in interconnected systems.
Physical Review X, 5(1):011027, 2015a.

Manlio De Domenico, Vincenzo Nicosia, Alexandre Arenas, and

Vito Latora. Structural reducibility of multilayer networks. Nature
communications, 6:6864, 2015b.

Manlio De Domenico, Mason A Porter, and Alex Arenas. Muxviz: a

tool for multilayer analysis and visualization of networks. Journal of
Complex Networks, 3(2):159–176, 2015c.

Manlio De Domenico, Albert Solé-Ribalta, Elisa Omodei, Sergio

Gómez, and Alex Arenas. Ranking in interconnected multilayer
networks reveals versatile nodes. Nature communications, 6:6868,
2015d.

Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. A

multilinear singular value decomposition. SIAM journal on Matrix
Analysis and Applications, 21(4):1253–1278, 2000.

Pasquale De Meo, Emilio Ferrara, Giacomo Fiumara, and Alessan-

dro Provetti. Mixing local and global information for community
detection in large networks. Journal of Computer and System Sciences,
80(1):72–87, 2014.

Yves-Alexandre De Montjoye, César A Hidalgo, Michel Verleysen,

and Vincent D Blondel. Unique in the crowd: The privacy bounds of
human mobility. Scientific reports, 3:1376, 2013.

Wouter De Nooy. A literary playground: Literary criticism and

balance theory. Poetics, 26(5-6):385–404, 1999.

Wouter De Nooy, Andrej Mrvar, and Vladimir Batagelj. Exploratory

social network analysis with Pajek. Cambridge University Press, 2018.

Ithiel de Sola Pool and Manfred Kochen. Contacts and influence.

Social networks, 1(1):5–51, 1978.

Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Con-

volutional neural networks on graphs with fast localized spectral
filtering. In NIPS, pages 3844–3852, 2016.

Morris H DeGroot. Reaching a consensus. Journal of the American

Statistical Association, 69(345):118–121, 1974.
690 the atlas for the aspiring network scientist

Michela Del Vicario, Alessandro Bessi, Fabiana Zollo, Fabio Petroni,

Antonio Scala, Guido Caldarelli, H Eugene Stanley, and Walter
Quattrociocchi. The spreading of misinformation online. PNAS, 113
(3):554–559, 2016a.

Michela Del Vicario, Gianna Vivaldo, Alessandro Bessi, Fabiana

Zollo, Antonio Scala, Guido Caldarelli, and Walter Quattrociocchi.
Echo chambers: Emotional contagion and group polarization on
facebook. Scientific reports, 6:37825, 2016b.

Michela Del Vicario, Antonio Scala, Guido Caldarelli, H Eugene

Stanley, and Walter Quattrociocchi. Modeling confirmation bias and
polarization. Scientific reports, 7:40391, 2017.

Fabio Della Rossa, Fabio Dercole, and Carlo Piccardi. Profiling core-
periphery network structure by random walkers. Scientific reports, 3:
1467, 2013.

J-C Delvenne, Sophia N Yaliraki, and Mauricio Barahona. Stability

of graph communities across time scales. Proceedings of the national
academy of sciences, 107(29):12755–12760, 2010.

Jean-Charles Delvenne, Michael T Schaub, Sophia N Yaliraki, and

Mauricio Barahona. The stability of a graph partition: A dynamics-
based framework for community detection. In Dynamics On and Of
Complex Networks, Volume 2, pages 221–242. Springer, 2013.

Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum

likelihood from incomplete data via the em algorithm. Journal of the
Royal Statistical Society: Series B (Methodological), 39(1):1–22, 1977.

Li Deng, Dong Yu, et al. Deep learning: methods and applications.

Foundations and Trends® in Signal Processing, 7(3–4):197–387, 2014.

Xiaoheng Deng, Genghao Li, and Mianxiong Dong. Finding overlap-

ping communities with random walks on line graph and attraction
intensity. In International Conference on Wireless Algorithms, Systems,
and Applications, pages 94–103. Springer, 2015.

Imre Derényi, Gergely Palla, and Tamás Vicsek. Clique percolation

in random networks. Physical review letters, 94(16):160202, 2005.

Karine Descormiers and Carlo Morselli. Alliances, conflicts, and

contradictions in montreal’s street gang landscape. International
Criminal Justice Review, 21(3):297–314, 2011.

Bruce A Desmarais and Skyler J Cranmer. Statistical inference for

valued-edge networks: The generalized exponential random graph
model. PloS one, 7(1):e30136, 2012.
 bibliography 691

T Dettmers, P Minervini, P Stenetorp, and S Riedel. Convolutional

2d knowledge graph embeddings. In 32nd AAAI Conference on
Artificial Intelligence, AAAI 2018, volume 32, pages 1811–1818. AAI
Publications, 2018.

Pierre Deville, Dashun Wang, Roberta Sinatra, Chaoming Song,

Vincent D Blondel, and Albert-László Barabási. Career on the move:
Geography, stratification, and scientific impact. Scientific reports, 4:
4770, 2014.

Pierre Deville, Chaoming Song, Nathan Eagle, Vincent D Blondel,

Albert-László Barabási, and Dashun Wang. Scaling identity connects
human mobility and social interactions. PNAS, 113(26):7047–7052,
2016.

Michel Marie Deza and Elena Deza. Encyclopedia of distances. In

Encyclopedia of distances, pages 1–583. Springer, 2009.

Inderjit S Dhillon. Co-clustering documents and words using

bipartite spectral graph partitioning. In SIGKDD, pages 269–274,
2001.

Inderjit S Dhillon, Subramanyam Mallela, and Dharmendra S

Modha. Information-theoretic co-clustering. In SIGKDD, pages 89–98,
2003.

Robert B Dial. Algorithm 360: Shortest-path forest with topological

ordering [h]. Communications of the ACM, 12(11):632–633, 1969.

Navid Dianati. Unwinding the hairball graph: pruning algorithms

for weighted complex networks. Physical Review E, 93(1):012304, 2016.

Mark E Dickison, Matteo Magnani, and Luca Rossi. Multilayer social

networks. Cambridge University Press, 2016.

Reinhard Diestel. Graph theory. Springer Publishing Company,

Incorporated, 2018.

Edsger W Dijkstra. A note on two problems in connexion with

graphs. Numerische mathematik, 1(1):269–271, 1959.

Andrew Dobson, Kiril Solovey, Rahul Shome, Dan Halperin, and

Kostas E Bekris. Scalable asymptotically-optimal multi-robot motion
planning. In 2017 International Symposium on Multi-Robot and Multi-
Agent Systems (MRS), pages 120–127. IEEE, 2017.

Ian Dobson, Benjamin A Carreras, and David E Newman. A loading-

dependent model of probabilistic cascading failure. Probability in the
Engineering and Informational Sciences, 19(1):15–32, 2005.
692 the atlas for the aspiring network scientist

Ian Dobson, Benjamin A Carreras, Vickie E Lynch, and David E

Newman. Complex systems analysis of series of blackouts: Cas-
cading failure, critical points, and self-organization. Chaos: An
Interdisciplinary Journal of Nonlinear Science, 17(2):026103, 2007.

Ugur Dogrusoz, Erhan Giral, Ahmet Cetintas, Ali Civril, and Emek
Demir. A layout algorithm for undirected compound graphs.
Information Sciences, 179(7):980–994, 2009.

Pedro Domingos and Matt Richardson. Mining the network value

of customers. In Proceedings of the seventh ACM SIGKDD international
conference on Knowledge discovery and data mining, pages 57–66. ACM,
2001.

Xiaowen Dong, Pascal Frossard, Pierre Vandergheynst, and Nikolai

Nefedov. Clustering on multi-layer graphs via subspace analysis on
grassmann manifolds. IEEE Transactions on signal processing, 62(4):
905–918, 2013.

Yuxiao Dong, Jie Tang, Sen Wu, Jilei Tian, Nitesh V Chawla, Jinghai
Rao, and Huanhuan Cao. Link prediction and recommendation
across heterogeneous social networks. In 2012 IEEE 12th International
conference on data mining, pages 181–190. IEEE, 2012.

Yuxiao Dong, Nitesh V Chawla, and Ananthram Swami. metap-

ath2vec: Scalable representation learning for heterogeneous net-
works. In SIGKDD, pages 135–144, 2017.

Stijn Dongen. A cluster algorithm for graphs. 2000.

Sergey N Dorogovtsev and Jose FF Mendes. Evolution of networks.

Advances in physics, 51(4):1079–1187, 2002.

Sergey N Dorogovtsev, José Fernando F Mendes, and Alexander N

Samukhin. Giant strongly connected component of directed networks.
Physical Review E, 64(2):025101, 2001.

Constantinos Apostolos Doxiadis et al. Ekistics; an introduction to

the science of human settlements. 1968.

Xianzhi Du, Mostafa El-Khamy, Jungwon Lee, and Larry Davis.

Fused dnn: A deep neural network fusion approach to fast and
robust pedestrian detection. In 2017 IEEE winter conference on
applications of computer vision (WACV), pages 953–961. IEEE, 2017.

Jordi Duch and Alex Arenas. Community detection in complex

networks using extremal optimization. Physical review E, 72(2):027104,
2005.
 bibliography 693

Olivier Duchenne, Francis Bach, In-So Kweon, and Jean Ponce.

A tensor-based algorithm for high-order graph matching. IEEE
transactions on pattern analysis and machine intelligence, 33(12):2383–
2395, 2011.

Daniel M Dunlavy, Tamara G Kolda, and Evrim Acar. Temporal link

prediction using matrix and tensor factorizations. ACM Transactions
on Knowledge Discovery from Data (TKDD), 5(2):10, 2011.

Cody Dunne and Ben Shneiderman. Motif simplification: improving

network visualization readability with fan, connector, and clique
glyphs. In Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems, pages 3247–3256, 2013.

Rick Durrett. Probability: theory and examples, volume 49. Duxbury

Press, 1996.

Rick Durrett. Some features of the spread of epidemics and infor-

mation on a random graph. Proceedings of the National Academy of
Sciences, 2010.

David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael

Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams.
Convolutional networks on graphs for learning molecular finger-
prints. In NIPS, pages 2224–2232, 2015.

Cynthia Dwork, Ravi Kumar, Moni Naor, and Dandapani Sivaku-

mar. Rank aggregation methods for the web. In Proceedings of the
10th international conference on World Wide Web, pages 613–622. ACM,
2001.

Tim Dwyer, Kim Marriott, and Michael Wybrow. Integrating edge

routing into force-directed layout. In International Symposium on Graph
Drawing, pages 8–19. Springer, 2006.

Raissa M D’Souza and Michael Mitzenmacher. Local cluster aggrega-

tion models of explosive percolation. Physical review letters, 104(19):
195702, 2010.

David Easley and Jon Kleinberg. Networks, crowds, and markets:

Reasoning about a highly connected world. Cambridge University Press,
2010.

Holger Ebel, Lutz-Ingo Mielsch, and Stefan Bornholdt. Scale-free

topology of e-mail networks. Physical review E, 66(3):035103, 2002.

Sergey Edunov, Carlos Diuk, Ismail Onur Filiz, Smriti Bhagat, and
Moira Burke. Three and a half degrees of separation. Research at
Facebook, 2016.
694 the atlas for the aspiring network scientist

Victor M Eguiluz, Dante R Chialvo, Guillermo A Cecchi, Marwan

Baliki, and A Vania Apkarian. Scale-free brain functional networks.
Physical review letters, 94(1):018102, 2005.

Albert Einstein. Über die von der molekularkinetischen theorie

der wärme geforderte bewegung von in ruhenden flüssigkeiten
suspendierten teilchen. Annalen der physik, 4, 1905.

Matthias Erbar, Martin Rumpf, Bernhard Schmitzer, and Stefan

Simon. Computation of optimal transport on discrete metric measure
spaces. arXiv preprint arXiv:1707.06859, 2017.

P Erdős and A Rényi. On random graphs. Publicationes Mathematicae

Debrecen, 6:290–297, 1959.

Paul Erdos and Alfréd Rényi. On the evolution of random graphs.

Publ. Math. Inst. Hung. Acad. Sci, 5(1):17–60, 1960.

Paul Erdős and Alfréd Rényi. On the strength of connectedness of a

random graph. Acta Mathematica Hungarica, 12(1-2):261–267, 1961.

Paul Erdos and Alfred Renyi. On random matrices. Magyar Tud. Akad.
Mat. Kutató Int. Közl, 8(455-461):1964, 1964.

Paul Erdos and Alfréd Rényi. On the existence of a factor of degree

one of a connected random graph. Acta Math. Acad. Sci. Hungar, 17
(3-4):359–368, 1966.

Paul Erdős and Miklós Simonovits. Supersaturated graphs and

hypergraphs. Combinatorica, 3(2):181–192, 1983.

Alcides Viamontes Esquivel and Martin Rosvall. Compression of flow

can reveal overlapping-module organization in networks. Physical
Review X, 1(2):021025, 2011.

Montacer Essid and Justin Solomon. Quadratically-regularized

optimal transport on graphs. arXiv preprint arXiv:1704.08200, 2017.

Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A

density-based algorithm for discovering clusters in large spatial
databases with noise. In Kdd, volume 96, pages 226–231, 1996.

Jean-Baptiste Estoup. Gammes sténographiques: méthode et exercices pour

l’acquisition de la vitesse. Institut sténographique, 1916.

Ernesto Estrada. The structure of complex networks: theory and applica-

tions. Oxford University Press, 2012.

Leonhard Euler. Solutio problematis ad geometriam situs pertinentis.

Commentarii academiae scientiarum Petropolitanae, pages 128–140, 1741.
 bibliography 695

Lawrence C Evans. Partial differential equations and monge-

kantorovich mass transfer. Current developments in mathematics, 1997
(1):65–126, 1997.

Tim S Evans. Clique graphs and overlapping communities. Journal of

Statistical Mechanics: Theory and Experiment, 2010(12):P12037, 2010.

TS Evans and Renaud Lambiotte. Line graphs, link partitions, and

overlapping communities. Physical Review E, 80(1):016105, 2009.

Shimon Even. Graph algorithms. Cambridge University Press, 2011.

Martin G Everett and Stephen P Borgatti. The dual-projection

approach for two-mode networks. Social Networks, 35(2):204–210,
2013.

Paul Expert, Tim S Evans, Vincent D Blondel, and Renaud Lam-

biotte. Uncovering space-independent communities in spatial
networks. Proceedings of the National Academy of Sciences, 108(19):
7663–7668, 2011.

Giorgio Fagiolo. Clustering in complex directed networks. Physical

Review E, 76(2):026107, 2007.

Tom Fawcett. An introduction to roc analysis. Pattern recognition

letters, 27(8):861–874, 2006.

Tomás Feder and Rajeev Motwani. Clique partitions, graph com-

pression and speeding-up algorithms. Journal of Computer and System
Sciences, 51(2):261–272, 1995.

Scott L Feld. Why your friends have more friends than you do.
American Journal of Sociology, 96(6):1464–1477, 1991.

Willliam Feller. An introduction to probability theory and its applications,

volume 2. John Wiley & Sons, 1968.

Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, and

Alessandro Flammini. The rise of social bots. Communications of the
ACM, 59(7):96–104, 2016.

Mathias Fiedler and Christian Borgelt. Support computation for

mining frequent subgraphs in a single graph. In MLG, 2007.

Miroslav Fiedler. Laplacian of graphs and algebraic connectivity.

Banach Center Publications, 25(1):57–70, 1989.

Alexander Fix, Aritanan Gruber, Endre Boros, and Ramin Zabih. A

graph cut algorithm for higher-order markov random fields. In 2011
International Conference on Computer Vision, pages 1020–1027. IEEE,
2011.
696 the atlas for the aspiring network scientist

Jessica C Flack, Michelle Girvan, Frans BM De Waal, and David C

Krakauer. Policing stabilizes construction of social niches in primates.
Nature, 439(7075):426–429, 2006.

Gary William Flake, Steve Lawrence, C Lee Giles, et al. Efficient

identification of web communities. In KDD, volume 2000, pages
150–160, 2000.

Pablo Fleurquin, José J Ramasco, and Victor M Eguiluz. Systemic

delay propagation in the us airport network. Scientific reports, 3:1159,
2013.

Robert W Floyd. Algorithm 97: shortest path. Communications of the

ACM, 5(6):345, 1962.

Klaus-Tycho Foerster, Linus Groner, Torsten Hoefler, Michael

Koenig, Sascha Schmid, and Roger Wattenhofer. Multi-agent
pathfinding with n agents on graphs with n vertices: Combinatorial
classification and tight algorithmic bounds. In International Conference
on Algorithms and Complexity, pages 247–259. Springer, 2017.

Fedor V Fomin and Dimitrios M Thilikos. An annotated bibliography

on guaranteed graph searching. Theoretical computer science, 399(3):
236–245, 2008.

Santo Fortunato. Community detection in graphs. Physics reports, 486

(3-5):75–174, 2010.

Santo Fortunato and Marc Barthelemy. Resolution limit in commu-

nity detection. Proceedings of the National Academy of Sciences, 104(1):
36–41, 2007.

Santo Fortunato and Darko Hric. Community detection in networks:

A user guide. Physics Reports, 659:1–44, 2016.

Santo Fortunato, Vito Latora, and Massimo Marchiori. Method to

find community structures based on information centrality. Physical
review E, 70(5):056104, 2004.

Santo Fortunato, Marián Boguñá, Alessandro Flammini, and Filippo

Menczer. Approximating pagerank from in-degree. In International
Workshop on Algorithms and Models for the Web-Graph, pages 59–71.
Springer, 2006.

Santo Fortunato, Carl T Bergstrom, Katy Börner, James A Evans,

Dirk Helbing, Staša Milojević, Alexander M Petersen, Filippo Radic-
chi, Roberta Sinatra, Brian Uzzi, et al. Science of science. Science, 359
(6379):eaao0185, 2018.
 bibliography 697

Bailey K Fosdick, Daniel B Larremore, Joel Nishimura, and Johan

Ugander. Configuring random graph models with fixed degree
sequences. SIAM Review, 60(2):315–355, 2018.

Jacob G Foster, David V Foster, Peter Grassberger, and Maya

Paczuski. Edge direction and the structure of networks. Proceedings of
the National Academy of Sciences, 107(24):10815–10820, 2010.

Samuel P Fraiberger, Roberta Sinatra, Magnus Resch, Christoph

Riedl, and Albert-László Barabási. Quantifying reputation and
success in art. Science, 362(6416):825–829, 2018.

B Francis. A course in H 1 control theory. Lectures notes in control and

information sciences, volume 88. Springer Verlag Berlin, 1987.

Ove Frank and David Strauss. Markov graphs. Journal of the american
Statistical association, 81(395):832–842, 1986.

Linton C Freeman. Centrality in social networks conceptual clarifica-

tion. Social networks, 1(3):215–239, 1978.

Linton C Freeman, Douglas Roeder, and Robert R Mulholland.

Centrality in social networks: Ii. experimental results. Social networks,
2(2):119–141, 1979.

Arne Frick, Andreas Ludwig, and Heiko Mehldau. A fast adaptive

layout algorithm for undirected graphs (extended abstract and
system demonstration). In International Symposium on Graph Drawing,
pages 388–403. Springer, 1994.

Jonathan Friedman and Eric J Alm. Inferring correlation networks

from genomic survey data. PLoS computational biology, 8(9):e1002687,
2012.

Nir Friedman, Dan Geiger, and Moises Goldszmidt. Bayesian

network classifiers. Machine learning, 29(2-3):131–163, 1997.

Nir Friedman, Lise Getoor, Daphne Koller, and Avi Pfeffer. Learning
probabilistic relational models. In IJCAI, volume 99, pages 1300–1309,
1999.

Karl Friston. The free-energy principle: a unified brain theory?

Nature reviews neuroscience, 11(2):127–138, 2010.

Thomas MJ Fruchterman and Edward M Reingold. Graph drawing

by force-directed placement. Software: Practice and experience, 21(11):
1129–1164, 1991.
698 the atlas for the aspiring network scientist

Wenjie Fu, Le Song, and Eric P Xing. Dynamic mixed membership

blockmodel for evolving networks. In Proceedings of the 26th annual
international conference on machine learning, pages 329–336. ACM,
2009.

Daniel Funke, Sebastian Lamm, Ulrich Meyer, Manuel Penschuck,

Peter Sanders, Christian Schulz, Darren Strash, and Moritz von
Looz. Communication-free massively distributed graph generation.
Journal of Parallel and Distributed Computing, 131:200–217, 2019.

E Gabasova. The star wars social network. Evelina Gabasova’s

Blog. Data available at: [Link] com/evelinag/StarWars-social-
network/tree/master/networks, 2015.

David J Galas, Gregory Dewey, James Kunert-Graf, and Nikita A

Sakhanenko. Expansion of the kullback-leibler divergence, and a new
class of information metrics. Axioms, 6(2):8, 2017.

Galileo Galilei. Dialogo sopra i due massimi sistemi del mondo, 1632.

Ryan J Gallagher, Jean-Gabriel Young, and Brooke Foucault Welles.

A clarified typology of core-periphery structure in networks. arXiv
preprint arXiv:2005.10191, 2020.

Ayalvadi Ganesh, Laurent Massoulié, and Don Towsley. The effect

of network topology on the spread of epidemics. In INFOCOM 2005.
24th Annual Joint Conference of the IEEE Computer and Communications
Societies. Proceedings IEEE, volume 2, pages 1455–1466. IEEE, 2005.

Emden R Gansner and Yifan Hu. Efficient node overlap removal

using a proximity stress model. In International Symposium on Graph
Drawing, pages 206–217. Springer, 2008.

Emden R Gansner and Yehuda Koren. Improved circular layouts. In

International Symposium on Graph Drawing, pages 386–398. Springer,
2006.

Emden R Gansner, Yifan Hu, Stephen North, and Carlos Scheideg-

ger. Multilevel agglomerative edge bundling for visualizing large
graphs. In 2011 IEEE Pacific Visualization Symposium, pages 187–194.
IEEE, 2011.

Jianxi Gao, Sergey V Buldyrev, Shlomo Havlin, and H Eugene

Stanley. Robustness of a network of networks. Phys. Rev. Lett., 107:
195701, Nov 2011. doi: 10.1103/PhysRevLett.107.195701.

Jianxi Gao, Sergey V Buldyrev, H Eugene Stanley, and Shlomo

Havlin. Networks formed from interdependent networks. Nature
physics, 8(1):40, 2012.
 bibliography 699

Jianxi Gao, Yang-Yu Liu, Raissa M D’souza, and Albert-László

Barabási. Target control of complex networks. Nature communications,
5(1):1–8, 2014.

Jianxi Gao, Baruch Barzel, and Albert-László Barabási. Universal

resilience patterns in complex networks. Nature, 530(7590):307–312,
2016.

Xinbo Gao, Bing Xiao, Dacheng Tao, and Xuelong Li. A survey of
graph edit distance. Pattern Analysis and applications, 13(1):113–129,
2010.

David Garcia. Leaking privacy and shadow profiles in online social

networks. Science advances, 3(8):e1701172, 2017.

Thomas Gärtner, Peter Flach, and Stefan Wrobel. On graph kernels:

Hardness results and efficient alternatives. In Learning theory and
kernel machines, pages 129–143. Springer, 2003.

Alexander J Gates and Yong-Yeol Ahn. The impact of random models

on clustering similarity. The Journal of Machine Learning Research, 18
(1):3049–3076, 2017.

Alexander J Gates, Ian B Wood, William P Hetrick, and Yong-Yeol

Ahn. Element-centric clustering comparison unifies overlaps and
hierarchy. Scientific reports, 9(1):8574, 2019.

Laetitia Gauvin, André Panisson, and Ciro Cattuto. Detecting the

community structure and activity patterns of temporal networks: a
non-negative tensor factorization approach. PloS one, 9(1):e86028,
2014.

Valerio Gemmetto, Alessio Cardillo, and Diego Garlaschelli. Irre-

ducible network backbones: unbiased graph filtering via maximum
entropy. arXiv preprint arXiv:1706.00230, 2017.

Lise Getoor and Christopher P Diehl. Link mining: a survey. Acm

Sigkdd Explorations Newsletter, 7(2):3–12, 2005.

Amir Ghasemian, Pan Zhang, Aaron Clauset, Cristopher Moore,

and Leto Peel. Detectability thresholds and optimal algorithms for
community structure in dynamic networks. Physical Review X, 6(3):
031005, 2016.

Amir Ghasemian, Homa Hosseinmardi, and Aaron Clauset. Evaluat-

ing overfit and underfit in models of network community structure.
TKDE, 2019.
700 the atlas for the aspiring network scientist

Amir Ghasemian, Homa Hosseinmardi, Aram Galstyan, Edoardo M

Airoldi, and Aaron Clauset. Stacking models for nearly optimal link
prediction in complex networks. Proceedings of the National Academy
of Sciences, 117(38):23393–23400, 2020.

Mina Ghashami, Edo Liberty, and Jeff M Phillips. Efficient frequent

directions algorithm for sparse matrices. In SIGKDD, pages 845–854,
2016.

Gourab Ghoshal and Albert-László Barabási. Ranking stability and

super-stable nodes in complex networks. Nature communications, 2:
394, 2011.

Christos Giatsidis, Dimitrios M Thilikos, and Michalis Vazirgiannis.

D-cores: Measuring collaboration of directed graphs based on
degeneracy. In Data Mining (ICDM), 2011 IEEE 11th International
Conference on, pages 201–210. IEEE, 2011.

Edgar N Gilbert. Random graphs. The Annals of Mathematical

Statistics, 30(4):1141–1144, 1959.

Walter R Gilks, Sylvia Richardson, and David Spiegelhalter. Markov

chain Monte Carlo in practice. Chapman and Hall/CRC, 1995.

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals,

and George E Dahl. Neural message passing for quantum chemistry.
In ICML, pages 1263–1272. JMLR. org, 2017.

A Gimenez-Salinas Framis. Illegal networks or criminal organi-

zations: Power, roles and facilitators in four cocaine trafficking
structures. In Third Annual Illicit Networks Workshop, 2011.

Donna K Ginther, Walter T Schaffer, Joshua Schnell, Beth Masimore,

Faye Liu, Laurel L Haak, and Raynard Kington. Race, ethnicity, and
nih research awards. Science, 333(6045):1015–1019, 2011.

Herbert Gintis. The bounds of reason: Game theory and the unification of
the behavioral sciences. Princeton University Press, 2014.

Michelle Girvan and Mark EJ Newman. Community structure in

social and biological networks. Proceedings of the national academy of
sciences, 99(12):7821–7826, 2002.

Minas Gjoka, Maciej Kurant, Carter T Butts, and Athina

Markopoulou. Walking in facebook: A case study of unbiased
sampling of osns. In 2010 Proceedings IEEE Infocom, pages 1–9. Ieee,
2010.
 bibliography 701

Minas Gjoka, Maciej Kurant, Carter T Butts, and Athina

Markopoulou. Practical recommendations on crawling online
social networks. IEEE Journal on Selected Areas in Communications, 29
(9):1872–1892, 2011.

James P Gleeson, Jonathan A Ward, Kevin P O’sullivan, and

William T Lee. Competition-induced criticality in a model of meme
popularity. Physical review letters, 112(4):048701, 2014.

Pablo M Gleiser and Leon Danon. Community structure in jazz.

Advances in complex systems, 6(04):565–573, 2003.

Kurt Gödel. Über formal unentscheidbare sätze der principia

mathematica und verwandter systeme i. Monatshefte für mathematik
und physik, 38(1):173–198, 1931.

Julio E Godoy, Ioannis Karamouzas, Stephen J Guy, and Maria

Gini. Adaptive learning for multi-agent navigation. In Int Conf
on Autonomous Agents and Multiagent Systems, pages 1577–1585.
International Foundation for Autonomous Agents and Multiagent
Systems, 2015.

Chris Godsil and Gordon F Royle. Algebraic graph theory, volume 207.
Springer Science & Business Media, 2013.

Mark Goldberg, Malik Magdon-Ismail, Srinivas Nambirajan, and

James Thompson. Tracking and predicting evolution of social
communities. In SocialCom, pages 780–783. IEEE, 2011.

Jacob Goldenberg, Barak Libai, and Eitan Muller. Using complex

systems analysis to advance marketing theory development: Model-
ing heterogeneity effects on new product growth through stochastic
cellular automata. Academy of Marketing Science Review, 9(3):1–18,
2001.

Oded Goldreich. Finding the shortest move-sequence in the graph-

generalized 15-puzzle is np-hard., 2011.

Alexander V Goltsev, Sergey N Dorogovtsev, and Jose Ferreira F

Mendes. k-core (bootstrap) percolation on complex networks: Critical
phenomena and nonlocal effects. Physical Review E, 73(5):056101,
2006.

Andres Gomez-Lievano, HyeJin Youn, and Luis MA Bettencourt. The

statistics of urban scaling and their connection to zipf’s law. PloS one,
7(7), 2012.

Andres Gomez-Lievano, Oscar Patterson-Lomba, and Ricardo

Hausmann. Explaining the prevalence, scaling and variance of urban
phenomena. Nature Energy, pages 1–9, 2018.
702 the atlas for the aspiring network scientist

Marta C González, Hans J Herrmann, J Kertész, and Tamás Vicsek.

Community structure and ethnic preferences in school friendship
networks. Physica A, 379(1):307–316, 2007.

Marta C Gonzalez, Cesar A Hidalgo, and Albert-Laszlo Barabasi.

Understanding individual human mobility patterns. nature, 453(7196):
779–782, 2008.

Benjamin H Good, Yves-Alexandre De Montjoye, and Aaron Clauset.

Performance of modularity maximization in practical contexts.
Physical Review E, 81(4):046106, 2010.

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning.

MIT press, 2016.

Leo A Goodman. Snowball sampling. The annals of mathematical

statistics, pages 148–170, 1961.

Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model

for learning in graph domains. In Proceedings. 2005 IEEE International
Joint Conference on Neural Networks, 2005., volume 2, pages 729–734.
IEEE, 2005.

Amit Goyal, Francesco Bonchi, and Laks VS Lakshmanan. Learning

influence probabilities in social networks. In Proceedings of the third
ACM international conference on Web search and data mining, pages
241–250. ACM, 2010.

Palash Goyal and Emilio Ferrara. Graph embedding techniques,

applications, and performance: A survey. Knowledge-Based Systems,
151:78–94, 2018.

Daniel Grady, Christian Thiemann, and Dirk Brockmann. Robust clas-

sification of salient links in complex networks. Nature communications,
3:864, 2012.

Ronald L Graham and Pavol Hell. On the history of the minimum

spanning tree problem. Annals of the History of Computing, 7(1):43–57,
1985.

Mark Granovetter. Threshold models of collective behavior. American

journal of sociology, 83(6):1420–1443, 1978.

Mark Granovetter. The strength of weak ties: A network theory

revisited. 1983.

Mark Granovetter and Roland Soong. Threshold models of diffusion

and collective behavior. Journal of Mathematical sociology, 9(3):165–179,
1983.
 bibliography 703

Mark Granovetter and Roland Soong. Threshold models of interper-

sonal effects in consumer demand. Journal of Economic Behavior &
Organization, 7(1):83–99, 1986.

Mark Granovetter and Roland Soong. Threshold models of diver-

sity: Chinese restaurants, residential segregation, and the spiral of
silence. Sociological methodology, pages 69–104, 1988.

Mark S Granovetter. The strength of weak ties. In Social networks,

pages 347–367. Elsevier, 1977.

Steve Gregory. An algorithm to find overlapping community

structure in networks. In European Conference on Principles of Data
Mining and Knowledge Discovery, pages 91–102. Springer, 2007.

Steve Gregory. Finding overlapping communities using disjoint

community detection algorithms. In Complex networks, pages 47–61.
Springer, 2009.

Steve Gregory. Finding overlapping communities in networks by

label propagation. New Journal of Physics, 12(10):103018, 2010.

Jacopo Grilli, György Barabás, Matthew J Michalska-Smith, and

Stefano Allesina. Higher-order interactions stabilize dynamics in
competitive network models. Nature, 548(7666):210, 2017.

Jonathan L Gross and Jay Yellen. Graph theory and its applications.
CRC press, 2005.

Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning

for networks. In Proceedings of the 22nd ACM SIGKDD international
conference on Knowledge discovery and data mining, pages 855–864.
ACM, 2016.

Jean-Loup Guillaume and Matthieu Latapy. Bipartite structure of all

complex networks. Information processing letters, 90:Issue–5, 2004.

Jeremy Guillon, Mario Chavez, Federico Battiston, Yohan Attal,

Valentina La Corte, Michel Thiebaut de Schotten, Bruno Dubois,
Denis Schwartz, Olivier Colliot, and Fabrizio De Vico Fallani. Dis-
rupted core-periphery structure of multimodal brain networks in
alzheimer’s disease. Network Neuroscience, 3(2):635–652, 2019.

Paulo R Guimaraes Jr and Paulo Guimaraes. Improving the analyses

of nestedness for large sets of matrices. Environmental Modelling &
Software, 21(10):1512–1513, 2006.

Roger Guimera and Luis A Nunes Amaral. Functional cartography

of complex metabolic networks. nature, 433(7028):895, 2005.
704 the atlas for the aspiring network scientist

Roger Guimerà and Marta Sales-Pardo. Missing and spurious

interactions and the reconstruction of complex networks. Proceedings
of the National Academy of Sciences, 106(52):22073–22078, 2009.

Roger Guimera, Marta Sales-Pardo, and Luís A Nunes Amaral.

Modularity from fluctuations in random graphs and complex
networks. Physical Review E, 70(2):025101, 2004.

Roger Guimerà, Marta Sales-Pardo, and Luís A Nunes Amaral.

Module identification in bipartite and directed networks. Physical
Review E, 76(3):036102, 2007.

Natali Gulbahce and Sune Lehmann. The art of community detection.

BioEssays, 30(10):934–938, 2008.

Mangesh Gupte, Pravin Shankar, Jing Li, Shanmugauelayut

Muthukrishnan, and Liviu Iftode. Finding hierarchy in directed
online social networks. In Proceedings of the 20th international confer-
ence on World wide web, pages 557–566. ACM, 2011.

Thomas R Hagadone. Molecular substructure similarity searching:

efficient retrieval in two-dimensional structure databases. Journal of
chemical information and computer sciences, 32(5):515–521, 1992.

Aric Hagberg, Pieter Swart, and Daniel S Chult. Exploring network

structure, dynamics, and function using networkx. Technical report,
Los Alamos National Lab.(LANL), Los Alamos, NM (United States),
2008.

Patric Hagmann, Leila Cammoun, Xavier Gigandet, Reto Meuli,

Christopher J Honey, Van J Wedeen, and Olaf Sporns. Mapping the
structural core of human cerebral cortex. PLoS biology, 6(7):e159, 2008.

Arda Halu, Raúl J Mondragón, Pietro Panzarasa, and Ginestra

Bianconi. Multiplex pagerank. PloS one, 8(10):e78293, 2013.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive rep-

resentation learning on large graphs. In NIPS, pages 1024–1034,
2017.

David K Hammond, Pierre Vandergheynst, and Rémi Gribonval.

Wavelets on graphs via spectral graph theory. Applied and Computa-
tional Harmonic Analysis, 30(2):129–150, 2011.

Jiawei Han, Jian Pei, and Yiwen Yin. Mining frequent patterns
without candidate generation. In ACM sigmod record, volume 29,
pages 1–12. ACM, 2000.
 bibliography 705

Jiawei Han, Hong Cheng, Dong Xin, and Xifeng Yan. Frequent
pattern mining: current status and future directions. Data mining and
knowledge discovery, 15(1):55–86, 2007.

James A Hanley and Barbara J McNeil. The meaning and use of the
area under a receiver operating characteristic (roc) curve. Radiology,
143(1):29–36, 1982.

Steve Hanneke, Wenjie Fu, Eric P Xing, et al. Discrete temporal

models of social networks. Electronic Journal of Statistics, 4:585–605,
2010.

Robert A Hanneman and Mark Riddle. Introduction to social

network methods. 2005.

Derek L Hansen, Ben Shneiderman, and Marc A Smith. Analyzing

social media networks with NodeXL: Insights from a connected world.
Morgan Kaufmann, 2010.

Lars Kai Hansen and Peter Salamon. Neural network ensembles.

IEEE transactions on pattern analysis and machine intelligence, 12(10):
993–1001, 1990.

Obaida Hanteer, Roberto Interdonato, Matteo Magnani, Andrea

Tagarelli, and Luca Rossi. Community detection in multiplex
networks, 2019.

Daniel Damir Harabor, Alban Grastien, et al. Online graph pruning

for pathfinding on grid maps. In AAAI, pages 1114–1119, 2011.

Frank Harary. On the measurement of structural balance. Behavioral

Science, 4(4):316–323, 1959.

Frank Harary. Graphs and matrices. SIAM Review, 9(1):83–90, 1967.

Frank Harary, Robert Zane Norman, and Dorwin Cartwright. Struc-

tural models: An introduction to the theory of directed graphs. Wiley,
1965.

Frank Harary, Juhani Nieminen, et al. Convexity in graphs. Journal of

Differential Geometry, 16(2):185–190, 1981.

Steve Harenberg, Gonzalo Bello, L Gjeltema, Stephen Ranshous,

Jitendra Harlalka, Ramona Seay, Kanchana Padmanabhan, and
Nagiza Samatova. Community detection in large-scale networks:
a survey and empirical evaluation. Wiley Interdisciplinary Reviews:
Computational Statistics, 6(6):426–439, 2014.

Mark Harrower and Cynthia A Brewer. Colorbrewer. org: an online

tool for selecting colour schemes for maps. The Cartographic Journal,
40(1):27–37, 2003.
706 the atlas for the aspiring network scientist

Richard A Harshman et al. Foundations of the parafac procedure:

Models and conditions for an" explanatory" multimodal factor
analysis. 1970.

Ki-ichiro Hashimoto. Zeta functions of finite graphs and representa-

tions of p-adic groups. In Automorphic forms and geometry of arithmetic
varieties, pages 211–280. Elsevier, 1989.

Ricardo Hausmann, César A Hidalgo, Sebastián Bustos, Michele

Coscia, Alexander Simoes, and Muhammed A Yildirim. The atlas of
economic complexity: Mapping paths to prosperity. Mit Press, 2014.

Taher H Haveliwala. Topic-sensitive pagerank. In Proceedings of the

11th international conference on World Wide Web, pages 517–526. ACM,
2002.

Friedrich August Hayek. The use of knowledge in society. The

American economic review, 35(4):519–530, 1945.

Douglas D Heckathorn and Christopher J Cameron. Network

sampling: From snowball and multiplicity to respondent-driven
sampling. Annual review of sociology, 43:101–119, 2017.

David Heckerman, Chris Meek, and Daphne Koller. Probabilistic

entity-relationship models, prms, and plate models. Introduction to
statistical relational learning, pages 201–238, 2007.

Jeffrey Heer, Stuart K Card, and James A Landay. Prefuse: a toolkit

for interactive information visualization. In Proceedings of the SIGCHI
conference on Human factors in computing systems, pages 421–430.
ACM, 2005.

Fritz Heider. The psychology of interpersonal relations. Psychology Press,

2013.

Mark Heimann, Haoming Shen, Tara Safavi, and Danai Koutra.

Regal: Representation learning-based graph alignment. In Proceedings
of the 27th ACM International Conference on Information and Knowledge
Management, pages 117–126, 2018.

Keith Henderson, Brian Gallagher, Tina Eliassi-Rad, Hanghang

Tong, Sugato Basu, Leman Akoglu, Danai Koutra, Christos Falout-
sos, and Lei Li. Rolx: structural role extraction & mining in large
graphs. In Proceedings of the 18th ACM SIGKDD international conference
on Knowledge discovery and data mining, pages 1231–1239. ACM, 2012.

Herbert W Hethcote. Three basic epidemiological models. In Applied

mathematical ecology, pages 119–144. Springer, 1989.
 bibliography 707

César A Hidalgo and Ricardo Hausmann. The building blocks of

economic complexity. Proceedings of the national academy of sciences, 106
(26):10570–10575, 2009.

César A Hidalgo, Bailey Klinger, A-L Barabási, and Ricardo Haus-

mann. The product space conditions the development of nations.
Science, 317(5837):482–487, 2007.

Frank L Hitchcock. The expression of a tensor or a polyadic as a sum

of products. Journal of Mathematics and Physics, 6(1-4):164–189, 1927.

Frank L Hitchcock. The distribution of a product from several

sources to numerous localities. Studies in Applied Mathematics, 20(1-4):
224–230, 1941.

Manel Hmimida and Rushed Kanawati. Community detection in

multiplex networks: A seed-centric approach. NHM, 10(1):71–85,
2015.

Qirong Ho, Le Song, and Eric Xing. Evolving cluster mixed-

membership blockmodel for time-evolving networks. In Proceedings
of the Fourteenth International Conference on Artificial Intelligence and
Statistics, pages 342–350, 2011.

Nathan Oken Hodas and Kristina Lerman. How visibility and

divided attention constrain social contagion. In SocialCom, pages
249–257. IEEE, 2012.

Douglas R Hofstadter. Gödel, Escher, Bach. Harvester press Hassocks,

Sussex, 1979.

Daniel A Hojman and Adam Szeidl. Core and periphery in networks.

Journal of Economic Theory, 139(1):295–309, 2008.

Paul W Holland and Samuel Leinhardt. Transitivity in structural

models of small groups. Comparative group studies, 2(2):107–124, 1971.

Paul W Holland and Samuel Leinhardt. An exponential family of

probability distributions for directed graphs. Journal of the american
Statistical association, 76(373):33–50, 1981.

Paul W Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt.

Stochastic blockmodels: First steps. Social networks, 5(2):109–137,
1983.

Petter Holme. Core-periphery organization of complex networks.

Phys. Rev. E, 72:046111, Oct 2005. doi: 10.1103/PhysRevE.72.046111.

Petter Holme and Beom Jun Kim. Growing scale-free networks with
tunable clustering. Physical review E, 65(2):026107, 2002.
708 the atlas for the aspiring network scientist

Petter Holme and Jari Saramäki. Temporal networks. Physics reports,

519(3):97–125, 2012.

Petter Holme, Mikael Huss, and Sang Hoon Lee. Atmospheric

reaction systems as null-models to identify structural traces of
evolution in metabolism. PLoS One, 6(5):e19759, 2011.

Danny Holten. Hierarchical edge bundles: Visualization of adjacency

relations in hierarchical data. IEEE Transactions on visualization and
computer graphics, 12(5):741–748, 2006.

Danny Holten and Jarke J Van Wijk. Force-directed edge bundling

for graph visualization. In Computer graphics forum, volume 28, pages
983–990. Wiley Online Library, 2009.

Sungpack Hong, Nicole C Rodia, and Kunle Olukotun. On fast

parallel detection of strongly connected components (scc) in small-
world graphs. In Proceedings of the International Conference on High
Performance Computing, Networking, Storage and Analysis, pages 1–11,
2013.

John Hopcroft, Omar Khan, Brian Kulis, and Bart Selman. Tracking
evolving communities in large linked networks. Proceedings of the
National Academy of Sciences, 101(suppl 1):5249–5253, 2004.

Harold Hotelling. Stability in competition. The Economic Journal, 39

(153):41–57, 1929.

Darko Hric, Richard K Darst, and Santo Fortunato. Community

detection in networks: Structural communities versus ground truth.
Physical Review E, 90(6):062805, 2014.

Desislava Hristova, Mirco Musolesi, and Cecilia Mascolo. Keep your

friends close and your facebook friends closer: A multiplex network
approach to the analysis of offline and online social ties. In Eighth
International AAAI Conference on Weblogs and Social Media, 2014.

Desislava Hristova, Anastasios Noulas, Chloë Brown, Mirco Mu-

solesi, and Cecilia Mascolo. A multilayer approach to multiplexity
and link prediction in online geo-social networks. EPJ Data Science,
5(1):24, 2016.

Shenglong Hu and Liqun Qi. Algebraic connectivity of an even

uniform hypergraph. Journal of Combinatorial Optimization, 24(4):
564–579, 2012.

Jun Huan, Wei Wang, and Jan Prins. Efficient mining of frequent
subgraphs in the presence of isomorphism. In Third IEEE International
Conference on Data Mining, pages 549–552. IEEE, 2003.
 bibliography 709

Jianbin Huang, Heli Sun, Jiawei Han, Hongbo Deng, Yizhou Sun,
and Yaguang Liu. Shrink: a structural clustering algorithm for
detecting hierarchical communities in networks. In Proceedings of
the 19th ACM international conference on Information and knowledge
management, pages 219–228. ACM, 2010.

Jianbin Huang, Heli Sun, Yaguang Liu, Qinbao Song, and Tim
Weninger. Towards online multiresolution community detection in
large-scale networks. PloS one, 6(8):e23829, 2011a.

Xuqing Huang, Jianxi Gao, Sergey V Buldyrev, Shlomo Havlin, and

H Eugene Stanley. Robustness of interdependent networks under
targeted attack. Physical Review E, 83(6):065101, 2011b.

Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal

of classification, 2(1):193–218, 1985.

Thomas Henry Huxley. Evidence as to Man’s Place in Nature. London,

Williams and Norgate, 1863.

Iacopo Iacopini, Giovanni Petri, Alain Barrat, and Vito Latora.

Simplicial models of social contagion. Nature communications, 10(1):
2485, 2019.

Jacopo Iacovacci, Zhihao Wu, and Ginestra Bianconi. Mesoscopic

structures reveal the network between the layers of multiplex data
sets. Physical Review E, 92(4):042806, 2015.

Vijay Ingalalli, Dino Ienco, and Pascal Poncelet. Sumgra: Querying

multigraphs via efficient indexing. In International Conference on
Database and Expert Systems Applications, pages 387–401. Springer,
2016.

Roberto Interdonato, Andrea Tagarelli, Dino Ienco, Arnaud Sal-

laberry, and Pascal Poncelet. Local community detection in multilayer
networks. DMKD, 31(5):1444–1479, 2017.

Md Shahadat Iqbal, Charisma F Choudhury, Pu Wang, and Marta C

González. Development of origin–destination matrices using mobile
phone call data. Transportation Research Part C: Emerging Technologies,
40:63–74, 2014.

Hiroshi Ishikawa. Higher-order clique reduction in binary graph cut.

In 2009 IEEE Conference on Computer Vision and Pattern Recognition,
pages 2993–3000. IEEE, 2009.

Ashesh Jain, Amir R Zamir, Silvio Savarese, and Ashutosh Saxena.

Structural-rnn: Deep learning on spatio-temporal graphs. In Proceed-
ings of the IEEE Conference on Computer Vision and Pattern Recognition,
pages 5308–5317, 2016.
710 the atlas for the aspiring network scientist

Mahdi Jalili, Yasin Orouskhani, Milad Asgari, Nazanin Alipourfard,

and Matjaž Perc. Link prediction in multiplex online social networks.
Royal Society open science, 4(2):160863, 2017.

Svante Janson, Donald E Knuth, Tomasz Łuczak, and Boris Pittel.

The birth of the giant component. Random Structures & Algorithms, 4
(3):233–358, 1993.

Edwin T Jaynes. Probability theory: The logic of science. Cambridge

university press, 2003.

Glen Jeh and Jennifer Widom. Scaling personalized web search.

In Proceedings of the 12th international conference on World Wide Web,
pages 271–279. Acm, 2003.

Finn V Jensen et al. An introduction to Bayesian networks, volume 210.

UCL press London, 1996.

Tommy R Jensen and Bjarne Toft. Graph coloring problems, volume 39.
John Wiley & Sons, 2011.

Lucas GS Jeub, Prakash Balachandran, Mason A Porter, Peter J

Mucha, and Michael W Mahoney. Think locally, act locally: Detection
of small, medium-sized, and large communities in large networks.
Physical Review E, 91(1):012821, 2015.

Lucas GS Jeub, Michael W Mahoney, Peter J Mucha, and Mason A

Porter. A local perspective on community structure in multilayer
networks. Network Science, 5(2):144–163, 2017.

Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowl-
edge graph embedding via dynamic mapping matrix. In IJCNLP,
pages 687–696, 2015.

Yuexin Jiang, Daniel I Bolnick, and Mark Kirkpatrick. Assortative

mating in animals. The American Naturalist, 181(6):E125–E138, 2013.

Gudlaugur Jóhannesson, Gunnlaugur Björnsson, and Einar H

Gudmundsson. Afterglow light curves and broken power laws: a
statistical study. The Astrophysical Journal Letters, 640(1):L5, 2006.

Samuel Johnson and Nick S Jones. Looplessness in networks is linked

to trophic coherence. Proceedings of the National Academy of Sciences,
114(22):5618–5623, 2017.

Samuel Jonhson, Virginia Domínguez-García, and Miguel A Muñoz.

Factors determining nestedness in complex networks. PloS one, 8(9):
e74025, 2013.

Jakob Jonsson. Simplicial complexes of graphs, volume 3. Springer, 2008.

 bibliography 711

Kara Joyner and Grace Kao. School racial composition and ado-
lescent racial homophily. Social science quarterly, pages 810–825,
2000.

Inderjit S Jutla, Lucas GS Jeub, and Peter J Mucha. A generalized

louvain method for community detection implemented in matlab.
URL [Link] amath. unc. edu/GenLouvain, 2011.

Tomihisa Kamada and Satoru Kawai. A simple method for com-

puting general position in displaying three-dimensional objects.
Computer Vision, Graphics, and Image Processing, 41(1):43–56, 1988.

Jermain Kaminski, Michael Schober, Raymond Albaladejo, Olek-

sandr Zastupailo, and Cesar Hidalgo. Moviegalaxies-social networks
in movies. 2018.

U Kang, Hanghang Tong, and Jimeng Sun. Fast random walk graph
kernel. In Proceedings of the 2012 SIAM international conference on data
mining, pages 828–838. SIAM, 2012.

Ravi Kannan, Santosh Vempala, and Adrian Vetta. On clusterings:

Good, bad and spectral. Journal of the ACM (JACM), 51(3):497–515,
2004.

George Karakostas. Faster approximation schemes for fractional

multicommodity flow problems. ACM Transactions on Algorithms
(TALG), 4(1):13, 2008.

Brian Karrer and Mark EJ Newman. Stochastic blockmodels and

community structure in networks. Physical review E, 83(1):016107,
2011.

Brian Karrer, Elizaveta Levina, and Mark EJ Newman. Robustness

of community structure in networks. Physical review E, 77(4):046119,
2008.

Brian Karrer, Mark EJ Newman, and Lenka Zdeborová. Percolation

on sparse networks. Physical review letters, 113(20):208702, 2014.

Leo Katz. A new status index derived from sociometric analysis.

Psychometrika, 18(1):39–43, 1953.

Brian P Kelley, Roded Sharan, Richard M Karp, Taylor Sittler,

David E Root, Brent R Stockwell, and Trey Ideker. Conserved
pathways within bacteria and yeast as revealed by global protein
network alignment. Proceedings of the National Academy of Sciences, 100
(20):11394–11399, 2003.
712 the atlas for the aspiring network scientist

David Kempe, Jon Kleinberg, and Éva Tardos. Maximizing the

spread of influence through a social network. In Proceedings of the
ninth ACM SIGKDD international conference on Knowledge discovery
and data mining, pages 137–146. ACM, 2003.

David Kempe, Jon Kleinberg, and Éva Tardos. Influential nodes in

a diffusion model for social networks. In International Colloquium on
Automata, Languages, and Programming, pages 1127–1138. Springer,
2005.

Dror Y Kenett, Matjaž Perc, and Stefano Boccaletti. Networks of

networks–an introduction. Chaos, Solitons & Fractals, 80:1–6, 2015.

William Ogilvy Kermack and Anderson G McKendrick. A contri-

bution to the mathematical theory of epidemics. Proceedings of the
Royal Society of London. Series A, Containing Papers of a Mathematical
and Physical Character, 115(772):700–721, 1927.

Elias Khalil, Hanjun Dai, Yuyu Zhang, Bistra Dilkina, and Le Song.
Learning combinatorial optimization algorithms over graphs. In
Advances in Neural Information Processing Systems, pages 6348–6358,
2017.

Masoumeh Kheirkhahzadeh, Andrea Lancichinetti, and Martin

Rosvall. Efficient community detection of network flows for varying
markov times and bipartite networks. Physical Review E, 93(3):032309,
2016.

Jin Seop Kim, Kwang-Il Goh, Byungnam Kahng, and Doochul Kim.
Fractality and self-similarity in scale-free networks. New Journal of
Physics, 9(6):177, 2007.

Jungeun Kim and Jae-Gil Lee. Community detection in multi-layer

graphs: A survey. ACM SIGMOD Record, 44(3):37–48, 2015.

Jungeun Kim, Jae-Gil Lee, and Sungsu Lim. Differential flattening:

A novel framework for community detection in multi-layer graphs.
ACM Transactions on Intelligent Systems and Technology (TIST), 8(2):27,
2017.

Youngdo Kim, Seung-Woo Son, and Hawoong Jeong. Finding

communities in directed networks. Physical Review E, 81(1):016103,
2010.

Gary King. Unifying political methodology: The likelihood theory of

statistical inference. University of Michigan Press, 1998.

Thomas Kipf and Max Welling. Semi-supervised classification with

graph convolutional networks. In ICLR, 2017.
 bibliography 713

Thomas N Kipf and Max Welling. Variational graph auto-encoders.

arXiv preprint arXiv:1611.07308, 2016.

Maksim Kitsak, Ivan Voitalov, and Dmitri Krioukov. Link prediction

with hyperbolic geometry. Physical Review Research, 2(4):043113, 2020.

Mikko Kivelä and Mason A Porter. Isomorphisms in multilayer

networks. IEEE Transactions on Network Science and Engineering, 5(3):
198–211, 2017.

Mikko Kivelä, Alex Arenas, Marc Barthelemy, James P Gleeson,

Yamir Moreno, and Mason A Porter. Multilayer networks. Journal of
complex networks, 2(3):203–271, 2014.

Gunnar W Klau. A new graph-based method for pairwise global

network alignment. BMC bioinformatics, 10(1):S59, 2009.

Jon M Kleinberg. Authoritative sources in a hyperlinked environment.

Journal of the ACM (JACM), 46(5):604–632, 1999.

Jon M Kleinberg, Ravi Kumar, Prabhakar Raghavan, Sridhar Ra-

jagopalan, and Andrew S Tomkins. The web as a graph: mea-
surements, models, and methods. In International Computing and
Combinatorics Conference, pages 1–17. Springer, 1999.

Kaj-Kolja Kleineberg, Marián Boguná, M Ángeles Serrano, and

Fragkiskos Papadopoulos. Hidden geometric correlations in real
multiplex networks. Nature Physics, 12(11):1076, 2016.

Yuval Kluger, Ronen Basri, Joseph T Chang, and Mark Gerstein.

Spectral biclustering of microarray data: coclustering genes and
conditions. Genome research, 13(4):703–716, 2003.

Donald Ervin Knuth. The Stanford GraphBase: a platform for combinato-

rial computing. AcM Press New York, 1993.

Jihoon Ko, Yunbum Kook, and Kijung Shin. Incremental lossless

graph summarization. In Proceedings of the 26th ACM SIGKDD
International Conference on Knowledge Discovery & Data Mining, pages
317–327, 2020.

Ron Kohavi et al. A study of cross-validation and bootstrap for

accuracy estimation and model selection. In Ijcai, volume 14, pages
1137–1145. Montreal, Canada, 1995.

Sadamori Kojaku and Naoki Masuda. Core-periphery structure

requires something else in the network. New Journal of Physics, 20(4):
043012, 2018.
714 the atlas for the aspiring network scientist

Tamara Kolda and Brett Bader. The tophits model for higher-order
web link analysis. In Workshop on link analysis, counterterrorism and
security, volume 7, pages 26–29, 2006.

Tamara G Kolda and Brett W Bader. Tensor decompositions and

applications. SIAM review, 51(3):455–500, 2009.

Giorgos Kollias, Shahin Mohammadi, and Ananth Grama. Network

similarity decomposition (nsd): A fast and scalable approach to net-
work alignment. IEEE Transactions on Knowledge and Data Engineering,
24(12):2232–2243, 2011.

Andrey Kolmogorov. Sulla determinazione empirica di una lgge di

distribuzione. Inst. Ital. Attuari, Giorn., 4:83–91, 1933.

Xiangnan Kong, Jiawei Zhang, and Philip S Yu. Inferring anchor

links across multiple heterogeneous social networks. In Proceedings
of the 22nd ACM international conference on Information & Knowledge
Management, pages 179–188. ACM, 2013.

Stephen Kosack, Michele Coscia, Evann Smith, Kim Albrecht,

Albert-László Barabási, and Ricardo Hausmann. Functional struc-
tures of us state governments. Proceedings of the National Academy of
Sciences, 115(46):11748–11753, 2018.

Danai Koutra, Joshua T Vogelstein, and Christos Faloutsos. Deltacon:

A principled massive-graph similarity function. In Proceedings of the
2013 SIAM International Conference on Data Mining, pages 162–170.
SIAM, 2013.

Danai Koutra, U Kang, Jilles Vreeken, and Christos Faloutsos. Vog:

Summarizing and understanding large graphs. In Proceedings of the
2014 SIAM international conference on data mining, pages 91–99. SIAM,
2014.

Lauri Kovanen, Márton Karsai, Kimmo Kaski, János Kertész, and

Jari Saramäki. Temporal motifs in time-dependent networks. Journal
of Statistical Mechanics: Theory and Experiment, 2011(11):P11005, 2011.

David Krackhardt, N Nohria, and B Eccles. The strength of strong

ties. Networks in the knowledge economy, 82, 2003.

József Krausz. Démonstration nouvelle d’une théoreme de whitney

sur les réseaux. Mat. Fiz. Lapok, 50(1):75–85, 1943.

Dmitri Krioukov, Fragkiskos Papadopoulos, Maksim Kitsak, Amin

Vahdat, and Marián Boguná. Hyperbolic geometry of complex
networks. Physical Review E, 82(3):036106, 2010.
 bibliography 715

Balachander Krishnamurthy, Phillipa Gill, and Martin Arlitt. A few

chirps about twitter. In Proceedings of the first workshop on Online social
networks, pages 19–24. ACM, 2008.

Pavel N Krivitsky. Exponential-family random graph models for

valued networks. Electronic journal of statistics, 6:1100, 2012.

Pavel N Krivitsky and Mark S Handcock. A separable model for

dynamic networks. Journal of the Royal Statistical Society. Series B,
Statistical Methodology, 76(1):29, 2014.

Joseph B Kruskal. On the shortest spanning subtree of a graph

and the traveling salesman problem. Proceedings of the American
Mathematical society, 7(1):48–50, 1956.

Joseph B Kruskal. Three-way arrays: rank and uniqueness of trilinear

decompositions, with application to arithmetic complexity and
statistics. Linear algebra and its applications, 18(2):95–138, 1977.

Florent Krzakala, Cristopher Moore, Elchanan Mossel, Joe Neeman,

Allan Sly, Lenka Zdeborová, and Pan Zhang. Spectral redemption
in clustering sparse networks. Proceedings of the National Academy of
Sciences, 110(52):20935–20940, 2013.

Martin Krzywinski, Inanc Birol, Steven JM Jones, and Marco A

Marra. Hive plots – rational approach to visualizing networks.
Briefings in bioinformatics, 13(5):627–644, 2011.

Oleksii Kuchaiev and Nataša Pržulj. Integrative network alignment

reveals large regions of global network similarity in yeast and
human. Bioinformatics, 27(10):1390–1396, 2011.

Jussi M Kumpula, Mikko Kivelä, Kimmo Kaski, and Jari Saramäki.

Sequential algorithm for fast clique percolation. Physical Review E, 78
(2):026109, 2008.

Zhana Kuncheva and Giovanni Montana. Community detection

in multiplex networks using locally adaptive random walks. In
ASONAM, pages 1308–1315. ACM, 2015.

Jérôme Kunegis. Konect: the koblenz network collection. In

Proceedings of the 22nd International Conference on World Wide Web,
pages 1343–1350, 2013.

Michihiro Kuramochi and George Karypis. Finding frequent patterns

in a large sparse graph. Data mining and knowledge discovery, 11(3):
243–271, 2005.
716 the atlas for the aspiring network scientist

Renaud Lambiotte and Marcel Ausloos. Uncovering collective

listening habits and music genres in bipartite networks. Physical
Review E, 72(6):066107, 2005.

Renaud Lambiotte and Marcel Ausloos. Collaborative tagging as a

tripartite network. In International Conference on Computational Science,
pages 1114–1117. Springer, 2006.

Renaud Lambiotte, J-C Delvenne, and Mauricio Barahona. Laplacian

dynamics and multiscale modular structure in networks. arXiv
preprint arXiv:0812.1770, 2008.

Godfrey N Lance and William T Williams. Computer programs

for hierarchical polythetic classification (“similarity analyses”). The
Computer Journal, 9(1):60–64, 1966.

Andrea Lancichinetti and Santo Fortunato. Community detection

algorithms: a comparative analysis. Physical review E, 80(5):056117,
2009.

Andrea Lancichinetti and Santo Fortunato. Limits of modularity

maximization in community detection. Physical review E, 84(6):066122,
2011.

Andrea Lancichinetti and Santo Fortunato. Consensus clustering in

complex networks. Scientific reports, 2:336, 2012.

Andrea Lancichinetti, Santo Fortunato, and Filippo Radicchi. Bench-

mark graphs for testing community detection algorithms. Physical
review E, 78(4):046110, 2008.

Andrea Lancichinetti, Santo Fortunato, and János Kertész. Detecting

the overlapping and hierarchical community structure in complex
networks. New Journal of Physics, 11(3):033015, 2009.

Andrea Lancichinetti, Filippo Radicchi, José J Ramasco, and Santo

Fortunato. Finding statistically significant communities in networks.
PloS one, 6(4):e18961, 2011.

Daniel B Larremore, Aaron Clauset, and Abigail Z Jacobs. Efficiently

inferring community structure in bipartite networks. Physical Review
E, 90(1):012805, 2014.

Vito Latora and Massimo Marchiori. Vulnerability and protection of

infrastructure networks. Physical Review E, 71(1):015103, 2005.

Vito Latora, Vincenzo Nicosia, and Giovanni Russo. Complex networks:

principles, methods and applications. Cambridge University Press, 2017.
 bibliography 717

Eugene L Lawler, Jan Karel Lenstra, AHG Rinnooy Kan,

David Bernard Shmoys, et al. The traveling salesman problem: a
guided tour of combinatorial optimization, volume 3. Wiley New York,
1985.

Anna Lázár, Dániel Ábel, and Tamás Vicsek. Modularity measure of

networks with overlapping communities. EPL, 90(1):18001, 2010.

Emmanuel Lazega and Marijtje Van Duijn. Position in formal

structure, personal characteristics and choices of advisors in a law
firm: A logistic regression model for dyadic network data. Social
networks, 19(4):375–397, 1997.

David Lazer, Alex Sandy Pentland, Lada Adamic, Sinan Aral, Al-
bert Laszlo Barabasi, Devon Brewer, Nicholas Christakis, Noshir
Contractor, James Fowler, Myron Gutmann, et al. Life in the network:
the coming age of computational social science. Science (New York,
NY), 323(5915):721, 2009.

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning.

nature, 521(7553):436–444, 2015.

Ernest Bruce Lee and Lawrence Markus. Foundations of optimal

control theory. Technical report, Minnesota Univ Minneapolis Center
For Control Sciences, 1967.

Sang Hoon Lee, Pan-Jun Kim, and Hawoong Jeong. Statistical

properties of sampled networks. Physical Review E, 73(1):016102, 2006.

Sang Hoon Lee et al. Network nestedness as generalized core-

periphery structures. Physical Review E, 93(2):022306, 2016.

Kristen LeFevre and Evimaria Terzi. Grass: Graph structure summa-

rization. In Proceedings of the 2010 SIAM International Conference on
Data Mining, pages 454–465. SIAM, 2010.

Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris

Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed
Morsey, Patrick Van Kleef, Sören Auer, et al. Dbpedia–a large-scale,
multilingual knowledge base extracted from wikipedia. Semantic Web,
6(2):167–195, 2015.

Elizabeth A Leicht and Mark EJ Newman. Community structure in

directed networks. Physical review letters, 100(11):118703, 2008.

Elizabeth A Leicht, Petter Holme, and Mark EJ Newman. Vertex

similarity in networks. Physical Review E, 73(2):026120, 2006.
718 the atlas for the aspiring network scientist

Ronny Lempel and Shlomo Moran. Salsa: the stochastic approach

for link-structure analysis. ACM Transactions on Information Systems
(TOIS), 19(2):131–160, 2001.

Kristina Lerman and Rumi Ghosh. Information contagion: An

empirical study of the spread of news on digg and twitter social
networks. In ICWSM, 2010.

Kristina Lerman, Xiaoran Yan, and Xin-Zeng Wu. The" majority

illusion" in social networks. PloS one, 11(2):e0147617, 2016.

Jure Leskovec and Christos Faloutsos. Sampling from large graphs.

In Proceedings of the 12th ACM SIGKDD international conference on
Knowledge discovery and data mining, pages 631–636. ACM, 2006.

Jure Leskovec and Christos Faloutsos. Scalable modeling of real

graphs using kronecker multiplication. In Proceedings of the 24th
international conference on Machine learning, pages 497–504. ACM,
2007.

Jure Leskovec and Julian J Mcauley. Learning to discover social

circles in ego networks. In Advances in neural information processing
systems, pages 539–547, 2012.

Jure Leskovec and Rok Sosič. Snap: A general-purpose network

analysis and graph-mining library. ACM Transactions on Intelligent
Systems and Technology (TIST), 8(1):1–20, 2016.

Jure Leskovec, Jon Kleinberg, and Christos Faloutsos. Graphs

over time: densification laws, shrinking diameters and possible
explanations. In Proceedings of the eleventh ACM SIGKDD international
conference on Knowledge discovery in data mining, pages 177–187. ACM,
2005a.

Jure Leskovec, Lada A Adamic, and Bernardo A Huberman. The

dynamics of viral marketing. ACM Transactions on the Web (TWEB), 1
(1):5, 2007a.

Jure Leskovec, Jon Kleinberg, and Christos Faloutsos. Graph evolu-

tion: Densification and shrinking diameters. ACM Transactions on
Knowledge Discovery from Data (TKDD), 1(1):2, 2007b.

Jure Leskovec, Kevin J Lang, Anirban Dasgupta, and Michael W

Mahoney. Statistical properties of community structure in large social
and information networks. In Proceedings of the 17th international
conference on World Wide Web, pages 695–704. ACM, 2008.

Jure Leskovec, Kevin J Lang, Anirban Dasgupta, and Michael W Ma-

honey. Community structure in large networks: Natural cluster sizes
 bibliography 719

and the absence of large well-defined clusters. Internet Mathematics,

6(1):29–123, 2009.

Jure Leskovec, Daniel Huttenlocher, and Jon Kleinberg. Signed

networks in social media. In Proceedings of the SIGCHI conference on
human factors in computing systems, pages 1361–1370. ACM, 2010a.

Jure Leskovec, Kevin J Lang, and Michael Mahoney. Empirical

comparison of algorithms for network community detection. In
WWW, pages 631–640. ACM, 2010b.

Jurij Leskovec, Deepayan Chakrabarti, Jon Kleinberg, and Christos

Faloutsos. Realistic, mathematically tractable graph generation and
evolution, using kronecker multiplication. In European Conference
on Principles of Data Mining and Knowledge Discovery, pages 133–145.
Springer, 2005b.

Vladimir I Levenshtein. Binary codes capable of correcting deletions,

insertions, and reversals. In Soviet physics doklady, volume 10, pages
707–710, 1966.

Cheng-Te Li and Shou-De Lin. Egocentric information abstraction

for heterogeneous social networks. In 2009 International Conference on
Advances in Social Network Analysis and Mining, pages 255–260. IEEE,
2009.

Chun-Hsien Li, Chiung-Chiou Tsai, and Suh-Yuh Yang. Analysis of

epidemic spreading of an sirs model in complex heterogeneous net-
works. Communications in Nonlinear Science and Numerical Simulation,
19(4):1042–1054, 2014.

Geng Li, Murat Semerci, Bulent Yener, and Mohammed J Zaki. Graph
classification via topological and label attributes. In Proceedings of the
9th international workshop on mining and learning with graphs (MLG),
San Diego, USA, volume 2, 2011.

Jiaoyang Li, Pavel Surynek, Ariel Felner, Hang Ma, TK Satish Ku-
mar, and Sven Koenig. Multi-agent path finding for large agents. In
Proceedings of the AAAI Conference on Artificial Intelligence, volume 33,
pages 7627–7634, 2019.

Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, Yi Chang, and Huan
Liu. Attributed network embedding for learning in a dynamic
environment. In Proceedings of the 2017 ACM on Conference on
Information and Knowledge Management, pages 387–396, 2017a.

Menghui Li, Ying Fan, Jiawei Chen, Liang Gao, Zengru Di, and
Jinshan Wu. Weighted networks of scientific communication: the
720 the atlas for the aspiring network scientist

measurement and topological role of weight. Physica A: Statistical

Mechanics and its Applications, 350(2-4):643–656, 2005.

Michael Y Li and James S Muldowney. Global stability for the seir

model in epidemiology. Mathematical biosciences, 125(2):155–164, 1995.

Michael Y Li, John R Graef, Liancheng Wang, and János Karsai.

Global dynamics of a seir model with varying total population size.
Mathematical biosciences, 160(2):191–213, 1999.

Siming Li, Christopher M Armstrong, Nicolas Bertin, Hui Ge, Stuart

Milstein, Mike Boxem, Pierre-Olivier Vidalain, Jing-Dong J Han,
Alban Chesneau, Tong Hao, et al. A map of the interactome network
of the metazoan c. elegans. Science, 303(5657):540–543, 2004.

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolu-
tional recurrent neural network: Data-driven traffic forecasting. arXiv
preprint arXiv:1707.01926, 2017b.

Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter
Battaglia. Learning deep generative models of graphs. arXiv preprint
arXiv:1803.03324, 2018.

Lizi Liao, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua.

Attributed social network embedding. IEEE Transactions on Knowledge
and Data Engineering, 30(12):2257–2270, 2018.

David Liben-Nowell and Jon Kleinberg. The link-prediction problem

for social networks. Journal of the American society for information
science and technology, 58(7):1019–1031, 2007.

Edo Liberty. Simple and deterministic matrix sketching. In SIGKDD,

pages 581–588, 2013.

Ryan N Lichtenwalter, Jake T Lussier, and Nitesh V Chawla. New

perspectives and methods in link prediction. In Proceedings of the 16th
ACM SIGKDD international conference on Knowledge discovery and data
mining, pages 243–252, 2010.

Ryan Lichtnwalter and Nitesh V Chawla. Link prediction: fair and

effective evaluation. In 2012 IEEE/ACM International Conference on
Advances in Social Networks Analysis and Mining, pages 376–383. IEEE,
2012.

Nan Lin. Foundations of social research. McGraw-Hill Companies, 1976.

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu.
Learning entity and relation embeddings for knowledge graph
completion. In Twenty-ninth AAAI conference on artificial intelligence,
2015.
 bibliography 721

Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu,

Bing Xiang, Bowen Zhou, and Yoshua Bengio. A structured self-
attentive sentence embedding. arXiv preprint arXiv:1703.03130,
2017.

Kun Liu and Evimaria Terzi. Towards identity anonymization

on graphs. In Proceedings of the 2008 ACM SIGMOD international
conference on Management of data, pages 93–106, 2008.

Lu Liu, Yang Wang, Roberta Sinatra, C Lee Giles, Chaoming Song,

and Dashun Wang. Hot streaks in artistic, cultural, and scientific
careers. Nature, 559(7714):396, 2018a.

Minghua Liu, Hang Ma, Jiaoyang Li, and Sven Koenig. Task and
path planning for multi-agent pickup and delivery. In Int Conf
on Autonomous Agents and MultiAgent Systems, pages 1152–1160.
IFAAMAS, 2019.

Weiping Liu and Linyuan Lü. Link prediction based on local random
walk. EPL (Europhysics Letters), 89(5):58007, 2010.

Xin Liu and Tsuyoshi Murata. Community detection in large-scale

bipartite networks. Transactions of the Japanese Society for Artificial
Intelligence, 25(1):16–24, 2010.

Yang Liu and Jeffrey Heer. Somewhere over the rainbow: An

empirical assessment of quantitative colormaps. In Proceedings of the
2018 CHI Conference on Human Factors in Computing Systems, pages
1–12, 2018.

Yang-Yu Liu, Jean-Jacques Slotine, and Albert-László Barabási.

Controllability of complex networks. nature, 473(7346):167, 2011.

Yike Liu, Tara Safavi, Abhilash Dighe, and Danai Koutra. Graph
summarization methods and applications: A survey. ACM Computing
Surveys (CSUR), 51(3):1–34, 2018b.

Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor
Darrell. Rethinking the value of network pruning. arXiv preprint
arXiv:1810.05270, 2018c.

Stuart Lloyd. Least squares quantization in pcm. IEEE transactions on

information theory, 28(2):129–137, 1982.

Dale F Lott. Dominance relations and breeding rate in mature male

american bison. Zeitschrift für Tierpsychologie, 49(4):418–432, 1979.

László Lovász et al. Random walks on graphs: A survey. Combina-

torics, Paul erdos is eighty, 2(1):1–46, 1993.
722 the atlas for the aspiring network scientist

Ada Lovelace. Notes on menabrea’s “sketch of the analytical engine

invented by charles babbage”, 1842.

Can Lu, Jeffrey Xu Yu, Rong-Hua Li, and Hao Wei. Exploring
hierarchies in online social networks. IEEE Transactions on Knowledge
and Data Engineering, 28(8):2086–2100, 2016.

Linyuan Lü and Weiping Liu. Information filtering via preferential

diffusion. Physical Review E, 83(6):066119, 2011.

Linyuan Lü and Tao Zhou. Link prediction in complex networks:

A survey. Physica A: statistical mechanics and its applications, 390(6):
1150–1170, 2011.

Xuesong Lu and Stéphane Bressan. Sampling connected induced sub-

graphs uniformly at random. In International Conference on Scientific
and Statistical Database Management, pages 195–212. Springer, 2012.

Flaminia L Luccio. Intruder capture in sierpinski graphs. In FUN,

pages 249–261. Springer, 2007.

Feng Luo, James Z Wang, and Eric Promislow. Exploring local

community structures in large networks. Web Intelligence and Agent
Systems: An International Journal, 6(4):387–400, 2008.

Jianxi Luo and Christopher L Magee. Detecting evolving patterns of

self-organizing networks by flow hierarchy measurement. Complexity,
16(6):53–61, 2011.

Athen Ma and Raúl J Mondragón. Rich-cores in networks. PloS one,

10(3):e0119678, 2015.

Hang Ma, TK Satish Kumar, and Sven Koenig. Multi-agent path

finding with delay probabilities. In Thirty-First AAAI Conference on
Artificial Intelligence, 2017.

Hao Ma, Haixuan Yang, Michael R Lyu, and Irwin King. Min-
ing social networks using heat diffusion processes for marketing
candidates selection. In Proceedings of the 17th ACM conference on
Information and knowledge management, pages 233–242. ACM, 2008.

Jan Maas. Gradient flows of the entropy for finite markov chains.
Journal of Functional Analysis, 261(8):2250–2292, 2011.

Laurens van der Maaten and Geoffrey Hinton. Visualizing data using
t-sne. Journal of machine learning research, 9(Nov):2579–2605, 2008.

Robert H Mac Arthur and Edward Osbornecoaut Wilson. The theory

of island biogeography. Technical report, 1967.
 bibliography 723

James MacQueen et al. Some methods for classification and analysis

of multivariate observations. In Berkeley symp on math statistics and
probability, volume 1, pages 281–297. Oakland, CA, USA, 1967.

Matteo Magnani and Luca Rossi. The ml-model for multi-layer social
networks. In ASONAM, pages 5–12. IEEE, 2011.

Arun S Maiya and Tanya Y Berger-Wolf. Benefits of bias: Towards

better characterization of network sampling. In Proceedings of the 17th
ACM SIGKDD international conference on Knowledge discovery and data
mining, pages 105–113. ACM, 2011.

Erkki Mäkinen. How to draw a hypergraph. International Journal of

Computer Mathematics, 34(3-4):177–185, 1990.

Fragkiskos D Malliaros and Michalis Vazirgiannis. Clustering and

community detection in directed networks: A survey. Physics Reports,
533(4):95–142, 2013.

Riccardo Marcaccioli and Giacomo Livan. A pólya urn approach to

information filtering in complex networks. Nature Communications,
10(1):745, 2019.

Massimo Marchiori and Vito Latora. Harmony in the small-world.

Physica A: Statistical Mechanics and its Applications, 285(3-4):539–546,
2000.

Luca Marotta, Salvatore Micciche, Yoshi Fujiwara, Hiroshi Iyetomi,

Hideaki Aoyama, Mauro Gallegati, and Rosario N Mantegna. Bank-
firm credit network in japan: an analysis of a bipartite network. PloS
one, 10(5):e0123079, 2015.

Travis Martin, Xiao Zhang, and Mark EJ Newman. Localization and

centrality in networks. Physical review E, 90(5):052808, 2014.

Víctor Martínez, Fernando Berzal, and Juan-Carlos Cubero. A survey

of link prediction in complex networks. ACM Computing Surveys
(CSUR), 49(4):69, 2017.

Guido Previde Massara, Tiziana Di Matteo, and Tomaso Aste.

Network filtering for big data: Triangulated maximally filtered
graph. Journal of complex Networks, 5(2):161–178, 2016.

Naoki Masuda and Renaud Lambiotte. A Guidance to Temporal

Networks. World Scientific, 2016.

Prabhaker Mateti and Narsingh Deo. On algorithms for enumerating

all circuits of a graph. SIAM Journal on Computing, 5(1):90–99, 1976.
724 the atlas for the aspiring network scientist

Michael Mathioudakis, Francesco Bonchi, Carlos Castillo, Aristides

Gionis, and Antti Ukkonen. Sparsification of influence networks.
In Proceedings of the 17th ACM SIGKDD international conference on
Knowledge discovery and data mining, pages 529–537, 2011.

Ryuta Matsuno and Tsuyoshi Murata. Mell: effective embedding

method for multiplex networks. In Companion Proceedings of the The
Web Conference 2018, pages 1261–1268, 2018.

Christopher L McClendon, Lan Hua, Gabriela Barreiro, and

Matthew P Jacobson. Comparing conformational ensembles us-
ing the kullback–leibler divergence expansion. Journal of chemical
theory and computation, 8(6):2115–2126, 2012.

Elizabeth Aura McClintock. When does race matter? race, sex, and
dating at an elite university. Journal of Marriage and Family, 72(1):
45–72, 2010.

Aaron F McDaid, Derek Greene, and Neil Hurley. Normalized

mutual information to evaluate overlapping community finding
algorithms. arXiv preprint arXiv:1110.2515, 2011.

Fintan McGee, Mohammad Ghoniem, Guy Melançon, Benoît Ot-

jacques, and Bruno Pinaud. The state of the art in multilayer network
visualization. In Computer Graphics Forum, volume 38, pages 125–149.
Wiley Online Library, 2019.

Andrew McGregor and Daniel Stubbs. Sketching earth-mover

distance on graph metrics. In Approximation, Randomization, and
Combinatorial Optimization. Algorithms and Techniques, pages 274–286.
Springer, 2013.

Brendan D McKay et al. Practical graph isomorphism. Department of

Computer Science, Vanderbilt University Tennessee, USA, 1981.

Miller McPherson, Lynn Smith-Lovin, and James M Cook. Birds of a

feather: Homophily in social networks. Annual review of sociology, 27
(1):415–444, 2001.

Yasir Mehmood, Nicola Barbieri, Francesco Bonchi, and Antti

Ukkonen. Csi: Community-level social influence analysis. In Joint
European Conference on Machine Learning and Knowledge Discovery in
Databases, pages 48–63. Springer, 2013.

Marina Meilă. Comparing clusterings—an information based

distance. Journal of multivariate analysis, 98(5):873–895, 2007.

Isabel Meirelles. Design for information: an introduction to the histories,

theories, and best practices behind effective information visualizations.
Rockport publishers, 2013.
 bibliography 725

Victor Gabriel Lopez Mejia, Frank L Lewis, Yan Wan, Edgar N

Sanchez, and Lingling Fan. Solutions for multiagent pursuit-
evasion games on communication graphs: Finite-time capture and
asymptotic behaviors. IEEE Transactions on Automatic Control, 2019.

David Melamed. Community structures in bipartite networks: A

dual-projection approach. PloS one, 9(5):e97823, 2014.

Sergey Melnik, Hector Garcia-Molina, and Erhard Rahm. Similarity

flooding: A versatile graph matching algorithm and its application
to schema matching. In Proceedings 18th International Conference on
Data Engineering, pages 117–128. IEEE, 2002.

Filippo Menczer, Santo Fortunato, and Clayton A Davis. A First

Course in Network Science. Cambridge University Press, 2020.

Robert K Merton. The matthew effect in science: The reward and

communication systems of science are considered. Science, 159(3810):
56–63, 1968.

Carl D Meyer. Matrix analysis and applied linear algebra, volume 71.
Siam, 2000.

Giovanni Micale, Alfredo Pulvirenti, Alfredo Ferro, Rosalba Giugno,

and Dennis Shasha. Fast methods for finding significant motifs on
labelled multi-relational networks. Journal of Complex Networks, 2019.

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient
estimation of word representations in vector space. arXiv preprint
arXiv:1301.3781, 2013a.

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff
Dean. Distributed representations of words and phrases and their
compositionality. Advances in neural information processing systems, 26:
3111–3119, 2013b.

Tijana Milenković, Weng Leong Ng, Wayne Hayes, and Nataša

Pržulj. Optimal network alignment with graphlet degree vectors.
Cancer informatics, 9:CIN–S4744, 2010.

Stanley Milgram. The small world problem. Psychology today, 2(1):

60–67, 1967.

Joel C Miller. Percolation and epidemics in random clustered

networks. Physical Review E, 80(2):020901, 2009.

K Miller and Tina Eliassi-Rad. Continuous time group discovery in

dynamic graphs. Technical report, LLNL, 2010.
726 the atlas for the aspiring network scientist

Barbara J Mills, Jeffery J Clark, Matthew A Peeples, W Randall

Haas, John M Roberts, J Brett Hill, Deborah L Huntley, Lewis Borck,
Ronald L Breiger, Aaron Clauset, et al. Transformation of social
networks in the late pre-hispanic us southwest. Proceedings of the
National Academy of Sciences, 110(15):5785–5790, 2013.

Ron Milo, Shai Shen-Orr, Shalev Itzkovitz, Nadav Kashtan, Dmitri

Chklovskii, and Uri Alon. Network motifs: simple building blocks
of complex networks. Science, 298(5594):824–827, 2002.

Staša Milojević. Power law distributions in information science:

Making the case for logarithmic binning. Journal of the American
Society for Information Science and Technology, 61(12):2417–2425, 2010.

Byungjoon Min, Su Do Yi, Kyu-Min Lee, and K-I Goh. Network

robustness of multiplex networks with interlayer degree correlations.
Physical Review E, 89(4):042811, 2014.

Takayuki Mizuno, Hideki Takayasu, and Misako Takayasu. Correla-

tion networks among currencies. Physica A: Statistical Mechanics and
its Applications, 364:336–342, 2006.

Michael Molloy and Bruce Reed. A critical point for random graphs
with a given degree sequence. Random structures & algorithms, 6(2-3):
161–180, 1995.

Michael Molloy and Bruce Reed. The size of the giant component
of a random graph with a given degree sequence. Combinatorics,
probability and computing, 7(3):295–305, 1998.

Raul J Mondragon, Jacopo Iacovacci, and Ginestra Bianconi. Multi-

link communities of multiplex networks. PloS one, 13(3):e0193821,
2018.

Enys Mones. Hierarchy in directed random networks. Physical Review

E, 87(2):022817, 2013.

Enys Mones, Lilla Vicsek, and Tamás Vicsek. Hierarchy measure for
complex networks. PloS one, 7(3):e33799, 2012.

Gaspard Monge. Mémoire sur la théorie des déblais et des remblais.

Histoire de l’Académie Royale des Sciences de Paris, 1781.

Bjarke Mønsted, Piotr Sapieżyński, Emilio Ferrara, and Sune

Lehmann. Evidence of complex contagion of information in so-
cial media: An experiment using twitter bots. PloS one, 12(9), 2017.

Todd K Moon. The expectation-maximization algorithm. IEEE Signal

processing magazine, 13(6):47–60, 1996.
 bibliography 727

Edward F Moore. The shortest path through a maze. In Proc. Int.

Symp. Switching Theory, 1959, pages 285–292, 1959.

Jacob Levy Moreno, Helen Hall Jennings, and Ernest Stagg Whitin.
Group method and group psychotherapy. Number 5. Beacon House, 1932.

Matteo Morini, Patrick Flandrin, Eric Fleury, Tommaso Venturini,

and Pablo Jensen. Revealing evolutions in dynamical networks. arXiv
preprint arXiv:1707.02114, 2017.

Christopher Morris, Martin Ritzert, Matthias Fey, William L Hamil-

ton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler
and leman go neural: Higher-order graph neural networks. In
Proceedings of the AAAI Conference on Artificial Intelligence, volume 33,
pages 4602–4609, 2019.

Fred Morstatter, Jürgen Pfeffer, Huan Liu, and Kathleen M Carley. Is

the sample good enough? comparing data from twitter’s streaming
api with twitter’s firehose. In Seventh international AAAI conference on
weblogs and social media, 2013.

Fred Morstatter, Jürgen Pfeffer, and Huan Liu. When is it biased?:

assessing the representativeness of twitter’s streaming api. In
Proceedings of the 23rd international conference on world wide web, pages
555–556. ACM, 2014.

Peter J Mucha, Thomas Richardson, Kevin Macon, Mason A Porter,

and Jukka-Pekka Onnela. Community structure in time-dependent,
multiscale, and multiplex networks. science, 328(5980):876–878, 2010.

Brian D Muegge, Justin Kuczynski, Dan Knights, Jose C Clemente,

Antonio González, Luigi Fontana, Bernard Henrissat, Rob Knight,
and Jeffrey I Gordon. Diet drives convergence in gut microbiome
functions across mammalian phylogeny and within humans. Science,
332(6032):970–974, 2011.

Abubakr Muhammad and Magnus Egerstedt. Control using higher

order laplacians in network topologies. In Proc. of 17th International
Symposium on Mathematical Theory of Networks and Systems, pages
1024–1038. Citeseer, 2006.

Tamara Munzner. Visualization analysis and design. AK Peters/CRC

Press, 2014.

Mirco Musolesi and Cecilia Mascolo. A community based mobility

model for ad hoc network research. In Proceedings of the 2nd inter-
national workshop on Multi-hop ad hoc networks: from theory to reality,
pages 31–38, 2006.
728 the atlas for the aspiring network scientist

Richard Myers, RC Wison, and Edwin R Hancock. Bayesian graph

edit distance. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 22(6):628–635, 2000.

Marc Najork, Sreenivas Gollapudi, and Rina Panigrahy. Less is more:

sampling the neighborhood graph makes salsa better and faster. In
Proceedings of the Second ACM International Conference on Web Search
and Data Mining, pages 242–251, 2009.

Sunil K Narang, Akshay Gadde, Eduard Sanou, and Antonio Or-

tega. Localized iterative methods for interpolation in graph struc-
tured data. In Global Conference on Signal and Information Processing
(GlobalSIP), 2013 IEEE, pages 491–494. IEEE, 2013.

Anand Narasimhamurthy, Derek Greene, Neil Hurley, and Pádraig

Cunningham. Community finding in large social networks through
problem decomposition. In Proc. 19th Irish Conference on Artificial
Intelligence and Cognitive Science, AICS, volume 8, 2008.

Arvind Narayanan and Vitaly Shmatikov. De-anonymizing social

networks. In 2009 30th IEEE symposium on security and privacy, pages
173–187. IEEE, 2009.

Saket Navlakha, Rajeev Rastogi, and Nisheeth Shrivastava. Graph

summarization with bounded error. In Proceedings of the 2008 ACM
SIGMOD international conference on Management of data, pages 419–
432, 2008.

Maziar Nekovee, Yamir Moreno, Ginestra Bianconi, and Matteo

Marsili. Theory of rumour spreading in complex social networks.
Physica A: Statistical Mechanics and its Applications, 374(1):457–470,
2007.

Azadeh Nematzadeh, Emilio Ferrara, Alessandro Flammini, and

Yong-Yeol Ahn. Optimal network modularity for information
diffusion. Physical review letters, 113(8):088701, 2014.

Tamás Nepusz, Andrea Petróczi, László Négyessy, and Fülöp Bazsó.

Fuzzy communities and the concept of bridgeness in complex
networks. Physical Review E, 77(1):016107, 2008.

Erich Neuwirth and R Color Brewer. Colorbrewer palettes. R package

version, pages 1–1, 2014.

M Newman. Network reconstruction and error estimation with noisy

network data. arXiv preprint arXiv:1803.02427, 2018a.

Mark Newman. Networks. Oxford university press, 2018b.

 bibliography 729

Mark EJ Newman. Clustering and preferential attachment in growing

networks. Physical review E, 64(2):025102, 2001a.

Mark EJ Newman. Scientific collaboration networks. i. network

construction and fundamental results. Physical review E, 64(1):016131,
2001b.

Mark EJ Newman. Scientific collaboration networks. ii. shortest paths,

weighted networks, and centrality. Physical review E, 64(1):016132,
2001c.

Mark EJ Newman. Assortative mixing in networks. Physical review

letters, 89(20):208701, 2002.

Mark EJ Newman. Mixing patterns in networks. Physical Review E, 67

(2):026126, 2003a.

Mark EJ Newman. The structure and function of complex networks.

SIAM review, 45(2):167–256, 2003b.

Mark EJ Newman. Analysis of weighted networks. Physical review E,

70(5):056131, 2004a.

Mark EJ Newman. Detecting community structure in networks. The

European Physical Journal B, 38(2):321–330, 2004b.

Mark EJ Newman. Fast algorithm for detecting community structure

in networks. Physical review E, 69(6):066133, 2004c.

Mark EJ Newman. A measure of betweenness centrality based on

random walks. Social networks, 27(1):39–54, 2005a.

Mark EJ Newman. Power laws, pareto distributions and zipf’s law.

Contemporary physics, 46(5):323–351, 2005b.

Mark EJ Newman. Finding community structure in networks using

the eigenvectors of matrices. Physical review E, 74(3):036104, 2006a.

Mark EJ Newman. Modularity and community structure in networks.

Proceedings of the national academy of sciences, 103(23):8577–8582,
2006b.

Mark EJ Newman. Random graphs with clustering. Physical review

letters, 103(5):058701, 2009.

Mark EJ Newman. Equivalence between modularity optimization

and maximum likelihood methods for community detection. Physical
Review E, 94(5):052315, 2016a.

Mark EJ Newman. Mathematics of networks. The new Palgrave

dictionary of economics, pages 1–8, 2016b.
730 the atlas for the aspiring network scientist

Mark EJ Newman and Michelle Girvan. Finding and evaluating

community structure in networks. Physical review E, 69(2):026113,
2004.

Mark EJ Newman and Duncan J Watts. Renormalization group

analysis of the small-world network model. Physics Letters A, 263(4-6):
341–346, 1999.

Mark EJ Newman, Steven H Strogatz, and Duncan J Watts. Random

graphs with arbitrary degree distributions and their applications.
Physical review E, 64(2):026118, 2001.

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. A three-

way model for collective learning on multi-relational data. In Icml,
volume 11, pages 809–816, 2011.

Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy

Gabrilovich. A review of relational machine learning for knowledge
graphs. Proceedings of the IEEE, 104(1):11–33, 2015.

Vincenzo Nicosia, Giuseppe Mangioni, Vincenza Carchiolo, and

Michele Malgeri. Extending the definition of modularity to directed
graphs with overlapping communities. Journal of Statistical Mechanics:
Theory and Experiment, 2009(03):P03024, 2009.

Vincenzo Nicosia, John Tang, Cecilia Mascolo, Mirco Musolesi,

Giovanni Russo, and Vito Latora. Graph metrics for temporal
networks. In Temporal networks, pages 15–40. Springer, 2013.

Mathias Niepert, Mathias Ahmed, and Konstantin Kutzkov. Learning

convolutional neural networks for graphs. In ICML, pages 2014–2023,
2016.

Siegfried Nijssen and Joost N Kok. The gaston tool for frequent
subgraph mining. Electronic Notes in Theoretical Computer Science, 127
(1):77–87, 2005.

Shirin Nilizadeh, Apu Kapadia, and Yong-Yeol Ahn. Community-

enhanced de-anonymization of online social networks. In Proceedings
of the 2014 acm sigsac conference on computer and communications
security, pages 537–548, 2014.

Jae Dong Noh, Hyeong-Chai Jeong, Yong-Yeol Ahn, and Hawoong

Jeong. Growing network model for community with group structure.
Physical Review E, 71(3):036131, 2005.

Mary L Northway. A method for depicting social relationships

obtained by sociometric testing. Sociometry, pages 144–150, 1940.
 bibliography 731

Anastasios Noulas, Salvatore Scellato, Renaud Lambiotte, Massim-

iliano Pontil, and Cecilia Mascolo. A tale of many cities: universal
patterns in human urban mobility. PloS one, 7(5), 2012.

Esko Nuutila and Eljas Soisalon-Soininen. On finding the strongly

connected components in a directed graph. Inf. Process. Lett., 49(1):
9–14, 1994.

Abdel R Omran. The epidemiologic transition: a theory of the

epidemiology of population change. The Milbank Quarterly, 83(4):
731–757, 2005.

Jukka-Pekka Onnela, Jari Saramäki, János Kertész, and Kimmo

Kaski. Intensity and coherence of motifs in weighted complex
networks. Physical Review E, 71(6):065103, 2005.

Jukka-Pekka Onnela, Samuel Arbesman, Marta C González, Albert-

László Barabási, and Nicholas A Christakis. Geographic constraints
on social network groups. PLoS one, 6(4):e16939, 2011.

Tore Opsahl, Filip Agneessens, and John Skvoretz. Node centrality in

weighted networks: Generalizing degree and shortest paths. Social
networks, 32(3):245–251, 2010.

Günce Keziban Orman and Vincent Labatut. A comparison of

community detection algorithms on artificial networks. In DS, pages
242–256. Springer, 2009.

Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu.
Asymmetric transitivity preserving graph embedding. In SIGKDD,
pages 1105–1114, 2016.

John F Padgett and Christopher K Ansell. Robust action and the rise
of the medici, 1400-1434. American journal of sociology, 98(6):1259–1319,
1993.

Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd.

The pagerank citation ranking: Bringing order to the web. Technical
report, Stanford InfoLab, 1999.

Gergely Palla, Albert-László Barabási, and Tamás Vicsek. Quantifying

social group evolution. Nature, 446(7136):664, 2007.

Raj Kumar Pan and Jari Saramäki. Path lengths, correlations, and
centrality in temporal networks. Physical Review E, 84(1):016105, 2011.

Shirui Pan, Jia Wu, Xingquan Zhu, Chengqi Zhang, and Yang Wang.
Tri-party deep network representation. Network, 11(9):12, 2016.
732 the atlas for the aspiring network scientist

Panagiotis Papadimitriou, Ali Dasdan, and Hector Garcia-Molina.

Web graph similarity for anomaly detection. Journal of Internet Services
and Applications, 1(1):19–30, 2010.

Spiros Papadimitriou, Jimeng Sun, Christos Faloutsos, and S Yu

Philip. Hierarchical, parameter-free community discovery. In Joint
European Conference on Machine Learning and Knowledge Discovery in
Databases, pages 170–187. Springer, 2008.

Fragkiskos Papadopoulos, Maksim Kitsak, M Ángeles Serrano,

Marián Boguná, and Dmitri Krioukov. Popularity versus similarity
in growing networks. Nature, 489(7417):537–540, 2012.

Symeon Papadopoulos, Andre Skusa, Athena Vakali, Yiannis Kom-

patsiaris, and Nadine Wagner. Bridge bounding: A local approach for
efficient community discovery in complex networks. arXiv preprint
arXiv:0902.0871, 2009.

Manos Papagelis, Gautam Das, and Nick Koudas. Sampling online

social networks. IEEE Transactions on knowledge and data engineering,
25(3):662–676, 2013.

Luca Pappalardo, Giulio Rossetti, and Dino Pedreschi. " how well
do we know each other?" detecting tie strength in multidimensional
social networks. In 2012 IEEE/ACM International Conference on
Advances in Social Networks Analysis and Mining, pages 1040–1045.
IEEE, 2012.

Luca Pappalardo, Filippo Simini, Salvatore Rinzivillo, Dino Pe-

dreschi, Fosca Giannotti, and Albert-László Barabási. Returners and
explorers dichotomy in human mobility. Nature communications, 6:
8166, 2015.

Ashwin Paranjape, Austin R Benson, and Jure Leskovec. Motifs in

temporal networks. In Proceedings of the Tenth ACM International
Conference on Web Search and Data Mining, pages 601–610. ACM,
2017.

Vilfredo Pareto. Manuale di economia politica con una introduzione alla

scienza sociale, volume 13. Società editrice libraria, 1919.

Namyong Park, Andrey Kan, Xin Luna Dong, Tong Zhao, and
Christos Faloutsos. Estimating node importance in knowledge graphs
using graph neural networks. In Proceedings of the 25th ACM SIGKDD
International Conference on Knowledge Discovery & Data Mining, pages
596–606, 2019.
 bibliography 733

Roni Parshani, Celine Rozenblat, Daniele Ietri, Cesar Ducruet, and

Shlomo Havlin. Inter-similarity between coupled networks. EPL
(Europhysics Letters), 92(6):68002, 2011.

Torrence D Parsons. Pursuit-evasion in a graph. In Theory and

applications of graphs, pages 426–441. Springer, 1978.

Srinivasan Parthasarathy, Yiye Ruan, and Venu Satuluri. Community

discovery in social networks: Applications, methods and emerging
trends. In Social network data analytics, pages 79–113. Springer, 2011.

Emanuel Parzen. On estimation of a probability density function and

mode. The annals of mathematical statistics, 33(3):1065–1076, 1962.

Romualdo Pastor-Satorras and Alessandro Vespignani. Epidemic

dynamics and endemic states in complex networks. Physical Review
E, 63(6):066117, 2001a.

Romualdo Pastor-Satorras and Alessandro Vespignani. Epidemic

spreading in scale-free networks. Physical review letters, 86(14):3200,
2001b.

Romualdo Pastor-Satorras, Alexei Vázquez, and Alessandro Vespig-

nani. Dynamical and correlation properties of the internet. Physical
review letters, 87(25):258701, 2001.

Romualdo Pastor-Satorras, Claudio Castellano, Piet Van Mieghem,

and Alessandro Vespignani. Epidemic processes in complex net-
works. Reviews of modern physics, 87(3):925, 2015.

Rob Patro and Carl Kingsford. Global network alignment using

multiscale spectral signatures. Bioinformatics, 28(23):3105–3114, 2012.

Bruce D Patterson. The principle of nested subsets and its implica-

tions for biological conservation. Conservation Biology, 1(4):323–334,
1987.

Mateusz Pawlik and Nikolaus Augsten. Rted: a robust algorithm for

the tree edit distance. arXiv preprint arXiv:1201.0230, 2011.

Claudia Payrató-Borras, Laura Hernández, and Yamir Moreno.

Breaking the spell of nestedness: The entropic origin of nestedness
in mutualistic systems. Physical Review X, 9(3):031024, 2019.

Judea Pearl and Dana Mackenzie. The book of why: the new science of
cause and effect. Basic Books, 2018.

Leto Peel and Aaron Clauset. Detecting change points in the

large-scale structure of evolving networks. In Twenty-Ninth AAAI
Conference on Artificial Intelligence, 2015.
734 the atlas for the aspiring network scientist

Leto Peel, Daniel B Larremore, and Aaron Clauset. The ground

truth about metadata and community detection in networks. Science
advances, 3(5):e1602548, 2017.

Jian Pei, Jiawei Han, Runying Mao, et al. Closet: An efficient

algorithm for mining frequent closed itemsets. In ACM SIGMOD
workshop on research issues in data mining and knowledge discovery,
volume 4, pages 21–30, 2000.

Tiago P Peixoto. Efficient monte carlo and greedy heuristic for the
inference of stochastic block models. Physical Review E, 89(1):012804,
2014a.

Tiago P Peixoto. The graph-tool python library. figshare, 2014b.

Tiago P Peixoto. Hierarchical block structures and high-resolution

model selection in large networks. Physical Review X, 4(1):011047,
2014c.

Tiago P Peixoto. Inferring the mesoscale structure of layered, edge-

valued, and time-varying networks. Physical Review E, 92(4):042807,
2015.

Tiago P Peixoto. Reconstructing networks with unknown and

heterogeneous errors. Physical Review X, 8(4):041011, 2018.

Ofir Pele and Michael Werman. A linear time histogram metric for
improved sift matching. In European conference on computer vision,
pages 495–508. Springer, 2008.

Ofir Pele and Michael Werman. Fast and robust earth mover’s
distances. In Computer vision, 2009 IEEE 12th international conference
on, pages 460–467. IEEE, 2009.

Shmuel Peleg, Michael Werman, and Hillel Rom. A unified approach

to the change of resolution: Space and gray-level. IEEE Transactions
on Pattern Analysis and Machine Intelligence, 11(7):739–742, 1989.

Diego Pennacchioli, Giulio Rossetti, Luca Pappalardo, Dino Pe-

dreschi, Fosca Giannotti, and Michele Coscia. The three dimensions
of social prominence. In International Conference on Social Informatics,
pages 319–332. Springer, 2013.

Diego Pennacchioli, Michele Coscia, Salvatore Rinzivillo, Fosca

Giannotti, and Dino Pedreschi. The retail market as a complex
system. EPJ Data Science, 3(1):33, 2014.

Mathew Penrose et al. Random geometric graphs, volume 5. Oxford

university press, 2003.
 bibliography 735

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online

learning of social representations. In Proceedings of the 20th ACM
SIGKDD international conference on Knowledge discovery and data
mining, pages 701–710. ACM, 2014.

Bryan Perozzi, Vivek Kulkarni, and Steven Skiena. Walklets: Mul-

tiscale graph embeddings for interpretable network classification.
arXiv preprint arXiv:1605.02115, 2016.

Thomas Piketty. Capital in the 21st century. 2014.

Ali Pinar, C Seshadhri, and Vaidyanathan Vishal. Escape: Efficiently

counting all 5-vertex subgraphs. In Proceedings of the 26th International
Conference on World Wide Web, pages 1431–1440. International World
Wide Web Conferences Steering Committee, 2017.

Clara Pizzuti. Ga-net: A genetic algorithm for community detection

in social networks. In International conference on parallel problem solving
from nature, pages 1081–1090. Springer, 2008.

Clara Pizzuti. A multiobjective genetic algorithm to find communities

in complex networks. IEEE Transactions on Evolutionary Computation,
16(3):418–430, 2012.

Henri Poincaré. The foundations of science. 1913.

Pascal Pons and Matthieu Latapy. Computing communities in large

networks using random walks. J. Graph Algorithms Appl., 10(2):
191–218, 2006.

Mason A Porter, Jukka-Pekka Onnela, and Peter J Mucha. Commu-

nities in networks. Notices of the AMS, 56(9):1082–1097, 2009.

Márton Pósfai, Yang-Yu Liu, Jean-Jacques Slotine, and Albert-László

Barabási. Effect of correlations on network controllability. Scientific
reports, 3:1067, 2013.

Eleanor A Power. Discerning devotion: Testing the signaling theory

of religion. Evolution and Human Behavior, 38(1):82–91, 2017.

Jonathan D Power, Alexander L Cohen, Steven M Nelson, Gagan S

Wig, Kelly Anne Barnes, Jessica A Church, Alecia C Vogel, Timo-
thy O Laumann, Fran M Miezin, Bradley L Schlaggar, et al. Func-
tional network organization of the human brain. Neuron, 72(4):
665–678, 2011.

David Martin Powers. Evaluation: from precision, recall and f-

measure to roc, informedness, markedness and correlation. 2011.
736 the atlas for the aspiring network scientist

William H Press, Saul A Teukolsky, William T Vetterling, and

Brian P Flannery. Numerical recipes 3rd edition: The art of scientific
computing. Cambridge university press, 2007.

Derek de Solla Price. A general theory of bibliometric and other

cumulative advantage processes. Journal of the American society for
Information science, 27(5):292–306, 1976.

Robert Clay Prim. Shortest connection networks and some general-

izations. Bell system technical journal, 36(6):1389–1401, 1957.

Manisha Pujari and Rushed Kanawati. Link prediction in multiplex

networks. NHM, 10(1):17–35, 2015.

John Punin and Mukkai Krishnamoorthy. Xgmml (extensible graph

markup and modeling language), 2001.

Manish Purohit, B Aditya Prakash, Chanhyun Kang, Yao Zhang,

and VS Subrahmanian. Fast influence-based coarsening for large
networks. In Proceedings of the 20th ACM SIGKDD international
conference on Knowledge discovery and data mining, pages 1296–1305,
2014.

Filippo Radicchi, Claudio Castellano, Federico Cecconi, Vittorio

Loreto, and Domenico Parisi. Defining and identifying communities
in networks. Proceedings of the National Academy of Sciences, 101(9):
2658–2663, 2004.

Filippo Radicchi, José J Ramasco, and Santo Fortunato. Information

filtering in complex weighted networks. Physical Review E, 83(4):
046101, 2011.

Usha Nandini Raghavan, Réka Albert, and Soundar Kumara. Near

linear time algorithm to detect community structures in large-scale
networks. Physical review E, 76(3):036106, 2007.

Kelly Raley, Megan Sweeney, and Danielle Wondra. The growing

racial and ethnic divide in us marriage patterns. Future of children, 25
(2):89, 2015.

Rajeev Raman. Recent results on the single-source shortest paths

problem. ACM SIGACT News, 28(2):81–87, 1997.

William M Rand. Objective criteria for the evaluation of clustering

methods. Journal of the American Statistical association, 66(336):846–850,
1971.

Amir H Rasti, Mojtaba Torkjazi, Reza Rejaie, D Stutzbach,

N Duffield, and W Willinger. Evaluating sampling techniques
for large dynamic graphs. Univ. Oregon, Tech. Rep. CIS-TR-08, 1, 2008.
 bibliography 737

Amir Hassan Rasti, Mojtaba Torkjazi, Reza Rejaie, Nick Duffield,

Walter Willinger, and Daniel Stutzbach. Respondent-driven sampling
for characterizing unstructured overlays. In IEEE INFOCOM 2009,
pages 2701–2705. IEEE, 2009.

Erzsébet Ravasz and Albert-László Barabási. Hierarchical organiza-

tion in complex networks. Physical review E, 67(2):026112, 2003.

Erzsébet Ravasz, Anna Lisa Somera, Dale A Mongru, Zoltán N

Oltvai, and A-L Barabási. Hierarchical organization of modularity in
metabolic networks. science, 297(5586):1551–1555, 2002.

John W Raymond, Eleanor J Gardiner, and Peter Willett. Rascal:

Calculation of graph similarity using maximum common edge
subgraphs. The Computer Journal, 45(6):631–644, 2002.

Jörg Reichardt and Stefan Bornholdt. Statistical mechanics of

community detection. Physical Review E, 74(1):016110, 2006.

Fergal Reid, Aaron McDaid, and Neil Hurley. Percolation computa-

tion in complex networks. In 2012 IEEE/ACM International Conference
on Advances in Social Networks Analysis and Mining, pages 274–281.
IEEE, 2012.

Davi de Castro Reis, Paulo Braz Golgher, Altigran Soares Silva, and
AlbertoF Laender. Automatic web news extraction using tree edit
distance. In Proceedings of the 13th international conference on World
Wide Web, pages 502–511, 2004.

Saulo DS Reis, Yanqing Hu, Andrés Babino, José S Andrade Jr,

Santiago Canals, Mariano Sigman, and Hernán A Makse. Avoiding
catastrophic failure in correlated networks of networks. Nature
Physics, 10(10):762, 2014.

CJ Rhodes and P Jones. Inferring missing links in partially observed

social networks. In OR, Defence and Security, pages 256–271. Springer,
2015.

Bruno Ribeiro and Don Towsley. Estimating and sampling graphs

with multidimensional random walks. In Proceedings of the 10th ACM
SIGCOMM conference on Internet measurement, pages 390–403, 2010.

Leonardo FR Ribeiro, Pedro HP Saverese, and Daniel R Figueiredo.

struc2vec: Learning node representations from structural identity. In
SIGKDD, pages 385–394, 2017.

Kaspar Riesen and Horst Bunke. Approximate graph edit distance

computation by means of bipartite graph matching. Image and Vision
computing, 27(7):950–959, 2009.
738 the atlas for the aspiring network scientist

Matteo Riondato, David García-Soriano, and Francesco Bonchi.

Graph summarization with quality guarantees. Data mining and
knowledge discovery, 31(2):314–349, 2017.

Elina Robeva and Anna Seigal. Singular vectors of orthogonally

decomposable tensors. Linear and Multilinear Algebra, 65(12):2457–
2471, 2017.

Garry Robins, Pip Pattison, Yuval Kalish, and Dean Lusher. An

introduction to exponential random graph (p*) models for social
networks. Social networks, 29(2):173–191, 2007.

Yannick Rochat. Closeness centrality extended to unconnected

graphs: The harmonic centrality index. Technical report, 2009.

M Puck Rombach, Mason A Porter, James H Fowler, and Peter J

Mucha. Core-periphery structure in networks. SIAM Journal on
Applied mathematics, 74(1):167–190, 2014.

Murray Rosenblatt. Remarks on some nonparametric estimates of a

density function. The Annals of Mathematical Statistics, pages 832–837,
1956.

Giulio Rossetti and Rémy Cazabet. Community discovery in dynamic

networks: a survey. ACM Computing Surveys (CSUR), 51(2):35, 2018.

Giulio Rossetti, Michele Berlingerio, and Fosca Giannotti. Scalable

link prediction on multidimensional networks. In 2011 IEEE 11th
International Conference on Data Mining Workshops, pages 979–986.
IEEE, 2011.

Giulio Rossetti, Luca Pappalardo, Dino Pedreschi, and Fosca Gian-

notti. Tiles: an online algorithm for community discovery in dynamic
social networks. Machine Learning, 106(8):1213–1241, 2017.

Giulio Rossetti, Letizia Milli, Salvatore Rinzivillo, Alina Sîrbu, Dino

Pedreschi, and Fosca Giannotti. Ndlib: a python library to model
and analyze diffusion processes over complex networks. International
Journal of Data Science and Analytics, 5(1):61–79, 2018.

Giulio Rossetti, Letizia Milli, and Rémy Cazabet. Cdlib: a python

library to extract, compare and evaluate communities from complex
networks. Applied Network Science, 4(1):52, 2019.

Martin Rosvall and Carl T Bergstrom. Maps of random walks on

complex networks reveal community structure. Proceedings of the
National Academy of Sciences, 105(4):1118–1123, 2008.
 bibliography 739

Martin Rosvall and Carl T Bergstrom. Multilevel compression of

random walks on networks reveals hierarchical organization in large
integrated systems. PloS one, 6(4):e18209, 2011.

Martin Rosvall, Alcides V Esquivel, Andrea Lancichinetti, Jevin D

West, and Renaud Lambiotte. Memory in network flows and its
effects on spreading dynamics and community detection. Nature
communications, 5:4630, 2014.

Sam T Roweis and Lawrence K Saul. Nonlinear dimensionality

reduction by locally linear embedding. Science, 290(5500):2323–2326,
2000.

A. Roxana Pamfil, Sam D. Howison, and Mason A. Porter. Edge

correlations in multilayer networks. arXiv preprint arXiv:1908.03875,
2019.

Bernard Roy. Transitivité et connexité. Comptes Rendus Hebdomadaires

Des Seances De L Academie Des Sciences, 249(2):216–218, 1959.

Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. The earth

mover’s distance as a metric for image retrieval. International journal
of computer vision, 40(2):99–121, 2000.

Neha Runwal, Richard M Low, and Mark Stamp. Opcode graph

similarity and metamorphic detection. Journal in computer virology, 8
(1-2):37–52, 2012.

Kerstin Sailer and Ian McCulloh. Social networks and spatial

configuration—how office layouts drive social interaction. Social
networks, 34(1):47–58, 2012.

Marta Sales-Pardo, Roger Guimera, André A Moreira, and Luís

A Nunes Amaral. Extracting the hierarchical organization of complex
systems. Proceedings of the National Academy of Sciences, 104(39):
15224–15229, 2007.

Matthew J Salganik and Douglas D Heckathorn. Sampling and es-

timation in hidden populations using respondent-driven sampling.
Sociological methodology, 34(1):193–240, 2004.

Vsevolod Salnikov, Daniele Cassese, and Renaud Lambiotte. Simpli-

cial complexes and complex systems. European Journal of Physics, 40
(1):014001, 2018.

Aliaksei Sandryhaila and Jose MF Moura. Discrete signal processing

on graphs: Frequency analysis. IEEE Trans. Signal Processing, 62(12):
3042–3054, 2014.
740 the atlas for the aspiring network scientist

Fabio Saracco, Mika J Straka, Riccardo Di Clemente, Andrea

Gabrielli, Guido Caldarelli, and Tiziano Squartini. Inferring monopar-
tite projections of bipartite networks: an entropy-based approach.
New Journal of Physics, 19(5):053022, 2017.

Jari Saramäki, Mikko Kivelä, Jukka-Pekka Onnela, Kimmo Kaski,

and Janos Kertesz. Generalizations of the clustering coefficient to
weighted complex networks. Physical Review E, 75(2):027105, 2007.

Vikram Saraph and Tijana Milenković. Magna: maximizing accuracy

in global network alignment. Bioinformatics, 30(20):2931–2940, 2014.

Venu Satuluri and Srinivasan Parthasarathy. Scalable graph clustering

using stochastic flows: applications to community discovery. In
SIGKDD conference, pages 737–746. ACM, 2009.

Venu Satuluri, Srinivasan Parthasarathy, and Yiye Ruan. Local

graph sparsification for scalable clustering. In Proceedings of the 2011
ACM SIGMOD International Conference on Management of data, pages
721–732, 2011.

JW Scannell, GAPC Burns, CC Hilgetag, MA O’Neil, and Malcolm P

Young. The connectional organization of the cortico-thalamic system
of the cat. Cerebral Cortex, 9(3):277–299, 1999.

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuch-

ner, and Gabriele Monfardini. The graph neural network model.
IEEE Transactions on Neural Networks, 20(1):61–80, 2008.

Salvatore Scellato, Anastasios Noulas, Renaud Lambiotte, and

Cecilia Mascolo. Socio-spatial properties of online location-based
social networks. In ICWSM, 2011.

Satu Elisa Schaeffer. Graph clustering. Computer science review, 1(1):

27–64, 2007.

Michael T Schaub, Jean-Charles Delvenne, Sophia N Yaliraki, and

Mauricio Barahona. Markov dynamics as a zooming lens for
multiscale community detection: non clique-like communities and
the field-of-view limit. PloS one, 7(2):e32210, 2012a.

Michael T Schaub, Renaud Lambiotte, and Mauricio Barahona.

Encoding dynamics for multiscale community detection: Markov
time sweeping for the map equation. Physical Review E, 86(2):026112,
2012b.

Michael T Schaub, Jean-Charles Delvenne, Renaud Lambiotte,

and Mauricio Barahona. Structured networks and coarse-grained
descriptions: A dynamical perspective. Advances in Network Clustering
and Blockmodeling, pages 333–361, 2019.
 bibliography 741

Martin W Schein and Milton H Fohrman. Social dominance relation-

ships in a herd of dairy cattle. The British Journal of Animal Behaviour,
3(2):45–55, 1955.

Thomas C Schelling. Hockey helmets, concealed weapons, and

daylight saving: A study of binary choices with externalities. Journal
of Conflict resolution, 17(3):381–428, 1973.

Maximilian Schich. Revealing matrices. 2010.

Maximilian Schich. Cultural analysis situs. 2019.

Maximilian Schich, Chaoming Song, Yong-Yeol Ahn, Alexander

Mirsky, Mauro Martino, Albert-László Barabási, and Dirk Helbing.
A network framework of cultural history. science, 345(6196):558–562,
2014.

Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van

Den Berg, Ivan Titov, and Max Welling. Modeling relational data
with graph convolutional networks. In European Semantic Web
Conference, pages 593–607. Springer, 2018.

Philipp Schuetz and Amedeo Caflisch. Efficient modularity optimiza-

tion by multistep greedy algorithm and vertex mover refinement.
Physical Review E, 77(4):046112, 2008.

Christoph Schulz, Arlind Nocaj, Jochen Goertler, Oliver Deussen,

Ulrik Brandes, and Daniel Weiskopf. Probabilistic graph layout for
uncertain network visualization. IEEE transactions on visualization and
computer graphics, 23(1):531–540, 2016.

John R Seeley. The net of reciprocal influence. a problem in treating

sociometric data. Canadian Journal of Experimental Psychology, 3:234,
1949.

Stephen B Seidman. Network structure and minimum degree. Social

networks, 5(3):269–287, 1983.

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian

Gallagher, and Tina Eliassi-Rad. Collective classification in network
data. AI magazine, 29(3):93–93, 2008.

M Angeles Serrano, Dmitri Krioukov, and Marián Boguná. Self-

similarity of complex networks and hidden metric spaces. Physical
review letters, 100(7):078701, 2008.

M Ángeles Serrano, Marián Boguná, and Alessandro Vespignani.

Extracting the multiscale backbone of complex weighted networks.
Proceedings of the national academy of sciences, 106(16):6483–6488, 2009.
742 the atlas for the aspiring network scientist

Neil Shah, Danai Koutra, Tianmin Zou, Brian Gallagher, and Chris-
tos Faloutsos. Timecrunch: Interpretable dynamic graph summariza-
tion. In Proceedings of the 21th ACM SIGKDD International Conference
on Knowledge Discovery and Data Mining, pages 1055–1064, 2015.

Haichuan Shang, Xuemin Lin, Ying Zhang, Jeffrey Xu Yu, and Wei
Wang. Connected substructure similarity search. In Proceedings of the
2010 ACM SIGMOD International Conference on Management of data,
pages 903–914, 2010.

Paul Shannon, Andrew Markiel, Owen Ozier, Nitin S Baliga,

Jonathan T Wang, Daniel Ramage, Nada Amin, Benno Schwikowski,
and Trey Ideker. Cytoscape: a software environment for integrated
models of biomolecular interaction networks. Genome research, 13(11):
2498–2504, 2003.

Chengcheng Shao, Giovanni Luca Ciampaglia, Onur Varol, Alessan-

dro Flammini, and Filippo Menczer. The spread of fake news by
social bots. arXiv, pages 96–104, 2017.

Jia Shao, Sergey V Buldyrev, Shlomo Havlin, and H Eugene Stanley.

Cascade of failures in coupled network systems with multiple
support-dependence relations. Phys. Rev. E, 83:036116, Mar 2011.
doi: 10.1103/PhysRevE.83.036116.

Hua-Wei Shen, Xue-Qi Cheng, and Jia-Feng Guo. Quantifying and

identifying the overlapping community structure in networks. Journal
of Statistical Mechanics: Theory and Experiment, 2009(07):P07042, 2009.

Zeqian Shen, Kwan-Liu Ma, and Tina Eliassi-Rad. Visual analysis

of large heterogeneous social networks by semantic and structural
abstraction. IEEE transactions on visualization and computer graphics, 12
(6):1427–1439, 2006.

Shai S Shen-Orr, Ron Milo, Shmoolik Mangan, and Uri Alon. Net-
work motifs in the transcriptional regulation network of escherichia
coli. Nature genetics, 31(1):64, 2002.

Jianbo Shi and Jitendra Malik. Normalized cuts and image segmen-
tation. Departmental Papers (CIS), page 107, 2000.

Yu Shi, Qi Zhu, Fang Guo, Chao Zhang, and Jiawei Han. Easing
embedding learning by comprehensive transcription of heteroge-
neous information networks. In Proceedings of the 24th ACM SIGKDD
International Conference on Knowledge Discovery & Data Mining, pages
2190–2199, 2018.
 bibliography 743

Ben Shneiderman. The eyes have it: A task by data type taxonomy
for information visualizations. In Proceedings 1996 IEEE symposium
on visual languages, pages 336–343. IEEE, 1996.

David Shuman, Sunil Narang, Pascal Frossard, Antonio Ortega, and

Pierre Vandergheynst. The emerging field of signal processing on
graphs: Extending high-dimensional data analysis to networks and
other irregular domains. IEEE Signal Processing Magazine, 3(30):83–98,
2013.

David I Shuman, Benjamin Ricaud, and Pierre Vandergheynst. Vertex-

frequency analysis on graphs. Applied and Computational Harmonic
Analysis, 40(2):260–291, 2016.

Gerard Sierksma and Han Hoogeveen. Seven criteria for integer

sequences being graphic. Journal of Graph theory, 15(2):223–231, 1991.

Arlei Silva, Wagner Meira Jr, and Mohammed J Zaki. Mining

attribute-structure correlated patterns in large attributed graphs.
Proceedings of the VLDB Endowment, 5(5):466–477, 2012.

Samuel Silva, Beatriz Sousa Santos, and Joaquim Madeira. Using

color in visualization: A survey. Computers & Graphics, 35(2):320–333,
2011.

Filippo Simini, Marta C González, Amos Maritan, and Albert-László

Barabási. A universal model for mobility and migration patterns.
Nature, 484(7392):96–100, 2012.

Herbert A Simon. Models of man; social and rational. 1957.

Roberta Sinatra, Dashun Wang, Pierre Deville, Chaoming Song, and

Albert-László Barabási. Quantifying the evolution of individual
scientific impact. Science, 354(6312):aaf5239, 2016.

Richard Sinkhorn. A relationship between arbitrary positive matrices

and doubly stochastic matrices. The annals of mathematical statistics,
35(2):876–879, 1964.

Richard Sinkhorn and Paul Knopp. Concerning nonnegative matrices

and doubly stochastic matrices. Pacific Journal of Mathematics, 21(2):
343–348, 1967.

Paul B Slater. A two-stage algorithm for extracting the multiscale

backbone of complex weighted networks. Proceedings of the National
Academy of Sciences, 106(26):E66–E66, 2009.

Nickolay Smirnov. Table for estimating the goodness of fit of

empirical distributions. The annals of mathematical statistics, 19(2):
279–281, 1948.
744 the atlas for the aspiring network scientist

Adam Smith. The Wealth of Nations. 1776.

Laura M Smith, Linhong Zhu, Kristina Lerman, and Allon G Percus.

Partitioning networks with node attributes by compressing informa-
tion flow. ACM Transactions on Knowledge Discovery from Data (TKDD),
11(2):15, 2016.

Jamie Snape, Jur Van Den Berg, Stephen J Guy, and Dinesh
Manocha. The hybrid reciprocal velocity obstacle. IEEE Transac-
tions on Robotics, 27(4):696–706, 2011.

Tom AB Snijders. Markov chain monte carlo estimation of expo-

nential random graph models. Journal of Social Structure, 3(2):1–40,
2002.

Tom AB Snijders, Philippa E Pattison, Garry L Robins, and Mark S

Handcock. New specifications for exponential random graph models.
Sociological methodology, 36(1):99–153, 2006.

Tom AB Snijders, Gerhard G Van de Bunt, and Christian EG

Steglich. Introduction to stochastic actor-based models for net-
work dynamics. Social networks, 32(1):44–60, 2010.

Justin Solomon, Raif Rustamov, Leonidas Guibas, and Adrian

Butscher. Continuous-flow graph transportation distances. arXiv
preprint arXiv:1603.06927, 2016.

Chaoming Song, Shlomo Havlin, and Hernan A Makse. Self-

similarity of complex networks. Nature, 433(7024):392–395, 2005.

Chaoming Song, Zehui Qu, Nicholas Blumm, and Albert-László

Barabási. Limits of predictability in human mobility. Science, 327
(5968):1018–1021, 2010.

Sébastien Sorlin and Christine Solnon. Reactive tabu search for

measuring graph similarity. In International Workshop on Graph-Based
Representations in Pattern Recognition, pages 172–182. Springer, 2005.

F Sorrentino, M Di Bernardo, G Huerta Cuellar, and S Boccaletti.

Synchronization in weighted scale-free networks with degree–
degree correlation. Physica D: nonlinear phenomena, 224(1-2):123–129,
2006.

Sucheta Soundarajan, Tina Eliassi-Rad, Brian Gallagher, and Ali

Pinar. Maxreach: Reducing network incompleteness through node
probes. In 2016 IEEE/ACM International Conference on Advances in
Social Networks Analysis and Mining (ASONAM), pages 152–157.
IEEE, 2016.
 bibliography 745

Sucheta Soundarajan, Tina Eliassi-Rad, Brian Gallagher, and Ali

Pinar. ε-wgx: Adaptive edge probing for enhancing incomplete
networks. In Proceedings of the 2017 ACM on Web Science Conference,
pages 161–170. ACM, 2017.

Daniel A Spielman and Shang-Hua Teng. Nearly-linear time al-

gorithms for graph partitioning, graph sparsification, and solving
linear systems. In Proceedings of the thirty-sixth annual ACM symposium
on Theory of computing, pages 81–90, 2004.

Olaf Sporns, Giulio Tononi, and Rolf Kötter. The human connectome:
a structural description of the human brain. PLoS computational
biology, 1(4), 2005.

Natalie Stanley, Saray Shai, Dane Taylor, and Peter J Mucha. Cluster-
ing network layers with the strata multilayer stochastic block model.
IEEE transactions on network science and engineering, 3(2):95–105, 2016.

Dietrich Stauffer and Ammon Aharony. Introduction to percolation

theory. Taylor & Francis, 2014.

Juliette Stehlé, François Charbonnier, Tristan Picard, Ciro Cattuto,

and Alain Barrat. Gender homophily from spatial behavior in a
primary school: a sociometric study. Social Networks, 35(4):604–613,
2013.

Stephen V Stehman. Selecting and interpreting measures of thematic

classification accuracy. Remote sensing of Environment, 62(1):77–89,
1997.

Hugo Steinhaus. Sur la division des corp materiels en parties. Bull.

Acad. Polon. Sci, 1(804):801, 1956.

Nicholas M Stiffler and Jason M O’Kane. Pursuit-evasion with

fixed beams. In 2016 IEEE International Conference on Robotics and
Automation (ICRA), pages 4251–4258. IEEE, 2016.

Gilbert Strang. Introduction to linear algebra. Wellesley-Cambridge

Press Wellesley, MA, 1993.

Alexander Strehl and Joydeep Ghosh. Cluster ensembles—a knowl-

edge reuse framework for combining multiple partitions. Journal of
machine learning research, 3(Dec):583–617, 2002.

Michael PH Stumpf and Mason A Porter. Critical truths about power

laws. Science, 335(6069):665–666, 2012.

Daniel Stutzbach, Reza Rejaie, Nick Duffield, Subhabrata Sen, and

Walter Willinger. Sampling techniques for large, dynamic graphs. In
746 the atlas for the aspiring network scientist

Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference

on Computer Communications, pages 1–6. IEEE, 2006.

Daniel Stutzbach, Reza Rejaie, Nick Duffield, Subhabrata Sen, and

Walter Willinger. On unbiased sampling for unstructured peer-to-
peer networks. IEEE/ACM Transactions on Networking (TON), 17(2):
377–390, 2009.

Lovro Šubelj. Convex skeletons of complex networks. Journal of The

Royal Society Interface, 15(145):20180422, 2018.

Lovro Šubelj and Marko Bajec. Robust network community detection

using balanced propagation. The European Physical Journal B, 81(3):
353–362, 2011.

Jiankai Sun, Deepak Ajwani, Patrick K Nicholson, Alessandra Sala,

and Srinivasan Parthasarathy. Breaking cycles in noisy hierarchies. In
Proceedings of the 2017 ACM on Web Science Conference, pages 151–160.
ACM, 2017.

Jimeng Sun, Christos Faloutsos, Spiros Papadimitriou, and Philip S

Yu. Graphscope: parameter-free mining of large time-evolving
graphs. In SIGKDD, pages 687–696. ACM, 2007a.

Jimeng Sun, Yinglian Xie, Hui Zhang, and Christos Faloutsos. Less
is more: Compact matrix decomposition for large sparse graphs. In
Proceedings of the 2007 SIAM International Conference on Data Mining,
pages 366–377. SIAM, 2007b.

Yizhou Sun and Jiawei Han. Mining heterogeneous information

networks: a structural analysis approach. Acm Sigkdd Explorations
Newsletter, 14(2):20–28, 2013.

Yizhou Sun, Jie Tang, Jiawei Han, Manish Gupta, and Bo Zhao. Com-
munity evolution detection in dynamic heterogeneous information
networks. In MLGraphs, pages 137–146. ACM, 2010.

Yizhou Sun, Rick Barber, Manish Gupta, Charu C Aggarwal, and

Jiawei Han. Co-author relationship prediction in heterogeneous
bibliographic networks. In 2011 International Conference on Advances
in Social Networks Analysis and Mining, pages 121–128. IEEE, 2011.

Yizhou Sun, Jiawei Han, Charu C Aggarwal, and Nitesh V Chawla.

When will it happen?: relationship prediction in heterogeneous
information networks. In Proceedings of the fifth ACM international
conference on Web search and data mining, pages 663–672. ACM, 2012.

Siva R Sundaresan, Ilya R Fischhoff, Jonathan Dushoff, and Daniel I

Rubenstein. Network metrics reveal differences in social organization
 bibliography 747

between two fission–fusion species, grevy’s zebra and onager.

Oecologia, 151(1):140–149, 2007.

Latanya Sweeney. k-anonymity: A model for protecting privacy.

International Journal of Uncertainty, Fuzziness and Knowledge-Based
Systems, 10(05):557–570, 2002.

Tracy M Sweet, Andrew C Thomas, and Brian W Junker. Hierarchical

mixed membership stochastic blockmodels for multiple networks
and experimental interventions. Handbook on mixed membership models
and their applications, pages 463–488, 2014.

Danielle Albers Szafir. Modeling color difference for visualization

design. IEEE transactions on visualization and computer graphics, 24(1):
392–401, 2017.

Gabor J Szekely and Maria L Rizzo. Hierarchical clustering via joint

between-within distances: Extending ward’s minimum variance
method. Journal of classification, 22(2):151–183, 2005.

Michael Szell, Renaud Lambiotte, and Stefan Thurner. Multirelational

organization of large-scale social networks in an online world.
Proceedings of the National Academy of Sciences, 107(31):13636–13641,
2010.

Andrea Tagarelli, Alessia Amelio, and Francesco Gullo. Ensemble-

based community detection in multilayer networks. Data Mining and
Knowledge Discovery, 31(5):1506–1543, 2017.

Nassim Nicholas Taleb. The black swan: The impact of the highly
improbable, volume 2. Random house, 2007.

Fei Tan, Yongxiang Xia, and Boyao Zhu. Link prediction in complex
networks: a mutual information perspective. PloS one, 9(9):e107056,
2014.

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and
Qiaozhu Mei. Line: Large-scale information network embedding. In
Proceedings of the 24th international conference on world wide web, pages
1067–1077. International World Wide Web Conferences Steering
Committee, 2015a.

Jian Tang, Jingzhou Liu, Ming Zhang, and Qiaozhu Mei. Visualizing
large-scale and high-dimensional data. In WWW, pages 287–297,
2016.

Jiliang Tang, Shiyu Chang, Charu Aggarwal, and Huan Liu. Negative
link prediction in social media. In Proceedings of the eighth ACM
international conference on web search and data mining, pages 87–96.
ACM, 2015b.
748 the atlas for the aspiring network scientist

Lei Tang, Huan Liu, Jianping Zhang, and Zohreh Nazeri. Community
evolution in dynamic multi-mode networks. In SIGKDD, pages
677–685. ACM, 2008.

Lei Tang, Xufei Wang, and Huan Liu. Community detection via
heterogeneous interaction analysis. Data mining and knowledge
discovery, 25(1):1–33, 2012.

Jun Tao, Jian Xu, Chaoli Wang, and Nitesh V Chawla. Honvis:
Visualizing and exploring higher-order networks. In 2017 IEEE Pacific
Visualization Symposium (PacificVis), pages 1–10. IEEE, 2017.

Robert Tarjan. Depth-first search and linear graph algorithms. SIAM

journal on computing, 1(2):146–160, 1972.

Marc Tarrés-Deulofeu, Antonia Godoy-Lorite, Roger Guimera, and

Marta Sales-Pardo. Tensorial and bipartite block models for link
prediction in layered networks and temporal networks. Physical
Review E, 99(3):032307, 2019.

Nikolaj Tatti. Hierarchies in directed networks. In 2015 IEEE

international conference on data mining, pages 991–996. IEEE, 2015.

Dane Taylor, Saray Shai, Natalie Stanley, and Peter J Mucha. En-
hanced detectability of community structure in multilayer networks
through layer aggregation. Physical review letters, 116(22):228301, 2016.

Mikkel Thorup. On ram priority queues. SIAM Journal on Computing,

30(1):86–109, 2000.

Yuanyuan Tian, Richard A Hankins, and Jignesh M Patel. Efficient

aggregation for graph summarization. In Proceedings of the 2008
ACM SIGMOD international conference on Management of data, pages
567–580, 2008.

Seth Tisue and Uri Wilensky. Netlogo: A simple environment for

modeling complexity. In International conference on complex systems,
volume 21, pages 16–21. Boston, MA, 2004.

Michele Tizzoni, Paolo Bajardi, Adeline Decuyper, Guillaume

Kon Kam King, Christian M Schneider, Vincent Blondel, Zbigniew
Smoreda, Marta C González, and Vittoria Colizza. On the use of
human mobility proxies for modeling epidemics. PLoS computational
biology, 10(7), 2014.

Hannu Toivonen, Fang Zhou, Aleksi Hartikainen, and Atte Hinkka.

Compression of weighted graphs. In Proceedings of the 17th ACM
SIGKDD international conference on Knowledge discovery and data
mining, pages 965–973, 2011.
 bibliography 749

Hanghang Tong, B Aditya Prakash, Charalampos Tsourakakis, Tina

Eliassi-Rad, Christos Faloutsos, and Duen Horng Chau. On the
vulnerability of large graphs. In 2010 IEEE International Conference on
Data Mining, pages 1091–1096. IEEE, 2010.

Jameson L Toole, Carlos Herrera-Yaqüe, Christian M Schneider, and

Marta C González. Coupling human mobility and social ties. Royal
Society Interface, 12(105):20141128, 2015.

Leo Torres, Pablo Suárez-Serrato, and Tina Eliassi-Rad. Non-

backtracking cycles: length spectrum theory and graph mining
applications. Applied Network Science, 4(1):41, 2019.

Vincent A Traag and Jeroen Bruggeman. Community detection in

networks with positive and negative links. Physical Review E, 80(3):
036115, 2009.

Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier,

and Guillaume Bouchard. Complex embeddings for simple link
prediction. International Conference on Machine Learning (ICML),
2016.

Huynh Thanh Trung, Nguyen Thanh Toan, Tong Van Vinh,

Hoang Thanh Dat, Duong Chi Thang, Nguyen Quoc Viet Hung,
and Abdul Sattar. A comparative study on network alignment
techniques. Expert Systems with Applications, 140:112883, 2020.

Sho Tsugawa and Hiroyuki Ohsaki. Benefits of bias in crawl-based

network sampling for identifying key node set. IEEE Access, 8:
75370–75380, 2020.

Ledyard R Tucker. Some mathematical notes on three-mode factor

analysis. Psychometrika, 31(3):279–311, 1966.

Edward Tufte and P Graves-Morris. The visual display of quantitative

information.; 1983, 2014.

Michele Tumminello, Tomaso Aste, Tiziana Di Matteo, and

Rosario N Mantegna. A tool for filtering information in com-
plex systems. Proceedings of the National Academy of Sciences, 102(30):
10421–10426, 2005.

Michele Tumminello, Salvatore Micciche, Fabrizio Lillo, Jyrki Piilo,

and Rosario N Mantegna. Statistically validated networks in bipartite
complex systems. PloS one, 6(3):e17994, 2011.

Peter Uetz, Loic Giot, Gerard Cagney, Traci A Mansfield, Richard S

Judson, James R Knight, Daniel Lockshon, Vaibhav Narayan,
Maithreyan Srinivasan, Pascale Pochart, et al. A comprehensive
750 the atlas for the aspiring network scientist

analysis of protein–protein interactions in saccharomyces cerevisiae.

Nature, 403(6770):623, 2000.

Eugenio Valdano, Luca Ferreri, Chiara Poletto, and Vittoria Colizza.

Analytical computation of the epidemic threshold on temporal
networks. Physical Review X, 5(2):021005, 2015.

Gerhard G Van de Bunt, Marijtje AJ Van Duijn, and Tom AB Sni-

jders. Friendship networks through time: An actor-oriented dynamic
statistical network model. Computational & Mathematical Organization
Theory, 5(2):167–192, 1999.

Marijtje AJ Van Duijn, Tom AB Snijders, and Bonne JH Zijlstra. p2: a

random effects model with covariates for directed graphs. Statistica
Neerlandica, 58(2):234–254, 2004.

Jørn Vatn. Finding minimal cut sets in a fault tree. Reliability

Engineering & System Safety, 36(1):59–62, 1992.

A Vazquez, R Dobrin, D Sergi, J-P Eckmann, ZN Oltvai, and A-L

Barabási. The topological relationship between the large-scale
attributes and local interaction patterns of complex networks. Pro-
ceedings of the National Academy of Sciences, 101(52):17940–17945,
2004.

Alexei Vázquez and Yamir Moreno. Resilience to damage of graphs

with degree correlations. Phys. Rev. E, 67:015101, Jan 2003. doi:
10.1103/PhysRevE.67.015101.

Petar Veličković, William Fedus, William L Hamilton, Pietro Liò,

Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. arXiv
preprint arXiv:1809.10341, 2018.

Tiphaine Viard, Matthieu Latapy, and Clémence Magnien. Comput-

ing maximal cliques in link streams. Theoretical Computer Science, 609:
245–252, 2016.

Nguyen Xuan Vinh, Julien Epps, and James Bailey. Information

theoretic measures for clusterings comparison: Variants, properties,
normalization and correction for chance. J. Mach. Learn. Res, 11(Oct):
2837–2854, 2010.

Pauli Virtanen, Ralf Gommers, Travis E Oliphant, Matt Haberland,

Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson,
Warren Weckesser, Jonathan Bright, et al. Scipy 1.0: fundamental
algorithms for scientific computing in python. Nature methods, pages
1–12, 2020.
 bibliography 751

SVN Vishwanathan, Karsten M Borgwardt, Nicol N Schraudolph,

et al. Fast computation of graph kernels. In NIPS, volume 19, pages
131–138, 2006.

Ivan Voitalov, Pim van der Hoorn, Remco van der Hofstad, and
Dmitri Krioukov. Scale-free networks well done. arXiv preprint
arXiv:1811.02071, 2018.

Vitaly Ivanovich Voloshin. Introduction to graph and hypergraph theory.

Nova Science Publishers Hauppauge, 2009.

Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics and

computing, 17(4):395–416, 2007.

John von Neumann. First draft of a report on the edvac. 1945.

Soroush Vosoughi, Deb Roy, and Sinan Aral. The spread of true and
false news online. Science, 359(6380):1146–1151, 2018.

Claudia Wagner, David Garcia, Mohsen Jadidi, and Markus

Strohmaier. It’s a man’s wikipedia? assessing gender inequality
in an online encyclopedia. In Ninth international AAAI conference on
web and social media, 2015.

Claudia Wagner, Eduardo Graells-Garrido, David Garcia, and Fil-

ippo Menczer. Women through the glass ceiling: gender asymmetries
in wikipedia. EPJ Data Science, 5(1):5, 2016.

Glenn Wagner and Howie Choset. Subdimensional expansion for

multirobot path planning. Artificial Intelligence, 219:1–24, 2015.

Thayne T Walker, David M Chan, and Nathan R Sturtevant. Using

hierarchical constraints to avoid conflicts in multi-agent pathfinding.
In Int Conf on Automated Planning and Scheduling, 2017.

Thayne T Walker, Nathan R Sturtevant, and Ariel Felner. Extended

increasing cost tree search for non-unit cost domains. In IJCAI, pages
534–540, 2018.

AR Wallace. On the tendency of varieties to depart indefinitely from

the original type. J. Linn. Soc. Lond. Zool., 3:53–62, 1858.

Bo Wang, Aziz M Mezlini, Feyyaz Demir, Marc Fiume, Zhuowen

Tu, Michael Brudno, Benjamin Haibe-Kains, and Anna Goldenberg.
Similarity network fusion for aggregating data types on a genomic
scale. Nature methods, 11(3):333, 2014a.

Daixin Wang, Peng Cui, and Wenwu Zhu. Structural deep network
embedding. In Proceedings of the 22nd ACM SIGKDD international
conference on Knowledge discovery and data mining, pages 1225–1234,
2016a.
752 the atlas for the aspiring network scientist

Dan J Wang, Xiaolin Shi, Daniel A McFarland, and Jure Leskovec.

Measurement error in network data: A re-classification. Social
Networks, 34(4):396–409, 2012a.

Dashun Wang, Dino Pedreschi, Chaoming Song, Fosca Giannotti,

and Albert-Laszlo Barabasi. Human mobility, social ties, and link
prediction. In SIGKDD, pages 1100–1108. Acm, 2011a.

Dashun Wang, Zhen Wen, Hanghang Tong, Ching-Yung Lin,

Chaoming Song, and Albert-László Barabási. Information spreading
in context. In Proceedings of the 20th international conference on World
wide web, pages 735–744. ACM, 2011b.

Dashun Wang, Chaoming Song, and Albert-László Barabási. Quanti-

fying long-term scientific impact. Science, 342(6154):127–132, 2013.

Liping Wang, Qing Li, Na Li, Guozhu Dong, and Yu Yang. Substruc-
ture similarity measurement in chinese recipes. In Proceedings of the
17th international conference on World Wide Web, pages 979–988, 2008a.

Peng Wang, BaoWen Xu, YuRong Wu, and XiaoYu Zhou. Link
prediction in social networks: the state-of-the-art. Science China
Information Sciences, 58(1):1–38, 2015a.

Pu Wang, Marta C González, César A Hidalgo, and Albert-László

Barabási. Understanding the spreading patterns of mobile phone
viruses. Science, 324(5930):1071–1076, 2009.

Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. Knowledge

graph embedding: A survey of approaches and applications. IEEE
Transactions on Knowledge and Data Engineering, 29(12):2724–2743,
2017a.

Quanxin Wang, Olaf Sporns, and Andreas Burkhalter. Network

analysis of corticocortical connections reveals ventral and dorsal
processing streams in mouse visual cortex. Journal of Neuroscience, 32
(13):4386–4399, 2012b.

Suhang Wang, Jiliang Tang, Charu Aggarwal, Yi Chang, and Huan

Liu. Signed network embedding in social media. In Proceedings of
the 2017 SIAM international conference on data mining, pages 327–335.
SIAM, 2017b.

Xiaohong Wang, Jun Huan, Aaron Smalter, and Gerald H Lush-

ington. G-hash: towards fast kernel-based similarity search in
large graph databases. In Graph Data Management: Techniques and
Applications, pages 176–213. IGI Global, 2012c.
 bibliography 753

Yang Wang, Deepayan Chakrabarti, Chenxi Wang, and Christos

Faloutsos. Epidemic spreading in real networks: An eigenvalue
viewpoint. In 22nd International Symposium on Reliable Distributed
Systems, 2003. Proceedings., pages 25–34. IEEE, 2003.

Yi Wang, Bin Wu, and Xin Pei. Commtracker: A core-based algorithm

of tracking community evolution. In ADMA, pages 229–240. Springer,
2008b.

Yu-Xiang Wang, James Sharpnack, Alex Smola, and Ryan J Tibshi-

rani. Trend filtering on graphs. Journal of Machine Learning Research,
17(105):1–41, 2016b.

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowl-
edge graph embedding by translating on hyperplanes. In Twenty-
Eighth AAAI conference on artificial intelligence, 2014b.

Zhen Wang, Lin Wang, Attila Szolnoki, and Matjaž Perc. Evolu-
tionary games on multilayer networks: a colloquium. The European
physical journal B, 88(5):124, 2015b.

Stephen Warshall. A theorem on boolean matrices. Journal of the ACM

(JACM), 9(1):11–12, 1962.

Stanley Wasserman and Katherine Faust. Social network analysis:

Methods and applications, volume 8. Cambridge university press, 1994.

Duncan J Watts. Networks, dynamics, and the small-world phe-

nomenon. American Journal of sociology, 105(2):493–527, 1999.

Duncan J Watts. A simple model of global cascades on random

networks. Proceedings of the National Academy of Sciences, 99(9):
5766–5771, 2002.

Duncan J Watts. Six degrees: The science of a connected age. WW Norton

& Company, 2004.

Duncan J Watts and Steven H Strogatz. Collective dynamics of

‘small-world’networks. nature, 393(6684):440, 1998.

Bernard M Waxman. Routing of multipoint connections. IEEE journal

on selected areas in communications, 6(9):1617–1622, 1988.

Samuel F Way, Allison C Morgan, Daniel B Larremore, and Aaron

Clauset. Productivity, prominence, and the effects of academic
environment. Proceedings of the National Academy of Sciences, 116(22):
10729–10733, 2019.
754 the atlas for the aspiring network scientist

Jan D Wegner, Javier A Montoya-Zegarra, and Konrad Schindler. A

higher-order crf model for road network extraction. In Proceedings of
the IEEE Conference on Computer Vision and Pattern Recognition, pages
1698–1705, 2013.

E Weinan, Tiejun Li, and Eric Vanden-Eijnden. Optimal partition and

effective dynamics of complex networks. Proceedings of the National
Academy of Sciences, 105(23):7907–7912, 2008.

Lilian Weng, Alessandro Flammini, Alessandro Vespignani, and

Fillipo Menczer. Competition among memes in a world with limited
attention. Scientific reports, 2:335, 2012.

Lilian Weng, Filippo Menczer, and Yong-Yeol Ahn. Virality prediction

and community structure in social networks. Scientific reports, 3:2522,
2013.

Lilian Weng, Filippo Menczer, and Yong-Yeol Ahn. Predicting

successful memes using network and community structure. In
ICWSM, 2014.

Douglas Brent West et al. Introduction to graph theory, volume 2.

Prentice hall Upper Saddle River, 2001.

Hassler Whitney. Congruent graphs and the connectivity of graphs.

American Journal of Mathematics, 54(1):150–168, 1932.

Roland Wiese, Markus Eiglsperger, and Michael Kaufmann.

yfiles—visualization and automatic layout of graphs. In Graph
Drawing Software, pages 173–191. Springer, 2004.

Michael Windzio. The network of global migration 1990–2013: Using

ergms to test theories of migration between countries. Social Networks,
53:20–29, 2018.

Ludwig Wittgenstein. Tractatus logico-philosophicus, 1921.

Ling Heng Wong, Philippa Pattison, and Garry Robins. A spatial

model for social networks. Physica A, 360(1):99–120, 2006.

Marc Wörlein, Thorsten Meinl, Ingrid Fischer, and Michael

Philippsen. A quantitative comparison of the subgraph miners
mofa, gspan, ffsm, and gaston. In European Conference on Principles of
Data Mining and Knowledge Discovery, pages 392–403. Springer, 2005.

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi

Zhang, and Philip S Yu. A comprehensive survey on graph neural
networks. arXiv preprint arXiv:1901.00596, 2019.
 bibliography 755

Jierui Xie, Boleslaw K Szymanski, and Xiaoming Liu. Slpa: Uncov-

ering overlapping communities in social networks via a speaker-
listener interaction dynamic process. In 2011 IEEE 11th International
Conference on Data Mining Workshops, pages 344–349. IEEE, 2011.

Jierui Xie, Stephen Kelley, and Boleslaw K Szymanski. Overlap-

ping community detection in networks: The state-of-the-art and
comparative study. Acm computing surveys (csur), 45(4):43, 2013.

Eric P Xing, Wenjie Fu, Le Song, et al. A state-space mixed member-

ship blockmodel for dynamic network tomography. The Annals of
Applied Statistics, 4(2):535–566, 2010.

Jian Xu, Thanuka L Wickramarathne, and Nitesh V Chawla. Repre-

senting higher-order dependencies in networks. Science advances, 2
(5):e1600028, 2016.

Kevin S Xu and Alfred O Hero. Dynamic stochastic blockmodels: Sta-

tistical models for time-evolving networks. In International conference
on social computing, behavioral-cultural modeling, and prediction, pages
201–210. Springer, 2013.

R. Xulvi-Brunet and I. M. Sokolov. Changing Correlations in

Networks: Assortativity and Dissortativity. Acta Physica Polonica B,
36:1431, 5 2005.

Ramon Xulvi-Brunet and Igor M Sokolov. Reshuffling scale-free

networks: From random to assortative. Physical Review E, 70(6):
066102, 2004.

Konstantin Yakovlev and Anton Andreychuk. Any-angle pathfinding

for multiple agents based on sipp algorithm. In Twenty-Seventh
International Conference on Automated Planning and Scheduling, 2017.

Bowen Yan and Steve Gregory. Detecting communities in networks

by merging cliques. In 2009 IEEE International Conference on Intelligent
Computing and Intelligent Systems, volume 1, pages 832–836. IEEE,
2009.

Gang Yan, Georgios Tsekenis, Baruch Barzel, Jean-Jacques Slotine,

Yang-Yu Liu, and Albert-László Barabási. Spectrum of controlling
and observing complex networks. Nature Physics, 11(9):779–786, 2015.

Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial temporal graph
convolutional networks for skeleton-based action recognition. In
Thirty-Second AAAI Conference on Artificial Intelligence, 2018.

Xifeng Yan and Jiawei Han. gspan: Graph-based substructure pattern

mining. In 2002 IEEE International Conference on Data Mining, 2002.
Proceedings., pages 721–724. IEEE, 2002.
756 the atlas for the aspiring network scientist

Xifeng Yan and Jiawei Han. Closegraph: mining closed frequent

graph patterns. In Proceedings of the ninth ACM SIGKDD international
conference on Knowledge discovery and data mining, pages 286–295.
ACM, 2003.

Xifeng Yan, Philip S Yu, and Jiawei Han. Substructure similarity

search in graph databases. In Proceedings of the 2005 ACM SIGMOD
international conference on Management of data, pages 766–777, 2005.

Jaewon Yang and Jure Leskovec. Community-affiliation graph model

for overlapping network community detection. In 2012 IEEE 12th
international conference on data mining, pages 1170–1175. IEEE, 2012.

Jaewon Yang and Jure Leskovec. Overlapping community detection

at scale: a nonnegative matrix factorization approach. In Proceedings
of the sixth ACM international conference on Web search and data mining,
pages 587–596. ACM, 2013.

Jaewon Yang and Jure Leskovec. Overlapping communities explain

core–periphery organization of networks. Proceedings of the IEEE, 102
(12):1892–1902, 2014.

Jaewon Yang and Jure Leskovec. Defining and evaluating network

communities based on ground-truth. Knowledge and Information
Systems, 42(1):181–213, 2015.

Yang Yang, Ryan N Lichtenwalter, and Nitesh V Chawla. Evaluating

link prediction methods. Knowledge and Information Systems, 45(3):
751–782, 2015.

Zhao Yang, René Algesheimer, and Claudio J Tessone. A comparative

analysis of community detection algorithms on artificial networks.
Scientific Reports, 6:30750, 2016a.

Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov. Revisiting

semi-supervised learning with graph embeddings. arXiv preprint
arXiv:1603.08861, 2016b.

Muhammed A Yildirim and Michele Coscia. Using random walks to

generate associations between objects. PloS one, 9(8):e104813, 2014.

Hao Yin, Austin R Benson, Jure Leskovec, and David F Gleich.

Local higher-order graph clustering. In Proceedings of the 23rd ACM
SIGKDD International Conference on Knowledge Discovery and Data
Mining, pages 555–564. ACM, 2017.

Vahan Yoghourdjian, Tim Dwyer, Karsten Klein, Kim Marriott, and

Michael Wybrow. Graph thumbnails: Identifying and comparing
multiple graphs at a glance. IEEE Transactions on Visualization and
Computer Graphics, 24(12):3081–3095, 2018.
 bibliography 757

Soon-Hyung Yook, Hawoong Jeong, A-L Barabási, and Yuhai Tu.

Weighted evolving networks. Physical review letters, 86(25):5835, 2001.

Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure
Leskovec. Graphrnn: Generating realistic graphs with deep auto-
regressive models. In International Conference on Machine Learning,
pages 5694–5703, 2018.

Hyejin Youn, Deborah Strumsky, Luis MA Bettencourt, and José

Lobo. Invention as a combinatorial process: evidence from us patents.
Journal of The Royal Society Interface, 12(106):20150272, 2015.

Jean-Gabriel Young, Giovanni Petri, Francesco Vaccarino, and Alice

Patania. Construction of and efficient sampling from the simplicial
configuration model. Physical Review E, 96(3):032312, 2017.

Amy Zhao Yu, Shahar Ronen, Kevin Hu, Tiffany Lu, and César A
Hidalgo. Pantheon 1.0, a manually verified dataset of globally
famous biographies. Scientific data, 3:150075, 2016.

Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph

convolutional networks: a deep learning framework for traffic
forecasting. In IJCAI, pages 3634–3640. AAAI Press, 2018.

Haiyuan Yu and Mark Gerstein. Genomic analysis of the hierarchical

structure of regulatory networks. Proceedings of the National Academy
of Sciences, 103(40):14724–14731, 2006.

Jingjin Yu and Daniela Rus. Pebble motion on graphs with rotations:

Efficient feasibility tests and planning algorithms. In Algorithmic
Foundations of Robotics XI, pages 729–746. Springer, 2015.

Kai Yu, Wei Chu, Shipeng Yu, Volker Tresp, and Zhao Xu. Stochastic
relational models for discriminative link prediction. In Advances in
neural information processing systems, pages 1553–1560, 2007.

Burcu Yucesoy and Albert-László Barabási. Untangling performance

from success. EPJ Data Science, 5(1):17, 2016.

Wayne W Zachary. An information flow model for conflict and

fission in small groups. Journal of anthropological research, 33(4):
452–473, 1977.

Laura A Zager and George C Verghese. Graph similarity scoring and

matching. Applied mathematics letters, 21(1):86–94, 2008.

G Zehfuss. Über eine gewisse determinante. Zeitschrift für Mathematik

und Physik, 3(1858):298–301, 1858.
758 the atlas for the aspiring network scientist

Zhiping Zeng, Jianyong Wang, Lizhu Zhou, and George Karypis.

Coherent closed quasi-clique discovery from large dense graph
databases. In Proceedings of the 12th ACM SIGKDD international
conference on Knowledge discovery and data mining, pages 797–802.
ACM, 2006.

Hongming Zhang, Liwei Qiu, Lingling Yi, and Yangqiu Song. Scal-
able multiplex network embedding. In IJCAI, volume 18, pages
3082–3088, 2018.

Jun Zhang and F Yu Kai. What’s the relative risk?: A method of

correcting the odds ratio in cohort studies of common outcomes.
Jama, 280(19):1690–1691, 1998.

Jun Zhang, Mark S Ackerman, and Lada Adamic. Expertise networks

in online communities: structure and algorithms. In Proceedings of the
16th international conference on World Wide Web, pages 221–230, 2007a.

Muhan Zhang and Yixin Chen. Link prediction based on graph

neural networks. In Advances in Neural Information Processing Systems,
pages 5165–5175, 2018.

Ning Zhang, Yuanyuan Tian, and Jignesh M Patel. Discovery-driven

graph summarization. In ICDE, pages 880–891. IEEE, 2010.

Pan Zhang and Cristopher Moore. Scalable detection of statistically

significant communities and hierarchies, using message passing for
modularity. Proceedings of the National Academy of Sciences, 111(51):
18144–18149, 2014.

Peng Zhang, Jinliang Wang, Xiaojia Li, Menghui Li, Zengru Di, and
Ying Fan. Clustering coefficient and community structure of bipartite
networks. Physica A: Statistical Mechanics and its Applications, 387(27):
6869–6875, 2008.

Shihua Zhang, Xuemei Ning, and Xiang-Sun Zhang. Identifica-

tion of functional modules in a ppi network by clique percolation
clustering. Computational biology and chemistry, 30(6):445–451, 2006.

Shihua Zhang, Rui-Sheng Wang, and Xiang-Sun Zhang. Identification

of overlapping community structure in complex networks using
fuzzy c-means clustering. Physica A: Statistical Mechanics and its
Applications, 374(1):483–490, 2007b.

Xiao Zhang, Travis Martin, and Mark EJ Newman. Identification of

core-periphery structure in networks. Physical Review E, 91(3):032803,
2015.
 bibliography 759

Zhong-Yuan Zhang and Yong-Yeol Ahn. Community detection

in bipartite networks using weighted symmetric binary matrix
factorization. International Journal of Modern Physics C, 26(09):1550096,
2015.

Peixiang Zhao, Xiaolei Li, Dong Xin, and Jiawei Han. Graph cube:
on warehousing and olap multidimensional networks. In Proceedings
of the 2011 ACM SIGMOD International Conference on Management of
data, pages 853–864, 2011.

Elena Zheleva and Lise Getoor. Preserving the privacy of sensitive

relationships in graph data. In International Workshop on Privacy,
Security, and Trust in KDD, pages 153–171. Springer, 2007.

Elena Zheleva and Lise Getoor. To join or not to join: the illusion
of privacy in social networks with mixed public and private user
profiles. In Proceedings of the 18th international conference on World wide
web, pages 531–540, 2009.

Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu,
and Maosong Sun. Graph neural networks: A review of methods and
applications. arXiv preprint arXiv:1812.08434, 2018.

Shi Zhou and Raúl J Mondragón. The rich-club phenomenon in the

internet topology. IEEE Communications Letters, 8(3):180–182, 2004.

Tao Zhou, Jie Ren, Matúš Medo, and Yi-Cheng Zhang. Bipartite
network projection and personal recommendation. Physical Review
E, 76(4):046115, 2007.

Tao Zhou, Linyuan Lü, and Yi-Cheng Zhang. Predicting missing

links via local information. The European Physical Journal B, 71(4):
623–630, 2009.

Tao Zhou, Zoltán Kuscsik, Jian-Guo Liu, Matúš Medo, Joseph Rush-
ton Wakeling, and Yi-Cheng Zhang. Solving the apparent diversity-
accuracy dilemma of recommender systems. Proceedings of the
National Academy of Sciences, 107(10):4511–4515, 2010.

GK Zipf. The psycho-biology of language. 1935.

Marinka Zitnik, Monica Agrawal, and Jure Leskovec. Modeling

polypharmacy side effects with graph convolutional networks.
Bioinformatics, 34(13):i457–i466, 2018.

Vinko Zlatić, Andrea Gabrielli, and Guido Caldarelli. Topologically

biased random walk and community finding in networks. Physical
Review E, 82(6):066109, 2010.
760 the atlas for the aspiring network scientist

Ezra W Zuckerman and John T Jost. What makes you think you’re so
popular? self-evaluation maintenance and the subjective side of the"
friendship paradox". Social Psychology Quarterly, pages 207–223, 2001.

Konrad Zuse. Der Plankalkül. Number 63. Gesellschaft für Mathe-

matik und Datenverarbeitung, 1972.