ARTICLE IN PRESS

0378-4371/$ - se

doi:10.1016/j.ph

�CorrespondE-mail addr

1We use the

weighted undir

Physica A 385 (2007) 385–396

www.elsevier.com/locate/physa

Information theoretic description of networks

Thomas Wilhelm�, Jens Hollunder

Theoretical Systems Biology, FLI Jena, Beutenbergstrasse 11, D-07745 Jena, Germany

Received 25 April 2007; received in revised form 31 May 2007

Available online 30 June 2007

Abstract

We present a new information theoretic approach for network characterizations. It is developed to describe the general

type of networks with n nodes and L directed and weighted links, i.e., it also works for the simpler undirected and

unweighted networks. The new information theoretic measures for network characterizations are based on a

transmitter–receiver analogy of effluxes and influxes. Based on these measures, we classify networks as either complex

or non-complex and as either democracy or dictatorship networks. Directed networks, in particular, are furthermore

classified as either information spreading and information collecting networks.

The complexity classification is based on the information theoretic network complexity measure medium articulation (MA).

It is proven that special networks with a medium number of links (L�n1:5) show the theoretical maximum complexity

MA ¼ ðlog nÞ2=2. A network is complex if its MA is larger than the average MA of appropriately randomized networks:

MA4MAr. A network is of the democracy type if its redundancy RoRr, otherwise it is a dictatorship network. In democracy

networks all nodes are, on average, of similar importance, whereas in dictatorship networks some nodes play distinguished roles

in network functioning. In other words, democracy networks are characterized by cycling of information (or mass, or energy),

while in dictatorship networks there is a straight through-flow from sources to sinks. The classification of directed networks into

information spreading and information collecting networks is based on the conditional entropies of the considered networks

(HðA=BÞ ¼ uncertainty of sender node if receiver node is known, HðB=AÞ ¼ uncertainty of receiver node if sender node is

known): if HðA=BÞ4HðB=AÞ, it is an information collecting network, otherwise an information spreading network.

Finally, different real networks (directed and undirected, weighted and unweighted) are classified according to our

general scheme.

r 2007 Elsevier B.V. All rights reserved.

PACS: 89.70.þc; 89.75.�k; 02.10.Ox

Keywords: Information theory; Complexity; Graph; Network; Network characterization

1. Introduction

The development of measures to characterize networks1 is a classical field of research in graph theory.An undirected unweighted network, i.e., a graph, can completely be characterized by its complete set of graph

e front matter r 2007 Elsevier B.V. All rights reserved.

ysa.2007.06.029

ing author: Tel./fax: +49 3641 65 6208/3641 65 6191.

ess: thomas.wilhelm@bbsrc.ac.uk (T. Wilhelm).

term network in the most general sense: it comprises undirected unweighted graphs, directed unweighted digraphs and

ected and directed networks.

ARTICLE IN PRESST. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396386

invariants, e.g. the spectrum of the adjacency matrix together with the canonical star basis [1]. That means, given onlythe complete set of graph invariants, the corresponding graph can unambiguously be reconstructed. Unfortunately,generally the time complexity of such graph invariants is high, even if polynomial time algorithms exist for manyclasses of graphs [1]. Because the computationally simplest graph invariants are unknown, the famous graphisomorphism problem is still unsolved. It represents an own complexity class. However, other measures can beinformative and useful without characterizing a graph or network completely. Recently, in a survey of correspondingmeasurements the authors raised the ‘‘important question of how to choose the most appropriate measurements’’ [2].

Starting with two pioneering works [3,4], the last years have seen a surge of papers dealing with networks onfields as diverse as social networks [5–7], food webs [8–10], communication networks [11,12], transportationnetworks [13], and sub-cellular networks, such as metabolic [14,15], protein interaction [16,17], and geneticnetworks [18]. Many interesting network properties have been found, providing us insights into both, thedynamics on networks (fixed links) [19], dynamics of networks (dynamic rewiring) [20], and (on another timescale) the evolutionary processes leading to such networks [4,21]. However, nearly all of these papers deal withgraphs, i.e., unweighted networks, where links are either present or absent. Here we go one step further andpresent a new characterization of the most general form of networks: directed weighted networks. Note thatboth, undirected and unweighted networks can also be analysed in this framework: in unweighted networksthe edges have only two different weights (0—no edge, 1—edge), and undirected networks can be understoodas directed with all links pointing in both directions (with the appropriate weight).

Many unweighted networks could better be described by their weighted form. For instance, in acquaintancenetworks one can quantify the acquaintances, or in collaboration networks one can count the number of jointlywritten papers. A weighted network description is especially important for food webs where no standard forestablishment exists: it depends on the personal choice of the ecologist if weak links are also counted. Forinstance, lions are usually considered as top predators, but gnats also bite them, so there is a small material fluxalso from lions to gnats which is usually not considered. However, the decision whether to take small fluxes intoaccount affects the statistics of the corresponding unweigted network [9,10]. The latter can simply be deducedfrom its weighted form [10]: a special cut-value is defined, if the weight is larger cut the link is set, if not it isneglected. We have shown that such a deduced unweighted network may not be distinguishable from a random(Erd+os-Renyi-) network, although the original weighted network was clearly structured [9]. Moreover, wepreviously argued that different network measures should better be discussed as a function of cut [10]. Up to nowthere is much ambiguousness in food web theory. Fourteen years ago the leading ‘‘students of food webs’’urgently demanded a common food web standard [22]. However, up to now no such standard has beenestablished, but it seems to be generally accepted that food webs should contain weighted links [10,22].

A simple Gedankenexperiment (thought experiment) shows the potential ambiguity of the characterizationof weighted networks with measures developed to describe unweighted networks: the highly weighted links canshow a scale-free property [4] (power-law distribution of node degrees, may imply preferential attachment innetwork evolution), but a different degree distribution may be obtained if also links with small weights areconsidered (may imply another network evolution).

Generally, it is important to develop a deeper understanding of weighted networks. Appropriate statisticalmeasures to characterise such networks are therefore needed. Starting already five decades ago [23], theoreticalecologists developed different information theoretic measures for the description of directed and weightedfood webs [9,10,24–28]. Here we present the general theory, capable to characterize each type of networks.

In the first part of this paper we motivate and introduce information theoretic measures for networkdescriptions, based on the transmitter–receiver analogy of effluxes and influxes of nodes. In the second part wepresent an information theoretic complexity measure for networks: MA. It is proven that networks with n

nodes cannot be more complex than MAmax ¼ ðlog nÞ2=2. In Section 4 we show that networks can be classifiedas complex or non-complex and as democracy or dictatorship networks. Finally, we discuss the correspondingclass-membership of different real networks.

2. Information theoretic measures for network characterisations

In the 1940s and 1950s Claude Elwood Shannon developed his celebrated theory of communication [29]. Inthis later called information theory he considered a transmitter sending for instance a string of symbols

ARTICLE IN PRESST. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396 387

ða2; a4; a1; a2; . . .Þ and a receiver receiving another string ðb2; b1; b1; b3; . . .Þ. According to Shannon, theinformation content (information capacity) of such a signal, for instance the sender, can be quantified with theentropy HðAÞ ¼ �

PipðaiÞ log pðaiÞ. The amount of information which was really transmitted from the sender

to the receiver (because in technical signal transductions there is always information lost due to noise) ismeasured by the mutual information (or transinformation, the joint information of sender and receiver)I ¼

Pi

Pjpðai; bjÞ logðpðai; bjÞ=pðaiÞpðbjÞÞ, etc. Thirty years ago, i.e., about 30 years after Shannon, Rutledge

et al. [24] found an interesting sender-receiver analogy for directed networks: all effluxes of nodes correspondto the sender, the influxes to the receiver. This approach was later used also by other ecologists [25,26], butit was always confined to food web descriptions with a terminology making it difficult to be applied forgeneral network descriptions. We now present the corresponding general information theoretic networkcharacterization.

Most generally, we consider directed weighted networks with n nodes. Fig. 1 shows a n ¼ 4 examplenetwork. tij denotes the weight of a flux from node i to node j. The total system throughflow is defined as

TST ¼P

i

Pj tij. In the following, Tij exclusively denotes the normalized weight of the flux from i to j:

Tij ¼ tij=TST . The sender–receiver analogy of networks is the following: the probability for a symbol in the

transmitter signal corresponds to the sum of all effluxes from a given node, and the probability for a symbol inthe receiver signal corresponds to the sum of all influxes to a given node (pðaiÞ can therefore be exchanged withpðTi:Þ ¼

PjTij , etc.): pðaiÞ2pðTi:Þ ¼

PjT ij, pðbjÞ2pðT :jÞ ¼

PiTij . The conditional probability of receiving

bj if ai was sent corresponds to the conditional probability that j is the sink node if i was the source

pðbj=aiÞ2pðT :j=Ti:Þ ¼ Tij=Ti: and the joint probability of ai sent and bj received corresponds to the joint

probability that i was the source and j the sink node: pðai; bjÞ2pðTi:;T :jÞ ¼ Tij. According to Shannon’s

deduction, the following formulas characterizing networks are simply obtained:

�

Entropy of effluxes ¼ uncertainty of sender

HðAÞ ¼ �X

i

Xj

Tij

!log

Xj

T ij

!.

�

Entropy of influxes ¼ uncertainty of receiver

HðBÞ ¼ �X

j

Xi

Tij

!log

Xi

T ij

!.

t33

t14

t13

t43

t21

t12

1

4

3

2

Fig. 1. A directed weighted four-node example network.

ARTICLE IN PRESST. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396388

Uncertainty of sender node if receiver is known to be node j

�

HðA=bjÞ ¼ �X

i

TijPkTkj

logTijPkTkj

.

�

Uncertainty of receiver node if sender is known to be node i

HðB=aiÞ ¼ �X

j

TijPkTik

logTijPkTik

.

�

Conditional entropy 1 ¼ uncertainty of sender if receiver is known

HðA=BÞ ¼X

j

pðbjÞHðA=bjÞ ¼ �X

i

Xj

T ij logTijPkTkj

.

�

Conditional entropy 2 ¼ uncertainty of receiver if sender is known

HðB=AÞ ¼X

i

pðaiÞHðB=aiÞ ¼ �X

i

Xj

Tij logTijPkTik

.

�

Redundancy ¼ conditional entropy 1þ 2

RðA;BÞ ¼ HðA=BÞ þHðB=AÞ ¼ �X

i

Xj

Tij logT2

ijPkTik

PlTlj

.

�

Joint entropy ¼ uncertainty of sender and receiver

HðA;BÞ ¼ �X

i

Xj

T ij logTij.

�

Mutual information ¼ information about sender if receiver is known and vice versa

IðA;BÞ ¼ HðA;BÞ � RðA;BÞ ¼ HðAÞ �HðA=BÞ ¼ HðBÞ �HðB=AÞ

¼X

i

Xj

Tij logTijP

kTik

PlTlj

.

These measures are applicable to the description of all networks (directed, undirected, weighted,unweighted). In the unweighted case some measures correspond to other well-known quantities, for instance,the joint entropy is simply the logarithm of the number of links, HðA=bjÞ is the logarithm of the in-degree ofnode j and HðB=aiÞ is the logarithm of the out-degree of node i. The maximum possible values (for unweightedand also for weighted networks) are 2 log n for HðA;BÞ and RðA;BÞ (cf. next section) and log n for all othermeasures. These maxima can be used for normalizations (e.g. in Table 1).

Note that the joint entropy carries no information about network topology, but only information about theweight distribution of links. Because HðA;BÞ ¼ RðA;BÞ þ IðA;BÞ, the topological information contained inRðA;BÞ and IðA;BÞ is the same (for any given link distribution of weights). Thus, network characterizationneeds only one of these measures, either RðA;BÞ or IðA;BÞ, it is simply a matter of taste which one is preferred.In Sections 4 and 5 we use RðA;BÞ for corresponding network classifications. A more detailed discussion ofRðA;BÞ and IðA;BÞ is given in the next section, building the basis for the deduction of a proper networkcomplexity measure that shows its maximum for networks with a medium number of links.

It can be seen from the corresponding formulas that pure information spreading networks, such asthe example in Fig. 2A, are characterized by HðAÞ ¼ HðA=BÞ ¼ IðA;BÞ ¼ 0 and HðB=AÞ ¼ HðB=AÞmax ¼

ARTICLE IN PRESS

Fig. 2. Pure dictatorship networks. (A) Information spreading network; (B) information collecting network. All links must have the same

weight to reach the extreme cases (A) HðA=BÞ ¼ 0 and (B) HðB=AÞ ¼ 0.

T. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396 389

RðA;BÞ ¼ HðA;BÞ ¼ log n. The entropy of the effluxes is zero, because there is no uncertainty of the source ofany flux in the network. Likewise, the conditional entropy for known receiver is zero, because there is, ofcourse, no remaining uncertainty of the sender if the receiver is known. The mutual information is also zero,because there is no mutual dependence between sender and receiver, or in other words, knowing the receiveradds no information about the sender and vice versa, knowing the sender adds no information aboutthe receiver. In contrast to pure information spreading networks, pure information collecting networks (e.g.Fig. 2B) are characterized by HðBÞ ¼ HðB=AÞ ¼ IðA;BÞ ¼ 0 and HðA=BÞ ¼ HðA=BÞmax ¼ RðA;BÞ ¼HðA;BÞ ¼ log n. Analog to the mutual dependence of RðA;BÞ and IðA;BÞ (both sum to HðA;BÞ), alsoHðA=BÞ and HðB=AÞ carry the same information about network structure: HðA=BÞ þHðB=AÞ ¼ RðA;BÞ.Generally, information spreading networks are characterized by HðB=AÞ4HðA=BÞ, information collectingnetworks are characterized by HðB=AÞoHðA=BÞ.

Note also that HðAÞ �HðA=BÞ ¼ HðBÞ �HðB=AÞ (¼ IðA;BÞ), i.e., the entropies HðAÞ and HðBÞ carry noadditional information compared to HðA=BÞ and HðB=AÞ. Again, it’s a matter of taste which measures areused for classification of directed networks as information spreading or information collecting. We use theconditional entropies, because this fits well to our also used RðA;BÞ: RðA;BÞ ¼ HðA=BÞ þHðB=AÞ.Summarizing, we use three information theoretic measures for network characterizations: the joint entropyHðA;BÞ to characterize the weight distribution of links, and two measures carrying information aboutnetwork topology: RðA;BÞ and HðA=BÞ. A fourth measure, yielding information about aspects of networkcomplexity, is presented in the next section.

3. The network complexity measure medium articulation

It directly follows from the definition of the joint entropy that HðA;BÞ ¼ HðA;BÞmax ¼ 2 log n ifTij ¼ 1=n28i; j. Such a network is shown in Fig. 3A. HðA;BÞ ¼ HðA;BÞmin ¼ 0 if there is just one link in thenetwork (Tij ¼ 1 for some i; j and the remaining links equal zero). Furthermore: RðA;BÞ ¼ RðA;BÞmax ¼

2 log n if Tij ¼ 1=n28i; j (Fig. 3A) and RðA;BÞ ¼ RðA;BÞmin ¼ 0 if T2ij ¼

PkTkj

PkTik8Tija0 (Fig. 3C).

IðA;BÞ ¼ IðA;BÞmin ¼ 0 if Tij ¼P

kTkj

PkTik8ti;ja0. (Fig. 3A); IðA;BÞ ¼ IðA;BÞmax ¼ log n if HðA;BÞ is as

large as possible (HðA;BÞjIðA;BÞmax¼ log n) under the condition RðA;BÞ ¼ RðA;BÞmin ¼ 0 (Fig. 3C). Note that

all networks in the extreme cases HðA;BÞ ¼ HðA;BÞmax ¼ RðA;BÞmax and IðA;BÞ ¼ IðA;BÞmax belong to theclass of Kirchhoff-networks where

PiT ij ¼

PiT ji 8j. Summarizing, highly connected networks are

characterized by high HðA;BÞ and high RðA;BÞ, but low IðA;BÞ-values. In contrast, sparsely connectednetworks with in- and effluxes per node, i.e., highly ‘‘articulated’’ networks [9], have low HðA;BÞ- and RðA;BÞ-values, but a high mutual information IðA;BÞ.

ARTICLE IN PRESST. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396390

Previously, strings of symbols have been characterized as complex if being neither monotonic nor purelyrandom, but somewhere in between: a measure being zero in the monotonic and maximum in the random case(e.g. entropy) and an opposite measure (e.g. maxentropy–entropy) have been multiplied to give thecorresponding complexity measure [30,31]. Using the same simple multiplication idea we quantify thecomplexity of networks with the measure MA [9]:

MA ¼ IðA;BÞRðA;BÞ

¼ IðA;BÞðHðA;BÞ � IðA;BÞÞ

¼ RðA;BÞðHðA;BÞ � RðA;BÞÞ. ð1Þ

In contrast to other network complexity measures [32–34], MA is a typical complexity measure in the sensethat it is zero in the extreme cases (here: if either IðA;BÞ ¼ 0 or RðA;BÞ ¼ 0), but maximum in between. Thus,MA ¼ 0 for the networks given in Figs. 3A and C.

In the following we prove that the complexity of a network with n nodes cannot be larger thanMAmax ¼ ðlog nÞ2=2. Such MAmax networks have L ¼ n1:5 links, lying exactly between sparse (L ¼ n) and fullyconnected networks (L ¼ n2). Fig. 3B shows a corresponding n ¼ 4 MAmax example network. Note that alllinks of the networks shown in Figs. 3A–C must have equal weights to represent the extreme cases RðA;BÞ ¼RðA;BÞmax (A), MA ¼MAmax (B), and IðA;BÞ ¼ IðA;BÞmax (C).

Fig. 4 shows IðA;BÞ and RðA;BÞ for all directed unweighted n ¼ 4 networks.Generally, MA ¼ RðA;BÞ � IðA;BÞ and HðA;BÞ ¼ RðA;BÞ þ IðA;BÞ, i.e., a network with a given number of

links L can only have maximum MA if HðA;BÞ is as large as possible, i.e., if all links have equal weights. Itfollows:

HewðA;BÞ ¼ RðA;BÞ þ IðA;BÞ ¼ logL. (2)

Fig. 4 shows for each L the lines RðA;BÞ ¼ log L� IðA;BÞ. Moreover, each MA corresponds to a line

RðA;BÞIðA;BÞ ¼ const. (3)

in the IðA;BÞ;RðA;BÞ-plane. The maximum MA for a given L is obtained at the osculation point of the twolines (2) and (3):

MAmaxðLÞ ¼MAðIop;RopÞ ¼MAlog L

2;log L

2

� �¼ðlog LÞ2

4. (4)

However, as can be seen from Fig. 4, all networks lie left of the line

RðA;BÞ ¼ f ðIðA;BÞÞ ¼ 2ðlog n� IðA;BÞÞ. (5)

It follows from (2) and (5) that the intersection point of these two lines is at

ðI iðA;BÞ;RiðA;BÞÞ ¼ ðlog ðn2=LÞ; log ðL2=n2ÞÞ. (6)

A comparison of the points (4) and (6) shows that the maximum MAmaxðn;LÞ (4) is for all L4n4=3 outside theallowed region (5) (n ¼ 4: all L46). That means, for all L4n4=3, the maximum MA is obtained by thenetworks with the lowest possible redundancy (the highest possible mutual information). It also follows thatthe absolute possible maximum MA ¼MAmax for a given n is obtained by the networks lying closest to theosculation point of the two lines (3) and (5):

MAmaxðnÞ ¼MAðI jMAmax;RjMAmax

Þ ¼MAlog n

2; log n

� �¼ðlog nÞ2

2. (7)

A comparison of RðA;BÞ in (6) and (7) shows that MAmax-networks have L ¼ n1:5 links if L=n is integer andL�n1:5 links if not.

The line (5) can be understood as follows: networks lying on this line have a mutual information IðA;BÞ ¼

logðn2=LÞ (6). It can be seen from the definition IðA;BÞ ¼P

i

PjT ij log

TijPkTik

PlTlj

that such a IðA;BÞ can only

be obtained by a network where each of the L fluxes (equal weights: Tij ¼ 1=L) comes from a node with L=n

effluxes and points to a node with L=n influxes. This network topology realizes the highest possible mutual

ARTICLE IN PRESS

Fig. 3. Three different 4-node-networks [26]. (A) maximally connected, HðA;BÞ ¼ HðA;BÞmax, RðA;BÞ ¼ RðA;BÞmax, IðA;BÞ ¼ 0,

MA ¼ 0; (B) moderately connected, i.e., moderately articulated, RðA;BÞ ¼ RðA;BÞmax=2, IðA;BÞ ¼ IðA;BÞmax=2, MA ¼MAmax; and (C)

minimally connected, i.e., maximally articulated, RðA;BÞ ¼ 0, IðA;BÞ ¼ IðA;BÞmax, MA ¼ 0.

T. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396 391

information, i.e., the average information about the source of a flux if one knows its sink and about the sink ofa flux if one knows its source is highest. In other words, such networks realize the lowest possible redundancy,i.e., the lowest possible sum of the conditional entropies. Of course, only networks with integer L=n values canlie exactly on the boundary (5). For n ¼ 4 this are networks with L ¼ 4; 8; 12; 16 (cf. Fig. 4).

4. Complex and non-complex democracy and dictatorship networks

Here we show that the measures described above can be used to classify all networks into one of fourdifferent classes. For that purpose, the redundancy RðA;BÞ and the Medium Articulation MA of a givennetwork are compared with the average RðA;BÞr and MAr of correspondingly randomized networks (edges arecompletely randomly rewired; for undirected unweighted networks this are the corresponding Erd +os-Renyi-graphs). Because HðA;BÞ does not depend on network topology, but only on the number and weights of theedges, it follows HðA;BÞ ¼ HðA;BÞr. Fig. 3C shows that RðA;BÞ ¼ 0 for minimally connected networks with aring structure. Fig. 2 shows that other minimally connected networks (‘‘stars’’) can have a vanishing mutualinformation, i.e., a maximum redundancy. Obviously, ring-networks with RðA;BÞ ¼ 0 have a lowerredundancy than their random counterparts RðA;BÞoRðA;BÞr. Because in such networks the nodes playequal roles we call them democracy networks. The information tends to cycle in democracy networks, whichcan therefore also be called cycling networks. Dictatorship networks, in contrast, have RðA;BÞ4RðA;BÞr and a

ARTICLE IN PRESS

Fig. 4. IðA;BÞ and RðA;BÞ for all directed unweighted n ¼ 4 networks (non-normalized values, binary logarithm). Each solid line

corresponds to a given number of links 1pLp16. The dashed line indicates the upper bound of IðA;BÞ (lower bound of RðA;BÞ) for allLX4.

T. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396392

tendency for straight information through-flow from sources to sinks. Thus, dictatorship networks could alsobe named source-sink or throughflow networks.

The network complexity measure MA serves to subdivide the two major groups: complex networks withMA4MAr lie between pure democracy and pure dictatorship networks. Summarizing, pure democracynetworks have RðA;BÞoRðA;BÞr;MApMAr, complex democracy networks RðA;BÞoRðA;BÞr;MA4MAr,complex dictatorship networks RðA;BÞ4RðA;BÞr;MA4MAr, and pure dictatorship networks RðA;BÞ4RðA;BÞr;MApMAr.

We illustrate this general classification scheme with some examples of directed unweighted networks, i.e.,Tij ¼ 1=L (8Tij40). Each directed unweighted network with L links has a joint entropy HewðA;BÞðn;LÞ ¼logL, for any number of nodes n.

Fig. 5A shows the redundancy RðA;BÞ and Medium Articulation MA for all networks with n ¼ 6 nodes andL ¼ 6 directed unweighted edges. It can be seen that all four different network types can be found within theclass of directed n ¼ 6;L ¼ 6 networks. Fig. 5B shows for some selected L the corresponding RðA;BÞ and MA

for all n ¼ 6 networks, as well as the corresponding random network values RðA;BÞr and MAr. It can be seenthat complex dictatorship networks only exist for small L. Analysis shows that for a given n there are manymore complex democracy, than complex dictatorship networks. Pure democracy networks only exist for evensmaller L. In other words, most democracy networks are complex, whereas most dictatorship networks arenon-complex.

5. Classification of real networks

In contrast to the well-known network classifications ‘‘small-world’’ [3] and ‘‘scale-free’’ [4], ourclassification scheme is of maximum generality. It is applicable to all four network types: directed andundirected, and weighted and unweighted networks. Table 1 shows the classification of some real networksbelonging to each of these types.

The analysed food webs (all directed networks) are always dictatorship networks. This is plausible, becauseof the underlying trophic hierarchy. Note furthermore that all weighted food webs are information collecting

ARTICLE IN PRESS

0.5

0.4

0.3

0.2

0.1

00 0.1 0.2 0.3 0.4 0.5

R (A,B)

R (A,B)r

MA

MAr

democracy dictatorship

complex

non-complex

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

00 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

R (A,B)

MA

L=9

L=18

L=24

L=30

L=12

L=11

L=8

L=7

L=6

L=5

L=4

L=3

L=2

Fig. 5. Classification of directed unweighted n ¼ 6 networks (normalized RðA;BÞ and MA). Each point denotes at least one network. For

the sake of clarity points are connected with splines. All networks above the horizontal MAr line are complex networks (below are non-

complex (pure) networks), all networks left of RðA;BÞr are democracy networks (right are dictatorship networks). A) All networks with

L ¼ 6 links. B) All networks with L ¼ 2; 3; ::: links. x indicates the corresponding arithmetic mean values of the randomized networks

(RðA;BÞr;MAr).

T. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396 393

networks (HðA=BÞ4HðB=AÞ). This is the expected result, because the measured fluxes are typically carbonflows and point from prey to predator. The unweighted food webs, in contrast, do not show this consistentresult: only two of the five networks are information collecting. This result presents another argument forweighted food webs, against unweighted ones [9,10]. Four of the five unweighted food webs are non-complex,whereas 11 of 12 weighted food webs are complex.

To compare the full information contained in weighted networks with the reduced information ofcorrespondingly deduced unweighted networks, we have taken the largest weighted network (n ¼ 66) as

ARTICLE IN PRESS

Table 1

Classification of real networksa

Networks n L HðA;BÞ RðA;BÞ RðA;BÞr MA MAr HðA=BÞ HðB=AÞ

Directed and unweighted (food webs)

Grassland 88 137 0.549 0.232 0.18 0.589 0.532 0.186 0.278

Little Rock Lake 183 2494 0.751 0.613 0.508 0.675 0.986 0.67 0.556

Silwood Park 154 370 0.587 0.396 0.219 0.605 0.645 0.24 0.552

St. Martin Island 45 224 0.711 0.524 0.446 0.784 0.945 0.532 0.515

Ythan Estuary 135 601 0.652 0.452 0.328 0.724 0.851 0.401 0.503

Directed and weighted (food webs)

fw1 21 82 0.212 0.11 0.037 0.09 0.044 0.176 0.043

fw2 21 61 0.222 0.124 0.036 0.097 0.047 0.202 0.046

fw3 36 122 0.400 0.186 0.104 0.319 0.243 0.191 0.182

fw4 36 166 0.346 0.295 0.084 0.121 0.173 0.300 0.290

fw5 21 55 0.550 0.303 0.249 0.598 0.595 0.360 0.246

fw6 66 791 0.263 0.159 0.022 0.132 0.041 0.233 0.084

fw7 43 348 0.291 0.233 0.057 0.107 0.104 0.313 0.154

fw8 32 158 0.380 0.263 0.099 0.247 0.22 0.274 0.252

fw9 51 270 0.439 0.269 0.133 0.365 0.323 0.287 0.251

fw10 34 158 0.476 0.259 0.164 0.451 0.407 0.273 0.245

fw11 34 149 0.441 0.245 0.139 0.385 0.333 0.264 0.227

fw12 34 115 0.488 0.284 0.177 0.462 0.438 0.293 0.276

neural networks

C. elegans (jsh) 190 4336 0.691 0.451 0.399 0.868 0.933 0.474 0.427

C. elegans (n2u) 202 3963 0.693 0.446 0.403 0.883 0.936 0.454 0.437

Undirected and weighted railway network 213 332 0.499 0.108 0.115 0.339 0.352 0.108 0.108

Undirected and unweighted E. coli prot–prot interaction 270 1432 0.649 0.403 0.43 0.793 0.966 0.403 0.403

aData from: directed unweighted food webs (www.cosin.org/extra/data/foodwebs/web.html), directed weighted food webs

(www.cbl.umces.edu/�ulan/ntwk/network.html, all networks with n420), directed weighted neural networks (elegans.swmed.edu/parts/

neurodata.txt), Brandenburg railway network (www.bahnstrecken.de/strecken.htm), and E.coli protein–protein interaction network

(www.cosin.org/extra/data/proteins).

All entries are normalized values. The last column carries no additional information (HðB=AÞ ¼ RðA;BÞ �HðA=BÞ, note the

normalization), but is shown to facilitate the direct comparison with HðA=BÞ. For simple readability, the corresponding larger values

are printed in bold: for dictatorship networks, complex networks, and information collecting networks RðA;BÞ-values, MA-values, and

HðA=BÞ-values are bold, respectively.

T. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396394

unweighted (i.e., all fluxes with weights above a cut-value are 1, the others are 0; if cut ¼ 0, it is Tij ¼ 1=66). Incontrast to the original weighted network (a complex dictatorship network), this unweighted network is a puredictatorship network (RðA;BÞ ¼ 0:684RðA;BÞr ¼ 0:60;MA ¼ 0:65oMAr ¼ 0:94), but with other cut-valuesone again obtains complex dictatorship networks (cut ¼ 1=10 000; 1=100; 1=10 �max tij). This ambiguity ofnetwork types depending on cut-values was first discussed in our previous papers [9,10] and needs additionalstudies in future.

In the analysed two neural networks the nodes are neurons and the weights correspond to the synapticstrength between the neurons (male adult worm (jsh), hermaphrodite worm (n2u)). Interestingly, bothnetworks are pure dictatorship networks, that means there is a tendency to straight information through-flowfrom sources to sinks. This feature is even more pronounced in the neural network of the male adult worm.

In the undirected weighted railway network of the German federal state Brandenburg nodes are stationsand weights correspond to spatial distances. It is a pure democracy network. The cycling property can easilybe understood, because the whole railway network has the form of a cycle: it is circled around Berlin (Fig. 6).It will be interesting to compare this result to other transportation networks.

The analysed protein–protein interaction network is of the pure democracy type, which indicates that, onaverage, proteins play similar roles in the corresponding networks and the information is cycling. It seemspossible to extract biologically important information from a classification analysis of different protein

ARTICLE IN PRESS

Fig. 6. The Brandenburg railway network.

T. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396 395

networks. A lower complexity could, for instance, indicate perturbation or disease, but future studies areneeded.

6. Discussion

We have presented information theoretic measures to characterize networks (directed and undirected,weighted and unweighted). The corresponding general classification scheme characterizes networks as eithercomplex or non-complex, and as either democracy (i.e., cycling) networks or dictatorship (i.e., throughflow)networks. Directed networks, in particular, are furthermore classified as information spreading or informationcollecting networks.

Our presented measures are computationally very simple because only links to/from nearest neighbours areconsidered. Inspection of the formulas in Section 2 shows that the complexity is at most OðLÞ: the double sumsmust be calculated for the L weights which are not zero. However, different extensions of this informationtheoretic network description are possible to capture also more global structural properties, at the expense ofcalculation time, of course. For instance, consideration of second neighbours can be based on the quantityp2ðai=bjÞ ¼ flux from second neighbour i to j/flux from i to all its second neighbours. It follows, e.g., theuncertainty of sender node two steps away from receiver node j: H2ðA=bjÞ ¼ �

Pip2ðai=bjÞ log p2ðai=bjÞ.

The quantification of network complexities is important in different fields of science: in the discussion ofsoftware complexity [32], to quantify the complexity of business process models [33], and in structuralchemistry to measure the complexity of chemical compounds [34]. However, all these authors considernetworks as complex if there are many links between the nodes, i.e., complete graphs (cliques) are complex. Incontrast, our Medium Articulation is a proper complexity measure for networks in the sense that networkswith a medium number of links are classified to be complex, a clique and a simple chain are both non-complex.A similar behaviour is shown by the recently proposed Offdiagonal Complexity [35] and some generalizedversions [36]. It will be interesting to compare these complexity quantifications to the recently suggestedcomplementary approach of identification of non-complex subnetworks [37].

Our analyses show that special types of real networks typically belong to special network classes. Anexample are weighted food webs, which are complex dictatorship networks. Of course, our general network

ARTICLE IN PRESST. Wilhelm, J. Hollunder / Physica A 385 (2007) 385–396396

description has numerous potential applications. It may be interesting to characterize subnetworks, e.g.communities, to see differences between different groups of nodes. Our approach also allows to quantify the‘‘importance’’ of nodes and edges [16,38], either directly (in the double sum formulas (Section 2) eachsummand corresponds to an edge, in the single sum formulas each summand corresponds to a node), or byconsideration of corresponding k-neighbourhood networks. The correlation of such ‘‘importances’’ to other(biological) quantities such as gene or metabolite essentiality or evolutionary conservation allows additionalinsight into the organization of (biological) networks [16,38].

References

[1] D. Cvetkovic, P. Rowlinson, S. Simic, Eigenspaces of Graphs Encyclopedia of Mathematics and its Applications 66, Cambridge

University Press, Cambridge, 1997.

[2] L. Costa, et al., Characterization of complex networks: a survey of measurements, Adv. Phys. 56 (1) (2007) 167–242.

[3] D.J. Watts, S.H. Strogatz, Collective dynamics of ‘small-world’ networks, Nature 393 (1998) 440–442.

[4] A.-L. Barabasi, R. Albert, Emergence of scaling in random networks, Science 286 (1999) 509–512.

[5] M.E.J. Newman, The structure of scientific collaboration networks, Proc. Natl. Acad. Sci. USA 98 (2001) 404–409.

[6] M.E.J. Newman, D.J. Watts, S.H. Strogatz, Random graph models of social networks, Proc. Natl. Acad. Sci. USA 99 (2002)

2566–2572.

[7] D.J. Watts, P.S. Dodds, M.E.J. Newman, Identity and search in social networks, Science 296 (2002) 1302–1305.

[8] A.E. Krause, K.A. Frank, D.M. Mason, R.E. Ulanowicz, W.W. Taylor, Compartments revealed in food-web structure, Nature 426

(2003) 282–285.

[9] T. Wilhelm, An elementary dynamic model for non-binary food-webs, Ecol. Model. 168 (2003) 145–152.

[10] T. Wilhelm, A study of equal node models for food webs, Ecol. Model. 181 (2005) 215–227.

[11] R. Cohen, K. Erez, D. Ben-Avraham, S. Havlin, Resilience of the internet to random breakdowns, Phys. Rev. Lett. 85 (2000)

4626–4628.

[12] T. Wilhelm, P. Hanggi, Power-law distributions resulting from finite resources, Physica A 329 (2003) 499–508.

[13] V. Latora, M. Marchiori, Is the Boston subway a small-world network?, Physica A 314 (2002) 109–113.

[14] S. Schuster, D.A. Fell, T. Dandekar, A general definition of metabolic pathways useful for systematic organization and analysis of

complex metabolic networks, Nat. Biotech. 18 (2000) 326–332.

[15] T. Wilhelm, J. Behre, S. Schuster, Analysis of structural robustness of metabolic networks, Syst. Biol. 1 (2004) 114–120.

[16] H. Jeong, S.P. Mason, A.-L. Barabasi, Z.N. Oltvai, Lethality and centrality in protein networks, Nature 411 (2001) 41–42.

[17] T. Wilhelm, H.-P. Nasheuer, S. Huang, Physical and functional modularity of the protein network in yeast, Mol. Cell. Prot. 2.5 (2003)

292–298.

[18] S. Shen-Orr, R. Milo, S. Mangan, U. Alon, Network motifs in the transkriptional regulation network of E. coli, Nat. Genet. 31 (2002)

64–68.

[19] D.J. Watts, Small Worlds, Princeton Univ. Press, Princeton, NJ, 1999.

[20] G. Palla, A.-L. Barabasi, T. Vicsek, Quantifying social group evolution, Nature 446 (2007) 664–667.

[21] R. Albert, A.-L. Barabasi, Statistical mechanics of complex networks, Rev. Mod. Phys. 74 (2002) 47–97.

[22] J.E. Cohen, et al., Improving food webs, Ecology 74 (1993) 252–258.

[23] R.H. MacArthur, Fluctuations of animal populations and a measure of community stability, Ecology 36 (1955) 533–536.

[24] R.W. Rutledge, B.L. Basore, R.J. Mulholland, Ecological stability: an information theory viewpoint, J. Theor. Biol. 57 (1976)

355–371.

[25] C. Pahl-Wostl, The Dynamic Nature of Ecosystems, Wiley, New York, 1995.

[26] R.E. Ulanowicz, Ecology, The Ascendent Perspective, Columbia Univ. Press, 1997.

[27] R.E. Ulanowicz, Information theory in ecology, Comput. Chem. 25 (2001) 393–399.

[28] R.E. Ulanowicz, Quantitative methods for ecological network analysis, Comput. Biol. Chem. 28 (2004) 321–339.

[29] C.E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J. 27 (1948) 379–423, 623–656.

[30] R. Lopez-Ruiz, H.L. Mancini, X. Calbet, A statistical measure of complexity, Phys. Lett. A 209 (1995) 321–326.

[31] J.S. Shiner, M. Davison, P.T. Landsberg, Simple measure for complexity, Phys. Rev. E 59 (1999) 1459–1464.

[32] N.R. Hall, S. Preiser, Combined network complexity measures, IBM J. Res. Develop. 28 (1984) 15–27.

[33] A.M. Latva-Koivisto, Finding a complexity measure for business process models, Research report, Helsinki University of

Technology, 2001.

[34] D. Bonchev, C.A. Buck, Quantitative measures of network complexity, Chapter 5, in: D. Bonchev, D.H. Rouvray (Eds.) Complexity

in Chemistry, Biology and Ecology, Springer, New York, 2004.

[35] J.C. Claussen, Characterization of networks by the offdiagonal complexity, Physica A 375 (2007) 365–373.

[36] A.D. Anastasiadis, et al., Measures of structural complexity in networks, in: Proceedings of the International Summer School on

Complex Systems, Santa Fe Institute, NM, 2005.

[37] L. Costa, Seeking for simplicity in complex networks, arXiv:physics/0702102, 2007.

[38] T. Manke, L. Demetrius, M. Vingron, Lethality and entropy of protein interaction networks, Genome Inf. 16 (2005) 159–163.