The Collaborative Image of The City: Mapping the

Inequality of Urban Perception

The MIT Faculty has made this article openly available. Please share
how this access benefits you. Your story matters.

Citation Salesses, Philip, Katja Schechtner, and Cesar A. Hidalgo. “The

Collaborative Image of The City: Mapping the Inequality of Urban
Perception.” Edited by Alain Barrat. PLoS ONE 8, no. 7 (July 24,
2013): e68400.
As Published [Link]
Publisher Public Library of Science

Version Final published version

Accessed Sun Mar 31 19:40:49 EDT 2019
Citable Link [Link]
Terms of Use Creative Commons Attribution
Detailed Terms [Link]
The Collaborative Image of The City: Mapping the
Inequality of Urban Perception
Philip Salesses1, Katja Schechtner1,2,3, César A. Hidalgo1,4,5*
1 The MIT Media Lab, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America, 2 Mobility Department, Austrian Institute of
Technology, Vienna, Austria, 3 Institute of Urban Design and Landscape Architecture, Vienna University of Technology, Vienna, Austria, 4 Engineering Systems Division,
Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America, 5 Instituto de Sistemas Complejos de Valparaiso, Valparaiso, Chile

Abstract
A traveler visiting Rio, Manila or Caracas does not need a report to learn that these cities are unequal; she can see it directly
from the taxicab window. This is because in most cities inequality is conspicuous, but also, because cities express different
forms of inequality that are evident to casual observers. Cities are highly heterogeneous and often unequal with respect to
the income of their residents, but also with respect to the cleanliness of their neighborhoods, the beauty of their
architecture, and the liveliness of their streets, among many other evaluative dimensions. Until now, however, our ability to
understand the effect of a city’s built environment on social and economic outcomes has been limited by the lack of
quantitative data on urban perception. Here, we build on the intuition that inequality is partly conspicuous to create
quantitative measure of a city’s contrasts. Using thousands of geo-tagged images, we measure the perception of safety,
class and uniqueness; in the cities of Boston and New York in the United States, and Linz and Salzburg in Austria, finding
that the range of perceptions elicited by the images of New York and Boston is larger than the range of perceptions elicited
by images from Linz and Salzburg. We interpret this as evidence that the cityscapes of Boston and New York are more
contrasting, or unequal, than those of Linz and Salzburg. Finally, we validate our measures by exploring the connection
between them and homicides, finding a significant correlation between the perceptions of safety and class and the number
of homicides in a NYC zip code, after controlling for the effects of income, population, area and age. Our results show that
online images can be used to create reproducible quantitative measures of urban perception and characterize the
inequality of different cities.

Citation: Salesses P, Schechtner K, Hidalgo CA (2013) The Collaborative Image of The City: Mapping the Inequality of Urban Perception. PLoS ONE 8(7): e68400.
doi:10.1371/[Link].0068400
Editor: Alain Barrat, Centre de Physique Théorique, France
Received November 5, 2012; Accepted May 22, 2013; Published July 24, 2013
Copyright: ß 2013 Salesses et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: Massachusetts Institute of Technology Media Lab Consortia Funding. The funders had no role in study design, data collection and analysis, decision to
publish, or preparation of the manuscript.
Competing Interests: César A Hidalgo is a PLOS ONE Editorial Board member. This does not alter the authors’ adherence to all the PLOS ONE policies on sharing
data and materials.
* E-mail: hidalgo@[Link]

Introduction evaluative aspects of cities that income based measures are unable
to fully capture.
In ‘‘The Image of The City’’, Kevin Lynch defines the city as a In this paper, we present a high-throughput method to quantify
form of temporal art [1]. Much like sculptures, cities are spatial people’s perception of cities, and their neighborhoods, and use it to
structures, but unlike sculptures, cities are too large to be measure the perceptual inequality of Boston, New York, Linz and
experienced in a single try. Hence, people experience cities Salzburg. The method is based on image ratings created from the
through unique temporal sequences that are reversed, interrupted pairwise comparison of images in response to evaluative questions,
and cut-across from the sequences experienced by others. such as ‘‘Which place looks safer?’’ or ‘‘Which place looks more
Ultimately, in a world in which people’s experiences of urban upper-class?’’ The data shows that the range of perceptions elicited by
environments is unique, this uniqueness can give rise to an images from Boston and NYC is wider than the range of perception
alternative form of inequality, where differences in the experiences elicited by the images of Linz and Salzburg. Finally, we validate our
elicited by different neighborhoods, rather than income, becomes measures of urban perception by studying the correlation between
an important source of interpersonal contrast. urban perception and homicides in New York City, finding a
Neighborhoods often differ in their demographics, such as the significant correlation between violent crime and urban perception
income and ethnicity of the people that inhabits them, but also on after controlling for income, population, area and age.
how safe they feel, how clean they are, how historical they look, We conclude that the method presented in the paper is able to
and how lively they are, among many other evaluative dimensions capture information about a city’s built environment that is
[2]. Certainly, many of these dimensions will correlate with relevant for the experiences of citizens, and not fully contained in
measures of income, but income will not necessarily be a complete income-based measures. Moreover, we conclude that these
proxy for all of them. Because of this, it is important to create measures can be used to estimate the contrasts – or inequality –
measures of cities–and their neighborhoods–that incorporate the of a city’s built environment with respect to these evaluative
dimensions.

PLOS ONE | [Link] 1 July 2013 | Volume 8 | Issue 7 | e68400

 Mapping the Inequality of Urban Perception

Figure 1. Images used in the study. A–D. Locations from which images were collected for: A Boston, B New York City, C Salzburg and D Linz. We
note that for many locations, more than one image was collected (with the camera looking in different directions).
doi:10.1371/[Link].0068400.g001

A tale of two literatures largely to two parallel branches. On the one hand, we have the
Cities, and their neighborhoods, are complex entities that weave literature advanced by urban planners and architects, and on the
together the physical components of the built environment, and other, we have the literature advanced by social scientists and
the social interactions of the citizens that inhabit them. Yet, the natural scientists.
study of cities does not belong to a unified stream of literature, but

Figure 2. Data Collection Methods. A. The website used to collect votes. Participants were presented a random pair of images and voted by
clicking on one in response to the question. B. Robustness of the urban perception metric (Q). B is the square of the Pearson correlation between two
disjoint subsets of votes of size v containing the same number of images.
doi:10.1371/[Link].0068400.g002

PLOS ONE | [Link] 2 July 2013 | Volume 8 | Issue 7 | e68400

 Mapping the Inequality of Urban Perception

Figure 3. Identifying places associated with different urban perceptions. A. High and low scoring images for safety B. social-class and C.
uniqueness. D. Scatter plot of Q-scores for safety and social-class with four examples illustrating images with different combinations of evaluative
criteria. E. Same as D, but for safety and uniqueness. G. Same as D, but for social-class and uniqueness.
doi:10.1371/[Link].0068400.g003

The literature advanced by architects and urban planners puts number of different evaluative dimensions [2]. This study is
special emphasis on a city’s built environment. During the 20th certainly inspired by these measures, which have been based
century, the development of this literature was punctuated by a mostly on visual surveys where people rate images on a 1–10 scale
series of movements, which have resulted in cities combining [2,6–14]. The justification of visual surveys is that urban
different architectural and planning styles [3]. Among the most environments have features, such as the exterior beauty of the
notable of these movements are: the City Beautiful or Civic Art architecture, or the neatness of the shrubbery, that are not traded
movement of Charles Mulford Robinson [4], which emphasizes in the market. Hence, these cannot be inferred from market
the aesthetic aspects of a city’s built environment – think of New mechanisms, such as the price system [2,14–15]. The offline and
York’s Grand Central Station; The Garden City of Ebenezer online studies conducted in the past, however, have lacked the
Howard [5], which proposed a mixture of low density housing throughput required to make comprehensive maps of urban
and parks – much like many modern suburbs; and the Radiant City perception (Table 2s in File S2), and hence, are limited in their
of Le Corbusier [3,6], which reconciled Howard’s Garden City ability to compare a large number of cities and neighborhoods.
with high density buildings – NYC Stuyvesant village being an Within the social sciences, the study of cities has focused mostly
excellent illustration of it. on the connection between demographic and economic variables,
The literature of architects and urban planners has also been with the physical appearance of the built environment playing little
active in the creation of measurements of urban perception along a or no role. The literature advanced by economists, for instance,

PLOS ONE | [Link] 3 July 2013 | Volume 8 | Issue 7 | e68400

 Mapping the Inequality of Urban Perception

Figure 4. Contrasts in urban perception. A. Scatter plot showing the Q-scores obtained for each image, city and question. Top and bottom
whiskers represent one standard deviation. B. Moran’s I z-scores for each city and question (all p-values,0.01, see SM). C. Spatial correlograms
showing the decay of spatial autocorrelation as a function of distance. D–F. Map of NYC showing statistically significant clusters of high -and low- Q-
scores for the perception of safety, class and uniqueness according to Getis Gi* statistic. Green shows clusters of positive perceptions (high Q-scores)
and red shows clusters of negative perceptions (low Q-scores).
doi:10.1371/[Link].0068400.g004

has focused on the creation of mathematical models, such as those examples here include the study of the fractal growth of cities [20–
involved in the new economic geography of Krugman, Fujita and 21] and the study of allometric relations connecting population to
Venables [16–17], or on the establishment of empirical patterns, a number of social and infrastructural variables [22]. Natural
such as the knowledge spillovers documented by Glaeser and scientists have also been keen to develop automated data collection
others [18–19]. methods that use big data to study the statistical properties of
Natural scientists, on the other hand, have a different focus than citizens, such as their human mobility patterns [23–25] and social
economists, but also rely on quantitative methods that do not networks [26–30].
incorporate the aesthetic features of the cities they study. Notable

PLOS ONE | [Link] 4 July 2013 | Volume 8 | Issue 7 | e68400

 Mapping the Inequality of Urban Perception

Table 1. Means and Standard Deviations of the Q-scores obtained for each city and question.

Linz Salzburg Boston NYC Manhattan Queens Brooklyn

Mean Safety 4.85 4.76 4.94 4.47 5.13 4.46 4.23

Unique 4.84 5.04 4.77 4.46 5.21 4.26 4.31
Class 5.01 4.89 4.97 4.31 5.17 4.22 4.06
Standard Safety 0.80 0.88 1.48 1.41 1.25 1.35 1.44
Deviation
Unique 0.93 0.90 1.22 1.18 1.17 1.06 1.16
Class 0.90 0.99 1.62 1.53 1.38 1.39 1.57

doi:10.1371/[Link].0068400.t001

Finally, the most direct connection between these two streams of All of these studies explore the link between people’s perception
literature is the work of Jane Jacobs [31–33] and the Broken of urban environments and social outcomes. Yet, the focus of this
Windows Theory (BWT) of Wilson and Kelling [34]. In ‘‘The literature has been mainly on the association between crime and
Death and Life of Great American Cities’’ [31], Jacobs emphasizes disorder, when this is only one of the many potential associations
the connections she observed between the physical environment of between the urban environment and social outcomes that can be
neighborhoods, and the social interactions between the citizens of interest. In effect, urban landscapes are complex enough to
that inhabited them. ‘‘Death and Life’’ is well cited among demand a number of evaluative dimensions to be characterized
architects and urban planners. Social scientists and economists, on [2], since beyond disorder places can look lively, modern,
the other hand, often build on Jacobs’ later works, including ‘‘The inspiring, classy, abandoned, congested, colorful or beautiful,
Economy of Cities’’ [32] and ‘‘Cities and The Wealth of Nations’’ among other things. These additional dimensions can be used to
[33]. Hence, the literature bridge represented by Jacobs’ work is explore connections between aspects of urban perception and
largely due to her participation in both streams of literatures–and other social dimensions, such as entrepreneurship, civic engage-
unfortunately – does not indicate a clear dialogue between them. ment and high-school completion, among other things. To explore
The Broken Windows Theory (BWT) of Wilson and Kelling these connections, however, we need to extend our quantitative
[34], on the other hand, represents a more direct connection methods of urban perception beyond measures of disorder. In this
between the study of urban forms and social outcomes. In brief, paper, we show that it is possible to capture detailed information
the Broken Windows Theory suggests that evidence of environ- about other evaluative dimensions and show that this information
mental disorder, such as broken windows, litter and graffiti, can can be used to characterize the inequality of cities with respect to
induce other kinds of disorder, like crime, and hence, policies that these dimensions. Finally, inspired by the BWT, we validate the
focus on the amelioration of minor offences can help fight more measures collected by comparing them with data on homicides for
severe forms of criminal activity. NYC.
The BWT has also been politically influential. For instance, it
was cited as a justification for New York City’s quality-of-life Data and Methods
initiative [35–36], an order-maintenance strategy that strictly
Data
enforces minor offenses, such as public drinking and turnstile
We collected data on urban perception by using 4,136 geo-tagged
jumping, as a way to prevent more substantial forms of crime, such
images from four cities (# of images): New York City (1,706) and
as robbery.
Boston (1,236) in the United States; and Salzburg (544) and Linz
Providing evidence to prove or disprove the BWT, however, has (650) in Austria, (Fig. 1A–D). Images from New York City (NYC)
not been easy. In fact, several observational and longitudinal and Boston were sourced digitally from Google Street View while
studies have argued in favor and against of the BWT [35–38]. images from Linz and Salzburg were collected manually onsite. The
Arguments against the BWT point to, among other things, the images and dataset used in the study can be downloaded from
existence of spurious correlations in which underlying environ- ([Link]
mental features, such as liquor stores, can lead to both crime and Perception data was collected using a website created for the
disorder [36]. Arguments in favor of the BWT include experi- study (Fig. 2A). Here users were shown two images, selected
ments, like the ones performed by Keizer et al. [39]. Here the randomly from the dataset, and asked to click on one in response
authors showed that in controlled settings, evidence of disorderly to one of three questions: ‘‘Which place looks safer?’’, ‘‘Which
behavior, such as graffiti or supermarket carts left unattended in place looks more upper-class?’’, or ‘‘Which place looks more
parking garages, were associated with an increase in the unique?’’. Users additionally had the option of indicating that both
probability of people breaking other social norms, such as littering images were perceived as equal. The spatial location of images was
or stealing. not revealed to participants during the study.
In recent years, the BWT has also been linked to health. For We selected the phrasing ‘‘Which place looks more X?’’ because
example, cases of gonorrhea in New Orleans have been shown to it reflected more accurately what could be evaluated from an
correlate more strongly with an index of neighborhood disorder image. We note that similar questions have been asked in
than with an index of neighborhood poverty [40], and residents of preceding evaluative studies (17). 7,872 unique participants from
disadvantaged neighborhoods in Illinois, where noise, graffiti and 91 countries contributed a total of 208,738 votes and self-reported
vandalism are more common, have been found to have worse age and gender (SM and table 1s in File S2).
health outcomes than residents of advantaged neighborhoods, Some limitations of the data include the constrained amount of
even after controlling for individual level disadvantages [41]. information that is captured in an image, since other sensory

PLOS ONE | [Link] 5 July 2013 | Volume 8 | Issue 7 | e68400

 Mapping the Inequality of Urban Perception

Table 2. Comparison between the means and standard deviations of the urban perception recorded for each city and question.

Difference in Means
T-test for equal means with unequal variances.

Safety (p-values)
Salzburg Boston New York Manhattan Queens Brooklyn
Linz 0.0482** 0.1152 0.0000*** 0.0004*** 0.0000*** 0.0000***
Salzburg 0.0015*** 0.0000*** 0.0000*** 0.0000*** 0.0000***
Boston 0.0000*** 0.0201** 0.0000*** 0.0000***
New York 0.0000*** 0.9193 0.0001***
Manhattan 0.0000*** 0.0000***
Queens 0.0028***
Unique (p-values)
Salzburg Boston New York Manhattan Queens Brooklyn
Linz 0.0001*** 0.1547 0.0000*** 0.0000*** 0.0000*** 0.0000***
Salzburg 0.0000*** 0.0000*** 0.0342** 0.0000*** 0.0000***
Boston 0.0000*** 0.0000*** 0.0000*** 0.0000***
New York 0.0000*** 0.0003*** 0.0033***
Manhattan 0.0000*** 0.0000***
Queens 0.4156
Class (p-values)
Salzburg Boston New York Manhattan Queens Brooklyn
Linz 0.0317** 0.4844 0.0000*** 0.0670* 0.0000*** 0.0000***
Salzburg 0.2129 0.0000*** 0.0019*** 0.0000*** 0.0000***
Boston 0.0000*** 0.0291** 0.0000*** 0.0000***
New York 0.0000*** 0.2114 0.0002***
Manhattan 0.0000*** 0.0000***
Queens 0.0535*
Difference in Variances
F-test
Safety (p-values)
Salzburg Boston New York Manhattan Queens Brooklyn
Linz 0.0257** 0.0000*** 0.0000*** 0.0000*** 0.0000*** 0.0000***
Salzburg 0.0000*** 0.0000*** 0.0000*** 0.0000*** 0.0000***
Boston 0.0633** 0.0003*** 0.0216** 0.4562
New York 0.0091*** 0.2913 0.4144
Manhattan 0.1296 0.0034***
Queens 0.1210
Unique (p-values)
Salzburg Boston New York Manhattan Queens Brooklyn
Linz 0.3764 0.0000*** 0.0000*** 0.0000*** 0.0018*** 0.0000***
Salzburg 0.0000*** 0.0000*** 0.0000*** 0.0001*** 0.0000***
Boston 0.2511 0.4196 0.0003*** 0.1611
New York 0.8950 0.0037*** 0.6196
Manhattan 0.0445** 0.8383
Queens 0.0252**
Class (p-values)
Salzburg Boston New York Manhattan Queens Brooklyn
Linz 0.0279** 0.0000*** 0.0000*** 0.0000*** 0.0000*** 0.0000***
Salzburg 0.0000*** 0.0000*** 0.0000*** 0.0000*** 0.0000***
Boston 0.0164** 0.0004*** 0.0000*** 0.3293
New York 0.0257** 0.0113** 0.2980
Manhattan 0.9122 0.0066***

PLOS ONE | [Link] 6 July 2013 | Volume 8 | Issue 7 | e68400

 Mapping the Inequality of Urban Perception

Table 2. Cont.

Difference in Means
T-test for equal means with unequal variances.

Queens66 0.0024***

Significance thresholds * p,0.1 **p,0.05 ***p,0.01.

doi:10.1371/[Link].0068400.t002

channels that can affect perception, such as sound and smell, are represents the maximum possible score for safety, social-class or
absent in pictographic depictions. Also, variation in image quality uniqueness, whereas Q = 0 represents the minimum.
(i.e. contrast, hue, saturation, brightness, tint and clarity), as well as
the time of day, and weather conditions, can introduce additional Robustness of Q
sources of variation in the perceptions associated with a digital We test the inter-rater, or inter-observer reproducibility of Q, by
image. We therefore interpret the urban perception data collected comparing the scores obtained using the same number of images,
through this method as a proxy for the perceptions elicited by the but extracted from non-overlapping subsets of votes of size v. We
actual locations [2]. do this using subsets containing up to 50% of the total votes,
Finally, we note that the mapping between images and locations because it is not possible to construct non-overlapping subsets that
is not one-to-one. In fact, for a large number of locations we are larger than 50% of the original sample. As our measure for
captured more than one image, by pointing the camera in two or inter-rater robustness (B), we use the average R2 of the Pearson
more directions. Hence, many locations are characterized by more correlation between rankings calculated using the same set of
than one quantitative value –usually two. We captured more than images, but a different set of votes. Formally, we define B as:
one image for many locations to take into account the variability of
using images that are not 360-degree representations of a place, 0P 12
but a 90-degree wedge. (Q1i (v){SQ1 (v)T)(Q2i (v){SQ2 (v)T)
B(v)~@ i A ð3Þ
s1 s2
Measures
We scored each image using the fraction of times it got selected
over another image, corrected by the ‘‘win’’ and ‘‘loss’’ ratios of all where Q1(v) and Q2(v) represent two sets of Q-scores calculated
images with which it was compared. This correction allowed us to using disjoint sets of participants of size v, ,. is used to indicate
adjust for the ‘‘strength of schedule’’ [42], since by chance some averages, and s1 and s2 are, respectively, the standard deviations
images were compared with others that were more likely to be of the Q-scores in the sets Q1 and Q2. We note that B is related to
selected favorably in pairwise comparisons. We define the win (W) Cronbach’s aand represents an estimate of the test-retest reliability
and loss (L) ratios of image i with respect to question u as: of the method. A value of B = 100% indicates a perfectly robust
ranking, since it would mean that the exact same set of Q-scores
was obtained by using data collected from different people.
wi,u li,u Figure 2B shows the average B obtained for subsets of different
Wi,u ~ , Li,u ~ ð1Þ
wi,u zli,u zti,u wi,u zli,u zti,u size v (thick line) for each question. We find that the behavior of B
as a function of the sample size v is well approximated by:
where w is the number of times an image was selected over its
paired image, l is the number of times that an image was not
chosen over its paired image, and t is the number of times when an B(v)~(1{exp(Bva ))2 ð4Þ
image was chosen as equal to its paired image. Using this, we
define the Q-score for each image i and question u as: where a and b are fitting parameters (R2 = 99.7% for safety,
R2 = 99.9% for social-class and R2 = 99.9% for uniqueness). We
0 1 use (4) to extrapolate the observed values (thin line Fig 2B) and
nw nl infer the values expected for the totality of our dataset, finding that
10 B 1 Xi
1 X i
C
Qi,u ~ @Wi,u z w Wj u { l Lj u z1A ð2Þ the 93,622 votes collected for the safety question (red square)
3 ni j ~1 1 ni j ~1 2
1 2 results in B = 86.3%, the 70,157 votes available for the social-class
question (blue square) results in B = 84.4%, and the 48,109 votes
where niw is equal to the total number of images i was preferred collected for uniqueness (green square) results in B = 56.0%.
over, nil is equal to the total number of images i was not preferred Finally, we test the internal consistency of the perceptions
over, and where the first sum extends over j1, the images that collected by looking at their transitivity. We find that the overall
image i was preferred over and the second sum extends over j2, level of transitivity of our data is high (86.76% for safety, 87.00%
the images that were preferred over i. for social-class, and 83.34% for uniqueness).
Equation (2) simply corrects an images win ratio (Wi,u) by As a rule of thumb, we find that between 22 and 32 votes per
adding the average win ratio of the images that it was selected over image are needed to produce a ranking with B.75% for each of
and by subtracting the loss ratio of the images that were selected the three questions.
over image i. By doing this, we incorporate information about the One important concern that needs to be addressed here is the
images that were paired together with each image. The numerical possible biases in the measures that might come from the
factors of 10/3 and 1 are used to scale the score to fit the range [0– demographic of participants that joined the online experiment.
10], and come from the theoretical minimum and maximums of To test for this, participants were asked to self-report age and
the analytic expression (2) (see SM). In sum, a score of Q = 10 gender after contributing five clicks. Self-reporting was high, with

PLOS ONE | [Link] 7 July 2013 | Volume 8 | Issue 7 | e68400

 Mapping the Inequality of Urban Perception

97.1% of the participants providing answers for age and gender. ably larger than those for Linz and Salzburg, even when there are
From these, 76.0% identified themselves as male and 21.1% as no significant differences in the mean (for example with the means
female. The median self-reported age was 28 years. Finally, of Linz and Boston for social-class). This suggests that Boston and
participants were geo-located using their IP addresses and the NYC are perceptually more unequal, since the average gap of the
7,872 unique IP addresses were located in 91 countries. evaluative response between images is larger in NYC and Boston
We test the significance of possible biases by comparing the Q- than in Linz and Salzburg. Moreover, we note that the standard
scores estimated using different subsets of participants. We do this deviation measured for NYC is not statistically larger than the one
for participants’ age (above and below the median), gender (male measured for Queens and Brooklyn, when it comes to the
and female), and location (United States vs non-United States). As perception of safety and class.
controls, we show the correlations obtained for random subsets of Next, we study the segregation of urban environments by asking
participants of the same size (Figures 1s, 2s and 3s in File S2). For if the places associated with similar perceptions of safety, social-
example, we compare the correlation of the scores obtained for class and uniqueness co-locate, and if so, to what extent. In
people older and younger than the median age of 28, with the principle, a wider range of values is observed for Boston and NYC,
correlation obtained for two disjoint random half-samples of but these could be spatially intermixed rather than clustered. To
participants. The same procedure was used to create controls for measure the spatial segregation of perceptions we use Moran’s I
the correlations observed between groups of participant with statistic [44]. Values of I range from 21 to 1. A value of 21
different sex and for participants from US and non-US locations, indicates perfect anti-correlation (e.g. a checkerboard), whereas a
as proxied by participants’ IP-addresses. Overall, we find that the value of 1 indicates that similar values are perfectly clustered. The
correlations obtained for groups of different demographics are not null-hypothesis of I is complete spatial randomness and produces
significantly lower than those obtained for the random controls, values near 0. Moran’s I statistic, however, cannot be used directly
indicating that the results of our sample are not driven by biases in to make statistical inferences, since its significance depends on the
age, gender or location of the study’s participants. sample size. Hence, we normalize the Moran I scores for each city
by subtracting the city’s average and dividing it by its standard
Results deviation (creating a z-score). We also control for differences in
sample size by randomly down-sampling the data for Boston,
We begin by asking whether perceptions of safety, class and NYC and Linz to match the 544 points available for Salzburg.
uniqueness are perfectly collinear, or whether they have significant This guarantees that all datasets have the same sample size and
orthogonal components. Figures 3A–3C show typical images ensure that variations are not due to differences in the number of
associated with high and low scores for safety, social-class and points considered.
uniqueness. Places perceived as safe are also more likely to be Figure 4B shows the z-scores associated with Moran’s I for each
perceived as upper-class (Fig. 3D R2 = 68.94%, p-value,0.0001) city and question (see Table 3s in File S2 for p-values). In general
and unique (Fig. 3E R2 = 35.32%, p-value,0.0001), yet, their we find that all cities exhibit positive spatial autocorrelation, with
orthogonal components (1-R2) are relatively large. This allows us Boston and New York having higher z-scores than Linz and
to identify images matching particular combinations of evaluative Salzburg. These results suggest that the American cities studied
criteria, such as images where the perception of safety matches have more segregated neighborhoods than the Austrian cities of
that of social-class (Fig. 3D–I and 3D–III) and where social-class Linz and Salzburg. To explore this further, we measure the length
and safety are inversely related (Fig. 3D–II and 3D–IV). Figure 3F
of the spatial autocorrelation using the autocorrelation function:
shows the analysis for the remaining combination of social-class
and uniqueness (R2 = 37.04%, p-value,0.0001). Together, these
results show that data collected through this method can be used to r){vQw)(Q(~
S(Q(~ rz~
d ){vQw)T
A(D~
d D)~ ð5Þ
identify images satisfying combinations of criteria, and therefore s2
can distinguish between the perceptions of safety, social-class and
uniqueness.
Next, we use Q to measure the contrast or inequality of urban Figure 4C shows the autocorrelation function (5) for each city
perception. We begin this by asking: how wide is the range of and for the three NYC boroughs of Manhattan, Queens and
perceptions elicited by the images of one city vis-a-vis another? Brooklyn. We note that since many locations contain more than
Figure 4A shows the distribution of scores characterizing each one image –images captured with the camera pointing in a
image, for each city and question (values are reported in Table 1). different direction–A(0),1, since this represents the correlation
Here, we see that images in Boston and NYC are distributed over between images captured in the same location but with a different
a wider range of values. Yet, since we have considerably more heading. Finally, we measure the correlation length of each of
images for Boston and NYC, than for Linz and Salzburg, we these using:
compare the standard deviations of these distributions (s), rather
than their range. We do this because the standard deviation of a ~
A~me{nDd D zg ð6Þ
distribution is independent of sample size and provides a good
comparator to measure the dispersion of the Q-scores calculated where m, g and g are fitting parameters. g is included to capture
for each city. Moreover, the distribution of Q-scores for each
the negative correlations observed for large values of D~ dD
question is close to normal (see SM and Figure 4s in File S2).
Table 2 compares the means and standard deviations of each (.5 [km]). To ease interpretation, we define l as the distance D~dD
city and question using, respectively, a t-test to compare the means at which A(D~
d D) = 0. To avoid measurement errors due to binning,
of distributions with different variances, and an F-test. The F-Test we take the average l calculated empirically using a series of bins
allows us to assess whether the difference between the standard ranging from 100 [m] to 1000 [m], for every 100 [m].
deviations of two distributions is significant, after taking into NYC is found to be the city with the largest autocorrelation
consideration their sample size [43]. We find that the standard length, having all l.4.75 [km]. Boston’s mean autocorrelation
deviations of the distribution for Boston and NYC are consider- length for the three questions is l.2.00 [km] whereas Linz and

PLOS ONE | [Link] 8 July 2013 | Volume 8 | Issue 7 | e68400

 Table 3. Getis Spatially Filtered Regression including variables for demographic and urban perception.

Getis Spatially Filtered Regression. Dependent Variable -. Log (Number of Homicides in Zip Code +1)

DEMOGRAPHICS URBAN PERCEPTION

Population and Area Income and Age Safety Class

MODEL 1 Log(Pop)* L_Log Log L_Log Log L_Log Log10 L_Log10 Qsafety* L_Qsafety SQ L_SQ Qclass* L_Qclass SQ L_SQClass
(Pop) (Area)* (Area) (Income)* (Income) (Age)* (Age) safety* safety Class*

PLOS ONE | [Link]

Coefficient 0.262** 0.188* 0.569*** 20.419 20.954*** 20.453 21.559** 214.89**
t-statistic 2.298 2.798 5.416 20.510 24.820 20.345 22.486 22.216
p-value 0.024 0.075 0.000 0.611 0.000 0.731 0.015 0.029
MODEL 2 R2 69.9%
Coefficient 0.868*** 20.599 20.181 20.220 20.181*** 21.033*** 20.109 20.416
t-statistic 6.362 20.453 21.132 21.465 23.655 22.746 21.269 20.745
p-value 0.000 0.651 0.260 0.146 0.000 0.007 0.208 0.458
MODEL 3 R2 47.8%
Coefficient 0.833*** 21.046 20.208 20.260** 20.180*** 20.713** 0.045 0.728
t-statistic 6.115 20.711 21.316 21.742 23.801 22.262 0.053 1.292
p-value 0.000 0.479 0.191 0.085 0.000 0.026 0.600 0.200
MODEL 4 R2 48.3%

9
Coefficient 0.837*** 21.737 20.222 20.257* 20.073 20.856 20.213** 22.480** 20.144 0.118 0.189* 2.560***
t-statistic 5.995 21.171 21.407 21.733 20.681 20.073 21.976 22.535 21.379 0.116 1.732 2.696
p-value 0.000 0.245 0.163 0.086 0.497 0.484 0.051 0.013 0.171 0.908 0.086 0.008
MODEL 5 R2 52.9%
Coefficient 0.392*** 0.347*** 0.481*** 0.936 21.183*** 22.252 21.545*** 222.45*** 20.035 22.717*** 20.210*** 21.103 0.033 3.511*** 0.180** 1.259*
t-statistic 3.172 2.957 4.774 0.803 25.642 21.228 22.686 23.033 20.465 23.089 22.732 21.341 0.444 4.487 2.336 1.917
p-value 0.002 0.004 0.000 0.424 0.000 0.223 0.009 0.003 0.643 0.003 0.008 0.183 0.658 0.000 0.020 0.058

The dependent variable is the logarithm–in base 10–of the number of homicides in a zip code plus one. The plus one was added to include zip codes in which the number of homicides is zero. Significance thresholds are: * p,0.1
**p,0.05 *** p,0.01.
doi:10.1371/[Link].0068400.t003

July 2013 | Volume 8 | Issue 7 | e68400

Mapping the Inequality of Urban Perception
 Mapping the Inequality of Urban Perception

Figure 5. Urban perception and violent crime. A Comparison between the location of crimes in NYC and the predictions of urban perception,
area and population (model [4]). B. Demographics (model [1]). C. All variables (model [5]).
doi:10.1371/[Link].0068400.g005

Salzburg have characteristic lengths of 1.6 [km] or less. This shows the spatial variation has been removed (x*). For each location i,
that locations associated with similar perceptions form larger and variable x, these variables are defined as:
spatial clusters in NYC (Figures 4 D–F) and Boston than in Linz
and Salzburg. Finally, we note that the NYC boroughs of xi Si
Manhattan, Brooklyn and Queens all exhibit strong autocorrela- xi ~ ð7Þ
Gi (n{1)
tion, with lengths only slightly smaller than that of NYC. This
suggests that the measures obtained for NYC also hold for smaller
spatial scales in that city, yet a detailed evaluation of the
association between the segregation of urban perception and city Lxi ~xi {xi ð8Þ
size will require data on a larger number of cities.
where Si = Sjsij is the sum of the spatial weights used to
characterize the spatial proximity between data points (in our
Urban perception and violent crime case 1/distance between locations i and j), n is the number of
Finally, we use homicide data for NYC to look at the correlation locations considered and
between the urban perception of inequality and homicides. We
note from the start that our intention is not to make a causal
statement, but simply to use this correlation to validate the value of P
wij xj
the information contained in our measures of urban perception. j
Because of the spatial nature of the dataset, we use Getis Spatially Gi ~ P for j?I (9)
xj
Filtered Regression (GSFR) [45–46], rather than an Ordinary j
Least Square (OLS) regression. In spatial datasets is not
appropriate to use OLS regressions because of the existence of Finally, a GSFR regression is an OLS regression where each
spatial auto correlations. In other words, the fact that neighboring variable x is replaced by its spatially filtered x* and varying
cells are characterized by similar values violates the independence component Lx. More details about this statistical technique can be
assumption needed to perform an OLS. So, an OLS is only found in [45]. To illustrate what the method doe consider the
justified if the residuals of the OLS regression are NOT spatially income of a zip code. This is a variable that is certainly spatially
auto-correlated. This is because the autocorrelation of the autocorrelated, since rich zipcodes are more likely to locate next to
residuals would indicate the existence of unexplained spatial other rich zipcodes. Instead of incorporating income as a variable,
variation, and therefore, the existence of a missing variable. In a GSFR will incorporate an income* variable, which would be the
statistics, we would say that in this case the model is under- income of a zip code that is not explained by the incomes of
specified. nearby zip codes, and a Lincome variable, that would capture the
GSFRs solve this problem by using a transformation that filters spatial variation of income across zip codes.
out the spatial component of each variable x, into two estimates: Table 3 shows the results of a GSFR where the dependent
one capturing the spatial variation of the variable (Lx), and the variable is the logarithm of the number of homicides in a NYC zip
other capturing the local variation of this variable remaining after code recorded between 2003 and 2011. We note that the Google

PLOS ONE | [Link] 10 July 2013 | Volume 8 | Issue 7 | e68400

 Mapping the Inequality of Urban Perception

Street View API does not provide information for the date and availability of data about urban perception has been limited,
time the images were captured. As explanatory factors we use the and so has our ability to compare cities with respect to them. In
average incomes of households in the zip-code, population, area, this paper, we presented a method to measure urban perception
age and four urban perception variables: the average Q-score for and found that the cities of Boston and NYC differ from the
safety and class (Qsafety, Qclass), and their respective standard Austrian cities of Linz and Salzburg in two important dimensions.
deviations (SQsafety, SQclass) calculated for each zip-code. Formally, First, the perceptions recorded for the cities of Boston and NYC
the regression takes the form: are distributed more broadly than the perceptions elicited by the
images from the two Austrian cities of Linz and Salzburg. Second,
x zB2 ~
log10 (Homicidesz1)~B1~ Lx ze ð10Þ positive and negative perceptions cluster more strongly in the two
American cities, than in their European counterparts. This means
that the recorded gap between ‘‘good’’ and ‘‘bad’’ neighborhoods
Table 3 presents 5 different specification of the statistical model. is larger in NYC and Boston and that both positively evaluated
All models include the population and area of a zip code, since and negatively evaluated images cluster more in these American
these are obvious correlates of crime. Model 1 includes also cities than in their Austrian counterparts. Finally, we showed that
income and age. Model 2 adds the perception of safety, while the inequality of perceptions helps explain the location of violent
model 3 includes the perception of class. Model 4 includes the crime in a NYC zip code, even after controlling for income,
perception of class and safety, but no information on age or population, area and age.
income. Finally, model 5 includes all variables –population, area, As the world gears towards building cities for hundreds of
income, age, average perception of safety, average perception of millions of individuals, the imperative of understanding cities
class, standard deviation in the perception of safety, and standard becomes ever more important [3]. Therefore, there is a strong
deviation in the perception of class. We note that for the full need to create quantitative bridges that can help us link urban
specification of our model (model [5]), we find no spatial perception with other social, political, economic and cultural
correlations among the residuals (Moran’s I z-score = 20.23, p- aspects of cities. In this paper, we present a method that can be
value = 0.82), indicating that the model is not underspecified and used to quantify urban perception and have applied it to the study
can be used for statistical inference. Hence, the results cannot be of a few cities and questions. Although the method offers an
interpreted as the result of a missing variable, such as policing or important improvement in throughput over previous studies, its
race [45–46]. ability to collect data is limited to web traffic and participation.
Model 5 explains nearly 80% of the variation of homicides Because of this, future iterations will need to consider the use of a
across zip codes. This correlation is 10% larger than what is combination of crowdsourcing and machine learning tools to
explained by income, age, population and area alone –from extend the patterns captured by the online participation data to
69.88% (model [1]) to 79.36% (model [5])). The increase is higher resolution and different latitudes. Moreover, future studies
statistically significant (F = 5.3, p-value,1.861025), and indicates might also explore the perceptual biases associated with the
that the measures of urban perception contain information on the measurement technique presented in this paper, as well as support
location of homicides that is not contained in income. the development of techniques that can help identify the features
Overall, we find that in the full model (model [5]), the spatial that determine the evaluative responses recorded. Ultimately, the
components (LQsafety, LQclass), and not the local intensity compo- goal of this study – and those similar to it – is to contribute to our
nents (Qsafety*, Qclass*) are statistically significant meaning that the understanding of the urban environments that we have built, with
spatial variation of urban perception across the city, is what the goal of improving them, and their ability to include their
correlates significantly with the location of homicides. Moreover, citizens, while also informing the construction of future cities.
we find that the local spread of perceptions within a zip-code
(SQclass*, SQsafety*) correlates with the number of homicides. These
results are consistent in the sense that spatial variations for the
Supporting Information
perceptions of safety and class (rather than their absolute values) File S1 Q scores.
correlate with violent crime, after introducing the control (XLS)
variables. A visual comparison of the statistical models presented
in table 3 is presented in figure 5. File S2 Supplementary material.
Finally, we notice that the regression coefficients of the safety (DOCX)
variables are negative (safer looking, less crime), whereas those of
class are positive (classier looking, more crime). As expected, Acknowledgments
coefficients of safety and class are negative when introduced We would like to thank Deepak Jagdish for compiling and organizing the
individually (models [2] and [3]), but the one for class reverse signs dataset before release. We would also thank Kiran Bhattaram, David
when we control for safety (models [4] and [5]). We interpret the Gelvez, Sep Kamvar, Kent Larson, Evan Marshall, Shahar Ronen, Alex
opposite signs of these coefficients as evidence that the orthogonal Simoes, Paul Sawaya, Michael Xu and Michael Wong for their comments
component between class and safety (Figure 3D) carries important and expertise. We acknowledge support from the MIT Media Lab
information, since it indicates that violent crime occurred in places consortia, and the ABC Career Development chair.
that look relatively more upper class after controlling for their
perception of safety. Author Contributions
Conceived and designed the experiments: CAH. Performed the experi-
Conclusions ments: PS. Analyzed the data: CAH PS. Contributed reagents/materials/
analysis tools: CAH PS KS. Wrote the paper: CAH KS PS. Original Idea:
The way a city looks is of central importance for the daily CAH.
experience of billions of city-dwellers. Yet until now, the

PLOS ONE | [Link] 11 July 2013 | Volume 8 | Issue 7 | e68400

 Mapping the Inequality of Urban Perception

References
1. Lynch K (1960) The image of the city (Vol. 1). MIT press. 25. de Montjoye YA, Hidalgo CA, Verleysen M, Blondel VD (2013) Unique in the
2. Nasar JL (1997) The Evaluative Image of the City. Sage Publications. Crowd: The privacy bounds of human mobility. Scientific reports 3.
3. Rybczynski W (2010) Makeshift metropolis: ideas about cities. Scribner. 26. Eagle N, Pentland A, Lazer D (2009) Inferring Social Network Structure using
4. Robinson CM (1909) Modern civic art: or, The city made beautiful. GP Mobile Phone Data, Proceedings of the National Academy of Sciences (PNAS)
Putnam’s sons. 106 (36):15274–15278.
5. Howard E, Osborn FJ (1965) Garden cities of to-morrow (Vol. 23). The MIT 27. Onnela JP, Saramaki J, Hyvonen J, Szabo G, Lazer D, et al. (2007) Structure
Press. and tie strengths in mobile communication networks. Proceedings of the
6. Scott JC (1998) Seeing like a state: How certain schemes to improve the human National Academy of Science 18:7332–7336
condition have failed. Yale University Press. 28. Hidalgo CA, Rodriguez-Sickert C (2008) The dynamics of a mobile phone
7. Devlin K, Nasar JL (1989) The Beauty and the Beast, Journal of Environmental network. Physica A: Statistical Mechanics and its Applications, 387(12):3017–
Psychology 9:333–344. 3024.
8. Peterson GL (1967) A Model of Preference, J Regional Sci 7:19–31. 29. Palla G, Barabasi AL, Vicsek T (2007) Quantifying social group evolution,
9. Schroeder HW, Anderson LM (1984) Perception of Personal Safety in Urban Nature 446 (7136): 664–667.
Recreation Sites, Journal of Leisure Research 16:178–194. 30. Eagle N, Macy M, Claxton R (2010) Network Diversity and Economic
10. Herzog TR, Kaplan S, Kaplan R (1976) The Prediction of Preference for Development, Science 328(5981):1029–1031.
Familiar Urban Places, Environment and Behavior 8:627–645. 31. Jacobs J (1961) The death and life of great American cities. Vintage.
32. Jacobs J (1970) The economy of cities. The economy of cities.
11. Roth M (2005) Online Visual Landscape Assessment Using Internet Survey
33. Jacobs J (1985) Cities and the wealth of nations: Principles of economic life. New
Techniques Trends in online landscape architecture. Proceedings at Anhalt
York: Vintage Books.
University of Applied Sciences, 121–130.
34. Kelling GL, Wilson JQ (1982) Broken windows. Atlantic monthly, 249(3):29–38.
12. Wherrett JR (2010) Creating Landscape Preference Models Using Internet
35. Bratton W, Kelling G (2006) There are no cracks in the broken windows.
Survey Techniques, Landscape Research 25:79–96.
National Review, 28.
13. Milgram S (1976) Psychological maps of Paris. Environmental psychology:
36. Harcourt BE (1998) Reflecting on the Subject: a Critique of the Social Influence
People and their physical settings, 104–124.
Conception of Deterrence, the Broken Windows Theory, and Order-
14. Wilson RL (1962) Livability of the city: attitudes and urban development. Urban Maintenance Policing New York Style, Michigan Law Review 97:291–389.
Growth Dynamics, 359–399. 37. Harcourt BE, Ludwig J (2006) Broken windows: New evidence from New York
15. Chapin FS, Weiss SF (1966) Urban Growth Dynamics in a Regional Cluster of City and a five-city social experiment. The University of Chicago Law Review:
Cities. 271–320.
16. Krugman P (1998) What’s new about the new economic geography?. Oxford 38. Jean PKS (2007) Pockets of crime: Broken windows, collective efficacy, and the criminal point
review of economic policy, 14(2):7–17. of view. University of Chicago Press.
17. Fujita M, Krugman P (2003) The new economic geography: Past, present and 39. Keizer K, Lindenberg S, Steg L (2008) The spreading of disorder. Science,
the future. Papers in regional science 83(1):139–164. 322(5908):1681–1685
18. Glaeser EL, Kallal HD, Scheinkman JA, Shleifer A (1992) Growth in Cities. 40. Cohen D, Spear S, Scribner R, Kissinger P, Mason K, Wildgen J (2000)
Journal of Political Economy, 100(6). ‘‘Broken windows’’ and the risk of gonorrhea. American Journal of Public
19. Ellison G, Glaeser EL (1997) Geographic Concentration in US Manufacturing Health, 90(2):230.
Industries: A Dartboard Approach. Journal of Political Economy, 105(5):889– 41. Ross CE, Mirowsky J (2001) Neighborhood disadvantage, disorder, and health.
927. Journal of health and social behavior, 258–276.
20. Batty M (2007) Cities and complexity: understanding cities with cellular automata, agent- 42. Park J, Newman ME (2005) A network-based ranking system for US college
based models, and fractals. The MIT press. football. Journal of Statistical Mechanics: Theory and Experiment,
21. Batty M, Longley PA (1994) Fractal cities: a geometry of form and function. Academic 2005(10):10014.
Press. 43. Lomax RG (2007) An introduction to statistical concepts.
22. Bettencourt LMA, Lobo J, Helbing D, Kuhnert C, West GB (2007) Growth, 44. Moran PA (1950) Notes on continuous stochastic phenomena. Biometrika, 37(1/
innovation, scaling, and the pace of life in cities. Proceedings of the National 2):17–23.
Academy of Sciences (PNAS) 104(17):7301–7306. 45. Getis A (1990) Screening for spatial dependence in regression analysis. In Papers
23. González MC, Hidalgo CA, Barabási AL (2008) Understanding Individual of the Regional Science Association 69(1):69–81 Springer-Verlag.
Human Mobility Patterns. Nature 435:779–782. 46. Anselin L, Getis A (2010) Spatial statistical analysis and geographic information
24. Brockmann D, Hufnagel L, Geisel T (2006) The scaling laws of human travel, systems. In Perspectives on Spatial Data Analysis 35–47. Springer Berlin
Nature 439(7075):462–465. Heidelberg.

PLOS ONE | [Link] 12 July 2013 | Volume 8 | Issue 7 | e68400