Simple Exposure Analysis

COMP3020 Social Web Analytics

School of Computing, Data and Mathematical Sciences

Module 3
 Outline

1 Facebook Insights

2 Reach and Demographics

Confidence interval for proportion

Chi-squared test for independence

3 Key Metrics

Social Web Analytics Simple Exposure Analysis 2 / 52

 Outline

1 Facebook Insights

2 Reach and Demographics

Confidence interval for proportion

Chi-squared test for independence

3 Key Metrics

Social Web Analytics 3 / 52

 Facebook Insights

If you own a facebook page and you have 30 “likes” then you get access to
facebook insights.
Facebook believes these Insights are useful - particularly to business.
The School of Computing, Engineering and Mathematics has a facebook page.

Social Web Analytics 4 / 52

 Likes, Reach and Talking About
Facebook Insights records data (over a fixed period) relating to the number of
likes, the reach and the number “Talking About”
This can be at the page or post level

Likes are just the number of unique people who click the like button on a
page or post.
Reach is the number of unique people who might have seen a page or post.
It includes likes, but also includes people who have seen it because it was
shared etc.
“Talking about” means actively interacting with a page or post. That is
likes, comments, tags or shares etc.

These measures can be obtained for various time periods, daily, weekly, 28-day,
as well as a cumulative total.
The measures are only available for pages that 30 or more of that measure.
Social Web Analytics 5 / 52
 Business Questions

The sort of things an organisation might be interested in include.

What is the Reach of our facebook presence?

What are the demographics of that Reach — are we getting to our target
audience?
What impacts have changes to our page/posts/presence made on Reach?

Facebook provide graphical visualisations of this data. We will look at

replicating this, and doing some simple statistical analysis.

Social Web Analytics 6 / 52

 Facebook Insights

Social Web Analytics 7 / 52

 Facebook Insights

The overview screen shows the Actions, Views, Likes, Reach,

Engagements, Videos, Followers of the Page.
The figures at the top are current cumulative activity number for the last
28 days.
The graph shows for 28 days period ending on a particular day, plotted
against that day.

This is all for the past month.

(There is a table of per post info also)

Social Web Analytics 8 / 52

 Facebook Insights

Social Web Analytics 9 / 52

 Facebook Insights

On the second screen the Likes are broken down by several demographic
factors

Gender, Age, Country, Town, and Language.

The graph is a bar plot of proportions by gender and age.

You can choose the time period that this covers, up to a 92 day period.
You can also Export the data for 180 days.

Social Web Analytics 10 / 52

 Export Data

The small button to the right of this second screen allows us to export data.

Data can be exported at a page or post level.

Dates can be chosen up to 180 days long.
XLS or CSV format can be used. XLS contains more information.

We will look at Page level, XLS data.

As exported by Facebook there are sometimes problems with this file, so open it
in Excel and re-save as an XLSX file.

Social Web Analytics 11 / 52

 Export Data
R has some facilities to read XLSX files. The readxl library is one. (This will
need to be installed)
library(readxl, quietly=TRUE)

sheets=excel_sheets(”Facebook Insights Data Export [Link]”)

length(sheets)
## [1] 38

sheets

## [1] ”Key metrics” ”Daily Like sources”

## [3] ”Daily Viral Reach by story type” ”Weekly Viral Reach by story...”
## [5] ”28 days Viral Reach by story...” ”Daily Viral Impressions by s...”
## [7] ”Weekly Viral Impressions by...” ”28 days Viral Impressions by...”
## [9] ”Daily Total frequency distri...” ”Weekly Total frequency distr...”
## [11] ”28 days Total frequency dist...” ”Daily Page posts frequency d...”
## [13] ”Weekly Page posts frequency...” ”28 days Page posts frequency...”
## [15] ”Daily Viral frequency distri...” ”Weekly Viral frequency distr...”
## [17] ”28 days Viral frequency dist...” ”Daily Talking About This by...”
## [19] ”Weekly Talking About This by...” ”28 days Talking About This b...”

Social Web Analytics 12 / 52

 Export Data
The XLSX file contains 65 sheets. We are primarily going to look at “Key
Metrics” and “Weekly Reach by Demographics”
(These are saved as CSV for ease of use, but…)

keyMetrics <- [Link](”Facebook Insights Data Export [Link]”, 1)

WeekReach <- [Link](”Facebook Insights Data Export [Link]”, 34)

keyMetrics <- [Link](”[Link]”, [Link]=TRUE)

dim(keyMetrics)

## [1] 161 90

WeekReach <- [Link](”[Link]”)

dim(WeekReach)

## [1] 160 21

The “[Link]=TRUE” prevents the key metrics being treated as factors. (see later)
Social Web Analytics 13 / 52
 Outline

1 Facebook Insights

2 Reach and Demographics

Confidence interval for proportion

Chi-squared test for independence

3 Key Metrics

Social Web Analytics 14 / 52

 Demographics

Demographics refers to the characteristics of individuals. In Facebook, the

most useful demographics are Age group and Gender. City, Country and
Language are also recorded.
Several sheets in the XLSX file have breakdown by demographics.
We will look at “Weekly Reach Demographics”

Social Web Analytics 15 / 52

 Reach and Demographics

Reading in the CSV file gives the following. At the moment, we are interested in
a particular (recent) date.

xx <- WeekReach[158,]
print(xx)

## Description Date F.13.17 F.18.24 F.25.34 F.35.44 F.45.54 F.55.64

## 158 8/5/13 1 45 15 6 2 3
## F.65. M.13.17 M.18.24 M.25.34 M.35.44 M.45.54 M.55.64 M.65. U.18.24
## 158 2 2 121 31 7 4 2 3 1
## U.25.34 U.35.44 U.45.54 U.65.
## 158 NA 1 1 NA

Social Web Analytics 16 / 52

 Reach and Demographics

First we make this into a table (matrix) after discarding the Date, and “U”
categories. And we set up meaningful row and columns names

tab <- matrix([Link](xx[3:16]), nrow=2, byrow=TRUE)

colnames(tab) <- c(”13-17”, ”18-24”, ”25-34”, ”35-44”,
”45-54”, ”55-64”, ”65+”)
rownames(tab) <- c(”Female”,”Male”)
print(tab)

## 13-17 18-24 25-34 35-44 45-54 55-64 65+

## Female 1 45 15 6 2 3 2
## Male 2 121 31 7 4 2 3

Social Web Analytics 17 / 52

 Reach and Demographics
So we can draw a graph similar to the Facebook one (bar plot)
barplot(tab, legend=TRUE, col=c(”pink”,”lightblue”))
150

Male
Female
100
50
0

13−17 18−24 25−34 35−44 45−54 55−64 65+

Social Web Analytics 18 / 52
 Reach and Demographics
However, side by side bars are sometimes easier to compare.
barplot(tab, legend=TRUE, col=c(”pink”,”lightblue”),
beside=TRUE)
120

Female
100

Male
80
60
40
20
0

13−17 18−24 25−34 35−44 45−54 55−64 65+

Social Web Analytics 19 / 52
 Questions of interest.

Business (or page owners) might be interested to know…

What proportion of Reach is … eg. Male or 18 to 24? If the page, represents

a product or service you might be interested in whether you are reaching
Males more than Females?
Is this proportion changing? (see later lecture)
Are the age profiles different for males and females? Is there something
different about the age profiles that you are reaching?

These are statistical questions…provided we are prepared to assume that the

reach is a random sample of all possible Reach. (Discuss?)

Social Web Analytics 20 / 52

 Outline

1 Facebook Insights

2 Reach and Demographics

Confidence interval for proportion

Chi-squared test for independence

3 Key Metrics

Social Web Analytics 21 / 52

 Proportions

The simple (point) estimate of the proportion in category XYZ, is simply the
number in that category divided by the total.

Number of males (2 + 121 + 31 + 7 + 4 + 2 + 3) = 170

Number of females (1 + 45 + 15 + 6 + 2 + 3 + 2) = 74
Total = 170+74 = 244

Therefore, our estimate of the proportion of males is p̂ = 170/244 = 0.697 or

69.7%

Social Web Analytics 22 / 52

 Proportions

The simple (point) estimate of the proportion in category XYZ, is simply the
number in that category divided by the total.

Number of males (2 + 121 + 31 + 7 + 4 + 2 + 3) = 170

Number of females (1 + 45 + 15 + 6 + 2 + 3 + 2) = 74,
Total = 170+74 = 244

Therefore, our estimate of the proportion of males is p̂ = 170/244 = 0.697 or

69.7%
But note that this is an estimate of the true proportion p. Providing only the
estimate p̂ does not tell us how different p̂ is to p.

Social Web Analytics 22 / 52

 Variance in sample proportions

Example
We want to estimate the proportion of 1s rolled when using a six sided dice. In
this experiment, we know that p = 1/6 (assuming the dice is fair). After rolling
the dice n = 10 times, we obtain the sample:

6, 1, 1, 2, 6, 3, 3, 3, 3, 5

Giving p̂ = 0.2.

Social Web Analytics 23 / 52

 Variance in sample proportions

Example
We want to estimate the proportion of 1s rolled when using a six sided dice. In
this experiment, we know that p = 1/6 (assuming the dice is fair). After rolling
the dice n = 10 times, we obtain the sample:

6, 1, 1, 2, 6, 3, 3, 3, 3, 5

Giving p̂ = 0.2.
After rolling the dice another 10 times:

4, 2, 5, 4, 3, 6, 4, 2, 6, 1

Giving p̂ = 0.1.

Social Web Analytics 23 / 52

 Sample proportion distribution
If we repeat the experiment 1000 times, we get the following distribution. If
p = 1/6, we can get values of p̂ from 0 to 0.5, when n = 10.

300

200
count

100

0
0.00 0.25 0.50 0.75 1.00
sample proportion

Social Web Analytics 24 / 52

 Confidence Interval for Proportion
We just showed that if we know the proportion p, we can estimate the variation
of p̂. But when we obtain a sample, we have p̂ (not p), so we have the reverse
question.
Confidence interval for p
Giving a sample proportion p̂ and sample size n, what range of values could p
take, with probability 1 − α?

Social Web Analytics 25 / 52

 Confidence Interval for Proportion
We just showed that if we know the proportion p, we can estimate the variation
of p̂. But when we obtain a sample, we have p̂ (not p), so we have the reverse
question.
Confidence interval for p
Giving a sample proportion p̂ and sample size n, what range of values could p
take, with probability 1 − α?

For a 95% confidence interval:

We must estimate the variation of p̂

Using bootstrapping, we sample with replacement from the original
sample and compute the bootstrap statistic p̂b
Repeat the process many times (at least 1000) to obtain a distribution of
the bootstrap statistic.
The confidence interval for p is the middle 95% of the bootstrap
distribution.
Social Web Analytics 25 / 52
 Confidence interval for proportion of males
The sample proportion of males p̂ = 0.697 with n = 244.
bootDist = replicate(1000, mean(sample(c(”M”,”F”), size = 244,
prob = c(0.697, 0.303), replace = TRUE) == ”M”))

150

100
count

50

0
0.60 0.65 0.70 0.75 0.80
Bootstrap distribution of proportion of males
Social Web Analytics 26 / 52
 Confidence interval for proportion of males
The 95% confidence interval is middle 95% of the bootstrap distribution:
lower = quantile(bootDist, 0.025)
upper = quantile(bootDist, 0.975)
print (c(lower, upper))

## 2.5% 97.5%
## 0.6434426 0.7581967

Therefore, we are 95% confident that the proportion of males is between

0.6434426 and 0.7581967.

Social Web Analytics 27 / 52

 Confidence interval for proportion of males
The 95% confidence interval is middle 95% of the bootstrap distribution:
lower = quantile(bootDist, 0.025)
upper = quantile(bootDist, 0.975)
print (c(lower, upper))

## 2.5% 97.5%
## 0.6434426 0.7581967

Therefore, we are 95% confident that the proportion of males is between

0.6434426 and 0.7581967.

The 90% confidence interval is middle 90% of the bootstrap distribution:

lower = quantile(bootDist, 0.05)
upper = quantile(bootDist, 0.95)
print (c(lower, upper))

## 5% 95%
## 0.6516393 0.7459016

Social Web Analytics 27 / 52

 Problem: confidence intervals
Problem
Estimate the 95% confidence interval from the following bootstrap
distributions.

14
20

8
10

4
0

0
20 30 40 50 60 70 4 6 8 12 16
80

20
40

10
0

0 2000 6000 1 2 3 4 5 6
Social Web Analytics 28 / 52
 Outline

1 Facebook Insights

2 Reach and Demographics

Confidence interval for proportion

Chi-squared test for independence

3 Key Metrics

Social Web Analytics 29 / 52

 Age profiles

Question
Are the age profiles of the males and females that we are reaching different?
That is, neglecting the the difference in overall number is the spread across
ages different?

print(tab)

## 13-17 18-24 25-34 35-44 45-54 55-64 65+

## Female 1 45 15 6 2 3 2
## Male 2 121 31 7 4 2 3

Is there evidence, that the age profiles of genders in our audience differs?
This is a hypothesis test. We want to test if the two variables are independent.

Social Web Analytics 30 / 52

 Independent events
Example
The outcomes of tossing a coin and rolling a dice are independent. Here are the
results from tossing a coin and rolling a dice 100 times.

Dice
1 2 3 4 5 6
Head 12 8 6 9 6 4
Tail 8 9 8 10 10 10

If we ran the experiment again, we would get different numbers, but the two
variables are still independent.

Social Web Analytics 31 / 52

 Independent events
Example
The outcomes of tossing a coin and rolling a dice are independent. Here are the
results from tossing a coin and rolling a dice 100 times.

Dice
1 2 3 4 5 6
Head 12 8 6 9 6 4
Tail 8 9 8 10 10 10

If we ran the experiment again, we would get different numbers, but the two
variables are still independent.
We know that the dice roll and coin toss are independent. For another
experiment where we are unsure of the dependence, how different do the
numbers have to be for us to have confidence that the variables are dependent?

Social Web Analytics 31 / 52

 Chi-squared statistic
The χ2 (Chi-squared) statistic can be used to summarise the similarity of a
table to the expected “independent” table.
∑ ∑ (Xij − npi qj )2
χ2 =
npi qj
i j

Xij as the count of the ith row, jth column,

n as the sample size (sum of all counts),
pi as the expected proportion of ith row and
qj as the expected proportion of the jth column.
npi qj is the expected count in cell ij, assuming independence between
rows and columns.

If all Xij are equal to the expected npi qj , χ2 = 0. The more different Xij is to
npi qj , the greater χ2 .
Social Web Analytics 32 / 52
 Chi-squared statistic of Dice and Coin sample
Example
Dice
1 2 3 4 5 6
Head 12 8 6 9 6 4
Tail 8 9 8 10 10 10

pi for each i is 0.45, 0.55

qj for each j is 0.20, 0.17, 0.14, 0.19, 0.16, 0.14
n = 100
Giving χ2 = 3.81

Social Web Analytics 33 / 52

 Chi-squared statistic of Dice and Coin sample
Example
Dice
1 2 3 4 5 6
Head 12 8 6 9 6 4
Tail 8 9 8 10 10 10

pi for each i is 0.45, 0.55

qj for each j is 0.20, 0.17, 0.14, 0.19, 0.16, 0.14
n = 100
Giving χ2 = 3.81
We can compute the χ2 value of our sample and obtain a number. If the number
is large, then the sample table is not similar to the expected independent table,
so the sample is probably not independent.

Social Web Analytics 33 / 52

 Chi-squared statistic of Dice and Coin sample
Example
Dice
1 2 3 4 5 6
Head 12 8 6 9 6 4
Tail 8 9 8 10 10 10

pi for each i is 0.45, 0.55

qj for each j is 0.20, 0.17, 0.14, 0.19, 0.16, 0.14
n = 100
Giving χ2 = 3.81
We can compute the χ2 value of our sample and obtain a number. If the number
is large, then the sample table is not similar to the expected independent table,
so the sample is probably not independent.
But how “large” is large? Is 3.81 large enough?
Social Web Analytics 33 / 52
 Examining the χ2 distribution for independent
tables.

To determine what “large enough” means for the χ2 statistic, we must examine
the χ2 statistic for tables where the random variables are independent. If our
χ2 statistic is larger than those, then our table variables are probably not
independent.
How do we observe other samples with independent rows and columns?

Social Web Analytics 34 / 52

 Examining the χ2 distribution for independent
tables.

To determine what “large enough” means for the χ2 statistic, we must examine
the χ2 statistic for tables where the random variables are independent. If our
χ2 statistic is larger than those, then our table variables are probably not
independent.
How do we observe other samples with independent rows and columns?
We use randomisation of our table.

Social Web Analytics 34 / 52

 Randomisation to break dependence

Example (continued)
If two random events are independent, we can independently shuffle the order
of their outcomes without effecting the probability of their joint outcome.

Dice 5 2 2 1 5 1 3 6 4 6
Coin T H H H H H H H H H

Social Web Analytics 35 / 52

 Randomisation to break dependence

Example (continued)
If two random events are independent, we can independently shuffle the order
of their outcomes without effecting the probability of their joint outcome.

Dice 5 2 2 1 5 1 3 6 4 6
Coin T H H H H H H H H H
↓ Random shuffle ↓
Dice 1 6 2 5 2 6 3 1 4 5
Coin H H H H H T H H H H

Social Web Analytics 35 / 52

 Randomisation to break dependence

Example (continued)
If two random events are independent, we can independently shuffle the order
of their outcomes without effecting the probability of their joint outcome.

Dice 5 2 2 1 5 1 3 6 4 6
Coin T H H H H H H H H H
↓ Random shuffle ↓
Dice 1 6 2 5 2 6 3 1 4 5
Coin H H H H H T H H H H

Note that the above two outcomes are not equivalent if dependence exists
between the coin toss and dice roll outcomes.

Social Web Analytics 35 / 52

 Observing independent samples

By repeatedly using random shuffling and computing the χ2 statistic of the new
table, we can observe what the dice vs. coin χ2 statistic will look like when the
two are independent.
We now ask “what is the probability that we could obtain our χ2 statistic or
greater, given that the coin and dice are independent?”

If the probability is low, then the dice and coin are probably not
independent (since the shuffled tables are independent).
If the probability is high, then we can’t say anything. We can assume that
they are independent.

Social Web Analytics 36 / 52

 Computing the p-value
Example (continued)
expectedIndependent = function(X) {
n = sum(X)
p = rowSums(X)/sum(X)
q = colSums(X)/sum(X)
return(p %o% q * n) # outer product creates table
}

chiSquaredStatistic = function(X, E) {
return(sum((X - E)^2/E))
}

E = expectedIndependent(X) # compute expected counts if independent

x2 = replicate(1000, { # compute 1000 randomised chi-squared statistics

diceShuffle = sample(dice)
coinShuffle = sample(coin)
Xindep = table(coinShuffle, diceShuffle)
chiSquaredStatistic(Xindep, E)
})
Social Web Analytics 37 / 52
 Chi-squared distribution

60
count

40

20

0
0 5 10 15 20
chi−squared statistics

Figure: Randomisation distribution of the dice vs. coin chi-squared statistic. Blue
region is the set of χ2 values that are greater than our sample χ2 statistic (3.81).
Social Web Analytics 38 / 52
 Problem: χ2 test
Problem
Our sample produces a χ2 statistic of 15. Determine if our sample does not
belong to the following χ2 distributions.
20

20
10
0

0
0 5 10 15 20 10 20 30 40
25

20
10
10
0

0 5 10 15 30 40 50 60 70
Social Web Analytics 39 / 52
 Hypothesis test
We just performed a Hypothesis test on the dice vs. coin table.

H0 : The dice and coin outcomes are independent.

HA : The dice and coin outcomes are not independent.

The p-value for the test is the proportion of blue in the previous histogram
(0.568).
Since the p-value is high, we cannot reject H0 .

Social Web Analytics 40 / 52

 Hypothesis test
We just performed a Hypothesis test on the dice vs. coin table.

H0 : The dice and coin outcomes are independent.

HA : The dice and coin outcomes are not independent.

The p-value for the test is the proportion of blue in the previous histogram
(0.568).
Since the p-value is high, we cannot reject H0 .
We can also compute the p-value using the R function:
[Link](X, [Link] = TRUE)

##
## Pearson’s Chi-squared test with simulated p-value (based on 2000
## replicates)
##
## data: X
## X-squared = 3.8067, df = NA, p-value = 0.5967

Social Web Analytics 40 / 52

 Age profiles: Hypothesis test for independence
We now continue our analysis of age vs. gender.

## 13-17 18-24 25-34 35-44 45-54 55-64 65+

## Female 1 45 15 6 2 3 2
## Male 2 121 31 7 4 2 3

If we assume (H0 ) that age and gender are independent (we get the same
distribution of ages no matter what gender we observe), the expected
frequencies are (npi qj )

## 13-17 18-24 25-34 35-44 45-54 55-64 65+

## Female 0.9098361 50.34426 13.95082 3.942623 1.819672 1.516393 1.516393
## Male 2.0901639 115.65574 32.04918 9.057377 4.180328 3.483607 3.483607

Giving us a χ2 value of 4.8116605.

Is the difference between the observed frequencies and expected frequencies
large enough for us to say that the age and gender are not independent?
Social Web Analytics 41 / 52
 Age profiles: randomisation distribution
Given the below distribution of χ2 if gender and age are independent, and our
sample having χ2 = 4.8116605, can we say that gender and age are not
independent?

60

40
count

20

0
0 5 10 15 20
chi−squared statistics
Social Web Analytics 42 / 52
 Age profiles: randomisation distribution
The blue region shows a p-value of 0.614, so we cannot say that gender and age
are not independent (cannot reject H0 ).

60

40
count

20

0
0 5 10 15 20
chi−squared statistics

Social Web Analytics 43 / 52

 Other uses of the χ2 test

We computed the χ2 distribution where the expected frequencies were those

generated using npiqj. This was to test if the two variables (age and gender) are
independent.
Note that if you are not using a randomisation distribution for a χ2 test (e.g.
using built in function in R [Link](), then expected count for each cell
must be greater than 5.
Note that we can test for any set of expected frequencies. If we want to test if
the proportion of males is 60% and females is 40% and the sample size is n, the
expected number of males is 0.6n and the expected number of females is 0.4n.
We will go through a problem concerning this in the lab.

Social Web Analytics 44 / 52

 Outline

1 Facebook Insights

2 Reach and Demographics

Confidence interval for proportion

Chi-squared test for independence

3 Key Metrics

Social Web Analytics 45 / 52

 Key Metrics
The Key Metrics data has a header row, and a second row that is a description.
There are 90 columns: 1,15,18 and 24 are;

Date The Date!

Daily Total Reach Daily number of people who have seen any content
associated with your Page. (Unique Users)
Daily Organic Reach Daily number of people who visited your Page, or
saw your Page or one of its posts in News Feed or ticker. These can be
people who have liked your Page and people who haven’t. (Unique Users)
Daily Viral Reach Daily number of people who saw your Page or one of
its posts from a story shared by a Friend. These stories include liking your
Page, posting to your Page’s Timeline, liking, commenting on or sharing
one of your Page posts, answering a question you posted, responding to
one of your events, mentioning your Page, tagging your Page in a photo or
checking in at your location. (Unique Users)

Social Web Analytics 46 / 52

 Key Metrics

dates <- keyMetrics[,1]

dates <- dates[-1]
dates <- strptime(dates, format=”%m/%d/%y”)
reach <- keyMetrics[,15]
reach[1]

## [1] ”Daily The number of people who have seen any content associated with your

reach <- [Link](reach[-1])

Social Web Analytics 47 / 52

 Key Metrics
plot(dates, reach, type=”l”)

200
150
reach

100
50
0

Mar Apr May Jun Jul Aug

dates
Social Web Analytics 48 / 52
 Key Metrics
keep <- (dates > [Link](”2013-07-01”))
plot(dates[keep], reach[keep], type=”l”)
150
reach[keep]

100
50
0

Jul 04 Jul 14 Jul 24 Aug 03

Social Web Analytics 49 / 52

 Weekly Total Reach
wreach <- [Link](keyMetrics[-1,16])
plot(dates[keep], wreach[keep], type=”l”)
250
200
wreach[keep]

150
100
50

Jul 04 Jul 14 Jul 24 Aug 03

Social Web Analytics 50 / 52

 Conclusion

So by exporting the data we can do a lot more than the usual insights provide

Testing of Reach demographics

Estimating features of Reach/Likes etc
Graphing different parameters
Graphing over different time periods

Social Web Analytics 51 / 52

 Next week

Text Mining 1: Indexing and Querying Text

Social Web Analytics 52 / 52