Agenda

• Properties of data
• The fact-based data model
• Benefits of a fact-based model for Big
Data
• Graph schemas

Presented by : [Link]
 The properties of data
• Information Dependency: means that some pieces of information can
be derived from others but not vice versa.
• This creates a one-way relationship, where more detailed data can
generate summarized views, but summarized views cannot be expanded
back into detailed data
• Example: Friendship Information in FaceSpace

Presented by : [Link]
 The properties of data
■ Data refers to the information that can’t be derived from anything else.
Data serves as the axioms from which everything else derives.
■ Queries are questions you ask of your data. For example, you query
your financial transaction history to determine your current bank
account balance.
■ Views are information that has been derived from your base data. They
are built to assist with answering specific types of queries

Presented by : [Link]
 The properties of data
• one person’s data can be another’s view

Presented by : [Link]
 The properties of data
Data vs. Views: Perspective Matters
A key concept in data management is that the classification of data depends
on perspective.
Example: Public vs. Private Data
• FaceSpace has access to Tom’s full birthdate (March 13, 1984).
• Tom only allows the public to see his birthday (March 13).
• To FaceSpace, the birthday is a view because it’s derived from the birthdate.
• To an advertiser scraping public profiles, the birthday is raw data since
they have no access to the full date.
Why does this matter?
• Organizations must be aware of what counts as data vs. a derived view when
managing access and security.
• Different users and systems may interpret the same information differently
• Introduce the key properties of data: rawness, immutability, and perpetuity (or the
“eternal trueness of data”).

Presented by : [Link]
 The properties of data
Data vs. Views: Perspective Matters
A key concept in data management is that the classification of data depends
on perspective.
Example: Public vs. Private Data
• FaceSpace has access to Tom’s full birthdate (March 13, 1984).
• Tom only allows the public to see his birthday (March 13).
• To FaceSpace, the birthday is a view because it’s derived from the birthdate.
• To an advertiser scraping public profiles, the birthday is raw data since
they have no access to the full date.
Why does this matter?
• Organizations must be aware of what counts as data vs. a derived view when
managing access and security.
• Different users and systems may interpret the same information differently
• Introduce the key properties of data: rawness, immutability, and perpetuity (or the
“eternal trueness of data”).

Presented by : [Link]
 The properties of data
1 Data is Raw
▪ Definition of Raw Data
▪ Raw data is unprocessed and retains its original form.
▪ The more raw the data, the more questions can be asked of
it.
▪ Importance of Raw Data in Big Data Systems
▪ A Big Data system is designed to answer questions based on
past information.
▪ Storing raw data enables answering unexpected future
questions.
▪ Example: FaceSpace Data vs. Advertiser Data
▪ FaceSpace data is more valuable because it can infer more
details about users like Tom.
▪ Raw data helps extract deeper insights compared to
summarized data. Presented by : [Link]
 The properties of data
1 Data is Raw
Example: Stock Market Data
o Stock prices are typically recorded using daily summaries (Open,
High, Low, Close).
o Summarized data may hide key details about market
movements.
• Granular Data Provides More Insights
o Storing data at finer time intervals (e.g., minute-by-minute) reveals
cause-effect relationships.
o More detailed data allows deeper analysis of trends and
competitor impact.
• More Raw Data = More Flexibility
o Keeping raw data allows for new insights over time.
o Summarizing, overwriting, or deleting data limits its usefulness.
o Big Data systems handle massive storage (petabytes, exabytes)
while allowing direct queries. Presented by : [Link]
 The properties of data
1 Data is Raw
• Unstructured vs. Normalized Data
• Definition of Semantic Normalization
o It’s the process of reshaping unstructured data into a structured
format.
o Example: FaceSpace converting "SF" into "San Francisco, CA."
• Why Store Unstructured Data?
o Normalization algorithms can improve over time.
o Keeping raw (unstructured) data allows reprocessing as
algorithms evolve.
o Example: "North Beach" might not be recognized today, but it can
be corrected later.
• When to Store Unstructured Data
• If an extraction algorithm is simple and accurate, store the
processed result.
• If the algorithm may change or improve, store the unstructured
version

Presented by : [Link]
 The properties of data
1 Data is Raw
• Raw Data vs. More Data
• More Data ≠ Raw Data
• More data doesn’t always mean it’s rawer.
• Example: Storing blog content for FaceSpace.
• What Counts as Raw Data?
• Plain text blog posts are less raw than annotated text (bold, italics,
headings).
• Annotated text provides richer context and helps with analysis.
• What is NOT Raw Data?
• Full HTML of a blog is not raw data—it includes CSS, JavaScript, and
styles that don’t contribute to content analysis.
• Raw data should be meaningful for analysis, not just a large dataset
Presented by : [Link]
 The properties of data
2 Data is Immutable
• Definition of Immutable Data
o Immutable data means once recorded, it cannot be updated or
deleted—only new data is added.
o Unlike traditional relational databases where updates are
common, immutable data follows an append-only model.
• Advantages of Immutability in Big Data Systems
a) Human-Fault Tolerance
o In a mutable model, errors can cause data loss because values
are overwritten.
o In an immutable model, past data remains, so errors can be fixed
by deleting bad entries and recomputing results from the master
dataset.
b) Simplicity
o Mutable models require indexing to locate and update records.
o Immutable models only append new records, eliminating the
need for indexing and making storage management easier.
o Storing a master dataset can be as simple as using flat files.
Presented by : [Link]
 The properties of data
2 Data is Immutable
Key Features of Immutable Data Storage
• Timestamps Ensure Historical Accuracy
o Each record includes a timestamp indicating when the data was
recorded. This allows tracking changes over time.
• Separate Tables for Different Data Types
o Each field (age, location, etc.) is stored in a separate table.
o When data changes, only the affected field gets a new record, reducing
storage duplication
Trade-offs of Immutable Data Models
• More Storage Space Required
o The database keeps all past records, not just the latest one.
o User ID is repeated in multiple records instead of being stored once per
row.
• Big Data Systems Handle Large Storage
o The extra storage is not a problem because Big Data technologies are
designed to store and manage vast amounts of data efficiently.
• Simpler and More Reliable Data Management
o The master dataset remains accurate and easy to manage.
o Data loss is minimized, and historical tracking is preserved.
Presented by : [Link]
 The properties of data
3 Data is Eternally True
• Definition of Eternal Truth in Data
o Once a piece of data is recorded as true, it must always
remain true.
o Immutability ensures that past data remains valid within the
context of its timestamp.
• Analogy: Learning History
o Historical facts remain true for their specific time period.
o Example:
▪ On July 4, 1776, the U.S. had 13 states.

▪ This remains eternally true, even though the number of

states has changed since then.

• Master Dataset Growth
o The dataset expands over time by adding new immutable
records.
o Old data is never updated or changed, only new data is
appended Presented by : [Link]
 The properties of data
3 Data is Eternally True
• Exceptions to Data Retention
Although data is eternally true, there are cases where data
deletion is necessary:
• Garbage Collection
o Low-value data can be deleted to optimize storage.
o Example: Instead of keeping all location changes, a
system may store only one location per year for each
user.
• Regulatory Compliance
o Some laws and government regulations require data
deletion for privacy or legal reasons.
o Example: GDPR (General Data Protection Regulation)
gives users the "right to be forgotten", meaning
companies must delete personal data upon request.

Presented by : [Link]
 The Fact-Based Model for Representing Data
• Instead of storing data in rigid tables, the fact-based model breaks it
down into small, fundamental units called facts.
• Each fact represents a single, immutable piece of information.
• Facts are stored with timestamps, so the dataset grows over time
instead of updating existing records.
How the Fact-Based Model Works
• Example: FaceSpace (Social Media Platform)
o When a user updates their location, instead of overwriting the old
value, a new fact is added with a timestamp.
• Tom is friends with David (2012/05/16 18:31:24)
• Tom lives in San Francisco, CA (2012/04/04 18:31:24)
• Tom works for Google (2012/04/04 18:31:24)
• This ensures that past data remains accurate and new insights can
be drawn from historical records. Presented by : [Link]
 The Fact-Based Model for Representing Data
How the Fact-Based Model Works

Presented by : [Link]
 The Fact-Based Model for Representing Data
Core Principles of the Fact-Based Model
• Atomicity: Each fact is independent and indivisible (e.g., a
user's friend list is multiple facts, not a single entry).
• Immutability: Facts are never changed or deleted, only new
facts are added with timestamps.
• Identifiability: Every fact should have a unique identifier (e.g.,
a nonce for pageviews) to prevent duplicates
Why Identifiability Matters
• Systems handling Big Data often retry sending data due to
network failures, leading to duplicate entries.
• Instead of using costly transactional writes, the fact-based
model ensures facts are distinguishable to handle duplicates
efficiently.
Presented by : [Link]
 The Fact-Based Model for Representing Data
Why Identifiability Matters
Example: In a pageview tracking system, consider this structure:
• struct PageView: DateTime timestamp String url String ip_address
• Problem: If multiple pageviews occur at the same time for the same
URL and IP, they will have identical records, making it impossible to
differentiate separate events
• Solution: Adding a Unique Identifier
• To ensure identifiability, we introduce a nonce (random 64-bit
number):
• struct PageView: DateTime timestamp String url String ip_address
Long nonce
• Impact: Now, even if all other fields match, each pageview is
uniquely identifiable, preventing accidental duplicate records.

Presented by : [Link]
 Benefits of the Fact-Based Model
1. Queryable at Any Time in History
• Traditional relational databases store only the current state of data.
• The fact-based model, however, keeps all past states by recording
facts with timestamps.
• “Updates” and “deletes” don’t modify existing records but instead
add new facts with updated timestamps.
• Enables time-travel queries, allowing you to reconstruct the state of
the data at any given point.
Example:
• Tom moves from San Francisco → Los Angeles (fact added).
• Later, this is corrected by deleting the Los Angeles fact.
• Queries automatically return San Francisco as Tom’s latest location..

Presented by : [Link]
 Benefits of the Fact-Based Model
2. Tolerates Human Errors
• Mistaken facts can be removed, revealing the previous correct state.
• No data corruption since the original facts remain.
• Eliminates the need for complex “undo” mechanisms.
Example:
• A wrong employment record is stored.
• Instead of modifying the existing data, a correction fact is added.
• Queries automatically retrieve the latest correct employment
information.

Presented by : [Link]
 Benefits of the Fact-Based Model
3. Easily Handles Partial Information
• Traditional relational databases often use NULL values for missing
data.
• The fact-based model avoids NULLs by storing only the available
facts.
• New information can be added incrementally without schema
changes.
Example:
• Tom provides his age and gender but not his location.
• The system only stores known facts—missing information doesn’t
clutter the dataset.

Presented by : [Link]
 Benefits of the Fact-Based Model
4. Combines the Benefits of Normalization and
Denormalization
• Normalization: Master dataset is fully normalized (no redundancy).
• Denormalization: Batch views index data efficiently for fast
querying.
• Prevents data inconsistency (issue in denormalized schemas).
• Allows efficient querying without expensive joins (issue in
normalized schemas).
Example:
• The master dataset has one record for each fact (normalized).
• The batch layer creates optimized views for different query types
(denormalized).
• Result: Queries run fast without redundancy issues.
Presented by : [Link]
 Benefits of the Fact-Based Model
4. Combines the Benefits of Normalization and
Denormalization

Presented by : [Link]
 Benefits of the Fact-Based Model
5. Lambda Architecture: Separation of Storage & Query
Processing
• The master dataset remains immutable and normalized.
• The batch layer continuously processes and indexes the data for
fast access.
• Batch views auto-rebuild from the master dataset—ensuring
consistent updates.
Key Benefit:
• Unlike relational databases, you don’t have to choose between data
consistency and query performance—you get both.

Presented by : [Link]