Lecture 4

Data Modeling in NoSQL

Databases
Design Principles and Consideration, Schema Design

Shahzad Rizwan
Data Modeling in NoSQL Databases
• A database schema is a formal description of the structure or
organization of a particular database (DB).
• The term database schema is most commonly used for relational
databases, which organize information in tables and use the SQL
query language.
• Non-relational (or “NoSQL”) databases come in several different
formats and don't have a “schema” in the same way that relational
databases do (although they do have an underlying structure).
Fundamental Components of the Data Schema
• There are two fundamental components of any database schema:
• Physical database schema: The physical database schema describes how
you physically store data in a storage system and the form of storage used
(files, key-value pairs, indices, etc.).
• Logical database schema: The logical database schema describes the
logical constraints applied to data and defines fields, tables, relations,
views, integrity constraints, etc. These requirements provide useful
information for programmers to apply to the physical design of a database.
The rules or constraints defined in this logical model determine how data in
different tables relate to one another.
• The definition of physical tables in the schema comes from the logical data
model. Entities become tables, entity attributes become table fields, etc.
Types of Database Schemas
• There are six most common database schema types below:
• Flat model: A flat model database schema organizes data in a single, two-
dimensional display—think of a Microsoft Excel spreadsheet or a CSV file.
This schema is best for simple tables and databases without complex
relationships between different entities.
• Hierarchical model: Database schemas in a hierarchical model have a
“tree-like” structure, with child nodes branching out from a root data node.
This schema is ideal for storing nested data—for example, family trees or
biological taxonomies.
• Network model: The network model, like the hierarchical model, treats
data as nodes connected to one other; however, it allows for more complex
connections, such as many-to-many relationships and cycles. This schema
can model the movement of goods and materials between locations or the
workflows required to accomplish a particular task.
Types of Database Schemas
• Relational model: As discussed above, this model organizes data in a series
of tables, rows, and columns, creating relationships between different
entities. The next section and the rest of this guide will focus on the
relational model.
• Star schema: The star schema is an evolution of the relational model that
organizes data into facts and dimensions. Fact data/table is numerical (for
example, the number of sales of a product), while dimensional data/table
is descriptive (for example, a product’s price, color, weight, etc.).
• Snowflake schema: The snowflake schema is a further abstraction on top
of the star schema. It contains a fact table that connects to a dimensional
table, expanding the descriptiveness possible within a database. i.e.
dimension tables are split into smaller sub-dimensions to keep data more
organized and detailed.
Star Schema Example
Snowflake Schema Example
Importance of Database Schema Design
• Inefficiently organized databases suck up tons of energy and resources and are
difficult to maintain and administer. That’s where database schema design comes
into play. You’ll struggle to get value from enterprise data without a clean,
efficient, consistent database schema. The right schema design removes
duplicated and inconsistent data in different locations.
• Relational database systems depend on a solid database schema. The goals of
good schema design include:
• Reducing or eliminating data redundancy.
• Preventing data inconsistencies and inaccuracies.
• Ensuring data integrity and correctness.
• Facilitating rapid data lookup, retrieval, and analysis.
• Keeping sensitive and confidential data secure and accessible to those who need
it.
Data modeling
• Data modeling refers to the organization of data within a database
and the links between related entities. Data in MongoDB has
a flexible schema model, which means:
• Documents within a single collection are not required to have the
same set of fields.
• A field's data type can differ between documents within a collection.
• Generally, documents in a collection share a similar structure. To
ensure consistency in your data model, you can create schema
validation rules.
Use Cases
• The flexible data model lets you organize your data to match your application's needs.
MongoDB is a document database, meaning you can embed related data in object and array
fields.
• A flexible schema is useful in the following scenarios:
• Your company tracks which department each employee works in. You can embed department
information inside of the employee collection to return relevant information in a single
query.
• Your e-commerce application shows the five most recent reviews when displaying a product.
You can store the recent reviews in the same collection as the product data, and store older
reviews in a separate collection because the older reviews are not accessed as frequently.
• Your clothing store needs to create a single-page application for a product catalog. Different
products have different attributes, and therefore use different document fields. However,
you can store all of the products in the same collection.
Understanding MongoDB Schema Design

• MongoDB stores data in a JSON-like format called BSON (Binary

JSON), which allows for flexible and dynamic schema design.
• Unlike traditional relational databases, MongoDB doesn’t enforce a
rigid schema, giving developers the freedom to structure documents
as needed.
• This flexibility is beneficial, but it also demands careful planning to
avoid performance bottlenecks and data inconsistencies.
Schema Design Principles
• Understand Your Application’s Access Patterns
Designing a schema should start with understanding how your
application will query data. Frequently accessed data should be
stored together to minimize the number of queries and optimize read
performance.
• Embed vs. Reference
One of the key decisions in MongoDB schema design is whether to
embed documents or reference them. In MongoDB, you can model
relationships between data using embedding or referencing.
Embedding Example
• Embedding: Embedding stores related data within a single document. This
is useful when the related data is frequently accessed together.
•{
"title": "MongoDB Schema Design",
"author": {
"name": "John Doe",
"email": "john@[Link]"
},
"tags": ["mongodb", "database", "schema"]
}
• In this example, the author’s details are embedded within the article
document.
Referencing Example
• Referencing: Referencing stores related data in separate documents
and links them using references (like foreign keys in relational
databases). This is useful when the related data is large or shared
across multiple documents.
Referencing Example
•{
"title": "MongoDB Schema Design",
"author_id": ObjectId("60c72b2f9b1d4f4d2c8b4567"),
"tags": ["mongodb", "database", "schema"]
}
Description
• Here, the author’s details are stored in a separate document, and the
author_id is used to reference it.
• Denormalization
• MongoDB encourages denormalization, where related data is stored
together to reduce the need for expensive joins. While this can lead to data
duplication, it often results in faster queries and a more responsive
application.
• Avoid Deep Nesting
• Deeply nested documents can lead to complexity and performance issues.
MongoDB has a limit on the maximum document size (16MB) and the
nesting depth (100 levels). Flattening structures where possible can
improve both performance and readability.
Schema Examples
• Let’s start with a simple example of a blog application where posts have comments.
• {
"_id": ObjectId("605c72af12345678abcd1234"),
"title": "Understanding MongoDB Schema Design",
"content": "MongoDB schema design is crucial...",
"author": "John Doe",
"comments": [
{
"author": "Jane Smith",
"comment": "Great article!",
"date": ISODate("2023-09-01T10:00:00Z")
},
{
"author": "Bob Brown",
"comment": "Very informative.",
"date": ISODate("2023-09-02T12:30:00Z")
}
],
"tags": ["MongoDB", "Schema Design", "NoSQL"]
}
• In this schema, comments are embedded within the post document. This design is efficient for scenarios
where comments are frequently accessed along with the post.
Schema Examples
• In an e-commerce application, products and orders are often modeled using references.

• Products Collection:

• {
"_id": ObjectId("605c72af12345678abcd5678"),
"name": "Smartphone",
"description": "Latest model with advanced features",
"price": 699.99,
"category": "Electronics"
• }
Schema Examples
• Orders Collection:

• {
"_id": ObjectId("605c72af12345678abcd9101"),
"orderNumber": "ORD123456",
"customerId": ObjectId("605c72af12345678abcd1122"),
"orderDate": ISODate("2023-09-03T14:45:00Z"),
"items": [
{
"productId": ObjectId("605c72af12345678abcd5678"),
"quantity": 2,
"price": 699.99
}
],
"totalAmount": 1399.98
• }
Schema Examples
• In the previous design, products are referenced in the orders
collection. This approach is suitable when the product information is
large or frequently updated, as it avoids duplicating product data
across multiple documents.
Schema Design for Web3 Projects
• In Web3 projects, schema design must account for decentralized and absolute data structures, where data
integrity and transparency are vital.
• Users Collection (for a Web3 Wallet Application):
• {
"_id": ObjectId("605c72af12345678abcd1313"),
"walletAddress": "0xAbC1234567890Def1234567890ABCdEF12345678",
"username": "cryptouser",
"createdAt": ISODate("2023-09-01T08:00:00Z"),
"transactions": [
{
"txHash": "0xabc1234567890def1234567890abcdef1234567890abcdef1234567890abcdef",
"amount": 1.5,
"token": "ETH",
"timestamp": ISODate("2023-09-01T08:30:00Z")
},
Schema Design for Web3 Projects
{
"txHash":
"0xdef1234567890abc1234567890abcdef1234567890abcdef1234567890abc
def",
"amount": 0.75,
"token": "DAI",
"timestamp": ISODate("2023-09-01T09:00:00Z")
}
]
•}
Smart Contracts Collection (for Design Apps):
•{
"_id": ObjectId("605c72af12345678abcd1414"),
"contractAddress":
"0xABC1234567890def1234567890Abcdef12345678",
"creator": "0xDef1234567890ABC1234567890defAbc12345678",
"creationDate": ISODate("2023-09-01T07:00:00Z"),
"contractABI": [ ... ],
"transactions": [
Smart Contracts Collection (for Design Apps):
{
"txHash": "0x1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
"event": "Transfer",
"from": "0xAbC1234567890Def1234567890ABCdEF12345678",
"to": "0xDef1234567890Abc1234567890abcdef12345678",
"value": 1000,
"token": "USDC",
"timestamp": ISODate("2023-09-01T08:30:00Z")
}
]
• }
• In this Web3-specific schema, wallet addresses, transaction hashes, and smart contract details are
crucial.
Best Practices for MongoDB Schema Design
• Optimize for Read Operations: Since MongoDB is often used for read-heavy
workloads, design your schema to reduce the number of queries needed to fetch
related data.
• Use Aggregation Pipelines: MongoDB’s aggregation framework allows for
powerful data processing within the database. Design your schema to take
advantage of these capabilities.
• Consider Sharding: For large-scale applications, consider how your schema will
perform when sharded. Design your shard key carefully to distribute data evenly
across the cluster.
• Balance Embedding and Referencing: Striking the right balance between
embedding and referencing is key to optimizing both performance and data
integrity.
• Plan for Growth: Think about how your data and application will evolve. Design
your schema to accommodate future features and scale.
Additional Data Modeling Considerations
• The following factors can impact how you plan your data model.
• Data Duplication and Consistency
• When you embed related data in a single document, you may duplicate data between two
collections. Duplicating data lets your application query related information about multiple
entities in a single query while logically separating entities in your model.
• For example, a products collection stores the five most recent reviews in a product document.
Those reviews are also stored in a reviews collection, which contains all product reviews. When a
new review is written, the following writes occur:
• The review is inserted into the reviews collection.
• The array of recent reviews in the products collection is updated with $pop and $push.
• If the duplicated data is not updated often, then there is minimal additional work required to
keep the two collections consistent. However, if the duplicated data is updated often, using a
reference to link related data may be a better approach.
• Before you duplicate data, consider the following factors:
• How often the duplicated data needs to be updated.
• The performance benefit for reads when data is duplicated.
Additional Data Modeling Considerations
• Indexing
• To improve performance for queries that your application runs frequently, create indexes on
commonly queried fields. As your application grows, monitor your deployment's index use to
ensure that your indexes are still supporting relevant queries.
• Hardware Constraints
• When you design your schema, consider your deployment's hardware, especially the amount of
available RAM. Larger documents use more RAM, which may cause your application to read from
disk and degrade performance. When possible, design your schema so only relevant fields are
returned by queries. This practice ensures that your application's working set does not grow
unnecessarily large.
• Single Document Atomicity
• In MongoDB, a write operation is atomic on the level of a single document, even if the operation
modifies multiple embedded documents within a single document. This means that if an update
operation affects several sub-documents, either all of those sub-documents are updated, or the
operation fails entirely and no updates occur.
• A denormalized data model with embedded data combines all related data in a single document
instead of normalizing across multiple documents and collections. This data model allows atomic
operations, in contrast to a normalized model where operations affect multiple documents.
Conclusion
• MongoDB’s flexible schema design offers both power and
responsibility. By understanding your application’s needs and
following best practices, you can create a schema that is both
efficient and scalable.
• Whether you’re building a traditional web application or a cutting-
edge Web3 project, mastering MongoDB schema design is crucial for
success.