Salesforce Marketing Cloud Data Modeling Guide
Salesforce Marketing Cloud Data Modeling Guide
Data Modeling
Contact Model
The Contact model defines how Marketing Cloud stores information about subscribers across
channels. It can be configured in the Data Designer tool within Contact Builder and allows to
relate data stored in channel applications.
Contact Model
The Contact model combines Subscribers data from different channel applications based on a
unique key referred to as Contact Key.
Contact Model
Subscribers
Subscriber data is managed in Marketing Cloud channel applications, such as Email Studio for
email, MobileConnect for SMS, or MobilePush for push notifications.
Contacts
Contact data is sourced from Marketing Cloud applications, such as All Subscribers in Email
Studio or All Contacts in Contact Builder, Journey Builder, MobileConnect, and MobilePush.
In addition, Contact data can also be sourced from external data sources through integrations
such as Marketing Cloud Connect for Salesforce Sales Cloud or Service Cloud.
Contact Key
Marketing Cloud combines Contact information across channels based on the Contact Key. In
Email Studio, the Contact Key is referred to as Subscriber Key.
The Contact Key is a unique value assigned each Contact and Subscriber. It allows Marketing
Cloud to identify Subscribers across channels as a single Contact.
If no Contact Key is defined for a Subscriber, the value is defaulted to address of the channel,
such as email address for emails, mobile number for SMS, or device ID for push.
Data Designer
The Contact model can be configured with the Data Designer tool within Contact Builder. It
consists of Attributes, Attribute Groups, Relationships, and Populations.
Data Designer
Attributes
An Attribute describes a single piece of information about a Contact. Attributes can be created
to define a relevant data model around Contacts.
1. Profile Attributes
2. Behavioural Attributes
Profile Attributes describe who a Contact is (e.g. name), and Behavioural Attributes describe
what a Contact has done (e.g. email clicks).
Attributes
Attribute Groups
Attribute Groups can be used to organize Data Extensions related to Contacts. They consist of
a collection of Data Extensions whose fields become Attributes.
Attribute Group
Relationships
Data Extensions in Attribute Groups can be linked to each other, to Contacts, and to
Populations. Relationships are defined based on a key, such as the Contact Key. Supported
cardinalities are one-to-one, one-to-many, and many-to-many.
Relationships
Populations
It is not necessary to create populations for simple segmentation, but only for advanced use
cases with complex queries or API integrations. The number of populations within an account
should be less than three.
Data Extensions
Data Extensions are custom database tables in Marketing Cloud to store information. They can
be related to each other to build a custom data model.
Data Extensions
Data Extensions allow to define custom data tables in Marketing Cloud, to store Subscriber
and Contact information. The can be created in Email Studio and Contact Builder, and consist
of keys, fields, and relationships.
Keys
Keys are fields to uniquely identify Data Extension records, and to relate Data Extension to
each other. There are three types of keys in Marketing Cloud:
1. Primary Key
2. Foreign Key
3. Contact Key
Primary keys are used to uniquely identify records in a Data Extension. A primary key value
must therefore be unique in a Data Extension. Duplicate records with the same primary key are
not possible.
Foreign keys allow to relate Data Extension records to other records in different Data
Extensions.
The Contact Key is a unique value to identify Subscribers across channels as a single Contact.
In Email Studio, the Contact Key is referred to as Subscriber Key.
Fields
Data Extensions consist of fields to store relevant information. Each Data Extensions field has
a data type and maximum length, which defines the kind of information that can be stored.
Supported field data types are: number, text, date, boolean, decimal, email, phone number, and
locale. For integer numbers the decimal data type can be used, by setting decimal places to 0.
Relationships
Data Extensions can be standalone or related to other Data Extensions. Relationships allow to
link Data Extensions based on a key and cardinality. The cardinality quantifies how records
relate to each other on both sides of the relationship. Supported cardinalities are one-to-one,
one-to-many, and many-to-many.
In addition to relating Data Extensions to each other, they can also be related to Contacts. Data
Extensions in Attribute Groups can be linked to Contacts based on the Contact Key.
Attribute Group
Standard Data Extensions can be used to create custom set of fields and keys to store
information in Marketing Cloud.
Filtered Data Extensions can be used to select records from a source Data Extensions based on
criteria, and store them in a new target Data Extension. Criteria can be based on field values as
well as measures.
Filtered Data Extension
They can be used to segment an audience based on specific attributes. This allows to target
marketing activities towards specified subsets of Contacts or Subscribers.
Random Data Extensions can be used to randomly select records from a source Data Extension
and store them in one or more target Data Extensions.
They can be used to segment an audience based random assignment. This allows to target
marketing activities towards a sample of contacts or Subscribers.
Sendable
Sendable Data Extensions can be used as an audience in Marketing Cloud applications such as
Email Studio or Journey Builder. They have a send relationship to map Data Extension records
to email Subscribers based on the Subscriber Key.
Send Relationship
If an email is sent to a sendable Data Extension, the Contacts in the Data Extension are added
to the All Subscribers list in Email Studio.
If a Data Extension is sendable, it can further be defined as testable. This allows to use the
Data Extension as an audience for email test sends in Email Studio.
Non-Sendable
Non-sendable Data Extensions can not be used as audiences, but can store reference or
personalization data linked to Subscribers.
Synchronized Data Extensions are only available to to the top-level business unit by default
and cannot be share with other business units. To achieve this, a filter activity must be created
which copies data from the synchronized Data Extension to shared Data Extension.
Contact Records
Marketing Cloud uses the Contact Key to identify and relate Subscribers across channels as
Contacts. Contact Builder consolidates related Contact data in Contact records.
Contact Key
Marketing Cloud combines Contact information across channels based on the Contact Key. In
Email Studio, the Contact Key is referred to as Subscriber Key.
Contact Model
The Contact Key is a unique value assigned each Contact and Subscriber. It allows Marketing
Cloud to identify Subscribers across channels as a single Contact.
If no Contact Key is defined for a Subscriber, the value is defaulted to address of the channel,
such as email address for emails, mobile number for SMS, or device ID for push.
Contact ID
Contacts can also be uniquely identified by the Contact ID. This ID is auto-generated by
Marketing Cloud when a Contact is created.
Compared to the Contact Key, the Contact ID cannot be chosen or changed. It is used
internally by Marketing Cloud to uniquely identify Contacts.
Contact Records
Contact records can be accessed in the All Contacts tool within Contact Builder. The tool
consolidates Contact data across channels applications and the Contact model, based on the
Contact Key.
Contact Record
Contact records consists of system data, email data from Email Studio, MobileConnect data,
MobilePush data, and Attribute Groups defined in the Contact model.
Data from additional Marketing Cloud applications can appear, if enabled in an account. For
example from GroupConnect or Einstein Predictive Intelligence.
Example
The following is an example how Subscribers data in channel applications roll up to Contact
records in Contact Builder.
Email Studio
Subscribers are stored in the All Subscribers list within Email Studio. Profile and Preference
Attributes can be defined to describe Subscribers, and the Subscriber Key is used to uniquely
identify them.
The All Subscribers list in Email Studio could look like this:
MobileConnect
Subscribers are stored in the All Contacts list within MobileConnect. Attributes can be defined
to describe Subscribers, and the Contact Key is used to uniquely identify them.
MobilePush
Subscribers are stored in the All Contacts list within MobilePush. Attributes can be defined to
describe Subscribers, and the Contact Key is used to uniquely identify them.
Contact Builder
Contact Builder combines Subscriber information from channel applications based on the
Subscriber Key and Contact Key. This allows to identify Subscribers across channels as a
single Contact.
The All Contacts list in Contact Builder could look like this:
Contact
Email Address Name Mobile Phone Operator Mobile Device ID Device
Key
CUST00 00018051705
john@[Link] John Alphacom 90QJDC0OEAU6IXD9 iOS
1 4
1. Suppression phase
2. Deletion phase
Suppression Phase
During the suppression phase, Contacts requested for deletion remain in Marketing Cloud but
are hidden in channel applications such as Email Studio, MobileConnect, or Mobile Push.
They can also not be accessed in Contact Builder or be imported again.
The default suppression period is 14 days but can be changed to delete Contacts sooner or
later. If the suppression period is set to 0, then the suppression period is skipped and deletions
processed direclty.
Suppression Period
Deletion Phase
In the deletion phase, suppressed Contacts are permanently removed from Marketing Cloud.
Deleted Contacts are removed from Contact Builder and all sendable Data Extensions. After a
Contact is deleted, it can be imported to Marketing Cloud again.
Contacts are not automatically removed from non-sendable Data Extensions. They are also not
removed from aggregated tracking data stored in Send Logs.
Contact Builder
A single Contacts can be deleted by selecting it in All Contacts within Contact Builder and
initiating the deletion process.
REST API
The REST API can be used to programmatically delete single or multiple Contacts.
A single Contact can be deleted by referencing its Contact Key, Subscriber Key, Contact ID, or
Subscriber ID in one of the following API routes:
POST /contacts/v1/contacts/actions/delete?type=ids
POST /contacts/v1/contacts/actions/delete?type=keys
Multiple contacts can be deleted by referencing a List or sendable Data Extension in following
API route:
POST /contacts/v1/contacts/actions/delete?type=listReference
When deleting multiple Contacts, the call limits each process to one million records. To delete
more than one million Contacts, the deletion process must be initiated multiple times.
Data Retention
In addition to the Contact delete process, retention policies can be configured for Data
Extensions. Data retention policies define when Data Extensions and records are removed
from Marketing Cloud. If data retention is enabled for a Data Extension, it can be changed but
not removed later.
Data Retention Policy
Retention policies can be applied to individual records, all records, or all records and the Data
Extensions itself. A retention period or a specific date can be defined, after which the selected
records are permanently deleted from Marketing Cloud. The retention period can be
automatically reset after a new import has been performed on the Data Extension.
Removing Data Extensions or records not equivalent to Contact deletion, which removes all
data related to a Contact, including tracking data.
Programmactic Languages
AMPscript
AMPscript is a proprietary Marketing Cloud scripting language which can be embedded in
emails, SMS messages, push notifications, and landing pages. The embedded code is processed
on server-side at send time or when a landing landing is requested.
Syntax
AMPscript can be added to messages and landing pages using following code block:
%%[
// AMPscript code
]%%
Personalization strings can be used to output values from different sources in a message or
landing page:
• Subscriber and system attributes from Lists and sendable Data Extensions:
• Variables:
%%=v(@MyVar)=%%
%%=UPPERCASE(@MyVar)=%%
Functions
AMPscript provides a large number of functions. They can be categorized in following groups:
1. API
2. Contacts
3. Content
4. Data Extensions
5. Data Time
6. Encryption
7. HTTP
8. Math
9. Microsoft Dynamics CRM
10. Salesforce
11. Sites
12. Social
13. String
14. Utilities
Example
The following AMPscript code can be added in an email message, to retrieve records from a
Data Extension and write the content of a field in the message body.
%%[
SET @rows = LookupRows("Certificates","Customer Id",_subscriberKey)
SSJS
Server-Side JavaScript (SSJS) is based on JavaScript but executes on server-side instead of
client-side. It includes standard JavaScript functions and Marketing Cloud specific
functionality from two proprietary libraries. External libraries are not compatible with SSJS
and the document object model (DOM) of messages or landing pages cannot be accessed.
Syntax
SSJS can be added to messages and landing pages using following code block:
<script runat=server>
// SSJS code
</script>
Optionally, the language can be specified when defining a SSJS code block:
<script runat=server language=javascript>
// SSJS code
</script>
Personalization Strings
Personalization strings can be used to output values from different sources in a message or
landing page:
• Subscriber and system attributes from Lists and sendable Data Extensions:
• Variables:
<ctrl:eval language=javascript>[Link]()</ctrl:eval>
Functions
In addition to most JavaScript functions, SSJS offers two libraries which provide Marketing
Cloud specific functions:
1. Platform Library
2. Core Library
The two libraries provide some overlapping functions but are used differently. The Platform
library can be used in messages and landing pages. However, the Core library can only be used
in landing pages and applications.
The platform library is included in default, and the core library can be loaded in landing pages
as follows:
<script runat=server>
[Link]("Core", "1");
</script>
Platform Library
The SSJS functions provided by the Platform library can be categorized in following groups:
1. API
2. Field and Attribute
3. HTTP
4. Content
5. Data Extensions
6. Date and Time
7. Client Browser
8. Utility
9. AMPscript Variables
10. HTTP Property
11. Content Syndication
Core Library
The SSJS functions provided by the Core library can be categorized in following groups:
1. Account
2. AccountUser
3. Content Area
4. Data Extension
5. Date and Time
6. Delivery Profile
7. Email
8. Events
9. Filter
10. Folder
11. HTTP
12. List
13. Portfolio
14. Query Definition
15. Recipient
16. Send Classification
17. Send
18. Sender Profile
19. Subscriber
20. Template
21. Triggered Send
22. Utilities
Example
The following SSJS code can be added in an email message, to retrieve records from a Data
Extension and write the content of a field in the message body.
<script runat="server">
var rows = [Link]('Certificates','Customer
Id',_subscriberKey);
On the other hand, AMPscript is better integrated in Marketing Cloud and therefore provides
better performance. It handles inline personalization and IF/ELSE statements more efficiently
and is preferred for programmatic content personalization. Furthermore, AMPscript is easier to
learn for new developers without prior experience.
AMPscript and SSJS can be used jointly to benefit from advantage in both languages.
Variables defined in AMPscript code can be get and set in SSJS. The SSJS Variable utility
functions can be used to achieve this:
[Link]("@AMPscriptVariable");
[Link]("@AMPscriptVariable","new value");
Testing
AMPscript and SSJS code should be thoroughly tested before it is deployed to customers.
For AMPscript and SSJS in email messages, the Markup Validation and Send Preview tools in
Email Studio can be used for testing. In addition, test sends can be performed to validate all
scenarios handled in code.
For AMPscript and SSJS in landing pages, preview feature in the Cloud Pages can be used for
testing. In addition, the location of landing pages can be changed from a test to a production
URL after testing is completed.
Debugging
Syntax validation for AMPscript in email messages is integrated in Content Builder. Syntax
errors will appear in the Preview and Test tab for debugging.
Other than this, Marketing Cloud provides no validation or debugging tools. To identify the
root cause of faulty code, it is recommended to narrow down the problem, by first reducing the
code and then slowly adding code while testing after each iteration. As part of this, it can be
helpful to output variables at different positions in the code.
Exception Handling
AMPscript and SSJS provide different features for exception handling. While AMPscript
provide simple exception handling, SSJS offers a more advanced features.
In AMPscript, the RaiseError() function can be used to handle exceptions. When this function
is called in code within an email message, it stops the email send for either all or individual
Subscribers. If Subscribers are excluded from a send because of this function, Marketing
Cloud still includes them in tracking data and counts them as sent towards the send limit. This
function is therefore not intended for segmentation, but only for error handling for small
numbers of email Subscribers.
In SSJS, standard try/catch blocks can be used for advanced exception handling. Code within
the try block can throw exceptions when it fails, which can be handled in the catch block.
try {
Block of code to try
}
catch(err) {
Block of code to handle errors
}
Attribute Values
Attribute values related to Contacts or Subscribers can be programatically retrieved. Values are
derived from the send context related to a Contact or Subscriber, including following data
sources:
Because attribute values can be empty in the context of a Contact or Subscriber, this exception
should be programatically handled.
In AMPscript, it can be checked with the AttributeValue() and Empty() functions. The latter
function checks if a value is null or empty.
SET @name = AttributeValue("Name")
IF EMPTY(@name) THEN
SET @name = "Valued Customer"
ENDIF
The same and additional validations should be applied if form data is handled on landing pages
or Marketing Cloud apps.
Impression Tracking
Impression tracking allows track to the performance of dynamic email content created with
AMPscript.
Impression regions can be nested and can include dynamic content, or references to static
content using the ContentArea() or ContentAreaByName() functions.
AMPscript Processing
AMPscript code gets executed top down when a message is sent or a landing page is requested
from the server. The order of execution is relevant for email messages consisting of HTML
and text versions.
Order of Execution
AMPscript is executed from top to bottom. This impacts variables, which need to be defined
before they can be referenced.
The code below would fail because the @name variable is not defined before it is referenced:
Output(Concat("Name: ", @name))
SET @name = "Miller"
Email Messages
Multi-part MIME emails consists of a HTML and text version. If AMPscript is used in email
messages, the code is executed in following order:
As a result, AMPscript variables need to be declared in the HTML body before they can be
used in the text body or subject line.
The code below would fail because the @name variable is not defined before it is referenced:
Subject Line:
set @name = AttributeValue("First Name")
HTML Body:
<h1>Hi %%=v(@name)=%%</h1>
Subscriber Exclusion
AMPscript exclusion scripts allow to dynamically exclude Subscribers from email sends at
send time. Exclusion scripts evaluate to a boolean value, which determines if a Subscriber is
included in or excluded from the send.
Exclusion Scripts
Exclusion scripts are executed at send time and for each individual Subscriber in the target
audience. They can include Subscriber attributes to evaluate a logical test. The script must
evaluate to a boolean, which determines if a Subscriber is included in or excluded from a send.
If the logical test of an exclusion script evaluates to TRUE, then the Subscribers is excluded
from the send. If it evaluates to FALSE, then the Subscriber is included in the send.
Exclusion Script
Availability
Exclusion scripts are available in following Marketing Cloud applications:
• Content Builder
• Journey Builder
• Triggered Sends
• Automation Studio
Limitations
Exclusion scripts are not intended to replace segmentation for large audiences and should only
be used in exceptional cases. Furthermore, not all AMPscript features are available in
exclusion scripts.
API
SOAP API
The SOAP API provides access to most Email Studio functionality in Marketing Cloud. It
allows external applications to create and send emails, as well as to receive tracking
information. This API can also be used to modify Marketing Cloud account information.
SOAP uses XML messages defined in WDSL, to exchange information between integrated
applications. The API consists of objects and methods, whereby methods can be applied to
different objects.
The SOAP API allows to perform requests synchronously as well as asynchronously. While
synchronous API calls can include up to 50 objects, asynchronous requests support up to 100
objects. Asynchronous API calls are queued by Marketing Cloud and help to avoiding
timeouts.
In summary, the SOAP API provides access to following Marketing Cloud functionalities:
• Emails
• Tracking
• Subscribers and Lists
• Automations
• Triggered Sends
REST API
The REST API provides broader access to Marketing Cloud than the SOAP API. It is not
limited to Email Studio functionalities, but is also integrated with newer Marketing Cloud
applications such as Journey Builder.
REST uses JSON messages and standard HTTP requests to exchange information between
integrated applications.
REST API calls are synchronous and timeout after 300 seconds for data-retrieval operations,
and after 120 for all other requests. The payload size of REST API requests is limited to 4
megabytes.
In summary, the REST API provides access to following Marketing Cloud functionalities:
• Contacts
• Content Builder
• Journey Builder
• MobileConnect
• MobilePush
• Campaigns
• Triggered Sends
• Platform Events
The Marketing Cloud REST API can be broken down into application-specific APIs for
Content Builder, Event Notifications, Journey Builder, GroupConnect, MobileConnect,
Personalization Builder, Transactional Messaging.
The Event Notification Service (ENS) API can be used to receive notifications when platform
events occur in Marketing Cloud. This allows external applications to respond to event
occurring in Marketing Cloud, such as log ins, password reset requests, email send errors, etc.
The Journey Builder API can be used to programatically create event-driven and personalized
campaigns across channels. Custom events and activities can be built to extend the out-of-the-
box features available in the Journey Builder application.
The GroupConnect Chat Messaging API can be used to send event-driven outbound messages
to Facebook Messenger and LINE.
MobileConnect API
The MobileConnect API can be used to send event-driven outbound SMS to Subscriber or
Lists. It can further be used to programatically opt-in Subscribers to keywords.
Personalization Builder API
The Personalization Builder API can be used to update the data used by Einstein
Recommendations to personalize messages, manage consumer privacy requests, and retrieve a
report listing the count of email recommendations.
The Transactional Messaging API can be used to send personalized transactional email and
SMS messages. By using the Event Notification Service API, immediate feedback about the
transactional messages can be retrieved.
Transactional messages sent using the Transactional Messaging API differ from triggered
sends within the Email Studio and MobileConnect applications. The Transactional Messaging
API provides better scalability and faster send performance. It is not compatible with triggered
send definitions created in Email Studio or MobileConnect.
Packages
Marketing Cloud packages allow to manage integrations from external applications towards
Marketing Cloud. A package consists of one or more component providing access to
Marketing Cloud functionality.
1. API Integration
2. Marketing Cloud App
3. Custom Content Block
4. Journey Builder Activity
5. Journey Builder Entry Source
Component Types
API Integration
The API Integration component allows to provision access to the Marketing Cloud SOAP and
REST API. The component generates endpoints and authentication credentials for external
applications. Only one API Integration component can be added per package. Once added it
cannot be removed without deleting the whole package.
Marketing Cloud App
A Marketing Cloud App is an external application which can be displayed in Marketing Cloud
using an iFrame. Marketing Cloud Apps can be accessed from a menu item in Marketing
Cloud. Only one Marketing Cloud App component can be added per package. Once added it
cannot be removed without deleting the whole package.
A Custom Content Block can be used to use a different HTML editor than Content Builder in
Marketing Cloud.
Journey Builder Activity components allow to create custom Journey Builder activities which
make REST API calls to external applications. Multiple Journey Builder Activity components
can be added per package.
Journey Builder Entry Source components allow to add Contacts from REST API calls to
journeys defined in Journey Builder. Multiple Journey Builder Entry Source components can
be added per package.
OAuth Authentication
Both the Marketing Cloud SOAP API and REST API support OAuth 2.0 authentication. The
authentication flow depends on the integration type of the API Integration component in a
Marketing Cloud package.
For Marketing Cloud the OAuth grant type and credentials are derived from the integration
type defined in the API Integration component of an installed package.
Integration Types
After adding an API Integration component to a Marketing Cloud package, an integration type
can be chosen. Following integration types are available:
1. Web App
2. Public App
3. Server-to-Server
Integration Types
The integration type defines the OAuth grant type and the available credentials for external
applications.
The Web App integration type makes API requests in the context of end users who access the
app. This integration is issued a client ID and client secret. It uses the Authorization Code
grant type.
Public App
The Public App integration type also makes API requests in the context of end users who
access the app. However, this integration is only issued a client ID as it cannot securely store a
client secret. It uses Authorization Code grant type.
Server-to-Server
The Server-to-Server integration type makes API requests in the context of a system user
instead of an end user. This integration is issued a client ID and client secret. It uses the Client
Credentials grant type.
Access Scope
After choosing the integration type for an API Integration component, the access scope can be
configured. This defines which permissions external applications using the API Integration
will have.
Properties
Authentication Flow
The authentication mechanism depends on the OAuth grant type and thus on the integration
type of a API Integration component.
Web App integrations use the Authorization Code grant type with client ID and client secret.
API Integration Credentials
With Web App integrations, external applications can authenticate towards Marketing Cloud
as follows:
The access permission of the external application is the combination of the Marketing Cloud
user permission and the access scope defined in the package component.
Public App integrations use the Authorization Code grant type with only client ID. However,
the authentication flow is the same as with Web App integrations.
With Public App integrations, external applications can authenticate towards Marketing Cloud
as follows:
The access permission of the external application is the combination of the Marketing Cloud
user permission and the access scope defined in the package component.
Server-to-Server Authentication
Server-to-Sever integrations use the Client Credentials grant type with client ID and client
secret. External applications need an access token to request the SOAP or REST API
resources.
External applications can authenticate towards Server-to-Server integrations with following
steps:
The access token can be requested by performing a POST call to the Marketing Cloud
Authentication Server and including the client ID and client secret.
Host: [Link]
POST /v2/token
Content-Type: application/json
{
"grant_type": "client_credentials",
"client_id": "CLIENT_ID",
"client_secret": "CLIENT_SECRET"
}
If the client ID and client secret are correct, the Authentication Service will respond with the
an access token and expiry time.
HTTP/1.1 200 OK
{
"access_token":"ACCESS_TOKEN",
"expires_in":1200,
"token_type":"Bearer",
"rest_instance_url":"[Link]",
"soap_instance_url":"[Link]",
"scope": "email_read email_write email_send"
}
The returned access token be used in subsequent API calls without further authentication and
without providing client ID or client secret.
For REST API calls, the access token can be added to the HTTP header:
GET [Link]
Accept: application/json
For SOAP API requests, the access token is added to the envelope:
<s:Envelope xmlns:s="[Link]
<s:Header>
<h:fueloauth xmlns="[Link]
xmlns:h="[Link]
ACCESS_TOKEN
</h:fueloauth>
</s:Header>
</s:Envelope>
OAuth 2.0 access tokens expire 20 minutes after they are issued. Access tokens from legacy
packages using an older OAuth version expire after 60 minutes. To get a new access token, the
Authentication Service can be called again.
Response Handling
The SOAP API uses XML for requests and responses, while the REST API uses JSON for the
same purpose. If a request fails, the two APIs respond with different error codes and messages.
SOAP API
SOAP API responses consist of a HTTP header and a XML body. The XML body indicates
whether the request was successful or not.
A successful API request returns the requested response in the XML body. Failed API requests
returns an error code and message in the body.
1. System
2. Validation
3. Rate Limited
System Errors
System errors indicate that an error occurred in Marketing Cloud. As an example, one of the
following error codes and messages could be returned:
Error
Description
Code
Error
Description
Code
Synchronous request cannot be completed at this time due to database problems. This request
13
was not processed.
Validation Errors
Validation errors indicate a problem with the information provided in the API request. As an
example, one of the following error codes and messages could be returned:
11 Member: [member] does not have AsyncAPI authorization. Request was denied
Marketing Cloud throttles API request to stabilize system performance. As an example, the
following error codes and messages could be returned:
Error
Message
Code
We noticed an abnormal spike in your call volume so your SOAP Web Service account has been
17
rate limited temporarily.
REST API
REST API responses consist of a HTTP header and a JSON body. The header contains an
HTTP code which indicates whether the request was successful or not.
A successful API request returns a HTTP response code 2XX and the requested resources in
the JSON response body. Failed API requests return an HTTP error code and an error message
in the body.
REST API error responses can be categorized as follows:
1. Validation
2. Authorization
3. Authentication
4. Runtime
5. Rate Limiting
Validation Errors
Validation errors indicate a problem with the information provided in the API request. They
return following HTTP error codes:
Authorization Errors
Authorization errors indicate a permission problem connected to the API request. They return
following HTTP error codes:
Authentication errors indicate an access problem connected to the API request. They return
following HTTP error codes:
403 General
Runtime Errors
Runtime errors indicate system errors in Marketing Cloud. They return following HTTP error
codes:
400 Generic
Import Activity
File Transfer and File Import activities in Automation Studio allow to load data from different
file locations into Marketing Cloud Data Extensions. The combination of both activities allows
to also import zipped and encrypted files.
Starting Source
The starting source of an Automation Studio workflow defines how the workflow gets
triggered. Available starting sources are:
1. Schedule
2. File Drop
Scheduled Automations
Scheduled automations can have a schedule which triggered the workflow at predefined dates
and times. If no schedule is defined, the automation run when trigged manually in Automation
Studio.
Scheduled Automation
• Never
• After x occurrence
• On a date
File Drop automations start whenever a new file is placed in a specific SFTP folder. A
filename pattern can be specified, to only triggered the automation when an appropriate file is
placed. An operator defines in which place place the pattern is matched. Supported operators
are begins with, ends with, and contains. If not filename pattern is defined, the automation runs
whenever any new file is placed in the specified folder.
Import File
Import files must be in a delimited format. Supported delimiters are comma, tab, or any other
single character.
The files can be located on the Marketing Cloud Safehouse, the Marketing Cloud Enhanced
FTP server, as well as on external FTP servers.
An import file can either be selected from the file location, or a file pattern can be specified.
This allows the activity to pick up new import files which have been places on the file location
by other processes.
Import File
Destination
The target Data Extension can be any existing Data Extension in Marketing Cloud.
Mapping
The field mapping defines how columns of the import file are mapped to target Data Extension
fields.
The mapping can be done automatically based on column header names, or manually based on
column ordinals or an example import file.
Data Actions
File Import activities can insert, update, upsert, and overwrite data in the target Data
Extensions.
The insert method only adds new records to the target Data Extension, and the update
operation only updates existing records based on a matching key. The upsert method combines
insert and update operations, by updating existing records based on a matching key and
inserting records for which no match is found. The overwrite operation replaces all records in
the Data Extension with the content of the import file.
A combination of File Transfer and Import File activities allows to import zipped and
encrypted files into Marketing Cloud Data Extensions.
Manage Files
The Manage File option of File Transfer activities allows to unzip or decrypt files from
different file locations. The decompression or decryption processes happens in the Safehouse,
a highly-redundant file location provided by Marketing Cloud.
File Transfer Manage Files
Move Files
The Move File option of File Transfer activities allows to move files from the Safehouse to
other file locations. In connection with the Manage File option, this allows to move unzipped
or decrypted files from the Safehouse to another location for further processing.
The Move File options further allows to decrypt a file in the Safehouse location. Both PGP and
GPG encryption methods are supported.
The results of SQL Query activities can be stored in a target Data Extensions. Query results
can be appended, updated, or overwritten in the target Data Extension.
The field mapping of query results to the target Data Extension is done automatically based on
column and field names. To change the mapping, aliases can be specified using
the AS keyword.
Select Statement
SELECT statements retrieve records from Data Extensions or Data Views. Fields to be
retrieved can be listed, or the * wildcard can be used to query all fields.
Query
SELECT ID, Name, Email, Gender FROM Customers
Result
ID Name Email Gender
Query
SELECT * FROM Orders
Result
ID Customer_ID Product Price
1 2 Green 39.99
2 3 Blue 19.99
3 4 Red 29.99
Where Clause
WHERE clauses filter SELECT statement based on a condition.
Query
SELECT Name, Email FROM Customers WHERE Gender = 'Female'
Result
Name Email
Susan susan@[Link]
Query
SELECT * FROM Orders WHERE Price > 20
Result
ID Customer_ID Product Price
1 2 Green 39.99
3 4 Red 29.99
Join Clauses
JOIN clauses combine records from two or more Data Extensions based on a matching field.
Following join types exist:
1. Left Join
2. Left Join with Exclusions
3. Right Join
4. Right Join with Exclusions
5. Inner Join
6. Full Outer Join
7. Outer Join
Left Join
LEFT JOIN returns all records from the left Data Extension, and all records from the right
Data Extension which match a common field.
Query
SELECT [Link], [Link], [Link], [Link]
FROM Customers
LEFT JOIN Orders
ON [Link] = Orders.Customer_ID
Result
Name Email Product Price
John john@[Link]
LEFT JOIN with Exclusions return all records from the left Data Extension which do not
match a common field on the right Data Extension.
Query
SELECT [Link], [Link], [Link], [Link]
FROM Customers
LEFT JOIN Orders
ON [Link] = Orders.Customer_ID
WHERE Orders.Customer_ID IS NULL
Result
Name Email Product Price
John john@[Link]
Right Join
RIGHT JOIN returns all records from the right Data Extension, and all records from the left
Data Extension which match a common field.
Query
SELECT [Link], [Link], [Link], [Link]
FROM Customers
RIGHT JOIN Orders
ON [Link] = Orders.Customer_ID
Result
Name Email Product Price
Red 29.99
Right Join with Exclusions
RIGHT JOIN with Exclusions returns all records from the left Data Extension which do not
match a common field on the right Data Extension.
Query
SELECT [Link], [Link], [Link], [Link]
FROM Customers
RIGHT JOIN Orders
ON [Link] = Orders.Customer_ID
WHERE [Link] IS NULL
Result
Name Email Product Price
Red 29.99
Inner Join
INNER JOIN returns all records from the left and the right Data Extension which match a
common field.
Query
SELECT [Link], [Link], [Link], [Link]
FROM Customers
INNER JOIN Orders
ON [Link] = Orders.Customer_ID
Result
Name Email Product Price
FULL OUTER JOIN returns all records from the left and the right Data Extension.
Query
SELECT [Link], [Link], [Link], [Link]
FROM Customers
FULL OUTER JOIN Orders
ON [Link] = Orders.Customer_ID
Result
Name Email Product Price
John john@[Link]
Red 29.99
Outer Join
OUTER JOIN returns all records from the left and the right Data Extension which do not
match a common field.
Query
SELECT [Link], [Link], [Link], [Link]
FROM Customers
FULL OUTER JOIN Orders
ON [Link] = Orders.Customer_ID
WHERE [Link] IS NULL OR Orders.Customer_ID IS NULL
Result
Name Email Product Price
John john@[Link]
Red 29.99
Union Clause
UNION combines the result of two or more SELECT statements. Each SELECT statement
must have the same number and order of fields and each filed must be of a similar data type.
Query
SELECT Name
FROM Customers
UNION
SELECT Product AS Name
FROM Orders
Result
Name
John
Mike
Susan
Green
Blue
Red
Limitations
Marketing Cloud does not support all standard SQL features. SQL can only be used to retrieve
data, but not to manipulate it.
• GOTO
• PRINT
• EXEC
• TEXT and IMAGE
• Variables
• Cursors
• User-Defined Functions
• Transaction and Locking
• Temporary Tables and Common Table Expressions
• Open-ended comment designations such as –
Extract Activity
Data Extract activities in Automation Studio allow to extract and export data from Marketing
Cloud to different file locations.
The output files of Data Extract activities can be further processed by File Transfer activities to
unzip, encrypt, or move files to external locations.
Following extract types can be provisioned and used in Data Extract activities:
1. Convert XML
2. Data Extension Extract
3. GlobalUnsubImport
4. Microsoft Dynamics CRM Tracking Data Extract
5. Tracking Extract
6. Zip
7. Security Action/Event Audit Log
Convert XML
The Convert XML extract type allows to convert simple XML files to delimited files. The
delimited files can then be used in Import File activities to load data into data extensions or
lists.
Supported output formats include comma-separated (CSV), tab-delimited, any other other
character-delimited files.
The Data Extension Extract type allows to store data extension information in delimited files.
The delimited files can then be used in File Transfer activities using the Move a File From
Safehouse method, to move the extract to another file location.
Supported output formats include comma-separated (CSV), tab-delimited, any other other
character-delimited files.
GlobalUnsubImport
The GlobalUnsubImport extract type allows to import data into the global unsubscribe list for
Enterprise 1.0 accounts.
This extract type is only supported by Enterprise 1.0 accounts with GlobalUnsub feature
enabled.
This data extract can be used to export tracking data relevant for Marketing Cloud accounts
using the Marketing Cloud for Microsoft Dynamics CRM integration.
Tracking Extract
The Tracking Extract type allows to export granular tracking data from Email Studio. This data
can be imported by external systems.
Tracking Extracts can include different types of tracking data. The data including in an extract
can be selected in the Data Extract activity. Following tracking data can be included:
• Attributes
• Bounces
• ClickImpression
• Clicks
• Conversions
• ListMembershipChanges
• Lists
• NotSent
• Opens
• SendImpression
• SendJobImpression
• SendJobs
• Sent
• Spam Complaints
• StatusChanges
• Subscribers
• Surveys
• Unsubscribes
Marketing Cloud outputs tracking extract data in a zip file. Supported output formats for files
included in the zip include comma-separated (CSV), tab-delimited, and XML files.
Zip
The Zip data extract type allows to compress exported files on file locations such as FTP
severs.
The Security Action Audit Log extract type can be used to extract log data related to user
operations from Marketing Cloud.
The Security Event Audit Log extract type allows to extract authentication related log data
from Marketing Cloud.
SQL Best Practices
Marketing Cloud cancels SQL Query activities in Automation Studio after 30 minutes. Best
practice should be followed to avoid timeouts, and reduce query complexities and data
volumes.
Data Volume
Querying all fields of a Data Extension using a wildcard like SELECT * should be avoided.
Instead, only relevant fields should be queried by specifying the necessary field names in the
SELECT statement. This also avoids issues with cached data, as a result of which new fields
might not be returned with SELECT * statements.
Return results from SELECT statements can be further reduced by using SELECT DISTINCT.
This returns only unique rows.
Similarly, SELECT TOP can be used to limit results to fixed number or percentage of records.
This query should be combined with ORDER BY to predict the results returned.
When querying tracking data, a cap of 6 months or less should be implemented. This reduces
the amount of data being processed and thus improves performance.
Query Efficiency
To optimze the performance of WHERE and JOIN clauses in SELECT statements, the primary
key of Data Extensions should be used as the predicate.
Non-searchable operators include OR, NOT, NOT EXISTS, NOT IN, NOT LIKE. As an
example, WHERE NOT age > 18 AND age < 18 should be replace with WHERE age = 18.
If a SQL Query activity is added to a step within an Automation Studio workflow, no other
activity should be executed in the same step. This ensures that the query can complete without
impacting concurrent activities in the same step.
The result of SQL Query activities is stored in target Data Extensions. The fields and field
types of the target Data Extension should match the expected values from the query.
Furthermore, the field lengths of target Data Extensions should be minimised. The total size of
Data Extension definitions should be kept below 8000 bytes (i.e. fields x length). Keeping the
table definition size low can significantly improvements in data retrieval performance.
Indexing
Intermediate Tables
Complex SQL queries should be split into multiple SQL Query activities in Automation
Studio. To chain those SQL Query activities, intermediate Data Extensions can be created.
Each SQL Query activity in the chain can store its results in an intermediate Data Extension,
which is used as input for the next SQL Query activity.
Intermediate Data Extensions help to reduce the complexity and data volume of individual
SQL Query activities, and allows Marketing Cloud to complete each activity one by one
without timeouts.
Send Logs
Send Logs are custom Data Extensions which are populated with email tracking data at send
time. They allow to track information for historical reporting, segmentation, and decision
making.
Standard Fields
Following standard fields are included by the Data Extension template and cannot be changed:
• JobID
• ListID
• BatchID
• SubID (i.e. subscriber id)
• TriggeredSendID
• ErrorCode
Custom Fields
Custom fields can be added to Send Log Data Extensions, to track additional information at
send time. For example, custom fields for subscriber or campaign attributes could be added
and used later for segmentation.
The name of the additional Send Log fields must exactly match the field names of the sendable
Data Extension, Subscriber Attributes, or calculated values used in the email send. Marketing
Cloud automatically maps those fields and logs values at send time.
Usage
To use a Send Log, the Retain Send Log Data option must be checked when sending an email.
This feature is available for emails sent using Content Builder, User-Initiated Sends, and
Triggered Email Send interactions.
Use Cases
Send Logs can be used to track and retain information which could be changed after an email
send. Retaining those values allows historical reporting for future segmentation and decision
making.
Send Log data can also be combined with standard tracking information for segmentation. In
addition, Send Logs can be processed with AMPscript or SSJS at send time for content
personalization based on historical information.
Best Practices
Best practices should be followed to limit the amount of tracking data stored in Send Logs, and
to reduce impact on send performance.
Only relevant custom fields should be added to Send Log Data Extensions and the length of
the custom fields should be minimized. In total no more than 10 custom fields should be
added.
Furthermore, a Data Retention Policy should be defined for Send Log Data Extension. Send
Log tracking data should not be stored longer than 10 days.
The number of parallel activities performed on a single Send Log Data Extension should be
limited. A Send Log Data Extension should not be queried at the same time as it is populated
during sends. Instead Send Logs should be copied to separate Data Extension for heavy
processing.
Contact Deletion
Depending on the tenant type, Contact deletions cascade differently in Marketing Cloud
accounts. In addition, Contact deletions does not propagate across all Marketing Cloud
applications and Data Extensions.
Tenant Types
Depending on the tenant type, Contact deletions cascade differently in Marketing Cloud:
Core One No
Agency One No
An Enterprise 2.0 tenant includes a top-level account and all child accounts consisting of
business units.
Contact deletions in Enterprise 2.0 tenants must originate from the top-level account and
deletions cascade to all child accounts.
Enterprise Tenant
An Enterprise tenant includes a top-level account and all child accounts consisting of On-
Your-Behalf and Lock & Publish business units.
Contact deletions in On-Your-Behalf accounts must originate from the top-level account and
deletions cascade to all child accounts.
Contact deletions in Lock & Publish accounts are isolated and don't cascade outside it.
Core Tenant
Contact deletions in Core tenants are isolated and don't cascade outside it.
Agency Tenant
An Agency tenant includes a single top-level account. All child accounts are considered as
separate Agency tenants.
Contact deletions in Agency tenants are isolated and don't cascade outside it.
Applications
Deleted Contacts are removed from following Marketing Cloud applications:
• Contact Builder
• Email Studio
• MobileConnect
• MobilePush
• Journey Builder
• Audience Builder
• GroupConnect
• Einstein (Predictive Intelligence)
MobileConnect and MobilePush do not preserve opt-in information for deleted Contacts. If a
deleted Contact gets imported to Marketing Cloud again, exiting preferences need to be
resorted from a backup or recollected to ensure regulatory compliance.
Email Studio retains unsubscribe information from email addresses after Contact deletion. This
is to ensure that previously unsubscribed email addresses do not receive messages from
Marketing Cloud in the future.
Data Extension
Deleted Contacts are removed from sendable Data Extensions which are linked to Contacts in
an Attribute Group.
However, Contacts are not automatically removed from non-sendable Data Extensions. They
are also not removed from aggregated tracking data stored in Send Logs.
Security
Security Features
Marketing Cloud provides different features to encrypt data and resources. Data can be
encrypted at system or application layer and on field or database level.
Key Management
Marketing Cloud encryption keys can be stored with the Key Management tool. This tool
allows manage keys used to encrypt and decrypt data, digitally sign email messages, and
implement SAML single-sign on.
1. Asymmetric
2. Symmetric
3. Initialization Vector
4. Salt
5. SSH
6. SSO Metadata
Key Management
AMPscript Encryption
AMPscript provides functions to programatically hash, encrypt, and decrypt data. Keys stored
in Key Management can be used in encryption and decryption functions.
• Decrypt Symmetric
• Encrypt Symmetric
• Base 64 Decode
• Base 64 Encode
• GUID
• MD5
• SHA1
• SHA256
• SHA512
As an example, the following AMPscript code uses keys stored in the Key Management tool,
to encrypt and decrypt a string.
%%[
SET @string = "My secret"
A single File Transfer activity can be used to encrypt a file in the Marketing Cloud account,
and move the encrypted file to an external file location. Supported encryption methods are
PGP or GPG.
Two File Transfer activities are needed to decrypt a file located on an external file location.
The first activity decrypts the file in the Safehouse location, and the second activity copies the
decrypted file from the SAfehouse to the Marketing Cloud account.
SSL Encryption
The standard Sender Authentication Package (SAP) does not include SSL certificates to
securely serve resources over the encrypted HTTPS protocol. However, SSL certificates can be
purchased separately and installed for the SAP domain.
Following resource types can be secured with SSL certificates and HTTPS:
To secure both types of resources, two separate SSL certificates need to be purchased and
installed. This is because Marketing Cloud uses an external content delivery network (CDN)
for serving portfolio content.
Field-Level Encryption
The Field-Level Encryption method uses a symmetric key and initialization vector stored in
the Key Management tool within Marketing Cloud. This allows to import encrypted data into
Marketing Cloud.
The Transparent Data Encryption method uses a symmetric or asymmetric key stored in the
boot record of the SQL server. Encryption and decryption of database files is performed real-
time, which allows to implement encryption without modifying existing application code.
The Transparent Data Encryption protects from physical data theft of media containing
database files, but does not provide application-layer or field-level encryption. It is also not
supported for data stored by the Einstein (Predictive Intelligence), Audience Builder, and
Social Studio applications.
Tokenized Sending
Tokenized Sending is an application-layer encryption method that uses tokens to mask data at
field-level. Tokens can be resolved to actual values at send time using API calls to an external
source system.
The Tokenized Sending method can be used to prevent sensitive data from being stored in
Marketing Cloud. Sensitive data can be tokenized in an external source system and loaded into
Marketing Cloud. Tokens are then resolved to actual values at send time. The Messaging
Engine calls a Token Resolve API towards the source system to replace the tokens with actual
values from the source system.
Secure Storage
OAuth refresh tokens as access tokens must be treated as credentials. Only the refresh token
can be securely stored in external applications. The access token must be kept in system
memory and a new token should only be requested when needed.
Secure in Transit
TLS must be enforced for API calls towards Marketing Cloud APIs. The OAuth access token
must only be transferred as authorisation header and never as query parameter in HTTP
requests.
Access Control
User sessions and permissions in external applications must to be verified before providing
access to any restricted data.
HTML Injection
HTML Injections are attacks in which HTML code is injected into vulnerable web
applications, which is then displayed as legitimate content.
SQL Injection
SQL Injections are attacks in which input data is manipulated on vulnerable applications, to
inject SQL commands which impact the execution of predefined SQL queries.
Arbitrary Redirect
Arbitrary Redirects describe attacks in which attackers manipulate vulnerable redirection
pages, to trick users into following trustworthy looking URLs which actually redirects them to
malicious web pages.