IDMC Best Practices and Standards

Revision History

Version Date Revised By Description of Revision

1.0 08/15/2023 Kandarp Sengar Initial Release

3/20/2024 Page 2 of 27
Contents
Purpose.................................................................................................................................................4
Overview...............................................................................................................................................5
IDMC Architecture................................................................................................................................6
High Availability Architecture............................................................................................................8
Runtime Environment........................................................................................................................8
Informatica Cloud Secure Agent....................................................................................................8
Secure Agent\Group......................................................................................................................8
Secure agent Architecture.................................................................................................................9
Organization and Secure agent architecture diagram.......................................................................9
On-boarding new applications (projects)...........................................................................................10
Users\Groups\Roles........................................................................................................................10
Project Setup...................................................................................................................................10
Development Best Practices...............................................................................................................11
Adherence to Standards and Best Practices....................................................................................11
Use Standard Naming Conventions.................................................................................................11
Connections.................................................................................................................................11
Environments..............................................................................................................................12
Secure Agent................................................................................................................................13
Assets and Components..............................................................................................................13
Mapping Transformations...........................................................................................................13
Task Flows....................................................................................................................................14
Leveraging Code Reusability and Concurrency................................................................................15
Error Handling Strategies....................................................................................................................16
Performance Tuning...........................................................................................................................16
Maximizing Performance using Pushdown Optimization..................................................................19
Migration Procedures.........................................................................................................................20
Native GitHub Integration with IDMC..............................................................................................21
Recommended Configuration and Best Practices............................................................................22
Setup phase:................................................................................................................................22
Development...............................................................................................................................22
Reference:.......................................................................................................................................23
Monitoring..........................................................................................................................................23
Operational Insights........................................................................................................................23
Informatica Network..........................................................................................................................24
Additional References.........................................................................................................................25

3/20/2024 Page 3 of 27
Additional Development Pointers......................................................................................................26

3/20/2024 Page 4 of 27
Purpose
The purpose of this best practices document is to provide standards, guidelines, and
recommendations on design, development, and optimization techniques for IDMC Development
team. As new versions are released this document will need to be updated to reflect new features
and recommended Best Practices.

Overview
To ensure consistency and facilitate easy maintenance postproduction it is important to define and
agree on development standards and best practices before development work has begun.

These standards and best practices will define the ground rules for the development team.
Standards can range in items from naming conventions to documentation standards to error
handling standards. Development work should adhere to these standards throughout the lifecycle
and new team members will be able to reference these standards to understand the requirements
placed upon the design and build activities.

While it may be impossible to predict every pattern and every type of object used in a project during
an early stage, it is important to define standards for the areas you are certain will be part of the
project and to continually update these standards throughout the project as necessary. Standards
should be published in a common area that is available to the full team of project developers
including internal staff, consultants, and offshore developers. Expectations should be set early on
that all developers should follow these standards as part of their development process. New team
members should be required to review these documents before beginning any design or
development work.

3/20/2024 Page 5 of 27
IDMC Architecture

IDMC Architecture components

❖ IDMC is an iPaaS-based application. This means that you can access it from any machine with
Internet access and a web browser application.

❖ Users interact with IDMC through a web client via the HTTPS protocol.

❖ The IDMC host contains services such as Data Integration, Application Integration, API
Management, and Integration Hub that can be accessed by users. IDMC implements security as a
design principle. These services are built on microservices and multi-tenant repositories on the back
end with a common login page and user interface shell on the front end.

❖ The customer employee acting as an administrator configures and manages the security of the
customer organization.

❖ Integration developers and business users use the web-based visual designer and wizard tools of
Intelligent Cloud Services to design integration and data management flows. IDMC also provides
organization (client) managed Secure agents as well.

❖ All metadata that defines this design is stored in a multitenant metadata repository in the IDMC
host.

3/20/2024 Page 6 of 27
❖ Unique tenant IDs and other means are used to ensure separation of the metadata across
tenants.

❖ The Secure Agent and/or the Informatica managed cloud runtime executes customer authored
integrations and processes, and thereby connect, transform, and move the data among source and
target data applications in batch and real time.

❖ Depending on customer choice, processing of data can be done by customer managed Secure
Agents or the Informatica managed cloud runtime. Customer-managed Secure Agents can be
deployed on-premise, behind the firewall, or in public/private clouds.

❖ The Informatica managed cloud runtime is deployed on the IDMC host.

❖ The multi-tenant data repository backend is used for securely storing customer data on the IDMC
host. Certain IDMC services like Cloud Integration Hub use the multi-tenant data repository backend
upon customer request.

❖ The customer employee acting as an administrator configures and manages the security of the
customer organization.

❖ Integration developers and business users use the web-based visual designer and wizard tools of
Intelligent Cloud Services to design integration and data management flows. IDMC also provides
organization (client) managed Secure agents as well.

3/20/2024 Page 7 of 27
High Availability Architecture

Each Informatica data centre uses (n+1) configuration at all levels of infrastructure. At any time, if
there is a system failure, another system is assigned in lieu of the failed system. Each data centre is
also paired with disaster recovery region. In case of catastrophic failure, the primary region will be
failed over to a disaster recovery region.

Runtime Environment

A runtime environment is the execution platform that runs a data integration or application
integration task. To run tasks in your organization, you must have at least one runtime environment
set-up in your organization.

Informatica Cloud Secure Agent

The Informatica Cloud Secure Agent is a type of runtime environment. In addition to the Informatica
Cloud application and repository, there is a local component called the Informatica Cloud Secure
Agent.

The Secure Agent is a lightweight, self-upgrading program that runs on a machine inside the
organization network. The Secure Agent is responsible for moving data directly from the source to
the target. It allows users to access local resources, for example, databases or applications that
reside behind the firewall.

Informatica Cloud Secure Agent is the local agent that runs the tasks. Therefore, application data
never gets staged or run through the Informatica Cloud servers. Each organizations data remains
completely secure and stays behind the firewall. It is available for Windows and Linux platforms.

Multiple agents can be installed within the organizations network. Only one agent is installed per
machine/server. The secure agent is automatically linked to the IDMC Org that it is installed from.

Secure Agent\Group

Secure Agent is a lightweight runtime environment installed behind customer firewall to execute the
Data integration tasks

Secure Agent belongs to a Secure Agent group. By default, when you create a Secure Agent, it is
added to its own group. If you have the Secure Agent Cluster license, you can add multiple
homogeneous agents i.e. multiple windows agent together or Linux agents together to form
a Secure Agent group.

The main advantage of having agent groups is:

1. High Availability - Your task will not fail because of not having an agent rather if there is another
agent that is available from the agent group, the task will run using that another agent.
Note: that the switch to another agent happens only when the task is about to run and if one of the agents is not present, it will be routed
to next available agent in the agent group. If the failure happens during execution of the task, the task will not be automatically switched
to another agent, rather the task will fail, and this is expected behaviour.

3/20/2024 Page 8 of 27
2. Load Balancing: The execution of various task will be distributed to various agents in the agent
group in round-robin fashion automatically. It's not possible to override the task assignment and the
task gets assigned in round-robin fashion automatically.

Secure agent Architecture

Organization and Secure agent architecture diagram

3/20/2024 Page 9 of 27
On-boarding new applications (projects)

Users\Groups\Roles

When on-boarding new applications areas like Users, User Groups and User Roles need to be
determined between the application team and the IDMC Admin team.

Project Setup

Once the Users\Groups \Roles are determined, create a Main project, projects specific to users for
the development and provide the necessary roles\privileges to the project group.

In example below RADA is the main project and user Chandra will have z_Chandra project to work
on his own development and testing.

3/20/2024 Page 10 of 27
Development Best Practices

Maintain Descriptions – The best way to ensure the code can be understood by others is to provide
appropriate descriptions of mappings, transformations, and tasks. Good annotation will not only
ensure that others can more easily interpret the code, but also help them understand how the
transformations are working. Writing a good description also serves as a good way to double check
the logic and ensure what we have created satisfies the requirements.

Adherence to Standards and Best Practices

Depending on the size and nature of the team, it can be difficult to ensure that all team members
are following standards & Best Practices. As part of defining the standards defining ways to measure
the adherence to the standards may also be considered.

Methods could include any of the following:

 Formal review of objects with development team members or standards leads

 Formal production move process that reviews changes/new objects against required
standards before migration to production

Use Standard Naming Conventions

One of the easiest ways to simplify readability of any mapping is to create and adhere to standards
for naming conventions. This should include all aspects of the process, including the mapping name,
the transformation names, the port or field names within the transformations and tasks. Below is a
suggested naming convention for various objects.

Connections

Environment Naming Convention (Post or Prefix)

Oracle ORA_<Schema Name>

Flat File FF_<Folder Name>

MS Access MS_Access_<Database Name>

MS Azure ADLS MS_AZ_ADLS_<Database Name>

FTP/SFTP FTP_<Application Name>_<Folder Name>

Salesforce SFDC_<Instance Name>

3/20/2024 Page 11 of 27
 Oracle Database Ingestion ORA_DB_IN_<Database Name>

Microsoft Excel MS_EXCEL_<Folder Name>

PostgreSQL PG_SQL_<Database Name>

MS SQlServer MS_SQL_<Database Name>

Web Service WS_<Application Name>_<Web Service Name>

Hive HIVE_<Application Name>

REST V2 RESTV2_<Application Name>

MARKETO MKTO_<Application Name>

Amazon Redshift AZ_RESHFT_<Application Name>

DB2 DB2_<Application Name>

Snowflake SNFLK_<Application Name>

Environments

Environment Naming Conventions (Prefix or Postfix)

Development DEV
Testing, System Integration Testing, TEST\SIT\QA
Quality Assurance
User Acceptance Testing UAT
Performance Testing PT\PFT\PERF
Pre-Production PREPROD
Production PD\PRD\PROD

3/20/2024 Page 12 of 27
Secure Agent

Secure Agent Abbreviation Naming Convention Example

Secure Agent SA SA_[Environment]_[Ecosystem]_[Server] SA_Azure_Linux01
Secure Agent Group SAG SAG_[Environment]_[Ecosystem]_[Server SAG_Azure_Linux
Set]

Assets and Components

One of the easiest ways to simplify readability of any project is to create and adhere to standards for
naming conventions. This should include all aspects of the process. Below is a suggested naming
convention for various objects:

Informatica Cloud Object Naming Conventions

Mapping m_[project_folder]_[Source]_[Object]_[Target]
_[Object]
Masking Task mskt_[project_folder]_[Source]_[Object]_
[Target]_[Object]
Mass Ingestion Task mit_[project_folder]_[Source]_[Object]_[T
arget]_[Object]
Synchronization Task st_[project_folder]_[Source]_[Object]_[Target]
_[Object]
PowerCenter Task pct_[project_folder]_[Source]_[Object]_[Targe
t]_[Object]
Replication Task drs_[project_folder]_[Source]_[Object]_[Targe
t]_[Object]
Mapping Task mtt_[project_folder]_[Source]_[Object]_[Targe
t]_[Object]
TaskFlow tf_[project_folder]_[Source]_[Target]_[Operati
on Type]
Scheduler schedule_[Frequency]

Mapping Transformations
Transformations are a part of a mapping that represent the operations that anyone wants to
perform on data. Transformations also define how data enters each transformation. Each
transformation performs a specific function. For example, a Source transformation reads data from a
source, and an Expression transformation performs row-level calculations.

 Use underscore to separate words

 Use consistent naming for different type of mappings
 Name structure:
 Mapping with single source:
M_<SOURCE>_<TARGET>_<NAME>_<VERSION>

3/20/2024 Page 13 of 27
  Mapping with multiple sources:
M_<SOURCE>_<ENTITY>_to_<TARGET>_<NAME>_<VE
RSION>

Examples:

 M_CDI_Accounts_FF_incremental
 M_CDI_Accounts_Contacts_to_SFDC

Task Flows

 Use underscore to separate words

 Use consistent and readable(mix case) naming
 Name structure: TF_<NAME>_<VERSION>

Examples:

 TF_CDI_Acount_Contacts_to_SFDC
 TF_Hierarchy_Change_Processing

Informatica Cloud Naming Conventions

Object
Source Src_{SOURCE_NAME}_{OBJECT}
Target Tgt_{update_types(s)}_{TARGET_NAME}
Sorter Src_{DESCRIPTION}
Aggregator Agg_{FUNCTION}
Expression Exp_{FUNCTION}
Filter Fil_ or Filt_{FUNCTION}
Joiner Jnr_{DESCRIPTION}
Lookup Lkp_{TABLE_NAME} or suffix with _{descriptor}
Normalizer Nrm_{FUNCTION}
Router Rtr_{DESCRIPTION}
Mapplet Mpl_{DESCRIPTION}
Rank Rnk_{DESCRIPTION}
Hierarchical Parser HPc_{DESCRIPTION}
Hierarchical Builder HBd_{DESCRIPTION}
Java Jva_{DESCRIPTION}
Sequence Sqn_{DESCRIPTION}
Generator
Transaction Tcl_{DESCRIPTION}
Control
Union Unn_{DESCRIPTION}
Webservice Wsv_{DESCRIPTION}
Cleanse Cln_{DESCRIPTION}
Mask Msk_{DESCRIPTION}
Deduplicate Ddp_{DESCRIPTION}

3/20/2024 Page 14 of 27
 Parse Prs_{DESCRIPTION}
Rule Specification Rsp_{DESCRIPTION}
Verifier Vrf_{DESCRIPTION}
Profile Profile_{Source}_{Table}_{DESCRIPTION}

Transformations Object Ports

Prefix Prefix Description

i_ For Input ports
o_ for Output ports
io_ For Input/Output Ports
var_ For Variable Ports
Lkp For returns from lookups
Mplt For returns from mapplets

Transformations – Below are some recommendations to make best use of available options in
transformations while developing mappings:

 Join all source tables in a single source if they all can be accessed from the same connection.
Also apply filters if any on the source to restrict the number of records getting processed.
Use only the required ports from source to subsequent transformations. However, be
careful with the amount of logic that is in the SQL Override so that is readable for
troubleshooting.
 Use variable ports to limit the number of redundant calculations or store values in complex
transformation, avoid datatype conversions, use operators (||, +, / etc.) instead of functions
and keep in mind that numeric operations are generally faster than string operations.
 If all the filter conditions can’t be applied at Source level, use filter or router transformations
as close to the source as possible. It is recommended to use aggregator after filter if possible
and to use sorted input for better performance.
 For optimizing Joiner transformation, designate the master source as the source with fewer
rows and join sorted data when possible.
 Call Unconnected Lookup in expressions if it is not required to be called for every row.
Replace large lookup tables with joins when possible. Review the database objects and add
indexes to database columns when possible.
 Use integer values in place of other datatypes when performing comparisons using Lookup
and Filter transformations when possible. If there are multiple Lookup conditions, make the
condition with the “=” sign first to optimize the lookup performance.
 The incoming field name should not be same as any port name of lookup transformation. If
not followed, the transformation and mapping will be valid, but the task will terminate
abruptly while running.
 Lookups to cloud applications such as SFDC can be expensive, as it will invoke the APIs
multiple times. This can be avoided by using Flat File Lookup on a downloaded file from the
corresponding application.

Leveraging Code Reusability and Concurrency

Using Parameters, Variable and Parameter Files

Use Parameters – Using parameters is very beneficial, if used appropriately it can allow to load data
for any time window making the code scalable. It is also recommended to create multiple mappings
instead of few complex ones. Use Staging Area and try to keep the processing logic as clear and
simple as possible.

3/20/2024 Page 15 of 27
Error Handling Strategies

User Defined Exception:

Data issues critical to the data quality, which might get loaded to the database
unless explicitly checked for quality.
Error Handling Functions
ERROR ( ) – Skip the row and issue user specified error message
IIF(TRANS_DATA>SYSDATE,ERROR(‘Invalid Transaction Date’))
ABORT ( ) – Stops the task with specified error message
IIF(ISNULL(CREDIT_CARD_NB),ABORT(‘Empty Card Number’))

Non-Fatal Exception:
Error which would get ignored by Informatica and cause the records dropout from target table
otherwise handled in the ETL logic.
Default Value Setting
Setting the default value is a good way to handle exception due to NULL and unexpected
transformation errors.
Row Error Logging
Cloud connectors internally handles the target error and writes it to error log file placed inside the
agent.

Fatal Exception
Errors such as database connection errors, which forces the workflow to stop.
Automating Connection Validation using IDMC REST API
Task Flow design to Stop/Warn on errors.

Performance Tuning
Below are some of the attributes that can be configured for a secure agent to increase the memory
properties for different operations of the secure agent.
 DTM: If you hit java out of memory or java heap space or out of memory or perGen space
error messages at run time of a task (either in session log or activity log or both), then
memory attributes need to be defined as JVMOptions under DTM. The default options will
be JVMOption1, JVMOption2, JVMOption3 and so on. If you have already used all the
available JVMOptions, then you can add more as custom properties on the agent, making
sure the sequence is maintained (if the default ones stop at JVMOption5, then the custom
properties should start with JVMOption6 and should have type as DTM and subtype as
INFO). Each JVMOption can take 1 JVM property.

3/20/2024 Page 16 of 27
The main JAVA memory properties are -Xms**m, -Xmx****m and -XX:MaxPermSize=***m.
Memory options on the attribute named INFA_MEMORY that is under tomcat JRE
Xms***m : The initial value- The Java virtual machine will start with this amount of memory. The
memory value mentioned here will be allocated as soon as the Java process starts.
Since this is the base/ initial value, the value given can be small like 64m or 128m. Since this is just
the initial value, it is just used for initialization of process and the java process will eventually keep
allocating space as and when it needs.

-Xmx****m : The maximum value- the maximum amount of memory the Java virtual machine can
allocate as heap. After the Java process starts, as needed, it will keep allocating more space to store
its objects, the allocation can continue up until it hits this maximum setting, once this maximum
value is hit, the java process will crash complaining about java heap space or out of memory.
Since this is the maximum value to which the Java heap can grow, the value has to be generally
large enough to hold all the java objects and classes. For a 64 bit agent, the -Xmx value has virtually
no boundaries, but is only limited by the system memory ( the amount of memory that the OS can
allocate on the system ). Generally, on a 64 bit agent, the value can be about 1024M or 2048M.
However, if your tasks fail with java heap space or out of memory errors then you will have to
increase it further. The value to which it has to be increased will be a trial and error as the value is
dependent on the amount of data, the classes loaded, how java and the OS stores the data.

XX:MaxPermSize=***m : The maximum permissible size the JVM can use at a time. If the JVM needs
more than the specified amount, then the java process will crash complaining about permGem Space
issue.
This can be set to an average value of about 512M, however make sure that the value is smaller than
-Xmx value. This needs to be increased only when the error is about permGen space.

Optimize JVM configurations

Modify JVM settings based on secure agent machine configuration and workloads.
In System Configuration Details modify below settings appropriately –

DTM JVMOption1 -Xmx1024m

Infa Memory configuration

Update INFA_MEMORY parameter to an optimal setting based on Secure Agent’s configuration

'-Xms512m -Xmx1024m -
Tomcat JRE INFA_MEMORY XX:MaxPermSize=512m'

3/20/2024 Page 17 of 27
Note: Recommendation to host Informatica cloud secure agent in a box with 2 cores and 4
CPUs/Logical Processors at least and minimum 16 GB RAM with at least 250 GB disk space.
[Link]

Optimize DB Read/Write parameters

For high volume batch read and write operations, the below highlighted parameters should be set to
1. By default, it is 0, this should be set to 1

OptimizeOracleSQLServerWrite
Enables array loading, which is a ODBC feature that makes writes to Oracle and SQL Server faster by
preparing the query once and binding it directly with an array of values, generating and issuing
queries much faster to the database. (Oracle and SQL Server connections use ODBC under the hood)
OptimizeODBCRead
Value set to ‘Yes' implies SQLExtendedFetch function is called and ‘No’ implies SQLFetch function is
called. By setting OptimizeODBCRead to Yes, the overall performance of the read operation is
improved, because a set of rows are read at once (rather than one row) and the SQLBindCol
operation is executed once for each set (rather than once for each row).
OptimizeODBCWrite
Enables Array inserts if set to ‘Yes’ which is an ODBC driver feature that allows you to insert more
than one row into a target table simultaneously.

Max DTM Processes

By default, it is set to 2. This should be increased to an optimal value depending on secure agent
configurations. Add this property in Custom Configuration Details for Secure Agent as depicted
below.

3/20/2024 Page 18 of 27
Recommendation: configure the maximum process as 6-7 , agent can support up to 20 -25
concurrent jobs based on server specification.

MCT Session properties

DTM Buffer Size: The DTM buffer size specifies the amount of buffer memory used when the DTM
processes a session.
Default Buffer Block Size: The buffer block size specifies the amount of buffer memory used to move a
block of data from the source to the target.
By default, these values are set to Auto.
In MCTs, DTM buffer size and Block Size should be set to an optimal value for better performance.
Edit these properties as depicted below –

Maximizing Performance using Pushdown Optimization

PushDown Optimization (PDO): PDO is a way to improve the performance by pushing the processing
logic to either source, target or both.
Informatica cloud currently supports PDO for Oracle, SQL Server, MySQL, ODBC and Amazon Redshift
type connections.
It can be enabled on Schedule tab of mapping configuration task setup, at the bottom under
"Advanced session properties", you can see an option named Pushdown Optimization. There are
multiple possible values supported for this parameter.

The possible values for Pushdown optimization property in Mapping configuration task is:

 None. The task processes all transformation logic for the task.
 To Source. The task pushes as much of the transformation logic to the source database as
possible.
 To Target. The task pushes as much of the transformation logic to the target database as
possible.
 Full. The task pushes as much of the transformation logic to the source and target databases
as possible. The task processes any transformation logic that it cannot push to a database.
Applicable when the source and target are the same databases.

3/20/2024 Page 19 of 27
  $$PushdownConfig. The task uses the pushdown optimization type specified in the user-
defined parameter file for the task. If you use $$PushdownConfig, ensure that the user-
defined parameter is configured in the parameter file.

Performance tuning

Partitioning
If partitioning license is enabled, the below partitioning types can be configured in mappings:

Key-range based partitioning for Relational and Redshift sources

Informatica Cloud service passes data to each partition depending on the ranges you specify for each
port. Each of the key ranges defined translates into partition and it will carry this partition
throughout entire mapping like as pass through and it is written in multiple threads at target as well,
so you do not have separate partitioning for Target.

Pass-through Partitioning for Flat-File sources

Informatica Cloud service processes data without Redistributing rows among partitions. All rows in a
single partition stay in that partition after crossing a pass-through partition point. Use pass-through
partitioning when the data throughput is to be increased, but not the number of partitions.

Migration Procedures

Organizations have different needs and may be in different phases of their CI/CD journey. Since no
two organizations are alike informatica provide multiple SDLC options to meet to your varying needs
in IDMC:
Leveraging export/import APIs:

3/20/2024 Page 20 of 27
Targeted towards administrators, devops, and developers alike looking to automate their backups,
and for organizations that already have custom built CI/CD workflows and want to plug IDMC assets
into the mix using IDMC APIs. Using this option requires users to invoke and process IDMC APIs and
requires advanced knowledge about these APIs and structure of the export zip format.
Native GitHub integration:

Targeted towards developers seeking collaborative development, and administrators enforcing a

robust source control management process leveraging their cloud hosted GitHub repository. This
option is ideal for developers seeking to directly source control their IDMC assets from their
development canvas to their GitHub repository.
IDMC asset management CLI utility:

Ideal for administrators, devops, operators primarily focused on deployment automation use cases
and looking to build CI/CD pipelines to promote their IDMC artifacts across different environments
(Dev/Test/Production).

If you are using a Github repository as the enterprise versioning tool. Migration approach that will
be followed is the native Github integration

Native GitHub Integration with IDMC

With the native GitHub integration with IDMC, Informatica provides source control management
capabilities in situ with your development experience. This greatly simplifies the ease of use for
versioning assets in the GitHub repository without needing to leave your development canvas.
This approach is best suited for users who prefer to use their own cloud hosted GitHub repository.
This way, you not only have full control over your repository but also prevent vendor lock-ins. This is
a plug and play offering, meaning you just need to link your GitHub repository to your IDMC
organization. Users can then immediately start performing check-out, check-in, pull, undo checkout,
and unlink actions directly from the IDMC explore page.
***This is applicable to only CDI\CAI service-related assets
The key use cases that this option addresses are:
• Source control management
• Backups
• Patching and fixes
• Rollbacks
• Collaborative development
• Promotion of assets across environments from the GitHub repository
• Release management

3/20/2024 Page 21 of 27
Recommended Configuration and Best Practices

To get you started with a good set of development practices, we have put together the following
best practices when using the native GitHub integration option with IDMC:
Setup phase:
 We recommend maintaining different Development, Test, Staging, and Production
organizations to ensure isolation across environments so that changes meant for say a test
environment do not accidentally get deployed to say a production environment. Such a
sanitized structure also ensures that users do not accidentally push changes to GitHub from
a non-development environment.
 Source as well as Target Org should reside in same PODs
 Development organizations must be configured for “read/write” (push) access to the GitHub
repository.
 Non-development organizations (Test/Staging/Production, etc.) should be configured for
“read-only” access to the GitHub repository.
 We highly recommend that you pair only one development organization with a GitHub
repository at a time. Although any number of non-development organizations can be linked
to the same GitHub repository, if you pair multiple development organizations with one
GitHub repository, then you must ensure that projects do not clash across organizations
because one user’s changes could overwrite another user’s changes.
 Start with a bare repository and make sure you do not have any folder by the name
“Explore” in the GitHub repository that is linked to IDMC because IDMC assets are stored
under “Explore” project.
 As with any good SDLC practice, do not share GitHub user credentials among multiple users.

Development
Project/Folder Setup
 Maintain golden/master copies of projects that are source controlled.

Managing Dependencies

3/20/2024 Page 22 of 27
  Before pulling assets from GitHub into an organization, create connections and runtime
environments in the organization. This allows you to easily and effectively reuse connections
and runtime environments in your design objects.
 Be aware of dependencies across projects. Make sure reusable objects such as mappings are
present in the repository and organization before, they get used, as IDMC does not allow
saving an asset such as mapping task when the dependent mapping is not present.

Check-in/Check-out
 Identify all dependencies before performing a check out on reusable objects like mappings,
mapplets, and user defined functions.
 Check out all dependent objects in one operation. (For example, when modifying mapping,
check out the mapping and all mapping tasks that call this mapping in one operation.) if
panning to migrate all these assets.
 Check in all dependent objects as part of a single commit if planning to migrate all these
assets.
 Use the comment field when performing a check in. For example, you could use a Release
Tag name in the Summary field and enter more descriptive comments in the Description
field. This way, the Git Summary field in IDMC will show the release tag associated with the
asset.

Cautions:
 Exercise caution or avoid deleting, moving, and renaming source-controlled objects that are
used by other assets. If not done correctly, references could be broken.
 For better performance with the GitHub repository service, when performing bulk check in
and bulk check out operations, limit the bulk operation set to under 1000.
 Informatica Cloud does not migrate schedule information with tasks or task flows. After you
migrate a task or task flow, you can assign it to a schedule defined in the target organization.
 As a security measure, Informatica Cloud migrates connections without passwords or
security tokens. Configure connection passwords and security tokens in the target
organization after migration.

Reference:

Refer to best practice document for source control in IDMC under below link

[Link]
language=en_US&type=external

Monitoring

Operational Insights

3/20/2024 Page 23 of 27
Operational Insights is a service in Informatica Intelligent Cloud Services that gives you visibility into
the performance and operational efficiency of your Informatica infrastructure. Use Operational
Insights to monitor your Informatica products on-premises and in Informatica Intelligent Cloud
Services.

Informatica Operational Insights highlights:

 Comprehensive monitoring statistics allow you to quickly assess the health of your Informatica
assets
 Data processing analytics based on job runs and data processing statistics enable you to assess
usage of your investment in Informatica.
 Run-time job execution, workflow, and task metrics allow you to troubleshoot performance
degradation and execution failures
 Resource consumption analytics help you predict when you need to increase capacity or
reallocate resources
 Email notifications alert you about problems within a Data Engineering Integration, Data Quality,
or PowerCenter domain or with a Secure Agent
 Insights in the form of recommendations and anomalous workflow detection help you identify
errors and abnormal behaviour within a domain

Informatica Network
One place shop for all the support needs.

Informatica provides you with a range of product resources through the Informatica Network and
other online portals. Use the resources to get the most from your Informatica products and solutions
and to learn from other Informatica users and subject matter experts.

• Informatica Network: [Link]

• Knowledge Base

• Documentation

• Communities

• PAM\Velocity

• Marketplace

• Global Customer Support

3/20/2024 Page 24 of 27
Additional References

IICS Minimum requirements and best practices when installing Informatica Cloud Secure Agent

FAQ: Secure Agent Cluster related queries in Informatica Cloud

FAQ: Questions on Secure Agent Group in Informatica Cloud (IICS)

How to library for connectors:

[Link]
[Link]#howtolibrary

Is possible to restrict a specific group of users to a secure agent group?

Yes, it is possible to restrict permission on secure agent group\agent for a specific users\groups

FAQ: All about Source Control (Version Control) in IICS

[Link]

3/20/2024 Page 25 of 27
FAQs on DR implementation for Secure Agent in IICS

[Link]
language=en_US&type=external

Cloud Data Integration logs for troubleshooting

[Link]

FAQ: Why Structure Parser is generating more number of output groups compared to Hierarchy
Parser?

[Link]

Structure Parser is not showing the path to the elements unlike Hierarchy Parser in CDI

[Link]
elements?language=en_US&type=external

FAQ: Can Hierarchy Parser transformation parse source file greater than 2GB in IICS?

[Link]
language=en_US&type=external

FAQ: What is the difference between XML Generator in PC and Hierarchical Builder in IICS?

[Link]

Additional Development Pointers

1. Create a Naming Conventions document detailing the standard naming conventions for
different types of Connections, Tasks, Taskflows and schedules within Informatica
Cloud. All the developers should rigorously follow these naming conventions. It has been
observed that when multiple people are working simultaneously, they tend to follow their
own naming standards and at the end, there are lot of tasks and it is very difficult to
identify tasks and administrator has to spend a good amount of time identifying the
correct tasks.
2. Add meaningful description to all your tasks, connections, taskflows and schedules in
such a way that they convey the purpose of their use and thus do not create confusion to
other users.
3. The machine the Informatica Cloud Secure Agent runs on must always be on. It must not
be in a sleep mode or in “idle”. This might be indicated when the Agent status fluctuates
between an active/inactive state. Make sure this computer is on or re-install the agent on a
computer that stays on continuously.
4. If you are using the CSV files as source or target, make sure that you match up the date
format in the associated connection for flat files, by dropping down the Date Format list
and choosing the matching format.
And If there isn’t a matching format in the drop down list, then you will need to explicitly
format the date in Step 5 of Data Synchronization task by using a transformation function
called TO_DATE.
5. If there is a requirement of performing a lookup on Salesforce objects, then do not create
a direct lookup. A direct lookup on Salesforce object will call the Salesforce object for

3/20/2024 Page 26 of 27
 each record processed and thus performance will decrease considerably. Instead, write the
data of Salesforce object in a flat file and then use the flat file for doing the lookup.
6. For incremental processing, use the “$LastRunTime” and “$LastRunDate” variables in
conjunction with a source field of “date” type. Informatica Cloud supports and maintains
these variables automatically. For example, if your source has a field called
LASTMODIFIEDDATE, you could set up your filter such that LASTMODIFIEDDATE
> $LastRunTime. If your schedule then runs the task on a daily basis, that means each day
you will only have to deal with the new/changed records from the previous day, instead of
worrying about ALL records.
7. In a multi-user environment where the number of tasks to be created is very high, create
Views based on logical groups for viewing similar group task in a single task view.
Similarly you can create views for connection and taskflows.
8. SYSDATE is the current datetime you can use to denote the current date and time.
9. Use logical operators like IIF and DECODE to encode conditional logic in the expression
in Step 5 of Data Synchronization task.
10. Know E-R diagram of the application you are loading to. Here is a good example of
Salesforce [Link]
sforce_api_erd_majors.htm Accounts, Contacts, and Opportunity are the most used objects in
Salesforce, and knowing the relationship between them is important.
11. Use different sandboxes for different projects, departments, geographies, and LOB for
development.
12. Most of the editions can have unlimited users, so get everyone an account.
13. Use the alerting feature to get failed or warning emails and download the Informatica
Administration App to monitor jobs from tablets and smartphones.
14. When you configure or run tasks that use Flat File or FTP/SFTP connection types, the Secure
Agent might require permissions to read or write information to related directories.
15. When you create FTP/SFTP connection, you need to provide a local directory path (accessible
by agents). While creating a task with FTP/SFTP as source or target, you need to place a copy
of the file (metadata only) in the local directory which you have specified during creation of
connection.
16. While creating a schedule, please keep in mind the time zone differences.
17. For creating a Salesforce sandbox connections, change the service url
from [Link] to: [Link]
18. For relational connections, make sure the agent can access the database - it needs to be on the
same named network.

3/20/2024 Page 27 of 27