SAP TPO Report for PepsiCo System
SAP TPO Report for PepsiCo System
Confidential
Database Oracle
This report contains confidential customer data and may be viewed only by SAP DBS employees, authorized SAP
partners, and customer employees. Do not distribute it to other parties.
SAP TPO for PO system PepsiCo, Inc 22.05.2025
1 Table of Contents
1 TABLE OF CONTENTS ........................................................................................................................................ 2
2 SUMMARY ............................................................................................................................................................ 4
2.1 CONTENT OF THIS DOCUMENT ............................................................................................................4
2.2 SERVICE PLAN ..................................................................................................................................5
3 PERFORMED CHECKS ........................................................................................................................................ 6
4 SAP PO SYSTEM LANDSCAPE .......................................................................................................................... 7
5 ISSUES AND RECOMMENDATIONS ................................................................................................................... 8
5.1 ISSUE 1: PI MESSAGE MONITOR PERFORMANCE ISSUES .......................................................................8
5.2 ISSUE 2: DISABLE LM CONFIGURATION WIZARD ............................................................................... 10
5.3 ISSUE 3: MISSING SECURITY NOTE IN PX3 ....................................................................................... 11
5.4 ISSUE 4: SCHEDULE IDOC MONITOR RELATED HOUSEKEEPING JOB ..................................................... 11
5.5 ISSUE 5: HIGH DB SIZE FOR TABLE BC_MSG_DUP_CHECK .......................................................... 12
5.6 ISSUE 6: MESSAGE COMPRESSION NOT ENABLED ............................................................................. 13
6 SOFTWARE CONFIGURATION.......................................................................................................................... 14
7 SYSTEM OVERVIEW .......................................................................................................................................... 16
7.1 PERSISTENCE LAYER OVERVIEW ..................................................................................................... 16
DB and Tables size/entries ............................................................................................................................ 16
7.2 CONFIGURATION OVERVIEW............................................................................................................ 16
Channel types ............................................................................................................................................... 16
Mappings....................................................................................................................................................... 18
Adapter Modules ........................................................................................................................................... 18
Adapter Engine Scheduler ............................................................................................................................ 18
CPA Cache history ........................................................................................................................................ 20
8 SAP PO RECOMMENDED SETTINGS ............................................................................................................... 21
8.1 CONFIGURATION SETTINGS ............................................................................................................. 21
Java Memory ................................................................................................................................................. 21
SAP JVM Parameters ................................................................................................................................... 21
Java Engine .................................................................................................................................................. 22
8.2 PERFORMANCE SETTINGS .............................................................................................................. 24
Java Log Configuration ................................................................................................................................. 24
AAE: Advanced Adapter Engine specific settings .......................................................................................... 25
Adapter Queues ....................................................................................................................................... 25
Receiver Parallelism ................................................................................................................................. 26
Prioritization and Dispatcher Queue ......................................................................................................... 26
Adapter Framework Scheduler ................................................................................................................. 27
Java IDoc Adapter (IDoc_AAE) ................................................................................................................ 28
Polling Channel Watchdog ....................................................................................................................... 30
Mapping Runtime ..................................................................................................................................... 30
Simple XPath Processor ........................................................................................................................... 31
Retry Configuration................................................................................................................................... 31
Large Messages ....................................................................................................................................... 32
EOIO processing ...................................................................................................................................... 33
8.3 HOUSEKEEPING ............................................................................................................................. 33
AAE housekeeping ........................................................................................................................................ 33
Retention Periods ..................................................................................................................................... 33
Housekeeping jobs ................................................................................................................................... 34
Message Compression on SAP PO application level ............................................................................... 35
Table of Contents 2/39
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
2 Summary
The goal of the SAP PO Technical performance optimization service was to identify issues on the configuration
and performance of your system and interfaces. The system in the service scope is the integration hub PX3 which
is based on SAP NetWeaver 7.50 SP 28 on Oracle DB
• Performed Checks
• SAP PO System Landscape
• Issues and Recommendations
• Software Configuration
• System Overview
• SAP PO Recommended Settings
The section “Performed Checks” gives a check list of most important aspects which were reviewed in scope of
SAP TPO service. “SAP PO System Landscape” gives a high-level overview of the landscape architecture and
structure of SAP PO solution. In “Software Configuration” we checked current version of software in PX3 system
and compared with the latest available at Service Marketplace.
“System Overview” provides monitoring statistics about system performance and configuration which was
retrieved and analyzed during service session. The paragraph “Issues and Recommendations” is based on the
results of the checks performed in your SAP PO system and can be considered as a main output of our activity.
The section “SAP PO Recommended Settings” provides you a comparison of current configuration of your system
with initial optimal parameters recommended for SAP PO.
Note: All the recommendations provided in this report are based on our general experience only. We advise you to
test our recommendations before using them in your production system. We use term “issue” in the report not only
for existing issues but also for potential issues / risks.
Summary 4/39
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Summary 5/39
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
3 Performed Checks
Based on our expertise and SAP documentation find below the check list of all items evaluated in your SAP PO
system. The outcome of the checks are described in details in the format of risks / issues in the subsequent
chapter “Issues and Recommendations” of this report.
Check Status
Performed?
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
PRODUCT
SID SAP Product Product Version
PX3 SAP NetWeaver PO 7.5 SP28
DATABASES
Database SID Database System Database Version Database Host
PX3 Oracle Oracle Database 19c Enterprise pepldr03154
Edition Release [Link].0 -
Production Version [Link].0)
SAP APPLICATION SERVERS
SID Host Instance Name Operating System JAVA
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
All the recommendations provided in this report are based on our general experience only. We advise you to test
our recommendations before using them in your production system. We use “issue” in the following not only for
existing issues but also for potential issues / risks.
Description: The loading of messages from the message monitor takes long time. During the service we noticed
that the global message retention duration on PX3 is set to 4 days. Currently we see around 48 million entries in
the BC_MSG table. It was observed the archive job is currently not working as expected due to the below error
(see screenshot below). Because of the error, the messages in DLVD state are not archived and deleted from the
BC_MSG and other related tables thus leading to the performance issue.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Impact on Business: Large storage required for database and database operations will slow down and will lead
to performance issues in the PO system.
Recommended Task 1: For the failing archive job, Modify parameter ARCHCONN_READTIMEOUT and increase
its value, for example to 600000 (10mins) or larger.
Recommended Task 2: Considering the current message load in the system, For specific integration scenarios,
where business or legal teams require a higher retention, periods can be defined on interface level following the
SAP Note 1960448 - Configure message retention period per interface. Set the global message retention to 1 day
and configure interface specific retention.
Recommended Task 3: Disable the checkbox "Log JSON message" from the REST based channels. If the
checkbox "Log JSON message" is activated, the converted JSON content is logged in the message monitor and it
will add records to BC_MSG_LOG table.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Recommended Task 4:
Refer to SAP Note 2518441- The tablespace PSAPSR3DB of PI system grows quickly. Resolution steps 1-4
should help
Additionally, refer SAP Note 872388 – “Troubleshooting Archiving and Deletion in PI”
For archive and deletion job on BC_MSG*
Description:
During the service we observed that application LM Configuration Wizard is not disabled yet on PX3 system.
WSDL URL of this service is still accessible.
[Link]
Impact on Business:
LM Configuration Wizard of SAP NetWeaver AS JAVA, does not perform an authentication check which allows an
attacker without prior authentication, to execute configuration tasks to perform critical actions against the SAP
Java system, including the ability to create an administrative user, and therefore compromising Confidentiality,
Integrity and Availability of the system.
Refer SAP Note 2934135 - [CVE-2020-6287] Multiple Vulnerabilities in SAP NetWeaver AS JAVA (LM
Configuration Wizard) for more information.
Recommended Task:
The disabling of this application is a defence in depth to reduce the risk of potential undetected flaws in the LM
Configuration Wizard. Implement the steps detailed in SAP Note 2939665 - Disable/Enable LM Configuration
Wizard | Critical API's in LM Configuration Wizard.
We recommend disabling the application aliases unless required. This application is used by a few SAP Lifecycle
procedures only, such as the initial technical setup, and it is not needed in day-to-day operations. You can enable
this application temporarily for executing these SAP lifecycle procedures.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Description:
During the service we observed that PX3 system is missing an important Security Note with a CVSS score of > 5
based on the current SP level and patch levels of installed Java Software Components.
Impact on Business:
Missing Security Notes might allow an unauthorized attacker to access configuration objects, including such that
grant administrative privileges. This could result in complete compromise of PX3 system confidentiality, integrity,
and availability.
Recommended Task:
It is recommended to implement the below missing Security Notes in PX3.
3434192 - [CVE-2024-28163] Information Disclosure vulnerability in SAP NetWeaver Process Integration (Support
Web Pages)
Description: We observed in PO database that the table XI_IDOC_IN_MSG has 81,858,104 entries and
XI_IDOC_OUT_MSG has 2,797,600 entries. Currently the IDOC related housekeeping job is not scheduled in
PX3.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
The job has a parameter Retention_period that can be configured so that idoc messages older than the retention
period are cleaned up
Description: We observed in PO database that table BC_MSG_DUP_CHECK has 56,273,129 entries. For each
asynchronous message (EO or EOIO) that enters the PI/PO system, an entry is stored in table
BC_MSG_DUP_CHECK and kept for 30 days by default to prevent reprocessing of the same message.
Recommended Task: The property '[Link]' specifies the duplicate check persist
time. The default value is 30 days. If you would like to delete the entries in table BC_MSG_DUP_CHECK earlier,
you can consider to decrease the value of property '[Link]'. You can modify this
property online under NWA > Configuration > Infrastructure > Java System Properties > Service "XPI Service:
Messaging System". SAP Note: 2778426 - How to decrease the size of table BC_MSG_DUP_CHECK
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Description: We observed that message compression in system is not enabled. You can configure the Messaging
System to compress the asynchronous message content before storing it in MSG_BYTES field. It will help to
decrease the size of BC_MSG and BC_MSG_VERSION tables and reduce the LOB space occupied by
MSG_BYTES field of these tables.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
6 Software Configuration
We reviewed software configuration and compared current software versions in system PX3 with the latest
available on Service Marketplace. SAP releases Support Packages stacks (including SAP kernel patches) on
regular basis. For more information, see SAP Support Package Stack information at [Link]
stacks.
PX3 is SAP Process Orchestration (PO) installation of SAP NetWeaver release 7.5 and SP-stack level 28. The
latest available SP for this release is 31: [Link]
information/[Link].
SAP AS Version
Installation Name Usage Scenario Release Current SP Level Latest SP Level
SAP JVM
VM Runtime Version Current Patch Level
In the following table you can review the current patch levels of Java Software components in PX3 against the
latest available on Service Marketplace
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Wily Introscope
URL Agent Version Enterprise Manager
Version
[Link] - -
SAP Note 797147 - Introscope Installation for SAP Customers contains release information and installation
procedure of CA Introscope (Enterprise Manager, Agent, Workstation and Webview). Known restrictions of this
version are listed in SAP Note 1565954 - Introscope 9 Release Notes / SAP Note 2071100 - Introscope 9.5
Release Notes / SAP Note 2138309 - Introscope 9.7 Release Notes for changes and open issues / SAP Note
2285189 - Introscope 10.1 Release Notes for changes and open issues.
For further information refer to the link [Link]
The latest available version of XPI Inspector can be downloaded from SAP Note 1514898 - XPI Inspector for
troubleshooting XI.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
7 System Overview
The following areas of your system are discussed in this section:
• Configuration Overview
• AAE Statistics.
Global retention period settings is currently set to 4 days PX3. For specific integration scenarios, retention periods
can be adjusted on interface level according to SAP Note 1960448 - Configure message retention period per
interface.
BC_MSG 45,359,208
BC_MSG_VERSION 7,150,323
BC_MSG_AUDIT 5,869,438
BC_MSG_LOG 137,060,781
XI_IDOC_IN_MSG 81,148,615
XI_IDOC_OUT_MSG 2,636,386
BC_JOB_LOG 279,385
XI_AF_PROF_PERF_CA 2,190,567
Channel types
The following chart gives the number of sender and receiver channels which are classified by adapter types.
Analysis is based on CPA Cache information. Totally there are 1780 communication channels (709 sender and
1071 receiver channels). Major part of channels is of type SOAP.
Not that this graph provides an overview of configuration only and does not reflect actual processing volumes for
adapter types / transport protocols.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Mappings
Mapping overview shows all the mappings in non-SAP namespace distributed according to mapping types.
Adapter Modules
Below chart reflects all the adapter modules that are used in PX3
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
The following chart shows the distribution of polling tasks with server nodes.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Next graph reflects the number of polling tasks per hour and per server node (i.e. number of execution of polling
tasks instances):
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Java Memory
Memory utilization is extremely important for systems based on Java stack. PX3 is configured with 6 Java
Application servers having 4 server nodes each .
Note that the Memory parameters are customer specific and heavily depend on application usage (BPM, AEX,
BRM).
The following parameters can be checked via SAP NetWeaver Administrator → Configuration Management →
Infrastructure → Java System Properties (or direct link [Link]
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Note that heap dump files ([Link]) can consume significant amount of disk space. The free space should be
at least the total of server nodes multiplied by the configured heap size. You may also set the heap dump path to a
directory of your convenience, however, note that in case the target file system is slow this will increase the restart
/ recovery time of server node.
Java Engine
The following configuration must be set via SAP NetWeaver Administrator → Configuration Management →
Infrastructure → Java System Properties (or direct link [Link]
Area: Kernel
Name Properties Current Recommended Recommended Source
Value Value (AAE) Value (CE)
ApplicationThreadManager MaxThreadCount 350 350 150-200 Note
937159
ThreadManager MaxThreadCount 140 140 150-200 Note
937159
ClusterManager* [Link] 300000 450000** 450000**
* In order to display properties of Cluster Manager you need to click on the button “Show Advanced Properties” in
Java System Properties.
** Cluster Manager property “[Link]” specifies max reconnect timeout (in milliseconds) to
another node. If reconnection does not finish in this timeout the node itself will be shut down. The default value is
300 000 ms. There is another property of the Message Server “ms/keepalive” (default value is 300 s) – the
Message Server makes a ping in order to see if the nodes of its participants list are alive. If some node does not
return response in the specified timeout it is being removed from the participants list. These two properties have
the same values and can overlap which can cause the following issue: a node is shut down and it can take up to
300 sec for the Message Server to understand that. Meanwhile another node tries to connect to the shutdown
node because it does not yet have the info that the node is gone from the MS. And as the node will not be able to
connect for 300 sec it will restart itself (exitcode = 888 (engine exit codes:
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Cluster communication issues may happen due to network problems and lead to automatic restart of the whole
SAP PO cluster. In order to avoid it SAP Note 2265959 - Missed broadcast due to network problems provides
recommendations to increase default values of several Cluster Manager parameters (for the values refer to SAP
Note):
• [Link]: Specifies the timeout after which a reconnection to the message server is no
longer possible. If the switchover software is not able to detect the need for a switch and start the central
services instance on a different host within this time, the server process(s) will restart (exit code: -335).
• [Link]: Amount of time that a joining cluster element waits for the confirmation replies
from the cluster elements. If one or more cluster elements do not send a confirmation message, the
server process will restart (exit code: -334).
• [Link]: Amount of time for receiving the acknowledgment from the message server that
the message has been delivered. If the message server fails to deliver the acknowledgment, the server
process will restart.
• [Link]: If there has been no communication between a server process and the message server for
the specified amount of time, the server process triggers a liveness check (ping / pong protocol). This
ping / pong mechanism is important for the server process to detect that the connection to the message
server is broken. Without sending messages to the message server, the connection failure cannot be
detected.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
* Recommended values for these parameters can impact the system if the use-case is different / customized.
Changing them must be done with extensive testing.
*** Refer to SAP Note 2205344 - Performance tuning of CML cache within XPI Service: CPA Cache.
Overall recommendation is setting Tracing Location to severity “Error” and Logging Categories to “Info” with some
exceptional categories for which “Info” writes in some cases too many details:
Note: some of aforementioned categories might be missing in the system as they depend on usage of a
respective adapter.
Having “Logging Categories” with severity “Error” may lead to insufficient information in case of RCA activities in
the system. The goal is to have troubleshooting information and additionally to avoid an overhead in productive
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
systems. More information about authentications logging is in SAP Note 2209271 - Failed Authentications Logged
in the Security Audit Log with Severity Info.
If the “Logging Categories” generates a lot of logs, it can also be set to “Warning” / “Error” in this case and specific
categories that are required for audit purposes can be kept on “Info” (all the unnecessary ones set to “Warning” /
“Error”).
Some Tracing Locations like deploy service or SMD related are by default set to low logging level in order to keep
the information from these (seldom) actions which do not impact the runtime.
There are most useful monitoring tools which simplifies the monitoring of an SAP PO system and analysis of
performance issues:
• Performance monitor is enabled in PX3 system
• Wily Introscope is configured for PX3 system
Adapter Queues
For the meaning of each Thread queue, refer to SAP Note 1129604 - New Queueing Scheme in XI 3.0 SP19 / 7.0
SP11. For the documentation of the Messaging System Service properties, refer to SAP Note 791655 -
Documentation of the XI Messaging System Service Properties.
NOTE: For Java Only installations only Consumer Threads related to the Sender Side are used.
Currently the following value of the property [Link] of XPI Service: AF Core is used in
PX3:
NOTE: There is another property “[Link]” of service “XPI Service: Messaging System”
that is used for tuning of special adapters like CIDX, RNIF, Java Proxy or AFW queues. Standard adapters like
SOAP have to be configured using service “XPI Service: AF Core” and property “[Link]”.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Receiver Parallelism
One more important setting is restriction of number of worker threads which can be blocked by one interface
(based on receiver Party / Service and Interface / Namespace information) in specific adapter: property
“[Link]” of “XPI Service: Messaging System”. By using this property
you can ensure that even during backlog situations resources are kept free for other interfaces.
The global setting limiting the throughput of all the interfaces can be set via the global parameters or starting with
7.31 SP11 / 7.40 SP06 it can be set as well per interface (refer to SAP Note 1916598 - Configure receiver
parallelism per Interface in Messaging System).
* “[Link]” defines the set of queues for which the restriction shall be
enforced. For releases 7.30 and higher the default behavior (empty value) is to enforce the max receiver
restriction on both the receive queue for classical scenarios and the send queue for Integrated Configurations (this
corresponds to the value “Recv, IcoAsync”).
The value “Recv” is only relevant for dual stack SAP PI installations and ignored in Java only systems like SAP
PO or PI-AEX. Therefore, the following values are possible:
• “IcoAsync” will enforce the max receiver restriction on all asynchronous messages of Integrated
Configurations (which corresponds to the send queue);
• “IcoSync” will enforce the max receiver restriction on all synchronous messages of Integrated
Configurations (which corresponds to the call queue);
• “IcoAll” will enforce the max receiver restriction on all messages of Integrated Configurations (send and
call queue).
NOTE: It is important to keep in mind that, for synchronous scenarios, this setting is not recommended (in default
case) due to the restriction of parallelism and risk of timeout. Therefore, initial recommended setting is
[Link]=IcoAsync.
Note: In order to avoid performance issue due to hanging messages in To Be Delivered status in Dispatcher
queue that may occur in case of restarting or cancelling messages in TBDL state if property [Link].
[Link] is set to “true”, apply correction from SAP Note 2575215 - Parallel message
processing restricted due to [Link] property.
Note that it does not make sense to assign “high” priority for many interfaces. Normally the majority of your
interfaces should have “normal” priority. And only several scenarios can be treated with “high” or “low” importance.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Configured rules can be checked / adjusted under PIMON → Configuration and Administration → Adapter Engine
→ Message Prioritization.
Message prioritization is currently being used.
Based on the priority, the Dispatcher Queue of the Messaging System will forward the messages to the standard
adapter-specific queues. The dispatcher queue must not be adjusted. If this queue is mentioned in the parameter
“[Link]” or number of threads is adjusted in “[Link].
dispatcherThreadCount”, there is a risk of issue with duplicate messages. There must be one only thread for the
dispatcher to ensure its consistent behavior.
Starting with 7.31 SP13 / 7.40 SP08, the Dispatcher queue can also be disabled as per SAP Note 2009152 –
Message Prioritization in Adapter Queues (without Dispatcher Queue) in case high backlogs are observed for the
DispatcherDisp queue (if the high backlog slows down the message dispatching of other time critical scenarios).
For File and JDBC based sender communication channels polling configuration can be adjusted on channel level
using advanced property “clusterSyncMode”:
• “Scheduler”: this is the default option that guarantees that at any time only one server node will have any
channel instance active. The adapter runs locally on a server node which is assigned by the scheduler.
The polling of the adapter continues on the same server node, until the node crashes or “relocMode”
value is reached;
• “lock”: this option will ensure that there is only one instance of the channel running across server node.
When this option is set, adapter acquires an enqueue service lock on the server node. Lock is taken on
channel object and it is exclusive and non-cumulative such that no second instance (even it is from the
same channel) can fetch the data until first instance has completed (success / fail) the processing.
Another key aspect of this setting is that the polling is submitted to all the server nodes. Therefore, the
processing happens at all server nodes depending on the lock acquired by the instance.
For File adapter the “clusterSyncMode” can be also enabled at Service level: property
“[Link]” of service “XPI Adapter: File”. Moreover, a new mode is provided to force the File
sender channel to execute its polling job at specific cluster node with SAP Note 2029410 - Assign a File Sender
Channel to poll at specific cluster node.
NOTE: Globally scheduled polling tasks are causing high overhead on enqueue server and SAP system. In
addition, it is quite often that such channels have issues with load balancing across server nodes (the same server
node gets the messages). Therefore, it is recommended to use default “Scheduler” behavior.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Note that most of the IDoc_AAE based channels should be configured in Default mode and use only one
inboundRA resource adapter:
• JRA inboundRA is used by Java IDoc sender adapter;
• JRA outboundRA is used by Java IDoc receiver adapter;
• JRA is shared between all IDoc communication channels configured in default mode – meaning that
there is no channel-specific isolation and the parallelism is for all channels in default mode together;
• The number of reader threads or parallel server connections registered at the gateway is configurable
(see below).
Cloned resource adapters and executor threads in the application thread pool:
If you need to isolate specific IDoc_AAE based integration scenarios by some criteria (interface, prioritization,
sender system), you may have few cloned Resource Adapters for them. In general, it is not recommended to use
many cloned Resource Adapters since it could lead to the issues during CPA cache updates (refer to SAP Note
2246340).
It is necessary to make sure there is a sufficient number of executor threads in the application thread pool when
adding new RAs or change capacity of already existing cloned RAs.
Number of threads can be configured under the following path: SAP NetWeaver Administrator → Configuration →
Infrastructure → Application Resources (or direct link [Link] → Show:
“Resource Adapters” → filter for the Resource Adapter by name (e.g. inboundRA).
MaxReaderThreadCount specifies the number of connections (registered programs) on the Gateway for each
server node of the SAP PO system. It should be a positive number and should be specified taking into account the
number of sender systems, the load of the sent IDoc documents / lists and the available resources (especially
memory and application executor thread usage) on the SAP PO system to process as much parallel calls as
defined. If inboundRA is started and registered properly on the Gateway there should be exactly: (number of sever
nodes) multiplied by the (MaxReaderThreadCount value) registered programs with the value of the ProgramID
property.
Note that multiRepository must never be changed. The data will be filled automatically when a message for a
channel is processed. If it is not filled automatically there is typically another issue (e.g. the required
XI_IDOC_DEFAULT_DESTINATION_<SID> is not there. In case of requirement to change the naming of the
destinations (e.g. because there are multiple clients), the CustomDestMap value of the Java IDoc Resource
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Adapter can be used (override default metadata destination). After specifying the relation there, the corresponding
entry will be automatically populated in the multiRepository.
The connection pool is automatically managed by default. If you require a higher pool size for particular channel
you can change this via SAP NetWeaver Administrator → Configuration → Infrastructure → Application Resources
(or direct link [Link] → Show: “JCA Connection Factories” → filter for the
Connection Factory by name. Then, open the Connection pooling tab and check the property “Maximum
Connections”. The default value is ‘-1’ (automatically managed). Refer to the SAP Help Portal:
[Link]
US/[Link]
* The property “Persistance” is used to enable / disable the additional persistency of IDoc documents in the SAP
PO / PI-AEX system. It should be set to “true” if you want to monitor the IDoc documents in the “IDoc Adapter
Monitor” of PIMON or if you need to process IDoc acknowledgements (ALEAUD IDocs) via IDoc_AAE adapter. If
the IDoc_AAE additional monitoring and ALEAUD processing are not used, then, it is recommended to turn off the
additional persistence. This functionality impacts on the size of tables like XI_IDOC_OUT_MSG,
XI_IDOC_IN_MSG, etc.
** The property “HandleMSEvent” specifies whether the IDoc_AAE adapter registers and processes the event
triggered by the Messaging System background Delete and Archive events. In order to reduce the number and
complexity of the database queries during database tables cleanup by the AAE Deletion / Archiving jobs, a
separate Java job was introduced to delete old entries from the Java IDoc database tables. For this purpose the
job IdocDBTableCleanupPeriodic can be scheduled to run regularly and property “HandleMSEvent” of the
JavaIdocAdapter resource adapter is disabled. Refer to the SAP Note 1918557 - IDoc_AAE: Additional
persistency tables cleanup for more details. In addition, there is a newer job
SAP PO Recommended Settings 29/39
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
IdocDBTableCleanupPeriodicAdvanced for regular deletion of the Java IDoc database tables. It offers more
options than IdocDBTableCleanupPeriodic. Refer to SAP Note 2624188 - IdocDBTableCleanupPeriodicAdvanced:
a new improved job for IDoc_AAE data deletion with optimized option for SAP ASE DB.
For this functionality there is an important correction from the SAP Note 2584443 - Polling Channel Watchdog
consumes many threads that should be applied in the system before activation of the watchdog as it may occupy
many application threads in the system. This problem can lead to an exhaustion of all application threads and to
severe problems with the system.
Mapping Runtime
Support Java8 features in Message Mappings of SAP Netweaver 7.5
New features of Java8 in User-Defined Functions and Function Library in Message Mappings of SAP Netweaver
7.5 can be activated by selecting the JDKCompliance from the dropdown present in functions tab of Message
Mapping editor, Function Library editor, Imported Archive editor. Per default in SAP Netweaver 7.5 the mappings
are generated using Java 5 compatible code.
Default value in the JDK Compliance can be enabled with the property [Link].
supportCompileWithVMJavaVersion of AII Config Service. For more information refer to SAP Note 2482929 -
NewFeature: Support for features of Java8 and above in UserDefinedFunctions and FunctionLibrary in Message
Mappings of SAP NetWeaver Process Integration 7.5 and above
The single thread execution for graphical mapping can be switched on:
• Globally for all mappings it can be enabled with property [Link] of
AII Config Service. For more information refer to SAP Note 1991275 - Execute graphical mapping in a
single thread.
• For specific Message Mapping: add the following statement in the init() method of the message
mapping in the “Functions” tab of the mapping editor: mt$setHeaderObj("SingleThread",
"true");
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
* Not recommended to enable globally, but it makes sense to configure it for specific Message Mappings where
small payloads and frequent mapping invocations are expected.
Ensure that all your XSLT transformations use JDK. The respective switch is done by disabling the checkbox “Use
SAP XML Toolkit” in the operation mapping that references to the corresponding custom XSLT transformation
program. Note that such a switch should be properly tested in quality environment. For additional information refer
to SAP Notes:
SAP Note 1731772 - Migrate XSLT mappings to JDK JAXP implementation
SAP Note 1634755 - PI Mapping: XSL compatibility SAP XML Toolkit to XSLTC
SAP Help: Recommendation Related to Usage of SAP XML Toolkit and JDK
[Link]
US/[Link]
New XPath parser is provided within SAP Note 2239367 - Simple XPath Processor that does not use a DOM tree,
requires less memory and is faster than the default parser. More information about configuration is in this SAP
Note.
NOTE: new parser has limitations on the complexity of the XPath expressions (no attributes in the middle of the
XPath expression and expressions with double slash are not allowed).
Retry Configuration
According to global retry settings each erroneous message is reprocessed maximum 3 times with interval 5
minutes per default. There are predefined pipeline steps where reprocessing takes place: Inbound XML Validation
(VI), Receiver Determination (MS), Mapping (AM) or Outbound XML Validation (VO). It is also possible to prevent
automatic retries in case of an error during execution of specific pipeline steps.
SAP PO Recommended Settings 31/39
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
The default settings are taken from the outbound properties for the Java service XPI Adapter: XI and receiver
communication channel specific configuration can be done in the Communication Channel Monitor (link
[Link] Configuration Settings tab ➔ Number of Retries and Retry Interval
(only relevant for receiver channels).
* the property “[Link]” can be used to prevent automatic retries in case of an error during
execution of specific pipeline steps – e.g. very often reprocessing of mapping and receiver determination errors
does not help as errors are typically permanent. This property with value “MS,AM” can help to avoid such not
beneficial reprocessing attempts at these specific steps and in the most of the cases configuration of this value is
recommended. This property will affect all asynchronous messages that are persisted at some previous pipeline
step according to their staging configuration.
Note that it is still customer specific as e.g. in case of many mappings with lookups reprocessing of mapping step
might be useful.
Large Messages
Normally SAP PO system can handle large messages without impact on the system runtime. However, problems
may occur in case of concurrent execution of several large messages or incoming one but extremely huge
message which might negatively affect system performance. There is a functionality which can help to protect your
SAP PO system in such circumstances.
Large Message Handling restricts the number of large messages that can be processed at the same time across a
configured set of Messaging System queues on one AS Java server node. This restriction helps to improve the
performance of message processing and protect your server node from out of memory problems.
A message is considered large when its size is larger than the threshold that is defined using the
“[Link]”. Messages that are larger than this threshold have to obtain respective
number of permits to continue message processing. When the parameter “[Link].
blacklistXLMessage” is activated, it will push the large message directly to be blacklisted (the support efforts will
be higher due to the blacklisted messages monitoring and manual processing). Correction from SAP Note
2604234 - Corrections related to blacklisting of large messages should be applied.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
EOIO processing
Quality of Service Exactly Once In Order (EOIO) guarantees that a message is only processed once its
predecessor reached a final status (either successful delivered, or manually cancelled). Therefore, all messages
with a non-final predecessor are put to “HOLD” status until the predecessor is finally processed.
There are some known issues that may lead to backlogs of EOIO messages that are put on “HOLD”. Ensure that
these fixes are applied in your system:
1) In systems with high memory consumption (caused by any running scenario) the message may remain in
holding state after delivery of predecessor message and no errors in default trace files generated at this time. SAP
Note 2381352 - EOIO message remains in Holding state after successful delivery of predecessor message should
be applied to fix the issue.
2) Similar symptoms may occur for the interfaces with Staging configuration before determination of
receiver (e.g. configuration on ICO level: staging BI=3;MS=3). Detailed description of the issue and fix is available
in SAP Note 1678427 - EOIO messages put on HOLD in AAE with message versioning and 2177278 - EOIO
messages remain in Holding state without blocking predecessor message. This issue is fixed in all new releases,
however, in specific circumstances (mappings between “old” and “new” serialization contexts are kept in memory
and in case of server node failover / restart it will be lost), manual restart of EOIO messages might be necessary
as queue may stuck. Therefore, it is better to avoid staging configuration for EOIO interfaces before Receiver
Determination.
It is possible to configure alerting functionality for notifying about stuck messages, e.g. in on “HOLD” state. This
functionality is available in the standard for release 7.50 SPS06, refer to SAP Note 2279043 - New Feature:
Alerting: changes and improvements in Component Based Message Alerting (#1). For older releases refer to SAP
blog [Link] the background job AlertStuckMessagesJob
can be deployed to your SAP PO / PI-AEX system and scheduled periodically to check for messages that are in a
non-final status and without an update / change for a certain time and send Alerts in case of such situation (non-
final message status on the AAE are System Error, Waiting, To Be Delivered, Delivering and Holding).
There is also a functionality of Extended EOIO Exception Handling that helps to configure rules how a serialized
message is handled after all retries of the first message in the queue have failed. It is possible to enable automatic
continue of EOIO sequence, even though a predecessor message cannot be delivered successfully. This
requirement can be considered as a controlled violation of the EOIO delivery guarantee. In general, this
functionality might be useful for some special interfaces. Refer to SAP Help portal:
[Link]
US/[Link]
8.3 Housekeeping
AAE housekeeping
Housekeeping needs to be defined in your SAP PO . It is important to reduce / avoid unnecessary data in the
system. Basically there are two options: either delete processed messages or archive them. By default only
Deletion is enabled in AAE engine. Explanation of message persistence in AAE is provided in SAP KBA 1872758 -
How XI/PI messages are persisted in Adapter Engine.
Retention Periods
The following configuration must be set via SAP NetWeaver Administrator → Configuration Management →
Infrastructure → Java System Properties (or direct link [Link]
Note: The value of 86,400,000 ms corresponds to 1 day retention period. Consequences of a large retention
period are visible in the increase of the other related tables: BC_MSG, BC_MSG_AUDIT,
BC_MSG_LOG_VERSION.
SAP PO Recommended Settings 33/39
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Starting with 7.31 SP12 / 7.40 SP07 the Java retention period can also be set at interface level: SAP 1960448 -
Configure message retention period per interface. For example, this setting allows to increase retention period
only for a specific interface for which data needs to be persisted for longer period days (e.g. due to legal / audit
reasons). If you need to configure retention period per interface for logged synchronous messages, ensure that
correction from SAP Note 2218407 - Message Retention Per Interface for logged synchronous messages is
applied.
NOTE: It is not recommended to configure local retention rules with all wildcards as it is better to adjust global
retention periods to influence on all interfaces. Global properties “[Link]” and
“[Link]” are changeable, i.e. online modifiable and restart is not required.
Scenario-specific rules are aimed to adjust the retention periods of messages of specific interfaces and provide
maximum flexibility to ensure that only critical messages are kept for a longer time in the system.
Currently, Interface level retention has been set to 4 days for all the interfaces.
Housekeeping jobs
As a general information on the SAP PO / PI-AEX Background Job Types, refer to the SAP Help Portal at link:
[Link]
US/[Link]
This job should be scheduled with the following parameters (refer to the screenshot below):
• The frequency must be defined;
• The input field “Max Number of Restarts” must be set and activated (value “3” is recommended);
• Parameter for blacklist messages with the checkbox unselected (value “false”) should be added.
It is recommended to create this job at earlier stage to avoid future overhead when there are already old
messages in error in the system (NDLV status) and setting up the Restart job for the first time can create a high
load on SAP PO application and the database.
Usually archiving in SAP PO is not required (unless strict legal requirements demand it), as data is already
available on sender and receiver business systems (SAP PO is not a business system). Note that Default Version
Archive Job might be necessary to take care of messages that were edited and need to be archived. Usually, it
needs to be activated and runs with default daily frequency. Rules cannot be assigned to this job.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
* WARNING: Scheduling the Restart job for the first time or after a long period of inactivity can restart very old
messages and/or a high volume of messages (risk of system overload, in extreme cases can lead to downtime),
so a cleanup of the old messages in NDLV is highly recommended before scheduling this job.
With SAP Note 2611192 it is possible to create a rule for a background job of type Archive, Restart, Cancel or
Delete Archive, to make the job skip certain messages, e.g. to configure an Archive job to archive all messages
except those belonging to specific scenario, with certain interface, sent from specific sender, etc.
For SAP PO housekeeping troubleshooting refer to SAP Note 872388 - Troubleshooting Archiving and Deletion in
PI that provides an approach how to analyze non-final messages and links to the Notes for known problems.
If you schedule this job in productive system, ensure that important fixes from SAP Note 2420535 - Delete Archive
background job may cause OutOfMemoryError and SAP Note 2447828 - Delete Archive job may never finish if
messages have been manually deleted from the archive store are applied as well.
XML DAS expects the information about the archived resources to be consistent in both file system and database.
In case of manual deletion of archived messages on file system, the DELETE method of XML DAS will fail as
there is still information in XML DAS database. To overcome this issue Delete Archive background job has a
configuration parameter “delete_mode” with possible values “default” or “ignore_io_error”. It is recommended to
use “ignore_io_error” only in case when it is verified that the failing resource is missing. In all other cases it is
better to use the “default” value to keep the consistency of the archived resources. Refer to SAP Note 2811802 -
New delete_mode parameter in XML DAS DELETE command and Delete Archive Job.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Note: important prerequisite for enabling payload compression is applying the fix from SAP Note 2434204 -
IndexOutOfBoundsException with payload compression enabled.
It means that if you archive an interface with active User-Defined Message Search configuration, you need to
configure UDS data archiving (xi_af_uds_msg set) according to SAP Help Portal:
[Link]
US/[Link]
With SAP Note 2424103 - Deletion of User-Defined Search attributes upon message archiving it is also possible
to deletes UDMS attributes from the database without archiving, even if original XML message has been archived
(if no requirement to search in the archive using UDS criteria). The property “[Link]” of
service “XPI Service: Lean Message Search” can be set to “archive” or “delete”.
The CleanJob is a housekeeping job for the NetWeaver Scheduler which ensures that outdated and obsolete
information is removed from the system. By default this job has retention period for Tasks 30 days (parameter
RemoveTasksEndedBefore). In case of many Single Timers in the system, i.e. one Task produces one Job, default
retention period for Tasks should be reduced to avoid growing BC_JOB_TASKS.
In order to reduce the growth of the change log, you should regularly remove obsolete entries. It is recommended
to set up automatic cleanup task to reduce the growth of the obsolete change log entries on a regular basis. Go to
“Cleanup Task Configuration” and schedule new task using Java Scheduler (you can check this job definition and
start condition also in SAP NetWeaver Administrator).
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Rolling Restart
Considering that SAP PO is based on the Java Stack and most of the parameters on the Java stack are not
dynamic, the required restart of this stack used to impact then the implementation of optimized parameterization
as well the update of some components.
With the configuration and procedure of the controlled restart, the SAP PO application servers can be restarted
with a minimized impact to business.
For further information refer to the SAP Blog “Controlled restart of a J2EE engine in a PI system with multiple
application servers” under the link:
[Link]
servers/
In order to achieve more secure communication, you can utilize security mechanisms on transport level and
message level. It should be noted that transport level and message level security mechanisms should not be
considered as substitutes – even though they both are targeted at securing communication between service
consumer and service provider, approach and toolset used for securing this communication is fundamentally
different.
Additional important area from security perspective is Data Storage Security (especially for AAE) which helps to
prevent unauthorized access to stored data, because data can contain sensitive information. During runtime
several message versions can be persisted in SAP PO and each message version can contain sensitive data.
There are several measures in order to increase data security:
• Encrypting Data – encrypt message content (payload) on database level;
• Reducing Storage Duration of Data and Deleting Data – change the storage duration according to your
needs;
• Access Protection – prevent unauthorized users from accessing different monitoring tools;
• Change Protection – prevent unauthorized users from modifying message content:
[Link]
US/[Link]
• Logging Data Access – access to encrypted message content is logged:
[Link]
US/[Link]
There are different tools to display messages in SAP PO. Depending on the tool, there are different measures to
increase access protection:
• In Open SQL Data Browser of SAP Netweaver Administrator you can restrict access to the content of
Java database tables. Refer to SAP Note 1611852 - Support-roles to access Java NW PI-DB Tables;
• In Message Monitor of PIMON you can restrict access to message payloads of certain types of
messages (e.g. excluding special interfaces or including only messages from a certain sender). Refer to
SAP Note 1370334 - Securing Payloads in Message-Monitoring.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
It is recommended to check regularly for SAP notes (especially security related ones) via the System
Recommendations functionality available in your SAP Solution Manager under the path: Change Management →
System Recommendations.
Clickjacking Protection
To protect your system from clickjacking attacks, it is recommended to enable the clickjacking framework, as
described in SAP Notes [1] – [4].
• SAP Note 2290783 - Whitelist based Clickjacking Framing Protection for Java Server Pages
• SAP Note 2170590 - Whitelist service for Clickjacking Framing Protection in AS JAVA
• SAP Note 2286679 - Whitelist Service API required for the Clickjacking Framing Protection in JAVA at the
framework or application level
• SAP Note 2319727 - Clickjacking protection framework in SAP Netweaver AS ABAP and AS Java
A Remote Code Execution vulnerability was found in the Services Registry. In order to fix it SAP Note 2800779 -
[CVE-2019-0351] Remote Code Execution(RCE) in SAP NetWeaver UDDI Server (Services Registry) needs to be
applied.
You can setup a weekly job to send data to SAP Service Marketplace and receive the SAP recommended notes
grouped by:
• Security Notes
[Link]
[Link]
• HotNews
• Performance Notes
• Legal Changes Notes
• Patch Notes
This procedure is also described in SAP Note 2041071 - SysRec: How to download latest Java patches using
System Recommendation.
Confidential
SAP TPO for PO system PepsiCo, Inc 22.05.2025
Below is the missing security patches in PX3 which have CVSS score 5 or higher.
3434192 - [CVE-2024-28163] Information Disclosure vulnerability in SAP NetWeaver Process Integration 5.3
(Support Web Pages)
LM Configuration Wizard
Check that application LM Configuration Wizard is disabled as there were critical vulnerabilities detected in the
API of this application. Refer to SAP Note 2939665 - Disable/Enable LM Configuration Wizard | Critical API's in
LM Configuration Wizard for more information. Check that the URL for WSDL of this service is not accessible (404
error)
[Link]
It is observed that LM Configuration Wizard is not disabled yet and is accessible in PX3.
Confidential
Regularly scheduling housekeeping jobs such as 'IdocDBTableCleanupPeriodicAdvanced' is vital to maintain optimal database performance and prevent table bloating. These jobs efficiently manage data by clearing old or redundant entries, thereby enhancing database speed, improving data retrieval times, and ensuring efficient database management operations in ELT (Extract, Load, Transform) processing environments .
Without the 'Restart' job, undelivered messages (NDLV) in the PX3 system are left unresolved, potentially causing a backlog. Scheduling this job ensures that such messages are automatically retried and processed, reducing manual intervention and improving system reliability and throughput .
Implementing security notes with a CVSS score of over 5 is essential to prevent unauthorized access to configuration objects in the PX3 system. Failure to do so risks allowing attackers to gain administrative privileges, leading to potential breaches of system confidentiality, integrity, and availability. Addressing these vulnerabilities is crucial for maintaining system security and safeguarding against potential exploits .
Increasing the ARCHCONN_READTIMEOUT parameter value is recommended because it may resolve the failing archive job that causes messages in the DLVD state to remain unarchived. This adjustment allows the system more time to read and process large amounts of data, reducing performance issues by effectively deleting and archiving messages, thus lowering storage requirements .
Currently, PX3 retains messages for four days generically, contributing to a large number of entries in database tables like BC_MSG. Reducing the global retention period to one day and configuring interface-specific retention based on business or legal needs could significantly enhance system efficiency. This approach balances storage demands against operational requirements, ensuring essential data retention without unnecessary database overhead .
Enabling message compression is necessary to reduce the size of the BC_MSG and BC_MSG_VERSION tables by compressing asynchronous message content before storage. This action decreases the space occupied by the MSG_BYTES field, potentially improving database performance by minimizing the load and volume of data to be processed. However, enabling compression could increase CPU overhead, so it requires careful threshold configuration to balance performance benefits with resource usage .
Not disabling the LM Configuration Wizard on PX3 has significant security implications, as this application does not require authentication checks. This vulnerability allows attackers to perform unauthorized configuration tasks, including creating administrative users, which could compromise the SAP Java system's confidentiality, integrity, and availability .
The high number of entries in the XI_IDOC_IN_MSG and XI_IDOC_OUT_MSG tables can lead to performance issues in the PO database. Without regular housekeeping jobs, such as IdocDBTableCleanupPeriodicAdvanced, these entries accumulate, negatively affecting database speed and efficiency. This results in slower processing times and could impair other system functionalities due to increased database load .
Modifying the 'messaging.duplicateCheck.persistTime' property can improve the PX3 system's performance by decreasing the number of entries in the BC_MSG_DUP_CHECK table earlier than the default 30 days. This reduces database size and load, potentially enhancing database operations and system performance .
Not enabling large message handling can lead to increased memory usage and potential out-of-memory errors, especially when processing concurrent or unusually large messages. By setting thresholds and permits for large message processing, the system can manage resources better, improving reliability and performance during high-load periods .