IBM Developer

IBM Integration IBM badges Resources Videos Blog Support Events About

Cloud Pak for Integration Integration

An approach to build DevOps pipeline for ACE on

Cloud Pak for Integration
[Link]
Published on March 20, 2020 / Updated on March 20, 2020
0

We had published a recipe in developerWorks to automate the build and deployment of ACE projects on Cloud Pak for
Integration. In this blog, we will discuss a pragmatic approach on how enterprises can quickly configure the pipeline for
the build and deployment of ACE integration flows on IBM Cloud Pak for Integration.

Link to Developer Works recipe: Building CI-CD Pipeline for IBM App Connect Enterprise on Cloud Pak for Integration

In this blog, we will demonstrate an approach by taking an example so that it is nearer to real time experience.

Scenario:
A customer wants to automate the build and release process for IBM App Connect Enterprise to deploy on CP4I. They
have considered using:

• Git as source control

• Nexus to store versioned BAR files

• Jenkins as the CI tool

The diagram below depicts their target state.

Note that you might have the multiple logical ACE environments on the same CP4I cluster isolated by namespaces or
different ACE environments could be on different CP4I clusters spread across private or public clouds.
As you can see, this diagram depicts a basic DevOps flow. Lots of interesting things can be done on top of this, like:

• Implement continuous testing

• Automatically rollback to previous successful release if test fails

• Automatically Create issue in a bug tracking tool, like JIRA, for failures and assign to a developer

• Further enhance it to DevSecOps by introducing security test and so on.

One of the major benefits of moving to containers is that it eliminates the ‘compatibility issues’. It completely
eliminates ‘it works on my machine’ problems. You can test your application with the same image locally that you are
going to use in live environments. You would typically pull the respective ACE image from the container registry and
deploy the app on local workstation. You can attach debugger for your flows running on containers on your local
workstation. You could follow the below Github documentation to deploy ACE containers on your local workstation for
development, testing and debugging as well.

[Link]

Let us see how this basic DevOps flow can be implemented. In most of the scenarios:

1. Developer will check-in the code after testing it locally with same image

2. There could be a manual or automated build trigger for Jenkins build job

3. Jenkins build job will compile the ACE application/service and create versioned BAR files for respective
environments by taking configuration values from respective environment’s properties files and tag the source
code with the respective version number

4. Jenkins Build Job will store these versioned BAR files into a repository, say Nexus

5. There could be a manual or automated trigger to Jenkins deployment job for DEV environment

6. Jenkins DEV deployment job will create an ACE image with BAR file(s) pulled from repository (here Nexus) and
other configurations baked into it and will push this image to the CP4I container registry

7. Jenkins DEV deployment job will perform deployment from the created DEV image and will configure other
artifacts like Horizontal autoscalar policy, Route etc.

8. Upon successful testing in DEV env, there could be a trigger to Jenkins QA deployment Job. The similar steps (6
and 7) will be performed for QA environment.

9. After validation and all required testing, similar steps (6 and 7) will be performed for Production environment.

You could create the image(s) for your ACE application during the build process itself; however building the image in a
separate job, i.e. in a deployment job, has the advantage described below:

• You have versioned BAR files for respective environments and the source code is tagged with that version. So you
can always trace the source code version associated with the respective BAR files.

• You may include BAR files from one or more build jobs to deploy them in one Integration Server.

• You have control over the deployment i.e. make deployment manual or automated. As in many scenarios, you may
want to do manual testing of the applications before deploying in target environment.

For the scenarios where more than one ACE applications/services need to be deployed in the same Integration Server,
the Jenkins deployment job can pull the respective BAR files from the repository (here nexus) and bake them into the
ACE image.

The diagram below depicts two ACE applications being built and deployed in an IntegrationServer on CP4I
1. Configure Build Server

Ensure that you have configured the build server by following the steps from 1 to 5 from the recipe mentioned below:

Building CI-CD Pipeline for IBM App Connect Enterprise on Cloud Pak for Integration

Note that the instructions in this blog do not use OpenShift plugin and Docker plugin, so you may skip to install these
plugins in Jenkins; however docker and oc client must be installed on build server.

2. Configure ACE project and Build Job

For this demonstration, we will create two additional folders inside the ACE project:

– Properties folder: This folder will contain properties files for different environments, e.g. [Link] and
[Link]. These properties files would contain the environment specific parameters to be used in
mqsiapplybaroverride command to override node properties and UDPs

– Build folder: This folder will contain Jenkins pipeline script for the ACE project. So the build script for the ACE project
is also checked-in to source control

For this demonstration, I have created two ACE projects which are checked-in at the location below:

[Link]: [Link]

Sum_Service: [Link]

Both these projects have two properties files ([Link] and [Link]) inside the ‘properties’ folder. Let us
examine the ‘jenkinsfile’ script for the [Link] project.
Now let us create the Jenkins build job for this [Link] project.

Create a Jenkins pipeline project with the name ‘[Link]’.

Specify to read pipeline script from SCM and enter the github project location, credential to access GitHub and path of
the jenkinsfile in the project.
Click on Save.

Now build the project so that it uploads the BAR files on the repository and tags the code. The snippet below shows a
successful build of the [Link] project.
Now let us look at the Nexus repository if BAR files for this version (version 10 here) have been uploaded. The
snapshot below shows the zip file containing the version 10 BAR files.

Now let us look at Github if the [Link] source code has been tagged with version 10. Below snippet shows the tag.

Similarly configure the second ACE project (Sum_Service).

3. Configure Deployment Job

Now configure the deployment jobs for each environment. You may have a single parameterized job to deploy to a
different CP4I environment; however due to security and compliance reasons, you would probably have different
deployment jobs for respective environments.

Before the Integration Server starts, the container is checked for the folder /home/aceuser/initial-config.
Refer to [Link] to understand how to supply the required configuration information for the
Integration Server.
I have a sample deployment job that contains these two files:

• Dockerfile: This file will have instructions to build the image baked with BAR files and necessary IntegrationServer
configurations

• Jenkinsfile: This file will have pipeline script to deploy to CP4I environment, create route, Horizontal Pod autoscalar
policy etc.

The sample deployment job is checked-in to Github:

[Link]

Let us examine the Dockerfile.

You can add other configuration parameters as well, for example to do SSL configuration etc. Follow the
documentation at [Link]

Let us examine the parts of the jenkinsfile. This pipeline has been parameterized to accept some parameters
dynamically. It does following tasks:

• Checks-out the deployment project

• Pulls the specified versions (ACE_APP1_VERSION and ACE_APP2_VERSION) of BAR files from Nexus repository
for ACE_APP1 and ACE_APP2.

• Logs-in to OCP registry of specified CP4I instance

• Builds the docker image using the Dockerfile and DEV BAR file and pushes to OCP registry

• Performs helm deployment. Notice the name of ACE image ([Link]) in helm install command. Since we are
using ace only image, so value of ‘imageType’ has been specified as ‘ace’. If you are using ACE with MQ client or
ACE with MQ server, value of ‘imageType’ would be ‘acemqclient’ and ‘acemqserver’ respectively. Make sure to
specify the image parameter based on imageType

• If it is a new helm release being installed, route and horizontal autoscalar policy (hpa) are being created

• If it is an upgrade to existing helm release, it only upgrades the helm release and route & horizontal autoscalar
policy are left unchanged

• Performs the clean-up

timestamps {

node () {

wrap([$class: 'Xvfb']) {
stage ('[Link] - Checkout') {
checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, ext
ensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'jenkins-github', url: '[Link]
m/awasthan/[Link]']]])
}
stage ('[Link] - Build') {

artifactResolver artifacts: [artifact(artifactId: '${ACE_APP1}', extension: 'zip', groupId: '[Link]', v

ersion: '${ACE_APP1_VERSION}'), artifact(artifactId: '${ACE_APP2}', extension: 'zip', groupId: '[Link]'
, version: '${ACE_APP2_VERSION}')], targetDirectory: '/var/icp-builds/${ACE_APP1}'
sh label: '', script: '''#!/bin/sh
cd /var/icp-builds/${ACE_APP1}
unzip ${ACE_APP1}-${ACE_APP1_VERSION}.zip
chmod 777 *
cp ${ACE_APP1}_DEV_${ACE_APP1_VERSION}.bar /var/lib/jenkins/jobs/${JOB_NAME}/workspace
unzip ${ACE_APP2}-${ACE_APP2_VERSION}.zip
chmod 777 *
cp ${ACE_APP2}_DEV_${ACE_APP2_VERSION}.bar /var/lib/jenkins/jobs/${JOB_NAME}/workspace
cd ${WORKSPACE}
oc login ${OCP_API_URL} -u admin -p ${REGISTRY_PASSWORD} --insecure-skip-tls-verify
docker login ${OpenshiftRegistryURL} -u $(oc whoami) -p $(oc whoami -t)
docker build -t ${imagename}:${BUILD_NUMBER} .
docker tag ${imagename}:${BUILD_NUMBER} ${targetrepo}/${imagename}:${tag}-amd64
docker push ${targetrepo}/${imagename}:${tag}-amd64
source /etc/profile
cloudctl login -a ${ICP_CONSOLE_URL} -n ${Namespace} -u admin -p ${REGISTRY_PASSWORD} –skip-ssl-validation
cd /opt/certs
helm init --client-only
helm repo add local-charts --ca-file ${HELM_CERT_FILE_NAME} ${ICP_CONSOLE_URL}:443/helm-repo/charts
if test ${DeploymentType} = \'install\'; then
helm ${DeploymentType} --name $ReleaseName local-charts/ibm-ace-server-icp4i-prod --namespace ${Namespace} -
-set imageType=ace --set [Link]=[Link]/ace/${imagename}:${t
ag} --set [Link]=[Link]/ace/ibm-ace-mqclient-server-pro
d:[Link].1 --set [Link]=[Link]/ace/ibm-ace-mq-server-prod:1
[Link].1 --set [Link]=[Link]/ace/ibm-ace-icp-configur
ator-prod:[Link].1 --set [Link]=[Link]/ace/ibm-ace-
designer-flows-prod:[Link].1 --set [Link]=${IMAGE_PULL_SECRET} --set [Link]=false -
-set [Link]=false --set [Link]=testace --set [Link]
t=1 --set license=accept --tls
oc project ace
oc expose svc ${ReleaseName}-ibm-ace-server-icp4i-prod --port=7800
oc autoscale deployment/${ReleaseName}-ibm-ace-server-icp4i-prod --min 1 --max 3 --cpu-percent=25
else
helm ${DeploymentType} $ReleaseName local-charts/ibm-ace-server-icp4i-prod --namespace ${Namespace} --set im
ageType=ace --set [Link]=[Link]/ace/${imagename}:${tag} --s
et [Link]=[Link]/ace/ibm-ace-mqclient-server-prod:11.0.
0.6.1 --set [Link]=[Link]/ace/ibm-ace-mq-server-prod:[Link]
.1 --set [Link]=[Link]/ace/ibm-ace-icp-configurator-pr
od:[Link].1 --set [Link]=[Link]/ace/ibm-ace-designe
r-flows-prod:[Link].1 --set [Link]=${IMAGE_PULL_SECRET} --set [Link]=false --set pe
[Link]=false --set [Link]=testace --set [Link]=1 --s
et license=accept --tls
fi
cd /var/icp-builds
rm -rf ${ACE_APP1}
rm -rf ${ACE_APP2}'''
}
}
cleanWs()
}
}

Now let us look at the Jenkins pipeline Job for this.

Parameters have been defined for all the placeholders in above jenkinsfile.

Pipeline script has been configured to pull from Github.

Similarly you can configure deployment job for QA environment and so on.

4. Perform Deployment

Now let us deploy an IntegrationServer that contains version 15 of [Link] and version 16 for Sum_Service. Also note
that, the base image of ACE being pulled in docker file is version [Link].1. In the next step we will apply fix-pac
[Link] on this IntegrationServer.

Click on ‘Build with Parameters’ for the deployment job.

Supply the parameter values

Click on Build.
Below is the snippet of build logs.
Let us take a look at the ACE Dashboard.

The [Link] contains a simple ping flow. You can test the flow by hitting the url in web-browser. The url will be:
[Link]

You can get the route by using ‘oc get route’. Also notice the hpa.
5. Apply Patch – Upgrading the release

You may need to apply patch to your IntegrationServer for various reasons. Some of the reasons could be:

• Apply new fix pac for ACE

• Deploying new version of an application

• Deploying another application on same IntegrationServer

• Removing a deployed application

• Changing configuration parameters

And so on…

In this scenario, let us assume that we want to apply new ACE fix pac. The current image has been built on ACE
[Link].1. So first of all you need to upload IBM provided new image for ACE fix-pac. Follow the step 10 in below
article to upload ACE fix-pac image in OCP registry.

Modernizing Integration – Migration from IIB to App Connect running on IBM Cloud Pak for Integration(CP4I)

If you are pulling the images online from IBM entitlement registry, you don’t need to do this step.

After this is done, let us update the Dockerfile of the deployment job to pull new ACE fix-pac, in this case [Link]-r1

Now run the click on ‘Build with Parameters’ for the deployment job.
Notice that the ‘DeploymentType’ is selected as ‘upgrade’ and the name of existing helm release for upgrade has been
provided. Selecting the value ‘upgrade’ for DeploymentType will execute else condition in jenkinsfile, which means it
will not try to recreate route and hpa policy.

Now click on Build. Below is the log snippet of the deployment job execution.
Now the IntegrationServer is running on ACE [Link].r1 fix pac.

6. Rolling back to previous version

Though as part of DevOps process, you can implement automatic rollback in case deployment failure or test failure.
However in certain scenarios, you may be asked to rollback to a previous version due to some other reason.
To do that, you can go to ‘Cloud Pak Foundation’→Administer→Helm Releases

Find the Helm release and click on Rollback.

Select the release to rollback to and click on Rollback.

You can perform the roll back using command line as well.

helm rollback <RELEASE> [REVISION] --tls

The sample Github projects used in this demo are attached here.

[Link]-master
Sum_Service-master
[Link]-master

Thanks to Amar Shah for his contribution to this article

TAGS ACE, CP4I, ACE ENVIRONMENT, CONFIGURE BUILD SERVER, JENKINS PIPELINE

[Link]

Join The Discussion

Your email address will not be published. Required fields are marked *

Enter your comments...

Name *
 Email *

Website

Post
Comment

Tags
ACE ACE on Cloud tutorial administration agile-integration-architecture api API Connect API Management APIs App Connect badge App Connect Enterprise App Connect
Enterprise v11 App Connect tutorial betaworks BTM cloud configuration containers develop-integration-solution dfdl docker docs documentation ESQL
festive2015 gdm get-started global-cache graphical-data-mapping HTTP hybrid integration ibm-application-integration ibm-integration ibm-integration-
bus ibm-integration-bus-manufacturing-pack ibm-integration-community IBM App Connect (IBM Cloud) IBM App Connect
Enterprise IBM App Connect Professional IBM Cloud IBM Cloud Pak for Integration IBM Cloud Private iib iib10 iib10lab iibv9 iibv10
IBM Event Streams ICP iiboc

IIBv10.0.0.4 integration integration-design-and-architecture integration-solution Integration Bus Kubernetes Learning Path lightweight-integration LoopBack
manufacturing manufacturing-pack microservices migration monitoring mq mqsi optimisation overview performance problem-determination redbooks resources
RestAPI runtime Salesforce SAP Scaling toolkit topology tuning tutorial V10 websphere-message-broker wesb-conversion wesb-conversion-tool whats-new wmb
wsrr Zero to Hero

Contact us

Contact Privacy Terms of use Accessibility Feedback Report Abuse Cookie Preferences