SOA Testing
Ahamad Patan
8th Sept 2009
[Link]@[Link]
Agenda
SOA Overview
SOA Testing – An End to End Approach
Testing Web Service API’s
API Testing Strategies
Exercise
Testing Web Service Applications
SOA Testing Challenges
SOA Test Automation Approach
QA Session
SOA Overview
What is SOA?
Service-Oriented Architecture (SOA) is an architectural
style.
Applications built using an SOA style deliver functionality as
services that can be used or reused when building
applications.
SOA Continued…
SOA in other terms
A service-oriented architecture is essentially a collection of
services
Services are based on heterogeneous technologies.
Application functions are modularized and presented as
services.
A service is an implementation of a well-defined business
functionality.
Service interface is independent of the implementation.
SOA Continued…
Services are platform independent, self describing
interfaces (XML)
Messages are formally defined
Services can be provided on any platform
SOA and Webservices
SOA is not Webservices, but web services are the
preferred standards-based way to realize SOA .
Web services are software systems designed to
support interoperable machine-to-machine
interaction over a network. This interoperability
is gained through a set of XML-based open
standards, such as WSDL, SOAP, and UDDI.
These standards provide a common approach for
defining, publishing, and using web services.
SOA and Webservices
continued…
Interoperability is the most important
principle of SOA. This can be realized
through the use of web services, as one of the
key benefits of web services is
interoperability, which allows different
distributed web services to run on a variety of
software platforms and hardware
architectures.
Webservices basics
Points to
UDDI
UDDI description
WSDL
Registry
Registry
Points to
service Describes
Finds Service
Service
SOAP Web
WebService
Service
Web
WebService
Service (J2EE, PL/SQL,
(J2EE, PL/SQL,
Client
Client .NET,C/C++,
.NET,C/C++,
(J2EE,
(J2EE,.NET,
.NET, Legacy
Legacy…)
…)
PL/SQL
PL/SQL…)…)
SOA Testing
Testing Web Service API’s
Testing Web Service Applications
Testing Web Service API’s
Following types of testing needs to be performed
Functionality
Performance
Re-usability
Security
Interoperability
Compliance
Backward compatibility
Testing web service API’s
Functional Testing
API Testing
Detect errors at an early stage, right from the
requirements level.
Ensure that real-world test data is used
Automate test case execution to improve productivity
Testing web service API’s
Performance Testing
Web Services have high usage and high reuse
opportunities. Example: A service has been
identified as serving 10000 concurrent users with 5
sec of response time. Typically, services are server
side components that do not have a front end. It is
therefore useful to have a sample GUI interface that
can be used to load test the service from a web
client.
Testing web service API’s
Reusability Testing
The more a service gets reused, the greater
the impact associated with its downtime will
be.
Make sure that the APIs would work in all
possible business scenarios, thus ensuring
consumability
Testing web service API’s
Security Testing
Web Services are exposed to the outside world, this
fact opens up a host of vulnerabilities, such as DOS
(denial-of-service) attacks, penetration, high
volumes of spam data, etc. Typical security policies
have to be enforced at the network level to create
bullet proof SOA. Specific test cases aimed at
targeting these policies need to be designed to fully
test SOA security.
Testing web service API’s
Security Testing
A denial-of-service attack (DoS attack) is an
attempt to make a computer resource unavailable to
its intended users.
Prevent an Internet site or service from functioning
efficiently, temporarily or indefinitely.
Consumption of computational resources, such as
bandwidth, disk space, or processor time
Crash the operating system itself.
Testing web service API’s
Security Testing
Testing - Simulate denial-of-service attacks
Popular denial-of-service mitigation products
from Arbor Networks
Testing web service API’s
Interoperability Testing
The ability of two or more systems or components
to exchange information and to use the information
that has been exchanged.
Involves different technology platforms and
development tools. Developers use many different
tools to generate the contracts (WSDL) and modify
them in the process.
Testing web service API’s
Interoperability Testing
Testing – should be tested on different
technology platforms.
Testing web service API’s
Compliance Testing
Testing to determine whether we meet some
specified standards
Validating WSDL definitions
Validating SOAP request/response messages
Testing web service API’s
Backward Compatibility Testing
As businesses evolve, changes may required in the Web Services, but
some consumers may want to continue the old services. The challenge is
to make sure that old service consumers are still supported.
SOA Testing – Challenges
SOA Testing Challenges
Major Challenges:
No user interface for the Web Services
Difficult to create Real time applications
Test Data should be close to the end user environment
Infinite consumers possible
SOA Testing Challenges
Test Automation
Client Simulators (Message simulation)
Service Simulators
Automatic test data creation
Choosing the right test automation tool
Maintaining pool of testing resources with SOA domain knowledge
SOA Test Automation Approach
Traditional Testing Tools
GUI Testing Approach
In House Test Tools
Web Service Testing Tools
SOA Test Automation Approach
Traditional Testing Tools
Web Services SOAP requests needs to be generated manually or using a
supported third party vendor testing tool
Integrating traditional functionality testing tool and other such third party tool is not
feasible due to its nature and limitations
A mix of manual and automated process is an additional overhead
It’s a time consuming and expensive effort
SOA Test Automation Approach
GUI Testing Approach
Web Service is UI less application.
GUI application should be developed
Web service interface changes demands often GUI changes
Web Services security testing that requires encrypted & singed data is not
possible
Overall cost involved is very high for testing,development & maintenance
efforts
SOA Test Automation Approach
In House Test Tools
Focus is drawn towards developing & maintaining the test tools than the actual
testing
In-house test tool itself need to undergo testing to ensure it is coded to the
specifications
Dependency on in-house tools will impact test schedules & quality of the deliverable
in demanding situations
Time Consuming and expensive approach
SOA Test Automation Approach
Web Services Testing Tools
Provides robust, reliable, scalable and interoperable solution
Best fit for UI less testing
Provides a solution for load testing
Test data generation for any WSDL version is possible and easily
maintainable.
Depending on existing legacy systems & back-end applications, web services test
tools can be tweaked by developing / modifying test scripts to provide an E2E
solution for backward compatibility
Unlike any other traditional testing tool, the cost involved is very minimal.
Example
A Web Service maps user input IP
addresses to countries
String findCountry(String ip)
Example
Test Plan
Description Input Value Expected Result
Valid ip address [Link] India
Valid Ipv6 address 3ffe:1900:4545:3: US
200:f8ff:fe21:67c
f
Invalid address [Link].77.88 Error, Not a valid input
Example – Strategy 1
Suppose this API belongs to the class IPHelper.
IPHelper iph = new IPHelper();
String result1 = [Link](“[Link]”);
If ([Link](“India”))
[Link](“Test1 Passed”);
Else
[Link](“Test1 Failed”);
Example – Strategy 2
Store the data in a xml file
<Testsuite>
<testcase1>
<input>[Link]</input>
<expectedresult>India</expectedresult>
</testcase1>
<testcase2>
<input>3ffe:1900:4545:3:200:f8ff:fe21:67cf </input>
<expectedresult>US</expectedresult>
</testcase2>
<testcase3>
<input>[Link].77.88</input>
<expectedresult>Invalid Input</expectedresult>
</testcase3>
</Testsuite>
Example – Strategy 2
Read the xml file as test data input.
Parse the xml file into an Object(xmlObject)
IPHelper iph = new IPHelper();
For (int I =0; I<[Link]; I++){
String result1 = [Link]([Link]);
If ([Link]([Link]))
[Link](“Test” + I + “Passed”);
Else
[Link](“Test” + I + “Failed”);
}
Exercise
Web Service API
Gives the current temparature of the city in
required units
Float getTemparature(String City, Char Unit)
Unit can be F(Farenheit) or C(Celsius)
Write a test plan – Better come up with
unique test cases.
Description input1 input2 Expected
Result
Exercise
Some Unique Test Cases
Case Sensitivity – bangalore, f, c, F, C
Exercise
Some Unique Test Cases
Multiple names for a same city – Mumbai,
Bombay
Exercise
Some Unique Test Cases
City name contains space – New Delhi
Exercise
Some Unique Test Cases
Same city name in many countries –
Hyderabad(Ind), Hyderabad(Pakistan)
Exercise
Some Unique Test Cases
City Name contains Single Quote – Gizzard’s
Cove
Exercise
Some Unique Test Cases
City Name contains Same string repeated –
Zu Zu
Exercise
Some Unique Test Cases
The City is in North/South Pole – should
return negative value for the expected result
Exercise
Some Unique Test Cases
City Name is very Long string - Guangzhou in
southern China, Name of the city –
LLANFAIRWLLGWYNGLLGOGERYYCHWY
RNDROBWILLLLANTYSILIOGOGOGOCH
In New Zealand, Name of the city -
Taumatawhakatangihangakouuauotamateatur
ipukakapikimaungahoronukupokaiwhenuakita
natahu
Testing web service Applications
Functional Testing
Services specifically designed to function as independent
entities may work perfectly well in isolation, but might not
function as expected when integrated into applications due to
underlying platform and network specific issues.
Consider a sequence of web service calls made to various web
services – This should not affect the functionality.
Since SOA involves a lot of asynchronous communication
across systems, it is possible that messages will not reach their
destinations in an orderly fashion because of delays in
processing at the service provider. Some systems will respond
quickly, and some slowly, depending on the load at any given
point in time.
Testing web service Applications
Integration Testing
Testing the composite application that is built using
sub-systems or modules that are independent of the
rest of the system.
Testing web service Applications
Performance
When a composite application is being tested for
load, stress, it is important to observe the throughput
of the web service to see if a bottleneck occurs.
Questions or Comments?
Questions
or
Comments?
Thanks
