> Issue LIX, February 2012 > Fundamentals of SOA Security Testing
Mamoon Yunus

Mamoon Yunus


Mamoon is an industry-honored CEO and visionary in Web Services- and SOA-based technologies. As the founder of Forum Systems, he pioneered Web Services Security Gateways and Firewalls. Mamoon has spearheaded Forum's direction and strategy for six generations of award-winning Web Services Security products. Prior to Forum Systems, he was a Global Systems Engineer for webMethods (NASD: WEBM) where he developed XML-based business integration and architecture plans for Global 2000 companies such as GE, Pepsi, Siemens, and Mass Mutual. Mamoon has held various high-level executive positions at Informix (acquired by IBM) and Cambridge Technology Group. He holds two Graduate Degrees in Engineering from MIT and a BSME from Georgia Institute of Technology. InfoWorld recognized Mamoon as one of 4 "Up and coming CTOs to watch in 2004." He is a sought after speaker at industry conferences such as RSA, Gartner, Web Services Edge, CSI, Network Interop, and Microsoft TechEd. Mamoon has the distinction of showcasing Forum Systems' entrepreneurial leadership as a case study at the MIT Sloan School of Management. He has also been featured on CNBC as Terry Bradshaw's "Pick of the Week."


rss  subscribe to this author


Fundamentals of SOA Security Testing

Published: February 16, 2012 • Service Technology Magazine Issue LIX PDF

Abstract: This article describes the foundations of SOA security testing including functional, performance, interoperability, and vulnerability testing. As service-oriented architecture (SOA) begins to form the fabric of IT infrastructure, active and aggressive testing has become crucial to ensure that services are exposed safely without compromising security. Robust security testing ensures that trust (identity, privacy and integrity) is maintained within systems and threats (denial of service, mal-ware and data leaks) are eliminated from SOA deployments.


Rich internet applications, service APIs, virtualization, and cloud services provide extensive integration of data for real-time information access. This drive to open up business applications for integration comes at a cost: service security. This article focuses on SOA security testing. The trustworthiness of a service is a crucial factor determined by whether or not a potential consumer makes use of a service offering. Interestingly, many service providers neglect this fact. They assume that functionally rich services are good enough for their customers. However, non-functional requirements often make the difference between successful service offerings and fruitless, academic attempts to push SOA into an organization.

Figure 1 – Foundation of SOA Security Testing.

SOA security testing requires significant rigor and discipline because of the complex nature of SOA that involves many systems, protocols, content-types, identity tokens, encryption mechanisms, and signature techniques. Detailed business transactions exposed through services contain complex business structures in areas such as customer data, purchase orders, change requests, tax returns, financial reports, and MRI scans. Testing such complex structures' security provisions (along with identity, privacy and integrity) across functional, performance, interoperability and vulnerability domains is necessary for building a secure SOA.

Security Functional Testing

Functional testing is the first foundation of testing SOA security. It enables testers to verify that the services perform as required with the necessary security enabled. As a first step, the business functions of the services are verified with a service request generating a successful response. IT professionals then setup desired regression test cases for automating their testing cases. Most services, however, have security provisions enabled right from the beginning and require functional testing for the following areas:

  1. Transport Protocols: Services are transport protocol independent. Most Internet facing services rely on HTTP or HTTPs for communication, however within the enterprise JMS, IBM MQ Series, Tibco EMS, and FTP are popular transport protocols. The diversity in protocols requires that the functional testing harness is capable of sending and receiving messages over such protocols. With SSL enabled over HTTP or JMS, managing the public key infrastructure (PKI) becomes necessary for secure functional testing.
  2. Identity Tokens: Most services require client authentication and authorization before the request is accepted and a response is returned. The identity tokens provided for this process may come in through a variety of channels. Listed in order from simple to complex, the common identity tokens that require functional testing are as follows:

    1. HTTP basic authentication
    2. Cookie-based authentication
    3. HTTP x.509 mutual authentication
    4. SAML, WS-UserName, WS-X.509
    5. Any ad-hoc content in the payload (header, message, attachment)
  3. Privacy & Integrity: SOA security utilizes privacy (encryption-decryption) and integrity (signature-verification) for information in motion as well as information at rest. Through SSL the transport layer is secured, whereas messages using WS-Security standards, SOAP and XML are encrypted and signed granularly at the content-level. The combination of transport-level and content-level privacy and integrity provides significant control for companies to implement their security policies. With such flexibility, the burden of testing possible variations in the standards used, as well as the content over which the privacy and integrity policies are implemented, is now the responsibility of the quality assurance and security team within the organization. SOA deployments require testing for the following privacy and integrity items:

    1. HTTP over SSL
    2. SOAP/XML encryption
    3. SOAP/XML signatures
    4. MTOM and SOAP with Attachments (SwA)

Functional security testing for SOA is a non-trivial endeavor. With a large variety to protocols, identity token types, privacy and integrity schemes, using established commercially available tools saves corporations time and costs by replacing the coding effort for such diverse standards with simple point-and-click test management. Once a set of test cases have been configured, the next step is to build automation test suites. With automation test suites, SOA security professionals can easily perform regression testing to ensure that new service releases behave as expected. By using commercial products to build regression tests, and by providing developers detailed success-failure reports rather than implementing complex standards, testers efficiently and accurately maintain a high quality SOA deployment.

Security Performance Testing

Performance testing is the second foundation of SOA security testing. QA testers should test the scalability and robustness of services. Testers should determine response times, latency, and throughput profiles for target services. They should also determine scalability by bombarding target services with varying SOAP/XML message sizes across a range of concurrent loading clients. The following areas have a significant impact on performance testing techniques for SOA security:

  1. Generating Unique Messages: Performance testing requires identifying latency and throughput bottlenecks for varying message sizes and concurrent client loads. Typically, a target service is inundated with a large number of messages and the transaction per second (TPS) is then determined based on the error rates or latency characteristics. In the case of XML messages, simply using static messages is inaccurate for SOA security testing. The consuming server should detect a repeated static message as an expired message, or a replay attack.

    During performance testing, a tester has to ensure that target services that require message-level security (such as WS-Signatures, WS-Encryption or SAML) can consume such messages without any errors. The tester has to ensure that SOAP messages have dynamic timestamps, nuances, and security headers. The specifications around timestamps and security elements require unique wire signatures for all messages.

  2. Scaling Clients and Messages: SOA encompasses a wide set of use cases. In system-to-system communication, usually only a few connections are established between the systems and a large number of transactions flow through established connections. Conversely, in a user-to-system portal model, a large number of short-lived concurrent connections are usually established. Testing performance characteristics of a service exposed to a varying number of clients is critical before moving a service into production.

    Similarly, services may be required to handle a wide range of data size. For example, a service used for processing taxes has to handle tax messages of a few kilobytes from small businesses as well as multiple gigabytes from global corporations. Testing such services requires the ability to send a wide range of messages with security provisions such as messages with encryption and signatures enabled. Without the ability to scale both the concurrent clients and the message size with security enabled, a thorough performance profile of the service cannot be determined.

  3. Understanding Security Operations and Keys: Privacy and integrity operations involve public-private key pairs. To establish privacy, the public key is used for encryption whereas the private key is used for decryption. To establish integrity, the private key is used for signing a message and the public key is used for verifying the signature. Private key operations are computationally more intensive than public key operations and have a greater impact on performance. While testing service performance characteristics, testers have to anticipate and test performance degradation based on the cryptographic operation being performed.

    Key size also has a significant impact on service performance, especially when private key operations (decryption or signatures) are being executed. For higher security strength, greater key sizes, up to 4096 bits, are required and a minimum 1024 bit keys are used. Going below 1024 bits is usually considered too weak and is rarely deployed. Private key operations for key sizes of 1024 bits and above are computationally expensive even while using high-end CPUs. For such operations, performance is improved by using dedicated cryptographic accelerators. SOA security testers should understand the key sizes required by corporate policies and ensure that the services are tested for performance using the mandated key size.

Performance testing for SOA security is significantly different from performance testing for websites. An erroneous but fairly common practice is to morph existing web application testing tools into services testing tools. Static WS-Security messages are generated using a "cut-and-paste" scheme where the message is moved into the load testing tool environment. This results in incorrect performance profiles since testing is conducted without sending unique messages.

Understanding the nuances of WS-Security can be overwhelming, but developing this skill-set and utilizing the right tools is essential for building secure and scalable performance testing suites suitable for service-oriented solutions. By using commercial testing tools, this replay issue during performance testing can be easily avoided. Most SOA testing tools allow dynamic time stamps, nuances, and security tokens, to be generated. Testers can build test suites to evaluate whether the services honor expired timestamps by sending stale messages and can replay attack detection by sending duplicate nuances.

Security Interoperability Testing

Interoperability testing is the third foundation of SOA security testing. One of the promises of SOA is that it enables ease of integration between applications and systems. Regardless of coding environments (such as Java, .NET, or PHP) or operating systems (such as Windows, Linux or Solaris), SOA enables applications to exchange service definitions at design-time and then figure out the request and response messaging characteristics later on. Once security provisions are enabled, the interoperability testing requirement increases significantly especially in the following areas:

  1. Transport Protocol Security: In most SOA deployments, HTTP with SSL (HTTPS) is used as the de facto transport protocol. While establishing the secure transport connection between a client and a server, the list of cryptographic algorithms (CipherSuites) supported by both the client and server is negotiated. For greater security, organizations turn off certain weaker algorithms on the server. This then has to be verified by the testers to ensure that only approved algorithms are being presented to the client for establishing a secure tunnel. Verifying that the SSL connection for a service is interoperable with a variety of client scenarios is the first and most crucial step in testing service security.
  2. Identity Interoperability: As highlighted in the Security Functional Testing section, a variety of identity tokens may be accepted by a service for client authentication and authorization. Identity tokens also have varying version types, for example SAML 1.1 and SAML 2.0 are popular content-based tokens used for services. Testers are responsible for verifying the token types accepted by services regardless of the application, development language or operating systems that generate the identity tokens. They are also responsible for ensuring that token types that are not supported by the services are rejected with the appropriate error messages.
  3. Privacy and Integrity Interoperability: Similar to the identity interoperability case, privacy and integrity interoperability have a variety of artifacts that can result in serious interoperability issues if not tested properly. Privacy requires a public key (encryption) and a private key (decryption) operation. In addition to these operations, sensitive content within the XML or SOAP message is selected for the encryption and decryption operation. Ensuring that only the correct key-pairs permit the encryption-decryption process is the responsibility of the SOA security tester. Standards used for signatures also require private key (sign) and public key (verify) operations. In addition to these operations, the signature standards permit including or excluding spaces in the message, enveloping or enveloped signatures, and a variety of other options. This flexibility is powerful, but puts a significant burden on testing signature interoperability within an SOA deployment.

While using a service, consumer applications need to determine both design-time and runtime interoperability characteristics. Developers and testers should run a set of comprehensive interoperability tests and report interoperability issues with the services. Building a comprehensive interoperability test suite ensures that SOA assets are interoperable and that services can work within heterogeneous .NET, Java and PHP environments. Comprehensive interoperability testing ensures that the SOA security provisions work seamlessly and that they are prevented in becoming a hurdle for SOA deployment. Early interoperability testing ensures that development teams avoid falling into the trap of reducing or, in extreme cases, turning off identity, privacy and integrity checks to meet project deadlines.

Vulnerability Testing

Vulnerability testing is the fourth foundation of SOA security testing. SOA exposes internal corporate IT assets to external trading partners for higher business efficiency and collaboration. SOA, therefore, tends to expand the "surface area" on which attacks can be launched. As services are exposed, the service definitions provide granular details on data types, protocols, and input and output messages. This information provides a detailed roadmap for building attack vectors for the following common attacks:

  1. Injection and Data Excavation: SQL Injection is a well-known technique that has been used extensively for extracting information from websites that have a backend database. Using database programming constructs, a malicious query is constructed and sent to the database through a publicly facing website. Depending on the query used, the entire database may be deleted or all the results from a target table (such as username and passwords) may be leaked from the database back to the website. The ability to plug such risks requires pre-processing code that prevents such queries from being executed through input fields on a website.

    With the rapid adoption of SOA, the SQL Injection threat is amplified since services that can carry such injection queries are designed to be re-useable across multiple applications and systems. XML nodes and SOAP messages can now serve as an attack vector for SQL Injection. SOA security testers are required to construct detailed test cases with SQL Injection queries and launch such tests against services. The responses are evaluated to ensure that sensitive data is not excavated from the target service.

  2. Viruses and Malware: SOA provides systems a mechanism to send any content type as an attachment to the XML or SOAP message. Corporations use this channel to exchange complex data such as MRI scans, X-Rays, vehicle designs, and general documents (e.g. pdf, doc, jpeg files). Malware and viruses can permeate through corporations through the service attachment channel. SOA security testers should take benign malware and viruses, send them over XML and SOAP and ensure that the target service rejects infected requests.
  3. Resource Depletion: Using information provided in service definitions, attack vectors such as buffer overflows, deeply nested nodes, and recursive payloads can create depleted hardware resources such as CPU cycles and memory. This depletion can result in a Denial of Service (DoS) to legitimate users and cause business disruption. SOA security testers can preemptively construct a set of test cases that identity such vulnerabilities in exposed services.

By creating specialized tests for a target service, SOA security testers can measure the vulnerability profiles. Security testers need to ensure that vulnerabilities such as buffer overflows, deeply nested nodes, recursive payloads, schema poisoning, and malware traveling over SOAP messages do not affect their critical services. They need the ability to rapidly scan services and assess areas of exposure, determine severity levels, provide vulnerability diagnosis, and publish remediation techniques. Services' vulnerability assessment is a crucial pre-production and post-production step that every developer and security professional must take to ensure risk mitigation within his or her service-oriented architecture.


SOA has changed the way businesses interact and expose information to one another. The significant increase in real-time electronic document exchange, use of cloud-based services, and access to corporate information has resulted in improved revenue and reduced costs. The adoption of SOA has focused the industry's attention towards security. Projects that construct or consume services have to build a detailed plan for functional, performance, interoperability, and security testing of services. Enterprises have to recognize that SOA security testing requires demanding domain skills, tools, and processes that go beyond testing simple websites. Building a competent SOA security-testing team, selecting comprehensive SOA testing tools, and establishing an SOA lifecycle testing framework are crucial for ensuring a successful SOA deployment.