ServiceTechMag.com > Archive > Issue XLIII: September 2010 > Modern SOA Methodology and SOA Adoption Using Agile Practices - Part II
Vijay Narayanan

Vijay Narayanan

Biography

Vijay Narayanan is a technical lead building reusable data services and business process automation components. He has worked on several software projects ranging from single user systems to large, distributed, multi-user platforms with several services.

Most recently, Vijay has been involved in research dedicated to combining agile practices with service-orientation. He is developing content for an upcoming book entitled Agile Service Reuse. Vijay contributed content pertaining to this research as well as additional methodology-related research to the book Next Generation SOA as part of a chapter on modern SOA methodologies.

He maintains a blog dedicated to software reuse at http://softwarereuse.wordpress.com/.

Contributions

rss  subscribe to this author

Bookmarks



Modern SOA Methodology and
SOA Adoption Using Agile Practices - Part II

Published: September 15, 2010 • SOA Magazine Issue XLIII
 

The following is the second part of a two part series on "Modern SOA Methodology and SOA Adoption Using Agile Practices". The first part of this article can be read here.


Release Planning

Instead of taking the approach of "build it and they will come" for services in your domain inventory, only build services when there is a known consumer. While planning upgrades to existing consumers, support backward compatibility using service wrappers and co-existence using versioning. Doing so will reduce the burden on service development team. Additionally, your new consumers access to the upgraded service while not breaking existing users. Providing adequate time window for co-existence will help stagger migration to a new version. Deliver business value early with service orientation by releasing high priority services during the initial part of a release. Bundle multiple enhancements to both new and existing services in the domain inventory as part of a release plan. By releasing early and often you enhance the reuse potential of services across initiatives.


Iteration Planning

Iterations help build reusable services over time greatly minimizing schedule risk. Plan the scope of an iteration using a prioritized list of user stories. Based on the tasks identified, you can come up with new services needing development and updates to existing ones. Be cognizant of actual work effort allocating time for service contract development, service implementation, legacy code analysis, design and code reviews, refactoring, integration of automated tests into the continuous build, as well as necessary documentation. Ideally you want to complete the service capability in a single iteration but if that isn’t feasible. In such cases, capture assumptions and known refactorings to continuously deliver working software and address limitations over multiple iterations.


What about Simple Design?

Some agile practitioners advocate simple design where there is very little or no emphasis placed on upfront architecture and design. As much as we want to avoid big upfront design efforts it is important to recognize that without a consistent architecture and reuse strategy it will be extremely challenging to build a domain inventory for your SOA efforts. Although teams can aspire to practice simple design it may not always be possible for the following reasons:

  • You will want to leverage existing legacy services to reduce time to market while building a new service. This means whatever complexities the legacy service carries as baggage will come along with the service. This could mean the need to sacrifice a simpler interface or the need to be backward compatible with legacy processes.
  • Major fundamental architecture flaws require costly course corrections especially for a SOA initiative. The design needs to be robust, scalable, and extensible requiring fair amount of domain analysis, environmental assessment, and operational stability.
  • Too simple a design may not always be appropriate when building service compositions that orchestrate across multiple systems. There is a real need to build an error handling mechanism, escalations, notifications/alerts, as well as compensation logic to ensure transactional integrity and operational stability.

Service Design Principles and Agility

The principles outlined in SOA Principles of Service Design loose coupling, abstraction, composability, standardized contract, autonomy, statelessness, and reusability can be used in conjunction with agile practices.

Loose Coupling

Service logic is built independent of specific product, technology, channel, and transport. Legacy services are wrapped to avoid direct coupling between providers and consumers. Both techniques are performed iteratively using refactoring. Decouple horizontal concerns such as logging, error handling, metrics, and security from core domain logic. Lower the coupling, higher the opportunity for services to be reused. Loose coupling and agility go hand in hand because the design makes fewer assumptions about upstream and downstream consumers. It also provides flexibility to service provider for performing continuous refactorings.

Abstraction

Abstraction clarifies domain intent and reduces needless coupling between providers and consumers. Additionally, it hides technical complexity associated with a service implementation by providing a clean domain relevant interface for the service consumer. It provides the flexibility for providers to change implementation, gather metrics, and vary quality of service based on consumer needs. Abstraction applies to both legacy services and new services developed for the domain inventory. For legacy services abstraction eliminates point to point coupling between providers and consumers and allows the provider to migrate to a new implementation gracefully. Abstraction thus is essential to both the consumer as well as the provider. Code reviews and refactoring are used to simplify and consolidate service contracts that aren’t well abstracted. This applies to contracts that expose needless technology or platform specific details. Adequate abstraction may not be possible in the first service version and when done over multiple iterations you have to plan for supporting co-existence of service versions. Regardless of how exactly you achieve abstraction it is a fundamental principle for your SOA efforts.

Composability

Composability is useful when implementing task services that orchestrate multiple services. Existing services in the domain inventory may require refactoring in order to support composability. Composability is often realized during analysis of user stories because they often require a combination of services to fulfill a business task. You can identify opportunities for compositions when brainstorming how you want to implement user stories. Pre-defined compositions can also be setup (e.g. using the template method design pattern) to orchestrate a specific set of services to fulfill common parts of functionality.

Standardized Contract

Code and design reviews will be used to ensure reuse of data types, standard set of business and technical error codes, and schema definitions. Standardized contracts create loose coupling between wrapped legacy services and consumers since they abstract syntactic and semantic characteristics of legacy behavior. As alluded to earlier, standardized contracts need to be reviewed so they don’t result in unnecessary couplings thus inhibiting their reuse potential (a service is not considered "done,done" unless the contract is reviewed and signed off). If a contract change is identified towards the end of a particular iteration, postpone the implementation and add needed refactorings to iteration backlog. Changes such as these must be fixed as part of a future iteration prior to releasing a service into production. Helps align contracts with logical data models for enterprise data entities. This principle assists in building service capabilities that complement one another and facilitates service reuse across multiple physical transports (e.g. HTTP and JMS).

Autonomy

As your domain inventory matures and grows over time the number of tightly coupled legacy services will start to decline. As more service capabilities get loosely coupled they increasingly become autonomous. Service autonomy is also achieved by hosting services in a standard runtime environment such as an Enterprise Service Bus (ESB) or an application server that can host and manage a catalog of services.

Statelessness

Stateless services make fewer assumptions about the consumer and are easier to thus reuse. The domain inventory roadmap can be used to identify stateless services that are of relevance to your business. As the roadmap is realized supporting one or more business processes the need for stateless services will become evident. They can also be identified as part of analyzing user stories and leveraging legacy capabilities. Equally important are code reviews and refactoring. These practices can be used to identify and fix unnecessary state contained in both the service contract and the implementation. Finally, stateless services are simpler to scale to accommodate increased usage.

Reusability

There are several practices that were described above that aid in building reusable services for the domain inventory. Sometimes they can be identified and easily mapped to user stories. Other times it isn’t so straightforward. As user stories become better understood future iterations should be used to refactor the service capability to make it reusable. Constantly exercise the DRY principle to locate opportunities for eliminating duplication among services. Reusable services are also built using legacy services and processes but care must be taken to reduce direct coupling. Iteration planning allocates time for wrapping legacy services, unit testing, and working on refactoring backlog to make services reusable as well. Release planning is performed to ensure services are built within the context of an overall domain inventory roadmap. Minimize service development without a real consumer and plan to meet the needs of consumers that you know will use a service capability. Product line techniques to design, implement, and support service variations to accommodate varying functional and quality of service characteristics. Thus the meet in the middle approach facilitates iterative and incremental manner for building reusable services.


Modern Services Technologies and SOA Methodology


REST Services

REST provides a core set of constructs (resources, operations on resources, URI addressability, and support for HTTP transport) and does not enforce a particular development language or methodology. REST based services are more suited to bottom up and meet in the middle approaches to SOA because of their emphasis on design simplicity and lack of support for Quality of Service (QoS) characteristics supported by the various WS-* set of standards. REST is supported by popular web development frameworks such as Ruby on Rails and Apache Axis that are geared towards agile practices. Additionally, REST facilitates as well as inhibits the goals of service orientation. REST supports interoperability by exchanging XML messages across heterogeneous platforms over HTTP. REST services can be used to federate multiple services as well as be used as part of larger compositions. However, REST services might require an ESB when they are meant to be exposed as a web service fulfilling certain QoS characteristics. REST promotes vendor diversification because it uses open standards such as XML and HTTP supported by most vendor platforms. REST supports interoperability via open standards such as XML and HTTP regardless of specific methodology. REST is non-proprietary facilitating increased federation. However, there is no intrinsic support for transactional integrity and security. REST is beginning to get supported in vendor products that enterprises use. REST libraries are available for .NET and Java. Custom development can be done to offer REST APIs on top of products. REST services are based on business entities and actions on these entities thus facilitating domain alignment. REST services can be utilized in business processes and federated services. It can support any SOA methodology but is more suited for bottom up and meet in the middle. In a top down approach the enterprise typically would standardize integration protocols and options and lack of intrinsic support for non-HTTP based transports will be a critical factor. REST based services provide increased ROI for user facing applications built using agile methodologies. REST is simpler to implement than WS-* based web services and thus allow development teams to respond to enhancements faster for user stories.

Supports plain text formats such as XML or JSON. Service contracts are restricted to HTTP transport only and are not described as a WSDL document and registered in a UDDI registry. Service capabilities obtain input parameters via HTTP GET or POST operations. REST uses resources, actions on resources, and URI addressability to simplify the service design. REST does not support the idea of web methods grouped as a Web Service although parameters can be used to route requests to specific handlers. Additionally, since REST does not support WS-* based reliable messaging and security standards. Depending on your enterprise standards and policies this might or might not impact SOA adoption. REST libraries are available in several programming languages such as c#, java, PHP, and Ruby. Simpler to develop when compared to Web Services since there is no need to support SOAP message structure and semantics. REST services supporting HTTP GET can be tested via browsers. Automated testing and continuous integration can be setup with xUnit family of testing frameworks.


Modern Enterprise Service Buses (ESBs)

Modern ESBs greatly influence the SOA methodology by providing several critical pieces of functionality that support a variety of SOA goals. ESBs can be used to host your firm's enterprise or domain service inventory. ESBs can help the enterprise achieve increased interoperability by providing a service mediation layer that can perform protocol bridging, data transformation, and connectivity with a variety of vendor packages. For instance, it can be used to expose a service available only via COBOL copybook over JMS. An ESB can also be used to connect with a variety of transports including plain HTTP, SOAP/HTTP, JMS (e.g. JMS providers such as WebSphere MQ and Tibco EMS), FTP servers, and SMTP. The ability to connect to a variety of messaging brokers facilitates asynchronous request/reply and publish/subscribe based integrations and processes. ESBs also perform complex data transformations including visual editors that can reduce the time and effort to generate XSLT stylesheets and mappings. As an abstraction layer an ESB can also be used to wrap legacy services. The ESB can use the enterprise's canonical schemas and perform data transformation to invoke legacy service capabilities. Thus it can help migration of legacy capabilities gracefully over time while aiding technology transformation.

Many modern ESBs do support security standards such as WS-Security as well as HTTPS and SSL. They can be used to specify security policies, SLA details, as well as advanced routing based on message characteristics. Modern ESBs are also at their core advanced orchestration engines supporting standards such as BPEL facilitating business process automation and the ability to invoke multiple services as part of orchestrations. ESBs can be used with any SOA methodology although it is more suited when taking the top down or meet in the middle approaches. An ESB can help identify integration approaches, policies, and standards for the SOA initiative and since these decisions are aligned with governance it is more suited for a top down or meet in the middle type approaches. ESBs don't hinder the bottom up approach per se but if each team uses the ESB to place services not aligned to the firm (or line of business) standards and policies the SOA capabilities will not be able to provide consistent QoS characteristics.


Cloud-based Services

Cloud-based services offer several advantages for organizations looking to reduce costs and achieve on-demand scalability. Cloud based services can be used for performing specific functions in a stand alone application as well as a utility for a line of business or even the entire enterprise. Cloud based services such as storage can be used to reduce IT's total cost of ownership and even speed up SOA adoption. In addition to providing a robust, scalable, and low-cost alternative to in-house services, cloud based hosting providers can provide programming interfaces, integration assistance, documentation, and production support for critical capabilities. Cloud based services can be used with any SOA methodology but with certain considerations. If your enterprise has security policies and guidelines the hosting providers will need to support them. For instance, if your services need to be secured using X509 digital certificates, can the provider support that? What about data encryption? Additionally, because there is network latency associated with invoking services outside the organization use case-specific SLA needs need to be examined carefully. Exception handling and transactional integrity are critical enterprise concerns that will need attention as well because different providers might offer different behavior. Given that cloud based services provide a cost advantage as you scale up usage it is useful to place hosting decisions within the context of your enterprise or domain inventory. If a cloud based solution can fulfill your organization's requirements it needs to be factored when designing an implementing service capabilities. Multiple service providers - even a mix of internal and external solutions- can be used in conjunction as well where a specific provider is used based on SLA considerations or on a use case by use case basis. On a related note, as a risk mitigation exercise before large scale adoption cloud based services can be used for a single pilot project and slowly expanded aligned to a meet in the middle approach.



Conclusion

Enterprises have pursued SOA efforts via top down, bottom up, and Meet in the middle approaches. Although both top down and bottom up have specific advantages to them they are ill-suited for adopting SOA in the enterprise. Instead, by taking a Meet in the Middle approach focusing on building out a domain inventory incrementally and iteratively not only reduces costs but also increases the likelihood of reuse, domain relevance, and most importantly alignment with business goals. This chapter demonstrated how agile development practices such as User Stories, Do not Repeat Yourself, Iteration and Release planning, and refactoring, and 'done,done' can be used to successfully pursue the Meet in the Middle approach. Finally, this chapter briefly touched upon modern services technologies such as REST, cloud-based services, and ESBs examining their impact on methodology within the context of attaining the goals for service orientation.