ServiceTechMag.com > Archive > Issue XLII: August 2010 > Modern SOA Methodology and SOA Adoption Using Agile Practices - Part I
Vijay Narayanan

Vijay Narayanan

Biography

Vijay Narayanan is a technical lead building reusable data services and business process automation components. He has worked on several software projects ranging from single user systems to large, distributed, multi-user platforms with several services.

Most recently, Vijay has been involved in research dedicated to combining agile practices with service-orientation. He is developing content for an upcoming book entitled Agile Service Reuse. Vijay contributed content pertaining to this research as well as additional methodology-related research to the book Next Generation SOA as part of a chapter on modern SOA methodologies.

He maintains a blog dedicated to software reuse at http://softwarereuse.wordpress.com/.

Contributions

rss  subscribe to this author

Bookmarks



Modern SOA Methodology and SOA Adoption Using Agile Practices - Part I

Published: August 9, 2010 • SOA Magazine Issue XLII
 

Introduction

Enterprises large and small are adopting Service Oriented Architecture (SOA) in recent years in order to gain cost savings from streamlined processes as well as create opportunities for revenue generation. Unlike previous information technology (IT) initiatives SOA places business goals front and center of the adoption strategy. Pursuing SOA as a technology-only effort will result in tactical wins for the enterprise but obtaining effective business results will remain elusive. The importance of SOA being relevant to business drivers and strategies cannot be overstated.

Success with SOA hinges on several key considerations such as the scope of the overall initiative, development methodology chosen, and the extent to which the effort is tailored to the enterprise‘s environment. There are a variety of SOA methodologies:

  • IBM’s Service-oriented modeling and architecture (SOMA)
  • Microsoft’s Motion methodology
  • OASIS Methodology of Service Architectures

All these methodologies provide specific techniques to identify, specify, and realize services that make up your enterprise’s SOA. They provide guidance on analysis and design activities that can be used to model services (using business, service, and technology perspectives) and implement them. Additionally, they provide architectural guidance for implementing services using a layered approach that leverages existing capabilities in the enterprise. From a governance perspective the standards and best practices, and policies for categorizing services, capturing and realizing quality of service characteristics are covered as well.

The methodologies also stress the importance of creating a SOA blueprint specifically tailored for your enterprise. This blueprint will align your SOA investments with business goals and serve as a guide as you undertake technology transformation across projects. The blueprint is typically made up of multiple phases and covers the salient business processes. It should assess the current state of the enterprise both from business and technology perspectives, identify major adoption roadblocks, and most importantly capture future state after adopting SOA. Adopting SOA is no easy undertaking and is the focus of this article.

There are three approaches that enterprises typically take when pursuing SOA:

  1. Top down
  2. Bottom up
  3. Meet in the middle

Top Down

The fundamental idea of top down is to build a service inventory to address an enterprise’s business needs starting from the organization’s vision, mission, and strategic goals. The top down approach can be pursued either to build an enterprise service inventory or a domain service inventory. The intent with an enterprise inventory is to identify and create a suite of services that will address the majority if not all of the organization’s business processes. In other words, the idea is to meet the needs of all the lines of businesses. Conversely, if the scope of the exercise is to build a domain inventory it is typically pursued to support a single line of business within the enterprise. This approach is typically a large scale enterprise-wide or line of business-wide effort that involves upfront analysis to understand the current state of the business, the change drivers, and the future state. A business case is created addressing how SOA will help the enterprise reduce costs, release new products, services, generate new revenue or increase customer satisfaction. The business case will outline the scope for investing in a services inventory and appropriate resources are allocated for the overall initiative. Closely following the analysis would be the identification, design, and implementation of services to automate business processes and functions. When top down is pursued at the enterprise level, the services inventory will be used to transform the portfolio of applications and processes into a leaner more nimble suite of business processes. The existing mix of applications including legacy systems and processes would then be redesigned to leverage the services from the enterprise inventory. The exact nature of the implementation effort is dependent on funding and business constraints.

The top down approach has a high probability of the SOA effort being relevant to the enterprise. A set of standards and best practices will be established to increase consistency and reduce incompatibilities among services. A governance function gets established as well for reviewing and evaluating architectures and service implementations thereby ensuring that Quality of Service (QoS) requirements such as availability, reliability, and scalability are met prior to placing services in production. Because of the detailed upfront analysis relevant business processes will be identified and services will be earmarked for use within these processes. The top down approach will ensure that there is minimal duplication of services across projects and a high degree of reuse across of the enterprise.

All this sounds great and you might be thinking about the need for other approaches. You need other approaches because the top down approach is the riskiest and the most expensive way to adopt service orientation. The big upfront analysis and design effort could run into months (if not years) prior to which the business landscape might have altered significantly. It is also important to realize that the initial analysis efforts might produce artifacts and not any production quality software deliverables. Additionally, the resources - time, money, and people - required for a top down approach are too hard to allocate in this era of budget cuts and overstretched information technology (IT) teams. IT typically has a backlog of projects and application development work that has to be fulfilled before resources can be freed up to pursue strategic initiatives. Top down sounds straight forward to pursue but extremely unlikely to be practiced in the real world.


Bottom Up

The bottom up approach to SOA is the opposite of top down, where development teams pursue services development as part of their project deliverables. There is neither an overarching enterprise SOA initiative nor any upfront investment into analyzing existing business processes and applications. The key characteristic of a bottom up approach is that the SOA effort is most likely a technology-only exercise divorced from buy-in and sponsorship from business stakeholders. This approach strives to identify, design, and implement services relevant to one or more projects. One or more IT team might embark on a services approach in order to fulfill their line of business requirements. Typically SOA governance concerns such as metrics, monitoring, and Quality of Service (QoS) characteristics are not managed at the enterprise level and it is left up to individual teams to ensure that due diligence is taken prior to deploying services into production.

The bottom up approach is low risk from the perspective of upfront investment in analysis, design and implementation. However, it is a high risk venture on several counts. Because bottom up efforts do not have explicit business alignment there is high probability that the services are not domain relevant and do not trace back to business priorities. As more teams start to pursue one-off SOA efforts there is an increased likelihood of ending up with duplicate and tactical set of services across the enterprise. They are likely to all have ad-hoc standards and practices and will not fulfill QoS characteristics in a consistent manner. For example, services exposed from legacy systems may not be integrated via an abstraction layer resulting in a rat’s nest of point to point integrations between service providers and consumers. Moreover, the bottom up approach may result in delivering services that are coupled to a specific project thus inhibiting the reuse potential for services across projects. A lot of teams in IT are either consciously or unconsciously undertaking a bottom up approach without fully appreciating the ramifications for both their team and the overall enterprise. The illusion of cost savings upfront will quickly lead to higher cost of maintenance and rework over the long term. Sustaining a bottom up approach is very challenging since it is unlikely to obtain funding without demonstrating business benefits.


Meet in the Middle

The third approach to pursue SOA is ‘Meet in the middle’ which strives to balance alignment with business goals and services development. Instead of taking a top down or a bottom up approach the meet in the middle aspires to provide a less risky approach to succeeding with SOA. The idea is to take the best of both top down a bottom up approaches and deliver SOA in an iterative and incremental fashion. You want to succeed with SOA by meeting real business needs while not breaking the bank. What you need is an approach that delivers services bit by bit within the context of an overall strategy.

The fundamental difference between the meet in the middle approach and top down is the scope and scale of the effort. While top down strives to build out an enterprise services inventory, meet in the middle focuses on realizing a domain inventory. This approach strives to fulfill the needs of a single business division or a line of business. Because the scope is more contained the meet in the middle approach typically starts in a single project and iteratively gets realized over time. Before concluding this approach to be a rehash of bottom up it is important to realize that there are several differences between the two. There is a business case established for SOA and it is explicitly aligned to business deliverables and constraints. The business sponsor is made aware of the need for SOA and the rationale for pursuing an iterative and incremental approach. There is a roadmap for SOA adoption and realization and that is fully aligned with business goals. The meet in the middle approach will strive to realize this roadmap incrementally transforming business processes. Additionally, though services are delivered within the context of projects explicit care is taken to make them project agnostic. Finally, there is a lightweight governance function to ensure that services are built using appropriate standards, guidelines, and consistent QoS characteristics.

Meet in the middle is low risk compared to both top down and bottom up because:

  • It is aligned with business needs thus increasing the chances of incremental funding for projects
  • Only attempts to realize a domain inventory as opposed to an enterprise inventory thereby significantly reducing the need for big upfront analysis and investment
  • Strives to build reusable services even though services are delivered within the context of a project
  • Does not result in a hodgepodge of incompatible or redundant services
  • Realizes business processes by utilizing services from the domain inventory thereby significantly increasing service reuse
  • Uses lightweight governance to ensure that QoS characteristics are considered and fulfilled in a consistent manner

If meet in the middle is a much better alternative to top down and bottom up then the logical question is how exactly to go about putting this in practice. Meet in the middle is about being lightweight but not compromising business deliverables. If meet in the middle sounds awfully close to the spirit of agile development, your instincts are right!


Realising Meet in the Middle the Agile Way

Many software development teams today are practicing agile software methodologies in order to continuously deliver working software in an incremental manner. They collaborate with their business partners, identify high priority features, and deliver them via one or more iterations. Using agile practices many teams are discovering that schedule risk and big upfront design efforts can be mitigated successfully. Discarding the hype, the successful teams have tailored agile practices to their enterprise. This entails carefully choosing a subset of agile practices to start with and not following practices at the expense of business requirements. These teams also adapt agile practices within the context of existing IT processes such as release management, incident management, quality assurance, enterprise architecture/governance, and program management.

There is widespread debate about the possibility of agile and SOA coexisting in an enterprise. One is about rolling up your sleeves and programming for today’s needs while the other is about big picture and reusable services for both today and the future. Although they seem contradictory agile and SOA have the same underlying intent and that is to deliver tangible business value using software on a continuous basis.

One way to pursue the meet in the middle approach is by explicitly tying service orientation goals and agile practices such as User Stories, Do not Repeat Yourself (DRY), Refactoring, Done-Done, Release and Iteration planning. Too many SOA efforts fail because they try to take on gargantuan task of transforming an enterprise and a much more focused, smaller in scope problem domain is ideal to realize a domain inventory that will be of value to meet both today’s needs and the future. At a high level:

  • Develop a roadmap for building a domain inventory of relevance to your enterprise
  • Prioritize services based on business requirements on a continuous basis
  • Leverage existing processes and applications for developing services. No need to start from scratch!
  • Refactor – either partially or completely – legacy services in order to build out the service inventory
  • Use a common architecture for designing and hosting services
  • Use design patterns and specific techniques such as software product line practices in order to support variations in service behavior of relevance to the domain
  • Never place technical perfection or design elegance over business goals. At the same token, use every user story in every iteration to align existing services and new services with the overall reuse roadmap
  • Use design reviews, code reviews, and pair programming to identify new services and refactor existing ones

Having reviewed the high level tactics it is time to explain how specific agile practices support the Meet in the middle approach. The section below covers a sub-set agile techniques and how they support the realization of a domain inventory.


User Stories

User stories need to be examined within the context of the overall domain inventory. Care must be taken to differentiate functionality for the problem domain vs. for a single application. The user story needs to be reviewed to see if existing services can be used as-is, new ones need to be developed, or existing capabilities need to be refactored. User stories are the primary means for getting requirements for services and can be used to recognize common functionality across projects. When the business brings up similar stories across projects you can map them to the domain inventory. In essence, stories can act as the bridge between business needs and domain inventory.


Don't Repeat Yourself

This is a fundamental agile practice that involves actively ensuring that a piece of functionality is implemented in a single place. Design reviews and code reviews are used to remove duplicate logic across existing and new services. These reviews need to consider multiple levels of granularity – source code snippets, classes, and services within the domain inventory. In order to increase the potential for services reuse, ensure that they do not have channel, technology platform, or transport specific logic coupled with the service’s domain logic. DRY principle thus keeps both the domain inventory and the applications that consume services lightweight. This principle is also relevant when leveraging internal and/or external solutions (such as open source solutions) for functionality that is outside of your core domain. For example, logging, web services infrastructure, and monitoring needs may be met by leveraging open source software. So why implement them again for your domain inventory?


Refactoring

Refactoring is the foundational technique for achieving agility and paying down technical debt in your domain inventory. It is heavily used for two primary purposes: iterative service development and legacy asset leverage. Refactoring is used to overcome both functional and technical limitations in your services. It is beneficial to pursue refactoring efforts with user stories. By aligning user stories to known set of service refactorings you can deliver business value as well as improve code quality in an iterative manner. Refactoring also used to wrap functionality from legacy systems and loosen coupling among multiple legacy systems. This will not only help reduce time to market but also leverage existing investments saving valuable resources. Continuous refactoring is also essential because subtle variations in your domain get understood over time and it is not possible to implement a service capability the “right” way the first time around. Additionally, business needs change over time and refactoring is a necessity to keep with the changes.


Done, Done

This is an agile practice which makes sure that a piece of code is complete from functional, testing, integration, packaging, and deployment perspectives. Being “done, done” with a service capability means implementing the service logic including necessary refactorings. Design and code reviews are done to minimize and if possible eliminate technology-specific or application-specific coupling when adding/updating services to the domain inventory. Additionally, service contracts must reuse XML schema definitions and data types and schema elements and attributes must be domain relevant and not specific to a single project. For web services described via WSDL documents tools such as the WSDL Analyzer from Web Services interoperability (WS-I) need to be used to ensure services can be consumed from several platforms. It also means writing unit tests, adding automated tests to a continuous integration/daily build process, and documenting service capabilities and limitations in the domain inventory’s service catalog.


Conclusion

The second part to this article is published in the September 2010 issue of The SOA Magazine.