ServiceTechMag.com > Archive > Issue XXXVI: February 2010 > SOA Governance: How Best To Embrace it - Part III: Governance Maturity, Tooling, Vitality and Success Patterns
Farzin Yashar

Farzin Yashar

Biography

Farzin Yashar is a Certified Solution Architect with the SOA Advanced Technologies of IBM's Software Group. In this capacity Farzin helps IBM customers in adoption and implementation of Service Oriented Architecture, creating Enterprise Architectures and SOA Roadmaps.

Farzin has 28 years of experience in the field of Information Technology in multiple industries. Since 1996 Farzin's special focus has been Enterprise Application Integration, Enterprise Information Integration and Process Automation using Service Orientation.

Farzin has a BSc in Chemical Engineering and an MSc in Computer Science. He has lead creation of the architectures for Banking, Insurance, Transportation, Medical Systems and Software Development Tools. He has worldwide experience in people management; he has directed teams of over a 130 individuals. Finally, he has held the title of Technical Director of Alliances building partner relationships and alliances. In this role he has presented and spoken at various Integration Events.

Contributions

rss  subscribe to this author

Bookmarks



SOA Governance: How Best To Embrace it
Part III: Governance Maturity, Tooling, Vitality and Success Patterns

Published: February, 2010 • SOA Magazine Issue XXXVI
 

Abstract: In part 1 and 2, we learned about governance, its lifecycle and the organizational aspect of an enterprise to support SOA and SOA governance. In the final part of the series, we will cover governance maturity, tooling, vitality and end this paper with governance success patterns.


Governance Maturity

If you don't know where you are, a map will not help. If you don't know where you are going, any road will do.

Usually, a maturity level provides a way to predict future performance within a given discipline or set of disciplines. But, most importantly, it helps to identify the gaps and helps in prioritizing the areas that are in need of development, improvement or enhancement. Hence, it tells you where you are and helps you to put a roadmap together.

Regardless of the current level of maturity, you need to understand that SOA governance maturity cannot be achieved overnight. It requires total commitment and some if not total culture change that is best achieved incrementally, following a roadmap with well-defined milestones and measurable results. The first step in achieving Governance maturity is assessing and measuring the organization's current status in each governance area. This measurement helps in identifying the focus areas and prioritizing their improvement accordingly.

Specifically, how SOA Governance relates to an organization with well defined ISO 9000 or CMMI processes. CMMI and ISO 9000 give direction on what a good development process must have (repeatability and verifiability of software, integration, and IT processes). The specific development process (such as agile, water-fall, latest-flavor, etc.) can be implemented under CMMI or ISO 9000. Implementing SOA does not mean that existing processes will not suffice, or may be thrown away. In fact, a governance model is required to provide a mechanism for infusing new concepts into existing processes and practices.

As such, SOA Governance is as much art as a science, but there does need to be more science than what SOA Governance practitioners have been using in the past. It is instructive to consider existing IT Governance models for maturity and then apply a services approach to realize an SOA Governance Maturity Model that we can use to understand the organizations governance needs in a more scientific, rigorous manner.

COBIT (Control Objectives for Information and related Technology) [REF-3] was created by the Information Systems Audit and Control Association (ISACA) and the IT Governance Institute (ITGI). COBIT provides a set of generally accepted measures, processes, and best practices for maximizing the benefits of information technology and developing IT Governance.

COBIT provides an IT Maturity Model, which itself is derived from the Software Engineering Institute's Capability Maturity Model (CMM). In the case of the CMM, for example, the basis for comparison would be the organizations' software development processes. The maturity model uses standard CMM-based maturity levels as modified by COBIT and described in the table below.


0 - Non-existent Complete lack of any processes. The enterprise has not recognized that this is an area to be addressed.
1 - Initial / Ad-hoc There is evidence that the enterprise has recognized that this is an area to be addressed. There are, however, no standardized processes; instead, there are ad hoc approaches that tend to be applied on an individual or case-by-case basis.
2 - Repeatable but intuitive Governance processes have developed to the stage where similar procedures are followed by different people undertaking the same task. There is no formal training or communication of standard procedures, and responsibility is left to the individual. There is a high degree of reliance on the knowledge of individuals and, therefore, errors are likely.
3 - Defined Process Governance procedures have been standardized and documented, and communicated. It is mandated that these processes should be followed; however, it is unlikely that deviations will be detected and corrected.
4 - Managed and Measurable Governance authorities monitor and measure compliance with governance procedures and takes action where processes appear not to be working effectively. Processes are under constant improvement and provide good practice. Automation and tools are used in a limited or fragmented way.
5 - Optimized Governance processes have been refined to a level of good practice, based on the results of continuous improvement. Governance is used in an integrated way across the enterprise to improve quality and effectiveness, making the enterprise quick to adapt.

Table 1 – Definition of Maturity levels


Measurement and Metrics

In order to identify the gaps, we need to understand exactly where we are and where we want to be. Then, we need to identify how the gaps might be filled. Maturity models help in doing that. Maturity levels are measured by the amount of success the organization has in achievement of identified goals for the areas or operational entities of the domain in question. In our case the domain is SOA Governance. So, we need to identify the areas within SOA Governance that need to be measured.

In-house measuring/assessments can be an efficient and cost-effective approach. Staff members can leverage their existing knowledge to do the measurements. However, they might not have the broad-based skill sets required to do this at the enterprise level and with a holistic viewpoint. In addition, this activity can drain vital resources away from other projects. Also, an inherent conflict of interest, political and organizational concerns, pride of authorship, and other factors may influence this measurement. In most cases, a guided measurement by a third party is more appropriate. They can distance themselves from the politics and issues more easily and usually provide unbiased and objective results. In-house staff members not too closely associated with the governance function, combined with expertise from a third party, can be an effective approach.


What Should be Measured

All the areas and components related to the SOA governance must be measured. There is no standard list of such areas and components. The following diagram shows examples of the areas that could be measured


View Larger Image
Figure 1 – Example of Governance areas for measurement

Prioritizing Governance Areas

Properly designed and implemented, SOA governance mobilizes SOA which in turn can help an organization reach its business goals more quickly and increase its level of agility and responsiveness to its market and consumers. A big bang approach is not recommended in implementation of SOA and neither is it recommended in implementing SOA governance. Assessing governance maturity level and finding gaps is the recommended way to identify the areas of focus that then can be prioritized for a phased implementation. However, lack of such a study should not prevent progress in SOA adoption. Until such time as a maturity assessment is completed, there is an opportunity to identify other priorities. The following is the recommended approach:

  • Focus on service lifecycle and its governance needs. Remember that governance is all about transparency. If you look at the service life cycle as a process in which the first step could be proposing the idea of a service and the last step could be service deprecation; then, the governance for the service lifecycle must clearly state what should be done in each step, the policies, standards, and patterns to be used or enforced in the step, and how it is done and what role(s) will be doing it. For example, everybody should know about the required policies and standards for a service and who should be reviewing the results.
  • Review the existing project lifecycle and identifying the impact of services on the project lifecycle governance.
  • Finally, you could look into the lifecycle of the enterprise transition to SOA.


Figure 2 – Recommended approach to implementing SOA Governance when no formal Governance Measuring/Assessment is done

Governance Tooling

Tooling around SOA governance is evolving as best practices and standards are being identified and developed. In this section we do not propose tools from specific vendors. Rather, we try to discuss governance needs generically in order to guide development of tool selection criteria.

There are many alternatives in coming up with the criteria that one could use to select the required tooling. In this paper we touch upon two alternatives. Of course, combinations of these alternatives will provide many other possibilities. The focus of the first one is governance areas and components that each area may cover. The second alternative is governance control points. There are no standards or best practices in these areas available at the time of this writing. Identification of such criteria is one of the functions best suited to the SOA Center of Excellence.


Identifying Tooling Criteria - Alternative One

A very pragmatic way to hone in on SOA tooling requirements is to identify the areas and activities in the enterprise that SOA Governance should cover, break the areas into components and find out how tooling could help. The following is an example of such a breakdown. For each area/activity, two components, as examples, are identified and briefly described. The CoE or any surrogate team functioning as the CoE in any organization should complete this breakdown and identify their exact tooling needs. By doing this, one may realize that some of the existing tools could satisfy some of the SOA governance tooling needs. Obvious examples are requirement management software packages and Asset Management Software packages that handle assets such as "Services" and "Policies" as enterprise assets.

Here is a list of the possible breakdown and two example components:

  • Strategy
    • Business Vision - Creation and maintenance of the business vision for an agile enterprise. Creation and maintenance of a business architecture that identifies service domains, business functions, business processes and the mapping of those to SOA services and IT applications.
    • Standards and design patterns - Relevant standards and patterns that must be followed during development life cycle service deployment and beyond.
  • Enablement
    • Ownership & Funding - Who funds what and how rights and obligations are distributed.
    • Vendor Management - Managing policies and standards for products and vendors.
  • Development
    • Regulatory Compliance - Ensure compliance with legal, state, county and international regulations.
    • Service Certification - Creation, maintenance and utilization of standards for a service certification process.
  • Program/Project management
    • Risk management - Identify sources for risk.
    • Change management - Processing change requests and providing input for version management.
  • Operations
    • Service Support - Problem reporting and support. Release management.
    • Service Monitoring - Setting performance measures and monitoring them.

Identifying Tooling Criteria - Alternative Two

As an alternative, a team could concentrate on the control points that are relevant to the SOA Governance. Again, there are different ways to go about this. We could use alternative ones described above and identify the control points. A better way would be to go through Service Life Cycle and the operational aspect of a service and identify the control points.

Before introducing a suggested list of control points, let's defines what control points are. Control points provide an opportunity to measure a process and decide whether the execution of the process needs adjusting. Certain critical activities within a process may be associated with a control point. At the end of each identified control point activity, the governance function decides if the process is ready to move to the next activity.

Knowing what decisions within the process are critical and understanding what measurements are needed as input to those decisions helps decide where control points are best placed. This is an important part of governance.

The following is a suggested list of control points with some examples of the requirements for tooling support:

  • Business Requirements Control Point - Business goals and the requirements driven by these goals. Identification and measurement of Key Performance Indicators.
  • Solution Architecture Control Point - Standards and policies to be followed. Architectural decisions are being recorded and maintained.
  • Service Identification and Specification Control Point - Identified services are in line with goals and objectives. Service specification is complete.
  • Service Design Control Point - Design policies and standards are being followed. Data messaging model and data access patterns are being used.
  • Service Build Control Point - Rules are policy based. Existing services are being considered.
  • Service Test Control Point - Load and stress testing. Security testing.
  • Service Certification & Deployment Control Point - Verify compliance.
  • Service Vitality Control Point - Governance processes and procedures are up-to-date.

Governance Vitality

As SOA evolves, governance processes will also need to evolve in order to stay relevant. This evolution of governance processes is the essence of Governance Vitality. In addition, changes in many aspects of an organization warrant changes in governance processes as well. In an environment where SOA is chosen to enable fast reaction to market and customer needs and agility is a concern, governance needs to always be the pillar to support and embrace the required changes. Governance is not a one off job. It is not possible to establish governance processes and expect them to run forever. In a real world everything changes and so should the associated governance processes.

In this section we present the manner in which organizations could maintain their governance vitality. We also review events that may trigger changes in governance processes.


How Is It Done And Who Should Be Doing It?

There is no defined way to assure governance vitality; indeed there should be vitality in the vitality process itself! As part of the governance planning process, there should be thought given to what aspects of the SOA journey should be measured. There should also be thought given as to whom, and how often, should review the metrics. Suggestions, based on observations, should be summarized into a plan that suggests changes to aspects of the governance process. There should be an explicit mechanism in place to log ideas and set up review meetings and ensure that a well-defined decision-making process is followed for every log entry. Governance vitality is an important task that usually is initially associated with a member of the CoE team. Once SOA governance is established, and its relationships with IT and Enterprise governance are defined, then it will be easier to identify who should be responsible for governance vitality and how the SOA Governance Vitality processes are triggered.

Best practices suggest that involvement in standard bodies also helps keep the organization abreast of industry changes. Staying abreast of technology standards is often associated with organizational efficiency. Involvement in standard bodies also provides enterprises with the opportunity of influencing standards bodies to ensure that industry standards are relevant to their enterprise.

The key to successful governance and ensuring governance vitality is education on how governance and the associated tooling helps improve daily tasks, as opposed to getting in the way. In this fashion, everyone becomes a stakeholder in improving governance. The responsibility for governance vitality will be embedded in the entire organization.


Events Triggering Review of the SOA Governance Processes

Certain events trigger governance vitality activities. As noted before, SOA governance is an augmentation of, and an extension to, existing IT and Enterprise governance; therefore, these triggers are useful frameworks to identify the where, what, and for whom governance in general should be considered appropriate.

  • Business Strategy changes. Any change in business strategy may cause removal or update of existing governance processes, procedures and policies. A business strategy change could be as simple as adding or removing one of the communication channels with the clients/consumers/partners, such as removing direct mail and adding email; or as complex as outsourcing a division of the business.
  • Business Process changes. For many reasons, business processes may change. This could be as simple as a change in sequence of activities, to the addition of new activities or removal of existing activities within processes. We need to always determine the impact of the change on the governance processes. Some organizations are adopting the notion of Processes Governance for their Business Process Management (BPM) initiatives - SOA Governance vitality would benefit from governance mechanisms should they exist.
  • Organizational changes. Decision rights may change as the result of organizational changes. This triggers review of the governance processes and procedures
  • Legislative and Policy changes. Legislative and policy changes can require significant changes to company operations (e.g. SOX, BASEL II etc) and as the result require changes in the governance processes and procedures.
  • Changes in Standards. Standards usually replace proprietary parts of the governance processes. Lack of standards force enterprise to devise their own way of doing things. Once standards are developed and available they warrant review of the existing governance processes to identify required changes.
  • Technology improvements. Organizations should always be on the lookout to automate processes as a way to improve productivity and save money. New technology can make existing tasks or processes easier, more accurate, or provide more control. In many organizations, there is a formal body charged to run a technology scan, this trigger is normally tied to their output
  • Assessment of current governance effectiveness. As part of the SOA 'measure' phase, metrics should be maintained that indicate the effectiveness of SOA adoption. SOA Governance is responsible for periodically reviewing these metrics and making the needed changes to governance policies, standards, and processes. Metrics for governance vitality tend to vary for each organization, but tend to include measures such as: Number of Control Gate Meetings (a low number probably means the process is not working or being bypassed), Number of Services Certified (is the certification process too stringent, is it being used?) and so on.

Conclusion: Governance Success Patterns

Successful patterns are derived from many years of successful practice in given areas and are a sign of maturity for that area. SOA, by no means, has reached the level of maturity associated with proven patterns in some other disciplines. We are still in the relatively early stages of SOA adoption. However, SOA Patterns are now emerging and we are at a point where we can talk about patterns that have been leading indicators of SOA success so far. In this section we will discuss the top 10 leading practices that have proven to make SOA adoption successful and the role of SOA governance in successful SOA adoption. The following patterns are presented in no particular order. (No priority or ranking should be inferred from the presentation.)


Assure Executive Sponsorship/Champion

The SOA journey requires executive level attention. One or more executive sponsors who are enthusiastic about SOA and realize its benefits could have a great impact on getting the SOA journey on its way. Such executive(s), on the one hand would help evangelize SOA and on the other hand will have the power to make the required resources available. Ideally the sponsorship should be a joint effort between Business and IT. SOA governance should emphasize and encourage executive sponsorship/champion especially at the early stages.


Create Real or Virtual/Interim CoE

Governance requires a supporting organization of some sort, no matter what you might choose to call it. The creation of an SOA Center of Excellence, as an example, has proven to be extremely helpful in many organizations. In cases where it is early in the SOA adoption process and the enterprise needs some more time to create a physical organization, then creating a "virtual" CoE could be helpful. A virtual CoE does most of the work that a real CoE does without a formal structure. This should be a very short-lived phase for the CoE. The benefit of having a virtual CoE instead of no CoE at all is that it can provide a short-term focal point for necessary initial SOA activities, albeit in an informal fashion. On the negative side, because the CoE is not formal, it may not have the required support and may not produce the needed results. Virtual, or interim CoEs, are often created by SOA pioneers in the enterprise. Candidate individuals include those in the enterprise who believe in the benefits of SOA and have a passion to establish SOA in the right way, and who understand the chaos that will results from the creation of services in an ad-hoc manner in an unstructured environment.


Communicate Business Values

The best sales people are those who first buy, and wind up believing in, the product they are supposed to sell. It simply means that they really understand the product and truly believe that it delivers on all of its promises. SOA has a huge promise of business value. SOA governance must assure that this becomes a reality and there must be a clear means of communicating that to all stakeholders. That way, everyone who is impacted has an understanding of the goals and objectives and can act as an evangelist for promoting the business value of SOA.


Grow into SOA, Don't Jump Into It.

The concept of service orientation and the agility enabled by the creation of reusable and consumable services has created an excitement in the technical and business community. In some organizations, services are being produced faster than traditional applications. The rigidity and lack of flexibility of the legacy applications forced us to consider a services approach. If we do not pay attention to the way we create new services, if we do not think about their granularity and if they are not created with a holistic view of the enterprise, these services become the smaller versions of the old stove piped applications. Instead of a dozen large rigid applications you may end up with hundreds of smaller rigid services. Due to lack of governance, there can be many variations of the same service that will result in additional development and support costs.


Adopt Policies

Policies make decision making faster and better. By having policies for situations that are likely to happen, management of those situations could be automated and less will be left to chance for those situations where explicit instructions are not available. Exceptions will always require human involvement and a decision making process. Policies play an important role in the service development life cycle, as well as management and monitoring. Therefore, policies become extremely important in SOA governance. As an example, a policy such as "an enterprise service needs to have a minimum of three consumers" helps in deciding what qualifies as a centrally hosted, enterprise-level service. Any service candidate that does not warrant three or more consumers could automatically be rejected, or simply classified as a departmental service and associated with specific resource constraints.

Policy based management is used in Network Resource management and makes it easier. Security measures, including access to portions of the network, or control over quality of the service, are best addressed through policies. In Policy Based Management, Policy Enforcement Points (PEP) and Policy Decision Points (PDP) are two components worthy of mentioning. They make tremendous sense in SOA environments. A PEP component sends a message to the PDP component asking if a condition in the policy is met, or not, in order to allow the process to continue, or stop the process. Governance plays a crucial role in defining PDPs and PEPs as well as their behavior.

The Control points discussed in Section 6 are analogous to PEPs.


Measure Every Step of the Way

Measurement is key to effective governance. Without measuring effectiveness, vision, mission and goals are soon forgotten. Good governance enforces defining and solidifying Vision and Mission statements; clearly defining goals and a set of metrics identifying when goals are achieved. This ensures there is a viable measurement capability to see how business objectives are being attained.


Employ Tooling and Establish a SOA Lab

Enforcing governance at times becomes repetitive and mundane. Tooling and automation has proven to be instrumental. Of course, many factors play in selecting the type of the tools and their functionality. Please use the guideline in section 6 for choosing the right tools as they play an important part in a successful SOA journey.

Put in place a SOA Lab, to conduct early prototyping, validation of technical concerns (hardware and software) and testing of processes and procedures such as Service Development Life Cycle. This way, technical obstacles are identified early on. Usually, the CoE runs the lab. In fact, one of the early activities of a CoE should be establishment of a SOA lab. The CoE needs to identify what should be in the lab, how and for what purpose the lab will be utilized, when the lab should become operational and for how long.


Understand your Maturity Level

Many organizations start service oriented architecture by introducing services into their IT environment and after a while they believe that they have implemented SOA. But, SOA is not only a bunch of services that are created because a few IT individuals decided to write Web services instead of applications. Services need to be created with a holistic view of the organization in mind. In order to understand how to proceed, we need to realize where we are. Therefore, understanding the current maturity level is very important. This maturity assessment should be done on Enterprise and IT governance as well as assessing the state of service orientation.


Create and Govern a SOA Roadmap

Creation of a SOA roadmap is an integral part of a successful SOA journey. This is not by any means an easy process as it requires understanding of the current organization, clear understanding of the vision, the mission and the goals of the organization and how they can be met through service orientation. Creation of a roadmap and subsequently governing its implementation is one of the most time consuming and difficult aspect of a successful SOA journey.


Govern the Return of Investment

In the business world, any investment should be associated with a return on that investment. No return on investment is unacceptable. Moving to SOA is no exception. Defining the right metrics in capturing and measuring return on investment has helped enterprises of all kinds in controlling their costs and achieving their goals faster. This is one of the most important functions of governance and a huge success factor for SOA overall.


References

[REF-1] "The Next Revolution in Productivity", Harvard Business Review Article, 1 June 2008, Ric Merrifield, Jack Calhoun, Dennis Stevens, http://harvardbusinessonline.hbsp.harvard.edu/relay.jhtml?name=itemdetail&id=R0806D

[REF-2] IBM EA Academy Study Team, Orlando Workshop, 12th-13th March 2004

[REF-3] See COBIT Framework, Page 19: http://www.isaca.org/AMTemplate.cfm?Section=Downloads&Template=/ContentManagement/ContentDisplay.cfm&ContentID=34172.

[REF-4] SOA and the Emergence of Business Technology: How Business Services are Changing the IT Landscape. Farzin Yashar & Robert Laird. http://www.soamag.com/I4/0207-3.php.

[REF-5] Several writings and presentations from Clive Gee, Phd.

Acknowledgements: I would like to thank the following individuals who reviewed and contributed to the original paper: Les Robinson (Boeing), Jay Pollack (Computer Sciences Corp), Bob Riley (Harris Corp), David Almeida (Harris Corp), Mike Moomaw (IBM), Robert Laird (IBM), John Falkl (IBM), Al Secen (Lockheed-Martin), Chris Francis (L3 Communications), Peter Bostrom (Oracle), Kathy Kearns (SITA) and Mansour Rezaei-Mazinani (SITA). My special thanks go to Mr. Robert Laird who generously volunteered his time and expertise providing me with his input on every section of this paper.