ServiceTechMag.com > Archive > Issue XXXIII: October 2009 > Preventing SOA Failures: A Revealing Interview About Effective SOA Governance
David Berry

David Berry

Biography

Dave Berry has more than 25 years of technology experience ranging from mainframes, integration, and SOA technologies in financial services, government, and vendor environments. He currently works in Oracle Server Technologies as a Senior Manager in the Fusion Middleware SOA Integration product team.

Oracle's comprehensive, standards-based family of middleware software, Oracle Fusion Middleware enables customers to adopt and manage SOAs in heterogeneous computing environments. More than 35,000 customers now use Oracle Fusion Middleware and include leading organizations in the Financial Services, Telecommunications, Manufacturing, Retail, Pharmaceuticals, Health Care and Public Sector industries. Oracle Fusion Middleware is also supported by 9,000 partners, including leading independent software vendors, value added resellers and system integrators.

Contributions

rss  subscribe to this author

Mike Van Alst

Mike Van Alst

Biography

Mike Van Alst has been the Oracle ACE Director since 2008. He has been working in the IT industry since 1984. He started out doing mainframe programming, but in the subsequent years he shifted focus to methodologies and architectures. Mike is currently working as an architect for IT-eye in the Netherlands.

Contributions

rss  subscribe to this author

Bookmarks



Preventing SOA Failures:
A Revealing Interview About Effective SOA Governance

Published: October, 2009 • SOA Magazine Issue XXXIII
 

Abstract: Governance requires collaboration. In order for projects to be considered a success, governance is an absolute must - this is particularly true of SOA projects. In this article we combined a number of common mistakes and remarks from a range of IT architects to illustrate and highlight common and critical governance issues. This article presents these issues in a mock interview with a fictional IT architect. All the examples in the interview are based on real-world experiences.


Could you give us some background of your company?

I work for a midsize local government. We have about 15,000 employees in 12 departments. I am the lead architect, responsible for architecture and governance. Driven by a real need for agility in our business processes, we embarked on a journey toward a service-oriented architecture about five years ago.

One of the first things we did was create a company architecture (often referred to as a reference architecture) which would define the overall architecture for the next few years. The component architecture described all the different (generic) components we expected to need in the next few years. But we didn't need all the components and all their functionality right at the start, so we defined several levels of functionality for each component. We based these levels on dependencies with other components and on business drivers-times when we would need specific functionality. What we in fact did was create an SOA roadmap to manage the creation of new functionality.


That sounds like a good approach. How did that work out?

It is a good place to start and we feel that we have been successful in building the architecture; however, not all components are completed and we continue to move forward until our road map is fully implemented.


What role did governance play at the start of the initiative?

When we started our SOA initiative we thought we could wait before introducing governance. We had more-pressing issues around planning and the strategy of the initiative itself. However, looking back, that may have been one of our biggest mistakes. We have seen some initiatives go sour because of the lack of governance. Governance must be implemented at every stage of the SOA lifecycle to track ongoing changes to the architecture, design, and implementation.


Could you give us a specific example?

We defined a case management component that was to be our centralized component, responsible for registering and tracking all cases. A case is any activity that our local government executes on behalf of our citizens and/or local companies. We started out with small proofs-of-concept, so we felt that we had everything under control. The proofs-of-concept were quite successful, and that made several departments very eager to join in the SOA initiative. Unfortunately we still didn't have any governance in place and not enough people within our core architecture team. Each project team did their own thing, though most kept within the bounds specified in the architecture.


What happened?

Four different departments felt the need for a case management component. Each department looked at the component architecture and decided to build the part of the component they needed. The Maintenance department was the first to build its own component. Unaware that the Maintenance department was already building one, the Social Security department also began building their own component. In the end there were four decentralized components instead on one centralized one. This structure did not fit in with our business requirements.


Do you think governance would have prevented this?

Yes. Every department did what they thought best and tried to follow the architecture. Nobody thought about checking with the architecture team or the other departments to see if they could work together. As a result, none of the departmental initiatives were aligned, and quite a bit of time and money was wasted. Just a little bit of executive governance would have prevented this, and would have saved almost 50 percent of the investments.


What is being done to prevent that from happening again?

After we realized that the problem was caused by the lack of steering, we initiated an Architecture Board, where all departments had to present their initiatives. The Board not only determines whether the proposals adhere to the architecture, but it also makes sure that no duplicate efforts are under way. Furthermore, the Board would help the departments align their initiatives with other departments. And almost every department is represented on the Architecture Board.


So is it just a matter of aligning initiatives?

That would have been nice, but not quite. As I mentioned, we had four components instead of just one. When the Finance department began their implementation of a case management component, they knew about the Social Security department's initiative. However, they decided not to make use of that initiative.


Why?

There are always departmental politics at play. When I talked to the technical architect responsible for the Finance department's project, he looked at me and said, "You don't think I'm going to rely on their implementation, do you?"


Would you say departmental politics are one of the most challenging aspects of governance?

Trust has to be earned. Within a political environment such as a local government, departments have a natural resistance to anything that originates from the other departments. They tend to be proud of their own independence. But even within departments we see this happening. The good thing is that it is rapidly diminishing.


Can you explain why?

As I said: trust has to be earned. Gradually, some components are being built in a collaborative fashion. This means that the quality of the component increases, because these components usually are more generic than they would be if they had been built by one department alone.


How did you get the departments to collaborate?

That turned out to be easier than we thought. The first thing we did was get all departments involved by inviting them into the Architecture Board. At those meetings when we discuss the upcoming initiatives, they usually see opportunities for themselves. Working together inevitably leads to better results and lower costs. And once they've seen that the collaboration works, it is kind of self-perpetuating.


Can a good governance strategy facilitate that collaboration?

Absolutely! Governance is aimed at stimulating desired behavior. That means you have to be actively involved. In our case this means that the central architecture department is actively participating in all projects that have passed the Architecture Board. We're not sitting in our ivory tower and wondering why our projects go wrong. Our architects bring the project-architects together to learn from our successes and failures.


Does it work?

It takes-and still is taking-time. You can't expect changes such as these to be implemented quickly. It's a kind of a cultural change-looking at governance from a more collaborative viewpoint. There is a certain amount of mandate necessary to make it work. Sometimes architectural concerns for the whole municipality outweigh the interests of a single department. What certainly helps is to show people how governance helps them. I do think that we're definitely moving in the right direction.


So in your opinion governance is mostly about collaboration and alignment?

When we started out, we felt that governance was mainly aimed at managing the lifecycles of all the assets you create in your company. Now we feel that even though governance is about enforcing rules and guidelines, it has become a much more collaborative affair, meaning it is about people and how they work together. It is also about the process of building and maintaining assets, responsibilities, and ownership, as well as the tools you need to be able to manage that.


Can you explain what you mean by tools?

Of course, especially in an SOA environment, you need tools to manage the architecture itself. You need to manage the collaboration, the ongoing initiatives, and all your assets, particularly the interdependencies. One of the characteristics of an SOA environments is that you have a lot more assets to manage. Managing means not only developing and deploying them, but also guaranteeing service-level agreements. The more assets you have in place, the harder it gets to find the right asset that you want or need.

Which raises the question of ownership. Who owns the assets? Who is responsible for managing the assets? We had to answer these questions before we were able to effectively reuse assets. This is even more relevant for public organizations such as ours. Departments are very reluctant to give up their independence by relying on other departments' assets. This is all part of governance as well.


Do you feel that governance touches many parts of the organization?

Yes, I do. It affects everything from enterprise architecture to the lowest level of technical assets. Each level has its own unique requirements. The architectural level of governance is mainly aimed at getting the departments working together and doing things correctly. You could say the Architecture Board owns and maintains the component architecture. The Board realized that it also needed to formalize the way we build and deploy services. The staff architects established standardized practices for delivering, among other things, a shared service portfolio to promote and implement best practices around reuse of common components. The key word here is process!


What is key about process?

Collaboration in and of itself is obviously very nice. However, because there are so many people involved, there is a clear need to define roles and responsibilities - both from an architectural point of view and from an ownership and development perspective. Governance provides the structure for the process, especially in regards to rules and guidelines. There needs to be a standard process for everything, starting with defining an initial project architecture, all the way to deployment. One important aspect of that process is a feedback loop to make sure that lessons learned are shared with everyone in the whole organization.


What kind of tooling do you need in an SOA environment?

First of all, it is important to look beyond just the technology and understand how people and processes enable the collaboration of stakeholders. You have to understand that, before you can think about tooling.

Having said that, the principal idea of service orientation is business agility, which is achieved by orchestrating and reusing existing assets. The basic premise remains that if there are assets and services out there that can be reused, doing so will make any other initiative faster and cheaper. To facilitate reuse, you need three things: insight into available services, insight into ongoing development in the services, and insight into consumers of services.

For us, the central tool in a governance environment is the asset manager. The asset manager provides a single source of truth for architecture design and project implementation. Assets must be managed across every stage of the lifecycle, from conception to implementation, and from deployment to retirement. A good asset manager includes features or functionality for automation, workflow approvals, extensible notifications, and event infrastructure.


Is tooling primarily used to automate the governance process?

In a way, yes. Automating the process is meant to support the people during execution. However, tooling also helps us become proactive rather than reactive. Having all the information in one place, makes it possible to check on adherence to standards and best practices, monitor service usage, and signal discrepancies.

In the end, tooling is really meant to make the process more predictable and drive continual improvements to quality of service. It ensures that proper procedures are followed, and that no changes occur that are not properly defined and accepted. The hardest part for us was to define ownership and responsibilities of all assets.


How did the tooling help that?

We built a matrix to illustrate how each stakeholder should be engaged at each phase during the lifecycle of assets. We found that there are a variety of requirements for engaging each stakeholder at the appropriate phase change. Notification of phase changes is one of the basic instruments to involve the stakeholders.

An example of good governance tooling is enabling stakeholders to choose their own tools. Every department has its own exchange server. So each stakeholder expected to be able to approve promotions using their native Outlook clients. This was accomplished by enabling them to respond to notification e-mails with an approval template that activates a workflow engine integrated with the asset manager's built-in approval process.


Any closing statement you'd like to share with us?

Governance is a tough process to implement, simply because nobody likes to be governed. But, if we approach governance from a more collaborative perspective, the advantages become quite clear. There's a benefit for all participants in the governance process and collaboration. Collaboration should be supported by structured processes and insight-giving tools. In the end, the balance between these aspects is crucial in any implementation.