ServiceTechMag.com > Archive > Issue XXXII: September 2009 > A Case Study on SOA and Process: Integrating E-Gov Travel Services with Federal Agency Financial Systems - Part I
Jian Jeff Zhong

Jian "Jeff" Zhong

Biography

Mr. Zhong is Chief SOA Architect at Futrend Technology Inc. In this capacity, he provides thought leadership, architecture guidance and hands-on implementation support for Futrend's federal clients in the areas of E-Government and mission critical systems' integration. He has been working in various architect roles for more than a decade and has extensive experience in software architecture, application architecture, enterprise architecture, security architecture, and Service-Oriented Architecture. In the last ten years, he has been providing professional services to clients in the public sector such as National Institute of Standards and Technology (NIST) of the U.S. Department of Commerce, the National Institutes of Health (NIH) at the U.S. Department of Health and Human Services, the Energy Information Administration (EIA) of the U.S. Department of Energy and the General Services Administration (GSA).

He is a Sun Certified Enterprise Architect (SCEA), and is certified in Architect, XML and ITIL from industry leaders such as Sun Microsystems, Oracle and IBM. Zhong earned his Masters of Science degree from University of Maryland College Park. He has had numerous articles published on Java EE, enterprise architecture, enterprise Single Sign-On, and XML related technologies. Mr.Zhong can be reached at http://www.linkedin.com/in/jeffzhong

Futrend Technology, Inc is a fast growing system integrator/consulting company specializing in major mission-critical federal business systems integration. Futrend has a proven track record in systems integration with major financial systems such as Oracle Financials, PeopleSoft Financials, Federal Financial System (FFS), and Momentum for federal agencies such as CMS, HHS, HUD, NIH, and Smithsonian Institution. For more information about Futrend's SOA practice, please visit http://www.futrend.com/soa

Contributions

rss  subscribe to this author

Bookmarks



A Case Study on SOA and Process:
Integrating E-Gov Travel Services with Federal Agency Financial Systems - Part I

Published: September 25, 2009 • SOA Magazine Issue XXXII
 

Abstract: In this article I will present a step-by-step service-oriented solution development methodology and then describe how it is being used for the successful implementation of Service-Oriented Architecture (SOA) at a federal agency to integrate its financial systems with E-Gov Travel Services. I will discuss major service components constructed, service-oriented processes followed, business benefits gained, best practices used, and lessons learned. After reading the article, readers should have a good understanding of how SOA is used in real world situations to integrate large scale, highly valued, and mission critical information systems within and across enterprise boundaries.


Introduction

A properly utilized methodology can consistently solve problems effectively and efficiently with great results. One of the best examples is "The Art of War", first drafted and practiced by the ancient military commander and king's advisor Sun Tzu. His methodology, when studied, fine-tuned, and applied in the real world by his followers, produced grand victories for thousands of years [REF-1-3]. The architectural principles and design patterns of Service-Oriented Architecture (SOA) are similar to the war principles and thirty-six stratagems presented in Sun Tzu's masterpiece [REF-3]. Both the "The Art of War" and SOA place deliberate planning, agility, and flexibility as strategic goals in order to respond to fast-changing environments, factors, and changing business objectives [REF-4-7]. In this sense, SOA can be considered "The Art of Services" - the way to use services effectively and efficiently to produce desirable and predictable business results. To be effective, the right services for the right situation must be designed and implemented. To be efficient, services must be identified and constructed on schedule, within budget, and with the promised quality. By extracting and documenting the patterns from successful engagements, leaders in any discipline can be assured of success. This paper will document one such successful engagement.

Methodology is so important, but where should one start? One way is to research industry standard bodies [REF-8-11], major vendors [REF-12-19], and prominent SOA experts [REF-20-23] and reflect on their core principles and best practices. The next step is to embrace, adapt, and fine tune the information to fit your own situation in order to solve your own business problem. In articles published in JavaWorld, I streamlined the Rational Unified Process and summarized its eight steps for J2EE solution development [REF-24-27]. I further elaborated how SOA should be used to migrate an "as-is" to a "to-be" enterprise architecture of an organization [REF-27]. This article will refine my previously published SOA concepts to formulate a simple and practical methodology, based on my past years of SOA practical experience and emerging SOA methodologies from industry experts [REF-4-7, 28, 29]. To develop an executable SOA solution, eight steps (activities) must be performed in one or more iterations. Those steps are:

  1. Requirements Analysis
  2. Service-Oriented Analysis
  3. Solution Architecture
  4. Service-Oriented Design
  5. Service Component Development
  6. Service Component Assembly and Deployment
  7. Verification and Validation
  8. Operations and Management

I will now present a case study to show how the SOA methodology is used to solve a real world business problem. The Art of War says one must know the big picture before acquiring the important details. So before I begin the steps, I will describe the business background.


The E-Gov Travel Services

The "E-Government Act of 2002" was passed by the United States Senate to improve electronic government services and processes by using Internet technology. Under the President's Management Agenda, the E-Gov Travel Services program [REF-30-31] was one of 24 E-Gov initiatives that were created. As a separate contract line item, E-Gov Travel Services provides integration services to allow the electronic flow of travel data to and from federal agency financial systems and the support of the disbursal of travel-related funds, the synchronization of traveler profiles, and the validation of accounting code information.


Agency Enterprise Business System

The federal agency in this article has an annual budget of over $30 billion (FY2009), with 84% of the total budget supporting over 325,000 extramural scientists and research personnel at more than 3,000 institutions nationwide [REF-32]. Their scientists and physicians travel world-wide to conduct medical research and the agency travels patients for clinical trial studies. The Agency Enterprise Business System (AEBS) is the system of record for all accounting and disbursing transactions processed within the agency. Major functions of AEBS include Accounts Payable, Accounts Receivable, Acquisition and Contracts, General Ledger, Procurement Management, Property Management, Service and Supply Funds Operations, and Travel Management.


Architectural Scope and Constraints

In order for SOA to be productive and gain acceptance, it must solve real world business problems that have critical business impact and have enough importance to secure project funding. Business problems arise as the in-house hosted travel management system is migrated to E-Gov Travel Services. Direct database link has been used to integrate the legacy travel system with backend financials. As the agency migrates to E-Gov Travel Services, direct database access is not practical and it becomes challenging, yet necessary, to allow the flow of financial and travel data between E-Gov Travel Services and AEBS.

The AEBS E-Gov Travel Services project is led and managed directly by the AEBS Director with oversight from the the agency's Chief Financial Officer. The enterprise architecture guidance and infrastructure support is provided by the agency's CIO, Enterprise Architect, and their staff.

This SOA integration approach can be used for E-Gov Travel Services AEBS integration as well as for future system integration projects such as eGrant and other internal and external systems. It is consistent and in compliance with Enterprise Architecture and SOA guidelines and practices [REF-33-37]. The architecture and design, as directed by agency management, is:

  • Enterprise scope, reusable across different projects
  • Forward compatible to the next Oracle applications (Financials) release
  • Leverage existing TIBCO and/or Java EE applications
  • Leverage industry best practices and XML data standards

A Step-by-Step SOA Methodology

Requirements Analysis

Federal travel is a set of very sophisticated business processes governed by the Federal Travel Regulation (FTR) [REF-34]. The E-Gov Travel Services involve many business systems including the agency business systems, bank travel card systems, data warehouse and report systems, global distribution systems, travel authorization and voucher system, travel management centers, online booking engines, etc. Readers who are interested in the details can research the E-Gov Travel Services RFP [REF-31]. The RFP presented 11 high-level business use cases common to all agencies:

  1. Create and Update Traveler Profile
  2. Prepare and Submit Travel Requests
  3. Book Travel & Reservations
  4. Change Itinerary
  5. Prepare and Submit Voucher
  6. Scan and Submit Receipt Images
  7. Retain Receipt Originals
  8. Designate Payment Modalities
  9. Receive Itineraries, E-Tickets, Notifications and Reminders
  10. Generate Personal Travel Reports
  11. Perform Personal Travel Queries

Travel at the agency has many unique business requirements. Overall, medical research and science are really about uniqueness. Each scientist, and associated institute, is very special and often has unique requirements within various levels of subordinate organizations. There are many business requirements; below, I describe only two to show that generic federal E-Gov Travel Services cannot be used for the agency without modification and improvement.

First, only travel planners use E-Gov Travel Services to plan a trip, book a ticket, or submit vouchers to reimburse expenses, while travelers use eVoucher to sign and certify travel documents with a few simple steps. The eVoucher Web application, previously developed specifically for the agency, reduced the huge amount of time, resources and effort that would be required to train (and maintain) more than 77,000 travelers in the use of E-Gov Travel Services. However, with the in-house travel system migration to E-Gov Travel Services, eVoucher no longer had access to the travel data. This problem made SOA attractive, because SOA enabled eVoucher access to E-Gov Travel Services data without changing existing user interfaces.

Second, as part of its mission, the agency travels a large number of patients to conduct scientific and medical research. Patient travel is critical to the agency's mission. Patient traveler profile information, travel authorizations to be approved, and expenses to be dispersed must be concluded as quickly as possible. All interfaces should be real-time or near real-time. When the agency first selected E-Gov Travel Services, most of the E-Gov Travel Services Enterprise Application Integration (EAI) was in batch processes placing a unique challenge on the integration effort. The real world complicated services and systems and the information exchanged among those systems have been simplified in the diagram below.



Figure 1: Data Exchange between E-Gov Travel Services and an Agency Business System

In addition to the financial transactions in the diagram above, travel profiles, project accounting, and sponsors' information also must be exchanged. The Requirements Analysis is a separate step from the Service-Oriented Analysis to emphasize that the methodology must be business driven and independent of Web services. In the next step, I will explore when Web services should be used.


Service-Oriented Analysis

The Service-Oriented Analysis is performed by architects to identify services required to support the business requirement in a vendor independent and technology agnostic manner. The top down Web Service Definition Language (WSDL) first design enables the development teams from various companies to begin work independently without being concerned about the technical details and platform used. Since the analysis is technology independent, it is an invaluable asset that can survive the change of technology in the future. Lastly, but not least, this will minimize vendor influence on the architecture development.

The data exchange patterns and formats were analyzed; WSDL and XML schemas were produced as a part of the Integration Agreement. The agreement contains details of the concept of operations and data flow between the E-Gov Travel Services system and the agency's financial system. The following services were identified as common to the integration effort of many agencies:


Web Service (Operations) Direction Description
ProjectAccountLookup E-Gov Travel Services->AEBS Real-time read-only access to AEBS Projects
SponsorsLookup E-Gov Travel Services->AEBS Real-time read-only access to AEBS Customer
FundsCheck E-Gov Travel Services->AEBS Real-time read-only access to AEBS Funds Availability
CreatePurchaseOrders E-GOV TRAVEL SERVICES EAI->AEBS Transform a travel authorization to many financial obligations and submit to Oracle application workflow
AmendPurchaseOrders E-GOV TRAVEL SERVICES EAI->AEBS Transform a travel authorization to many financial obligations and submit the amendment to Oracle applications workflow
CancelPurchaseOrders E-GOV TRAVEL SERVICES EAI->AEBS Cancel a trip and all related Transform a travel authorization to many financial obligations and submit the amendment to Oracle applications workflow
CreateVoucher E-GOV TRAVEL SERVICES EAI->AEBS An expense voucher (regular, local, supplemental) is transformed to account payables, receivables and general ledger.
PositiveAcknowledgement/Reject AEBS->E-GOV TRAVEL SERVICES EAI Positive acknowledgement of financial transactions
AdviceOfPayment AEBS->E-GOV TRAVEL SERVICES EAI Payment notification
GetTravelDocumentList AEBS->E-Gov Travel Services Required for eVoucher service enable portlet
GetTravelDocumentDetails AEBS->E-Gov Travel Services Required for eVoucher service enable portlet
TravelerProfileNew AEBS->E-Gov Travel Services Traveler profile synchronization
TravelerProfileUpdate AEBS->E-Gov Travel Services Traveler profile synchronization
TravelerProfileTerminate AEBS->E-Gov Travel Services Traveler profile synchronization
TravelerProfileReactivate AEBS->E-Gov Travel Services Traveler profile synchronization
OrganizationChange AEBS->E-Gov Travel Services Traveler profile synchronization

Table 1

The primary purpose for AEBS Web services was to satisfy immediate business requirements. However, the data standard for all services was generic to future projects and could be reused. These services were first produced by architects and then fine tuned by Web service developers with deep domain knowledge. The diagram below is a domain model for the message content in the CreatePOs Web services to illustrate the data models in the XML schema and abstract WSDL. Because AEBS historically mapped a 1:1 relationship between purchase orders and lines, a customized approach to the integration was necessary.



Figure 2: Travel Authorization Class Diagram

Solution Architecture

In the last two sections, I have discussed business problem analysis and the design of an abstract solution. Now I will show how to select vendor technologies and develop an executable architecture with sufficient detail in order to begin our concrete design and construction. Like any technology solution, one must locate commercial tools to facilitate the development effort. The tools for service development, deployment, and operation and management are part of the infrastructure. First, the AEBS eTravel technical team researched Enterprise Architecture (EA) policies and standards. EA has policies with direct impact on:

  • Platform Selection
  • Certification and Accreditation
  • Data Standard
  • Federal Information Security Management Act (FISMA) Compliance

In 2004, AEBS conducted a three-month study on the agency's unique requirements and technical challenges necessary to migrate to E-Gov Travel Services. A prototype was also developed using Apache Axis open source software. In 2007, the agency developed and published integration architecture guidelines for all future integration projects, and decided to use TIBCO products for the enterprise-wide services to interface with the AEBS system, while utilizing Oracle products for the business processing logic. The following infrastructure components were available for our project application architecture.

  • F5 Load Balancer and SSL Accelerator
  • CA SiteMinder Federated Authentication for Web Applications and Services
  • Enterprise Directory
  • TIBCO BusinessWorks
  • TIBCO Enterprise Message Bus
  • Oracle 10G Database Real Application Clustering
  • Oracle 10G Application Server
  • Oracle Plumtree Portal
  • Oracle 10G Advanced Queuing (AQ)
  • Oracle Applications (Account Payable, Accounts Receivable, General Ledger, HR, Supplier)
  • Oracle JDeveloper
  • Oracle Enterprise Manager
  • Sun Solaris on Sun Fire E25K

After prototyping and many executive briefings and technical discussions, the following diagram illustrates how the above infrastructure components were connected in our high-level application architecture.



Figure 3: Integration Architecture

The architecture diagram above follows the standard n-tier application architecture for Web applications and services [REF-25-27]. In the process of analyzing business requirements and architectural constraints, the following major architectural considerations were relevant to our solution design.


Architecture Items Decision
Web Service Transport Layer Security All external services used FIPS compliant SSL connection.
Web Service Federated Authentication In addition to transport level security, all external services used Web Service Security (WSSE) message level authentication.
Loosely Coupled Systems Decoupled Web service end point from E-Gov Travel Services integration project so future projects could use the service directly without modification.
Decoupled TIBCO and Oracle so each product suite could be managed and upgraded independently.
Web Service Transaction All transactional services supported atomic transaction. A transaction had to be committed fully or completely rolled back.
Web Service Reliable Messaging Message from transactional service was submitted and processed exactly once. No redundant or missing obligations or invoices.
Event Driven SOA Patient traveler profile was placed on the service bus and transmitted to E-Gov Travel Services immediately after record was created in AEBS.
Service Enabled Portlet Existing eVoucher Web application user interface remained unchanged and was powered by Web services.
Three Way SSO The enterprise portal, E-Gov Travel Services and eVoucher all used federated authentication supported by SAML.
Message Driven Bean Durable Subscriber E-Gov Travel Services specific business logic was implemented in Oracle Message Driven Beans attached to the TIBCO EMS as durable subscribers.
Persistent Publisher All message publishers had to be persistent.
Component Based Architecture To facilitate reuse, all E-Gov Travel Services specific code used plug-n-play components.
Service Object Mapping XML messages were automatically mapped into Java objects by the architectural framework when business logic was performed.
Object Relational Mapping After business logic was performed, Java objects were automatically mapped into relational data as an atomic transaction by the architectural framework.
Real-time Request Response Services went directly to the database to perform read-only lookup access.
Real-time Message Requestor Most of pub/sub transactions were performed asynchronously. There were a few situations performed synchronously.
Generic Value Objects A generic value object was used for all business objects.
Content Based Message Routing All reusable services used content-based routing to implement project specific business logic.

Table 2

All relevant technical, security, application, and data architecture specifications were formally documented in the Interconnect Security Agreement (ISA), the Memorandum of Understanding (MOU), and the Data Integration Agreement. These documents governed the relationship between the agency, GSA, and the E-Gov Travel Services provider, including designated managerial and technical staff, in the absence of a common management authority.


From Prototype to Frameworks

Before the design effort, AEBS performed prototype work to help formulate and verify architectural decisions so that there would be no major surprises on the road to implementation. As always, after the prototype, some approaches were validated, and some design ideas were confirmed and fine-tuned. The relevant parts of the components were migrated into an architectural framework to be carried over into the design and construction phase.

Service-Oriented Design

This process, with all the required details in a specific enterprise technology environment, according to Service-Oriented Analysis results and architectural decisions and guidelines, produced a blueprint. It was platform independent and fully utilized vendor capabilities and value added-on features. Using CreatePOs Web service as an example of the scope of the design, a high-level workflow was developed. Major business activities within the workflow are described below.

  1. After a travel authorization was approved in E-Gov Travel Services, up to four purchase orders were created.
  2. E-Gov Travel Enterprise Application Integration (EAI) made a Web service call to the CreatePOs service for PO creation in AEBS.
  3. Upon TIBCO BusinessWorks (BW) receipt of the request, appropriate security measures were applied. When it was a valid user, a notification message was immediately sent to the E-Gov Travel EAI component.
  4. TIBCO BW then published the request to the Enterprise Message Service (EMS). The subscriber (Oracle AppServer) received and processed the message, which traveled through a set of validations for E-Gov Travel related PO edits.
  5. When a request passed all edits and validations, up to four purchase orders were inserted into the following Oracle Open Interface tables: PO_HEADERS_INTERFACE, PO_LINES_INTERFACE, and PO_DISTRIBUTIONS_INTERFACE. Each PO line may have had multiple distribution lines associated with it.
  6. When the PO creation was successfully completed in the Open Interface tables, the interface sent a notification message to TIBCO EMS.
  7. This triggered the PosAck (positive acknowledgement) Web service request from TIBCO BW to E-Gov Travel EAI, completing the CreatePOs Web service transaction.

After the business activities had been identified, AEBS added enterprise-wide components such as Web Service Security (WSSE),and logging, and exception handling. Starting from abstract WSDL, TIBCO BusinessWorks was used to generate concrete WSDLs for all inbound services. The combined diagram that includes the business activities, security, and infrastructure components is shown below.



Figure 4: CreatePOs Web Service Workflow Diagram

When the design progressed into component level, such as Message-Driven Beans for transactional business logic, I felt that Object-Oriented Design still added significant value to SOA. From this point of view, I agree with Thomas Erl's opinion and proved that Service-Oriented design and Object-Oriented design are not mutually exclusive.

At the end, coarse grained services must be implemented in executable software components. AEBS clearly separated the generic framework components such as service-to-object mapping, object-to-relational mapping, and persistent framework from business logic components; then further separated AEBS generic components from travel specific components so that the framework could be used for any future SOA project without much modification.


References

[REF-1] http://en.wikipedia.org/wiki/The_Art_of_War

[REF-2] http://en.wikipedia.org/wiki/Sun_Tzu

[REF-3] http://en.wikipedia.org/wiki/Thirty-Six_Strategies

[REF-4] http://www.soamethodology.com/

[REF-5] An SOA Vendor Evaluation Methodology; http://www.soamag.com/I20/0708-3.php

[REF-6] SOA and EDA: Using Events to Bridge Decoupled Service Boundaries; http://www.soamag.com/I4/0207-2.php

[REF-7] Service-Oriented architecture (SOA) was first described by Gartner in 1996; http://www.gartner.com/DisplayDocument?doc_cd=114358

[REF-8] http://www.opengroup.org/projects/soa/

[REF-9] http://soa.omg.org/

[REF-10] http://www.oasis-open.org/committees/tc_cat.php?cat=soa

[REF-11] Navigating the SOA Open Standards Landscape Around Architecture; A paper edited by: Heather Kreger, IBM Jeff Estefan, NASA/Jet Propulsion Laboratory July 2009; https://www.opengroup.org/projects/soa/uploads/40/20044/W096.pdf

[REF-12] http://www-01.ibm.com/software/solutions/soa/

[REF-13] http://www.ibm.com/developerworks/webservices/library/ws-soaintro2/

[REF-14] http://www.microsoft.com/soa/

[REF-15] http://www.oracle.com/technologies/soa/index.html

[REF-16] TIBCO

[REF-17] http://www.softwareag.com/corporate/products/wm/default.asp

[REF-18] http://www.jboss.com/resources/soa/

[REF-19] http://www.sap.com/platform/soa/index.epx

[REF-20] http://www.linkedin.com/pub/anne-thomas-manes/0/1/71

[REF-21] http://blogs.oracle.com/davidchappell/

[REF-22] http://booksbysandy.com/

[REF-23] http://www.thomaserl.com/

[REF-24] From Stove-piped Projects to Unified Enterprise Architecture: Strategic considerations for E-Authentication service development; http://www.javaworld.com/javaworld/jw-03-2003/jw-0321-sso.html

[REF-25] U.S. Department of Energy Signs on to J2EE: Create a secure single sign-on Web service for multiple n-tier Web applications. JavaWorld May 2002; http://www.javaworld.com/javaworld/jw-05-2002/jw-0524-signon.html

[REF-26] A Java Case Study: The power of J2EE How the Energy Information Administration secured a complete J2EE solution in less than five months. JavaWorld January 2002; http://www.javaworld.com/javaworld/jw-01-2002/jw-0118-j2ee.html

[REF-27] Step into the J2EE Architecture and Process: Develop complete J2EE solutions with an eight-step cycle. JavaWorld September 2001; http://www.javaworld.com/javaworld/jw-09-2001/jw-0928-rup.html

[REF-28] Working with SOA and RUP by Solmaz Boroumand; http://www.soamag.com/I16/0308-1.php

[REF-29] Event-Driven SOA>

[REF-30] http://www.gsa.gov/etravel

[REF-31] GSA Solicitation FBGT-CD-030001-N available at https://www.fbo.gov/notices/993abc7c2c6f22ef8bfeb6f94c8861de

[REF-32] http://www.nih.gov/

[REF-33] http://enterprisearchitecture.nih.gov/ArchLib/AT/IA/Integration/IntegrationPrinciples.htm

[REF-34] http://www.gsa.gov/ftr

[REF-35] http://acquisition.gov/far/

[REF-36] http://csrc.nist.gov/publications/fips/fips140-2/fips1402.pdf

[REF-37] http://enterprisearchitecture.nih.gov

[REF-38] http://java.sun.com/javaee/

[REF-39] http://www.ibm.com/developerworks/library/specification/ws-rm/

[REF-40] https://www.ibm.com/developerworks/webservices/library/ws-wsilover/

[REF-41] http://www.ibm.com/developerworks/library/specification/ws-rm/

[REF-42] http://www.ibm.com/developerworks/library/specification/ws-tx/

[REF-43] http://www.ibm.com/developerworks/library/ar-archtemp/

[REF-44] http://download.oracle.com/docs/cd/B34956_01/current/html/docset.html

[REF-45] http://download.oracle.com/technology/tech/soa/soa_best_practices_1013x_drop3.pdf

[REF-46] http://convergence.satpathy.org/

[REF-47] http://www.oracle.com/technology/products/applications/events/oow-2008/S298465_VeshaalSingh-SOA-EBS.pdf

[REF-48] http://www.idmanagement.gov/


Disclaimer

The SOA methodology discussed in this article is based on my knowledge or direct work experience on E-Gov Travel Service integration projects from federal agencies. The views presented in this article are all personal; the author does not represent any of the government agencies. Some technical details were simplified, generalized, omitted, or sanitized.