ServiceTechMag.com > Issue LII: July 2011 > Empowering the Discipline of Cloud Integration - Part III
Pethuru Cheliah

Pethuru Cheliah

Biography

Dr. Pethuru Raj has been working as a TOGAF-certified enterprise architecture (EA) consultant in Wipro Technologies, Bangalore. On the educational front, armed with the competitive UGC research fellowship, he could proceed with his research activities and was awarded the prestigious PhD degree by Anna University, Chennai, India. He then could acquire the meritorious CSIR fellowship to work as a postdoctoral researcher in the Department of Computer Science and Automation (CSA), Indian Institute of Science (IISc), Bangalore. Thereafter, he was granted a couple of international research fellowships (JSPS and JST) to work as a research scientist for 3 years in two leading Japanese universities. Dr. Raj also had a fruitful stint as a lead architect in the corporate research (CR) division of Robert Bosch, India, for 1.5 years.

Dr. Raj has more than 12 years of IT industry experience. Primarily, he has been a technical architect and currently he is providing technology advisory services for worldwide business behemoths on the transformation capabilities of enterprise architecture (EA) in synchronization with some of the emerging technologies such as the Internet of Things (IoT) / Cyber Physical Systems (CPS) / Machine-to-Machine (M2M) Integration, Big Data, Cloud and Service Computing paradigms, Real-time Analytics of Big data using Cloud-based NoSQL databases, Hadoop framework, etc. and Mobility. He has made use of the opportunities that came on his way to focus on a few business domains, including telecommunication, retail, government, energy, and health care.

Dr. Raj has contributed book chapters for a number of technology books that were edited by internationally acclaimed professors and published by leading publishing houses. Currently he is writing a comprehensive book with the title "The Internet of Things (IoT) Technologies for the Envisioned Smarter Planet" for a world-leading book house. The CRC Press, USA has just released his book on "Cloud Enterprise Architecture" and you can find the book details in the page http://www.peterindia.net/peterbook.html

Contributions

rss  subscribe to this author

Bookmarks



Empowering the Discipline of Cloud Integration – Part III

Published: July 17, 2011 • Service Technology Magazine Issue LII PDF
 

This is the third in a multi-part article series. The first part is published at Empowering the Discipline of Cloud Integration – Part I and the second part is published at Empowering the Discipline of Cloud Integration – Part II


Explaining Integration as a Service (IaaS)

More companies are adopting SaaS applications, fuelled by a competitive business environment. When potential technologies emerge, companies need to adapt them so that their IT departments can deliver innovative technology solutions rapidly at a lower cost. Frequently these days, we come across SaaS applications and providers. Core business applications are hence gradually finding their new homes in clouds. In a recent article, Accenture has indicated that it has saved a lot of money for one of its valuable customers by putting Oracle enterprise solutions in the cloud. SaaS solutions are also termed as on-demand applications that are attractive to IT because of the lower number of IT resources needed for deployment and effective management. Business users can start using the new systems as SaaS solutions are intuitive and easier to tinker with. Also, everybody likes subscription pricing because it means fewer budgetary issues get in the way of procuring and operating new applications. Hence, there is no surprise that SaaS model is inspiring many to jump into this cost-effective delivery and usage model. SaaS is ok but enabling a SaaS application to interact with other SaaS applications or with the indispensable and onsite enterprise packages is a large task.

SaaS providers now offer integration solution suites supporting complete end-to-end processes, such as source-to-pay, order-to-cash and even complete back-office operations, such as financials and human resources. Leading SaaS providers have also developed EAI approaches that marry proven business-to-business integration (B2Bi) methods with the delivery and cost benefits of SaaS to permit a flexible array of data integration options (batch to real-time messaging) between SaaS solutions and installed enterprise applications.


The Key Drivers for SaaS-based Integration

Developing, deploying and using SaaS applications comes with a challenge. Many IT departments become frustrated soon after they start using SaaS applications because the critical business information they need are locked in onsite back-end systems, and getting SaaS applications to access and update these information systems becomes a larger problem than anybody anticipated in the beginning. Integration has been as always an after-thought activity. Integration has to be the core portion at the strategizing and planning stage itself.

Despite the attractive benefits identified, SaaS applications will deliver limited value if the associated integration issues are not considered and resolved upfront. Getting the new SaaS application running is easy, but users expect instant access to all of their business information about customers, products, orders, and more. Although SaaS applications are simple, their integration with other systems in a company are critical to realizing the value.

Since SaaS applications offer the same benefits to all customers, all advantages gained by one company are equally available to its competitors. In order for SaaS vendors to keep their flocks intact have to look out for supplying value-added services in a consistent mode. One momentous addition is to provide a seamless connectivity with customized back-end applications that are core for a company’s successful running. Retaining this differentiation and making the enormous quantity of corporate data available to online SaaS systems brings the issue of application integration to the forefront.

Also, instead of putting the appropriate integration infrastructures for both types (SaaS-to-SaaS and SaaS-to-Onsite integrations) at the customer side, they are being hosted in the clouds that are a kind of fabric, grid, cluster or peer-to-peer networking of thousands of processing and data nodes. This new IaaS model treats integration as a fully hosted service, eliminating many of the burdens and costs of implementing and maintaining enterprise integration using traditional software and methods. IaaS delivers a multitude of pre-defined adapters between SaaS module and leading enterprise applications. IaaS ascribes to a flexible and subscription-based pricing model that scales based on the level of integration required.


SaaS Integration Challenges

There are important differences that make SaaS integration more challenging than the conventional integration.

  • Since SaaS applications exist outside the corporate firewall, a new set of protocols and interfaces are required to handle data integration
  • Ease of SaaS application customization by business users makes integration change management difficult for IT to deal with.
  • Cloud-enabling traditional IT integration tools might be too much, and cost prohibitive for small to mid-size companies or departmental deployments, which make up the majority of SaaS projects.
  • SaaS solutions are naturally multi-tenant, therefore conventional integration schemes may fail here.

IaaS Realization Approaches

Companies are trying different ways for SaaS integration and for providing integration activity as a network service. There are mainly technology, tool, process and artifact-based solutions. Fortunately, there is a growing army of innovators in this space and several niche vendors have come onto the SaaS integration stage with their offerings based on evolving techniques. The prominent IaaS approaches include the cloud-compliant middleware and the portfolio method.


Middleware in the Cloud Approach

We have an array of standards-compliant and proprietary integration products. That is, the traditional middleware are remedied to fit in the clouds, therefore the terminology of cloud middleware is becoming clearer. Different vendors have come out with a wide range of patentable cloud middleware in order to solve generic as well as specific integration requirements. We have given details about most of the cloud middleware in this chapter. The reference section contains the links for the companies’ URL and from there users can extract valuable information about the products and their features.


Prominent Steps towards IaaS

The common tasks for establishing connectivity between SaaS and customers’ business applications include the following:

  1. Getting Information into a SaaS System – Information is a strategic asset for any growing concern. As business applications are made SaaS-aware, corporate data have to be made available to SaaS systems to be productive and valuable. The goals of data accessibility and consumability are always important. This involves data extraction from the company-owned data sources, cleansing, transformation and loading into the requesting SaaS application.
  2. Synchronizing Information between SaaS and Back-end Systems – Back-end systems host all the corporate information about customers, products, transactions, and more. Therefore it is important to establish perfect data synchronization by decimating all kinds of data conflicts among SaaS and corporate solutions in order to provide a single, accurate and real-time view of customers and products to SaaS users. This synchronization process happens on the SaaS side.
  3. Extracting Information from a SaaS System – Once SaaS possesses the synchronized information, then the final act is to query and retrieve the relevant information from SaaS systems and to deliver them to users or user agents.

Portfolio Approach

Informatica has taken the proven portfolio route for solving the SaaS integration problem. First, it has introduced a best-in-class set of SaaS connectors to support IT users with powerful and scalable capabilities for enterprise SaaS implementations. More importantly, the company has developed a set of on-demand integration services, specifically designed for SaaS administrators that empower them to manage their own data integration needs while minimizing their reliance on IT. These services are web-based and easy to use, can be deployed in minutes, and are licensed at a compelling price point.

For example, the Informatica on-demand data synchronization service provides robust bi-directional synchronization between salesforce.com, or the Force.com platform and back office systems and SQL databases through a simple, yet powerful, “set and forget,” web-based environment. This service integrates on-premise systems, salesforce.com, and other SaaS applications, enabling account synchronization, automated order, accounting and shipping processes between SaaS, CRM and ERP systems. Powerful data mapping and data transformations using an Excel-like function palette enable administrators to manipulate and manage data without requiring multiple or manual steps. Automated task scheduling and task sequencing assure timely execution and manipulation of complex multi-step jobs.


Key Functionalities of IaaS Solutions

IaaS product shares the layered characteristics of the SaaS model.

  • Transport and Messaging – Reliance on sophisticated protocols for ensuring synchronous/asynchronous communication with guaranteed delivery and advanced security features.
  • Transformation – Due to a number of data formats, data mappers are forced to perform the bulk of integration, for example, EDI to XML, CSV to XML.
  • Consistency & Compliance – The validity of data is becoming more and more crucial given the growth of integration transactions. That is, data consistency in the midst of multiple applications interaction together besides stricter compliance to national rules is very vital
  • BPM, BAM & Workflows – In order for vendors to show to show their superiority, they need to and to preempt their market competitors, provide advanced features embedded. Some vendors enable the orchestration and management of multiple integration projects. Some attach more sophisticated monitoring capabilities.
  • Value-added Services – Marketplaces are proposing functions for specific industries or processes like Supply Chain, Invoicing, etc.

IaaS’ Exemplary Benefits

For the growing number of businesses that are moving towards SaaS solutions for their IT needs, this IaaS model provides benefits ranging from the information visibility achieved through the application integration to rapid deployment, minimal impact on IT resources and low total cost of ownership (TCO).

  • The ability to hand integration responsibilities and most of the integration infrastructures over to the SaaS provider relieves IT departments of costly in-house integration projects, software upgrades and the need for on-board integration specialists. It also reduces the need of installing and managing integration-related servers, databases and storage area networks (SANs) in the user's data center, keeping equipment and overhead to a minimum.
  • The use of pre-built connectors accelerates the integration process and lowers implementation risks. The security provided by the communication backbones and connectors aids compliance with regulatory requirements such as Sarbanes-Oxley. Once the integration infrastructure is set up for one SaaS solution, it can be extended to connect to other on-demand systems or to external trading partners.
  • Integration platforms are prohibitively expensive and by bundling integration expenses into the SaaS fee is a huge saving for IaaS consumers. Thus IaaS has become the solid, stimulating and selling point for SaaS. IaaS single-handedly has tilted the balance towards the SaaS model for not only small and medium companies but also for large organizations.

These work well for both SaaS and IaaS. Whether we need to connect our hosted software to any enterprise applications, packages and data sources, IaaS works quickly and cost-effectively.


Detailing IaaS Infrastructures and Services

IaaS is a promising paradigm but there is still a need for high-quality infrastructures and platforms. This is the key driver for the popularity for cloud computing. In this section, we are to discuss the distinct characteristics of cloud computing.

Computing is consistently on the progressive path. There have been requests with a number of generic and specific computing technologies to tackle different kinds of business problems. For instance, to realize proactive systems, we have proactive computing initiated by Intel. Similarly there are trustworthy and secure computing models by Microsoft, liquid computing by BEA system (Oracle), autonomic, deep, edge, pervasive, and on-demand models by IBM, affordable computing by Sun (Oracle), and adaptive computing by HP. For realizing high-performing, highly available and scalable applications, we have paradigms such as distributed, multicore, fabric, cluster and grid computing. There are unconventional computing models such as molecular or DNA, optical, and quantum computing. Other prominent computing concepts include model-driven, service-oriented, nature-inspired, activity and agent-based computing paradigms. With the adoption of the Internet, there arise computing styles in order to beneficially leverage the cheaper channel and platform offered for a variety of online business applications. In this evolution cycle, we now come across cloud computing, which is the multi-tenant, elastic, cheaper, on-demand (utility) and enterprise computing model.

Cloud Computing is the new-generation computing paradigm comprising of a large number of data and compute servers interconnected to work together. This provisioning infrastructure hosts application data and services to be delivered to a large group of users with heterogeneous I/O devices over the Internet. Cloud computing is aptly termed as the web-scale and elastic computing and is being viewed by experts as the much-expected paradigm-shift in IT. The next-generation IT is remarked as Cloud IT and computer science 2.0. The extraordinary distinction is the extreme elasticity of clouds through its features of expansion and contraction as demands fluctuate. Clouds will improve the utilization rate of expensive and underutilized IT systems to maximize returns and enlarge the space and scope of innovation significantly. Clouds are capable of hosting all kinds of IT resources and software engineering activities. This transition from the traditional on-premise to cloud hosting gives enterprises and individuals the benefit. Users could connect with clouds in order to avail any kind of personal and professional services. With the maturity of composition methods on the rise, smart and business-aligned applications, services, processes, and data will be created and hosted in clouds. Less upfront and management costs, quick deployment, high availability, performance and assurance etc., are some of the critical differentiators with the emergence of cloud concepts, processes, products, and services.

This kind of web-based storage, processing and delivery was initiated years ago, that initiative did not move forward due to lack of business cases and robust technologies. Now with the realization of several cutting-edge technologies ranging from virtualization, dynamic provisioning, multi-tenancy, SOA, security, to green, the applications hosted in remote server farms and delivered as services over the Internet have succeeded.

Clouds lead to a series of path-breaking innovations and today many venture capitalists, investors and entrepreneurs are confidently working with the into this expansive cloud space. Professionals are working on cloud infrastructures, patterns, utilities, methodologies, frameworks. Researchers are trying to identify the possible limitations to come out with competitive solutions. Consolidation gets an altogether new meaning and there is a convergence of personal and mainframe computing paradigms. In short, centralization, componentization, the global-scale connectivity and collaboration technologies are collectively contributing for the rapid realization of the cloud era. Analysts are understanding and highlighting what impacts cloud computing will make on IT.


Motivations for Cloud IT

From taxonomy-enabled information repository such as Google to social networking sites such as Youtube, SlideShare, Facebook and MySpace, online businesses are seeing growth due to the global reach of the Internet and the maturity of content-making, personalizing, and delivery technologies. In an effort to meet the soaring demands of their users, these firms have spent billions of dollars on storage appliances, servers and switches that, taken together, form a complex infrastructure. But such a big growth is putting the infrastructure under stress. Low utilization levels of investments on IT products due to structure, rising administration and governance costs, shortage of skilled, well-qualified, and experienced data center and server farm professionals, etc. are mandating for trend-setting technologies to bring a sense of moderation and meaning into this computing ecosystem.

Enterprises are putting their offices and teams in different places; this enforces the need to establish one or more data centers in each location. This scenario is not supporting the cost-cutting measures of business managers in this period. Executives are searching for virtualized, clustered, centralized, highly available, synchronized, all-encompassing, and real-time infrastructure to avoid extra cost. In order to mitigate the rising complexity of this structure, there came a number of efforts and results in the form of novel computing models, virtualization and dynamic provisioning technologies, web 2.0, and its allied disciplines, service composition methods, etc. Each one delivers something however; there is no coherent integration among them obstructing the liquid-like flow of information, and there is no rationing, adaptivity, federation and sharing.


Characteristics of Cloud Computing

Supercomputers today are used mainly by the military and government intelligence agencies, universities and research labs, and large companies for complex calculations for such tasks as simulating nuclear explosions, predicting climate changes, designing airplanes, and analyzing which proteins in the body are likely to bind with potentially new drugs. Cloud computing aims to apply that kind of power (measured in tens of trillions of computations per second) to problems like analyzing risks in financial portfolios, delivering personalized medical information, even powering immersive computer games, in a way that users can tap through the web. It does that by networking large groups of higher-end servers that are most often underutilized and even sometimes unutilized. This enhanced physical as well as virtual consolidation and subsequent utilization through workload sharing helps to keep the management and maintenance costs down. Further on, capital investment on IT products will come down to a major extent. Interestingly, there are some unique properties for this style of computing as enumerated below.

  • Virtual – Physical location and underlying infrastructure details are totally transparent to users. That is, users do not know which cloud from which part of the world fulfils the request.
  • Scalable – Able to break complex workloads into pieces to be served across an incrementally expandable infrastructure
  • Efficient – Capable of leveraging service orientation (SO) concepts for dynamic provisioning of shared compute resources
  • Flexible – Can serve a variety of workload types – both consumer and commercial

Cloud Building Blocks and Enabling Technologies

Clouds are the large, cost-effective, and open computing infrastructure for hosting a variety of web and service-enabled software applications to be delivered over the Internet. This flexible infrastructure can host applications and data. Further on, a number of integrated development environments (IDEs), specialized engines, and adapters, messaging middleware and service containers are also being hosted on this resizable and adaptable cloud infrastructure. Web 2.0 represents a collection of user interface technologies and every cloud application has to have a Web 2.0-compatible interface for interaction with people. There are a few big cloud providers (Google, Amazon, IBM etc.) today. IBM has leveraged some open source and its own platforms for establishing clouds successfully.

  • Open source Linux platform
  • Xen, the fastest and most secure virtualization software
  • Automated provisioning of computing resources by Tivoli Provisioning Manager
  • Systems management and monitoring by IBM Tivoli Monitoring
  • Parallel computing clusters using Apache Hadoop
  • Open source Eclipse-based development tools for parallel applications

Google's search engine and productivity (office suite) applications are among the early products out of efforts to locate fluctuating processing power of compute servers, rather than on desktop PCs. Microsoft has released an online software called Windows Live for photo-sharing, file storage, and other applications served from new data centers. Yahoo has taken similar steps. IBM has devoted 200 researchers to its cloud computing project. And Amazon.com recently broadened access for software developers to its "Elastic Compute Cloud (EC2)" service, which lets small software companies pay for processing power streamed from Amazon's data centers.


Cloud Benefits

Businesses are cost-cutting and they try to run their operations on a lower budget. In this scenario, cloud ideas have come as an advantage for them. Furthermore, cloud is a disruptive and transformative technology for both beginners as well as established enterprises. Clouds are able to:

  • enable new business, pricing, and delivery model
  • bring in a bucket of buzzwords such as Information, integration and infrastructure as a service, platform, process, policy, and portlet as a service, software, security and storage as a service, etc. with the overwhelming acceptance and adoption of clouds, composites, mashups, Web 2.0, virtualization, SaaS etc.
  • mitigate growing and widening IT complexity in a single stroke
  • increase business visibility, agility, recovery, resilience, and productivity
  • guarantee real-time operations when cloud middleware is empowered to receive, analyze, store, and mine business events.
  • host multiple applications from multiple organizations and respond to multiple users over the Internet in real-time
  • support easy customization and personalization of hosted applications
  • provision and de-provision servers according to the prevailing situation, cluster participating systems to scale up as per the need, orchestrate services to generate business-aligned and smart services.
  • assure fault and stress-tolerance
  • ensure better accessibility and resource management
  • maintain reduced cost for infrastructures and improved performance

In addition, clouds can host development platforms and toolkits in order to empower developers to develop, test, troubleshoot, and tweak software on the clouds. Further on, developed and verified components may be deployed on the cloud-compatible containers in order to serve their subscribers guaranteeing operation as well as service level agreements. Clouds promise a supercomputing facility at a very reasonable rate. Cloud computing is mass-based serving to a large base of web users. Thus, the user base and market opportunity are large. Clouds are for not only common people but also for entrepreneurs, executives, idea creators, service developers, knowledge workers, middleware vendors, infrastructure providers, etc.


Constraints and Challenges of Cloud Computing

Critics argue that the benefits of cloud computing (greater computing efficiencies and the accessibility of data and applications from anywhere) are offset by the privacy implication of declining control over our personal data.  Moreover, a growing number of environmental groups note with alarm that the enormous energy requirements to power (and keep cool) hundreds of thousands of compute servers are huge. Issues including offline access to files, the need for ubiquitous broadband, etc. abound. Like all technological advances, cloud computing is not without risks. For instance, there are security risks related to commingling our data with that of other companies. And reliability concerns arise whenever we depend on a third parties’ systems to be up and running 24/7.

Furthermore, there have been a lot of deliberations and discourses on cloud computing and its unique ability to replace expensive licensed software and to reduce hardware costs with low-power and cheap Internet access devices. Cloud computing continues to mature by assimilating an array of enterprise technologies. What this means is that we are at the beginning of a transition from the model of the past where applications and all the content created for them were stored locally to the hosted model. This steady shift has the potential to increase the types of Internet-connected devices we use to consume and create content. As businesses face pressure to be as adept as possible, they will be forced to adapt to evolving computing models to remain competitive.


Conclusion

The next article will conclude this series.