Abstract: Business Process Management (BPM) is complex, expensive, and often fails! If you agree (in the year of 2012+), then you should read the following rules to do BPM correctly in your next project.
This article does not give an introduction to BPM. It starts with a use case immediately to show best practices and common miscues regarding BPM. If you are not familiar with BPM, BPMN, WS-BPEL or related topics, then you should begin with a BPM introduction [REF-1].
Use Case: Loan Request
Figure 1 shows a stateful, long-running business process, which is applied in each paragraph to explain best practices and possible miscues in BPM projects. Here are the steps in this process: First, the business process covers a loan request. After the loan request is received, some automatic scripts and service tasks are executed. Following that, user interaction is required to approve or reject the loan request. Depending on the evaluation, the request is processed or denied. Stakeholders here have either the job role of a business expert or a developer.
Figure 1 – Business Process for a Loan Request
Looking at this use case, let's discuss several different aspects of BPM to learn important best practices and miscues in real world projects.
Goals of BPM
BPM attempts to improve processes continuously. It can therefore be described as a "process optimization process". Business-IT-Alignment is the key to improve processes continuously. The important word here is "alignment". It is not just about IT! It is not just about business! It is about working together - one of the major challenges of many BPM project. Besides this, BPM has further goals (which can be reached as consequence of good Business-IT-Alignment):
- Increased efficiency
- Higher transparency
- Better quality
- Reduced costs
- Enabling new business models
Use Case: The primary goal is to reach higher transparency and improve communication between business and IT. When introducing BPM, the focus should be on this goal right from the very beginning. The goal has to be supported and made transparent by decision makers. It would be wrong to introduce BPM without any key goals. In addition to this, key performance indicators should also be defined to measure the success of BPM.
Rule 1: If you want to do BPM correctly, then be aware that your primary goal is to improve Business-IT-Alignment!
BPM and SOA
BPM is an offspring of Service-oriented Architecture (SOA) as Anne Thomas Manes explained in her well-known article "SOA is Dead" in 2009 [REF-2]. SOA is the foundation for modern approaches such as BPM, cloud computing or mashups. So, "BPM based on SOA is technology's response to the growing demand for a flexible business environment that is not hindered by application silos. The integration technology must loosely couple the applications and resources that make up the process, otherwise the logic of a process will get hard coded into a particular technology platform, which may be expensive to change and therefore defeat the entire purpose of BPM." [REF-3]
Many risks have to be considered when pursuing BPM without SOA because advantages of BPM cannot be gained without services [REF-4].
Use Case: The loan request is already split up into several loosely coupled services (script and Java service tasks, human interactions). Therefore, it is ready to be realized with BPM. If everything is regarded as one silo, BPM would not be possible, and higher flexibility and increased efficiency could not be reached.
Rule 2: If you want to do BPM correctly, then do not try to realize it without a service-oriented Architecture!
Let's first tell you what BPM is not: BPM is not just tooling. Instead, BPM is an engineering process with several different stakeholders, technologies and tools. The lifecycle of BPM is iterative and has the following phases:
- Start again: Design
Of course, tooling should support the whole lifecycle. Therefore, BPM tools should be evaluated after some processes are modeled and designed, and when it is clear how business processes shall be executed and monitored.
It is important to introduce BPM iteratively. Do not try to "boil the ocean by implementing a BPM / SOA solution for an entire business group or division as one massive upgrade or roll out. BPM and SOA efforts are both inherently evolutionary in that they are best implemented in small, constrained, and frequent capability releases that grow upon each other." [REF-5]
Use Case: First choosing a tool and then starting realization is the wrong approach, and can lead quickly to failure. Several requirements may be unknown at the beginning of the project. For instance, no complex business rules have to be defined. So therefore, a BPM tool that integrates a complex (and expensive) rules engine would be a very bad decision. Following the BPM lifecycle iteratively reduces efforts, helps analyzing existing services and roles, and enables reusing available knowledge and existing source code.
Rule 3: If you want to do BPM correctly, then be aware that BPM is not just tooling, but an engineering process with a huge, iterative lifecycle!
Benefits of BPM
BPM is no Swiss army knife, which can solve every problem. Do not use BPM in wrong situations. I don't like seeing people using BPM tools just because there is a workflow. You don't need a BPM tool to realize a workflow. In this case, you can use (for instance) a Java method that does some business logic, and then calls another Java method that does some business logic, and then calls another Java method, and so on...
BPM brings benefits especially if you need:
- Long-running stateful workflows
- Frequently changing processes
- Human interaction
Use Case: BPM is an excellent choice due to long-running stateful workflows and human interaction. Several benefits are gained, e.g. better transparency, and higher flexibility. Introducing BPM pays off quickly. Implementing such process logic by oneself is a lot of effort, error-prone, and just unnecessary.
Rule 4: If you want to do BPM correctly, then use it only when you really gain benefits!
Myths of BPM
John Cingari has defined four myths of BPM projects, which are true in most organizations [REF-6]:
- Business analysts WILL create executable process models
- Business analysts CAN create executable process models
- Business analysts WANT to create executable process models
- IT WANTS business analysts to create executable process models
There will always be business guys and IT guys, and they have a different understanding of things. Business guys will never be able to model complex business processes. Therefore, Business-IT-Alignment is very important, and tooling cannot solve this problem. Business and IT have to work together to do BPM successfully.
Use Case: Business analysts have a lot of knowledge about loan request procedure. If possible, let them model the processes. Developers just add technical details. However, many business people cannot think this way, or simply do not want to. On the other hand, developers learned this way of thinking and modeling from the beginning of their carrier. It is fine if developers model all business processes. It is much more important to pick everybody up from business. Hold meetings to gather information, then model processes, and then do review meetings. Iterate this process.
Rule 5: If you want to do BPM correctly, then do NOT believe in BPM myths!
Standards are important for BPM to be future-proof and vendor-independent. This way, different tools can be used for different phases / by different people. A lot of standards exist for BPM, such as BPMN, XPDL, BPEL, jPDL, WF-XML, ARIS EPC, and others.
The most important standard for BPM is BPMN [REF-7] (Business Process Model and Notation) - at least since version 2.0 (released in March 2011). BPMN 2.0 added a standardized XML description format. Now, BPMN is not just flow charts: Sufficient restrictions and constraints specify the execution of business processes non-ambiguously. Another important new addition to BPMN 2.0 is its standardized extension points (because every vendor always adds his own specific features to a product). For instance, the open source BPM framework Activiti offers an extension for a Java service task.
WSBPEL [REF-8] (Web Services Business Process Execution Language, short: BPEL) is the second most important BPM standard. However, BPEL can only be used for SOAP Web Service execution. Due to the fact that BPMN 2.0 also includes execution of business processes, new products do not need to use BPEL for execution. Nevertheless, a BPMN 2.0 implementation must align to BPEL to be 100% standard conform. If you ask yourself about is the relationship between BPMN 2.0 and BPEL 2.0 and if BPMN 2.0 makes BPEL 2.0 redundant, then you should read the interesting interview at [REF-9].
Use Case: BPMN is the defacto standard for BPM in 2012+. It is suitable for business and IT people, and it can be used for modeling and execution. Many other standards are outdated or way too complex. Choosing WS-BPEL instead of BPM would add several restrictions and increase complexity. If the available tool in your organization just supports WS-BPEL, you can still use BPMN for modeling and WS-BPEL for execution.
Rule 6: If you want to do BPM correctly, then use BPMN 2.0!
Countless BPM products are available on the market. How do you decide which one to choose? You can do an evaluation comparing several criteria, e.g.:
- Open Source vs. Proprietary
- Supported Technologies
- Supported Standards
- GUI Designer
- Community, Documentation, Commercial Support
- ... and so on
Wrong! This procedure might work for most product or framework comparisons. I often created such criteria lists before by myself. However, for BPM tools, there is one important criterion, which must be discussed before you look at all the other ones: Do you want to use a developer-focused tool or a designer-focused tool?
Why are these so different?
A developer-focused tool such as Activiti or JBoss jBPM is embedded directly into your container or standalone application. You use your common development tooling and environment. You write source code including corresponding unit tests. Usually, developer-focused tools are very lightweight, easy-to use, open source and offer an API for your preferred language (e.g. Activiti is Java-based). You can start using it after some minutes. However, developer-focused open source tools are not perfect. For instance, they often provide bad documentation and a very rudimentary web frontend for modeling and monitoring.
On the other hand, there are designer-focused tools, which promise zero-coding. These tools are real products, not just embeddable frameworks. You do not write source code. You create business processes in a graphical designer via drag and drop. No coding is required. Decide for yourself if this is a pro or con. Sometimes, zero-coding might be a con (remember the four myths about BPM - business analyst cannot model business processes). If you cannot code anything or if you have to use ugly workarounds, then this can be a show-stopper for real world projects, where the tool does not offer the feature, which you require.
Proprietary, designer-focused tools are very expensive and heavyweight solutions (e.g. you do not need to buy and install just Oracle BPEL Suite, but also Oracle WebLogic Application Server, and several other products from the Oracle stack). If you start installing such a suite on day 1, you probably will not be able to use it until at least day 2 (because installation is so complex and time-consuming). Contrary to developer-focused open source tools, proprietary products offer high stability, powerful web frontends and good integration into other parts of your development environment (if you use the whole stack from one vendor).
Besides heavyweight, proprietary BPM tools from large vendors such as Oracle or IBM, there is also some designer-focused open source tools available, e.g. ProcessMaker or Bonita Open Solution. They are easy to setup and learn. Usually, designer-focused open source tools are not as powerful as proprietary products. However, you can add required functionality by yourself because they are open source.
So, the first question to answer for BPM tool choice is: Do I need / want a developer-focused tool or a designer-focused tool? After you have made this decision, you can evaluate BPM tools by other criteria: Which features are supported (e.g. a complex rules engine)? Is the tool extendable, can other software be integrated easily? Which technologies are supported? How shall business processes be monitored? And so on...
Use Case: Process logic of the loan request contains only business information. No complex logics or flows are needed. Technical information is not important on this level. Therefore, business analysts are able to model this process. Afterwards, developers just add technical interfaces for execution. This use case is a perfect match for a lightweight, designer-focused product.
Rule 7: If you want to do BPM correctly, then choose the right BPM tool for the job!
BPM and Systems Integration
But is this a good solution? It may be fine, if you just have to call one or two services in your process. In all other situations, the magic word(s) is "separation of concerns" Otherwise you will end up in complex spaghetti solutions. This is why a SOA is so important for introducing BPM.
A BPM tool is not an integration tool, but a process engine (plus some addons, of course). Use BPM for its benefits, e.g. realizing long running stateful processes or human interactions. For systems integration, use a tool that is built for this job, i.e. a lightweight integration framework such as Apache Camel, or an Enterprise Service Bus (ESB) such as IBM WebSphere Message Broker. Wolfgang Pleus wrote a great (short) blog post about combining a BPM engine and an integration framework [REF-10].
Use Case: Some technical services have to be called from the loan request business process. Deep technical knowledge for routing, transformation, and connectivity to different systems is required. Therefore, this logic should be separated from the process logic. The business process just contains the interfaces. All other integration logic is realized with an integration solution. Effort for separation of concerns is low due to an flexible, loosely coupled SOA fundament.
Rule 8: If you want to do BPM correctly, then do NOT use it for systems integration!
[REF-1] Business Process Management (BPM) http://en.wikipedia.org/wiki/Business_process_management
[REF-2] SOA is dead http://apsblog.burtongroup.com/2009/01/soa-is-dead-long-live-services.html
[REF-3] SOA and BPM http://www.information-management.com/issues/20070501/1082553-1.html
[REF-4] Risks of BPM without SOA (http://artofsoftwarereuse.com/2009/09/03/risks-with-pursuing-bpm-without-soa/)
[REF-5] BPM Lifecycle http://www.ebizq.net/topics/bpm/features/11569.html
[REF-6] BPM Myths http://www.activevos.com/blog/soa/the-four-myths-of-bpm-projects-what-it-project-teams-need-to-know/2011/01/18/
[REF-7] Business Process Modeling and Notation (BPMN) http://www.bpmn.org/
[REF-8] WS-BPEL http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsbpel
[REF-9] BPMN 2.0 and WS-BPEL www.infoq.com/articles/bpmn-2
[REF-10] Combination of a BPM Engine and an Integration Framework http://www.pleus.net/blog/?p=1028