Benedikt is a Consultant working in Integration and CRM projects for large enterprises. He has experience with different technologies, governance frameworks and roles, from developing and designing to leading and budgeting. Benedikt is an active contributor to Bitcoin meetups, and he follows discussions and congresses about cryptography and enterprise security. One of his main interests is tackling well-known enterprise software challenges with cryptography and blockchain technology.
Integration – The Blockchain 'Killer Usecase' – Part II Published: March 31, 2016 • Service Technology Magazine Issue XCIV PDF
This is the second part of a two-part article series. The first part is published at Integration – The Blockchain 'Killer Usecase' – Part I.
Why Blockchain Integration in the Enterprise has its Limits
Here is the problem: standards are an issue, because they would need to be followed. The Bitcoin protocol – ironically enough, because it is a decentralization approach – has an extreme point of view, when it comes to standardizing and centralizing the language necessary to execute transactions and eventually store data. There is just nothing else then Bitcoin Blockchain. The wallets are 'mere slaves', who have no other local Data Store and no other way to talk about Bitcoin than how the Blockchain commands them. That works for some use cases like finance and potentially legal, but is this a scenario that could resonate with the enterprise application landscape? Certainly not, there is an abundance of Applications out there all with different ways of describing concepts and storing data and often these applications are just what a group of users once wanted. Stripping them of their Data Models and forcing their data into a Blockchain or even their application logic onto the Blockchain with a Turing complete language would make little sense for most of them. There is no need for the specifics of the Blockchain way of storing data and in many cases things might scale or just be over – engineered. Using Bitcoin Blockchain Integration Pattern as a general Integration approach is not a good idea, because it is too restrictive and would have most general purpose Applications suffocate.
Just to be clear: no one ever said it would be a good idea to use a Bitcoin style Integration Approach as a general way to integrate Applications. We started out investigating, why the Bitcoin Blockchain is able to quickly integrate wallets and have them talk to each other, while in enterprise Application Integration Archtiecture making Applications talk to each other poses a serious difficulty. The answer we believe to be correct is that the Bitcoin Blockchain not only masters and centralizes data but also governs the database to store data and nomenclature how to speak about transactions. Following this pattern in an Enterprise approach can help to allow switching applications rapidly (no Data Migrations, just like a wallet) and it can have benefits for certain financial (currencies) and legal (smart contracts) use cases but it is not valuable when seen as a general approach to Integrate Applications.
We will come back at the end of the article of two positive (but admittedly
speculative) takeaways from the Bitcoin protocol for the Integration question. Now, we want to proceed to suggest a System inspired by Bitcoins prove of work, meant to be able to overcome the described 'Integration Translation' problem.
The Babelchain – A Consensus Algorithm Translating Machine Language
In the following we will outline how an algorithm that can solve the described integration problems would look like is used within a Blockchain and the principle of Bitcoin like decentralization and using computer solvable riddles. We will give an example of an algorithm that fulfills criteria we suggest but we do not claim to have a ready made and worked out solution here yet.
Blockchain protocols favoring a 'proof of stake' consensus algorithm do these among other reasons to not induce the high costs of a 'proof work' (mainly electricity costs) consensus algorithm upon the eco system. One point to notice here is that scenarios both algorithms describe to undertake a successful attack (51% attack, long range attack) seem to some extent artificial because any Blockchain System will some level of adoption will be regulated eventually by social and legal conventions. If a player manages to attack, undoubtedly members in the network will realize that and punish the malicious member by excluding him from transactions or bringing to justice, where ever the legal system already implemented measures for crypto currencies and contracts. Building security into the 'guts of the protocol' is a necessary approach, especially for financial transactions in a network like Bitcoin, where members don't even know each other Having said that, well known social sanctions to enforce rules will at some level of adoption and acceptance of a System have to take over.
Reverting to a more energy efficient 'proof of stake' is one way of dealing with energy waste, another way is too simple find a way to make ' proof work' algorithms useful, that is have the CPU execute some useful operation. There are proposals to have CPUs produce mathematically useful numbers, that communities in mathematics might need. We will try here to suggest an algorithm, which will enable translation between machine messages and at the same can be used as a proof work mechanism and in a hybrid approach with a stake and reputation system achieve the level of security and immutability necessary for the System we want to suggest.
Bitcoins starting point could be summarized in the following question:
"How can machines securely come to an agreement about the state of a shared ledger without resorting to trusted third parties?"
This question assumes, that machines necessarily understand each other, but that is not the case. It is the case, if the Blockchain forces everyone to its language and data store. But look at a 'normal IT landscape', where we have separate Data Stores and we have different dialects talking about the same thing in different ways. In that respect Bitcoin looks at a special case, because it looks at the case where n – Data Stores and n – dialects are reduced to one, which is perfectly reasonable for building a currency.
An interesting similarity between the question of how to create trust and how to create translations is that in both cases, reverting to a trusted, understanding both middle man' is a natural design solution. Language translation is a typical example where people revert to a competent middle man like a translator. When we want to make machines talk to each other, we introduce the notion of an ESB. The question that we want to ask is therefore
"How can machines come to an agreement about the meaning of a message without resorting to a trusted third parties?"
Meaning is a fuzzy concept, there are myriads of books written about this concept in linguistics and philosophy. We will we further below offer a simple technical definition. Before we dive into the details, we should note that translating without a translator is an every day concept. Anyone travelling to a foreign country know, that we are able to find out the meaning of words without being able to use a translator. Transporting meaning with the help of what one might call 'cryptographic quizzes' is also a technique well known and successfully used in public private ket cryptography.
To make this technically work, we want to suggest a mechanism we re going to call 'proof of understanding'. If it works we would have for obvious reasons a 'talking Blockchain, which we might want to call Babelchain. Those participants in the network, which facilitate will be called Translator – they do what Miners do in the Bitcoin Blockchain.
'Understanding' is work and the algorithm we want to present is therefore a version of the proof of work mechanisms that anti email spam system or crypto currencies like Bitcoin use. We will suggest – but not go into the details – to use this proof of work in combination with a stake and reputation system with in the system to decide which the next valid transaction in the Blockchain will be.
This is necessary because the proof of understanding we suggest will not have the desirable mathematical attributes which allow Bitcoin to design the puzzle such that on average it will be solved every 10 minutes. Finding Translations between messages can be trivial and can be impossible, if for example one participant doesn't want to be understood and tries to attack the network with an unsolvable riddle as a Denial of Service attack.
Another reason why we need a backup for a proof work will be that 'Translators' (these are miners in Bitcoin) will learn how different machines communicate and store that knowledge. The work they need to translate will decrease and at some point in some cases be too little to be able serving as a proof of work. We will not describe how such a learning message would look like, but only postulate that an algorithm solving the translation problem would need to have such a feature.
Here are the attributes such an algorithm would need to fulfill:
Principle 1–3 are directly reflecting the Bitcoin Patterns. The smart approach we want to follow here is that Bitcoin reformulated the problem of trust in a way which makes it manageable by CPU and storage. In fact, it uses Computers where they are best (storing large datasets, calculating and repeating the same steps over and over again) and clearly outrun human brains. Reformulating a task such that Computers can tackle it with its strong features sets the stage for solving the problem autonomously and de-central instead of reverting to a trusted and costly middleman. Using a crypto currency can be seen as a means to assemble the computing power needed, when the primary goal of the network is other than only allowing payments.
Principle 4 is a logical result of 2, the fact that participants in the network work for rewards and hence will be interested in working as efficient. We don't have to describe as part of the protocol, how these translators will learn. We just should be aware that they will start understanding hoe certain applications talk to each other and what the best algorithms are to achieve this.
The last principle is necessary because the average of the proof of work mechanism cannot be predicted and generally will tend to go down as translators learn how certain machines and applications communicate.
A Babelchain 'proof of understanding' algorithm
With a missing translation you might want to call the solution to the 'translation
puzzle' is asymmetrically shared between sender and receiver. The sender knows what he wants to say, but he doesn't know how to communicate it such that the receiver will understand him. The receiver would know, which message formats he understands, but he doesn't understand the content of the message of the sender.
To clarify this we want to define 'meaning' in this context as a key value pair, where the keys can be placed in some hierarchical order. Think of dataexchange formats like xml, csv or byte formats, where we have values located pre-determined by the data format.
<body>Don't forget me this weekend!</body>
<importance> high <importance/>
An xml format is a particularly easy to read format, but any format that can be described via regular expression has a meaning in that sense. Messages in that understanding consist of three parts: Format, content and assignment rules. Here is an example.
Format of the message:
Content of the message
Don't forget me this weekend
Note that in reality any order of the content parts is possible, in fact without a message format (here: the end of line feed) it isn't even clear that values like 'Tove' and 'Jani' are two disparate values. Content and Format only result in a message, if the content gets assigned to the format, the slots that the format provides for values to be assigned.
The 1st round of the quiz is designed to solve the 'sender's format' problem.
1st Quiz round Find Format Sender and Receiver agree to use
Reward here could mean a point system, an own currency or simply that this translator and e.g. the top10 who solved the riddle first go the next round. At this stage, we only segregate the winners from losers, the process is not yet finished and the Translation cannot be inserted into an immutable log yet.
Obviously, there is a lot to say about how such an algorithm could work. Would we just have an algorithm run through all combinations of ASCII Codes until both sender and receiver claim to understand a message format? That could be a last resort option, which might necessary if a new machine with an entirely unknown message format wants to participate in a conversation. The network would need to be able to claim higher rewards for such puzzles and eventually throttle or exclude a machine if solving a puzzle consumes too much resources or might be practically (deliberately, in the case of a Denial of Service Attack) unsolvable. In many cases there will be heuristics and pragmatics at work to find a common message format. Applications for example could quickly revert to industry standards or best practices.
The first part of the riddle was about file formats: what is a message information format that both sides (claim to) understand and can work with?
The second round of the quiz is about the values, which get assigned to keys: the
values as such are not the 'riddle', they could be easily sent. But without a data exchange format it wouldn't be clear how to interpret those. The question here is therefore, in which slots one assigns the values into the format to form a message. Technically, there is an empty set or structure of keys, upon which sender and receiver agreed, there are values and the challenge is how to assign values to keys.
The 2nd round of the quiz is designed to solve the 'receiver's content' problem.
2nd Quiz Round: Reconfirm Content is assigned correctly to the agree format:
In the best case, the result of the algorithm will be that we have one agreed message format and values exchanged based on that format. The translated message would be written into the Babelchain and get secured by a combination of proof of work, reputation and stake. The algorithm is meant to be an exemplification with the 5 requirements listed above showing, how the translation problem could be formulated such that CPU's can work on solving puzzles with their superior computing power and no trusted middleman. This algorithm could (like most real life tasks) come in the worst case not to a conclusion (step 3 in both quiz rounds could always result in disagreement), which we will discuss in the next section.
Rewards, Denial of Service, Accounts and Transactions
One important question here is how to set the reward properly. The first part of the puzzle was potentially significantly more difficult than the 2nd part, but the message is confirmed only after the 2nd round. Reward would need to be split up such between these two parties, that nodes are incented to execute this translation work.
Rewarding nodes is closely connected to how the mechanism to have translations going to the Babelchain will eventually be calibrated. If this is the 1st time machines talk to each other, the only 'judge' for the second part of the riddle, before the message gets inserted into the Babelchain will be the sender. He is the only one, who would know what he meant. If machines repeatedly talk to each other, the translator nodes would start remembering, what translations were accepted before hands. At that point in time we would have a reputation and stake system deciding together with the initial sender, what the correct translation is, what translation will get inserted into the Babelchain and which translation node receives rewards. 'Reputation' could be defined as to how many correct versus incorrect message formats and content assignments were offered by a node. Nodes, which offer many good solutions and make little mistakes would have a higher vote on deciding what a correct translation is. 'Stake' could be defined as the amount of transactions participants run or the amount of currencies a member holds. This would result in 'voting power', the overall idea being that someone using the system heavily and holding its currency will be interested in translating correctly and without working. The combination of 'work to understand' (mostly for new and unknown formats), 'reputation' and 'stake' is meant to give means to calibrate a solution which will allow members to have sufficient trust in the translations made for any practical means.
Accounts in a Babelblock should be seen as a mixture between an application endpoint and a Bitcoin Wallet. Like Bitcoin accounts they would have a balance, which is derived from the underlying Babelchain. We will call this currency 'spreak' which is an artificial word and a mixture of 'speak' and 'spread' (the word). Hence, an application in the Babelchain eco system would have
Here is what a Transaction, we will refer to it as a Translation, would be composed off for the Babelchain.
Separating the message into data and format is a prep-step we need the sender to do for the described quiz rounds. Giving Quiz Hints will allow the sender to make it very easy for translators to find a format the sender will accept, for example if he can refer to an industry standard. He will loose security since the proof of work will be trivial and in many cases, he will not be able giving such hints or they ight not be successful.
The described Quiz will suffer from a version of the Turing halting problem, that is to say it is not clear if certain riddles can ever be solved. Ethereum uses a notion of 'gas' senders need to have to be able to block processes running ad–infinitum on their turning complete machine.
Imagine, sender and receiver might simply not come to an agreement (step 3 in both quiz rounds) and the algorithm would run into an infinite loop and never halt. This could be intentional as a denial of service attack or the message could be so 'unlikely' that no heuristic and pragmatic rules can guess the right common format and the network starts going through all combinations of ASCII codes without solving the quiz. But only alone to handle Denial of Service Attacks the System needs to be able to stop a Translation. This could be done by asking senders to pay for transactions upfront and place the payment into the transaction. This will be handled by the Reward field. In the positive outcome, the translators who solved the quiz rounds would split up the reward in a to be defined proportion. But if the reward doesn't cover the work that is imposed onto the network by a translation and the translation isn't finished, the sender would loose this reward to the translators that executed the most work. The reward would therefore have the double function to reward and incent Translators and to penalize Senders, if they require difficulties or impossible quizzes to solve. This way of penalizing would require translating computational steps into costs expressed in spreaks, the currency of the network.
Open and Closed Questions
A high level design leaves naturally a number of questions open. For a currency system, we would need to have much closer look at money supply and incentives. If senders pay for translations, money supply could be such that senders have to buy the crypto currency spreak in a real world currency. That would be a very different approach than Bitcoins Mining approach, which has a limited and decreasing money supply. Rewards between miners would need to be split, with a learning network, if will be a question also, if many translations are so trivial (since already known) that translators might not be incented to do small pieces of work for such quizzes. Setting up a properly calibrated consensus Mechanism with mechanisms of work, stake and reputation proof requires more investigation.
And of course, there are questions to solve concerning the algorithm. Because if our algorithms don't manage to translate messages and don't do that efficiently the economic value of the entire System is not given. There are investigations in how difficult quizzes can get in the worst case (not quiz hints) necessary to assess the feasibility of certain scenarios. On Receiver side there will also be a set of pragmatic rules necessary to determine, which message one can really accept. Because at the end of the day, a system needs to be able to act upon a message it receives for example update a customer address. Knowing in which slots of an abstract content structure a value needs to be placed is only the first step. The system then needs to be able to do something with these values.
All these questions would require more investigation and will not be an easy task to design and implement. What we hope to have demonstrated is that algorithms with the afore listed five attributes are possible to design and that this can help solving the described challenges of Application Integration.
Conclusion–eliminate the trusted middleman
Even though a plain reuse of the Blockchain for an Enterprise Context has a certain set of useful but eventually limited use cases, from a broader perspective we can learn from the Bitcoin Approach. The main design principle should be eradicate any form of centralization in the System.
Bitcoin started with a disgruntle over the Banking Systems during the financial crisis from 2008. In fact, whenever we get involved in a transaction, unless we pay cash to the guy next to us, a bank or some kind of payment Systems is located 'in the middle' between us and the recipient. In many situations, that's fine and we might not even realize it. In many situations however, we might pay transaction fees, which can be significant. Even more, there is unease over the power we give to these 'evil middlemen'. Capital is gathered and those institutions, which are designed to facilitate economic activities, are moved into a power situation, which many feel is not justified by the value they add. How Bitcoin and protocols like Ethereum could change the power–balance towards less centralized Systems would be an exciting. Here however we want to point out some similarities between Banks and many of nowadays Integration layers that facilitate communication between IT Systems. We will try to describe (and for the sake of the exercise force) the Integration problem as a version of the 'evil middle man' problem known from the banking world.
If there is one clear similarity between a bank and an enterprise service bus, then simply that it is overhead, usually an annoying one. If you want to send money to your cousin on the other side of the world, you just want to get the money there. You might be indifferent or even opposing supporting huge and rich institutions 'in the middle' between you and your cousin. The moment they charge you transaction fees, you will find dislike the financial impact. If you are political conscious you might be alarmed by the damage banks cause within the economy, the bonuses they pay out to themselves and the political power they gain by hosting your capital. Paradoxically an institution meant to serve an ecosystem puts a huge burden on it and poses a risk of destabilizing it.
In many enterprises working with large and monolithic Enterprise Service Bus you will find just the same kind overhead and just as a bank, this overhead implies costs and damages the ecosystem it is meant to enable. There are less political consequences attached here. But the economic consequences for those in need of such Information Integration Brokers are severe. Interesting enough, more recent Service Oriented Approach is evolving around the concept of so called 'Micro services'. In one (admittedly oversimplified) reading of this concept one could summarize this as: 'Micro Service = Service Oriented Architecture – Enterprise Service Bus'. The idea is that applications in an enterprise build up point to point connections to the systems they need to talk to, there is no central information broker like an ESB. Often, such Micro Services run in so called containers, which is a light version of a virtual machine, where many containers can run on top of one operating systems and even share resources. Even though Micro Services have different set of arguments for their conclusion (and we didn't described them by any means comprehensively) they come to a similar conclusion, that centralization is a problematic design approach for Enterprise Integration Architecture
Just as Bitcoin designs an algorithm to create trust and secure transactions without a man in the middle we are challenged with creating understanding and enabling translations without a man in the middle.
Work starts here. It is 'easy enough' to describe a vision, the 'nitty gritty' details of Implementation will bring up difficulties we can't even foresee now. And even before an Implementation there are some major architectural questions to dive into, of which we even only some brought up here.
We hope to have given evidence to the view that a centralized Integration pattern is falling into the 'evil middle man' trap and that avoiding this form of centralization can unburden IT systems of Enterprise Service Busses like Bitcoin unburdens payment Systems from Banks. Our conclusion here is similar to modern service oriented approaches like Micro Service, but we draw the Conclusion from different arguments and suggest to find different solutions that are inspired from Bitcoin and Blockchain design Patterns.
We presented two Integration patterns inspired by the Blockchain and Consensus Models. Transferring the Bitcoin Integration Approach towards Enterprise Application could help mainly with financial and legal use cases similar to those Bitcoin and Blockchain facilitate but would in our view fail to cover a broader set of use cases as it requires too much of standardization and centralization for a common language and database.
Finding a decentralized 'proof of understanding' algorithm and embedding it such in a stake and reputation system that Systems can achieve consensus on its status is a more promising approach of reusing Bitcoin principles like decentralization and usage of CPU capacity. Further research will have to show if algorithms with the described properties can be designed having the necessary mathematical attributes and eventually leading to successful Implementations.
The hope we have is that taking a fresh look at the Integration landscape from the perspective of Blockchain and Consensus Technology can help solving well known Integration Problems and at the same time will help an innovative Technology and its Blockchain eco system to flourish even more, while nurturing on thee industry 'Integration killer usecase'.