The Future of EDI

Introduction

With the end of each year comes an inevitable desire to look back over the last few years to assess what has happened, and wonder what the years ahead are likely to bring. Unfortunately, however, when it comes to predictions regarding the future of EDI and B2B integration, industry commentators have a poor track record. Many of those who have peered into the tea leaves in an attempt to discern what the future holds for EDI have forecasted its swift demise. In spite of these predictions, however, EDI is still here and still going strong.

Yet the question of whether EDI will still be essential to supply chains in the future is still a valid one. Thanks to the emergence and continued development of potentially game-changing technologies such as AI, APIs and blockchain in recent years, fresh questions are being asked of EDI. Perhaps the most prevalent of these is ‘will EDI be able to incorporate these new developments or be superseded by them?’

By considering the views of industry commentators and addressing each of these new technologies and their potential relationship with EDI moving forward, in this article we aim to answer this question and provide clarity on a topic historically clouded by complexity and conjecture.

EDI’s History

In order to predict what the future holds for EDI it is first necessary to consider its history.

Although computer systems couldn’t exchange data until the 1960s, EDI’s roots date back to a system developed by US Army Master Sergeant Ed Guilbert for managing cargo information during the Berlin airlift of 1948-9.

By 1965 Guilbert’s original paper-based system had evolved into an electronic one, and the first EDI messages were sent in the form of trans-Atlantic cargo documents via telex.  Inevitably, as technology improved, faster and more efficient methods of data exchange were developed. However, universally recognised standards were needed to avoid confusion. Consequently, the Transportation Data Coordinating Committee (TDCC) was formed in 1968 by US automotive transport organisations, and released the first EDI standards in 1975. Six years later the American National Standards Institute published the first multi-industry national standard, X12, which was followed by the creation by the United Nations of a global standard, UN/EDIFACT, in 1985.

Significantly, the 90s brought the need for new protocols governing the transmission of EDI over the public Internet (EDIINT). This led to the creation of protocols AS1, AS2, AS3 and AS4, based on Simple Mail Transfer Protocol (SMTP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP) and Web Services respectively.

Why EDI is still standing

Since its inception, EDI has been constantly evolving alongside technological developments as businesses demand faster and more efficient document exchange processes. Such has been the hype surrounding some of these developments (like the introduction of XML and SOAP), that many have even forecasted they would replace EDI entirely. As yet, however, all such predictions have failed to come true. As Founder and CEO of Celigo, Jan Arendtsz, identifies, predicting EDI’s imminent demise is now almost “an annual ritual”:

“Competing standards, web services and modern APIs — all have been forecast to end EDI at one time or another. But EDI is here to stay for now as it still works well for many users.”

Likewise, Program Vice President of Supply Chain Strategies at IDC, Simon Ellis, notes:

“There have been many contenders to overthrow EDI over the years, and none of them have succeeded because EDI does what it does pretty well”.

Consider the development of the telephone over the past 50 years, for example. Phones today are almost completely unrecognisable from the rotary dial landline telephones of the 60s and 70s, both in appearance and how we use them. Yet whilst the technology has hugely improved, the key challenge – to connect people far from one another – remains the same. The evolution of B2B data exchange is remarkably similar. Whilst modern message formats may be far removed from the telex messages of the first EDI exchanges, the key business challenge – i.e. exchanging business information in as efficient a fashion as possible – remains as important as ever.

Meanwhile a similarity can also be drawn between the predicted decline of EDI and that of oil over the past 20 years. Indeed, despite repeated stating of the importance of moving away from fossil fuels, yearly global oil consumption today is over 25% higher than in 2000. Likewise, in direct contrast to the predictions of its detractors, EDI usage is rising, with a recent report by Forrester Wave placing yearly global EDI transactions at over 20 billion and growing. And the good news – in contrast to oil, the supply of EDI will not deplete.

But what about recent IT developments? Do they have the potential to replace EDI? Will they supplement EDI? Or will they have little impact on B2B integration moving forward? In order to answer these questions we need to look at each technology in turn.

API

Perhaps more than any of the other technologies in this article, APIs are widely viewed as posing the biggest ‘threat’ to traditional EDI over the short to medium term. Although few predict an imminent migration by companies from EDI to APIs, some, such as Erik Kiser (Founder and CEO of Orderful), foresee APIs as “eliminating EDI altogether” in as little as “five to ten years”. Similarly, Gartner predicts that over half of all B2B transactions will take place via real-time APIs by 2023.

What are APIs?

An API, or Application Programming Interface, is a collection of rules and protocols used by software developers which specifies how the different components of applications should interact. By exposing a particular API, a business can enable partners to access selected data directly, without the need for this information to be requested and delivered.

How APIs are changing B2B integration

Essentially HTTP-based APIs represent a natural development from RPC (remote procedure call), SOA (service oriented architecture) and SOAP (simple object access protocol), which have been around for decades. Following the same conceptual approach as RPC and SOA/SOAP, modern APIs have proved popular thanks to their ease of use and the existence of better support and more advanced technology. In particular, modern APIs are well suited to enterprise application integration (EAI) and in environments where the developer has control of the system and can dictate the process.

In theory, by allowing direct, real-time access to relevant information from trading partners, modern APIs have the potential to streamline B2B interactions by providing faster system integration. However, talk of APIs replacing EDI is short-sighted and in many ways mirrors similar discussions around the turn of the millennium concerning XML doing the same. For example, Nate Haines’s recent article ‘EDI Must Die’, in which he claims EDI to be an “outdated technology that companies should actively work on abandoning in order to move into the modern digital age” shares much with Uche Ogbuji’s 2001 article ‘XML – The Future of EDI?’ in which he argues that XML “has the potential of taking EDI from an arcane, if venerable technique to the rapidly developing center of enterprise information technology”. In the articles both authors argue that EDI is undermined by the fact that traditional formats are hard to read compared to XML/JSON, with Haines calling the JSON body of an API call “far superior” to the equivalent “unwieldy block of EDI text”. Unfortunately, this completely overlooks the fact that EDI messages are not designed to be human-readable; they are designed to facilitate the most efficient data exchange possible, which happens to be machine-to-machine with minimal human interaction. Likewise, although bandwidth availability is growing and data storage is cheap, message size is still important, and EDIFACT considerably outperforms both XML and JSON in this respect.

Another common argument for APIs over EDI is that APIs allow businesses to design more bespoke processes. However, whilst potentially useful on a small scale, this approach becomes increasingly impractical the larger the supply chain.

Consider a global supplier with 500 supermarket partners, for example. With every partner using APIs, it is possible that each would require the supplier to follow a different process with different data.

Three examples of such different processes might be:

• The supermarket wants the supplier to expose an API and pushes the data there
• The supplier is expected to pull the order from the API of the supermarket
• The supplier is expected both to pull the order by calling the API from the supermarket and to confirm the receipt of the order via a push to a different API

Typically, as the buyer, the supermarket dictates the process, which the supplier must then adhere to. That includes the protocols being used as well as the data formats being exchanged. Such a jumble of processes is not conducive to efficient data exchange for various reasons:

• There is no standardised way of designing APIs. One partner might request pull, one pull and yet another a mix of both approaches.
• The underlying process is not standardised. While one partner might request a purchase order response, another might not care about the response and instead take the successful order placement as confirmation. Meanwhile a different partner might request the use of purchase order changes. Thus, APIs typically differ in regard to the supported process choreography.
• The semantics of the exchanged data is not standardised. Classic EDI standards such as ANSI X12 or EDIFACT strongly build on the concept of information identifiers. These information identifiers are based on globally accepted code lists and unambiguously define the semantics, e.g., of a purchase order reference, an acknowledgement identifier, a date format, etc. APIs do not follow such globally agreed concepts, thus, greatly increasing the risk of reinventing the wheel with every API implementation.
• The payload of an API call does not define the encoding. In particular with modern JSON/REST API it is the transport protocol, namely HTTP, which must define the encoding, in which the data is being sent. Otherwise the receiver has no chance to process the received information correctly. While in most cases UTF-8 is used as the standard encoding for messages exchanges, other encodings such as ISO-8859-1, ISO-8859-2 etc. are often used by the IT systems generating a business document as well. In contrast to EDIFACT or XML, where the encoding is defined in the message itself (namely the UNB segment in case of EDIFACT or the processing instruction in case of XML), JSON does not foresee such a mechanism.
• API documentations are not standardised. Although there is tool support for documenting APIs, there is no common and standardised approach regarding how an API is documented for business analysts and software developers alike.
• APIs often miss routing-critical information such as logical sender address, logical receiver address or document type definitions. While these fields are not necessary in a point-to-point integration scenario, they are of utmost importance when communicating with third-party-networks, which in particular large corporations heavily rely on.

By contrast, exchanging self-contained messages in an asynchronous way offers much more reliable cross-company integration. By using for instance EDIFACT, all of the challenges mentioned above can easily be met.

But it is not only the technology that matters. The real challenges of modern EDI don’t concern syntax or communication protocols, but the following:

• Coordinating what data has to be exchanged between sender and receiver (on a business/conceptual level)
• Setting up new relations, including testing of all the different business scenarios, e.g. invoices containing different VAT rates, surcharges/rebates, purchase order changes, etc.
• Getting the right data in/out of the ERP system (this can still be cumbersome)
• Translating/mapping the semantics between different ERP Systems, e.g. mapping information from document level to line item level
• Having a streamlined and efficient onboarding process

With APIs only point-to-point integrations can be achieved in an EDI scenario. Unfortunately, this does not scale when connecting a larger number of EDI partners. By contrast EDI networks help to scale by quickly establishing connectivity and integration with a larger number of EDI partners.

Yet API integration is by no means an inferior technology to EDI. APIs are simply better suited for different application scenarios than EDI. Integrations based on APIs excel when it comes to the direct integration of various systems in the context of enterprise application integration – for instance connecting a CRM system to an ERP system. Similarly, this technology is useful when it comes to interfaces of the public administration, where a single API is used by all companies due to legal requirements. An example for this is the United Kingdom’s MTD (Making Tax Digital) initiative. Other examples include Web Services offered by governments for electronic invoicing such as the SDI system in Italy. However, also in these domains we already see a strong uptake of Peppol, an EDI communication protocol for electronic invoices. In this case, as is often so, it is the EDI technology replacing point-to-point API integration approaches and not vice versa.

Bearing the above in mind, the following result of a survey conducted by Project 44 among 200 supply chain executives is not unsurprising. Although 63% predicted APIs will play a significant role in the future of B2B integration, only 5% predicted that EDI, by contrast, will abandon its key role.

In short, both EDI and API technologies have their own specific application domains in which they both perform exceptionally well. As such, neither technology is on a collision course with the other (as suggested by some). When it comes to the next ten years, we are likely to see highly flexible and loosely coupled integration scenarios using message-based systems dominate supply chains. APIs will coexist in these scenarios and perform well when it comes to deeply integrated system-to-system integration. Meanwhile EDI will remain the main workhorse of any logistic or financial supply chains.

Blockchain

Identified by TechCrunch as “the next step in the integration evolution”, blockchain, like API technology, has often been cited as posing a threat to traditional EDI.

What is blockchain?

Originally developed to power Bitcoin, Blockchain is a ledger-based technology that records changes/developments relating to anything of value (e.g. goods or currency). Blocks of data are linked chronologically, with new blocks created and added to the end of the chain to log any changes made to the information concerned. Once created, each block is essentially a read-only record of a change. As blockchain was designed to be decentralised, before a new block can be added to the chain, the computer seeking to add the block must solve a puzzle and provide proof-of-work to other computers on the network, which must in turn verify the information. Once created, blocks then can’t be tampered with. This system ensures data accuracy whilst enabling maximum visibility for all parties.

Blockchain and EDI

So how exactly could blockchain impact EDI and supply chain data exchange? Many believe that digital shared ledgers (DSLs) will become more popular as organisations move away from point-to-point communication towards a more collaborative approach in search of faster, more reliable B2B communication and increased data visibility. Whilst IBM has identified the positive capacity for blockchain to deliver “a tamper-proof record of relevant events […] for all supply chain participants”, several have also noted the potential for blockchain to boost supply chain analytics and data error monitoring. Meanwhile, Stan Gibson of Digitalist points to the enormous benefits for supply chain organisations concerned with provenance (or where exactly products come from).

For example, blockchain could prove very helpful to businesses in the meat industry as it allows for potentially universal access to key information such as where the cattle was grown, where the slaughtering took place, where the meat was processed and deep frozen, etc.

Some, such as Karthikeyan Mani, CEO of ByteAlly Software, believe blockchain will become the preferred medium for EDI messages to be transmitted, completely removing the need for trading partners to exchange files. Thanks to the advantages of improved data visibility, Mani notes “There is simply too much to gain […] for us to ignore blockchain as the transmission medium of EDI”.

Certainly, there does seem to be increased interest in blockchain, with Matthias Roese (Chief Technologist for Manufacturing, Automotive and IoT at Hewlett Packard) noting that “everyone is looking into it and doing pilots”. Indeed, according to IDC, investment in blockchain is almost doubling year on year and is expected to exceed £7.5 billion by next year.

However, as identified by Bilgin Ibryam (Principal Architect at Red Hat), blockchain becomes complicated in a multi-party B2B integration landscape. In his article The next integration evolution – blockchain, Ibryam explains that in order for blockchain to be successful in supply chains, businesses which “do not trust each other” must agree to implement “a new breed of […] technology that relies not only on sharing of the protocols and contracts, but sharing of the end-to-end business processes and state”.

Yet even if businesses are successful in doing this, most acknowledge that this doesn’t signal the end for EDI. Whilst traditional EDI may admittedly need to evolve in order to enable businesses to experience some of the benefits of blockchain technology, as IBM clarifies, “EDI is alive and well and will remain critical to business for many years to come”. Even if blockchain was to become EDI’s principal transmission medium, similar transitions (such as the move to web-based protocols kickstarted by Walmart in 2002) have happened before with little negative impact on EDI as a whole.

In short, whilst blockchain is a great concept for generating trust in a decentralised network, this is not the principal problem that EDI is focused on fixing. EDI is concerned with solving integration challenges, which blockchain is unable to help with. Even if EDI is done via blockchain moving forward, the need for integration between information in the blockchain and the ERP systems will remain. So, therefore, will the need for EDI.

Essentially, though they may both play a significant role in B2B interactions in the near future, as EDI and blockchain solve different problems, for the moment at least they cannot be combined. Rather than merging, they will work in parallel with one another, as shown in the diagram below.

Artificial Intelligence (AI)

Although less frequently touted as a potential successor to EDI, AI is increasingly being discussed in conversations concerning the future of EDI, with IBM noting that “the true future lies in using and evolving EDI alongside disruptive technologies such as IoT, blockchain and AI, to deliver innovative levels of multi-party supply chain collaboration”.

What is AI?

In simple terms, AI is ‘intelligent’ software / technology that has been developed to learn patterns and solve problems in an almost human way.

How could AI change EDI?

One of the key benefits of modern EDI is the extent to which it is able to streamline B2B data exchange and remove the need for time-intensive and error-prone manual processes. However, all traditional EDI systems require maintenance to ensure they continue to function at maximum efficiency. AI could potentially reduce the effort required to keep systems running at optimum level and ensure errors are caught and resolved faster.

For example, AI could learn to spot patterns in customer ordering and flag when this pattern is interrupted as a usual order is missing. Similarly, AI could be used to automatically correct certain data exchange errors rather than simply flagging them for a human to resolve. AI could even be used to benefit business strategy by making recommendations based on advanced data forecasting by identifying patterns in customer demand and preparing to adapt accordingly, for example. Another key use of AI could be helping to prepare mappings if the requirements of both sides are known.

Unfortunately, however, AI is unlikely to be able to help streamline the most time-consuming steps in the EDI process today – namely the human coordination between different parties during partner onboardings, semantic discussions and definition of import/export formats.

Whilst AI could help to augment EDI moving forward through improving message conversion, it has little relevance to message transmission and thus, like APIs and blockchain, falls short of constituting a threat to the future of EDI as a whole. Rather, like APIs and blockchain, AI will evolve alongside EDI, with improvements in both helping businesses to streamline essential business processes.

Conclusion

With so many new and powerful technologies and such renewed interest in B2B integration, the coming years should bring exciting developments. As recent projects such as Peppol have shown, we are slowly moving away from point-to-point connections towards a more collaborative B2B environment with better standardised communication between networks. Further, APIs, blockchain and AI could potentially dramatically increase the efficiency and transparency of important data exchange, with IDC research suggesting businesses will gain a 308% ROI with modernised B2B integration – or more than £4 in benefits per £1 invested!

As we have explored, however, none of these new developments or technologies signal significant change to the role of EDI in the future of supply chain communication. Amongst the clamour of voices every year heralding a new era in B2B integration, it is important to remember that unlike API, blockchain and AI, EDI itself is not so much a technology as a concept. Ultimately anything that can be classified as computer to computer exchange of B2B documents is electronic data interchange. In as far as EDI is considered to have challenges and issues, therefore, those issues are not solved by changing the syntax from EDIFACT or X12 to JSON, as has been proven by similar discussions concerning XML in the past.

Simply put, as long as companies need to communicate with one another, EDI, and the concept of data exchange networks, will exist. As a result, despite the gloomy predictions of some commentators, Grand View Research forecasts that the global EDI market will be worth nearly six billion dollars by 2025 with a cumulative annual growth rate of (CAGR) of 9.4%.

Alongside APIs, blockchain, AI and other upcoming technologies that are enabling businesses to improve more and more business processes, EDI continues to solve a crucial problem for supply chain organisations, and as such isn’t going anywhere. However, whilst none of the technologies we’ve looked at in this article are capable of replacing EDI, as they evolve alongside one another, what exactly EDI will look like in another ten years is up for debate.

References:

1. BP, 2019, ‘BP Statistical Review of World Energy’, accessed 31 December 2019, https://www.bp.com/content/dam/bp/business-sites/en/global/corporate/pdfs/energy-economics/statistical-review/bp-stats-review-2019-full-report.pdf
2. Christopher, Claire, 2019, ‘5 EDI Software Trends: The Future of EDI’, accessed 31 December 2019, http://sandhill.com/article/5-edi-software-trends-the-future-of-edi/
3. Gartner, 2019, ‘Use APIs to Modernize EDI for B2B Ecosystem Integration’, accessed 31 December 2019, https://www.gartner.com/doc/reprints?id=1-1OC4DRR1&ct=190730&st=sb
4. Gibson, Stan, 2019, ‘Can Blockchain Replace EDI in the Supply Chain?’, accessed 31 December 2019, https://www.digitalistmag.com/digital-supply-networks/2019/03/04/can-blockchain-replace-edi-in-supply-chain-06196784
5. Haines, Nate, 2019, ‘EDI Must Die’, accessed 31 December 2019, https://www.dsco.io/blog/article/edi-must-die/
6. IBM, 2019, ‘The Future of EDI: An IBM Point-of-View’, accessed 31 December 2019, https://www.ibm.com/downloads/cas/WQB6NBJ7
7. Ibryam, Bilgin, 2019, ‘The Next Integration Evolution – Blockchain’, accessed 31 December 2019, https://techcrunch.com/2019/02/05/blockchain-as-integration-evolution/
8. Mani, Karthekeyan, 2019, ‘Will Blockchain Replace EDI? Yes And No’, accessed 31 December 2019, https://www.forbes.com/sites/forbestechcouncil/2019/04/03/will-blockchain-replace-edi-yes-and-no/#118d5fbb47d7
9. Ogbuji, Uche, 2001, ‘XML The future of EDI?’, accessed 31 December 2019, http://uche.ogbuji.net/etc/tech/articles/xmledi.html
10. O’Shaughnessey. Kim, 2019, ‘EDI Software Trends: The Future of EDI for 2020’, accessed 31 December 2019, https://www.selecthub.com/electronic-data-interchange/edi-software-trends/