Like any technology ecosystem, the integration world is always evolving. We spend a lot of time evaluating integration technologies from commercial offerings to open source products to homegrown integrations. There are A LOT of options out there, so it can be hard to see the macro trends that are occurring.
In this post we'll look at some of those trends and try to paint a picture of what the future looks like for integration. We'll also talk about why it's different than what we have today.
Some of these trends are already occurring. Others are more aspirational. Hit us up in 2030 so we can circle back about our predictions!
The Future of Integration is Adaptive
APIs evolve. It's just the reality of software development. New features are released. Bugs are fixed. Either of those can result in changes to the contract between the software and the outside world. This is one of integration's fundamental challenges.
Let's say you build an integration to XYZ system, and XYZ does quarterly releases. That means once per quarter, your engineering team will need to digest the release notes for XYZ, understand if any API changes are represented, assess if those changes impact your integration, then push a software release to react to them. Opportunity abounds for mistakes, missed emails, misunderstood documentation, or delays in updating. That's just for a single integration.
Now imagine your product has 10, 25, or 100 integrations, all with different release cycles, types of APIs, and quality of documentation. You'll have an entire team dedicated only to maintaining your existing integrations. That team isn't adding functionality to your core product. It isn't expanding your market presence by building new integrations. It's dedicated to maintenance.
Today, there are metadata frameworks like OpenAPI that help APIs communicate what they are to the world, but they are merely static. They do not describe what about APIs is new and different, nor do they provide any context for why those changes matter. Those who consume the APIs must rely on human brains to make such assessments.
In the future APIs will be able to describe themselves and their changes, and artificial intelligence will be used to dynamically adapt API consumers according to those changes. This will allow software products that consume these APIs to immediately adapt to these changes.
Human intervention will still be necessary, mostly to guide intention and to govern that the AI doesn't spin out of control, but the majority of the adaptation to API changes will be automatic. This will free up engineers' brain-share to focus on more value-additive activities.
Some really smart people are already working on this problem.
The Future of Integration is Intelligent
Not only will the future of integration be adaptive; it will be intelligent.
Today, product managers must partner with market-facing teams to decide what integrations are important. What do customers want? Where might there be more customers? Where can we add more value to our existing customers?
Nothing will replace the power of human creativity and humans come up with some pretty creative ways to make software products work together. But, that doesn't mean it has to be a solely human effort.
In the future AI will assist integration teams by recommending integration requirements. This may be enhancements to existing integrations (e.g. adding a new data flow) or it maybe discovering an entirely new product to integrate to!
Imagine something like the friend suggestions that pop up in Instagram, except recommending new integration partners.
Products like Crossbeam and their partner-reference product Partnerbase are the start of datasets that represent who plays with whom. In mature form, these types of datasets will inform recommendation engines such that they identify integration use cases your team wouldn't otherwise see.
The Future of Integration is Private
We're in an early chapter in the book about data privacy. The human race is contending with this massive, open thing we've built (the Internet), and we are starting to reconcile the good and the bad it brings. The coming decades will see more and more regulation from governments, NGOs, private citizens, and even corporations regarding protecting private, sensitive, and personal data.
Companies use many software products to market to customers, facilitate delivery of their services, charge them money, and do generally all the things that a company does. Those companies must use many products, because no single product does everything. Therefore, companies must integrate those products to achieve their goals.
These companies are already at risk of data breaches, because they hold sensitive information in that portfolio of commercial and homegrown products. Data at motion is at even greater risk, and motion is the sole purpose of integration. More specifically that risk is largely the responsibility of the integration provider, be they a software vendor or a services provider.
The future will require integration providers to take oversized responsibility for data security and privacy. Regulation will certainly dictate such, but market dynamics will ultimately drive the change. If you cannot guarantee the safety of the data moving through your integration product or with your service, your competitor will.
This will encourage integrators to invest in security and privacy, to invent new innovative ways of protecting data, and to view data privacy as a fundamental feature.
The Future of Integration is Decentralized
The economics of an integration business are tough. You sell a thing that nobody really wants. The only way to delight a customer is to remove the impediments to what they really want (the collective value of two products working together). The best customer experience for an integrator is one where the customer doesn't even know you're there!
However, integration costs money. The integration provider needs servers and software developers and computers. They are software companies just like any other. Therefore, they must build a product and sell it to the people who need it.
Think about that. The market is forced to pay for a thing that they don't really want. It's basically a tax, imposed because the software ecosystem can't inherently deliver on the overlapping product experience that end users actually want.
Thus we have an ecosystem where integration is almost always facilitated by a central authority--be it an integration services provider or an integration software provider. That central authority must make money. Therefore the ecosystem and ultimately its end customers must pay.
This centralized integration model is like an orchestra. There are many moving pieces (cellists and percussionists and flautists), but the value they provide (music) is coordinated by a central authority (the conductor). Without that central authority, the orchestra would be disconnected and unable to deliver that value.
Integration exists this way, because the technology and lack of widespread understanding of best practice integration obligate it to exist. Integration is a specialized engineering discipline, so those who have it become the conductors of that set of use cases.
But, it doesn't have to be this way (and it won't be).
The future of integration will be decentralized, because of the adaptive, intelligent integration technologies that will come. It will be decentralized because the knowledge of how to do integration well will also become common.
In this future, integration will be more like jazz. Each member will be autonomous and qualified, but none will necessarily be in charge. They'll organically work together to produce the music, and it may not be choreographed in advance.
Instead of a bunch of individual software products in an ecosystem integrated with a set of integration services providers, imagine those individual software products opting into an organic data sharing network. You plug in and have locally controlled access to other providers' data and likewise they do to your data.
These integrations are governed by the participants in the network, not by the central authority who controls the data rails. This would remove the added layer of cost that is eventually pushed down to the end customer, opening the market for investment in other innovations and business models.
The Future of Integration is Not iPaaS
Today, integration is a world of ETL (extract, transform, load) tools, data streaming tools, ESBs (enterprise service bus), and iPaaS (integration platform-as-a-service). Even integration services (not product) providers almost always deliver their services using one or some of these tools.
Full transparency: Blended Edge is one of these.
But, the future of integration is a set of tools that probably haven't been named yet. Perhaps iPaaS evolves into something that creates adaptive, intelligent, and decentralize integration ecosystems. But, at that point is it even iPaaS anymore?
While there are incredibly useful, innovative integration products available today, none fundamentally solve the integration problem: nobody wants it, but everybody needs it.
