A modern business-to-business software product requires integrations. So, if running or working on a B2B software product team isn’t hard enough already, add learning how to build integration to your list of to-dos.
Building a product team that is successful with delivering and managing integration requires three foundational areas of expertise: integration expertise, domain expertise, and system expertise.
These three areas complement each other and are equally important. When you lack any of the three in the context of a single integration or your integration program as a whole, you risk creating technical debt, building the wrong thing, or investing in integrations that don’t provide enough value for you or your users.
In this post, we’ll get into what those three types of expertise are and what it looks like to develop them yourself and on your product team.
Note: In this post, we’ll use the term “product team” pretty broadly. Not all teams are structured the same, but we mean the group of people who design, build, and deliver your software product.
Integration Expertise
Integration expertise is the ability to build, scale, and manage integrations, generally. In other words, put aside the details of a specific integration project.
Integration expertise is largely technical, but especially within a SaaS product team, there’s a large process part to it too. That’s because product teams build integrations that serve many users, which have a higher cost of ownership than one-off integrations.
This type of expertise is typically under the purview of the CTO or engineering leader, but not exclusively so. You’re also starting to see companies who put this responsibility under a product leader or even the partnership team.
Building scalable integrations is hard. It takes a lot more than just knowing how to write code that talks to an API. There are specific things that an integration expert (or a team with integration expertise) should absolutely know.
An integration expert or a team with integration expertise should have mastery over the following topics:
- Methods of Authentication/Authorization
- Common API Patterns and Types
- How to Move Large Amounts of Data
- A Specific Integration Technology
- Operational Best Practices
Authentication and Authorization
Authentication/Authorization is part of building an integration that trips up a lot of teams.
Authentication is the process of validating that a person (or system) is who they say they are and Authorization is the process of deciding whether that person or system has access to the data requested. These are required concepts for building integrations, which usually involve your product accessing data from someone else’s product via an API.
Authentication and authorization often seem one in the same. This article has a much deeper explanation of the differences and relationship between them. For the rest of this article, we’ll group them into the general term “auth”, unless the differences are important.
Auth is hard for a few reasons, even though there are fairly well understood patterns, practices, and standards for doing it right. Those reasons include:
- Those practices can be confusing if you’ve never used them before (we’re looking at you Oauth).
- Auth is very “under the hood” so it’s hard to play around with.
- Not every system implements the standards correctly or even uses a commonly accepted practice for auth.
Mastering auth is mostly a matter of experience. The more types of auth you see and have implemented, the more patterns you familiarize yourself with, and the easier the next one is. However there are some things you can do to level up yourself or your team:
- Learn the Oauth 2.0 standard. It’s increasingly used for product APIs, because it allows a three-way handshake between both products and the end user.
- Understand and play around with token-based and username/password-based authentication mechanisms, which are others you’ll likely see. (The former is a better practice.)
- Learn the basics of cryptographic hashing. For APIs that use unexpected auth mechanisms, basic understanding of securing API credentials helps.
Common API Patterns & Types
To some extent, every API is a unique animal. There will always be some amount of “learning the API” that will be required to build an integration that interacts with it.
Even if you can’t predict exactly how every API will look, you can be prepared by understanding some of the most commonly used API types. This will help you build an integration methodology around familiar API concepts.
Depending on where you look, the stats roughly say that anywhere from 70% to 90% of APIs are implemented as REST APIs. This is far and away the most important API approach to understand, especially if you are building integrations to software-as-a-service (SaaS) products.
To understand REST, you really need to understand HTTP and the request-response cycle that powers the modern Internet. Understanding of that often overlaps with understanding REST, which is simply a pattern for making more predictable requests and responses.
Depending on your industry and its general technology landscape, you may also want to consider learning more about:
- SOAP Web Services (common in ERPs and systems that came up in the early 2000s)
- GraphQL APIs
- Remote Procedure Call (RPC) APIs
- HTTP APIs that don’t follow the REST standard
Do some research on the kinds of APIs you are likely to come across. Dont, for example, invest a bunch of time into mastering SOAP if you don’t have evidence you’ll likely be building integrations to SOAP web services.
..unless you’re into that kind of thing.
Moving Data at High Volumes
One of the things that makes building native product integrations harder than building a one-off integration is the higher data volume requirements.
Totally made up scenario… Let’s pretend all of the potential users of an integration between your product and product X have exactly the same data volume: 1,000 records/day. If you were to build a one-off integration for one of those users, you need to build the integration to handle 1,000 records a day.
If you are building a native product integration that serves the same purpose but for potentially (ideally) many users, what does that look like? 100k records/day? 10 million?
Moving 1,000 records is pretty easy. Moving 10 million requires thoughtful system design to make sure those records move in a timely manner, that as few as possible fall on the floor, and that the ones that do fall can be easily surfaced for debugging.
This usually means data needs to stream asynchronously between systems. There are different underlying technology approaches, but most of them include some form of the data waiting in queues or some kind of holding pattern until processed as workers are available.
At scale, data should usually be processed asynchronously meaning the sender doesn’t get word back whether the receiver ever saw the record. Not having to wait for a response for every record means records can move faster and more fluidly.
But, while faster, the monitoring and exception handling requirements change. Remember, you don’t get a confirmation at the end.
Moving data at high volumes also means you have to consider the capacity limitations of any APIs you’re building to, including your own. Most systems don’t let you freely throw however much data you want at them. At a certain threshold they start to return errors asking you to slow down.
You need to factor into your integration design what API throttling will look like for a given system and what you should do about it. Also consider that some systems aren’t smart enough to tell you to slow down. They simply get overwhelmed and give you misleading errors back.
This quickly gets into a tech conversation (see the next point), but generally speaking, to get better at moving data at high volumes, you or your team can:
- Learn about how webhooks work and how they differentiate from querying an API
- Familiarize yourself with microservices and data streaming platforms
- Learn about queuing technologies, like RabbitMQ
- Familiarize yourself with commonly used API throttling patterns (calls/second, leaky bucket algorithm, etc.)
- Get familiar with various integration technologies–many of which are open source and freely available to experiment with
This is definitely a task that will be owned by the technologists in your product team, but it’s helpful for everyone in the organization to understand the basics of moving data in large volumes.
Specific Integration Technology
Your team should almost certainly not be building integration infrastructure from scratch.
The concepts of communicating with APIs, transforming data structures, moving data asynchronously, and moving it in large volumes are not new. These are problems that, from a technical perspective, have been solved many times before.
It’s extremely rare that your user’s integration requirements are so unique that they cannot be solved by existing technologies. That means, for your product team to get really good at integration, you need to get really good at one (or maybe a few) integration technology(ies) that serve those needs.
How to pick a specific integration technology is a topic for a whole other post, but there are some points worth considering, now:
- Will this be used by engineers or less technical teammates? Should you be looking for a developer platform or a low-code tool? No integration technology is right for everyone.
- How important is it to keep data in your company’s computing environment versus letting someone move data on your behalf within theirs “as a service”?
- Are you open to open source solutions or do you prefer fully commercial solutions?
- How are you thinking about the ROI for the overall integration program? The integration technology you choose will likely be a significant part of the investment you make.
Even if your integration technology is imperfect, and they all are in some way, mastery of that tech will make your product team far more effective at delivering integrations.
Operational Best Practices
Your product team, specifically the engineers, likely have processes in place that help them deliver new features in your product efficiently and safely. Not only will they need to have some mastery of those processes for product features, they will need to understand how to apply them to an integration context.
Integrations are different from typical product features in a few important ways:
- They are features that overlap with those of another product, built by another team, with their own priorities and product roadmap.
- They typically have to process large volumes of data.
- They are features that users tend to struggle to fully understand.
- Yet, they are features that tend to be critical to your role in solving a broader, cross-product problem for your user.
Those realities mean that the operational processes typically used in deploying and managing a software product must be looked at and potentially adapted when managing integrations.
For example, when you release updates to your SaaS product, you may get to roll out updates if and when you see fit. As you roll out those features, users get to take advantage of them.
Rolling out an integration means you need to coordinate with corresponding changes on the other system’s API. It might mean you also need to support past versions for some time, until users can migrate to the new. You don’t always get to just replace what’s there.
While the principles for how your product team deploys and operates your product generally still apply to integrations, the way they must apply them might vary.
Domain Expertise
Integration’s relationship to the business of building and selling software can get lost in that technical conversation, because it gets technical so quickly.
Domain expertise is the understanding of who your product’s users are, what they are trying to accomplish, and generally what their reality looks like. It means understanding your user’s business and where your product fits into it. It also means understanding your own business.
Put differently, domain expertise is the understanding of how your company makes money solving your customers’ problems.
The reality is, your product’s native integrations are simply one part of the way your company makes money selling a service to your customers. A product team with domain expertise can make better decisions about what integrations to build, how to deliver the ones you don’t want to build, and where it all fits into the overall strategy.
A domain expert or a team with domain expertise should have mastery over the following topics:
- Needs and Wants of Users
- Your User’s Business Model
- Your Company’s Business Model
- Your Ecosystem Strategy
- Your Company’s Organizational Structure
Needs and Wants of Users
User’s don’t want integrations. They want cross-product experiences that make them better/faster/etc. at their jobs.
Integrations are simply the means of providing those experiences.
Being really good at building integrations means being effective in producing integrations that add the most value for your users. How can you possibly do that if you don’t know what your user needs, wants, or what’s currently holding them back?
This is just a rehashing of the classic product management mantra to “get out of the office”. To build a product that solves users’ needs, you have to get to know the users intimately. Even if they can’t tell you exactly what they need, they can certainly tell you what unsolved problems they have to inspire you to come up with solutions.
With integrations, the same principles apply, but the scope changes. Now, you aren’t just looking for user insights to look inwardly at your own product and its features. You’re including a potentially vast patchwork of other products that you may (or may not) combine with to solve your users’ problems.
User interviews, surveys, and product analytics–all things you’re probably doing–are still great tools for doing this. Just consider broadening the scope of the questions you’re asking and the activities you’re monitoring for.
Look for the following:
- What is the overall process that your product plays part in?
- What other products or categories of products are playing the other parts?
- Where are users doing manual or repetitive tasks?
- Where are users making mistakes?
- What would give your users superpowers?
Your User’s Business Model
To build integrations that effectively solve your users’ problems, you need to understand their business model.
How do they make money? What keeps them up at night? What dent are they trying to make in the world? What do their margins look like?
This is a classic place where product teams get lost, delivering integrations in ways that aren’t compatible with how the user’s business runs. For example: If your user isn’t accustomed to working with services providers to build custom integrations, you shouldn’t try to sell them an expensive “custom” integration project as part of your onboarding.
Your user’s business model influences your product’s overall pricing. Included in that pricing conversation is understanding how much of the revenue can be carved out to support integrations.
It is possible that you face the challenge of customers with a Pinto budget and Ferrari expectations. This misalignment has a higher risk of bleeding into the integration part of your relationship, because everyone involve tends to be less comfortable with integration.
Understand the way your user makes money. Understand it deeply, so you don’t provide integration solutions that aren’t aligned.
But, to do that, you also have to know yourself quite well.
Your Business Model
It’s really important to factor exactly how you make money. This is relevant when deciding what integrations you build, how you build them, and how your customer experiences those integrations, even if you aren’t directly selling the integrations themselves.
A common challenge that integrations create for software companies is that they can throw off their unit economics. Here’s a classic example:
You sell a software as a service subscription for one year at $800/month meaning the contract’s full value is $9,600.
What if you sell that contract, contingent on building a simple integration? What if that integration goes a little off the rails, taking two people two or three weeks to build? Your internal cost for that customer could be 75% or more of the contract value just to get the integration!
There may be reasons to eat that cost. Maybe you’ll break even or lose money on the first customer, but the integration can be reused for dozens more of those $9,600 customers. Often that ends up not being the case, even if it was intended.
On the other hand, if you were selling to an $8,000/month customer with a $96,000 contract value, those two or three works are probably well worth it.
Hypotheticals aside, the point is that understanding the value a user has to your company and the value your product has to that user are both important. They help you to understand how much margin exists in the revenue to support the integration.
While the unit economics for everyone are different, there are some starting guidelines to work from:
- Larger ACV (average contract value) means the more room there is for complex integrations and/or custom integration projects.
- But, larger ACV also probably means your customer has more complex needs related to integrating your product.
- If you don’t have a professional services part of your customer relationship, your integrations probably need to be user-configurable.
- You should look for 10x the ROI on any integration you build, whether custom or productized. At least question any integrations that can’t obviously provide that.
Bottom line: how you build integrations, what they cost to manage and update, and how or if you monetize them with your customer are all key decisions to be made. If you don’t understand the unit economics of the business you’re running, you can make expensive decisions without realizing it.
Your Ecosystem Strategy
No software product, including yours, lives on its own or solves every problem for your users. You exist in an ecosystem of products with features that overlap with, complement, replace, or differentiate from your product’s features.
Often starting as a simple technology partnership program, your ecosystem strategy–how you plan to leverage this reality for growth–has a huge impact on how you build your integrations and what integrations you’ll build.
Consider the following questions:
- What’s the roadmap for technology partnerships? How does it relate to your product roadmap?
- What integrations are important enough for us to build and own?
- What integrations are important, but you’d like to rely on a partner to build and own?
- Do you centralize and manage all of our integrations or open up a platform for others to build integrations? Do those integrations live in a marketplace?
Developing and adhering to an ecosystem strategy is a big task, but you can start small.
Consider how you prioritize integrations and how you decide to deliver each integration. Work closely with your partnership team to understand the opportunities they see in the market. Consult your sales team about what integrations prospects are asking for.
The most important thing is not to fall into a pattern of building integrations in isolation of these market-facing teams or building integrations as unchecked reactions to customer requests. Having even the start of an intentional ecosystem strategy in place helps to avoid those mistakes.
Your Organizational Structure
Integration is a cross-functional concern for a software company. Consider how each of the following parts of the business are typically involved:
- Engineers: Responsible for building and maintaining the integrations. Measured on things like quality, time to deliver, uptime, and bug resolution, including of integrations.
- Product Leaders: Responsible for defining the product experiences, including cross-product experiences, that help users solve their problems.
- Sales Team: The front line for integration requests, which tend to come up first in sales conversations. Responsible for recording that information.
- Partnership Team: Responsible for the “offline” relationship that comes part and parcel with most integrations. Also a huge source of integration requests.
- Support Team: Responsible for fielding questions and resolving issues for users. Many times this means helping users solve problems with integrations.
- Marketing: Works to position the product in the market, made more effective by being able to articulate how various integrations add value for users.
Your process for delivering integrations, the tools you select to deliver them, which ones you offload to a partner, and who owns the integration once live all depend on how your company is organized. While there are common structures among different software companies, no two org structures are exactly the same.
You have to be intentional about how responsibilities and information flow through the various parts of your organization. That also means understanding who is responsible for taking ownership of integration initiatives.
For example, some companies are very product-led, in which case it may be the product leader’s job is to articulate to the company where integration fits into the overall product strategy. Some are engineering led. Others tend to be more marketing led.
Understanding who needs to do what related to an integration will help you develop processes for delivering integrations that don’t cause friction between teams.
System Expertise
System expertise is simply about understanding your product and its APIs and understanding the product to which you are integrating and its APIs.
Understanding Your Product
Providing a native product integration is usually harder than building a one-off integration. However, in this case, you get somewhat of an advantage building a product integration: one side of the integration is always your product.
It kind of goes without saying that, as a product team, you should have a deep expertise with what your product does and how. If you’re a small team with a single product manager and some engineers, everyone probably does understand it well, but as you scale this can get more difficult.
Growth typically requires specialization and separation of responsibilities within a product team. You eventually have different sub-teams working on different parts of the product.
Sometimes how product responsibilities are distributed has to do with how the technical parts of the product break down (API vs UI vs other). Sometimes they are distributed by functional areas or foundational problems that the product solves. Sometimes, if you’ve grown through acquisitions, you have separate teams for separate products that are being sold together as one solution.
All of these things put a strain on how you deliver integrations to address users’ cross-product needs, because it throttles everyone's understanding of the broader product strategy. Seek to ensure the team responsible for building integrations has everything they need to understand your own product(s) intimately.
Understanding the Integrated Product
Understanding the product you’re integrating to almost certainly increases the likelihood of creating a highly useful product for your user. On its face, this sounds kind of obvious too, but let’s break down some of the reasons why:
- You can know where the bodies are buried. Every product has flaws. If you know where the other product’s are, you can avoid falling victim to them when building your integration.
- This also makes you more equipped to support the integration. A good example is your support team knowing what common API errors mean.
- You can understand how people use the other product more deeply. This includes how users move through the product to accomplish jobs. This is a deeper understanding than just knowing the checklist of available features.
- You can understand the cultural qualities of that product's users. Every product has its own user culture and understanding theirs will help you blend it with yours.
- You can understand where the product is going, so you don’t build something that becomes obsolete when they release new features next quarter.
Understanding the other product requires all the tactical technical stuff: knowing how the API works, knowing how to use the product, etc. You can also build deeper relationships with the teams in that company who build and support the product.
This gives you a direct connection to the people who are just like you, but on the other side. It’ll help you deepen your partnership while deepening the functionality of the integration.
How much time should you invest in learning other products?
The challenge with understanding the other/integrated product is that you have limited time. What if you have 100 products on your potential integration roadmap? Is it really scalable to believe you’ll build deep expertise in all of them?
In most cases, trying to become expert in every product to which you integrate is not a good use of your team’s or your time. Instead, categorize and then approach those categories differently.
For ecosystem imperative integrations (the deepest, most important ones you’ll build), making the investment in learning the other system deeply is worth the effort.
In those cases, your teams will be delivering joint solutions, commonly. Your salespeople will have to field specifics about how the products work together. Your support team will end up supporting the integration, which could be impacted by either product. Invest in doing all of these things well, alongside that tech partner who should reciprocate.
As a general rule, this level of depth should only be afforded to the top 20% of your integrations, at most. Beyond that, you risk scaling your integration expenses too far beyond the potential revenue impact integrations can have.
For less important (but still important) integrations, build best practices around how to integrate into different categories of products.
Maybe Salesforce is an ecosystem imperative integration for which you invest a lot of time, but the long tail of other CRMs are not. Instead, you should develop best practices around what features any CRM will probably support and how those intermingle with your product’s features.
From there, you can define the abstract flows of data between your products to embody those practices. When it comes to building that next CRM integration, you’re starting from a place of likely best practices, based on that definition.
Essentially, you start to think of this second tier of integrations as blanks to be filled in later. Maybe you won’t know the specific details of the next CRM’s API, but you know what you’ll be looking for to fill in to fit that system into your standard CRM integration design.
For less important integrations, the ones that you’re definitely not going to provide natively, your goal is the opposite. Seek to enable the other product team (who may view you as ecosystem imperative) to build and maintain their integration to your product. Focus on developer experience, documentation, and being a supportive partner.
You’ll still play a role in supporting that integration. After all, they are your users too. But, you should try to ride shotgun on integrations that aren’t critical to your ecosystem and growth strategies.
Overvaluing System Expertise
A few words of caution…
System expertise is important, especially expertise with your own product. But, of the three types of expertise described in this post, system expertise is the one that tends to be most overvalued.
When evaluating what it will take for your team to build an integration or evaluating a third party that you may hire to help you, don’t assume that deep system expertise (with the other product) is going to make or break the project. It helps, but most of the time it’s not absolutely required.
In fact, seeking to build individual teams or leverage external ones who only have that deep expertise can really slow down your integration program. It will take far more time to build such knowledge and the program will run far less effectively unless it’s managed extremely well.
There are cases where deep expertise of “the other” system is important. They are less common than most teams consider them to be.
Bringing In Outside Help
As your company and your teams grow, there will be an ongoing effort to improve your expertise in all three of these categories. You should seek to grow those together, never getting too far ahead or behind with any one.
Easier said than done, though!
Building this kind of integration competency takes a lot of work, but you can accelerate that by bringing in outside help. Some common ways this is done are:
- Consultants to help define and execute your ecosystem strategy
- Outsourced development of certain integrations
- Creating a platform where third parties can submit their own integrations (really just a mature, more automated version of the last bullet)
- Partnering with solution partners with expertise in the integrated systems
All of these are valid, commonly used tactics, as long as they are used appropriately.
For most teams, the right answer is someone in the middle of “we’ll do it all ourselves” and “outsource it all”. As your product and your team progress, have a constant conversation about how to use outside help to fill in gaps you identify among the three types of expertise. This means you’ll always use outside help to solve exactly the right problems.
The right partners will really help you to accelerate your ecosystem and go-to-market strategies.
