"How do you organize a product integration team?" is a common question from software teams.
That question is becoming more common as the partnership team emerges as a primary function of a software business. It's also becoming more critical, because an API integration strategy is now non-negotiable for a software vendor.
This post will dive into how to structure a product integration team. While no two companies are exactly alike, this should give the appropriate guidance for doing so at any point in your company's growth.
This is not an Agile software development blog post, but a concept prevalent in Agile is helpful when thinking about your integration team: Agile pods.
This definition from Agile Connection explains the term well:
Agile pods are small custom agile teams, ranging from four to eight members, responsible for a single task, requirement, or part of the backlog. This organizational system is a step toward realizing the maximum potential of agile teams by involving members of different expertise and specialization, giving complete ownership and freedom, and expecting the best quality output.
The key takeaways relevant to this post are:
Structuring "pods" to deal with product integrations effectively addresses integration's particular challenges. For most software teams, most/all of the following are true:
The Agile pod concept is the basis for a successful product integration team, even if it doesn't literally mean you run Agile Scrum as a project management methodology. The pod addresses the complexities that are especially acute in integration projects by providing the right mix of domain specialization, diversity of skill, and autonomy to address a problem.
But, who should be part of these pods?
Scaling a successful product integration team is about understanding the roles that are required on the team, then understanding how to organize those roles to be successful. So, first, we'll dive into what those roles are.
The term "role" is being used very specifically, because depending the size of your team and the amount of resources you invest, these roles may be shared among many people. Early on, one or more of these roles may just be handled as one part of someone's overall job. (Hopefully, not for long.)
The role's function is more important than the job title, although these will likely converge as your team grows and your team members specialize into these roles.
In the simplest terms, a successful product integration team contains three roles:
These three roles work together, bringing complementary and somewhat overlapping skills to the table. These three roles are optimized to address the specific complexities an integration project requires.
The three roles on an integration team should be specialized to their tasks (see the following descriptions). However, you should try to respect an 80-20 rule of specialization.
80% of what a person in any of these roles brings to the table should be specialized to that role. However, 20% should overlap with that of the others.
The analyst and lead should have a basic understanding of the technology used to build integrations. The technologist and analyst should have a basic understanding of the lead's strategy for managing stakeholder expectations. The lead and technologist should have a basic understanding of thee analyst's requirements methodology.
A successful integration team, like a successful Agile pod, is one where all members share responsibility for success, even as they primarily bring their unique and most valuable skills to the table.
The Integration Lead's role is to coordinate and facilitate the success of integration projects that a pod takes on. This role is critical because it lets the technologist focus on the technology and the analyst focus on translating actual requirements between the technologist and stakeholders. The lead absorbs the disruptive overhead.
Again, actually executing Scrum (or any Agile framework) is not required, but this role in many ways is like the Scrum Master role. They remove barriers, coordinate work, and facilitate the team's progress.
This role does have an aspect of project management to it, but the focus should be enablement of progress over administrative project management. Both are necessary to some extent, but the former is more important.
It's also recommended that this role not be the formal manager of other pod members. Their "authority" as a lead should be that of facilitating success of the integration pod, not promotions, compensation, and skills development.
This role should also play a heavy hand in stakeholder collaboration.
Ideally an integration project includes cooperation from the customers or partners who will benefit from the integration. But, those people tend to need some wrangling. The integration project will not be the stakeholder's primary focus.
The integration lead adds a lot of value to the team by shielding the rest of the team members from those distractions, keeping stakeholders on track.
*****
Success with a product integration program requires two fundamentals: best practice technology on which you deploy your integrations and best practice process to triage, build, and manage them. You must have both.
The next two roles are the people that execute on both of those fundamentals, respectively.
The integration technologist's role is to own the integration technology that is deployed and leverage it to deploy integrations. They are the hands-on builders, wiring up APIs, data transformations, and integration flows.
The technologist should have a strong understanding of APIs, the mechanics of how data moves, and certainly how to use the integration technology that's in place. If you use an integration platform-as-a-service (iPaaS) to facilitate your integrations, they should be experts in that system.
Breadth of the right technology expertise makes this person successful. Broad experience with different types of APIs, including older, less common types, is important. Understanding of data structures like flat files, JSON, XML, etc. is important. Understanding programming-like languages for data transformation, like JSONata, JSON Path, or XSLT, are also helpful.
Often, this person has a background as a software engineer/programmer/developer or some comparable role and they've migrated from that to something less technical. You'll also see people with an MIS (management information systems) or like background who grows their technical skills be successful in this role.
This person needs to understand how data moves across the internet in small and large quantities, because they will be assembling the parts that make it so. They are the mechanics. They are the data plumbers.
The technologist is one key role. It is not the only key role. Many integration efforts fall flat, because leadership sees integration as purely a technical problem.
All software developers are not equipped to be a successful integration technologist. Likewise, many successful integration technologists don't have the programming chops to actually build software.
These are two fundamentally different roles with different, albeit occasionally overlapping, requirements of the person filling the role. "Integration is technical therefore give it to the developer to solve," is a trap. This approach does not maximize your probability for success.
That said, there is nothing wrong with software developers acting as integration technologists, if they have or are given the tools they need to learn this specialized role.
The integration analyst is the liaison between stakeholders and the integration technologist. To understand this role's importance, consider what a product integration is really intended to do:
A product integration should allow your software product to share data with another software product so that an end user (of both products) has an improved inter-product experience. The stakeholder becomes better/faster/less expensive/more accurate/whatever at some job that they have to accomplish that involves both software products when they are integrated.
The product integration must also consider that there isn't one end user. There is a population of stakeholders who have similar requests of the integration, but with some amount of variation.
Remember: the stakeholder does not want an integration. They want an improved inter-product experience.
The way they describe the problem (or opportunity) behind the integration will be fundamentally different than how you or your team would. You are too inside of the problem. And, the reality is:
The integration analyst's job is to serve the disconnect between the technologist (the person who needs to build the integration) and the stakeholders (the people who need it to deliver value). They are the translator. And, often times, they are the negotiator, when stakeholder wants and technology, budget, or timeline realities do not align.
Someone with an MIS, business analyst, business process, or solution engineering background is a great fit for such a role. Written and oral communication are very important. So are high levels of emotional intelligence.
This person must be able to speak to the technologist in the way they need to be spoken to and the stakeholder the way they need to be spoken to. Those two communication styles can be very different.
It usually helps if someone in this role also has domain expertise. Its most important they have domain expertise related to your product and the customer need it serves. (That should be a given.) It's helpful if they also have domain expertise in the type of products they'll be integrating to.
The analyst should have support from the lead when interacting with stakeholders, but ultimately the analyst must take on the burden of stakeholder value for the integration team. They are the "voice of the customer."
Knowing the three roles you want on an integration team is important. Actually organizing the people filling these roles in an effective manner is the other piece to this puzzle.
We'll share some thoughts on how to do that, which are by no means prescriptive. Every company has its own organizational realities, but starting with the following as a baseline will help you build a better integration team within those realities.
Your team should have at least people playing these three roles. Ideally that's three different people, but someone may have to play a role part time. You also may have to assign someone to do double duty (analyst + technologist). This is hard to do well and it sacrifices diversity of background. Avoid this if you can.
The most important advice here is to not avoid a role because of resource or time constraints. All three roles are important. Leaving one out is a fast train to an unsuccessful integration effort.
You likely have a bigger integration backlog than you'll ever completely address. Three people probably doesn't get this done for long.
As long as you keep the pod size to no larger than 8 or so people, you can add analysts and engineers as required.
You may add them in a 1:1 ratio, but you don't have to. Some teams must be more analysis heavy, some more technology heavy.
Typically within a pod, a single lead is sufficient, though you may reduce your maximum pod size to only 5 or 6 if leads must play a heavier than average role per integration project. A pod will take on more than one project at a time. It shouldn't be a number higher than what a single lead can manage.
There may be reason to involve others in your organization who aren't core members of the integration pod, but who are valuable when it comes to designing and deploying product integrations. Examples can include:
These extra pod members are probably involved part time and should contribute to, but not take responsibility for the pod's deliverables. The integration lead should manage these relationships.
It's also important to keep team sizes down. With some outside help, maybe you flex up to 8 to 10 people, but any more than that is probably too big.
Eventually the velocity at which you need to deploy integrations may go beyond what a single integration pod can deliver. At this point, you need to start scaling to multiple pods.
Scaling integration pods is very like scaling an Agile software team. You have multiple small groups coordinated at the "lead" level (called a Scrum of Scrums in Agile terms).
Nothing changes about how each individual pod operates. They still have ownership over their assigned problems and authority to solve them. They should include diverse backgrounds. The three roles should still be present.
However, it is difficult to coordinate multiple pods without a single person with that responsibility. As you scale to two pods, and definitely three and beyond, you should consider a formal role as "head of integration". Whether that's a VP, Director, or Manager role is up to you, and it depends on your business. Whether that role reports to the CTO, the head of product, or the head of partnerships is a topic for a future post.
The head of integration should own accountability for all integration projects. They should deliver integrations to serve the other parts of the organization who require them: customer success/implementation, sales, partnerships, etc.
While the analysts and technologists shouldn't probably report directly to their pod leads, the leads should report directly to the head of integration. This will help with standardization, shared knowledge, and coordination.
Scaling integration pods horizontally will open up a new set of questions about how you assign people and responsibilities to each pod. There is no single right answer. The only wrong answer is to go at it haphazardly. Consider the following ways to organize your pods:
You could also consider rotating team members through pods to keep things mixed up. This can create some additional project delivery challenges, but it'll help spread expertise throughout your integration team.
It's important to recognize that your product integration program is part engineering, but it's not an engineering task. It's part partnerships, but it's not solely a partnership task. It's part product, but it's not just product.
If your growth is dependent on integration, you don't have a choice but to get good at it--even if it isn't your core business. That means investing in dedicated resources for integration and organizing them for success.
Even if you can't afford or aren't ready for a large integration team today, understanding what this looks like as it grows will help you start where you are. Don't hand it all off to someone else.
