The strategic choices behind data connectivity

Jul 13, 2026

The strategic choices behind data connectivity
‍

“We need Kafka”

It is probably one of the most common phrases heard in IT over the last few years. Even before we know what we are going to build, we already know it will run on Kafka. But here is the catch, when the technology is chosen before the use-case is defined, we risk building solutions which are unnecessarily complex, over-budget and based on an unstable foundation.

To prevent this, those of us in technical roles need to explain the stakeholders the risks and complexities of the technologies and thus consulting them in choosing the best fitted technology rather than the most hyped one. And it is just that conversation which might be the most complex part of the project.

From a technical perspective, we speak a very different language than stakeholders. To overcome this gap, we don’t need complex diagrams with architectural designs, or endless slide decks filled with features. What we do need is a common language where we ask business-focused questions that provide the technical answers we need.

Simply said, we need to move the conversation from “Which tool is the coolest?” to “Which tool delivers the most business value?”

‍

The Pillars of the Translation Layer

‍

I’ve started structuring my technical questions to trigger business awareness with the stakeholders while I can make technical conclusions. So where I ,in the past, would just challenge the decision of the stakeholders with lots of questions, pointing out technical “risks” and “complexities”, I now focus on terms that translate directly into business value: cost, user experience, and continuity

A word of advice: Expect the obvious responses. Stakeholders will always want instant reaction time and 100% reliability. It is our job to dare to challenge those requirements. Example given: Is streaming truly required, or just a nice-to-have with a nice price tag?

‍

Timing & urgency (Latency)

‍

First I always like to focus on user experience This topic is highly critical to stakeholders because it directly impacts customer happiness. Technical insights to be gained are asynchronous vs synchronous.

‍

What you ask the stakeholder What you discover technically The Strategic Insight
“Is there a person actively waiting at a screen for a response to continue their work, or does this happen in the background?” Interaction Model: Request/Response vs. Eventing If a user clicks a button and needs immediate feedback, you need REST/API. If a service runs in the background, MQ or Kafka is likely what you are looking for.
“What happens to the user experience—or our revenue—if this action takes 10 seconds instead of 100 milliseconds?” The “Patience” Threshold & Latency Requirements If a 5-second delay is acceptable, synchronous communication over REST becomes a high risk for timeouts and unnecessary retries.
“If the shipping department's software crashes, should that block a customer from placing a new order on our website?” Consistency & System Dependency If a downstream system is slow or down, the frontend webshop should still be able to take orders. This heavily points toward asynchronous decoupling.

‍

Reliability & guarantees (Durability)

‍

In this section, we shift the topic from “How fast?” to “How safe?”. Technically we are looking for information like persistency, idempotency and write confirmation On the business side, we talk about trust, liability and business continuity.

  • The “Lost record” impact: “What is the actual business cost if we lose a single message?”
    • Insight: If the message is business critical (e.g. Payments, Purchases, Contract signed,…) it will be a huge issue. If the info contains less critical information like metrics, persistence might not be needed.
  • History vs transient: “Do we need to do analytics on past events later on, or do we only care about the latest status?”
    • Insight: By default MQ uses destructive reading while Kafka keeps the data “forever” allowing systems to look back in time.
  • Error handling ownership: “If a transaction fails due to a system error, who is responsible for retrying it—our IT team behind the scenes, or the user clicking 'retry'?”
    • Insight: : If IT must fix it invisibly, you need an asynchronous, persistent buffer. If the customer can just "try again" (like a failed credit card swipe), a synchronous integration with a clear error message provides a better experience.

‍

Volume & growth (scalability)

‍

For resource planning we need knowledge on throughput, retention and sizing. We need to find out how stable the business traffic is, or if we should prepare for massive spikes.

  • Traffic profile: “Is our business volume steady throughout the day, or are we highly sensitive to sudden peaks (like Black Friday or marketing campaigns)?”
    • Insight: Highly volatile peaks require architectural cushioning (like message brokers) to absorb the shock without crashing back-end applications.
  • Load protection: “When our systems are at their absolute busiest, is it acceptable to slow down the customer's interface, or should we accept the work instantly and catch up on processing later?”
    • Insight: This is about Resilience. A REST call forces the customer to wait (or fail) if the back-end is busy. MQ/Kafka allows the business to "accept the order" instantly, protecting the user experience during peak times.

‍

The Strategic Backbone

‍

When designing the topology, we must decide where to put the logic and how to distribute responsibility

  • Service coupling & autonomy: “If we take 'System A' offline for maintenance, should 'System B' be able to keep working as if nothing happened, or is it acceptable that the entire business process pauses?”
    • Insight:   If your business processes are interdependent and require real-time feedback, go for a direct REST/API. If you want autonomous departments that don't want to be bothered by each other's downtime, a Kafka or MQ backbone is the strategic choice.
  • Data Ownership & Availability: “Does every department need to be able to "look back in time" and access data independently, or is the providing department still owner of the data and do the consumers request the latest status?”
    • Insight: The source of truth is centralized and each consumer is responsible to understand the data or the producer remains the owner and provides the correct information on demand.

‍

Conclusion: Integration is not an IT-puzzle, but a business strategy

‍

Selecting the correct integration technology is much more than picking the latest or most popular tool. It is a fundamental strategic decision that balances speed, security, availability, and cost.

There is no one-size-fits-all solution. Every single use case, whether it’s a real-time payment processing system or a quarterly data analytics pipeline, has its own unique requirements. Choosing the wrong “bridge”, creates a costly bottleneck or a reliability risk for the future.

Next time you have the opportunity to advise stakeholders on technology, switch the focus from “which product is better” to “which trade-offs is the business willing to make”. By using these 4 pillars as an example, you ensure that your integration layer is not just a bridge between connections, but a robust backbone that supports the company’s growth.

‍

←
back to Blogs