Rohit Saini Making code go brrr (but predictably).

Designing for Scale

March 12, 2026 · 6 min read

Modern systems rarely fail because of a single large mistake. They fail because of small assumptions that compound.

A field that’s always populated — until it isn’t. A service that responds in 50ms — until traffic doubles. A queue that never backs up — until a downstream dependency has a bad day.

When designing for scale, the first instinct is usually to reach for more infrastructure. More caches. More queues. More services. More replicas.

That instinct is wrong about 80% of the time.

Scale Is About Clarity

Over the last few years, I’ve come to believe that scale is mostly about clarity. Not clever algorithms. Not exotic databases. Just clarity about four things.

1. Clarity of Data Flow

If you can’t draw how data moves through your system on a napkin, you’re going to have a bad time scaling it.

Here’s a real scenario. We had a service that called three downstream APIs sequentially:

Request → Service A → Service B → Service C → Response

Total latency: sum of all three. At low traffic, this was fine — maybe 200ms total. At high traffic, one slow response from Service C would back up everything upstream.

The fix wasn’t adding more instances of our service. It was restructuring the data flow so that we only needed Service A’s response synchronously. Services B and C could be handled asynchronously via events:

Request → Service A → Response (sync, fast)
              ↓
         Kafka Event
          ↓       ↓
     Service B   Service C (async, eventually consistent)

Same outcome. Dramatically different scaling characteristics.

2. Clarity of Ownership

One of the fastest ways to create scaling problems is shared ownership. Two teams writing to the same database table. Three services all maintaining their own copy of user preferences. Five consumers all computing the same aggregation differently.

I follow a simple rule: every piece of data has exactly one owner. The owner is the only service that writes to it. Everyone else reads, caches, or subscribes to change events.

This sounds obvious, but in practice, it’s the first thing that gets violated when teams are moving fast. “We’ll just add a column to their table, it’s faster.” Three months later, you have two services with competing opinions about what a user’s status should be.

3. Clarity of Failure Handling

At small scale, you can get away with “it probably won’t fail.” At large scale, everything that can fail will fail, and probably all at the same time on Friday evening.

The question isn’t “will this fail?” but “when this fails, what happens?”

Some patterns that have saved me:

  • Circuit breakers on every external call. When a dependency goes down, fail fast instead of queueing up thousands of requests.
  • Timeouts everywhere. Not just HTTP timeouts. Database query timeouts. Kafka consumer processing timeouts. Redis connection timeouts. I’ve seen systems grind to a halt because a single missing timeout let a slow query hold a connection pool hostage.
  • Dead-letter queues for every consumer. When a message can’t be processed after N retries, park it somewhere you can inspect later instead of blocking the entire queue.
# A timeout that actually works
import signal

class TimeoutError(Exception):
    pass

def timeout_handler(signum, frame):
    raise TimeoutError("Operation timed out")

def with_timeout(fn, seconds=5):
    signal.signal(signal.SIGALRM, timeout_handler)
    signal.alarm(seconds)
    try:
        return fn()
    finally:
        signal.alarm(0)

4. Clarity of Observability

You can’t scale what you can’t see.

I’ve worked on systems where the only “monitoring” was someone checking if the API returned 200. Green meant good, right? Wrong. The API was returning 200 while silently dropping 30% of messages because a Kafka producer was failing and the error was being swallowed.

Good observability at scale means:

  • Metrics that matter. Not just CPU and memory. Business metrics. Messages processed per second. P99 latency per endpoint. Queue depth over time. Error rate by type.
  • Structured logging. Key-value pairs, not strings. When you’re searching through millions of log lines, {"thread_id": "abc", "action": "send", "latency_ms": 342} is infinitely more useful than Sent message to thread abc in 342ms.
  • Traces that connect the dots. When a request touches five services, a correlation ID that flows through all of them is the difference between “something is slow” and “the Elasticsearch query in Service C is slow because the index isn’t optimized for this access pattern.”

The Trap of Premature Scaling

The biggest trap I see: designing for scale you don’t have yet. Adding Kafka when you have 100 requests per minute. Splitting into microservices when your team is four people. Introducing a distributed cache before profiling where the actual bottleneck is.

Every scaling mechanism you add is also a complexity mechanism. A cache means you now need cache invalidation strategy. A message queue means you need idempotency guarantees. Microservices mean you need service discovery, distributed tracing, and a deployment pipeline that doesn’t take 45 minutes.

Start simple. Measure. Scale the parts that are actually slow.

The One Rule

If I had to condense everything into one rule, it would be this:

The simplest architecture that clearly defines boundaries will almost always outperform a complex one that doesn’t.

Scaling is less about adding components and more about removing ambiguity. When every engineer on the team can look at the system and immediately understand where data flows, who owns what, how failures are handled, and where to look when things go wrong — that system is ready to scale.

Not because the infrastructure is impressive. Because the thinking is clear.

← Back to Writings