How I layer caches around nodes, indexers, and APIs without lying to users

Introduction

Blockchain backends are slow in all the wrong places.

Nodes are network‑bound. Indexers are I/O‑bound. Price feeds and KYC providers are rate‑limited. On top of that, chains have an awkward mix of properties: most historical data is immutable, but “latest balance” is always moving and “pending” is nowhere near final.

Caching is how you turn that into a responsive product without inventing your own reality.

In this article I’ll walk through how I design multi‑layer caching for blockchain systems using:

  • local in‑process caches in each application instance,
  • a shared Redis layer for cross‑instance caching,
  • simple invalidation strategies that respect on‑chain truth.

Examples come from explorers, DEX backends, and launchpads, but the patterns are chain‑agnostic.

1. Where caching actually sits in a blockchain stack

I mentally draw the stack like this:

[ Browser / dApp ]
        |
    (L0) browser cache / HTTP cache / CDN
        |
[ API gateway / BFF ]
        |
    (L1) in-process cache (per instance)
        |
    (L2) Redis / distributed cache
        |
[ Indexer / DB ]  (Postgres, KV, TSDB)
        |
[ Node / RPC / external APIs ]

There are four obvious places to cache:

  • L0: browser / CDN – static assets, some GET responses.
  • L1: in‑process – tiny hot sets per app instance (hash maps, Guava/Caffeine, @Cacheable).
  • L2: Redis / distributed cache – cross‑instance cache for heavy, shared lookups.
  • Storage‑side – materialised views, pre‑computed aggregates, DB‑side caching.

Architecture guides and cloud docs describe exactly this pattern: keep frequently accessed data close to the application, with a distributed cache like Redis sitting between app and database for heavier reuse.(Microsoft Learn)

For blockchain workloads, the structure is the same; the difference is how we define “fresh enough” for data that comes from a moving chain.

2. What to cache (and what not to)

Not everything deserves a cache entry. I split data into three buckets.

2.1 Immutable data

Blocks by hash, transactions by hash, confirmed logs/events. Once they’re beyond the reorg horizon, they never change.

Examples:

- block 123456 by hash
- tx 0xABC... decoded details
- event log entries for a confirmed transaction

These are ideal for long TTLs or even “no expiry” caches, as long as you’ve decided how many blocks of reorg risk you care about.

2.2 Slow‑changing aggregates

These change, but not on every request:

- address balances and token holdings
- pool reserves, TVL, APRs
- validator statistics, staking summaries
- token metadata and logo URLs

They are expensive to compute from scratch and get hit frequently. They get short TTLs (seconds to minutes) or event‑driven invalidation.

2.3 External / off‑chain data

Anything behind a paid or rate‑limited API:

- fiat and crypto prices
- oracle feeds
- KYC decisions
- payment gateway invoice status

These sit behind tighter TTLs and often small per‑user/per‑API‑key caches to avoid hammering providers. Web3 product guides call this out explicitly: a caching layer in front of external services can be the difference between “works in tests” and “survives production traffic”.(LeewayHertz - AI Development Company)

The goal is simple: cache expensive or repeated reads. Don’t cache everything just because you can.

3. Redis as the shared “L2” cache

Redis is my default L2 cache:

  • in‑memory, low latency;
  • supports replication and clustering;
  • offers flexible data structures (strings, hashes, sorted sets, streams).

Official Redis materials treat it as the standard tool for response caching, object caching, and rate limiting.(Redis)

In blockchain services, I commonly use Redis to:

  • cache explorer responses keyed by (method + path + query) for a few seconds;
  • cache address balance snapshots keyed by (chain, address) for dashboards;
  • cache recent transaction lists for popular addresses or contracts;
  • implement rate limiting (per IP, per API key) using counters and TTLs.(Redis)

Important rule: Redis is a performance accelerator, not a system of record.

- Everything in Redis must be reconstructible from DB + chain + external APIs.
- Redis outages should hurt latency and throughput, not correctness.

As soon as business logic starts treating “whatever is in Redis” as canonical, you’ve crossed a line.

4. Multi‑layer caching: local + Redis + DB

For high‑traffic blockchain APIs, a two‑level cache (local + Redis) pays off quickly:

+----------------------+
|  Application process |
|  (per instance)      |
|                      |
|  [L1] local cache    |  – tiny, ultra-fast
+----------+-----------+
           |
           v
+----------+-----------+
|  [L2] Redis cluster  |  – shared across instances
+----------+-----------+
           |
           v
+----------+-----------+
|   Database / Index   |
+----------------------+

L1 (local cache) Very small working set: the handful of addresses, pools, or tokens that this instance is serving heavily right now. Think “few thousand keys”, eviction by LRU, and short TTLs.

L2 (Redis) Shared across instances, holds a larger working set. It smooths traffic across the entire fleet and survives process restarts.

Multi‑layer caching guides describe this pattern in detail: use local memory for the hottest keys, Redis for shared reuse, and the database as the final source.(Medium)

In blockchain systems, this shines when:

  • a small set of addresses/contracts are extremely hot,
  • you have many app instances behind a load balancer,
  • you need to avoid each instance hitting the database for the same objects.

5. Read/write patterns: how data flows through caches

Underneath “where” you cache, there’s “how” you keep cache and storage in sync.

The usual patterns are: cache‑aside, read‑through, write‑through, write‑behind.(AWS Documentation)

For blockchain services, I almost always default to:

5.1 Cache‑aside (lazy loading)

The application code does:

1) Check cache for key.
2) If hit, return value.
3) If miss:
     - load from DB / indexer / node / provider
     - store in cache with TTL
     - return value

This is ideal for read‑heavy, not hyper‑sensitive data: balance summaries, recent tx lists, token metadata, pool stats.

5.2 Read‑through (library‑managed)

A library or data access layer hides cache vs DB from callers: they just “get X”, and on cache miss the library loads from storage and populates the cache.

This is helpful when you want consistent caching behaviour across many services without re‑implementing the logic everywhere.

5.3 Write‑through / write‑behind

For on‑chain state I avoid these patterns entirely: the only canonical writes happen on the chain and in the indexer derived from it.

For purely off‑chain tables (user accounts, KYC flags, API keys):

  • write‑through can make sense when you want reads to always see fresh data in cache;
  • write‑behind (writes to cache, flushes to DB asynchronously) buys throughput but risks data loss on failure. Redis and AWS docs are clear about the trade‑offs.(Redis, AWS Documentation)

Rule of thumb: for anything derived from the blockchain, cache should mirror storage, not front‑run it.

6. Cache invalidation with blocks and slots

Classic joke: “There are two hard problems in computer science: cache invalidation and naming things.” Blockchains add “and deciding which block you think reality is”.

I use three mechanisms together.

6.1 Time‑based (TTL)

Everything gets a TTL unless there is a very good reason otherwise.

For blockchain:

- immutable items:
    block by hash / tx by hash beyond reorg horizon
    TTL: hours or even days

- "latest" views:
    latest blocks, recent swaps, current validators
    TTL: seconds

- balances and portfolio summaries:
    TTL: tens of seconds, sometimes chain-dependent

Caching best‑practice docs from cloud providers hammer on this: TTLs are guard rails against forgotten invalidations and runaway memory.(Amazon Web Services, Inc.)

6.2 Event‑based

Your indexer already knows exactly what changed when a block is indexed. It can emit events such as:

BALANCE_CHANGED(chain, address)
POOL_UPDATED(chain, pool_id)
TOKEN_METADATA_CHANGED(chain, token_id)

API services subscribe to these (via Kafka, Redis Pub/Sub, or internal queues) and evict or refresh specific keys.

This architecture mirrors “blockchain‑aware caching” research and whitepapers: use chain events to drive precise invalidation while keeping response times low.(PMC)

6.3 Staleness‑aware UX

Sometimes you deliberately accept a little staleness and expose it:

Portfolio (as of block 12,345,678)   [Refresh]

The execution path (trading engine, withdrawal logic) still uses strict, uncached reads. The UI can aggressively cache summaries, as long as it clearly labels their freshness and offers a manual refresh.

This split between “displayed view” and “execution view” is important. It lets you safely cache the former while keeping the latter conservative.

7. Blockchain‑specific pitfalls

A few recurring failure modes.

7.1 Treating pending as final

If you store pending transactions and speculative balances in Redis and treat those as truth, you’ve built an off‑chain ledger with its own double‑spend risks.

This comes up in questions like “can I cache my transactions in Redis and write to the blockchain later?” The answer is: you can, but you’re now running a payment system, with all the failure modes that implies.(Stack Overflow)

For pending tx, it’s fine to:

  • cache UI state (“we submitted tx X, here is its mempool status”),
  • show predicted balances with “including pending” labels.

It is not fine to permanently commit financial decisions based solely on that cached, pre‑confirmation state.

7.2 Hidden cross‑service coupling

If Service A writes some cached representation of a balance and Service B silently trusts it, you’ve created an implicit shared state contract.

I prefer:

  • each service owns its own keys and invalidation logic,
  • shared Redis cluster, but no service reads keys it doesn’t own by convention.

That makes it much easier to evolve services independently.

7.3 No explicit “source of truth”

For each domain you must be able to answer: where is the canonical value?

  • on‑chain state → chain + indexer;
  • KYC → KYC provider + your off‑chain DB;
  • prices → oracle / price providers + your DB.

The cache is never the source. It’s a convenience layer. Once that line blurs, incident analysis becomes painful.

8. What I actually cache in practice

A few concrete patterns that have worked reliably.

8.1 Explorers

I cache:

- "latest blocks" and "latest tx" lists (TTL: 3–10s)
- address activity pages for hot addresses (TTL: 5–30s)
- token transfer lists for popular contracts (TTL: 5–30s)

This is classic cache‑aside with Redis. Source tables are append‑only; TTLs are short; a miss is just a DB query. It offloads a huge amount of repeated “same block range, same filters” traffic.

8.2 DEX / launchpad dashboards

I cache:

- aggregated pool stats, TVL, APRs
- per‑user portfolio snapshots
- sale progress (current raised vs cap)

These combine on‑chain data, external prices, and off‑chain config. I use:

  • TTLs in the tens of seconds, and
  • explicit invalidation triggered by backend events (e.g. “sale moved to next phase”, “new block updated pool state”).

8.3 External API calls

I cache:

- market price snapshots for a short window
- KYC status per user (bounded TTL, usually minutes)
- payment gateway invoice status

Cloud and Redis guidance is unanimous here: caching expensive external calls with short TTLs is one of the simplest ways to cut costs and failure cascades.(Amazon Web Services, Inc.)

9. Observability: hit rates and memory are first‑class

A caching layer you can’t see is a liability.

Metrics I consider mandatory:

- cache hit/miss rate per route or key category
- Redis command latency and error rate
- memory usage and eviction counts
- keyspace size (cardinality per prefix)
- TTL distribution (how much junk are we hoarding?)

Redis monitoring guides emphasise hit rate, latency, and memory/evictions as the core signals of cache health.(Groundcover)

On the application side, I log hit/miss for a few critical endpoints. If an endpoint is 90% miss, the cache isn’t helping; either adjust the strategy (coarser keys, longer TTL) or remove it and simplify.

10. Experience callouts

From the trenches – Redis isn’t a band‑aid. On one indexer we bolted Redis in front of a poorly indexed address_tx table and declared victory. It helped until traffic grew and the keyspace exploded with long‑tail addresses. Redis started thrashing; Postgres was still unhappy. Fixing the schema and adding proper partitioning made a bigger difference than caching. Redis went back to its real job: smoothing spikes, not hiding structural problems.

From the trenches – TTLs as a safety net. We once had a bug where a vesting dashboard never invalidated a “round is ongoing” key. Because TTL was set to 5 minutes “just in case”, the stale view self‑healed instead of lingering for days. AWS and general caching best‑practice docs are right: TTLs aren’t optional; they’re part of the design.(Amazon Web Services, Inc.)

Conclusion

Caching around blockchain systems is less about clever tricks and more about discipline:

  • be explicit about where truth lives (chain, indexer, DB, provider);
  • treat Redis and in‑process caches strictly as accelerators, never as ledgers;
  • use simple, well‑understood patterns like cache‑aside and read‑through;
  • rely on TTLs and, where possible, indexer events to keep caches fresh enough;
  • never let cached pending or off‑chain state masquerade as final on‑chain reality.

If you get those basics right, you can keep explorers, DEX dashboards, launchpads, and analytics responsive without melting nodes and databases every time the market wakes up.