Grade My Investments Technology Observatory
GMI · TECHNOLOGY OBSERVATORY // ALL SYSTEMS NOMINAL
ENGINEERED BY LEOPARD DATA

The Next Cache Tier

A memory mirror and a Redis stage — fully designed, priced, and deliberately not built. The triggers that will change that are written down in advance.

THE TIERS
rendering diagram…
flowchart LR
    C[Consumers<br/>report engine, page builder,<br/>warmer, macro engine]
    M[RAM mirror<br/>stage 1-3: in-process<br/>stage 4: Azure Redis<br/>NOT YET BUILT]
    DB[(MySQL cache<br/>durable record<br/>metered IO)]
    F[FMP API<br/>$0.005 per call]
    C --> M
    M -- miss or fault --> DB
    DB -- miss or fault --> F
    F -. write back .-> DB
    DB -. read-through fill .-> M
    style M stroke-dasharray: 5 5
Every tier fails toward the one behind it — a memory fault becomes a database read, a database fault becomes a paid provider call. Correctness always survives; only cost changes.

An Optimization We Designed and Then Refused

The platform's provider cache stores market data in MySQL so a filed fiscal quarter is bought once, not once per reader. But a cache hit still costs a database round trip — and on this database, storage IO is metered, so trips are literally billed. The obvious next move is a tier in front: the cache's working set in process memory, where reads cost nothing.

We designed it, priced it, and did not build it — because the load model said not to. A memory layer only pays when the same data is read repeatedly within a short window, and every reader of this cache is a batch process that touches each symbol once per run. The page builder sweeps 2,516 tickers — thousands of different rows, once each. User dashboards never touch this cache at all; they read pre-computed artifacts from blob storage. First touches dominate, and a desk in front of a filing cabinet does nothing for first touches.

The interesting part is what that refusal produces: a design document with trigger conditions instead of a deployment. The thresholds that would change the answer — report volume, measured latency, the IO meter doubling, any user-facing feature reading the cache per page view — are written down now, while the reasoning is fresh, so the build decision later is a lookup rather than a debate.

The Staircase: Four Stages, Each Abandonable

StageWhat it addsCost
1. Watermarks + statements in RAM The ~28 database trips a warm symbol still makes collapse to one or two. Settled statements are immutable by construction, so a mirrored copy can never be stale. ~230 MB of process memory. Free.
2. A long-lived report worker The report generator is currently spawned per job — a per-job process gets a cold mirror every time, which is the real architectural constraint. A long-lived worker keeps its warmth across jobs. An architecture change, not a purchase.
3. Full mirror including prices ~731 MB of cache data (measured) becomes ~1.1–1.5 GB resident. Only if measurement shows the price reads matter. RAM headroom on a 3.9 GB host — a sizing decision, not an afterthought.
4. Azure Cache for Redis The mirror moves out of process and becomes shared: every consumer starts warm, warmth survives the many-times-daily deploys, and MySQL is relieved of nearly all cache reads. ~$40–80/month, always on — a 25–50% increase on the entire current cloud bill.

Stage 4 is not a rival to stages 1–3 — it is the floor above them. Building it without having outgrown the in-process mirror is renting a warehouse to store one filing cabinet.

The Redis Stage, Sold Honestly

Redis is usually pitched on speed. Here that pitch would be wrong, and the design says so: Redis over the virtual network is a ~1 ms hop against local MySQL's ~1–3 ms — modestly faster, while in-process memory beats both by three orders of magnitude. Anyone selling this stage on latency has mis-read it.

What Redis actually buys is shared warmth: one mirror that every process reads, that a freshly spawned per-job worker inherits instead of rebuilding, and that survives deployment restarts on a platform that ships many times a day. Plus the quiet retirement of two ceilings at once — metered database IO, and the database connection budget.

The design names four conditions and requires two before the ~$40–80 monthly rent is justified: simultaneous hot readers across processes, deploy frequency defeating in-process warmth, the database ceilings actually being approached, and the cost being small against revenue. At a thousand users — where the provider cache alone saves more per month than Redis costs — all four are plausibly true. Today, none are.

What Never Changes, Whatever Gets Built

  • MySQL stays the only source of truth. Memory and Redis are read-path conveniences, rebuilt from the database at will and never written ahead of it. No write-behind — the first crash would prove whose data was real.
  • Every tier fails toward the one behind it. A mirror fault is a database read; a database fault is a paid provider call. Failure degrades to cost, never to a wrong answer — the property that has already saved this system three times.
  • The immutability rules do the hard work. The provider cache's settlement design established which data can never change; a cache tier above it inherits that for free. Consistency is easy here because correctness was designed one layer down.
  • The success condition may be never building it. The cheapest tier is the one you never had to run. If no trigger fires, the design document was still worth writing — it is the reasoning, preserved.

Related Reading