Designing a Cache Layer

A cache in front of a datastore improves read throughput and latency. Key considerations:

• Eviction policy: LRU is common; LFU for frequency-based workloads.
• TTL vs explicit invalidation: TTL is simple; explicit invalidation required for strong consistency.
• Write patterns: write-through vs write-back.

Trade-offs:

• Higher hit ratio reduces DB load but increases stale data risk.
• Sharding and distributed caches (Redis cluster) add operational complexity.

Mini checklist:

1. Choose eviction policy (LRU default).
2. Decide TTL vs invalidation strategy.
3. Measure hit ratio and tail latency.

Code (pseudo):

1
# naive in-memory LRU using OrderedDict
2
from collections import OrderedDict
3

4
class LRUCache:
5
  def __init__(self, capacity):
6
    self.cap = capacity
7
    self.d = OrderedDict()
8

9
  def get(self, k):
10
    if k not in self.d: return None
11
    self.d.move_to_end(k)
12
    return self.d[k]
13

14
  def put(self, k, v):
15
    self.d[k] = v
16
    self.d.move_to_end(k)
17
    if len(self.d) > self.cap:
18
      self.d.popitem(last=False)