Platform · 2011–now · Cloud Scale

Apache Kafka

Durable log as nervous system. Event-driven architecture with real operational teeth.

Kafka stuck because ordered, replayable logs solve integration without brittle point-to-point RPC. Ops cost is real but predictable compared to ESB fantasy. It failed only when teams treated topics as a database.

streaming events distributed

Case studies

Twitter outgrows pure Rails — without declaring Rails dead — Years of JVM/Scala investment alongside Ruby; Fail Whale era ops pain, not a total rewrite bill

Patterns

Cargo-cult hyperscale

Context

Split everything, then pay for the glue

Hyperscale patterns escaped the companies that needed them. Containers unified packaging; Kubernetes became the cloud OS; microservices and NoSQL were sold as defaults. Mobile-first stuck because screens changed. Cargo-cult distributed systems stuck around as invoices. The durable move was packaging and ops maturity — not rewriting every app into a mesh.

Compare with

vs Microservices Everywhere vs Redis vs Apache Spark vs Unix / POSIX

Architecture · 2012–now

Microservices Everywhere

Costly fad

Independent deployability as religion. Correct for some; ruinous cargo-cult for most.

$ Teams split 4-person products into 40 services, then hired SREs to babysit the mesh. Many later consolidated — after years of distributed tracing invoices and on-call trauma. See case studies on premature microservices and big-bang cutovers for named failure modes.

Platform · 2009–now

Redis

Stuck

Fast structures in memory with persistence options. Cache, queue, session store — pick your footgun.

Platform · 2014–now

Apache Spark