Platform · 2006–2018 · Cloud Scale
Hadoop Everywhere
MapReduce as lifestyle. The elephant in rooms that needed a spreadsheet.
Hadoop failed as a default because most data problems are warehouse-sized, not Google-sized. Clusters became resume-driven infrastructure with 3am pager duty. Spark and cloud SQL absorbed the real workloads; Hadoop became a cautionary sticker.
Cost of the fad
Enterprises stood up Hadoop clusters for gigabytes of data that fit on one Postgres instance. Hadoop admins, ZooKeeper nightmares, and ETL rewrites burned millions before Spark and cloud warehouses retired the elephant.
Patterns
Context
Split everything, then pay for the glue
Hyperscale patterns escaped the companies that needed them. Containers unified packaging; Kubernetes became the cloud OS; microservices and NoSQL were sold as defaults. Mobile-first stuck because screens changed. Cargo-cult distributed systems stuck around as invoices. The durable move was packaging and ops maturity — not rewriting every app into a mesh.
Compare with
Related
Platform · 2014–now
Apache Spark
In-memory data processing that made Hadoop feel overnight. Still the batch/stream workhorse under many lakehouses.
Practice · 2009–2019
NoSQL for Everything
Schema-optional databases sold as a lifestyle. Right tool for some jobs; default for none.
$ Startups ditched Postgres for document stores "because scale," then rebuilt relational integrity in application code — and paid twice when joins came back as a product requirement.
Platform · 2010–now
Elasticsearch
Full-text search and log analytics at scale. Powerful, hungry, and everywhere in observability stacks.