Platform · 2014–now · Cloud Scale
Apache Spark
In-memory data processing that made Hadoop feel overnight. Still the batch/stream workhorse under many lakehouses.
Why Apache Spark stuck
Spark stuck by meeting analysts where Python and SQL already lived while beating MapReduce ergonomics. It is not magic — shuffle still hurts — but it ships pipelines. The fad was calling every CSV a lake; Spark survived the hangover.
Compare with
Related
Platform · 2006–2018
Hadoop Everywhere
MapReduce as lifestyle. The elephant in rooms that needed a spreadsheet.
$ Enterprises stood up Hadoop clusters for gigabytes of data that fit on one Postgres instance. Hadoop admins, ZooKeeper nightmares, and ETL rewrites burned millions before Spark and cloud warehouses retired the elephant.
Language · 2004–now
Scala
JVM language with functional teeth. Spark and Kafka culture kept it relevant long after the Twitter hype cooled.
Language · 1991–now
Python
From scripting glue to the default language of data and AI. Readability as a growth strategy.
Platform · 2011–now
Apache Kafka
Durable log as nervous system. Event-driven architecture with real operational teeth.