Who Is Hiring? Search
open the tool →

Who Is Hiring? Search › Top 5 data engineering

Which Data Engineering Skills Are Most in Demand? (Kafka, Spark, Airflow, Snowflake)

The data-pipeline stack companies hire for in Hacker News 'Who is hiring?' posts: Kafka, Spark, Hadoop, Airflow and Snowflake, charted by share of postings since 2011.

Explore Top 5 data engineering in the Who Is Hiring? tool

Ranked by demand (all-time)

  1. 1.Kafka2,192 postings · peak Aug 2021
  2. 2.Spark2,104 postings · peak Jan 2017
  3. 3.Hadoop1,342 postings · peak Aug 2015
  4. 4.Airflow489 postings · peak May 2021
  5. 5.Snowflake353 postings · peak Apr 2022
tip: use | to OR several terms into one bar, e.g. backend|sre|devops
charting…

Job postings

Hover (or click) a bar segment to read the job postings that mention that term in that month.

Each calendar month since 2011 as one bar. Switch to share % to stack the bands to 100% and see each one’s slice of the category’s postings, narrow the window, or click a month to read the postings behind the bar.

What the chart shows

The data-engineering chart tracks how the pipeline stack modernized. Hadoop and Spark lead the early big-data years; Kafka becomes the near-default streaming backbone; Airflow rises as orchestration standardizes; Snowflake climbs as the cloud-warehouse era arrives and Hadoop fades. Read the relative bands as a timeline of how teams moved data, and the sample postings to see which tools cluster together - Kafka and Spark on the streaming side, Airflow and Snowflake on the warehouse side.

Popular Kafka postings (the category leader)

  1. lars_francke · read the thread · view on HN

    Stackable | Product Engineer - Web & UI | Remote (Germany UK) | ~20 people | Full or part-time We build an open source data platform on Kubernetes. The product (Stackable Data Platform - SDP) bundles 12+ open source data tools (Trino, Apache Kafka, Apache Airflow, Apache…

  2. nicksnyder · read the thread · view on HN

    Buf | Software Engineer, Engineering Manager | Remote (US or Canada) | Full time | https: buf.build Buf is building a new paradigm for APIs. Our open source and commercial software helps companies adopt Protobuf schemas throughout their stack, from gRPC APIs to streaming data in…

Each skill on its own

Build your own comparison on the full Who Is Hiring? chart, or see how these terms trend across all of Hacker News.