# Software Engineer, Data Platform (Foundations Lead)
**Company:** [Notion](https://scaleengineer.com/companies/notion)
Software Engineer, Data Platform (Foundations Lead) is a hands-on technical leadership role at Notion, where you'll architect and operate large-scale batch and streaming data infrastructure powering product features, analytics, search, and AI systems. You'll lead multi-quarter initiatives across data lakes, streaming platforms, distributed compute, governance, and reliability systems while building and developing a high-performing engineering team. This position requires deep data platform expertise, proven leadership of infrastructure teams, exceptional talent development capabilities, and the ability to influence across organizational boundaries in Notion's collaborative AI workspace environment.
**Role:** Lead Engineer
**Seniority:** Lead
**Locations:** San Francisco, California
**Salary:** 281000–334000 USD
[Apply](https://jobs.ashbyhq.com/notion/e9bf7ffa-9cf2-4360-a2e4-e9fa66859927)
Canonical: https://scaleengineer.com/jobs/notion/software-engineer-data-platform-foundations-lead
---
## Responsibilities

- Strategic Direction and Vision Setting: Define and articulate multi-quarter technical strategy for Data Foundations team across data lake, streaming infrastructure, distributed compute, governance, and platform reliability. Convert ambiguous company and customer needs into clear architectural directions, sequencing, and success metrics. Align Data Engineering, Data Product, Search, AI, Infrastructure, and Security teams around unified technical vision.
- Enterprise Data Platform Readiness: Lead initiatives to make Notion's data platform enterprise-ready including external key management (EKM), data residency compliance, granular access controls, data lifecycle management, and comprehensive governance frameworks. Design unified approaches to recurring requirements rather than building separate systems for each use case.
- Lakehouse Architecture Evolution: Lead the evolution of Notion's data lake toward modern S3-based lakehouse architecture with Apache Iceberg, implementing stronger governance, schema evolution capabilities, data discoverability, and automated lifecycle policies. Balance innovation with operational stability during large-scale migrations.
- Streaming Infrastructure at Scale: Own scaling and reliability of Kafka and Debezium-based streaming infrastructure powering product features, analytics, and AI systems. Lead capacity planning, performance optimization, disaster recovery improvements, and operational runbook development. Establish service level objectives and ownership models.
- Distributed Compute Optimization: Improve Apache Spark and distributed compute performance, reliability, and developer experience across EMR, Athena, and related platforms. Lead initiatives around workload isolation, cost efficiency, query optimization, and self-serve compute abstractions that increase adoption while reducing operational burden.
- Team Building and Talent Development: Attract strong engineering talent, build psychological safety and clear ownership culture, develop technical leaders for future growth, provide direct feedback and coaching, and establish performance clarity. Create environment where engineers are empowered to make strong technical decisions and grow across career stages.
- Complex Initiative Leadership and De-risking: Lead high-complexity, multi-month technical initiatives from conception through production operation. Establish strong technical ownership, prototype architectures to validate approaches before full commitment, break large work into shippable milestones, and coordinate execution across multiple teams. Navigate ambiguity and technical risk.
- Reliability and Operational Excellence: Identify systemic risks, architectural gaps, and operational pain points in data platform. Lead migrations and modernization efforts to improve reliability, reduce toil, and simplify operations. Establish clear service ownership models, SLO/SLA frameworks, on-call rotations, and incident response processes.
- Platform Abstractions and Developer Experience: Create reusable abstractions, paved paths, and self-serve primitives for internal customers across Data Engineering, Search, AI, and product teams. Design ergonomic platform interfaces, documentation, and tooling that reduce operational burden on consuming teams. Measure adoption and iterate based on feedback.
- Technical Decision-Making and Prototyping: Stay close to critical architectural decisions and systems designs while preserving team ownership. Use prototyping and proof-of-concepts to evaluate build vs. buy vs. migrate decisions. Mentor team on technical judgment, introducing new technologies thoughtfully with clear evaluation criteria and migration plans.
- Cross-Functional Collaboration and Influence: Build strong working relationships with Data Product, Search, AI, Infrastructure, and Security teams. Surface uncomfortable technical and organizational risks early, facilitate architectural discussions across team boundaries, and enable other teams through context, delegation, and feedback.
- Organizational Scaling and Knowledge Multiplication: Scale platform expertise and technical capability across engineering organization through coaching, documentation, design reviews, and direct collaboration. Create feedback loops that raise overall quality of technical decisions. Build systems and processes that don't depend on individual heroes.

## Requirements

### education

- {"name":"Bachelor's Degree in Computer Science or Related Field","description":"Bachelor's degree in Computer Science, Computer Engineering, Mathematics, or related technical discipline. Equivalent practical experience and demonstrated expertise may be considered as alternative qualifications."}

### technical

- {"name":"Large-Scale Batch and Streaming Systems Design","description":"Proven expertise designing, architecting, and operating large-scale batch processing and streaming data systems. Must understand distributed systems concepts including storage, compute, ingestion pipelines, orchestration layers, data serving, governance frameworks, and production operations at enterprise scale."}
- {"name":"Data Lake and Lakehouse Architecture","description":"Deep hands-on experience with modern data lake technologies including S3, Iceberg, Delta Lake, or similar object storage-based architectures. Must demonstrate proficiency with schema evolution, lifecycle management, governance controls, and cost optimization strategies for cloud data platforms."}
- {"name":"Streaming Infrastructure and Real-Time Systems","description":"Extensive experience with Apache Kafka, event streaming platforms, change data capture (CDC) tools like Debezium, and real-time data pipelines. Should have shipped production systems processing high-volume streams and understand operational challenges at scale including monitoring, alerting, and disaster recovery."}
- {"name":"Distributed Compute Frameworks","description":"Strong proficiency with Apache Spark, distributed SQL engines, and compute orchestration platforms. Must understand performance optimization, resource allocation, workload isolation, cost efficiency mechanisms, and developer experience considerations for self-serve compute platforms."}
- {"name":"Data Governance and Security","description":"Hands-on experience implementing data governance frameworks, access controls, external key management (EKM), data residency requirements, compliance controls, and security-first data platform design. Must have navigated enterprise security requirements and regulatory compliance (SOC2, GDPR, etc.)."}

### experience

- {"name":"Data Platform or Infrastructure Leadership","description":"5+ years of experience with 2+ years leading a data platform, infrastructure, or core systems engineering team through multi-quarter technical initiatives. Must demonstrate proven success identifying high-impact problems, defining technical strategy, sequencing execution, and driving architectural decisions with long-term consequences."}
- {"name":"Team Building and Talent Development","description":"Demonstrated track record attracting and recruiting strong engineers, growing people across multiple career stages (junior to senior), developing technical leaders, and creating performance management clarity when needed. Must show comfort with difficult conversations and organizational changes."}
- {"name":"Cross-Functional Collaboration and Influence","description":"Experience influencing technical decisions and strategy across team boundaries with partner teams (search, analytics, AI, product, infrastructure, security). Must excel at surfacing technical and organizational risks early, building consensus around complex decisions, and enabling teams without direct authority."}
- {"name":"Production Systems and Operational Excellence","description":"Hands-on experience operating production data systems including incident response, reliability engineering, observability design, cost optimization, and creating runbooks/documentation. Should understand SLO/SLA frameworks and reliability culture."}
- {"name":"Platform Product Thinking","description":"Experience building internal or external platforms where you've translated partner feedback into self-serve, ergonomic abstractions rather than accumulating bespoke solutions. Should demonstrate understanding of platform APIs, developer experience design, and adoption metrics."}

## Skills

### required

- {"name":"Apache Kafka","description":"Production experience operating Apache Kafka clusters, Kafka connectors, topic design patterns, replication strategies, and scaling considerations for high-throughput streaming workloads."}
- {"name":"Apache Spark","description":"Deep hands-on expertise with Apache Spark including RDDs, DataFrames, SQL optimization, DAG execution, memory management, and performance tuning for batch and streaming applications."}
- {"name":"S3 and Object Storage Architectures","description":"Production experience designing data systems on cloud object storage (S3, GCS, or Azure Blob Storage) with understanding of partitioning strategies, cost optimization, and integration with compute engines."}
- {"name":"Data Governance and Access Control","description":"Hands-on experience implementing data governance frameworks, role-based access controls (RBAC), external key management (EKM), data classification, and compliance controls (SOC2, HIPAA, GDPR)."}
- {"name":"Distributed Systems Design","description":"Strong understanding of distributed systems principles including consensus algorithms, eventual consistency, failure modes, replication strategies, and trade-offs in CAP theorem contexts."}
- {"name":"Technical Leadership and Mentorship","description":"Proven ability to mentor engineers, conduct effective technical design reviews, provide constructive feedback, and help teams make sound architectural decisions. Must demonstrate investment in others' growth."}
- {"name":"Strategic Problem-Solving","description":"Ability to identify leverage points in complex technical systems, decompose ambiguous problems into clear initiatives, define success metrics, and sequence work for maximum impact with constrained resources."}

### preferred

- {"name":"Debezium and Change Data Capture (CDC)","description":"Experience with Debezium, Maxwell, or similar CDC tools for streaming database changes to event systems. Understanding of Postgres/MySQL replication, WAL concepts, and CDC use cases."}
- {"name":"Apache Iceberg or Delta Lake","description":"Production experience with modern open-source table formats including schema evolution, time travel, ACID transactions, and metadata management capabilities."}
- {"name":"AWS Data Services","description":"Hands-on experience with AWS EMR, Athena, Glue, Lambda, or kinesis for building data pipelines and analytics platforms on AWS infrastructure."}
- {"name":"Data Orchestration Platforms","description":"Experience with Airflow, dbt, Dagster, or similar orchestration and transformation frameworks for managing complex multi-stage data pipelines."}
- {"name":"AI/ML Infrastructure","description":"Experience supporting machine learning workloads including feature engineering, experiment tracking infrastructure, MLOps platforms, or AI-powered data systems."}
- {"name":"Cost Optimization and FinOps","description":"Demonstrated expertise optimizing cloud infrastructure costs, implementing cost tracking and allocation frameworks, and designing systems for cost-efficiency without sacrificing reliability."}
- {"name":"Enterprise Architecture and Compliance","description":"Experience designing systems for enterprise requirements including data residency, multi-region deployments, regulatory compliance, and audit controls."}
- {"name":"AI Tools and Collaboration","description":"Intellectual curiosity about AI systems and demonstrated comfort using AI as a collaborative tool for problem-solving, code assistance, and accelerating technical work without treating it as a novelty."}

## Tech stack

### tools

- {"name":"AWS EMR","description":"Managed Hadoop and Spark cluster service for large-scale data processing, distributed computing, and analytics workloads."}
- {"name":"AWS Athena","description":"Serverless SQL query engine for analyzing data directly in S3 without ETL, enabling cost-efficient analytics and data exploration."}
- {"name":"dbt (data build tool)","description":"Modern data transformation framework enabling analytics engineers to work with version control, testing, and documentation for SQL-based transformations."}
- {"name":"Prometheus and Grafana","description":"Observability stack for monitoring, alerting, and visualizing metrics from distributed data platform systems."}

### others

- {"name":"Data Governance and Metadata Management","description":"Systems and practices for managing data catalogs, lineage, access controls, quality metrics, and compliance controls across data platforms."}
- {"name":"Kubernetes and Container Orchestration","description":"Container platforms for deploying and managing data processing workloads, streaming systems, and microservices at scale."}
- {"name":"Infrastructure as Code (Terraform, CloudFormation)","description":"Declarative infrastructure provisioning for repeatable, version-controlled deployment of data platform components and cloud resources."}
- {"name":"CI/CD and GitOps","description":"Continuous integration and deployment practices for data pipelines, infrastructure, and platform code ensuring reliability and auditability."}

### databases

- {"name":"Apache Iceberg","description":"Open-source table format for large-scale analytics providing ACID transactions, schema evolution, time travel, and hidden partitioning for data lakes."}
- {"name":"S3 Object Storage","description":"Cloud-native object storage serving as foundation for data lake architecture with support for partitioning, lifecycle policies, and cost optimization."}
- {"name":"PostgreSQL","description":"Source database system for change data capture and transactional source data in modern data stacks."}

### languages

- {"name":"Scala","description":"Commonly used with Apache Spark for distributed computing and functional programming patterns in data processing pipelines."}
- {"name":"Python","description":"Dominant language for data engineering, data science, and platform development work including orchestration, analytics, and infrastructure as code."}
- {"name":"Java","description":"Foundational language for JVM-based systems including Kafka, Spark, Hadoop ecosystem, and enterprise data platform infrastructure."}
- {"name":"SQL","description":"Essential for data querying, transformation logic, and analytics across data warehouses, data lakes, and SQL-on-object-storage platforms."}

### frameworks

- {"name":"Apache Spark","description":"Distributed batch and streaming processing engine for large-scale data transformations, ETL, and analytics workloads across the data platform."}
- {"name":"Apache Kafka","description":"Event streaming platform powering real-time data pipelines, CDC integration, and streaming analytics infrastructure."}
- {"name":"Debezium","description":"Change data capture framework enabling real-time streaming of database changes from MySQL, Postgres, and other databases into Kafka topics."}
- {"name":"Airflow or Dagster","description":"Workflow orchestration frameworks for scheduling, monitoring, and managing complex multi-stage data pipelines and ETL processes."}

## Benefits

### benefits

## Compensation

- **max:** 334000
- **min:** 281000
- **currency:** USD
- **stockOptions:** true

## Interview process

### steps

## Full description
# Who We Are

Notion is the collaborative AI workspace where teams and agents [think together](https://www.youtube.com/watch?v=vkpYpWfEK5s). We're building one place where your knowledge, projects, meetings, and AI tools live side by side, so work is faster, clearer, and less fragmented. Millions of individuals, small teams, and large companies run their work on Notion.

Notinos (our employees) are customer zero in bringing this future of work to life. We care about craft, building things that last, and the belief that great work is still fundamentally human. Our goal isn’t to ship the next feature. Each and every team of Notinos is working to set the standard for how humans work together in the AI era. From building a business’s system of record to making and managing AI agents to automating away the busy work, we care deeply about giving our customers more time for their life’s work.

# About the Role:

Notion’s Data Foundations team builds and operates the batch and streaming infrastructure behind our product features, analytics, search, and AI experiences. We’re looking for a hands-on technical leader to shape the next generation of this platform as Notion serves larger customers, expands globally, and supports more data-intensive products. You’ll identify the highest-leverage problems, set direction, build and develop a high-performing team, and lead multi-quarter initiatives across our data lake, streaming, distributed-compute, governance, and reliability systems. You’ll stay close to critical technical decisions while creating clear ownership, growing engineers and technical leaders, and helping the team execute as one—partnering closely with Data Engineering, Data Product, Search, AI, Infrastructure, and Security.

This role can be based in either San Francisco or New York City. We work from our offices on Mondays, Tuesdays and Thursdays (our Anchor Days) because we do our best thinking and building together in person. We’re looking for someone who’s excited to work alongside the team during those days. 

# What You'll Achieve:

* **Set the direction for Data Foundations.** Turn ambiguous company and customer needs into a clear multi-quarter strategy across the data lake, streaming, distributed compute, governance, and platform reliability—and align engineers and partner teams around it.
* **Build and develop a high-performing team.** Attract strong engineers, set clear expectations, grow technical leaders, and create an environment of ownership, direct feedback, and accountability.
* **Lead the team’s hardest initiatives from idea to operation.** Ensure complex work has strong technical ownership, de-risk architectures through prototypes, break work into shippable milestones, and coordinate execution across engineers and teams.
* **Make Notion’s data platform enterprise-ready.** Guide the team toward durable foundations for external key management, data residency, access controls, lifecycle management, and governance without creating separate systems for every requirement.
* **Raise reliability, simplicity, and cost efficiency.** Identify systemic risks and architectural gaps, establish clear service ownership, and lead migrations that make Kafka, Debezium, S3/Iceberg, Spark, EMR, Athena, and related systems easier to operate at scale.
* **Multiply the engineering organization.** Establish reusable abstractions and paved paths, delegate meaningful ownership, raise the quality of technical decisions, and scale expertise through coaching, documentation, tooling, design reviews, and direct collaboration.

# Skills You'll Need to Bring: 

* **Deep data-platform expertise.** You have designed and operated large-scale batch and streaming systems and can reason across storage, compute, ingestion, orchestration, serving, governance, and production operations.
* **Direction-setting leadership.** You have led a Data Platform or infrastructure team through multi-quarter work—finding the right problems, defining a strategy, sequencing execution, and driving decisions with long-term consequences.
* **Team-building and talent development.** You have attracted strong engineers, grown people across career stages, developed technical leaders, and created clarity when expectations or performance needed to change.
* **Hands-on engineering judgment.** You stay close to critical systems and designs and know when to prototype, simplify, buy, build, or migrate. You can go deep while preserving ownership for the engineers leading the work.
* **Platform-product thinking.** You understand the needs behind partner requests and turn repeated patterns into safe, ergonomic, self-serve capabilities rather than accumulating bespoke solutions.
* **Organizational influence.** You create clarity across team boundaries, surface uncomfortable technical or organizational risks early, and help engineers do their best work through strong context, delegation, feedback, and coaching.
* You don’t need deep AI expertise for every role, but we do expect every Notino to be intellectually curious, drawn to tinkering and discovery, and excited to use AI as a real collaborator in their work. People who thrive here don’t treat AI as a novelty. They use it to think better, move faster, and build more creatively.

# The Systems and Problems You May Lead

* Evolving Notion’s data lake toward an S3 + Iceberg lakehouse with stronger governance, discoverability, schema evolution, and lifecycle controls
* Scaling Kafka and Debezium-based streaming infrastructure that powers real-time product, analytics, and AI use cases
* Improving Spark and distributed-compute performance, workload isolation, developer experience, and cost efficiency
* Designing EKM, data residency, and compliant processing capabilities across Notion’s data estate
* Creating clear platform interfaces and self-serve primitives for Data Engineering, Search, AI, and product teams

## Why This Role Matters

Notion’s data infrastructure is no longer only an internal analytics layer. It increasingly powers customer-facing features, AI systems, enterprise controls, and high-volume product workflows. The team needs a leader who can see across these systems, build the technical leadership and operating culture required for that responsibility, and find the problems that matter before they become roadmap blockers. Success means Data Foundations has a strong, growing team with clear direction and dependable execution—and teams across Notion can build ambitious data-powered products faster without compromising customer trust.

Notion is committed to providing highly competitive cash compensation, equity, and benefits. The compensation offered for this role will be based on multiple factors such as location, the role’s scope and complexity, and the candidate’s experience and expertise, and may vary from the range provided below. For roles based in San Francisco, the estimated base salary range for this role is $281,000 - $334,000 per year.

By clicking “Submit Application”, I understand and agree that Notion and its affiliates and subsidiaries will collect and process my information in accordance with Notion’s [Global Recruiting Privacy Policy](https://notion.notion.site/Notion-Global-Recruiting-Privacy-Policy-fc3eb4e829354a26a2bb6fd5e289b550)[.](https://dev.notion.so/a3ce416456924276a59de91bca740028?pvs=21)

#LI-Onsite

# A Note on AI

You don’t need deep AI expertise for every role, but we do expect every Notino to be intellectually curious, drawn to tinkering and discovery, and excited to use AI as a real collaborator in their work. For some roles, AI fluency is a core requirement — when that’s the case, we'll say so explicitly in the qualifications. People who thrive here don’t treat AI as a novelty. They use it to think better, and make their work easier for others to build on.

# Equal Opportunity & Accommodations

We hire talented people from a wide range of backgrounds. If you’re excited about this role but don’t meet every bullet, we still encourage you to apply. Notion is an equal opportunity employer and does not discriminate on the basis of any legally protected characteristic. Consistent with applicable law, we will consider for employment qualified applicants with arrest and conviction records. Notion provides reasonable accommodations during the application process; if you need one, please let your recruiter know.

Notion is proud to be an equal opportunity employer. We do not discriminate in hiring or any employment decision based on race, color, religion, national origin, age, sex (including pregnancy, childbirth, or related medical conditions), marital status, ancestry, physical or mental disability, genetic information, veteran status, gender identity or expression, sexual orientation, or other applicable legally protected characteristic. Notion considers qualified applicants with criminal histories, consistent with applicable federal, state and local law. Notion is also committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, please let your recruiter know.