Staff Software Engineer I

Staff Engineer · Staff · Full Time · Remote

*Job Posting Only: USA1 · RemoteUSD 261k – 307k1mo ago
Apply for this role

Opens Confluent's application page

Role

What you'll do.

Staff Software Engineer I at Confluent will design and implement innovative strategies for table materialization, compaction, and multi-tenant infrastructure as part of the Tableflow project, advancing Kora's multi-modal storage engine. This role requires deep expertise in distributed systems, storage engines, and platform engineering to handle streaming data at massive scale across Confluent Cloud. The ideal candidate brings a strong computer science foundation, open-source contributions (Kafka, Iceberg, Flink), and proven ability to influence stakeholders while architecting highly available, scalable infrastructure.

Responsibilities

  • Storage Engine Architecture & Implementation: Design and implement innovative strategies for table schematization, materialization, and compaction at massive scale across Confluent Cloud. Architect efficient data structures and algorithms that balance performance, cost, and reliability for stream-table duality processing.
  • Multi-Tenant Infrastructure Engineering: Build and maintain highly available, resilient multi-tenant compute infrastructure powering Kora's background computational tasks. Ensure platform optimization for scalability, reliability, and resource efficiency while managing infrastructure across distributed cloud environments.
  • Tableflow Project Leadership: Advance key layers of the Tableflow project, focusing on core components responsible for table materialization and background maintenance. Drive technical decisions and implementation strategies for transforming Kora into a true multi-modal storage engine supporting seamless stream and table consumption.
  • Cross-Functional Technical Collaboration: Partner with product management, design, and engineering teams across the organization to ensure seamless integration of storage features and infrastructure components. Communicate complex technical concepts to non-technical stakeholders and guide architectural decisions that align with organizational objectives.
  • Distributed Systems Problem Solving: Tackle complex distributed storage systems challenges including high availability strategies, scalability patterns, and maintenance efficiency for massive-scale data processing. Develop and implement solutions for materialization and maintenance challenges specific to streaming data platforms.

Qualifications

What we look for.

Technical

  • Distributed Systems Design

    Deep expertise in designing and implementing distributed storage systems, data infrastructure, and multi-tenant architectures capable of handling massive scale data processing with high availability and consistency guarantees.

  • Stream Processing & Data Systems

    Strong understanding of streaming data platforms, stream-table duality concepts, and experience with materializing views or maintaining derived data structures at scale in distributed environments.

  • Database & Storage Engine Fundamentals

    Comprehensive knowledge of database internals, storage engines, table materialization techniques, compaction strategies, and indexing approaches for optimizing query performance and storage efficiency.

  • Cloud Infrastructure & DevOps

    Demonstrated proficiency with cloud platforms (AWS, Azure, or GCP), containerization, orchestration, and infrastructure-as-code practices for building scalable, resilient systems in production environments.

  • Performance Optimization & Scalability

    Proven track record optimizing systems for throughput, latency, and resource efficiency at scale. Experience identifying bottlenecks, designing horizontal scaling strategies, and implementing high-performance data processing pipelines.

Education

  • Computer Science or Related Field

    Bachelor's, Master's, or PhD degree in Computer Science, Computer Engineering, Software Engineering, or equivalent field. Equivalent professional experience in systems engineering or distributed systems development is acceptable as an alternative.

Experience

  • Senior-Level Systems Engineering (8+ years)

    Minimum 8+ years of professional software engineering experience with substantial time spent designing and implementing distributed systems, data infrastructure, or storage solutions. Demonstrated progression to senior technical leadership roles.

  • Storage & Infrastructure Projects

    Proven experience building production storage systems, data platforms, or infrastructure components handling massive scale. Track record of shipping features that improved system performance, reliability, or scalability in distributed environments.

  • Open-Source Contributions

    Active contributions to open-source data systems projects, particularly Apache Kafka, Apache Iceberg, or Apache Flink. Demonstrated understanding of streaming data platforms and their architectural patterns through community involvement.

  • Stakeholder Leadership & Communication

    Experience influencing and guiding technical decisions across multiple teams and organizational levels. Proven ability to communicate complex technical concepts to diverse audiences including product, design, and executive stakeholders.

Skills

Required

  • Distributed Systems Architecture

    Expert-level design and implementation of distributed systems with expertise in consensus protocols, replication strategies, fault tolerance, and consistency models. Ability to architect systems that scale horizontally while maintaining reliability.

  • Java or Scala Programming

    Advanced proficiency in Java or Scala for building production systems. Deep understanding of JVM performance characteristics, garbage collection, memory management, and concurrent programming patterns essential for high-throughput data systems.

  • Data Systems Engineering

    Strong expertise in designing data systems including query optimization, indexing strategies, data serialization formats, and schema evolution. Understanding of columnar formats, compression techniques, and materialization patterns.

  • Database Internals

    Comprehensive knowledge of relational and columnar database engines, query planning, execution optimization, ACID properties, transactions, and distributed query processing. Familiarity with modern data warehouse and lakehouse architectures.

  • Cloud Platform Operations

    Production experience with AWS, Azure, or GCP. Proficiency in services like EC2, RDS, S3, Kubernetes, load balancing, monitoring, and infrastructure automation for deploying and managing large-scale systems.

Preferred

  • Apache Kafka Expertise

    Nice to have

    Deep hands-on experience or open-source contributions to Apache Kafka. Understanding of Kafka's architecture, replication, topic design, and integration patterns for event streaming platforms. Familiarity with Confluent's ecosystem and extensions.

  • Apache Iceberg Knowledge

    Nice to have

    Experience with Apache Iceberg table format, including schema evolution, partitioning strategies, hidden partitions, and time-travel queries. Understanding of how Iceberg solves data reliability and query optimization challenges in data lakes.

  • Apache Flink Proficiency

    Nice to have

    Hands-on experience with Apache Flink for stream processing, including stateful computations, windowing, checkpointing, and distributed state management. Understanding of Flink's execution model and optimization techniques.

  • Stream-Table Duality

    Nice to have

    Conceptual and practical understanding of stream-table duality and how streaming and batch perspectives can be unified. Experience implementing dual-mode systems that provide consistent semantics across stream and batch processing.

  • Performance Benchmarking & Profiling

    Nice to have

    Expertise in profiling distributed systems, identifying performance bottlenecks, and conducting rigorous benchmarking studies. Experience with tools like Java Flight Recorder, flame graphs, and continuous performance monitoring in production.

  • Kubernetes & Container Orchestration

    Nice to have

    Production experience with Kubernetes for deploying, scaling, and managing containerized data systems. Proficiency with helm charts, custom resources, and orchestration patterns for complex multi-tenant architectures.

Tech stack

Languages

JavaScalaSQLPython

Frameworks

Apache KafkaApache IcebergApache FlinkKubernetes

Databases

Apache KafkaApache IcebergPostgreSQLS3 / Cloud Object Storage

Tools

Git / GitHubDockerTerraform / Infrastructure as CodePrometheus / ELK / DatadogApache Maven / Gradle

Other

Distributed Consensus ProtocolsCloud Architecture (AWS/Azure/GCP)Data Serialization FormatsQuery Optimization & Execution Planning

Compensation

Pay and benefits.

Base·USD 261,300 – 307,000

Equity·Stock options

Benefits

  • Comprehensive Health Coverage

    Medical, dental, and vision insurance with plans covering employees, dependents, and qualifying family members. Access to preventive care, specialist networks, and prescription drug coverage.

  • Competitive Retirement Plans

    401(k) retirement savings plan with company matching contributions to help build long-term financial security and retirement readiness.

  • Generous Time Off

    Unlimited paid time off policy allowing flexibility for vacation, personal wellness, and life events. Additional company holidays and paid leave for specific circumstances.

  • Equity & Stock Options

    Participate in company success through stock options and equity grants vesting over time. Opportunity to benefit from company growth as a technical leader.

  • Professional Development

    Learning budgets for conferences, courses, and certifications. Access to mentorship programs, internal technical workshops, and opportunities to present at industry events.

  • Workplace Flexibility

    Remote work options and flexible scheduling to balance work and personal life. Distributed team structure respecting diverse time zones and work preferences.

  • Wellness Programs

    Gym memberships, mental health resources, meditation apps, and wellness initiatives supporting employee physical and mental well-being.

  • Parental & Family Support

    Paid parental leave for primary and secondary caregivers, adoption assistance, and family planning benefits supporting employees at all life stages.

Process

Interview steps.

  1. 01

    Initial Screening Call

    Recruiter conversation to understand background, career trajectory, and motivation for the role. Discussion of technical interests, experience with distributed systems, and alignment with Confluent's culture and mission.

  2. 02

    Technical Phone Interview

    Focused discussion on distributed systems design principles, data platform architecture, and problem-solving approach. Potential discussion of Apache Kafka, Iceberg, or Flink experience. No live coding required at this stage.

  3. 03

    Architecture Design Discussion

    Deep-dive session exploring how you would design key components of the Tableflow system or similar distributed infrastructure challenges. Discussion of tradeoffs, scalability considerations, and production deployment strategies.

  4. 04

    System Design Interview

    Collaborative problem-solving session designing a large-scale distributed system from first principles. Focus on handling massive data volumes, multi-tenancy, high availability, and operational concerns at production scale.

  5. 05

    Stakeholder Impact Conversation

    Discussion with team members about cross-functional collaboration, communication approach, and experience influencing technical decisions. Questions about working with product, design, and other engineering teams.

  6. 06

    Leadership & Culture Fit Discussion

    Conversation with senior leaders or hiring manager exploring technical vision, leadership philosophy, and values alignment with Confluent's collaborative, inclusive culture and commitment to engineering excellence.

Full posting

Original listing.

We’re not just building better tech. We’re rewriting how data moves and what the world can do with it. With Confluent, data doesn’t sit still. Our platform puts information in motion, streaming in near real-time so companies can react faster, build smarter, and deliver experiences as dynamic as the world around them.

It takes a certain kind of person to join this team. Those who ask hard questions, give honest feedback, and show up for each other. No egos, no solo acts. Just smart, curious humans pushing toward something bigger, together.

One Confluent. One Team. One Data Streaming Platform.

About the Role:

We are a team of passionate engineers who love solving complex distributed storage systems and infrastructure problems. Through the Tableflow project, we are building the next generation of storage solutions for Kora, transforming it into a true multi-modal storage engine. Our goal is to enable streaming data to be consumed both as a table and as a stream seamlessly. If you're interested in learning more about the challenges we are tackling, check out our blog post: Tableflow: The Stream/Table, Kafka/Iceberg Duality.

As a Staff Software Engineer I on the team, you will focus on advancing key layers of the Tableflow project, including its multi-tenant, highly available compute infrastructure, and the core components responsible for table materialization and background maintenance. You will tackle challenges such as developing strategies for efficient table materialization and maintenance at massive scale across all of Confluent Cloud, ensuring high availability and scalability for background storage tasks.

What You Will Do:

Storage Engine Development: Design and implement innovative strategies for table schematization, materialization, and compaction at massive scale.

Platform Engineering: Build and maintain multi-tenant, highly available infrastructure that powers background computational tasks for Kora’s long-term storage. Ensure that the platform is optimized for scalability, reliability, and resilience, in alignment with organizational objectives.

Cross-Functional Collaboration: Collaborate with product management, design, and other engineering teams to ensure seamless integration of storage features and infrastructure with the broader organization.

What You Will Bring:

  • BS, MS, or PhD in computer science or a related field, or equivalent work experience

  • Excellent communication and collaboration skills, with the ability to influence and guide stakeholders at all levels.

  • What Gives You an Edge:

  • Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink

  • Experience/knowledge with public clouds (AWS, Azure or GCP)

Ready to build what's next? Let’s get in motion.

Come As You Are

Belonging isn’t a perk here. It’s the baseline. We work across time zones and backgrounds, knowing the best ideas come from different perspectives. And we make space for everyone to lead, grow, and challenge what’s possible.

We’re proud to be an equal opportunity workplace. Employment decisions are based on job-related criteria, without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other classification protected by law.

Privacy Statement

Confluent is an IBM subsidiary which has been acquired by IBM and will be integrated into the IBM organization. By proceeding with this application, you understand that Confluent will share your personal information with other IBM affiliates involved in your recruitment process, wherever these are located. More Information on how IBM protects your personal information, including the safeguards in case of cross-border data transfer, are available here.

Redirects to Confluent's application page.

Other roles

More at Confluent.

View all 21 roles