Staff Software Engineer I
Staff Engineer · Staff · Full Time · Remote
Opens Confluent's application page
Role
What you'll do.
Staff Software Engineer I at Confluent will design and implement innovative strategies for table materialization, compaction, and multi-tenant infrastructure as part of the Tableflow project, advancing Kora's multi-modal storage engine. This role requires deep expertise in distributed systems, storage engines, and platform engineering to handle streaming data at massive scale across Confluent Cloud. The ideal candidate brings a strong computer science foundation, open-source contributions (Kafka, Iceberg, Flink), and proven ability to influence stakeholders while architecting highly available, scalable infrastructure.
Responsibilities
- Storage Engine Architecture & Implementation: Design and implement innovative strategies for table schematization, materialization, and compaction at massive scale across Confluent Cloud. Architect efficient data structures and algorithms that balance performance, cost, and reliability for stream-table duality processing.
- Multi-Tenant Infrastructure Engineering: Build and maintain highly available, resilient multi-tenant compute infrastructure powering Kora's background computational tasks. Ensure platform optimization for scalability, reliability, and resource efficiency while managing infrastructure across distributed cloud environments.
- Tableflow Project Leadership: Advance key layers of the Tableflow project, focusing on core components responsible for table materialization and background maintenance. Drive technical decisions and implementation strategies for transforming Kora into a true multi-modal storage engine supporting seamless stream and table consumption.
- Cross-Functional Technical Collaboration: Partner with product management, design, and engineering teams across the organization to ensure seamless integration of storage features and infrastructure components. Communicate complex technical concepts to non-technical stakeholders and guide architectural decisions that align with organizational objectives.
- Distributed Systems Problem Solving: Tackle complex distributed storage systems challenges including high availability strategies, scalability patterns, and maintenance efficiency for massive-scale data processing. Develop and implement solutions for materialization and maintenance challenges specific to streaming data platforms.
Qualifications
What we look for.
Technical
Distributed Systems Design
Deep expertise in designing and implementing distributed storage systems, data infrastructure, and multi-tenant architectures capable of handling massive scale data processing with high availability and consistency guarantees.
Stream Processing & Data Systems
Strong understanding of streaming data platforms, stream-table duality concepts, and experience with materializing views or maintaining derived data structures at scale in distributed environments.
Database & Storage Engine Fundamentals
Comprehensive knowledge of database internals, storage engines, table materialization techniques, compaction strategies, and indexing approaches for optimizing query performance and storage efficiency.
Cloud Infrastructure & DevOps
Demonstrated proficiency with cloud platforms (AWS, Azure, or GCP), containerization, orchestration, and infrastructure-as-code practices for building scalable, resilient systems in production environments.
Performance Optimization & Scalability
Proven track record optimizing systems for throughput, latency, and resource efficiency at scale. Experience identifying bottlenecks, designing horizontal scaling strategies, and implementing high-performance data processing pipelines.
Education
Computer Science or Related Field
Bachelor's, Master's, or PhD degree in Computer Science, Computer Engineering, Software Engineering, or equivalent field. Equivalent professional experience in systems engineering or distributed systems development is acceptable as an alternative.
Experience
Senior-Level Systems Engineering (8+ years)
Minimum 8+ years of professional software engineering experience with substantial time spent designing and implementing distributed systems, data infrastructure, or storage solutions. Demonstrated progression to senior technical leadership roles.
Storage & Infrastructure Projects
Proven experience building production storage systems, data platforms, or infrastructure components handling massive scale. Track record of shipping features that improved system performance, reliability, or scalability in distributed environments.
Open-Source Contributions
Active contributions to open-source data systems projects, particularly Apache Kafka, Apache Iceberg, or Apache Flink. Demonstrated understanding of streaming data platforms and their architectural patterns through community involvement.
Stakeholder Leadership & Communication
Experience influencing and guiding technical decisions across multiple teams and organizational levels. Proven ability to communicate complex technical concepts to diverse audiences including product, design, and executive stakeholders.
Skills
Required
Distributed Systems Architecture
Expert-level design and implementation of distributed systems with expertise in consensus protocols, replication strategies, fault tolerance, and consistency models. Ability to architect systems that scale horizontally while maintaining reliability.
Java or Scala Programming
Advanced proficiency in Java or Scala for building production systems. Deep understanding of JVM performance characteristics, garbage collection, memory management, and concurrent programming patterns essential for high-throughput data systems.
Data Systems Engineering
Strong expertise in designing data systems including query optimization, indexing strategies, data serialization formats, and schema evolution. Understanding of columnar formats, compression techniques, and materialization patterns.
Database Internals
Comprehensive knowledge of relational and columnar database engines, query planning, execution optimization, ACID properties, transactions, and distributed query processing. Familiarity with modern data warehouse and lakehouse architectures.
Cloud Platform Operations
Production experience with AWS, Azure, or GCP. Proficiency in services like EC2, RDS, S3, Kubernetes, load balancing, monitoring, and infrastructure automation for deploying and managing large-scale systems.
Preferred
Apache Kafka Expertise
Nice to haveDeep hands-on experience or open-source contributions to Apache Kafka. Understanding of Kafka's architecture, replication, topic design, and integration patterns for event streaming platforms. Familiarity with Confluent's ecosystem and extensions.
Apache Iceberg Knowledge
Nice to haveExperience with Apache Iceberg table format, including schema evolution, partitioning strategies, hidden partitions, and time-travel queries. Understanding of how Iceberg solves data reliability and query optimization challenges in data lakes.
Apache Flink Proficiency
Nice to haveHands-on experience with Apache Flink for stream processing, including stateful computations, windowing, checkpointing, and distributed state management. Understanding of Flink's execution model and optimization techniques.
Stream-Table Duality
Nice to haveConceptual and practical understanding of stream-table duality and how streaming and batch perspectives can be unified. Experience implementing dual-mode systems that provide consistent semantics across stream and batch processing.
Performance Benchmarking & Profiling
Nice to haveExpertise in profiling distributed systems, identifying performance bottlenecks, and conducting rigorous benchmarking studies. Experience with tools like Java Flight Recorder, flame graphs, and continuous performance monitoring in production.
Kubernetes & Container Orchestration
Nice to haveProduction experience with Kubernetes for deploying, scaling, and managing containerized data systems. Proficiency with helm charts, custom resources, and orchestration patterns for complex multi-tenant architectures.
Tech stack
Languages
Frameworks
Databases
Tools
Other
Compensation
Pay and benefits.
Base·USD 261,300 – 307,000
Equity·Stock options
Benefits
Comprehensive Health Coverage
Medical, dental, and vision insurance with plans covering employees, dependents, and qualifying family members. Access to preventive care, specialist networks, and prescription drug coverage.
Competitive Retirement Plans
401(k) retirement savings plan with company matching contributions to help build long-term financial security and retirement readiness.
Generous Time Off
Unlimited paid time off policy allowing flexibility for vacation, personal wellness, and life events. Additional company holidays and paid leave for specific circumstances.
Equity & Stock Options
Participate in company success through stock options and equity grants vesting over time. Opportunity to benefit from company growth as a technical leader.
Professional Development
Learning budgets for conferences, courses, and certifications. Access to mentorship programs, internal technical workshops, and opportunities to present at industry events.
Workplace Flexibility
Remote work options and flexible scheduling to balance work and personal life. Distributed team structure respecting diverse time zones and work preferences.
Wellness Programs
Gym memberships, mental health resources, meditation apps, and wellness initiatives supporting employee physical and mental well-being.
Parental & Family Support
Paid parental leave for primary and secondary caregivers, adoption assistance, and family planning benefits supporting employees at all life stages.
Process
Interview steps.
- 01
Initial Screening Call
Recruiter conversation to understand background, career trajectory, and motivation for the role. Discussion of technical interests, experience with distributed systems, and alignment with Confluent's culture and mission.
- 02
Technical Phone Interview
Focused discussion on distributed systems design principles, data platform architecture, and problem-solving approach. Potential discussion of Apache Kafka, Iceberg, or Flink experience. No live coding required at this stage.
- 03
Architecture Design Discussion
Deep-dive session exploring how you would design key components of the Tableflow system or similar distributed infrastructure challenges. Discussion of tradeoffs, scalability considerations, and production deployment strategies.
- 04
System Design Interview
Collaborative problem-solving session designing a large-scale distributed system from first principles. Focus on handling massive data volumes, multi-tenancy, high availability, and operational concerns at production scale.
- 05
Stakeholder Impact Conversation
Discussion with team members about cross-functional collaboration, communication approach, and experience influencing technical decisions. Questions about working with product, design, and other engineering teams.
- 06
Leadership & Culture Fit Discussion
Conversation with senior leaders or hiring manager exploring technical vision, leadership philosophy, and values alignment with Confluent's collaborative, inclusive culture and commitment to engineering excellence.
Full posting
Original listing.
We’re not just building better tech. We’re rewriting how data moves and what the world can do with it. With Confluent, data doesn’t sit still. Our platform puts information in motion, streaming in near real-time so companies can react faster, build smarter, and deliver experiences as dynamic as the world around them.
It takes a certain kind of person to join this team. Those who ask hard questions, give honest feedback, and show up for each other. No egos, no solo acts. Just smart, curious humans pushing toward something bigger, together.
One Confluent. One Team. One Data Streaming Platform.
About the Role:
We are a team of passionate engineers who love solving complex distributed storage systems and infrastructure problems. Through the Tableflow project, we are building the next generation of storage solutions for Kora, transforming it into a true multi-modal storage engine. Our goal is to enable streaming data to be consumed both as a table and as a stream seamlessly. If you're interested in learning more about the challenges we are tackling, check out our blog post: Tableflow: The Stream/Table, Kafka/Iceberg Duality.
As a Staff Software Engineer I on the team, you will focus on advancing key layers of the Tableflow project, including its multi-tenant, highly available compute infrastructure, and the core components responsible for table materialization and background maintenance. You will tackle challenges such as developing strategies for efficient table materialization and maintenance at massive scale across all of Confluent Cloud, ensuring high availability and scalability for background storage tasks.
What You Will Do:
Storage Engine Development: Design and implement innovative strategies for table schematization, materialization, and compaction at massive scale.
Platform Engineering: Build and maintain multi-tenant, highly available infrastructure that powers background computational tasks for Kora’s long-term storage. Ensure that the platform is optimized for scalability, reliability, and resilience, in alignment with organizational objectives.
Cross-Functional Collaboration: Collaborate with product management, design, and other engineering teams to ensure seamless integration of storage features and infrastructure with the broader organization.
What You Will Bring:
BS, MS, or PhD in computer science or a related field, or equivalent work experience
Excellent communication and collaboration skills, with the ability to influence and guide stakeholders at all levels.
What Gives You an Edge:
Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink
Experience/knowledge with public clouds (AWS, Azure or GCP)
Ready to build what's next? Let’s get in motion.
Come As You Are
Belonging isn’t a perk here. It’s the baseline. We work across time zones and backgrounds, knowing the best ideas come from different perspectives. And we make space for everyone to lead, grow, and challenge what’s possible.
We’re proud to be an equal opportunity workplace. Employment decisions are based on job-related criteria, without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other classification protected by law.
Privacy Statement
Confluent is an IBM subsidiary which has been acquired by IBM and will be integrated into the IBM organization. By proceeding with this application, you understand that Confluent will share your personal information with other IBM affiliates involved in your recruitment process, wherever these are located. More Information on how IBM protects your personal information, including the safeguards in case of cross-border data transfer, are available here.
Redirects to Confluent's application page.
Other roles
More at Confluent.
Senior Manager, Detection & Response (Security Engineering)
Manager
Staff Software Engineer
Staff
Distributed Systems Software Engineer - WarpSteam
Senior
Senior Software Engineer - Streaming AI (Remote - Ontario / British Columbia)
Senior
Staff Software Engineer I - Confluent Platform
Staff