Staff Software Engineer - Data Infrastructure
Data Engineer · Staff · Full Time
Opens Plaid's application page
Role
What you'll do.
Staff Software Engineer - Data Infrastructure at Plaid focuses on architecting and scaling data warehouse, data lakehouse, and streaming infrastructure systems. You'll own key technical initiatives to enable machine learning, ETL pipelines, and data platform abstractions while mentoring engineers across the organization. This role requires 6+ years of hands-on software engineering expertise in data infrastructure systems, along with demonstrated leadership capabilities and cross-functional collaboration skills.
Responsibilities
- Technical Roadmap Leadership: Contribute to and guide the long-term technical roadmap for data-driven and machine learning iteration at Plaid, ensuring alignment with business objectives and technical excellence. Establish best practices and architectural patterns for data infrastructure systems.
- Key Data Infrastructure Project Ownership: Lead critical data infrastructure initiatives including ML development golden paths, offline streaming solutions for data freshness, new ETL pipeline infrastructure development, and evolution of data warehouse and data lakehouse capabilities. Make architectural decisions that impact platform scalability and performance.
- Cross-Functional Stakeholder Collaboration: Work closely with engineering, product, and business teams to define technical roadmaps for backend systems and data abstractions. Communicate complex technical decisions to non-technical stakeholders and gather requirements from diverse team functions.
- Platform Operations and Reliability: Debug, troubleshoot, and reduce operational burden on the Data Platform. Identify performance bottlenecks, implement cost optimization strategies, and ensure platform reliability and data quality across the organization.
- Team Growth and Mentorship: Mentor and develop junior and mid-level engineers through code reviews, technical document reviews, and pair programming sessions. Establish engineering standards, conduct architectural reviews, and guide team members on system design best practices and career development.
Qualifications
What we look for.
Technical
Data Infrastructure Systems Architecture
Expert-level knowledge of data warehouse, data lakehouse, and distributed data systems architecture. Deep understanding of performance tuning, cost optimization, and scalability patterns in large-scale data platforms.
Apache Spark and Distributed Computing
Extensive hands-on experience with Apache Spark, including RDD, DataFrame, and SQL APIs. Proficiency in optimizing Spark jobs, managing memory, and debugging distributed computing challenges in production environments.
Workflow Orchestration and Scheduling
Strong experience designing, building, and maintaining workflow orchestration systems. Understanding of distributed scheduling, fault tolerance, monitoring, and SLA management in data pipelines.
Streaming Data Infrastructure
Hands-on experience building and maintaining streaming data systems for real-time data processing. Knowledge of event-driven architectures, exactly-once semantics, backpressure handling, and streaming framework trade-offs.
Coding and System Design
Advanced proficiency in software engineering fundamentals including design patterns, testing strategies, code organization, and API design. Proven ability to write scalable, maintainable code with comprehensive testing coverage.
Education
Bachelor's Degree in Computer Science or Related Field
Formal education in computer science, software engineering, or related discipline. Equivalent professional experience can substitute for formal education requirements.
Experience
Software Engineering Experience
6+ years of professional software engineering experience with demonstrated expertise in building and scaling production systems. Track record of successful project delivery and technical decision-making at scale.
Data Infrastructure Domain Experience
Substantial hands-on experience in data infrastructure or platform engineering roles at mid to large-sized technology companies. Proven success designing and implementing systems that thousands of engineers or teams depend upon.
Leadership and Mentorship
Demonstrated experience mentoring junior and mid-level engineers, conducting code reviews, and driving technical decisions. Ability to influence across teams and establish engineering standards and best practices.
Cross-Functional Collaboration
Proven ability to work effectively with product managers, stakeholders, and engineers across multiple teams. Experience gathering requirements, communicating technical constraints, and delivering solutions that balance technical and business needs.
Skills
Required
Apache Spark
Production-level proficiency in Apache Spark for large-scale distributed data processing, including performance optimization and troubleshooting.
Data Warehouse Design
Strong understanding of data warehouse architecture, schema design, query optimization, and analytics workload patterns. Experience with modern cloud data warehouses.
Workflow Orchestration
Hands-on experience building data pipelines with orchestration tools and managing complex ETL workflows at scale.
Python
Strong Python programming skills for building data infrastructure tools, utilities, and backend systems.
System Design
Advanced system design skills including scalability analysis, performance optimization, and architectural decision-making for data platforms.
SQL
Expert-level SQL proficiency for query writing, optimization, and complex data analysis.
Preferred
Databricks Platform
Nice to haveProduction experience with Databricks unified analytics platform, including Delta Lake, MLflow, and managed workflows.
Apache Airflow
Nice to haveHands-on experience with Apache Airflow for workflow orchestration, DAG design, and operational management.
AWS EMR and Cloud Data Infrastructure
Nice to haveExperience deploying and managing Hadoop/Spark clusters on AWS EMR or building cloud-native data infrastructure on AWS.
Streaming Technologies
Nice to haveProduction experience with Apache Kafka, Apache Flink, or similar streaming platforms for real-time data processing.
Data Lakehouse Architecture
Nice to haveHands-on experience building or maintaining data lakehouse systems combining data lake and warehouse capabilities.
ML Platform Engineering
Nice to haveExperience building infrastructure supporting machine learning workflows, feature stores, or MLOps platforms.
Tech stack
Languages
Frameworks
Databases
Tools
Other
Compensation
Pay and benefits.
Base·USD 207,600 – 273,600
Equity·Stock options
Benefits
Comprehensive Medical, Dental, and Vision Coverage
Full health insurance coverage including medical, dental, and vision plans with employer contributions.
401(k) Retirement Plan
Tax-advantaged retirement savings plan with company match eligibility.
Equity and Stock Options
Competitive equity grants or stock options as part of comprehensive compensation package for staff-level engineers.
Professional Development and Learning
Access to learning resources, conference attendance budgets, and professional development opportunities to advance technical expertise.
Flexible Work Arrangement
Work environment supporting flexible schedules and collaboration across Plaid's offices in San Francisco, New York, Washington D.C., London, and Amsterdam.
Mentorship and Career Growth
Structured mentorship programs, technical leadership opportunities, and clear career advancement paths for staff engineers.
Process
Interview steps.
- 01
Application Review and Initial Screening
Plaid's recruiting team reviews your application, focusing on relevant data infrastructure experience, technical depth, and alignment with staff-level expectations. Prior experience at scaled companies with large data systems is particularly valuable.
- 02
Technical Phone Screen
Initial 45-60 minute conversation with a data infrastructure engineer covering your background, key projects, system design approaches, and technical decision-making philosophy. Expect questions about distributed systems, data pipeline architecture, and your experience with relevant technologies.
- 03
System Design Interview
In-depth technical interview focused on designing data infrastructure systems at scale. You'll discuss tradeoffs in warehouse vs. lakehouse architectures, workflow orchestration patterns, streaming data handling, and cost optimization strategies. Emphasis on your ability to think through complex architectural decisions.
- 04
Technical Deep Dive
Detailed conversation about your most significant data infrastructure projects, including challenges faced, solutions designed, and lessons learned. Expect technical depth probing your expertise in Spark, workflow orchestration, or streaming technologies.
- 05
Leadership and Impact Assessment
Behavioral interview with senior engineering leadership evaluating your mentorship approach, cross-functional collaboration skills, and track record of delivering large technical initiatives. Discussion of how you've influenced technical roadmaps and grown engineering teams.
- 06
Leadership and Hiring Manager Interview
Final conversation with the Data Infrastructure team lead or director covering team dynamics, long-term vision for the platform, your leadership style, and how you approach mentoring and building high-performing teams.
Full posting
Original listing.
We believe that the way people interact with their finances will drastically improve in the next few years. We’re dedicated to empowering this transformation by building the tools and experiences that thousands of developers use to create their own products. Plaid powers the tools millions of people rely on to live a healthier financial life. We work with thousands of companies like Venmo, SoFi, several of the Fortune 500, and many of the largest banks to make it easy for people to connect their financial accounts to the apps and services they want to use. Plaid’s network covers 12,000 financial institutions across the US, Canada, UK and Europe. Founded in 2013, the company is headquartered in San Francisco with offices in New York, Washington D.C., London and Amsterdam.
Making data driven decisions is key to Plaid's culture. To support that, we need to scale our data systems while maintaining correct and complete data. We provide tooling and guidance to teams across engineering, product, and business and help them explore our data quickly and safely to get the data insights they need, which ultimately helps Plaid serve our customers more effectively.
Engineers on Data Infrastructure are domain experts in Data Warehouse, Data Lakehouse, Spark, Workflow Orchestration, and Streaming technologies. We scale our existing data pipelines in a performant and cost efficient way while creating the necessary abstractions to make developing on top of this platform extremely simple for other engineers at Plaid.
Responsibilities
Contribute towards the long-term technical roadmap for data-driven and machine learning iteration at Plaid
Leading key data infrastructure projects such as improving ML development golden paths, implementing offline streaming solutions for data freshness, building net new ETL pipeline infrastructure, and evolving data warehouse or data lakehouse capabilities.
Working with stakeholders in other teams and functions to define technical roadmaps for key backend systems and abstractions across Plaid.
Debugging, troubleshooting, and reducing operational burden for our Data Platform.
Growing the team via mentorship and leadership, reviewing technical documents and code changes.
Qualifications
6+ years of software engineering experience
Extensive hands-on software engineering experience, with a strong track record of delivering successful projects within the Data Infrastructure or Platform domain at similar or larger companies.
Deep understanding of one of the below:
Data Infrastructure systems, including Data Warehouses, Data Lakehouses, Apache Spark, Streaming Infrastructure, Workflow Orchestration.
Strong cross-functional collaboration, communication, and project management skills, with proven ability to coordinate effectively.
Proficiency in coding, testing, and system design, ensuring reliable and scalable solutions.
Demonstrated leadership abilities, including experience mentoring and guiding junior engineers.
Nice-to-Have:
Experience with Databricks
Experience with Airflow
Experience with AWS EMR
Experience with Python
Our mission at Plaid is to unlock financial freedom for everyone. To support that mission, we seek to build a diverse team of driven individuals who care deeply about making the financial ecosystem more equitable. We recognize that strong qualifications can come from both prior work experiences and lived experiences. We encourage you to apply to a role even if your experience doesn't fully match the job description. We are always looking for team members that will bring something unique to Plaid!
Plaid is proud to be an equal opportunity employer and values diversity at our company. We do not discriminate based on race, color, national origin, ethnicity, religion or religious belief, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, military or veteran status, disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state, and local laws. Plaid is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance with your application or interviews due to a disability, please let us know at [email protected].
Please review our Candidate Privacy Notice here.
Additional compensation in the form(s) of equity and/or commission are dependent on the position offered. Plaid provides a comprehensive benefit plan, including medical, dental, vision, and 401(k). Pay is based on factors such as (but not limited to) scope and responsibilities of the position, candidate's work experience and skillset, and location. Pay and benefits are subject to change at any time, consistent with the terms of any applicable compensation or benefit plans.
Redirects to Plaid's application page.
Other roles
More at Plaid.
Engineering Manager - Consumer
Manager
Engineering Manager - Network Health
Manager
Senior Data Engineer
Senior
Software Engineer, Full Stack
Mid
Senior Software Engineer, Full Stack
Senior