# Staff Software Engineer - Data Infrastructure **Company:** [Plaid](https://scaleengineer.com/companies/plaid) Staff Software Engineer - Data Infrastructure at Plaid focuses on architecting and scaling data warehouse, data lakehouse, and streaming infrastructure systems. You'll own key technical initiatives to enable machine learning, ETL pipelines, and data platform abstractions while mentoring engineers across the organization. This role requires 6+ years of hands-on software engineering expertise in data infrastructure systems, along with demonstrated leadership capabilities and cross-functional collaboration skills. **Role:** Data Engineer **Seniority:** Staff **Locations:** San Francisco HQ **Salary:** 207600–273600 USD [Apply](https://jobs.ashbyhq.com/plaid/127d31ff-3a16-488b-ba01-6e87dbb96957) Canonical: https://scaleengineer.com/jobs/plaid/staff-software-engineer-data-infrastructure --- ## Responsibilities - Technical Roadmap Leadership: Contribute to and guide the long-term technical roadmap for data-driven and machine learning iteration at Plaid, ensuring alignment with business objectives and technical excellence. Establish best practices and architectural patterns for data infrastructure systems. - Key Data Infrastructure Project Ownership: Lead critical data infrastructure initiatives including ML development golden paths, offline streaming solutions for data freshness, new ETL pipeline infrastructure development, and evolution of data warehouse and data lakehouse capabilities. Make architectural decisions that impact platform scalability and performance. - Cross-Functional Stakeholder Collaboration: Work closely with engineering, product, and business teams to define technical roadmaps for backend systems and data abstractions. Communicate complex technical decisions to non-technical stakeholders and gather requirements from diverse team functions. - Platform Operations and Reliability: Debug, troubleshoot, and reduce operational burden on the Data Platform. Identify performance bottlenecks, implement cost optimization strategies, and ensure platform reliability and data quality across the organization. - Team Growth and Mentorship: Mentor and develop junior and mid-level engineers through code reviews, technical document reviews, and pair programming sessions. Establish engineering standards, conduct architectural reviews, and guide team members on system design best practices and career development. ## Requirements ### education - {"name":"Bachelor's Degree in Computer Science or Related Field","description":"Formal education in computer science, software engineering, or related discipline. Equivalent professional experience can substitute for formal education requirements."} ### technical - {"name":"Data Infrastructure Systems Architecture","description":"Expert-level knowledge of data warehouse, data lakehouse, and distributed data systems architecture. Deep understanding of performance tuning, cost optimization, and scalability patterns in large-scale data platforms."} - {"name":"Apache Spark and Distributed Computing","description":"Extensive hands-on experience with Apache Spark, including RDD, DataFrame, and SQL APIs. Proficiency in optimizing Spark jobs, managing memory, and debugging distributed computing challenges in production environments."} - {"name":"Workflow Orchestration and Scheduling","description":"Strong experience designing, building, and maintaining workflow orchestration systems. Understanding of distributed scheduling, fault tolerance, monitoring, and SLA management in data pipelines."} - {"name":"Streaming Data Infrastructure","description":"Hands-on experience building and maintaining streaming data systems for real-time data processing. Knowledge of event-driven architectures, exactly-once semantics, backpressure handling, and streaming framework trade-offs."} - {"name":"Coding and System Design","description":"Advanced proficiency in software engineering fundamentals including design patterns, testing strategies, code organization, and API design. Proven ability to write scalable, maintainable code with comprehensive testing coverage."} ### experience - {"name":"Software Engineering Experience","description":"6+ years of professional software engineering experience with demonstrated expertise in building and scaling production systems. Track record of successful project delivery and technical decision-making at scale."} - {"name":"Data Infrastructure Domain Experience","description":"Substantial hands-on experience in data infrastructure or platform engineering roles at mid to large-sized technology companies. Proven success designing and implementing systems that thousands of engineers or teams depend upon."} - {"name":"Leadership and Mentorship","description":"Demonstrated experience mentoring junior and mid-level engineers, conducting code reviews, and driving technical decisions. Ability to influence across teams and establish engineering standards and best practices."} - {"name":"Cross-Functional Collaboration","description":"Proven ability to work effectively with product managers, stakeholders, and engineers across multiple teams. Experience gathering requirements, communicating technical constraints, and delivering solutions that balance technical and business needs."} ## Skills ### required - {"name":"Apache Spark","description":"Production-level proficiency in Apache Spark for large-scale distributed data processing, including performance optimization and troubleshooting."} - {"name":"Data Warehouse Design","description":"Strong understanding of data warehouse architecture, schema design, query optimization, and analytics workload patterns. Experience with modern cloud data warehouses."} - {"name":"Workflow Orchestration","description":"Hands-on experience building data pipelines with orchestration tools and managing complex ETL workflows at scale."} - {"name":"Python","description":"Strong Python programming skills for building data infrastructure tools, utilities, and backend systems."} - {"name":"System Design","description":"Advanced system design skills including scalability analysis, performance optimization, and architectural decision-making for data platforms."} - {"name":"SQL","description":"Expert-level SQL proficiency for query writing, optimization, and complex data analysis."} ### preferred - {"name":"Databricks Platform","description":"Production experience with Databricks unified analytics platform, including Delta Lake, MLflow, and managed workflows."} - {"name":"Apache Airflow","description":"Hands-on experience with Apache Airflow for workflow orchestration, DAG design, and operational management."} - {"name":"AWS EMR and Cloud Data Infrastructure","description":"Experience deploying and managing Hadoop/Spark clusters on AWS EMR or building cloud-native data infrastructure on AWS."} - {"name":"Streaming Technologies","description":"Production experience with Apache Kafka, Apache Flink, or similar streaming platforms for real-time data processing."} - {"name":"Data Lakehouse Architecture","description":"Hands-on experience building or maintaining data lakehouse systems combining data lake and warehouse capabilities."} - {"name":"ML Platform Engineering","description":"Experience building infrastructure supporting machine learning workflows, feature stores, or MLOps platforms."} ## Tech stack ### tools - {"name":"AWS EMR","description":"Managed Hadoop and Spark clusters on AWS for distributed data processing and analytics infrastructure."} - {"name":"AWS S3","description":"Object storage service for data lake architecture and distributed data processing storage layers."} - {"name":"Apache Kafka","description":"Distributed event streaming platform for real-time data pipelines and event-driven architectures."} - {"name":"dbt","description":"Data transformation tool for building analytics engineering workflows with modular SQL transformations."} - {"name":"Git and Version Control","description":"Source code management for infrastructure-as-code, pipeline versioning, and collaborative development."} ### others - {"name":"Distributed Systems Design","description":"Understanding of consistency models, fault tolerance, and performance characteristics in large-scale distributed data systems."} - {"name":"Data Architecture and Modeling","description":"Data modeling techniques, dimensional modeling, slowly changing dimensions, and analytics schema design patterns."} - {"name":"Performance Optimization","description":"Query optimization, indexing strategies, data compression, and cost optimization techniques for data infrastructure."} - {"name":"Infrastructure as Code","description":"Terraform, CloudFormation, or similar tools for managing cloud infrastructure, data platform resources, and automation."} ### databases - {"name":"Data Warehouse (Snowflake, BigQuery, Redshift)","description":"Cloud-native data warehouse systems for analytics, reporting, and business intelligence workloads."} - {"name":"Delta Lake","description":"Open-source storage format providing ACID transactions and unified data lakehouse capabilities on cloud object storage."} - {"name":"Apache Hudi","description":"Incremental processing framework for managing large datasets with support for upserts and incremental queries."} ### languages - {"name":"Python","description":"Primary language for data infrastructure tooling, pipeline development, and automation."} - {"name":"SQL","description":"Essential for data warehouse query optimization, schema design, and analytics workload development."} - {"name":"Scala","description":"Used for Apache Spark development and distributed computing systems."} ### frameworks - {"name":"Apache Spark","description":"Core distributed data processing framework for ETL, batch processing, and data transformations at scale."} - {"name":"Apache Airflow","description":"Workflow orchestration platform for scheduling and managing complex data pipeline dependencies."} - {"name":"Databricks","description":"Unified analytics platform providing collaborative notebooks, Delta Lake, and managed Spark clusters."} ## Benefits ### benefits - {"name":"Comprehensive Medical, Dental, and Vision Coverage","description":"Full health insurance coverage including medical, dental, and vision plans with employer contributions."} - {"name":"401(k) Retirement Plan","description":"Tax-advantaged retirement savings plan with company match eligibility."} - {"name":"Equity and Stock Options","description":"Competitive equity grants or stock options as part of comprehensive compensation package for staff-level engineers."} - {"name":"Professional Development and Learning","description":"Access to learning resources, conference attendance budgets, and professional development opportunities to advance technical expertise."} - {"name":"Flexible Work Arrangement","description":"Work environment supporting flexible schedules and collaboration across Plaid's offices in San Francisco, New York, Washington D.C., London, and Amsterdam."} - {"name":"Mentorship and Career Growth","description":"Structured mentorship programs, technical leadership opportunities, and clear career advancement paths for staff engineers."} ## Compensation - **max:** 280000 - **min:** 200000 - **currency:** USD - **stockOptions:** true ## Interview process ### steps - {"name":"Application Review and Initial Screening","description":"Plaid's recruiting team reviews your application, focusing on relevant data infrastructure experience, technical depth, and alignment with staff-level expectations. Prior experience at scaled companies with large data systems is particularly valuable."} - {"name":"Technical Phone Screen","description":"Initial 45-60 minute conversation with a data infrastructure engineer covering your background, key projects, system design approaches, and technical decision-making philosophy. Expect questions about distributed systems, data pipeline architecture, and your experience with relevant technologies."} - {"name":"System Design Interview","description":"In-depth technical interview focused on designing data infrastructure systems at scale. You'll discuss tradeoffs in warehouse vs. lakehouse architectures, workflow orchestration patterns, streaming data handling, and cost optimization strategies. Emphasis on your ability to think through complex architectural decisions."} - {"name":"Technical Deep Dive","description":"Detailed conversation about your most significant data infrastructure projects, including challenges faced, solutions designed, and lessons learned. Expect technical depth probing your expertise in Spark, workflow orchestration, or streaming technologies."} - {"name":"Leadership and Impact Assessment","description":"Behavioral interview with senior engineering leadership evaluating your mentorship approach, cross-functional collaboration skills, and track record of delivering large technical initiatives. Discussion of how you've influenced technical roadmaps and grown engineering teams."} - {"name":"Leadership and Hiring Manager Interview","description":"Final conversation with the Data Infrastructure team lead or director covering team dynamics, long-term vision for the platform, your leadership style, and how you approach mentoring and building high-performing teams."} ## Full description We believe that the way people interact with their finances will drastically improve in the next few years. We’re dedicated to empowering this transformation by building the tools and experiences that thousands of developers use to create their own products. Plaid powers the tools millions of people rely on to live a healthier financial life. We work with thousands of companies like Venmo, SoFi, several of the Fortune 500, and many of the largest banks to make it easy for people to connect their financial accounts to the apps and services they want to use. Plaid’s network covers 12,000 financial institutions across the US, Canada, UK and Europe. Founded in 2013, the company is headquartered in San Francisco with offices in New York, Washington D.C., London and Amsterdam. Making data driven decisions is key to Plaid's culture. To support that, we need to scale our data systems while maintaining correct and complete data. We provide tooling and guidance to teams across engineering, product, and business and help them explore our data quickly and safely to get the data insights they need, which ultimately helps Plaid serve our customers more effectively. Engineers on Data Infrastructure are domain experts in Data Warehouse, Data Lakehouse, Spark, Workflow Orchestration, and Streaming technologies. We scale our existing data pipelines in a performant and cost efficient way while creating the necessary abstractions to make developing on top of this platform extremely simple for other engineers at Plaid. **Responsibilities** * Contribute towards the long-term technical roadmap for data-driven and machine learning iteration at Plaid * Leading key data infrastructure projects such as improving ML development golden paths, implementing offline streaming solutions for data freshness, building net new ETL pipeline infrastructure, and evolving data warehouse or data lakehouse capabilities. * Working with stakeholders in other teams and functions to define technical roadmaps for key backend systems and abstractions across Plaid. * Debugging, troubleshooting, and reducing operational burden for our Data Platform. * Growing the team via mentorship and leadership, reviewing technical documents and code changes. **Qualifications** * 6+ years of software engineering experience * Extensive hands-on software engineering experience, with a strong track record of delivering successful projects within the Data Infrastructure or Platform domain at similar or larger companies. * Deep understanding of one of the below: * Data Infrastructure systems, including Data Warehouses, Data Lakehouses, Apache Spark, Streaming Infrastructure, Workflow Orchestration. * Strong cross-functional collaboration, communication, and project management skills, with proven ability to coordinate effectively. * Proficiency in coding, testing, and system design, ensuring reliable and scalable solutions. * Demonstrated leadership abilities, including experience mentoring and guiding junior engineers. * Nice-to-Have: * Experience with Databricks * Experience with Airflow * Experience with AWS EMR * Experience with Python Our mission at Plaid is to unlock financial freedom for everyone. To support that mission, we seek to build a diverse team of driven individuals who care deeply about making the financial ecosystem more equitable. We recognize that strong qualifications can come from both prior work experiences and lived experiences. We encourage you to apply to a role even if your experience doesn't fully match the job description. We are always looking for team members that will bring something unique to Plaid! Plaid is proud to be an equal opportunity employer and values diversity at our company. We do not discriminate based on race, color, national origin, ethnicity, religion or religious belief, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, military or veteran status, disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state, and local laws. Plaid is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance with your application or interviews due to a disability, please let us know at accommodations@plaid.com. Please review our Candidate Privacy Notice [here](https://plaid.com/legal/#candidate-privacy-notice). Additional compensation in the form(s) of equity and/or commission are dependent on the position offered. Plaid provides a comprehensive benefit plan, including medical, dental, vision, and 401(k). Pay is based on factors such as (but not limited to) scope and responsibilities of the position, candidate's work experience and skillset, and location. Pay and benefits are subject to change at any time, consistent with the terms of any applicable compensation or benefit plans.