Scale Engineer

Machine Learning Engineer, Core Experimentation

ML Engineer · Senior · Full Time

SeattleUSD 437k – 485k4w ago
Apply for this role

Opens OpenAI's application page

Role

What you'll do.

Lead the technical direction for ML-powered experimentation and insights at OpenAI's Statsig platform. This 0-to-1 role combines machine learning, statistical methods, LLM systems, and product design to build production systems that generate evidence-backed insights and enable teams across ChatGPT, Codex, and developer products to make high-confidence decisions. You'll design end-to-end systems that learn from privacy-protected data while maintaining calibration, uncertainty quantification, and human oversight as core product requirements.

Responsibilities

  • Technical Roadmap Leadership: Set and execute the complete technical roadmap for Generative Insights and Predictive Experimentation from early prototypes through production adoption, working across the full ML lifecycle to deliver capabilities that support evidence-based decision making at scale.
  • Cross-Experiment Learning Systems: Build systems that retrieve and synthesize historical experiments, detect recurring effects and behavioral segments, reanalyze prior results when data or methods improve, and generate hypotheses with clear evidence and provenance to accelerate organizational learning.
  • Predictive Modeling and Simulation: Develop predictive models and simulation workflows including simulation-based evaluation approaches to estimate likely impact, affected segments, regression risk, and uncertainty before full live experiments, reducing costly learning cycles.
  • Data Engineering and Pipeline Development: Create high-quality datasets and feature or retrieval pipelines from exposures, events, metrics, experiment metadata, and replay data with strong lineage, freshness, privacy, and data-quality controls as foundational platform infrastructure.
  • Rigorous Model Evaluation and Monitoring: Establish rigorous evaluation through offline benchmarks, backtests, calibration analysis, drift monitoring, prediction-to-outcome comparisons, and explicit failure or abstention behavior to ensure models are safe for production decision-making.
  • Product Operationalization: Transform research models into durable product, API, and agent workflows that move seamlessly from insight generation to experiment design, approval-gated action, and measured learning outcomes.
  • Cross-Functional Collaboration: Partner deeply with data science and product teams on experiment design, causal inference, sequential decision-making, and variance reduction techniques, establishing clear boundaries between prediction and causal evidence.
  • Production ML Quality Leadership: Build reliable services and intuitive workflows that make sophisticated ML capabilities understandable and useful to teams making high-stakes product decisions, providing technical leadership and raising ML quality standards across the platform.

Qualifications

What we look for.

Technical

  • Production ML Development

    Demonstrated ability to lead end-to-end 0-to-1 production ML products where success is measured by real-world decision quality and organizational impact, not merely offline model metrics.

  • Full ML Lifecycle Expertise

    Hands-on experience across the complete ML lifecycle including dataset design, model training or adaptation, evaluation methodology, production deployment, monitoring systems, and iterative improvement.

  • Domain Depth

    Deep expertise in one or more of: LLM and retrieval systems, ranking and recommendation systems, forecasting or anomaly detection, causal ML and experiment analysis, or simulation-based methods.

  • Software Engineering Fundamentals

    Strong software engineering foundations with proven ability to build high-quality production systems in Python while working effectively across data, backend, and platform infrastructure boundaries.

  • Statistical and Causal Reasoning

    Strong grounding in experimentation and statistical reasoning with deep understanding of why predictive accuracy differs from causal validity and how to design experiments that support reliable causal inference.

  • Production System Design

    Experience designing and deploying systems that prioritize calibration, uncertainty quantification, provenance tracking, privacy protection, and human review as core product requirements rather than post-hoc additions.

Education

  • Advanced Degree or Equivalent

    Advanced degree (MS or PhD) in machine learning, statistics, computer science, or related field, or equivalent practical experience demonstrating mastery of ML fundamentals and statistical theory.

Experience

  • Senior ML Engineering (5+ years)

    Minimum 5+ years of professional experience building production machine learning systems at scale, with demonstrated track record of taking ambiguous problems from concept to organizational adoption.

  • 0-to-1 Product Development

    Proven experience leading greenfield ML product development where you owned the technical direction, shaped feature prioritization, and drove adoption across internal or external stakeholders.

  • Cross-Functional Leadership

    Experience translating ambiguous business or product questions into technical roadmaps and working effectively with product managers, data scientists, research teams, and infrastructure partners.

Skills

Required

  • Python Development

    Expert-level Python programming with ability to write production-quality code that scales and maintains reliability in complex distributed systems.

  • Machine Learning Systems

    Deep hands-on experience building, training, evaluating, and deploying machine learning models in production environments with full ownership of end-to-end system quality.

  • Statistical Inference

    Strong foundation in statistical methods, experimental design, hypothesis testing, and causal inference with ability to communicate statistical concepts to technical and non-technical stakeholders.

  • Data Engineering

    Proficiency in designing data pipelines, managing feature engineering workflows, handling data quality and lineage, and working with large-scale datasets for training and evaluation.

  • Experimentation and A/B Testing

    Deep understanding of experimental methodology, A/B testing design, statistical power analysis, multiple testing correction, and causal inference in observational settings.

  • Backend Systems Design

    Comfortable designing backend services, APIs, and distributed systems that support ML inference at scale with proper monitoring, logging, and failure handling.

Preferred

  • LLM and Retrieval Systems

    Nice to have

    Experience working with large language models, retrieval-augmented generation, prompt engineering, or building systems that leverage LLM capabilities for analysis or insights.

  • Causal ML Methods

    Nice to have

    Hands-on experience with causal inference techniques, propensity score matching, instrumental variables, or other methods for estimating treatment effects in observational data.

  • Ranking and Recommendation Systems

    Nice to have

    Experience building ranking algorithms, recommendation systems, or learning-to-rank approaches in production environments with optimization for business metrics.

  • Simulation and Forecasting

    Nice to have

    Background in simulation methodologies, time series forecasting, anomaly detection, or predictive modeling for resource planning or impact estimation.

  • ML Infrastructure and Tooling

    Nice to have

    Familiarity with ML infrastructure, experiment tracking systems, model serving platforms, or MLOps practices that support reproducible and reliable ML deployments.

  • Privacy-Preserving ML

    Nice to have

    Experience with privacy-preserving techniques such as differential privacy, federated learning, or secure data handling in production ML systems.

Tech stack

Languages

PythonSQL

Frameworks

PyTorch or TensorFlowScikit-LearnLLM Frameworks

Databases

Data Warehousing SolutionsTimeseries Databases

Tools

Jupyter Notebooks and IDEsVersion Control SystemsExperiment Tracking and ML PlatformsObservability and Monitoring

Other

Distributed ComputingStatistical Testing and Causal InferenceAPI Design and Deployment

Compensation

Pay and benefits.

Base·USD 437,000 – 485,000

Equity·Stock options

Benefits

  • Equity Compensation

    Competitive stock options reflecting your role and impact as a senior technical leader helping shape OpenAI's experimentation infrastructure.

  • Comprehensive Health Benefits

    Medical, dental, and vision coverage with significant employer contributions supporting you and your family's wellbeing.

  • Retirement Planning

    401(k) retirement savings plan with competitive employer matching to support long-term financial security.

  • Flexible Time Off

    Generous vacation and paid time off policy allowing you to balance intensive project work with personal wellness and recovery.

  • Professional Development

    Support for continued learning, conference attendance, and skill development to stay current with ML research and industry trends.

  • Collaborative Work Environment

    Bellevue office infrastructure designed for close collaboration with product teams, encouraging rapid problem-solving and knowledge sharing.

  • Relocation Assistance

    Support for relocating to the Bellevue, Washington office area, recognizing the importance of in-person collaboration to this role.

Process

Interview steps.

  1. 01

    Initial Screening Call

    Introductory conversation with recruiter to discuss your background, experience with production ML systems, and alignment with the role's 0-to-1 product development focus.

  2. 02

    Technical Deep Dive Interview

    Detailed discussion of your production ML experience including specific projects, the full lifecycle from conception through deployment, metrics used to measure success, and decisions made around model architecture and evaluation.

  3. 03

    ML System Design Interview

    Collaborative discussion of how you would approach building one of the core experimentation and insights capabilities, including data pipeline design, model architecture, evaluation strategy, and production deployment considerations.

  4. 04

    Statistical and Causal Reasoning Interview

    In-depth conversation about experimentation methodology, causal inference, your understanding of why predictive accuracy differs from causal validity, and how you've navigated statistical complexity in production systems.

  5. 05

    Cross-Functional Collaboration Interview

    Discussion with data scientists or product managers about how you translate ambiguous business questions into technical solutions, examples of successful cross-team projects, and your approach to raising standards across organizations.

  6. 06

    Leadership and Culture Fit

    Conversation about your leadership philosophy, how you've shaped technical direction in previous roles, your approach to building high-quality systems, and your values around collaboration, transparency, and impact.

Full posting

Original listing.

About the Team

The Statsig team within OpenAI builds the experimentation, feature rollout, dynamic configuration, and analytics systems that help OpenAI ship products with speed, safety, and evidence. Our work sits on the critical path for how product, engineering, research, and go-to-market teams learn from real-world usage and make high-confidence decisions.

Statsig began as an independent company focused on helping builders move faster through trustworthy experimentation and feature management. After Statsig joined OpenAI, the team began the next chapter: bringing that deep product expertise, customer intuition, and mature platform infrastructure into OpenAI as the experimentation and rollout platform for every product we ship.

Today, we support teams across ChatGPT, Codex, model measurement, consumer experiences, business subscriptions, developer products, and the shared infrastructure that connects them. These teams rely on Statsig to safely introduce new capabilities, compare product and model behavior, measure impact, and roll changes forward or back with confidence.

We are at a defining moment in the platform journey. OpenAI has the data, product surface area, and pace of innovation to learn faster than almost any organization in the world, but that potential only becomes real if teams can experiment responsibly, measure clearly, and roll out changes safely.

We are evolving experimentation systems to help teams learn from product behavior and make better evidence-based decisions.

Based out of OpenAI’s Bellevue office, we are a close-knit team that values in-person collaboration, urgency, craft, and impact. We build for other builders, and the best version of this team is one where every OpenAI product team can move faster because the experimentation and rollout layer is dependable, fast, and easy to use.

About the Role

We are looking for a Machine Learning Engineer to lead the technical direction for ML-powered experimentation and insights capabilities. You will build production systems that learn from privacy-protected product and experimentation data to generate evidence-backed insights and support decision-making, and help teams decide which ideas are worth testing live.

This is an end-to-end, 0-to-1 role. You will work across ML modeling, retrieval and LLM systems, statistical methods, simulation, data and training pipelines, backend services, and user- and agent-facing product experiences. The hard part is not merely producing a plausible answer. It is making each insight and prediction traceable, calibrated, useful, and safe enough to influence real product decisions.

Live experiments remain the source of causal validation. You will design systems that make uncertainty explicit, backtest against historical outcomes, compare predictions with online results, learn from misses, and abstain when the evidence is weak. You will preserve clear review, permission, and approval boundaries as automation becomes more powerful.

You will collaborate closely with teams building ChatGPT, Codex, model measurement workflows, consumer products, Growth, business subscription experiences, developer products, and shared infrastructure. You will turn their most important learning and decision problems into general platform capabilities that can support the full company.

In This Role, You Will

  • Set and execute the technical roadmap for Generative Insights and Predictive Experimentation, from early prototypes through production adoption.

  • Build cross-experiment learning systems that retrieve and synthesize historical experiments, detect recurring effects and segment behavior, reanalyze prior results when data or methods improve, and generate hypotheses with clear evidence and provenance.

  • Develop predictive models and simulation workflows, including simulation-based evaluation approaches, to estimate likely impact, affected segments, regression risk, and uncertainty before a full live experiment.

  • Create high-quality datasets and feature or retrieval pipelines from exposures, events, metrics, experiment metadata, and replay data, with strong lineage, freshness, privacy, and data-quality controls.

  • Establish rigorous evaluation through offline benchmarks, backtests, calibration, drift monitoring, prediction-to-outcome comparisons, and explicit failure or abstention behavior.

  • Turn models into durable product, API, and agent workflows that move from an insight to experiment design, approval-gated action, and measured learning.

  • Partner deeply with data science and product teams on experiment design, causal inference, sequential decision-making, variance reduction, and the boundary between prediction and causal evidence.

  • Build reliable services and intuitive workflows so sophisticated ML capabilities are understandable and useful to teams making high-stakes product decisions.

  • Provide technical leadership across engineering, product, data science, and research partners, and raise the bar for production ML quality across the platform.

You Might Thrive In This Role If You

  • Have led ambiguous 0-to-1 production ML products where success was measured by better real-world decisions, not only offline model metrics.

  • Have strong hands-on experience across the ML lifecycle: dataset design, training or adaptation, evaluation, deployment, monitoring, and iteration.

  • Bring depth in one or more of LLM and retrieval systems, ranking or recommendation, forecasting or anomaly detection, causal ML or experiment analysis, or simulation. You do not need to have done all of them.

  • Have strong software engineering fundamentals and can build high-quality production systems in Python while working comfortably across data, backend, and platform boundaries.

  • Have a strong grounding in machine learning, statistics, computer science, or a related field through formal study or equivalent practical experience.

  • Understand experimentation and statistical reasoning, especially why predictive accuracy is not the same as causal validity.

  • Treat calibration, uncertainty, provenance, privacy, and human review as product requirements, not cleanup work.

  • Can translate ambiguous partner questions into a product and technical roadmap, and work well with product, data science, research, and infrastructure partners.

  • Enjoy building for internal power users and agents, and can make sophisticated ML capabilities feel clear and actionable.

  • Value in-person collaboration and want to help shape a growing Bellevue-based team.

Location and Workplace

This role is based in Bellevue, Washington. The team works in person and uses that time to move quickly, solve ambiguous problems together, and stay close to the product teams we support.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. 

We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic.

For additional information, please see OpenAI’s Affirmative Action and Equal Employment Opportunity Policy Statement.

Background checks for applicants will be administered in accordance with applicable law, and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws, including the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act, for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft, loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary, confidential, and non-public information. In addition, job duties require access to secure and protected information technology systems and related data security obligations.

To notify OpenAI that you believe this job posting is non-compliant, please submit a report through this form. No response will be provided to inquiries unrelated to job posting compliance.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Interview prep

5 guides for OpenAI

Apply for this position

Redirects to OpenAI's application page.

Other roles

More at OpenAI.

View all 127 roles