ChatGPT Performance Engineer

Performance Engineer · Senior · Full Time

San FranciscoUSD 325k – 405k4mo ago
Apply for this role

Opens OpenAI's application page

Role

What you'll do.

OpenAI is seeking an experienced Performance Engineer to optimize the performance, reliability, and efficiency of mission-critical systems powering ChatGPT and the OpenAI API. This highly technical individual contributor role requires deep expertise in systems optimization, performance profiling, and distributed systems scaling. You'll work cross-functionally to drive latency, throughput, and cost-efficiency improvements across the entire technology stack, from GPU utilization to networking and application runtime optimization.

Responsibilities

  • Multi-Layer Performance Analysis and Optimization: Conduct comprehensive performance profiling and optimization across application, middleware, runtime, and infrastructure layers including networking, storage, Python runtime, GPU utilization, and beyond. Utilize advanced tracing and observability tools to identify bottlenecks and implement systematic improvements that directly impact system efficiency and user experience.
  • Observability and Instrumentation Development: Design and implement sophisticated tooling and metrics systems that provide deep observability into system performance across distributed environments. Create dashboards, logging frameworks, and monitoring solutions that enable real-time visibility into critical performance indicators and emerging degradation patterns.
  • Cross-Functional Collaboration and Architecture Influence: Partner closely with infrastructure, platform, training, and product teams to identify key performance goals, define SLAs/SLOs, and drive systemic improvements. Influence critical architecture and design decisions at scale to prioritize latency, throughput, and efficiency from inception rather than as post-hoc optimizations.
  • Production Performance Investigation and Resolution: Lead root-cause analysis investigations into high-impact performance regressions, scalability issues, and production incidents affecting mission-critical services like ChatGPT and the OpenAI API. Develop remediation strategies and implement fixes that prevent recurrence while documenting findings for organizational learning.
  • Performance Testing and SLA/SLO Definition: Design and execute comprehensive performance testing strategies for critical systems operating at scale. Establish and maintain Service Level Agreements (SLAs) and Service Level Objectives (SLOs) around latency and throughput, ensuring all stakeholders have clear performance expectations and measurement frameworks.
  • Scalability and Efficiency Optimization: Apply deep technical expertise to push latency, throughput, and cost-efficiency to the next level across mission-critical products. Identify and eliminate performance bottlenecks that prevent optimal resource utilization and system scaling, with particular focus on high-impact opportunities in large-scale distributed systems.

Qualifications

What we look for.

Technical

  • Performance Profiling and Tracing Tools Expertise

    Advanced proficiency with performance profiling tools, distributed tracing systems, and APM solutions. Demonstrated ability to identify performance bottlenecks and interpret profiling output across multiple platforms and architectures.

  • Multi-Layer Stack Optimization Experience

    Proven experience optimizing performance across one or more layers including database query optimization, network protocols and latency reduction, storage IO patterns, application runtime tuning, garbage collection configuration, Python/Golang internals, CUDA optimization, and GPU utilization maximization.

  • Operating Systems and Systems-Level Understanding

    Strong foundational knowledge of OS internals, CPU scheduling, context switching, memory management hierarchy (cache, virtual memory, NUMA), and IO patterns. Ability to analyze system behavior at the kernel level and optimize for efficient resource utilization.

  • Distributed Systems and Infrastructure Knowledge

    Deep understanding of distributed systems architecture, including load balancing, service mesh patterns, replication strategies, and consistency tradeoffs. Experience with large-scale infrastructure systems and the performance implications of distributed design decisions.

  • Observability Infrastructure Implementation

    Experience building or contributing to observability systems at scale, including metrics collection, distributed tracing, logging aggregation, and performance dashboards. Familiarity with open-source and commercial observability platforms.

Education

  • Computer Science or Related Field

    Bachelor's degree in Computer Science, Computer Engineering, or related technical field. Advanced degrees in systems, networks, or performance engineering are a plus but equivalent professional experience may substitute.

Experience

  • 7+ Years Software Engineering Experience

    Minimum 7 years of professional software engineering experience with demonstrated expertise in performance optimization, reliability engineering, or systems engineering roles at technology companies with significant scale requirements.

  • High-Scale Distributed Systems Track Record

    Proven track record optimizing performance and reliability of high-scale distributed systems handling significant traffic volume or computational load. Experience navigating ambiguity and aligning multiple stakeholders around performance goals and competing priorities.

  • Benchmark and Performance Testing Background

    Demonstrated success contributing to benchmarking frameworks, performance testing infrastructure, or performance-focused optimization initiatives at scale. Experience establishing baselines and measuring improvements quantitatively.

Skills

Required

  • Performance Profiling

    Expertise with flame graphs, sampling profilers, and continuous profiling tools to identify hot paths and CPU bottlenecks in production systems.

  • Distributed Tracing

    Proficiency with distributed tracing systems to understand request flows, latency attribution, and dependency analysis across microservices.

  • Python Performance Optimization

    Deep knowledge of Python runtime internals, GIL implications, memory profiling, and optimization techniques specific to Python-based systems.

  • Systems-Level Debugging

    Ability to use kernel-level tools (strace, perf, BPF) to investigate system behavior and identify performance bottlenecks at the OS level.

  • Infrastructure and Networking

    Understanding of networking protocols, TCP/UDP optimization, DNS resolution impacts, and infrastructure-level performance considerations.

  • Load and Stress Testing

    Experience designing and executing load tests, stress tests, and chaos engineering experiments to validate system behavior under real-world conditions.

  • SQL Query Optimization

    Proficiency with database query analysis, index optimization, and query execution plan interpretation for relational databases.

  • Cross-Functional Communication

    Ability to communicate complex technical findings to non-technical stakeholders and align teams around performance goals and tradeoffs.

Preferred

  • GPU Performance Optimization

    Nice to have

    Experience optimizing CUDA-based systems or working with GPU workloads, understanding memory transfers, kernel execution, and efficiency improvements.

  • Compiled Language Performance

    Nice to have

    Background with Go, Rust, or C++ performance optimization, including knowledge of compiler flags, SIMD optimization, and low-level tuning.

  • Machine Learning Systems Knowledge

    Nice to have

    Understanding of ML inference and training systems, including model serving frameworks, throughput optimization, and latency reduction in ML pipelines.

  • Open-Source Observability Contributions

    Nice to have

    Contributions to open-source observability tools such as Prometheus, Grafana, Jaeger, or similar platforms demonstrating commitment to instrumentation excellence.

  • Kubernetes and Container Orchestration

    Nice to have

    Experience optimizing performance of containerized systems and Kubernetes deployments, including resource allocation and scheduling efficiency.

  • Large Language Model Optimization

    Nice to have

    Familiarity with LLM serving infrastructure, inference optimization techniques, token processing efficiency, and real-time API performance considerations.

Tech stack

Languages

PythonGoC/C++

Frameworks

PyTorch or TensorFlowvLLM or Similar Serving Frameworks

Databases

PostgreSQLRedisTime-Series Databases

Tools

Flame Graphs and PerfDistributed Tracing PlatformsPrometheus and GrafanaBPF and eBPFLoad Testing FrameworksKubernetes

Other

GPU Architecture and CUDAMicroservices Architecture PatternsContinuous Integration and Deployment

Compensation

Pay and benefits.

Base·USD 325,000 – 405,000

Equity·Stock options

Benefits

  • Equity Compensation

    Competitive stock options package aligned with company performance and your individual contributions, providing meaningful ownership and upside participation as OpenAI advances its mission.

  • Comprehensive Health Coverage

    Extensive medical, dental, and vision insurance plans covering employees and dependents with low or no out-of-pocket costs for preventative care and specialist services.

  • Retirement Planning

    401(k) plan with company matching contributions to support long-term financial security and retirement savings with tax-advantaged growth opportunities.

  • Unlimited PTO

    Flexible paid time off policy recognizing the importance of work-life balance, wellbeing, and personal recovery time to maintain high performance and job satisfaction.

  • Professional Development and Learning

    Budgets and support for technical conferences, specialized training, certification programs, and continuous learning opportunities to stay current with advancing technologies and industry practices.

  • Mental Health and Wellness Support

    Access to mental health services, counseling, wellness programs, and employee assistance programs supporting overall wellbeing and resilience in demanding technical roles.

  • Parental Leave

    Generous parental leave policies supporting new parents with extended paid leave and flexible return-to-work arrangements.

  • Home Office Equipment

    Support for remote work infrastructure including equipment stipends, ergonomic setups, and technology allowances to create productive work environments.

  • Commuter and Relocation Benefits

    Flexible commuting support in San Francisco area including transit passes or parking benefits, plus relocation assistance for candidates relocating to the Bay Area.

Process

Interview steps.

  1. 01

    Initial Technical Screening

    Brief conversation with recruiting team to validate background, discuss your performance engineering experience, and assess cultural fit with OpenAI's mission-driven approach. This stage confirms baseline qualifications and enthusiasm for the role.

  2. 02

    Technical Deep-Dive Interview

    Comprehensive discussion with engineering leaders covering your hands-on experience with performance profiling tools, specific optimization projects you've led, and your approach to root-cause analysis. Expect questions about architecture trade-offs, instrumentation strategies, and real-world performance challenges you've solved.

  3. 03

    Systems Design and Problem-Solving

    Technical interview focused on your ability to reason about large-scale systems, identify performance bottlenecks, and propose optimization strategies. You may be presented with performance scenarios, latency issues, or scaling challenges to demonstrate your analytical approach and systems thinking.

  4. 04

    Cross-Functional Collaboration Discussion

    Conversation with members of the infrastructure, platform, and product teams to assess your ability to navigate ambiguity, communicate with diverse stakeholders, and influence architectural decisions. OpenAI values collaborative problem-solvers who can align teams around performance goals.

  5. 05

    Leadership and Impact Assessment

    Discussion with senior engineering leadership evaluating your track record of driving measurable business impact through performance improvements. Expect conversations about prioritization under constraints, how you've navigated competing stakeholder interests, and your philosophy on technical rigor and simplicity.

  6. 06

    Offer and Onboarding Discussion

    Final conversation covering compensation, benefits, equity details, and technical onboarding expectations. OpenAI will discuss team structure, current performance priorities, and how you'll ramp up on their systems and culture.

Full posting

Original listing.

About the Team

We bring OpenAI's technology to the world through products like ChatGPT and the OpenAI API.

We seek to learn from deployment and distribute the benefits of AI, while ensuring that this powerful tool is used responsibly and safely. Safety is more important to us than unfettered growth.

About the Role

OpenAI is looking for an experienced Performance Engineer to help us scale the performance, reliability, and efficiency of our systems. In this role, you'll apply deep technical expertise to optimize infrastructure and application-level performance across mission-critical products like ChatGPT and our developer API. You’ll work cross-functionally with teams building core services, training models, and developing real-time user experiences to push our latency, throughput, and cost-efficiency to the next level.

We are looking for engineers who thrive in ambiguous environments, value deep systems understanding, and are motivated by delivering measurable impact. This is a highly technical, individual contributor role focused on root-cause analysis, profiling, instrumentation, and architecture-level performance improvements across our stack.

In this role, you will:

  • Analyze and optimize performance across application, middleware, runtime, and infrastructure layers—networking, storage, Python runtime, GPU utilization, and beyond.

  • Develop tooling and metrics that provide deep observability into system performance.

  • Collaborate closely with infra, platform, training, and product teams to identify key performance goals and drive systemic improvements.

  • Influence architecture and design decisions to prioritize latency, throughput, and efficiency at scale.

  • Lead investigations into high-impact performance regressions or scalability issues in production.

  • Drive performance testing strategies and help define SLAs/SLOs around latency and throughput for critical systems.

You might thrive in this role if you:

  • Have 7+ years of experience in software engineering with a strong track record in performance or reliability of high-scale distributed systems.

  • Are deeply comfortable with performance profiling tools and tracing systems.

  • Have experience optimizing performance across one or more layers of the stack (e.g., database, networking, storage, application runtime, GC tuning, Python/Golang internals, GPU utilization).

  • Have a strong understanding of OS internals, scheduling, memory management, and IO patterns.

  • Have contributed to observability, benchmarking, or performance-focused infrastructure at scale.

  • Have demonstrated success navigating ambiguity and aligning stakeholders around performance goals.

  • Value simplicity, rigor, and collaboration when solving complex systems problems.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. 

We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic.

For additional information, please see OpenAI’s Affirmative Action and Equal Employment Opportunity Policy Statement.

Background checks for applicants will be administered in accordance with applicable law, and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws, including the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act, for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft, loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary, confidential, and non-public information. In addition, job duties require access to secure and protected information technology systems and related data security obligations.

To notify OpenAI that you believe this job posting is non-compliant, please submit a report through this form. No response will be provided to inquiries unrelated to job posting compliance.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Redirects to OpenAI's application page.

Other roles

More at OpenAI.

View all 108 roles