Scale Engineer

Rust Engineer

Senior · Full Time

San FranciscoUSD 200k – 260k2mo ago
Apply for this role

Opens LiteLLM's application page

Role

What you'll do.

LiteLLM is seeking a highly skilled Rust Engineer to lead a critical migration from Python to Rust, focusing on building the world's fastest AI gateway. The ideal candidate will own the end-to-end migration process, driving performance improvements and ensuring system stability for enterprise-level AI infrastructure.

Responsibilities

  • Migration Architecture: Design and plan the comprehensive migration strategy from Python to Rust, working closely with the CTO
  • Performance Optimization: Drive performance improvements across latency, throughput, and resource efficiency for 1B+ monthly requests
  • Incremental Rollout: Execute a gradual, stability-first migration approach to minimize risks to live, high-volume traffic
  • Codebase Restructuring: Prepare and restructure the existing codebase to facilitate a clean, incremental transition to Rust
  • Cross-Functional Collaboration: Ensure migration aligns with backend priorities in performance, reliability, and security

Qualifications

What we look for.

Technical

  • Rust Proficiency

    Extensive experience in production-level Rust development

  • Performance Engineering

    Deep understanding of system performance optimization techniques

  • Backend Infrastructure

    Comprehensive knowledge of backend systems and high-volume production environments

Education

  • Computer Science

    Bachelor's or Master's degree in Computer Science, Software Engineering, or related technical field

Experience

  • Rust Development

    Minimum 3-5 years of professional experience writing production Rust code

  • System Migration

    Proven track record of successful large-scale system migrations

  • Performance-Critical Systems

    Experience developing and maintaining high-performance, scalable systems

Skills

Required

  • Rust Programming

    Deep expertise in production-level Rust development with years of experience

  • Performance Optimization

    Proven ability to improve system latency, throughput, and resource efficiency

  • System Migration

    Experience in large-scale codebase migration and incremental system transitions

  • Backend Infrastructure

    Strong understanding of backend systems, high-volume production environments, and database technologies

Preferred

  • Language Migration

    Nice to have

    Prior experience leading migrations from another language to Rust

  • Performance-Critical Systems

    Nice to have

    Track record of developing and shipping high-performance system architectures

  • Database Technologies

    Nice to have

    Familiarity with PostgreSQL, Redis, and distributed system databases

Tech stack

Languages

RustPython

Databases

PostgreSQLRedis

Tools

GitHubCI/CD

Other

AI GatewayOpenAI Format

Compensation

Pay and benefits.

Base·USD 200,000 – 260,000

Equity·Stock options

Benefits

  • Competitive Salary

    Salary range of $200,000 - $260,000 per year

  • Equity

    Stock options or equity compensation

  • Healthcare

    Comprehensive health, dental, and vision insurance

  • Technical Ownership

    Opportunity to define the architectural foundation for a cutting-edge AI platform

  • Fast-Paced Environment

    Work in a dynamic, growth-oriented startup with significant technical challenges

Process

Interview steps.

  1. 01

    Initial Screening

    Phone or video call with recruiting team to assess background and initial fit

  2. 02

    Technical Assessment

    Comprehensive Rust and system design technical interview

  3. 03

    Performance Migration Challenge

    Practical coding challenge focusing on system migration and optimization

  4. 04

    CTO Interview

    In-depth discussion about migration strategy and technical approach

  5. 05

    Final Interview

    Team fit and final evaluation with engineering leadership

Full posting

Original listing.

Rust Engineer

LiteLLM is the world's most popular AI Gateway, trusted by top companies like Adobe, Netflix, and NASA. Our platform empowers developers by providing secure, reliable access to LLMs and adjacent services. We're looking for a Rust Engineer to own the migration of LiteLLM from Python to Rust and make us the fastest AI gateway ever built.

About The Role

You'll own LiteLLM's migration to Rust end to end. This is a high-stakes, high-visibility role: you'll plan the migration directly with the CTO, structure the codebase so it's ready for Rust, and lead a gradual, stability-first rollout. The work is fundamentally about performance. Every decision you make will move us toward being the lowest-latency, highest-throughput gateway in the category, without ever compromising the reliability our enterprise customers depend on.

This is a single-owner mandate. You will be the person leading the entire Python-to-Rust migration.

Responsibilities

  • Own the end-to-end migration of LiteLLM from Python to Rust, from architecture through rollout.

  • Plan the migration alongside the CTO, sequencing work to minimize risk to live, high-volume traffic.

  • Design and execute a gradual rollout that holds stability as the top constraint while the migration progresses.

  • Restructure the existing codebase so it's positioned for a clean, incremental move to Rust.

  • Drive performance gains across latency, throughput, and resource efficiency at 1B+ requests/mo.

  • Collaborate cross-functionally to ensure the migration aligns with backend priorities across performance, reliability, and security.

What We're Looking For

  • Deep, long-standing expertise in Rust. You've been writing production Rust for years and know it cold.

  • Obsessive about performance. You care about latency and throughput at a level most engineers don't.

  • Strong instinct for stability, with a track record of balancing ambitious change against reliability.

  • Major bonus: prior experience leading codebase migrations from another language to Rust.

  • Major bonus: a demonstrated history of shipping performance-critical systems.

  • Familiarity with backend infrastructure, databases (PostgreSQL, Redis), and high-volume production systems.

Why Join LiteLLM?

  • Own one of the most important technical bets in the company's roadmap.

  • Define the architectural foundation that the entire platform will run on.

  • Fast-paced growth environment with deep technical ownership.

  • Competitive salary, health, dental, and vision benefits.

About LiteLLM

LiteLLM (https://github.com/BerriAI/litellm) is a Python SDK and Proxy Server enabling seamless calls to 100+ LLM APIs in the OpenAI format, trusted by industry leaders worldwide.

Ready to make LiteLLM the fastest AI gateway ever? Apply now!

Apply for this position

Redirects to LiteLLM's application page.

Other roles

More at LiteLLM.

View all 7 roles