LiteLLM

Rust Engineer

LiteLLM2 weeks ago
Location

San Francisco

Type

Full Time

Salary

USD 200,000 – 260,000

Level

Senior

Role

Rust Engineer

Posted

Jun 16, 2026

Full TimeSenior

The role

Summary

LiteLLM is seeking a highly skilled Rust Engineer to lead a critical migration from Python to Rust, focusing on building the world's fastest AI gateway. The ideal candidate will own the end-to-end migration process, driving performance improvements and ensuring system stability for enterprise-level AI infrastructure.

What you'll do

Migration Architecture: Design and plan the comprehensive migration strategy from Python to Rust, working closely with the CTO
Performance Optimization: Drive performance improvements across latency, throughput, and resource efficiency for 1B+ monthly requests
Incremental Rollout: Execute a gradual, stability-first migration approach to minimize risks to live, high-volume traffic
Codebase Restructuring: Prepare and restructure the existing codebase to facilitate a clean, incremental transition to Rust
Cross-Functional Collaboration: Ensure migration aligns with backend priorities in performance, reliability, and security

What we look for

Technical

Rust ProficiencyExtensive experience in production-level Rust development
Performance EngineeringDeep understanding of system performance optimization techniques
Backend InfrastructureComprehensive knowledge of backend systems and high-volume production environments

Education

Computer ScienceBachelor's or Master's degree in Computer Science, Software Engineering, or related technical field

Experience

Rust DevelopmentMinimum 3-5 years of professional experience writing production Rust code
System MigrationProven track record of successful large-scale system migrations
Performance-Critical SystemsExperience developing and maintaining high-performance, scalable systems

Skills

Required skills

Rust ProgrammingDeep expertise in production-level Rust development with years of experience
Performance OptimizationProven ability to improve system latency, throughput, and resource efficiency
System MigrationExperience in large-scale codebase migration and incremental system transitions
Backend InfrastructureStrong understanding of backend systems, high-volume production environments, and database technologies

Nice to have

Language MigrationPrior experience leading migrations from another language to Rust
Performance-Critical SystemsTrack record of developing and shipping high-performance system architectures
Database TechnologiesFamiliarity with PostgreSQL, Redis, and distributed system databases

Compensation & benefits

Salary

USD 200,000 – 260,000 (annual)

Stock options

Available

Benefits

Competitive Salary

Salary range of $200,000 - $260,000 per year

Equity

Stock options or equity compensation

Healthcare

Comprehensive health, dental, and vision insurance

Technical Ownership

Opportunity to define the architectural foundation for a cutting-edge AI platform

Fast-Paced Environment

Work in a dynamic, growth-oriented startup with significant technical challenges

Interview process

  1. 1
    Initial Screening — Phone or video call with recruiting team to assess background and initial fit
  2. 2
    Technical Assessment — Comprehensive Rust and system design technical interview
  3. 3
    Performance Migration Challenge — Practical coding challenge focusing on system migration and optimization
  4. 4
    CTO Interview — In-depth discussion about migration strategy and technical approach
  5. 5
    Final Interview — Team fit and final evaluation with engineering leadership
Apply for this position

You'll be redirected to the company's application page

LiteLLM

LiteLLM

View all jobs

LiteLLM is a platform that provides a unified interface for accessing multiple large language models (LLMs) from different providers. The company offers tools and infrastructure that enable developers and organizations to seamlessly integrate various AI models into their applications while managing costs, performance, and deployment complexity. LiteLLM operates in the artificial intelligence and developer tools market, serving businesses that need flexible access to different language models without being locked into a single provider's ecosystem.

litellm.ai

Tech Stack

Languages
RustPython
Databases
PostgreSQLRedis
Tools
GitHubCI/CD
Other
AI GatewayOpenAI Format
Apply Now

On this page