LiteLLM

AI Engineer

LiteLLM4 weeks ago
Location

San Francisco

Type

Full Time

Salary

USD 170,000 – 220,000

Level

Mid

Role

AI Engineer

Posted

Mar 4, 2026

Full TimeMid

The role

Summary

LiteLLM is seeking an experienced AI Engineer to enhance their open-source Python SDK and FastAPI Server, which provides a unified gateway for accessing 100+ LLM APIs. The ideal candidate will work on critical system design projects, improve LLM interoperability, and support enterprise-level features for developers worldwide.

What you'll do

LLM API Unification: Design and implement transformations to convert API requests across different LLM provider formats, ensuring compatibility with OpenAI and Anthropic specifications
System Design: Develop provider-level features at the gateway level, including skills management, request compaction, and batch processing
Developer Experience: Enhance the LiteLLM Python SDK to support millions of users, focusing on intuitive and efficient API interactions
Interoperability: Maintain an excellent LLM interoperability layer, supporting multiple MCP authentication flows and enabling diverse agent use cases

What we look for

Technical

Backend Development1-2 years of experience with production backend systems
API IntegrationProven experience working with OpenAI API, understanding nuanced API specifications
System ArchitectureAbility to design scalable and efficient system architectures for LLM gateways

Education

Computer ScienceBachelor's degree in Computer Science, Software Engineering, or related technical field preferred

Experience

Production SystemsDemonstrated experience in developing and maintaining production-level software systems
Open SourceActive participation and contribution to open-source projects

Skills

Required skills

PythonCore programming language for backend development and LLM interactions
LLM APIsDeep understanding of Large Language Model API integrations and specifications
FastAPIWeb framework for building APIs with Python

Nice to have

OpenAI APIFamiliarity with OpenAI API specifications and nuanced API interactions
System DesignExperience in designing scalable backend systems for enterprise-level applications
API IntegrationExpertise in unifying and transforming API request formats across different providers

Compensation & benefits

Salary

USD 170,000 – 220,000 (annual)

Benefits

Open Source Impact

Opportunity to contribute to a widely-used open-source project with global enterprise adoption

Cutting-Edge Technology

Work with advanced AI technologies and large language model integrations

Startup Environment

Collaborative work culture with direct interaction with company leadership

Interview process

  1. 1
    Intro Call — 30-minute behavioral discussion covering working style, expectations, and company direction
  2. 2
    Hackerrank Assessment — 1-hour technical assessment focusing on Python programming skills
  3. 3
    Virtual Onsite — 3-hour virtual interview with founders, including collaborative GitHub issue resolution, technical project presentation, and system design problem-solving
Apply for this position

You'll be redirected to the company's application page

LiteLLM

LiteLLM

View all jobs

LiteLLM is a platform that provides a unified interface for accessing multiple large language models (LLMs) from different providers. The company offers tools and infrastructure that enable developers and organizations to seamlessly integrate various AI models into their applications while managing costs, performance, and deployment complexity. LiteLLM operates in the artificial intelligence and developer tools market, serving businesses that need flexible access to different language models without being locked into a single provider's ecosystem.

litellm.ai

Tech Stack

Languages
Python
Frameworks
FastAPI
Databases
PostgresRedis
Tools
LLM APIs
Other
Open Source
Apply Now

On this page