LiteLLM

Senior Backend Engineer

LiteLLMYesterday
Location

San Francisco

Type

Full Time

Salary

USD 170,000 – 230,000

Level

Senior

Role

Backend Engineer

Posted

Mar 31, 2026

Full TimeSenior

The role

Summary

LiteLLM is seeking a Senior Backend Engineer to enhance their AI Gateway platform's guardrails, observability, and logging infrastructure. The ideal candidate will develop robust backend systems that ensure secure, traceable, and high-performance interactions with large language models (LLMs) for enterprise customers.

What you'll do

Backend System Development: Build and scale product infrastructure ensuring high performance and reliability
Logging and Traceability: Implement comprehensive logging for guardrail and policy enforcement calls
Security Implementation: Design CPU-level guardrails to protect against common LLM API attacks
Error Handling: Develop robust error handling mechanisms with transparent user feedback
Observability Integration: Configure and maintain monitoring solutions for backend systems handling 1B+ monthly requests

What we look for

Technical

Backend DevelopmentExpertise in building scalable backend systems with Python
ObservabilityAdvanced knowledge of monitoring and logging platforms

Education

Computer Science DegreeBachelor's or Master's in Computer Science or related technical field

Experience

Backend Engineering4+ years of experience with Python and backend web frameworks
Systems ObservabilityProven track record in implementing comprehensive logging and monitoring solutions

Skills

Required skills

PythonPrimary backend programming language with extensive experience
Backend FrameworksProficiency in FastAPI or Flask for building scalable web services
Database TechnologiesExperience with PostgreSQL, Redis, and database integration
Observability ToolsExpertise in Datadog, Splunk, Prometheus, and OpenTelemetry

Nice to have

AI InfrastructureUnderstanding of LLM API integration and security
Error HandlingAdvanced techniques for robust logging and error tracing
System PerformanceExperience optimizing high-volume request processing

Compensation & benefits

Salary

USD 170,000 – 230,000 (annual)

Stock options

Available

Benefits

Health Insurance

Comprehensive medical, dental, and vision coverage

Equity

Stock options in a high-growth AI infrastructure startup

Technical Growth

Opportunity to work on cutting-edge AI and observability technologies

Impact

Contribute to infrastructure used by enterprise customers like Adobe, Netflix, and NASA

Interview process

  1. 1
    Initial Screening — Phone or video call with recruiting team to discuss background and experience
  2. 2
    Technical Assessment — Coding challenge focused on backend development, logging, and system design
  3. 3
    Technical Interview — In-depth discussion of system architecture, observability, and AI infrastructure
  4. 4
    System Design Interview — Evaluate candidate's approach to building scalable, secure backend systems
  5. 5
    Final Interview — Meeting with engineering leadership to assess team fit and long-term potential
Apply for this position

You'll be redirected to the company's application page

LiteLLM

LiteLLM

View all jobs

LiteLLM is a platform that provides a unified interface for accessing multiple large language models (LLMs) from different providers. The company offers tools and infrastructure that enable developers and organizations to seamlessly integrate various AI models into their applications while managing costs, performance, and deployment complexity. LiteLLM operates in the artificial intelligence and developer tools market, serving businesses that need flexible access to different language models without being locked into a single provider's ecosystem.

litellm.ai

Tech Stack

Languages
Python
Frameworks
FastAPIFlask
Databases
PostgreSQLRedis
Tools
DatadogSplunkPrometheusOpenTelemetry
Other
LLM Integration
Apply Now

On this page