LiteLLM

Backend Engineer (New Grad)

LiteLLM3 weeks ago
Location

San Francisco

Type

Full Time

Salary

USD 120,000 – 160,000

Level

Junior

Role

Backend Engineer

Posted

Mar 11, 2026

Full TimeJunior

The role

Summary

LiteLLM is seeking a Backend Engineer (New Grad) to enhance their AI Gateway platform's guardrails, logging, and observability systems. The ideal candidate will focus on creating robust backend infrastructure that ensures secure, traceable, and transparent AI service interactions for enterprise-level clients.

What you'll do

Guardrail Logging: Ensure comprehensive logging of guardrail and policy enforcement calls with full traceability
Error Handling: Identify and resolve silent failures in guardrail creation, registration, and policy application
Observability Integration: Implement and maintain monitoring solutions using tools like Datadog, Splunk, and Prometheus
Performance Optimization: Refactor and enhance monitoring integrations to support high-traffic environments
Cross-functional Collaboration: Work with team members to improve backend engineering priorities

What we look for

Technical

Programming ProficiencyStrong skills in Python and backend web frameworks
Observability ToolsExperience with monitoring and logging platforms
Database ManagementFamiliarity with database integration and troubleshooting

Education

Computer Science DegreeBachelor's or Master's in Computer Science or related technical field

Experience

New GraduateRecent graduate with foundational software engineering skills

Skills

Required skills

PythonCore programming language for backend development
FastAPIBackend web framework for building APIs
Backend DevelopmentCreating scalable and reliable server-side systems
Error HandlingImplementing robust error management and logging practices

Nice to have

PrometheusMonitoring and alerting toolkit
OpenTelemetryObservability framework for cloud-native software
PostgreSQLRelational database management
RedisIn-memory data structure store

Compensation & benefits

Salary

USD 120,000 – 160,000 (annual)

Benefits

Health Insurance

Comprehensive health coverage

Dental Insurance

Dental care benefits

Vision Insurance

Vision care coverage

High-Impact Work

Opportunity to contribute to mission-critical AI infrastructure

Technical Ownership

Room for personal growth and technical leadership

Interview process

  1. 1
    Initial Screening — Phone or video call with recruiting team
  2. 2
    Technical Assessment — Coding challenge or technical interview focusing on backend development skills
  3. 3
    Onsite/Virtual Interviews — Multiple rounds of interviews with engineering team members
  4. 4
    Final Interview — Discussion with senior leadership about role fit and company vision
Apply for this position

You'll be redirected to the company's application page

LiteLLM

LiteLLM

View all jobs

LiteLLM is a platform that provides a unified interface for accessing multiple large language models (LLMs) from different providers. The company offers tools and infrastructure that enable developers and organizations to seamlessly integrate various AI models into their applications while managing costs, performance, and deployment complexity. LiteLLM operates in the artificial intelligence and developer tools market, serving businesses that need flexible access to different language models without being locked into a single provider's ecosystem.

litellm.ai

Tech Stack

Languages
Python
Frameworks
FastAPIFlask
Databases
PostgreSQLRedis
Tools
DatadogSplunkPrometheusOpenTelemetry
Other
AI Gateway
Apply Now

On this page