Software Engineer, Model Routing & Inference

Cursor3 days ago

Location

New York

Type

Full Time

Salary

USD 180,000 – 250,000

Level

Senior

Role

Software Engineer

Posted

Apr 7, 2026

Full TimeSenior

The role

Summary

Cursor is seeking a highly skilled Software Engineer to join their Model Routing & Inference team, responsible for building a cutting-edge inference platform that powers AI interactions across their product. The ideal candidate will design and implement high-performance, reliable distributed systems for routing and serving AI model inference at massive scale.

What you'll do

Inference Platform Development: Build and evolve the inference gateway that provides a unified abstraction over multiple AI model provider APIs, enabling seamless model onboarding and configuration.

System Reliability Engineering: Design intelligent cross-provider failover mechanisms to ensure high availability and minimize user-facing service disruptions.

Traffic Management: Implement advanced routing backpressure and admission control strategies to manage traffic spikes and optimize system performance.

Scalability Optimization: Analyze and optimize GPU utilization, provider economics, and capacity planning to improve system cost-effectiveness and performance.

What we look for

Technical

Distributed SystemsExtensive experience building high-throughput, low-latency distributed systems with a focus on inference serving or real-time data pipelines

Performance OptimizationProven ability to reason about complex performance and cost tradeoffs at large-scale infrastructure

Education

Computer Science DegreeBachelor's or Master's degree in Computer Science, Software Engineering, or related technical field

Experience

Production SystemsDemonstrated track record of shipping robust production systems handling millions of requests

AI InfrastructureStrong background in AI model inference, routing, and infrastructure technologies

Skills

Required skills

Distributed SystemsDeep expertise in designing and implementing high-performance distributed systems

System DesignAdvanced system design skills with ability to make nuanced architectural decisions

Nice to have

Machine Learning InfrastructureExperience with AI model serving, routing, and infrastructure technologies

Cloud PlatformsFamiliarity with cloud infrastructure and scalable computing environments

Compensation & benefits

Salary

USD 180,000 – 250,000 (annual)

Stock options

Available

Benefits

Competitive Compensation

Highly competitive salary with potential equity compensation

Innovative Work Environment

Opportunity to work on cutting-edge AI infrastructure with a talented, passionate team

Professional Growth

Flat organizational structure with opportunities for rapid skill development and impact

Interview process

1
Initial Screening — Preliminary review of candidate's background and qualifications
2
Technical Interviews — 2-3 focused technical interviews assessing system design and problem-solving skills
3
Onsite Interview — In-office project work, technical discussions, and team meetings