Scale Engineer

Software Engineer, Compute Infra

Backend Engineer · Senior · Full Time

San FranciscoUSD 165k – 250k3d ago
Apply for this role

Opens Cursor's application page

Role

What you'll do.

Join SpaceXAI's Compute Infrastructure team as a Software Engineer to architect and build large-scale sandbox infrastructure powering millions of concurrent VMs across distributed datacenters. You'll design mission-critical systems including virtualization layers, control plane schedulers, and networking infrastructure supporting coding agents, reinforcement learning, and advanced AI applications at unprecedented scale.

Responsibilities

  • Control Plane and Scheduler Architecture: Lead the rearchitecture of the distributed control plane and scheduler to support 1 million concurrent VMs per cluster, implementing sophisticated resource allocation algorithms and managing cluster state across multiple datacenters
  • Storage and Block Device Development: Design and implement advanced ublk block device implementations with support for content deduplication, lazy loading mechanisms, and efficient snapshot management to optimize VM provisioning and data storage efficiency
  • Network Architecture and Security: Architect and develop egress proxy systems and network isolation layers that securely filter, monitor, and intelligently route agent traffic while maintaining low latency and high throughput across infrastructure
  • Multi-Datacenter Resilience Engineering: Design resilient, fault-tolerant systems capable of operating across hybrid infrastructure combining AWS, GCP, and on-premise datacenters with automatic recovery and state consistency
  • Performance Optimization and Monitoring: Continuously optimize infrastructure performance metrics including VM startup latency, scheduling efficiency, and resource utilization while implementing comprehensive observability and alerting systems
  • Infrastructure Primitive Development: Anticipate and build infrastructure primitives and abstractions that future AI agents will require, designing flexible APIs and patterns that enable innovative applications
  • Cross-Functional Collaboration: Work closely with AI/ML teams, platform engineers, and product teams to understand workload requirements and translate them into efficient infrastructure solutions

Qualifications

What we look for.

Technical

  • Virtualization Layer Architecture

    Design and implement virtualization stacks using microVM technologies (Firecracker, Cloud Hypervisor) supporting multi-million scale concurrent instances

  • Scheduler and Control Plane Development

    Build highly scalable scheduling systems managing 1M+ concurrent VMs with intelligent resource allocation and bin-packing algorithms

  • Block Device and Storage Optimization

    Develop ublk block device implementations with advanced features including content deduplication, lazy loading, and efficient snapshot management

  • Network Infrastructure and Isolation

    Design egress proxies and network isolation layers providing secure routing, traffic monitoring, and policy enforcement for agent workloads

  • Multi-Datacenter Resilience

    Engineer systems capable of operating seamlessly across hybrid cloud and on-premise infrastructure with automatic failover and consistency guarantees

  • Performance Optimization

    Optimize infrastructure for minimal latency, maximum throughput, and efficient resource utilization across millions of CPU cores

Education

  • Bachelor's Degree in Computer Science, Engineering, or Related Field

    Strong theoretical foundation in computer science, algorithms, mathematics, or engineering disciplines providing deep systems thinking capability

Experience

  • Compute Orchestration Platform Development

    2+ years of hands-on experience building, optimizing, or operating compute orchestration systems like Kubernetes or Mesos at meaningful scale

  • Large-Scale Infrastructure Operations

    Demonstrated experience building, deploying, and operating infrastructure supporting thousands to millions of concurrent workloads

  • Virtualization and Sandbox Systems

    Proven track record building or operating VM fleets, container systems, or isolated sandbox environments for running untrusted or multi-tenant code

  • Backend Systems Engineering

    Professional software engineering experience (2+ years) building distributed systems, databases, or infrastructure platforms in production environments

Skills

Required

  • Distributed Systems Design

    Design and implement scalable, fault-tolerant systems that operate reliably across multiple datacenters and cloud providers

  • Linux Kernel and Virtualization

    Proficiency with kernel internals, system calls, container technologies, and hypervisor-level abstractions for building sandboxed environments

  • Infrastructure as Code

    Experience building and managing infrastructure through code using Terraform, Ansible, or similar tools for reproducible deployments

  • Systems Programming

    Strong foundation in low-level programming using Rust, C, or C++ to optimize performance-critical infrastructure components

  • Cluster Operations and Orchestration

    Hands-on experience operating Kubernetes, Mesos, or similar orchestration platforms at scale

  • Networking and Security

    Understanding of network protocols, firewalls, proxy systems, and security isolation techniques for multi-tenant sandbox environments

Preferred

  • MicroVM and Firecracker Experience

    Nice to have

    Previous experience working with lightweight virtualization technologies and Firecracker specifically

  • eBPF and Network Programming

    Nice to have

    Advanced networking expertise using eBPF for packet filtering, monitoring, and custom network layer functionality

  • Large-Scale Fleet Management

    Nice to have

    Operational experience managing compute fleets with millions of cores across distributed infrastructure

  • Sandbox and VM Ecosystem

    Nice to have

    Background building and operating VM isolation systems for coding environments, testing platforms, or AI agent infrastructure

  • Storage Systems Optimization

    Nice to have

    Experience with block device optimization, deduplication techniques, snapshot systems, and lazy-loading data structures

  • ML Infrastructure and Reinforcement Learning

    Nice to have

    Understanding of infrastructure requirements for training and inference workloads, especially for agent-based systems

  • On-Premise and Hybrid Cloud Operations

    Nice to have

    Experience designing systems that span multiple deployment models including data centers and public cloud providers

Tech stack

Languages

RustGoC/C++Python

Frameworks

KubernetesgRPCetcd

Databases

etcdRocksDB

Tools

Linux KernelFirecrackerCloud HypervisoreBPFPrometheusGit

Other

Virtualization TechnologiesNetwork ProtocolsDistributed SystemsCloud Architecture

Compensation

Pay and benefits.

Base·USD 165,000 – 250,000

Equity·Stock options

Benefits

  • Equity Compensation

    Significant stock option grants reflecting senior-level contribution to company success in rapidly growing AI infrastructure sector

  • Comprehensive Health Coverage

    Full medical, dental, and vision insurance plans with company contributions for employee and eligible dependents

  • 401(k) Retirement Plan

    Competitive retirement savings plan with company matching contributions for long-term financial security

  • Professional Development

    Learning budget for conferences, courses, and certifications including Linux kernel deep-dives and infrastructure optimization workshops

  • Flexible Work Arrangement

    Flexible schedules supporting work-life balance for high-performance engineering teams

  • Hardware and Tools

    Top-tier workstations, development machines, and access to infrastructure lab environments for hands-on system development

  • Unlimited PTO

    Flexible paid time off policy supporting personal well-being and recovery

Process

Interview steps.

  1. 01

    Initial Technical Screening

    30-minute call with recruiting team to discuss background in distributed systems, virtualization experience, and alignment with infrastructure engineering challenges

  2. 02

    Systems Design Interview

    90-minute technical interview assessing your ability to architect large-scale systems, handle design tradeoffs, and communicate complex infrastructure solutions

  3. 03

    Infrastructure Coding Challenge

    Take-home coding assessment focusing on systems programming, likely involving control plane logic, scheduling algorithms, or networking layer implementation in Rust or C++

  4. 04

    Deep Dive Infrastructure Discussion

    60-minute technical conversation with senior infrastructure engineers exploring your experience with compute orchestration, virtualization, and operational challenges at scale

  5. 05

    Leadership and Values Alignment

    30-minute conversation with team lead or manager discussing career goals, engineering philosophy, and fit with SpaceXAI's mission-driven culture

Full posting

Original listing.

SpaceX was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today, SpaceX is actively developing the technologies to make this possible, with the ultimate goal of making life multiplanetary and expanding consciousness to the stars.

SpaceXSI is building Super Intelligence to help humanity understand the universe. Our teams are building the full Super Intelligence stack: from our Colossus supercomputers, to Grok models, to our suite of applications, including Grok Bot, Grok Build, Grok Imagine, Grok Voice, and X. We're creating a powerful flywheel where more compute trains better models, better models power more useful products, and what we learn from people using those products shapes what we build next.

About the Role

As a Software Engineer on Compute Infra at SpaceXAI, you'll build the sandbox infrastructure to give every agent their own VM. These VMs power coding agents, Grok Bot, reinforcement learning, and more. We operate one of the largest set of sandbox clusters in the world, at millions of CPU cores, across third- and first-party datacenters. You’ll own developing a cloud for agents, including the virtualization layer (Firecracker and Cloud Hypervisor microVMs), control plane scaling, and the networking/connectivity layer on top.

Responsibilities

  • Rearchitecting the control plane / scheduler to support 1 million concurrent VMs per cluster.

  • Building a ublk block device that can dedupe, lazy load, and efficiently snapshot VM data.

  • Designing an egress proxy and network isolation to securely filter, monitor, and route agent traffic.

  • Designing resilient systems that can operate across a mixture of cloud and on-premise datacenters.

Basic Qualifications

  • Bachelor's degree in computer science, engineering, math, or scientific discipline; OR 2+ years of professional experience building software in lieu of a degree

Preferred Skills and Experience

  • You have deep experience with building compute orchestration (e.g. Kubernetes, Mesos).

  • You have built and operated large fleets of VMs / isolated sandboxes.

  • You have experience with the Linux kernel and virtualization technologies.

  • You enjoy thinking about infrastructure primitives that future agents will require.


Apply for this position

Redirects to Cursor's application page.

Other roles

More at Cursor.

View all 39 roles