# AI Systems Research and Development Engineer – LLM Inference Systems & Optimization
**Company:** [Snowflake](https://scaleengineer.com/companies/snowflake)
Snowflake AI Research seeks an experienced systems engineer to design and optimize high-performance LLM inference systems spanning distributed serving, GPU kernels, and model-system co-design. This role requires 5+ years in LLM inference, GPU systems, or distributed AI, with hands-on expertise in inference frameworks like vLLM and deep understanding of performance optimization across compute, memory, and networking layers. You will advance state-of-the-art inference optimization techniques—from speculative decoding and adaptive parallelism to KV-cache management and quantization—while leveraging AI-native engineering approaches to accelerate system development and deployment at production scale.
**Role:** AI Systems Research and Development Engineer
**Seniority:** Senior
**Locations:** US-WA-Bellevue
**Salary:** 236000–330000 USD
[Apply](https://jobs.ashbyhq.com/snowflake/9e0ae021-f264-4efb-bc80-26c04bd06df6)
Canonical: https://scaleengineer.com/jobs/snowflake/ai-systems-research-and-development-engineer-llm-inference-systems-optimization
---
## Responsibilities

- Design and Develop High-Performance LLM Inference Systems: Architect and implement production-grade LLM inference systems spanning the complete stack—distributed serving architecture, runtime systems optimization, GPU execution strategies, and performance-critical kernel development. Focus on creating systems that achieve optimal latency, throughput, and resource efficiency across diverse hardware configurations.
- Develop Advanced Inference Optimization Techniques: Research and implement cutting-edge optimization methodologies including speculative and parallel decoding, prefill/decode disaggregation, adaptive parallelism strategies, continuous batching algorithms, KV-cache management approaches, quantization techniques, and communication layer optimization to maximize inference speed and efficiency.
- Explore Model-System Co-Design: Collaborate with model researchers to identify post-training and architectural modifications that unlock substantially more efficient inference. Design approaches that bridge the gap between model capabilities and system performance constraints, enabling novel inference acceleration techniques.
- Build Adaptive and Intelligent Inference Systems: Develop autonomous systems that automatically optimize execution for new model architectures, diverse hardware platforms, varying workload characteristics, and different deployment environments. Implement automated profiling, bottleneck identification, and runtime configuration selection with minimal manual intervention.
- Apply AI-Native Engineering Approaches to Systems Development: Leverage AI as both workload and development tool—using machine learning techniques for automated profiling, configuration search, code generation, experimentation acceleration, runtime strategy selection, debugging automation, and performance tuning to maximize engineering productivity.
- Design and Implement Distributed Inference Strategies: Engineer sophisticated parallelism approaches across GPUs and compute nodes, including tensor parallelism, sequence parallelism, pipeline parallelism, data parallelism, and expert parallelism. Optimize communication patterns and synchronization to minimize latency and maximize throughput.
- Develop Multi-Model Serving and Dynamic Resource Management: Design efficient approaches for serving multiple models simultaneously with dynamic resource allocation. Implement model loading, swapping mechanisms, and workload-aware scheduling algorithms that maximize resource utilization and minimize switching overhead.
- Analyze and Optimize GPU Kernels: Profile and optimize performance-critical GPU kernels and operators, including attention mechanisms, mixture-of-experts layers, communication primitives, and other bottleneck components. Use GPU profiling tools to identify optimization opportunities and implement efficient kernel implementations.
- Independently Identify and Solve High-Impact Problems: Demonstrate ownership by recognizing critical performance and systems challenges with limited direction, formulating data-driven hypotheses, rapidly prototyping solutions, and driving promising research initiatives from conceptualization through production deployment.
- Perform End-to-End Workload Profiling and Benchmarking: Conduct comprehensive performance analysis across compute, memory, communication, networking, scheduling, and model execution layers. Identify critical bottlenecks, measure performance tradeoffs, and establish baselines for optimization efforts using industry-standard profiling methodologies.
- Collaborate Across Research and Production Teams: Work closely with model researchers, infrastructure engineering teams, and product teams to translate research innovations into production systems. Facilitate cross-functional knowledge sharing and ensure successful deployment of advanced inference optimizations at scale.
- Publish and Open-Source Research Contributions: Document and publish research innovations through technical blogs and submissions to top-tier systems and machine learning conferences. Contribute to open-source projects like Arctic Inference to advance the broader AI systems community.

## Requirements

### education

- {"name":"Bachelor's Degree in Computer Science or Related Field","description":"Minimum requirement: Bachelor's degree in Computer Science, Electrical Engineering, Computer Engineering, or related discipline with strong foundation in systems, algorithms, and computer architecture."}
- {"name":"Advanced Degree Preferred","description":"Master's degree or PhD in Computer Science, Machine Learning, Computer Engineering, or related field is strongly preferred, particularly if focused on systems, distributed computing, or machine learning systems."}

### technical

- {"name":"Modern LLM Inference Architectures","description":"Deep understanding of contemporary LLM inference system architectures including prefill/decode separation, token generation optimization, attention mechanism implementations, memory management strategies, and performance tradeoff analysis for large-scale model serving."}
- {"name":"Distributed System Design and Execution","description":"Strong ability to reason about distributed execution patterns, synchronization strategies, communication optimization, fault tolerance, and scalability. Understanding of tensor, sequence, pipeline, and expert parallelism approaches for distributed inference."}
- {"name":"GPU Architecture and Optimization","description":"Solid grasp of GPU architecture including memory hierarchies, execution units, warp scheduling, and optimization techniques. Experience with performance-oriented libraries including CUTLASS, cuBLAS, and cuDNN for low-level optimization."}
- {"name":"End-to-End Performance Analysis","description":"Ability to holistically analyze system performance across model, runtime, distributed system, and hardware layers while reasoning about complex performance tradeoffs between latency, throughput, memory efficiency, and cost."}
- {"name":"AI-Native Systems Engineering","description":"Experience or strong interest in applying machine learning techniques to systems engineering challenges including automated profiling, configuration optimization, code generation, experimentation acceleration, and dynamic system adaptation."}
- {"name":"Independent Problem-Solving and Ownership","description":"Demonstrated ability to operate independently—recognizing impactful technical problems with minimal guidance, defining appropriate research questions, navigating technical ambiguity, and driving solutions from conception to production."}

### experience

- {"name":"5+ Years in LLM Inference Systems or Distributed AI","description":"Demonstrated professional experience designing, implementing, or optimizing LLM inference systems, distributed AI systems, GPU computing systems, or high-performance computing environments with measurable performance improvements."}
- {"name":"Production LLM Inference Framework Experience","description":"Hands-on experience extending, customizing, or optimizing modern LLM inference frameworks such as vLLM, SGLang, or TensorRT-LLM. Understanding of framework architecture, capabilities, and limitations in production environments."}
- {"name":"Inference Runtime and Optimization Expertise","description":"Direct experience designing or optimizing inference runtime systems in areas including scheduling algorithms, batching strategies, KV-cache management, distributed execution, parallelism schemes, speculative decoding, or disaggregated inference architectures."}
- {"name":"GPU Systems and CUDA Programming","description":"Substantial hands-on experience with GPU architecture fundamentals and GPU programming using CUDA, Triton, or equivalent environments. Demonstrated ability to write performance-critical GPU code and understand GPU memory hierarchies and execution models."}
- {"name":"Performance Profiling and Optimization","description":"Proven track record profiling, diagnosing, and optimizing complex system performance using tools like Nsight Systems, Nsight Compute, or equivalent profiling infrastructure. Ability to identify bottlenecks and implement targeted optimizations across hardware and software layers."}

## Skills

### required

- {"name":"LLM Inference System Design","description":"Expertise in designing and implementing production LLM inference systems with focus on optimizing latency, throughput, memory efficiency, and cost at scale."}
- {"name":"CUDA and GPU Programming","description":"Advanced proficiency with CUDA programming model, GPU memory management, kernel optimization, and understanding of GPU architecture constraints and capabilities."}
- {"name":"Distributed Systems Programming","description":"Strong experience implementing distributed algorithms, managing communication patterns, synchronization primitives, and debugging distributed systems at scale."}
- {"name":"Performance Profiling and Optimization","description":"Hands-on expertise using GPU profiling tools (Nsight Systems, Nsight Compute) and system-level analysis tools to identify bottlenecks and implement targeted optimizations."}
- {"name":"C++ Systems Programming","description":"Expert-level C++ programming for systems code, including understanding of memory management, performance optimization, and low-level optimization techniques."}
- {"name":"vLLM, SGLang, or TensorRT-LLM","description":"Practical experience with at least one major open-source LLM inference framework, including ability to extend, customize, and optimize for specific workloads."}

### preferred

- {"name":"Python Systems Development","description":"Proficiency with Python for rapid prototyping, system development, and integration with GPU acceleration libraries and inference frameworks."}
- {"name":"Triton GPU Programming","description":"Experience with Triton for writing performance-portable GPU kernels at a higher abstraction level than CUDA while maintaining performance."}
- {"name":"Machine Learning Systems Knowledge","description":"Understanding of transformer architectures, attention mechanisms, mixture-of-experts models, and other modern LLM components that impact inference performance."}
- {"name":"Quantization and Model Compression","description":"Knowledge of model quantization techniques (INT8, FP8, FP4), pruning, and other model compression approaches that enable efficient inference."}
- {"name":"AI-Native Engineering and Automation","description":"Experience applying machine learning techniques to systems engineering including automated profiling, configuration search, code generation, and adaptive optimization."}
- {"name":"Research Publication and Open-Source Contribution","description":"Track record of publishing systems research at top-tier conferences or contributing to influential open-source projects in AI infrastructure and systems."}

## Tech stack

### tools

- {"name":"Nsight Systems","description":"NVIDIA's comprehensive system-level profiler for analyzing GPU utilization, CPU-GPU synchronization, memory bandwidth, and identifying performance bottlenecks across the full inference stack."}
- {"name":"Nsight Compute","description":"NVIDIA's kernel-level profiler for deep analysis of GPU kernel performance, memory access patterns, compute utilization, and targeted kernel optimization opportunities."}
- {"name":"CUTLASS","description":"NVIDIA's CUDA templates library for optimized tensor operations, enabling efficient implementation of matrix multiplication and other compute-intensive inference operations."}
- {"name":"cuBLAS","description":"NVIDIA's basic linear algebra library providing highly-optimized implementations of matrix operations fundamental to LLM inference and model operations."}
- {"name":"cuDNN","description":"NVIDIA's deep neural network library offering optimized implementations of neural network operations including convolutions, activations, and attention mechanisms."}
- {"name":"Git and Version Control","description":"Essential for managing inference system codebase, coordinating contributions across research and engineering teams, and maintaining multiple optimization branches."}

### others

- {"name":"Speculative Decoding Techniques","description":"Advanced inference optimization method enabling parallel generation of multiple tokens using smaller draft models, significantly reducing time-to-first-token and overall latency."}
- {"name":"Adaptive Parallelism","description":"Dynamic parallelism strategies that automatically adjust tensor, sequence, and pipeline parallelism configurations based on workload characteristics and hardware capabilities."}
- {"name":"KV-Cache Optimization","description":"Memory optimization techniques for key-value cache management during token generation, including cache compression, disaggregation, and efficient prefill-decode coordination."}
- {"name":"Distributed Inference Scheduling","description":"Algorithms for optimal request scheduling, batching, and resource allocation across multiple GPUs and nodes to maximize throughput and minimize latency variance."}
- {"name":"Mixture-of-Experts (MoE) Optimization","description":"Specialized inference optimization approaches for models using conditional computation, including expert selection, load balancing, and communication reduction techniques."}
- {"name":"Model Quantization Strategies","description":"Post-training quantization techniques (INT8, FP8) and awareness training methods that enable efficient inference on reduced precision hardware while maintaining model quality."}
- {"name":"Disaggregated Inference Serving","description":"Architectural patterns separating prefill and decode stages across different hardware resources, optimizing for their distinct compute and memory characteristics."}
- {"name":"AI-Native Systems Engineering","description":"Systematic application of machine learning to systems engineering including automated performance prediction, configuration optimization, code generation, and runtime adaptation."}

### databases

- {"name":"GPU Memory Management","description":"Advanced understanding of GPU memory hierarchies, including global memory, shared memory, and cache optimization for maximizing data locality in inference workloads."}

### languages

- {"name":"C++","description":"Primary language for building high-performance inference systems, GPU kernel integration, and production-grade runtime implementations requiring low-latency execution."}
- {"name":"Python","description":"Used for system prototyping, rapid experimentation, framework development, integration with inference frameworks, and automation of profiling and optimization workflows."}
- {"name":"CUDA","description":"GPU programming language for implementing custom kernels, optimizing model components, and developing performance-critical inference operations on NVIDIA GPUs."}

### frameworks

- {"name":"vLLM","description":"Industry-leading open-source LLM inference framework featuring continuous batching, advanced scheduling, and optimizations for high-throughput inference serving."}
- {"name":"SGLang","description":"Structured generation framework enabling speculative decoding and other advanced inference optimizations for efficient token generation."}
- {"name":"TensorRT-LLM","description":"NVIDIA's LLM inference optimization framework providing compiled inference engines with support for quantization, tensor parallelism, and hardware-specific optimizations."}
- {"name":"Arctic Inference","description":"Snowflake's open-source inference system implementing advanced techniques like Shift Parallelism, SwiftKV, and Semi-Persistence for optimized multi-model serving."}
- {"name":"PyTorch","description":"Deep learning framework used for model training, evaluation, and as the foundation for many inference system implementations and optimizations."}

## Benefits

### benefits

## Compensation

- **max:** 0
- **min:** 0
- **currency:** 
- **stockOptions:** false

## Interview process

### steps

## Full description
At Snowflake, we are powering the era of the agentic enterprise. To usher in this new era, we seek AI-native thinkers across every function who are energized by the opportunity to reinvent how they work. You don’t just use tools; you possess an innate curiosity, treating AI as a high-trust collaborator that is core to how you solve problems and accelerate your impact. We look for low-ego individuals who thrive in dynamic and fast-moving environments and move with an experimental mindset — who rapidly test emerging capabilities to discover simpler, more powerful ways to deliver results. At Snowflake, your role isn't just to execute a function, but to help redefine the future of how work gets done.

We are looking for talented systems developers and researchers to join the Snowflake AI Research team and advance the state of the art in **LLM inference systems and optimization**.

Our mission is to build the next generation of **high-performance and intelligent inference systems**. We optimize not only how fast and efficiently models run, but also how quickly inference systems can adapt to new models, architectures, hardware, and workloads.

Our work spans the full inference stack—from distributed serving and runtime systems to GPU kernels and model-system co-design. We explore techniques such as **adaptive parallelism, speculative and parallel decoding, disaggregated inference, scheduling and batching, KV-cache optimization, model swapping, quantization, and GPU kernel optimization** to push the frontier of latency, throughput, scalability, and cost.

Beyond optimizing individual models, we are building **intelligent and adaptive inference systems** that can automate performance optimization—rapidly profiling new models and workloads, identifying bottlenecks, selecting effective execution strategies, and adapting system configurations with minimal manual tuning. We embrace **AI-native engineering**, using AI not only as the workload we optimize, but also as a tool to accelerate system development, experimentation, debugging, optimization, and adaptation to new models. Our goal is to accelerate both the **speed of inference and the agility of inference development**.

Recent innovations from Snowflake AI Research include **Arctic Inference**, our open-source inference system, and technologies such as **Shift Parallelism**, which dynamically adapts parallelism to workload characteristics; **SwiftKV**, which reduces redundant prefill computation; **Arctic Speculator and SuffixDecoding** for fast speculative decoding; **Jacobi Forcing** for causal parallel decoding; and **Semi-Persistence** for fast model swapping and dynamic multi-model serving.

This is an exciting opportunity to collaborate with a world-class team, including founding members of DeepSpeed, vLLM, and TensorFlow. Together, we will push the boundaries of AI systems and bring cutting-edge research into production-scale AI.

**Responsibilities**

* Design and develop **high-performance LLM inference systems**, spanning distributed serving, runtime systems, GPU execution, and performance-critical kernels.
* Develop novel techniques to improve **inference latency, generation speed, throughput, memory efficiency, scalability, and cost**.
* Explore advanced inference techniques including **speculative and parallel decoding, prefill/decode disaggregation, adaptive parallelism, continuous batching and scheduling, KV-cache management, quantization, and communication optimization**.
* Develop **adaptive and intelligent inference systems** that automatically optimize execution for new model architectures, hardware platforms, workload characteristics, and deployment environments.
* Apply **AI-driven and AI-native approaches to systems engineering**, including automated profiling, bottleneck identification, configuration search, code generation, experimentation, runtime strategy selection, debugging, and performance tuning.
* Independently identify high-impact performance and systems problems, formulate hypotheses, prototype solutions, and drive promising ideas from research through production.
* Design distributed inference strategies across GPUs and nodes, including tensor, sequence, pipeline, data, and expert parallelism.
* Develop efficient approaches for **multi-model serving, dynamic resource management, model loading and swapping, and workload-aware scheduling**.
* Analyze and optimize GPU kernels and operators for attention, MoE, communication, and other performance-critical model components.
* Explore **model-system co-design**, including model or post-training techniques that unlock substantially more efficient inference.
* Profile and benchmark end-to-end workloads to identify bottlenecks across compute, memory, communication, networking, scheduling, and model execution.
* Collaborate closely with model researchers, infrastructure teams, and product teams to deploy research innovations in production.
* Open-source and publish innovations through technical blogs and top-tier systems and machine learning conferences.

**Requirements**

* Bachelor’s degree in Computer Science, Electrical Engineering, or a related field. A Master’s degree or PhD is preferred.
* 5+ years of experience in one or more of the following areas: **LLM inference systems, distributed AI systems, GPU systems, or high-performance computing**.
* Strong understanding of modern LLM inference architectures and the performance tradeoffs involved in serving large-scale models.
* Hands-on experience with modern **LLM inference and serving frameworks**, such as **vLLM, SGLang, TensorRT-LLM**, or similar systems.
* Experience designing, extending, or optimizing inference runtimes, including areas such as **scheduling, batching, KV-cache management, distributed execution, parallelism, speculative decoding, or disaggregated serving**.
* Strong understanding of GPU architectures and experience with **CUDA, Triton**, or similar GPU programming environments.
* Experience with performance-oriented libraries and frameworks such as **CUTLASS, cuBLAS, cuDNN**, or related technologies.
* Experience profiling and diagnosing end-to-end system performance using **Nsight Systems, Nsight Compute**, or equivalent tools.
* Demonstrated ability to operate as an **independent problem identifier and solver**—recognizing important problems with limited direction, defining the right technical questions, and driving solutions through ambiguity.
* Strong ability to work across **model, runtime, distributed system, and hardware layers** and reason about end-to-end performance tradeoffs.
* Experience using **AI-native engineering approaches** to accelerate software development, experimentation, debugging, optimization, or system adaptation is a strong plus.
* Excellent communication skills and the ability to collaborate effectively across research, engineering, and product teams.

Snowflake is growing fast, and we’re scaling our team to help enable and accelerate our growth. We are looking for people who share our values, challenge ordinary thinking, and push the pace of innovation while building a future for themselves and Snowflake.

How do you want to make your impact?

For jobs located in the United States, please visit the job posting on the Snowflake Careers Site for salary and benefits information: [careers.snowflake.com](http://careers.snowflake.com)