# Software Engineer, Model Capabilities
**Company:** [Notion](https://scaleengineer.com/companies/notion)
Join Notion's Model Capabilities team as a Senior Backend Engineer focused on integrating frontier AI models into production, ensuring reliable inference at scale, and building the model layer that powers Notion AI. This role requires deep production AI systems experience, strong understanding of LLM economics and deployment patterns, and the ability to work cross-functionally with product teams. You'll be at the forefront of AI infrastructure, bringing cutting-edge models to market quickly while maintaining reliability, observability, and cost efficiency across Notion's distributed AI systems.
**Role:** Backend Engineer
**Seniority:** Senior
**Locations:** San Francisco, California
**Salary:** 255000–300000 USD
[Apply](https://jobs.ashbyhq.com/notion/ab383335-1dd5-4d6a-971f-439cebf48a9d)
Canonical: https://scaleengineer.com/jobs/notion/software-engineer-model-capabilities
---
## Responsibilities

- Production Model Integration: Drive the rapid integration of frontier large language models into Notion's production infrastructure, evaluating new model releases monthly, conducting technical feasibility assessments, and orchestrating deployment pipelines to make models available to users and engineering teams with minimal latency. Implement versioning strategies and A/B testing frameworks to safely roll out new models to production workloads.
- Inference Reliability & Resilience: Design and implement comprehensive error categorization systems for LLM API calls, build intelligent self-healing retry mechanisms with exponential backoff and circuit breaker patterns, and develop multi-provider failover strategies to ensure service continuity. Monitor and reduce inference failure rates across multiple model providers, implementing graceful degradation strategies and fallback mechanisms.
- Observability & Monitoring Architecture: Own the complete observability stack for the model layer, implementing advanced monitoring, tracing, and logging systems that minimize both time-to-detection and time-to-fix for production incidents. Create dashboards and alerts for model performance metrics, token consumption, latency percentiles, and cost tracking across inference workloads.
- Model-Level Capabilities Development: Build and ship new capabilities at the model abstraction layer such as prompt optimization, token budget management, context window handling, and tool-calling orchestration. Work closely with product teams to enable adoption of these capabilities, providing SDKs, documentation, and technical guidance for integration into consumer-facing features.
- Cross-Team AI Infrastructure Leadership: Act as the connective tissue across Notion's AI teams, identifying infrastructure gaps, unblocking dependent teams, and ensuring infrastructure improvements reach the right owners. Lead technical discussions around token economics, inference cost optimization, and model selection trade-offs. Facilitate knowledge sharing and best practices across AI-focused engineering teams.
- Data-Driven Performance Optimization: Establish rigorous evaluation frameworks and metrics for assessing model changes, prompt modifications, and inference optimizations. Implement A/B testing and statistical analysis methodologies before shipping changes, treating data as the source of truth for performance decisions. Track latency, throughput, token efficiency, and cost metrics across all model layer changes.
- Cost & Performance Management: Treat inference latency and operational costs as first-class product qualities, building models and dashboards to understand token consumption patterns, model pricing differences, and cost-per-inference by feature. Optimize the trade-off space between response quality, latency targets, and operational costs, implementing caching strategies and smart prompt engineering to reduce token spend.

## Requirements

### education

- {"name":"Computer Science Fundamentals","description":"Bachelor's degree in Computer Science, Software Engineering, or equivalent demonstrable experience with core CS concepts including algorithms, data structures, systems design, and distributed systems principles."}

### technical

- {"name":"Production LLM Systems Experience","description":"Demonstrated experience building and operating production systems on top of LLM APIs (OpenAI, Anthropic, Google, etc.), not just prototyping. Deep understanding of token economics, streaming responses, function/tool calling, context window limits, and how these fail under production load conditions."}
- {"name":"Inference Infrastructure Knowledge","description":"Solid understanding of inference optimization techniques including batching, caching, quantization, and latency optimization. Experience with LLM API rate limits, quota management, and multi-provider orchestration patterns. Knowledge of how to measure and optimize for time-to-first-token and end-to-end latency."}
- {"name":"Distributed Systems & Reliability Engineering","description":"Strong foundation in designing reliable distributed systems with focus on fault tolerance, error handling, and observability. Experience building systems that handle partial failures gracefully, implementing circuit breakers, retry mechanisms, and health checks for external service dependencies."}
- {"name":"Monitoring & Observability","description":"Practical experience designing and implementing observability systems using modern tools like distributed tracing, structured logging, and metrics collection. Ability to set up meaningful alerts, dashboards, and SLIs/SLOs for infrastructure and API-dependent services."}
- {"name":"Backend API Design","description":"Experience designing scalable backend APIs and services that integrate with external LLM providers. Proficiency with REST or gRPC API design patterns, rate limiting, authentication, and API versioning strategies. Understanding of API design trade-offs between simplicity and power."}

### experience

- {"name":"Production AI Systems at Scale","description":"3+ years of experience building and operating production AI or machine learning systems serving real users. Experience managing costs, latency, and reliability of systems with external API dependencies or inference workloads."}
- {"name":"Backend Infrastructure Development","description":"5+ years of professional backend or infrastructure engineering experience, with at least 2 years focused on reliability, observability, or platform systems. Track record of shipping features that impact production reliability and user experience."}
- {"name":"LLM API Integration","description":"Hands-on experience integrating with multiple large language model providers, understanding the operational implications of each provider's pricing model, rate limits, model availability, and SLA characteristics."}

## Skills

### required

- {"name":"Production LLM Integration","description":"Hands-on experience integrating frontier language models from providers like OpenAI (GPT-4, GPT-4o), Anthropic (Claude), Google (Gemini), or Meta (Llama) into production systems. Understanding of model versioning, upgrade strategies, and how to handle model deprecations."}
- {"name":"Backend Systems Design","description":"Strong ability to design scalable, reliable backend systems and services. Proficiency with service-oriented architecture patterns, designing APIs that handle distributed failure modes, and implementing proper error handling and retry logic."}
- {"name":"Python or Go","description":"Production-level proficiency in Python or Go for building backend services, infrastructure code, and data processing pipelines. Ability to write testable, maintainable code that can be deployed to production at scale."}
- {"name":"Inference Cost & Performance Optimization","description":"Demonstrated ability to reason about token economics, model costs, and latency trade-offs. Experience optimizing for efficiency: caching strategies, batching, prompt optimization, and understanding how to measure and improve cost-per-request metrics."}
- {"name":"Observability & Debugging","description":"Practical experience with distributed tracing, structured logging, and metrics collection. Ability to diagnose production issues in systems with external dependencies using observability tools and data. Comfort with tools like Datadog, New Relic, Prometheus, or similar platforms."}
- {"name":"Active AI Tool Usage","description":"Serious, ongoing use of AI tools (ChatGPT, Claude, GitHub Copilot, etc.) in your own work with well-developed opinions about where they add value and where they fall short. Ability to evaluate AI capabilities critically and pragmatically."}

### preferred

- {"name":"Machine Learning Operations (MLOps)","description":"Experience with ML/AI ops infrastructure including model versioning, feature stores, experiment tracking, or A/B testing frameworks. Familiarity with data validation and model monitoring in production environments."}
- {"name":"TypeScript/Node.js","description":"Additional proficiency with TypeScript and Node.js ecosystem for building services that interoperate with frontend teams or support full-stack AI features."}
- {"name":"Multi-Provider API Orchestration","description":"Experience working with multiple external API providers simultaneously, implementing strategies for provider selection, failover, and cost optimization across heterogeneous services."}
- {"name":"Real-time Systems","description":"Background with real-time or streaming systems, including message queues, event streaming platforms like Kafka, or systems requiring low-latency response times."}
- {"name":"Database Optimization","description":"Experience optimizing database query performance, understanding caching layers, and designing efficient data access patterns for high-throughput services."}
- {"name":"Open Source Contributions","description":"Active contributions to open source projects, particularly those in the AI/ML, infrastructure, or distributed systems domains. Evidence of code quality, collaboration skills, and technical depth."}

## Tech stack

### tools

- {"name":"Docker & Kubernetes","description":"Containerization and orchestration for deploying inference services, managing auto-scaling, and ensuring reliable service deployments in production."}
- {"name":"Datadog/New Relic","description":"Observability and monitoring platforms for distributed tracing, metrics collection, alerting, and incident response across inference infrastructure."}
- {"name":"GitHub/Git","description":"Version control and collaboration platform for managing infrastructure code, model integration code, and cross-team coordination."}
- {"name":"CI/CD Pipelines (GitHub Actions/Jenkins)","description":"Automation for testing, validating, and deploying model integration changes and infrastructure updates to production."}

### others

- {"name":"LLM Provider APIs","description":"Deep familiarity with OpenAI API (GPT-4, GPT-4o, completion endpoints, embeddings), Anthropic API (Claude models), Google Vertex AI, and other frontier model providers including authentication, rate limiting, and cost monitoring."}
- {"name":"Prompt Engineering & Evaluation","description":"Ability to iteratively develop and optimize prompts, design evaluation metrics, and implement automated testing frameworks for assessing model behavior under production conditions."}
- {"name":"Cost Analytics & Monitoring","description":"Building dashboards and analytics for tracking token consumption, cost-per-inference, model pricing comparisons, and cost optimization opportunities across inference workloads."}
- {"name":"A/B Testing Frameworks","description":"Implementing statistical frameworks for A/B testing model changes, prompt modifications, and inference optimizations with proper statistical rigor and minimum sample size calculations."}

### databases

- {"name":"PostgreSQL","description":"Primary relational database for storing model metadata, inference logs, cost tracking, and system state."}
- {"name":"Redis","description":"In-memory cache for reducing inference latency, caching model responses, managing rate limits, and session state."}
- {"name":"Datadog or Similar Observability Databases","description":"Time-series databases and observability platforms for storing metrics, traces, and logs from production inference systems."}

### languages

- {"name":"Python","description":"Primary language for building backend services, inference wrappers, model integration code, and data analysis tooling in AI/ML systems at Notion."}
- {"name":"Go","description":"Used for high-performance infrastructure services, concurrent systems, and services requiring low latency and efficient resource utilization."}
- {"name":"TypeScript/JavaScript","description":"JavaScript ecosystem support for full-stack integration, API design, and interoperability with Notion's frontend infrastructure."}

### frameworks

- {"name":"FastAPI","description":"Lightweight Python web framework used for building high-performance API services that expose LLM capabilities and handle inference orchestration."}
- {"name":"LangChain","description":"Framework for building applications powered by language models, providing abstractions for prompt engineering, chains, and tool integration patterns."}
- {"name":"LiteLLM","description":"Abstraction layer for unified interface to multiple LLM providers, enabling provider-agnostic model integration and automatic failover."}

## Benefits

### benefits

- {"name":"Competitive Equity Package","description":"Meaningful stock options with vesting schedules aligned to long-term value creation. Equity grants recognizing contributions to Notion's growth and success in the rapidly evolving AI workspace market."}
- {"name":"Health & Wellness Benefits","description":"Comprehensive medical, dental, and vision insurance plans with employer contributions. Mental health support, wellness programs, and resources supporting physical and mental wellbeing."}
- {"name":"Flexible Work Arrangements","description":"Hybrid work model with Anchor Days (Monday, Tuesday, Thursday) in San Francisco or New York City offices. Flexibility to structure work around deep focus time and collaboration needs."}
- {"name":"Professional Development","description":"Investment in continuous learning including conference attendance, training budgets, and dedicated time for exploring new technologies. Access to cutting-edge AI research and model releases as part of day-to-day work."}
- {"name":"Unlimited PTO","description":"Flexible time off policy encouraging work-life balance and recovery time. Additional paid holidays and company-wide break periods."}
- {"name":"401(k) Retirement Plan","description":"Employer-sponsored 401(k) retirement plan with company matching contributions, supporting long-term financial security."}
- {"name":"Notion Workspace License","description":"Free access to Notion's full platform as an employee, experiencing firsthand the tools and AI capabilities being built while using them in daily work."}

## Compensation

- **max:** 300000
- **min:** 255000
- **currency:** USD
- **stockOptions:** true

## Interview process

### steps

## Full description
# Who We Are

Notion is the collaborative AI workspace where teams and agents [think together](https://www.youtube.com/watch?v=vkpYpWfEK5s). We're building one place where your knowledge, projects, meetings, and AI tools live side by side, so work is faster, clearer, and less fragmented. Millions of individuals, small teams, and large companies run their work on Notion.

Notinos (our employees) are customer zero in bringing this future of work to life. We care about craft, building things that last, and the belief that great work is still fundamentally human. Our goal isn’t to ship the next feature. Each and every team of Notinos is working to set the standard for how humans work together in the AI era. From building a business’s system of record to making and managing AI agents to automating away the busy work, we care deeply about giving our customers more time for their life’s work.

# About the Role:

Notion has been at the cutting edge of AI since before ChatGPT launched. The job of the Model Capabilities team is to keep us there. We own the model layer of Notion AI: integrating frontier models as they ship, keeping inference reliable and economical at scale, and building new capabilities that other teams take advantage of.

This role can be based in either San Francisco or New York City. We work from our offices on Mondays, Tuesdays and Thursdays (our Anchor Days) because we do our best thinking and building together in person. We’re looking for someone who’s excited to work alongside the team during those days. 

# What You'll Achieve:

* Bring new frontier models into production quickly, making them available for our users and our engineers.
* Make inference reliable: better error categorization, self-healing retries, and cross-provider failover.
* Own observability for the model layer, driving down both time to detection and time to fix.
* Build new model-level capabilities and help product teams adopt them.
* Act as connective tissue across Notion's AI teams: find the gaps, unblock people, and make sure fixes land with the right owner.

# Skills You'll Need to Bring: 

* **Comfort with a moving frontier:** The model landscape changes monthly. You keep up with it, form your own view of what's real, and stay pragmatic about what's worth adopting now.
* **AI fluency as a practitioner:** You use AI tools seriously in your own work and have opinions about where they help and where they don't.
* **Production AI systems experience:** You've built and operated systems on top of LLM APIs, not just prototyped with them. You understand token economics, streaming, tool calling, context limits, and how each of these fails in production.
* **Measurement before conviction:** You don't ship a prompt or model change on vibes. You define what "better" means, build or use the evaluation to test it, and are willing to be proven wrong by your own data.
* **Cost and performance instincts:** You treat latency and dollars as product qualities, not afterthoughts, and you can reason about the tradeoff between them and output quality.

## 

Notion is committed to providing highly competitive cash compensation, equity, and benefits. The compensation offered for this role will be based on multiple factors such as location, the role’s scope and complexity, and the candidate’s experience and expertise, and may vary from the range provided below. For roles based in San Francisco or New York City, the estimated base salary range for this role is $255,000 – $300,000 per year.

By clicking “Submit Application”, I understand and agree that Notion and its affiliates and subsidiaries will collect and process my information in accordance with Notion’s [Global Recruiting Privacy Policy](https://notion.notion.site/Notion-Global-Recruiting-Privacy-Policy-fc3eb4e829354a26a2bb6fd5e289b550?pvs=74) and [NYLL 144](https://notion.notion.site/Ashby-AI-Bias-Audit-2b0efdeead05803bbbfae159ec86c528).

#LI-Onsite

# A Note on AI

You don’t need deep AI expertise for every role, but we do expect every Notino to be intellectually curious, drawn to tinkering and discovery, and excited to use AI as a real collaborator in their work. For some roles, AI fluency is a core requirement — when that’s the case, we'll say so explicitly in the qualifications. People who thrive here don’t treat AI as a novelty. They use it to think better, and make their work easier for others to build on.

# Equal Opportunity & Accommodations

We hire talented people from a wide range of backgrounds. If you’re excited about this role but don’t meet every bullet, we still encourage you to apply. Notion is an equal opportunity employer and does not discriminate on the basis of any legally protected characteristic. Consistent with applicable law, we will consider for employment qualified applicants with arrest and conviction records. Notion provides reasonable accommodations during the application process; if you need one, please let your recruiter know.

Notion is proud to be an equal opportunity employer. We do not discriminate in hiring or any employment decision based on race, color, religion, national origin, age, sex (including pregnancy, childbirth, or related medical conditions), marital status, ancestry, physical or mental disability, genetic information, veteran status, gender identity or expression, sexual orientation, or other applicable legally protected characteristic. Notion considers qualified applicants with criminal histories, consistent with applicable federal, state and local law. Notion is also committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, please let your recruiter know.