Software Engineer, Model Capabilities
Backend Engineer · Senior · Full Time
Opens Notion's application page
Role
What you'll do.
Join Notion's Model Capabilities team as a Senior Backend Engineer focused on integrating frontier AI models into production, ensuring reliable inference at scale, and building the model layer that powers Notion AI. This role requires deep production AI systems experience, strong understanding of LLM economics and deployment patterns, and the ability to work cross-functionally with product teams. You'll be at the forefront of AI infrastructure, bringing cutting-edge models to market quickly while maintaining reliability, observability, and cost efficiency across Notion's distributed AI systems.
Responsibilities
- Production Model Integration: Drive the rapid integration of frontier large language models into Notion's production infrastructure, evaluating new model releases monthly, conducting technical feasibility assessments, and orchestrating deployment pipelines to make models available to users and engineering teams with minimal latency. Implement versioning strategies and A/B testing frameworks to safely roll out new models to production workloads.
- Inference Reliability & Resilience: Design and implement comprehensive error categorization systems for LLM API calls, build intelligent self-healing retry mechanisms with exponential backoff and circuit breaker patterns, and develop multi-provider failover strategies to ensure service continuity. Monitor and reduce inference failure rates across multiple model providers, implementing graceful degradation strategies and fallback mechanisms.
- Observability & Monitoring Architecture: Own the complete observability stack for the model layer, implementing advanced monitoring, tracing, and logging systems that minimize both time-to-detection and time-to-fix for production incidents. Create dashboards and alerts for model performance metrics, token consumption, latency percentiles, and cost tracking across inference workloads.
- Model-Level Capabilities Development: Build and ship new capabilities at the model abstraction layer such as prompt optimization, token budget management, context window handling, and tool-calling orchestration. Work closely with product teams to enable adoption of these capabilities, providing SDKs, documentation, and technical guidance for integration into consumer-facing features.
- Cross-Team AI Infrastructure Leadership: Act as the connective tissue across Notion's AI teams, identifying infrastructure gaps, unblocking dependent teams, and ensuring infrastructure improvements reach the right owners. Lead technical discussions around token economics, inference cost optimization, and model selection trade-offs. Facilitate knowledge sharing and best practices across AI-focused engineering teams.
- Data-Driven Performance Optimization: Establish rigorous evaluation frameworks and metrics for assessing model changes, prompt modifications, and inference optimizations. Implement A/B testing and statistical analysis methodologies before shipping changes, treating data as the source of truth for performance decisions. Track latency, throughput, token efficiency, and cost metrics across all model layer changes.
- Cost & Performance Management: Treat inference latency and operational costs as first-class product qualities, building models and dashboards to understand token consumption patterns, model pricing differences, and cost-per-inference by feature. Optimize the trade-off space between response quality, latency targets, and operational costs, implementing caching strategies and smart prompt engineering to reduce token spend.
Qualifications
What we look for.
Technical
Production LLM Systems Experience
Demonstrated experience building and operating production systems on top of LLM APIs (OpenAI, Anthropic, Google, etc.), not just prototyping. Deep understanding of token economics, streaming responses, function/tool calling, context window limits, and how these fail under production load conditions.
Inference Infrastructure Knowledge
Solid understanding of inference optimization techniques including batching, caching, quantization, and latency optimization. Experience with LLM API rate limits, quota management, and multi-provider orchestration patterns. Knowledge of how to measure and optimize for time-to-first-token and end-to-end latency.
Distributed Systems & Reliability Engineering
Strong foundation in designing reliable distributed systems with focus on fault tolerance, error handling, and observability. Experience building systems that handle partial failures gracefully, implementing circuit breakers, retry mechanisms, and health checks for external service dependencies.
Monitoring & Observability
Practical experience designing and implementing observability systems using modern tools like distributed tracing, structured logging, and metrics collection. Ability to set up meaningful alerts, dashboards, and SLIs/SLOs for infrastructure and API-dependent services.
Backend API Design
Experience designing scalable backend APIs and services that integrate with external LLM providers. Proficiency with REST or gRPC API design patterns, rate limiting, authentication, and API versioning strategies. Understanding of API design trade-offs between simplicity and power.
Education
Computer Science Fundamentals
Bachelor's degree in Computer Science, Software Engineering, or equivalent demonstrable experience with core CS concepts including algorithms, data structures, systems design, and distributed systems principles.
Experience
Production AI Systems at Scale
3+ years of experience building and operating production AI or machine learning systems serving real users. Experience managing costs, latency, and reliability of systems with external API dependencies or inference workloads.
Backend Infrastructure Development
5+ years of professional backend or infrastructure engineering experience, with at least 2 years focused on reliability, observability, or platform systems. Track record of shipping features that impact production reliability and user experience.
LLM API Integration
Hands-on experience integrating with multiple large language model providers, understanding the operational implications of each provider's pricing model, rate limits, model availability, and SLA characteristics.
Skills
Required
Production LLM Integration
Hands-on experience integrating frontier language models from providers like OpenAI (GPT-4, GPT-4o), Anthropic (Claude), Google (Gemini), or Meta (Llama) into production systems. Understanding of model versioning, upgrade strategies, and how to handle model deprecations.
Backend Systems Design
Strong ability to design scalable, reliable backend systems and services. Proficiency with service-oriented architecture patterns, designing APIs that handle distributed failure modes, and implementing proper error handling and retry logic.
Python or Go
Production-level proficiency in Python or Go for building backend services, infrastructure code, and data processing pipelines. Ability to write testable, maintainable code that can be deployed to production at scale.
Inference Cost & Performance Optimization
Demonstrated ability to reason about token economics, model costs, and latency trade-offs. Experience optimizing for efficiency: caching strategies, batching, prompt optimization, and understanding how to measure and improve cost-per-request metrics.
Observability & Debugging
Practical experience with distributed tracing, structured logging, and metrics collection. Ability to diagnose production issues in systems with external dependencies using observability tools and data. Comfort with tools like Datadog, New Relic, Prometheus, or similar platforms.
Active AI Tool Usage
Serious, ongoing use of AI tools (ChatGPT, Claude, GitHub Copilot, etc.) in your own work with well-developed opinions about where they add value and where they fall short. Ability to evaluate AI capabilities critically and pragmatically.
Preferred
Machine Learning Operations (MLOps)
Nice to haveExperience with ML/AI ops infrastructure including model versioning, feature stores, experiment tracking, or A/B testing frameworks. Familiarity with data validation and model monitoring in production environments.
TypeScript/Node.js
Nice to haveAdditional proficiency with TypeScript and Node.js ecosystem for building services that interoperate with frontend teams or support full-stack AI features.
Multi-Provider API Orchestration
Nice to haveExperience working with multiple external API providers simultaneously, implementing strategies for provider selection, failover, and cost optimization across heterogeneous services.
Real-time Systems
Nice to haveBackground with real-time or streaming systems, including message queues, event streaming platforms like Kafka, or systems requiring low-latency response times.
Database Optimization
Nice to haveExperience optimizing database query performance, understanding caching layers, and designing efficient data access patterns for high-throughput services.
Open Source Contributions
Nice to haveActive contributions to open source projects, particularly those in the AI/ML, infrastructure, or distributed systems domains. Evidence of code quality, collaboration skills, and technical depth.
Tech stack
Languages
Frameworks
Databases
Tools
Other
Compensation
Pay and benefits.
Base·USD 255,000 – 300,000
Equity·Stock options
Benefits
Competitive Equity Package
Meaningful stock options with vesting schedules aligned to long-term value creation. Equity grants recognizing contributions to Notion's growth and success in the rapidly evolving AI workspace market.
Health & Wellness Benefits
Comprehensive medical, dental, and vision insurance plans with employer contributions. Mental health support, wellness programs, and resources supporting physical and mental wellbeing.
Flexible Work Arrangements
Hybrid work model with Anchor Days (Monday, Tuesday, Thursday) in San Francisco or New York City offices. Flexibility to structure work around deep focus time and collaboration needs.
Professional Development
Investment in continuous learning including conference attendance, training budgets, and dedicated time for exploring new technologies. Access to cutting-edge AI research and model releases as part of day-to-day work.
Unlimited PTO
Flexible time off policy encouraging work-life balance and recovery time. Additional paid holidays and company-wide break periods.
401(k) Retirement Plan
Employer-sponsored 401(k) retirement plan with company matching contributions, supporting long-term financial security.
Notion Workspace License
Free access to Notion's full platform as an employee, experiencing firsthand the tools and AI capabilities being built while using them in daily work.
Full posting
Original listing.
Who We Are
Notion is the collaborative AI workspace where teams and agents think together. We're building one place where your knowledge, projects, meetings, and AI tools live side by side, so work is faster, clearer, and less fragmented. Millions of individuals, small teams, and large companies run their work on Notion.
Notinos (our employees) are customer zero in bringing this future of work to life. We care about craft, building things that last, and the belief that great work is still fundamentally human. Our goal isn’t to ship the next feature. Each and every team of Notinos is working to set the standard for how humans work together in the AI era. From building a business’s system of record to making and managing AI agents to automating away the busy work, we care deeply about giving our customers more time for their life’s work.
About the Role:
Notion has been at the cutting edge of AI since before ChatGPT launched. The job of the Model Capabilities team is to keep us there. We own the model layer of Notion AI: integrating frontier models as they ship, keeping inference reliable and economical at scale, and building new capabilities that other teams take advantage of.
This role can be based in either San Francisco or New York City. We work from our offices on Mondays, Tuesdays and Thursdays (our Anchor Days) because we do our best thinking and building together in person. We’re looking for someone who’s excited to work alongside the team during those days.
What You'll Achieve:
Bring new frontier models into production quickly, making them available for our users and our engineers.
Make inference reliable: better error categorization, self-healing retries, and cross-provider failover.
Own observability for the model layer, driving down both time to detection and time to fix.
Build new model-level capabilities and help product teams adopt them.
Act as connective tissue across Notion's AI teams: find the gaps, unblock people, and make sure fixes land with the right owner.
Skills You'll Need to Bring:
Comfort with a moving frontier: The model landscape changes monthly. You keep up with it, form your own view of what's real, and stay pragmatic about what's worth adopting now.
AI fluency as a practitioner: You use AI tools seriously in your own work and have opinions about where they help and where they don't.
Production AI systems experience: You've built and operated systems on top of LLM APIs, not just prototyped with them. You understand token economics, streaming, tool calling, context limits, and how each of these fails in production.
Measurement before conviction: You don't ship a prompt or model change on vibes. You define what "better" means, build or use the evaluation to test it, and are willing to be proven wrong by your own data.
Cost and performance instincts: You treat latency and dollars as product qualities, not afterthoughts, and you can reason about the tradeoff between them and output quality.
Notion is committed to providing highly competitive cash compensation, equity, and benefits. The compensation offered for this role will be based on multiple factors such as location, the role’s scope and complexity, and the candidate’s experience and expertise, and may vary from the range provided below. For roles based in San Francisco or New York City, the estimated base salary range for this role is $255,000 – $300,000 per year.
By clicking “Submit Application”, I understand and agree that Notion and its affiliates and subsidiaries will collect and process my information in accordance with Notion’s Global Recruiting Privacy Policy and NYLL 144.
#LI-Onsite
A Note on AI
You don’t need deep AI expertise for every role, but we do expect every Notino to be intellectually curious, drawn to tinkering and discovery, and excited to use AI as a real collaborator in their work. For some roles, AI fluency is a core requirement — when that’s the case, we'll say so explicitly in the qualifications. People who thrive here don’t treat AI as a novelty. They use it to think better, and make their work easier for others to build on.
Equal Opportunity & Accommodations
We hire talented people from a wide range of backgrounds. If you’re excited about this role but don’t meet every bullet, we still encourage you to apply. Notion is an equal opportunity employer and does not discriminate on the basis of any legally protected characteristic. Consistent with applicable law, we will consider for employment qualified applicants with arrest and conviction records. Notion provides reasonable accommodations during the application process; if you need one, please let your recruiter know.
Notion is proud to be an equal opportunity employer. We do not discriminate in hiring or any employment decision based on race, color, religion, national origin, age, sex (including pregnancy, childbirth, or related medical conditions), marital status, ancestry, physical or mental disability, genetic information, veteran status, gender identity or expression, sexual orientation, or other applicable legally protected characteristic. Notion considers qualified applicants with criminal histories, consistent with applicable federal, state and local law. Notion is also committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, please let your recruiter know.
Redirects to Notion's application page.
Other roles
More at Notion.
Software Engineer, Developer Platform
Senior
Software Engineer, Mobile Core (Android)
Mid
Software Engineer, Mobile Platform (iOS)
Senior
Software Engineer, Data Platform (Foundations Lead)
Lead
Software Engineer, User Database Infrastructure
Senior