# Software Engineer, RL Data **Company:** [Cursor](https://scaleengineer.com/companies/cursor) Join Cursor's RL Data team as a Software Engineer to design training tasks, reward functions, and environments for frontier coding agents. You'll work with real user data to create datasets and evaluation systems that directly improve agent performance, combining software engineering fundamentals with data systems expertise in a flat, talent-dense organization focused on automating coding. **Role:** ML Engineer **Seniority:** Mid **Locations:** San Francisco **Salary:** 180000–280000 USD [Apply](https://jobs.ashbyhq.com/cursor/9a175381-45e2-42b6-bd64-6c84532365f9) Canonical: https://scaleengineer.com/jobs/cursor/software-engineer-rl-data --- ## Responsibilities - Design and Iterate Training Task Sets: Create curated task datasets that teach specific agent capabilities, such as code generation quality, debugging, or complex programming patterns. Analyze traces and evaluation metrics to identify what's working and iterate on task design based on model performance data, ensuring continuous improvement in agent effectiveness. - Analyze Agent Traces and Identify Failure Modes: Conduct deep analysis of model behavior from training traces and inference logs to discover failure patterns, unexpected behaviors, or edge cases. Build systematic approaches to surface and reproduce these modes, converting ad-hoc findings into scalable datasets that help the model learn from mistakes. - Develop Reward Functions and Evaluation Systems: Design reward functions that accurately measure task performance and align with Cursor's goals for agent effectiveness. Create robust evaluation systems that provide meaningful signals about model progress, enabling data-driven decisions about training effectiveness and dataset quality. - Build Reusable Data Infrastructure: Transform one-off scripts and recipes into production-ready systems that other teams can leverage. Abstract reward logic, environment setup, and data quality checks into maintainable, well-documented components that scale across the organization and improve data team productivity. - Collaborate with Research Teams: Partner with researchers to validate that datasets are actually teaching the intended capabilities, not just improving benchmark scores. Conduct analyses to understand model learning dynamics, propose data improvements, and ensure training datasets align with real-world deployment requirements. - Own Data Quality and Validation: Establish and maintain high standards for training data quality, including validation pipelines, outlier detection, and data lineage tracking. Ensure datasets are clean, representative, and properly documented for reproducibility and future reference. - Scale Real User Data Integration: Work on systems to incorporate real user data into RL training pipelines, ensuring privacy, quality, and relevance. Design scalable data collection and processing infrastructure that enables frontier coding agents to learn from actual user interactions and coding patterns. ## Requirements ### education - {"name":"Bachelor's Degree in Computer Science or Related Field","description":"Foundation in computer science, mathematics, physics, or engineering with strong algorithmic thinking and computational problem-solving skills."} ### technical - {"name":"Software Engineering Excellence","description":"Demonstrated ability to write production-quality code with strong fundamentals in software design, testing, debugging, and code organization. Experience shipping code and maintaining systems in production environments."} - {"name":"Data Systems Expertise","description":"Solid background in data infrastructure, ETL pipelines, or distributed data processing. Proficiency with data storage systems, query optimization, and designing systems that handle large-scale datasets efficiently."} - {"name":"Python and Scientific Computing","description":"Advanced Python programming skills with experience using scientific computing libraries like NumPy, Pandas, or similar tools for data manipulation and analysis at scale."} - {"name":"Problem Solving Under Ambiguity","description":"Comfort with ill-defined problems and ability to research, decompose complex challenges into concrete, measurable components with clear success criteria."} ### experience - {"name":"3-5 Years of Software Engineering","description":"Solid professional experience building production systems, with proven ability to own features end-to-end and deliver high-quality engineering solutions in team environments."} - {"name":"Data Infrastructure or ML Systems Background","description":"Background in data engineering, ML infrastructure, backend systems, or distributed systems. Experience with training machine learning models, optimization, or large-scale data processing is valuable."} - {"name":"Analytical Mindset with Shipping Velocity","description":"Track record of moving quickly while maintaining high code quality. Ability to balance perfectionism with pragmatism, knowing when to iterate versus when to over-engineer."} ## Skills ### required - {"name":"Software Engineering Fundamentals","description":"Strong ability to write clean, efficient, and maintainable code with proficiency in version control, testing, and code design patterns essential for production systems."} - {"name":"Data Systems and Pipeline Design","description":"Experience building, maintaining, and optimizing data pipelines and systems that process large-scale datasets, with understanding of data quality, validation, and lineage."} - {"name":"Python Programming","description":"Proficiency in Python for data processing, scripting, and system development, which is foundational for RL data infrastructure and model training workflows."} - {"name":"Machine Learning Concepts","description":"Understanding of ML fundamentals including training loops, loss functions, evaluation metrics, and how datasets impact model behavior and performance."} - {"name":"Problem Decomposition","description":"Ability to break down complex, ambiguous problems into measurable, concrete tasks and define appropriate success metrics and evaluation criteria."} ### preferred - {"name":"Reinforcement Learning Experience","description":"Prior experience with RL algorithms, training environments, reward design, or agent development is valuable but not required for this role."} - {"name":"Distributed Systems","description":"Background working with distributed computing, parallel processing, or scalable infrastructure for handling large-scale machine learning workloads."} - {"name":"Infrastructure and DevOps","description":"Experience with data infrastructure, experiment tracking systems, or backend services that support ML operations and scalability."} - {"name":"Behavioral Analysis and Data Exploration","description":"Skill in analyzing complex system behavior, reading logs and traces, and discovering patterns or failure modes in real-world agent performance."} - {"name":"AI/ML Model Training","description":"Experience training large language models, coding models, or other frontier AI systems at scale with understanding of dataset quality impact."} ## Tech stack ### tools - {"name":"Git and Version Control","description":"Essential for managing code repositories, collaboration, and maintaining the infrastructure that defines training tasks and evaluation systems."} - {"name":"Experiment Tracking Tools","description":"Tools like Weights & Biases or MLflow for tracking model training experiments, dataset versions, and evaluation results to measure improvements."} - {"name":"Jupyter Notebooks","description":"Interactive environment for exploratory data analysis, understanding agent traces, and prototyping reward functions and task designs."} ### others - {"name":"RESTful APIs","description":"Building and consuming APIs for integration between data systems, model training infrastructure, and evaluation platforms."} - {"name":"Docker and Containerization","description":"Containerizing reproducible training environments and ensuring consistent execution of RL training tasks across distributed systems."} ### databases - {"name":"Data Warehousing Systems","description":"Working with scalable data storage for large-scale training datasets, agent traces, and evaluation results from model training runs."} ### languages - {"name":"Python","description":"Primary language for data pipeline development, evaluation scripts, reward function implementation, and ML tooling in the RL training infrastructure."} - {"name":"SQL","description":"Essential for querying training datasets, analyzing agent traces, and working with data warehouses storing model training and evaluation data."} ### frameworks - {"name":"PyTorch","description":"Common framework for defining RL environments, reward functions, and integration with model training pipelines at Cursor."} - {"name":"Ray RLlib or OpenAI Gym","description":"Standard frameworks for defining RL environments, training agents, and managing distributed training infrastructure for reinforcement learning systems."} ## Benefits ### benefits - {"name":"Equity Compensation","description":"Significant stock option grants as an early-stage employee at Cursor, a well-funded AI startup working on frontier technology with potential for substantial upside."} - {"name":"Health and Wellness","description":"Comprehensive health insurance coverage including medical, dental, and vision plans for employees and their families."} - {"name":"Professional Development","description":"Access to learning resources, conference attendance budgets, and opportunities to work on cutting-edge AI research and machine learning systems."} - {"name":"Flexible Work Arrangement","description":"Remote-friendly work environment with flexibility to work from San Francisco or distributed, enabling work-life balance while collaborating with a talent-dense team."} - {"name":"Collaborative Culture","description":"Flat organizational structure with direct access to leadership, emphasis on spirited technical debate, and an environment that values creative problem-solving and shipping fast."} - {"name":"Meaningful Impact","description":"Direct influence on frontier coding agent capabilities that shape the future of AI-assisted software development, working on problems at the frontier of reinforcement learning and code generation."} ## Compensation - **max:** 280000 - **min:** 180000 - **currency:** USD - **stockOptions:** true ## Interview process ### steps - {"name":"Initial Screening","description":"Phone or video screening with a recruiter to assess background, experience with data systems or RL, and general fit with Cursor's mission and culture focused on building productive tools."} - {"name":"Technical Problem Discussion","description":"Conversation with a senior engineer covering data systems design, problem decomposition, and how you approach building reliable infrastructure. Expect discussions around real-world tradeoffs in data pipeline design."} - {"name":"RL and Data Fundamentals","description":"Technical discussion with the RL Data team lead exploring your understanding of reinforcement learning concepts, task design, evaluation metrics, and how you think about dataset quality and agent performance measurement."} - {"name":"System Design and Collaboration","description":"Work through a realistic scenario involving designing a training task, defining rewards, and building infrastructure to scale it. This assesses how you think about user-facing problems and collaborate across teams."} - {"name":"Leadership and Culture Fit","description":"Conversation with Cursor leadership exploring your approach to shipping code, handling ambiguity, and working in a flat organization. Emphasis on truth-seeking, creative problem-solving, and working with a high-performance team."} ## Full description Our mission is to automate coding. The first step in our journey is to build the best tool for professional programmers, using a combination of inventive research, design, and engineering. Our organization is very flat, and our team is small and talent dense. We particularly like people who are truth-seeking, passionate, and creative. We enjoy spirited debate, crazy ideas, and shipping code. # Software Engineer, Reinforcement learning Cursor is building the future of coding. We train [frontier coding agents](https://cursor.com/blog/composer) and scale RL on real user data to make them increasingly effective. ## About the role As a Software Engineer on the RL Data team at Cursor, you'll create the tasks, rewards, and environments that train our coding agents. The team owns the data that goes into training: what the model is asked to do, how we score it, and the setups it learns in. ## **What you’ll do** * Designing a task set that teaches a specific agent capability, then iterating on it from traces and evals until the model actually gets better. * Reading a pile of agent traces, finding a failure mode or a surprising behavior, and building a system that surfaces more of the same. * Turning a one-off recipe into something other teams can reuse: better rewards, cleaner environments, tighter data quality. * Partnering with research on whether a dataset is actually teaching the thing we think it is. ## **You may be a fit if** * You write careful, fast code and have strong software engineering fundamentals. * You like setting tasks: breaking a fuzzy capability into something concrete you can measure. * You have an infra, data, or distributed systems background. RL experience is a plus, not a requirement. * You enjoy looking at messy real-world agent behavior and turning it into a dataset or a tool.