Principal Software Engineer, LLM Gateway
Principal Engineer · Principal · Full Time
Opens UiPath's application page
Role
What you'll do.
As a Principal Software Engineer on UiPath's LLM Gateway team, you'll architect and lead the AI Trust Layer—a globally distributed, high-throughput gateway serving as the critical infrastructure for every UiPath product and agent built by customers to access models from OpenAI, Anthropic, Google, and other providers. This role demands 7+ years of backend and cloud development expertise with proven ability to design secure, scalable distributed systems that operate seamlessly across public cloud, sovereign clouds, and customer-hosted environments while managing intelligent routing, rate limiting, cost controls, and comprehensive data residency guarantees.
Responsibilities
- Lead Architecture and Technical Direction: Own the technical strategy and architecture of UiPath's AI Trust Layer and LLM Gateway infrastructure, serving as the critical proxy layer for all AI model access. Define the technical roadmap for complex backend and cloud-native services that support text, image, voice, and emerging modalities with intelligent routing and cost optimization.
- Design Distributed Systems at Scale: Architect secure, reliable, and high-performing distributed systems capable of globally distributed, high-throughput operations across multiple cloud environments. Design systems that handle intelligent routing, rate limiting, cost controls, and model capacity balancing across vendors, tiers, and regions with zero data loss and guaranteed data residency.
- Manage LLM Infrastructure Operations: Own and optimize the LLM infrastructure that sources, balances, and manages model capacity across vendors, tiers, and geographic regions. Ensure seamless operation in public cloud, government clouds, sovereign clouds, and customer-hosted deployments with consistent performance and cost efficiency.
- Drive Customer Access and Governance Controls: Lead the design and implementation of comprehensive AI governance and access control systems that enable customers to determine which models their users and automations can access, including customer-provided models. Implement policies, data protection mechanisms, audit trails, and compliance frameworks that meet enterprise security standards.
- Cross-Functional Leadership and Influence: Partner strategically with product teams, security, legal, model vendors, and engineering leaders to shape technical solutions and drive architecture decisions across teams. Influence organizational technical direction and establish engineering standards for system design, code quality, testing, observability, and incident management.
- Technical Mentorship and Engineering Excellence: Raise the engineering bar of the team through hands-on technical leadership, mentoring, and guidance. Conduct design reviews, lead technical discussions, provide architecture feedback, and foster a culture of excellence in distributed systems thinking, performance optimization, and production operations.
- Identify and Solve Long-Term Challenges: Proactively identify scalability, reliability, security, performance, and cost challenges in the LLM Gateway infrastructure. Lead the design and implementation of sustainable solutions that balance customer experience, system reliability, security posture, scalability requirements, and cloud cost optimization.
Qualifications
What we look for.
Technical
Distributed Systems Architecture
Proven expertise designing and building production-grade distributed systems including microservices architectures, API design patterns, service mesh approaches, and cloud-native deployment patterns. Experience with high-throughput proxy systems, intelligent routing mechanisms, rate limiting strategies, and global load balancing across multiple regions and cloud providers.
Cloud Platform Operations
Deep hands-on experience building and operating production services in major cloud environments including Azure, AWS, or GCP. Demonstrated expertise with container orchestration via Kubernetes, infrastructure-as-code, CI/CD pipeline design, service mesh implementations, and multi-cloud deployment strategies with data residency considerations.
Backend Programming Languages
Strong hands-on proficiency with one or more modern programming languages including C#, Java, Python, or Go. Ability to make architectural decisions about language selection based on performance requirements, operational characteristics, team expertise, and ecosystem maturity for backend and gateway services.
Data and Messaging Systems
Comprehensive experience with relational databases, NoSQL database designs, distributed cache patterns, message broker architectures, event-driven system design, and high-scale data processing. Understanding of eventual consistency models, transaction management, and data flow optimization for high-throughput systems.
Performance and Reliability Engineering
Advanced knowledge of system design principles, data structures, algorithm optimization, multithreading models, asynchronous programming patterns, and concurrency control. Experience with performance profiling, bottleneck identification, reliability patterns, fault tolerance mechanisms, and observability architecture.
Production Operations and Observability
Demonstrated expertise with production monitoring, logging aggregation, distributed tracing, metrics collection, and observability platform design. Experience with incident management, on-call procedures, alerting strategies, and SLO/SLI definitions for highly available systems.
Education
Computer Science or Related Degree (Optional)
Bachelor's degree in Computer Science, Software Engineering, or related technical field is preferred but not required. Equivalent professional experience demonstrating deep technical knowledge of distributed systems and software architecture can substitute for formal education.
Experience
Backend and Cloud Development
Minimum 7+ years of professional software engineering experience with substantial focus on backend development and cloud-native architectures. Track record of designing and implementing systems at scale in production environments serving enterprise customers.
Technical Leadership and Influence
Proven ability to influence technical direction and drive architecture decisions across engineering teams. Demonstrated impact on engineering culture, standards, and practices through mentorship and technical thought leadership.
Enterprise Software or Platform Engineering
Experience building developer platforms, enterprise software infrastructure, large-scale cloud systems, or automation platforms. Understanding of the challenges specific to enterprise deployments including compliance, security, multi-tenancy, and global operations.
Skills
Required
Distributed Systems Design
Ability to design fault-tolerant, scalable, and secure distributed systems with focus on consistency models, consensus algorithms, service discovery, load balancing, and cross-region failover.
Cloud Architecture
Expertise in designing cloud-native applications leveraging managed services, serverless patterns, containerization, orchestration, and multi-cloud strategies.
Microservices and APIs
Deep understanding of microservices design patterns, RESTful and asynchronous API design, service boundaries, dependency management, and inter-service communication patterns.
System Design and Trade-off Analysis
Ability to balance multiple competing concerns including performance, cost, security, reliability, and user experience when making architectural decisions.
Technical Leadership
Demonstrated ability to guide teams through technical ambiguity, mentor engineers at various levels, conduct effective design reviews, and drive consensus on complex architectural decisions.
Backend Programming
Production-level proficiency in one or more modern backend languages such as C#, Java, Python, or Go with understanding of language-specific idioms and ecosystem.
Preferred
AI and LLM Infrastructure
Nice to haveExperience with large language model infrastructure, model serving platforms, prompt optimization, fine-tuning pipelines, or AI-powered system design. Knowledge of model governance, prompt caching, or token optimization strategies.
Automation and RPA Experience
Nice to haveBackground in robotic process automation, business process automation, or low-code/no-code platforms that demonstrate understanding of enterprise automation challenges and solutions.
Security and Compliance
Nice to haveDeep expertise in API security, data encryption, compliance frameworks (SOC2, HIPAA, GDPR), audit logging, and threat modeling for cloud-native systems.
Gateway and Proxy Architecture
Nice to haveSpecific experience building API gateways, reverse proxies, service mesh data planes, or similar infrastructure components with intelligent routing and policy enforcement.
Developer Platform Engineering
Nice to haveExperience building developer-focused platforms, internal developer platforms (IDP), or infrastructure-as-a-service solutions with focus on developer experience and self-service.
Cost Optimization
Nice to haveDemonstrated ability to optimize cloud infrastructure costs through architectural decisions, resource right-sizing, usage monitoring, and FinOps practices while maintaining performance.
Tech stack
Languages
Frameworks
Databases
Tools
Other
Compensation
Pay and benefits.
Base·USD 180,000 – 280,000
Equity·Stock options
Benefits
Comprehensive Health Coverage
Medical, dental, and vision insurance with company contribution covering eligible dependents and preventive care
Retirement Planning
401(k) matching program to support long-term financial planning and wealth accumulation
Equity Compensation
Competitive stock options enabling participation in UiPath's growth and long-term value creation
Flexible Work Arrangements
Hybrid, remote, and office-based flexibility depending on business needs and team requirements with location flexibility for many roles
Paid Time Off
Generous vacation, personal days, and paid time off policies supporting work-life balance and employee wellness
Professional Development
Learning stipends, conference attendance, training programs, and mentorship opportunities for continuous skill development
Parental Leave
Comprehensive parental leave policies supporting employees during major life transitions
Mental Health and Wellness
Access to mental health resources, counseling services, and wellness programs promoting employee wellbeing
Full posting
Original listing.
Life at UiPath
The people at UiPath believe in the transformative power of automation to change how the world works. We’re committed to creating category-leading enterprise software that unleashes that power.
To make that happen, we need people who are curious, self-propelled, generous, and genuine. People who love being part of a fast-moving, fast-thinking growth company. And people who care—about each other, about UiPath, and about our larger purpose.
Could that be you?
Your mission
As a Principal Software Engineer on the LLM Gateway team, you will set the technical direction of UiPath’s AI Trust Layer: the gateway through which every UiPath product, and every agent our customers build, reaches models from OpenAI, Anthropic, Google and other commercial and open-weight providers. You will lead the architecture of a secure, globally distributed, high-throughput proxy serving text, image, voice and emerging modalities, with intelligent routing, rate limiting and cost controls, running the same way in our public cloud, in government and sovereign clouds, and in customer-hosted deployments. You will own the LLM infrastructure behind it: sourcing and balancing model capacity across vendors, tiers and regions, with data residency guaranteed in every geography. You will also own how customers see and control AI in their organizations: which models their users and automations can access, including the models they bring themselves, under what conditions, and with what policies, data protection and audit trail. You will raise the engineering bar of the team, work closely with product teams, security, legal and our model vendors, and help shape how we build software with AI agents.
What you'll do at UiPath
Lead the architecture and technical direction of complex backend and cloud-native services across the UiPath platform.
Design scalable, secure, reliable, and high-performing distributed systems across cloud, on-premises, and hybrid environments.
Partner with engineering leaders, architects, product managers, and cross-functional teams to shape technical solutions for high-impact product and platform areas.
Drive technical planning and roadmap input for complex services, platform capabilities, or cross-team initiatives.
Identify long-term scalability, reliability, security, performance, and cost challenges, and lead the design of sustainable solutions.
Set high standards for system design, code quality, testing, observability, automation, incident management, and engineering excellence.
Mentor and guide engineers through design reviews, technical discussions, architecture decisions, and hands-on technical feedback.
What you'll bring to the team
7+ years of software engineering experience, with strong backend and cloud development experience.
Hands-on experience with one or more modern programming languages such as C#, Java, Python, or Go.
Strong experience designing and building distributed systems, microservices, APIs, and cloud-native architectures.
Proven ability to influence technical direction and drive architecture decisions across teams.
Experience building and operating production-grade services in cloud environments such as Azure, AWS, or GCP.
Good knowledge of containers, Kubernetes, CI/CD, monitoring, logging, observability, and production operations.
Strong understanding of system design, data structures, algorithms, multithreading, asynchronous programming, and performance optimization.
Experience with relational and/or NoSQL databases, messaging systems, event-driven architectures, or high-scale data processing.
Ability to balance customer experience, reliability, security, scalability, and cloud cost when making technical decisions.
Strong technical leadership, ownership, and communication skills, with the ability to guide teams through ambiguity.
Interest in automation, AI, developer platforms, enterprise software, or large-scale cloud systems is a plus.
Maybe you don’t tick all the boxes above—but still think you’d be great for the job? Go ahead, apply anyway. Please. Because we know that experience comes in all shapes and sizes—and passion can’t be learned.
Many of our roles allow for flexibility in when and where work gets done. Depending on the needs of the business and the role, the number of hybrid, office-based, and remote workers will vary from team to team. Applications are assessed on a rolling basis and there is no fixed deadline for this requisition. The application window may change depending on the volume of applications received or may close immediately if a qualified candidate is selected.
We value a range of diverse backgrounds, experiences and ideas. We pride ourselves on our diversity and inclusive workplace that provides equal opportunities to all persons regardless of age, race, color, religion, sex, sexual orientation, gender identity, and expression, national origin, disability, neurodiversity, military and/or veteran status, or any other protected classes. Additionally, UiPath provides reasonable accommodations for candidates on request and respects applicants' privacy rights. To review these and other legal disclosures, visit our privacy policy.
Redirects to UiPath's application page.
Other roles
More at UiPath.
Principal Software Engineer, Growth
Principal
Senior Forward Deployed Engineer
Senior
Principal Software Engineer, Growth
Principal
Principal Software Engineer
Principal
Senior Software Engineer - Frontend
Senior