# Principal Software Engineer, LLM Gateway
**Company:** [UiPath](https://scaleengineer.com/companies/uipath)
As a Principal Software Engineer on UiPath's LLM Gateway team, you'll architect and lead the AI Trust Layer—a globally distributed, high-throughput gateway serving as the critical infrastructure for every UiPath product and agent built by customers to access models from OpenAI, Anthropic, Google, and other providers. This role demands 7+ years of backend and cloud development expertise with proven ability to design secure, scalable distributed systems that operate seamlessly across public cloud, sovereign clouds, and customer-hosted environments while managing intelligent routing, rate limiting, cost controls, and comprehensive data residency guarantees.
**Role:** Principal Engineer
**Seniority:** Principal
**Locations:** Bucharest
**Salary:** 180000–280000 USD
[Apply](https://jobs.ashbyhq.com/uipath/cbe38caa-a673-485f-82a2-748701f76dd8)
Canonical: https://scaleengineer.com/jobs/uipath/principal-software-engineer-llm-gateway
---
## Responsibilities

- Lead Architecture and Technical Direction: Own the technical strategy and architecture of UiPath's AI Trust Layer and LLM Gateway infrastructure, serving as the critical proxy layer for all AI model access. Define the technical roadmap for complex backend and cloud-native services that support text, image, voice, and emerging modalities with intelligent routing and cost optimization.
- Design Distributed Systems at Scale: Architect secure, reliable, and high-performing distributed systems capable of globally distributed, high-throughput operations across multiple cloud environments. Design systems that handle intelligent routing, rate limiting, cost controls, and model capacity balancing across vendors, tiers, and regions with zero data loss and guaranteed data residency.
- Manage LLM Infrastructure Operations: Own and optimize the LLM infrastructure that sources, balances, and manages model capacity across vendors, tiers, and geographic regions. Ensure seamless operation in public cloud, government clouds, sovereign clouds, and customer-hosted deployments with consistent performance and cost efficiency.
- Drive Customer Access and Governance Controls: Lead the design and implementation of comprehensive AI governance and access control systems that enable customers to determine which models their users and automations can access, including customer-provided models. Implement policies, data protection mechanisms, audit trails, and compliance frameworks that meet enterprise security standards.
- Cross-Functional Leadership and Influence: Partner strategically with product teams, security, legal, model vendors, and engineering leaders to shape technical solutions and drive architecture decisions across teams. Influence organizational technical direction and establish engineering standards for system design, code quality, testing, observability, and incident management.
- Technical Mentorship and Engineering Excellence: Raise the engineering bar of the team through hands-on technical leadership, mentoring, and guidance. Conduct design reviews, lead technical discussions, provide architecture feedback, and foster a culture of excellence in distributed systems thinking, performance optimization, and production operations.
- Identify and Solve Long-Term Challenges: Proactively identify scalability, reliability, security, performance, and cost challenges in the LLM Gateway infrastructure. Lead the design and implementation of sustainable solutions that balance customer experience, system reliability, security posture, scalability requirements, and cloud cost optimization.

## Requirements

### education

- {"name":"Computer Science or Related Degree (Optional)","description":"Bachelor's degree in Computer Science, Software Engineering, or related technical field is preferred but not required. Equivalent professional experience demonstrating deep technical knowledge of distributed systems and software architecture can substitute for formal education."}

### technical

- {"name":"Distributed Systems Architecture","description":"Proven expertise designing and building production-grade distributed systems including microservices architectures, API design patterns, service mesh approaches, and cloud-native deployment patterns. Experience with high-throughput proxy systems, intelligent routing mechanisms, rate limiting strategies, and global load balancing across multiple regions and cloud providers."}
- {"name":"Cloud Platform Operations","description":"Deep hands-on experience building and operating production services in major cloud environments including Azure, AWS, or GCP. Demonstrated expertise with container orchestration via Kubernetes, infrastructure-as-code, CI/CD pipeline design, service mesh implementations, and multi-cloud deployment strategies with data residency considerations."}
- {"name":"Backend Programming Languages","description":"Strong hands-on proficiency with one or more modern programming languages including C#, Java, Python, or Go. Ability to make architectural decisions about language selection based on performance requirements, operational characteristics, team expertise, and ecosystem maturity for backend and gateway services."}
- {"name":"Data and Messaging Systems","description":"Comprehensive experience with relational databases, NoSQL database designs, distributed cache patterns, message broker architectures, event-driven system design, and high-scale data processing. Understanding of eventual consistency models, transaction management, and data flow optimization for high-throughput systems."}
- {"name":"Performance and Reliability Engineering","description":"Advanced knowledge of system design principles, data structures, algorithm optimization, multithreading models, asynchronous programming patterns, and concurrency control. Experience with performance profiling, bottleneck identification, reliability patterns, fault tolerance mechanisms, and observability architecture."}
- {"name":"Production Operations and Observability","description":"Demonstrated expertise with production monitoring, logging aggregation, distributed tracing, metrics collection, and observability platform design. Experience with incident management, on-call procedures, alerting strategies, and SLO/SLI definitions for highly available systems."}

### experience

- {"name":"Backend and Cloud Development","description":"Minimum 7+ years of professional software engineering experience with substantial focus on backend development and cloud-native architectures. Track record of designing and implementing systems at scale in production environments serving enterprise customers."}
- {"name":"Technical Leadership and Influence","description":"Proven ability to influence technical direction and drive architecture decisions across engineering teams. Demonstrated impact on engineering culture, standards, and practices through mentorship and technical thought leadership."}
- {"name":"Enterprise Software or Platform Engineering","description":"Experience building developer platforms, enterprise software infrastructure, large-scale cloud systems, or automation platforms. Understanding of the challenges specific to enterprise deployments including compliance, security, multi-tenancy, and global operations."}

## Skills

### required

- {"name":"Distributed Systems Design","description":"Ability to design fault-tolerant, scalable, and secure distributed systems with focus on consistency models, consensus algorithms, service discovery, load balancing, and cross-region failover."}
- {"name":"Cloud Architecture","description":"Expertise in designing cloud-native applications leveraging managed services, serverless patterns, containerization, orchestration, and multi-cloud strategies."}
- {"name":"Microservices and APIs","description":"Deep understanding of microservices design patterns, RESTful and asynchronous API design, service boundaries, dependency management, and inter-service communication patterns."}
- {"name":"System Design and Trade-off Analysis","description":"Ability to balance multiple competing concerns including performance, cost, security, reliability, and user experience when making architectural decisions."}
- {"name":"Technical Leadership","description":"Demonstrated ability to guide teams through technical ambiguity, mentor engineers at various levels, conduct effective design reviews, and drive consensus on complex architectural decisions."}
- {"name":"Backend Programming","description":"Production-level proficiency in one or more modern backend languages such as C#, Java, Python, or Go with understanding of language-specific idioms and ecosystem."}

### preferred

- {"name":"AI and LLM Infrastructure","description":"Experience with large language model infrastructure, model serving platforms, prompt optimization, fine-tuning pipelines, or AI-powered system design. Knowledge of model governance, prompt caching, or token optimization strategies."}
- {"name":"Automation and RPA Experience","description":"Background in robotic process automation, business process automation, or low-code/no-code platforms that demonstrate understanding of enterprise automation challenges and solutions."}
- {"name":"Security and Compliance","description":"Deep expertise in API security, data encryption, compliance frameworks (SOC2, HIPAA, GDPR), audit logging, and threat modeling for cloud-native systems."}
- {"name":"Gateway and Proxy Architecture","description":"Specific experience building API gateways, reverse proxies, service mesh data planes, or similar infrastructure components with intelligent routing and policy enforcement."}
- {"name":"Developer Platform Engineering","description":"Experience building developer-focused platforms, internal developer platforms (IDP), or infrastructure-as-a-service solutions with focus on developer experience and self-service."}
- {"name":"Cost Optimization","description":"Demonstrated ability to optimize cloud infrastructure costs through architectural decisions, resource right-sizing, usage monitoring, and FinOps practices while maintaining performance."}

## Tech stack

### tools

- {"name":"Kubernetes","description":"Container orchestration platform for managing distributed deployment of gateway services across regions and cloud environments"}
- {"name":"Docker","description":"Containerization technology for packaging services with consistent environments across development, testing, and production"}
- {"name":"Azure / AWS / GCP","description":"Major cloud platforms for hosting, managing, and scaling the LLM gateway infrastructure globally"}
- {"name":"Terraform","description":"Infrastructure-as-code tool for defining and managing cloud infrastructure in reproducible and version-controlled manner"}
- {"name":"Prometheus","description":"Metrics collection and monitoring system for observability, alerting, and performance tracking of gateway services"}
- {"name":"Grafana","description":"Visualization and dashboarding platform for real-time monitoring and operational visibility"}
- {"name":"ELK Stack / DataDog / New Relic","description":"Distributed logging and APM platforms for comprehensive observability, trace analysis, and performance troubleshooting"}
- {"name":"CI/CD Pipelines","description":"Automated deployment pipelines using GitHub Actions, GitLab CI, or similar for continuous integration and delivery"}
- {"name":"Vault / Secrets Management","description":"Secure credential and secret management systems for handling API keys, model credentials, and sensitive configuration"}

### others

- {"name":"Microservices Architecture","description":"Distributed system design patterns including service boundaries, communication protocols, and operational patterns for scalability"}
- {"name":"API Gateway Patterns","description":"Implementation of routing, rate limiting, authentication, authorization, request transformation, and policy enforcement at the gateway layer"}
- {"name":"Multi-Cloud Deployment","description":"Strategies for consistent deployment and operation across multiple cloud providers with data residency and sovereignty considerations"}
- {"name":"Rate Limiting and Quota Management","description":"Token bucket algorithms, sliding window counters, and distributed rate limiting mechanisms for controlling API consumption"}
- {"name":"Data Residency and Compliance","description":"Technical patterns for ensuring data remains in specified geographic regions while maintaining system performance and reliability"}
- {"name":"LLM Model Serving","description":"Infrastructure and architecture patterns for serving large language models including model versioning, A/B testing, and cost optimization"}
- {"name":"Event-Driven Architectures","description":"Design and implementation of systems based on asynchronous event propagation for decoupling and scalability"}
- {"name":"Observability and SRE Practices","description":"Comprehensive monitoring, logging, tracing, alerting, and incident response procedures following SRE principles"}

### databases

- {"name":"PostgreSQL","description":"Relational database for transactional data, access control policies, audit logs, and metadata storage"}
- {"name":"MongoDB","description":"NoSQL document database for flexible schema requirements in configuration, routing rules, and customer model preferences"}
- {"name":"Redis","description":"In-memory data store for caching, rate limiting state, session management, and high-throughput data access patterns"}
- {"name":"Elasticsearch","description":"Search and analytics engine for audit trail indexing, logging aggregation, and observability data analysis"}

### languages

- {"name":"C#","description":"Primary language for UiPath platform development; strong ecosytem for cloud services and backend APIs on Azure"}
- {"name":"Java","description":"Enterprise backend development language with mature ecosystem for distributed systems, microservices, and cloud-native frameworks"}
- {"name":"Python","description":"Data processing, infrastructure automation, and AI/ML integration scripting for gateway configuration and monitoring"}
- {"name":"Go","description":"Cloud-native development language optimized for building high-performance, scalable backend services and infrastructure components"}

### frameworks

- {"name":".NET Core / .NET 6+","description":"Modern cross-platform framework for building high-performance cloud-native services and microservices with async/await patterns"}
- {"name":"Spring Boot","description":"Enterprise Java framework for building production-grade distributed systems with spring cloud ecosystem for cloud-native features"}
- {"name":"FastAPI","description":"Modern Python framework for building high-performance APIs suitable for gateway and proxy implementations"}
- {"name":"gRPC","description":"High-performance RPC framework for inter-service communication with protocol buffers and multiplexing capabilities"}

## Benefits

### benefits

- {"name":"Comprehensive Health Coverage","description":"Medical, dental, and vision insurance with company contribution covering eligible dependents and preventive care"}
- {"name":"Retirement Planning","description":"401(k) matching program to support long-term financial planning and wealth accumulation"}
- {"name":"Equity Compensation","description":"Competitive stock options enabling participation in UiPath's growth and long-term value creation"}
- {"name":"Flexible Work Arrangements","description":"Hybrid, remote, and office-based flexibility depending on business needs and team requirements with location flexibility for many roles"}
- {"name":"Paid Time Off","description":"Generous vacation, personal days, and paid time off policies supporting work-life balance and employee wellness"}
- {"name":"Professional Development","description":"Learning stipends, conference attendance, training programs, and mentorship opportunities for continuous skill development"}
- {"name":"Parental Leave","description":"Comprehensive parental leave policies supporting employees during major life transitions"}
- {"name":"Mental Health and Wellness","description":"Access to mental health resources, counseling services, and wellness programs promoting employee wellbeing"}

## Compensation

- **max:** 280000
- **min:** 180000
- **currency:** USD
- **stockOptions:** true

## Interview process

### steps

## Full description
# **Life at UiPath**

The people at UiPath believe in the transformative power of automation to change how the world works. We’re committed to creating category-leading enterprise software that unleashes that power.

To make that happen, we need people who are curious, self-propelled, generous, and genuine. People who love being part of a fast-moving, fast-thinking growth company. And people who care—about each other, about UiPath, and about our larger purpose.

Could that be you?

# **Your mission**

As a Principal Software Engineer on the LLM Gateway team, you will set the technical direction of UiPath’s AI Trust Layer: the gateway through which every UiPath product, and every agent our customers build, reaches models from OpenAI, Anthropic, Google and other commercial and open-weight providers. You will lead the architecture of a secure, globally distributed, high-throughput proxy serving text, image, voice and emerging modalities, with intelligent routing, rate limiting and cost controls, running the same way in our public cloud, in government and sovereign clouds, and in customer-hosted deployments. You will own the LLM infrastructure behind it: sourcing and balancing model capacity across vendors, tiers and regions, with data residency guaranteed in every geography. You will also own how customers see and control AI in their organizations: which models their users and automations can access, including the models they bring themselves, under what conditions, and with what policies, data protection and audit trail. You will raise the engineering bar of the team, work closely with product teams, security, legal and our model vendors, and help shape how we build software with AI agents.

# **What you'll do at UiPath**

* Lead the architecture and technical direction of complex backend and cloud-native services across the UiPath platform.
* Design scalable, secure, reliable, and high-performing distributed systems across cloud, on-premises, and hybrid environments.
* Partner with engineering leaders, architects, product managers, and cross-functional teams to shape technical solutions for high-impact product and platform areas.
* Drive technical planning and roadmap input for complex services, platform capabilities, or cross-team initiatives.
* Identify long-term scalability, reliability, security, performance, and cost challenges, and lead the design of sustainable solutions.
* Set high standards for system design, code quality, testing, observability, automation, incident management, and engineering excellence.
* Mentor and guide engineers through design reviews, technical discussions, architecture decisions, and hands-on technical feedback.

# **What you'll bring to the team**

* 7+ years of software engineering experience, with strong backend and cloud development experience.
* Hands-on experience with one or more modern programming languages such as C#, Java, Python, or Go.
* Strong experience designing and building distributed systems, microservices, APIs, and cloud-native architectures.
* Proven ability to influence technical direction and drive architecture decisions across teams.
* Experience building and operating production-grade services in cloud environments such as Azure, AWS, or GCP.
* Good knowledge of containers, Kubernetes, CI/CD, monitoring, logging, observability, and production operations.
* Strong understanding of system design, data structures, algorithms, multithreading, asynchronous programming, and performance optimization.
* Experience with relational and/or NoSQL databases, messaging systems, event-driven architectures, or high-scale data processing.
* Ability to balance customer experience, reliability, security, scalability, and cloud cost when making technical decisions.
* Strong technical leadership, ownership, and communication skills, with the ability to guide teams through ambiguity.
* Interest in automation, AI, developer platforms, enterprise software, or large-scale cloud systems is a plus.

Maybe you don’t tick all the boxes above—but still think you’d be great for the job? Go ahead, apply anyway. Please. Because we know that experience comes in all shapes and sizes—and passion can’t be learned.

Many of our roles allow for flexibility in when and where work gets done. Depending on the needs of the business and the role, the number of hybrid, office-based, and remote workers will vary from team to team. Applications are assessed on a rolling basis and there is no fixed deadline for this requisition. The application window may change depending on the volume of applications received or may close immediately if a qualified candidate is selected.

We value a range of diverse backgrounds, experiences and ideas. We pride ourselves on our diversity and inclusive workplace that provides equal opportunities to all persons regardless of age, race, color, religion, sex, sexual orientation, gender identity, and expression, national origin, disability, neurodiversity, military and/or veteran status, or any other protected classes. Additionally, UiPath provides reasonable accommodations for candidates on request and respects applicants' privacy rights. To review these and other legal disclosures, visit our [privacy policy](https://www.uipath.com/legal/trust-and-security/privacy-policy).