Forward Deployed AI Engineer (New York)
Edra5 days ago
Location
New York
Type
Full Time
Salary
USD 195,000 – 275,000
Level
Mid
Role
AI Engineer
Posted
Jul 20, 2026
Full TimeMid
The role
Summary
Edra is seeking a Forward Deployed AI Engineer to build production-grade LLM systems and multi-agent AI solutions for enterprise customers. You'll own the complete technical journey—from understanding customer operations to designing scalable AI architectures, implementing agentic workflows, and driving adoption. This role requires deep experience building complex LLM-based systems, designing evaluation frameworks, and balancing automation with human oversight in production environments.
What you'll do
Enterprise Customer Engagement: Work directly with enterprise customers throughout the complete engagement lifecycle—from initial discovery and problem identification through solution prototyping, production deployment, user adoption, and iterative expansion. Develop deep understanding of customer operations and translate business requirements into technical architectures.
LLM System Architecture & Design: Design and architect production-grade LLM-powered systems and agentic workflows from concept through deployment. Make critical architectural decisions for scalability, reliability, and performance. Build multi-layered systems that handle complex AI orchestration patterns and manage end-to-end request flows.
Agentic Workflow Implementation: Develop autonomous agent systems capable of executing complex enterprise operations. Build agentic features for knowledge base management, including agents that can autonomously edit, update, and maintain large-scale knowledge repositories. Implement agent orchestration patterns and multi-turn reasoning capabilities.
Reliability & Confidence Systems: Design and implement evaluation frameworks, confidence scoring mechanisms, and human-in-the-loop escalation logic. Build systems that intelligently determine when to automate operations versus escalate to human operators. Develop robust monitoring and validation systems for production deployments.
Context Learning Library Extension: Contribute to and extend Edra's core context-learning platform. Build reusable capabilities that solve specific customer problems while creating generalizable components that benefit the broader platform. Balance product-specific solutions with platform-wide improvements.
Scalable System Architecture: Architect asynchronous, scalable systems capable of handling complex AI orchestration at enterprise scale. Design systems that efficiently process millions of requests while maintaining low latency and high reliability. Implement distributed tracing, performance optimization, and capacity planning.
Production Code Development: Write production-quality code with strong architectural decisions and comprehensive testing. Own code quality, performance optimization, and system reliability. Implement best practices for LLM application development including prompt engineering, context management, and error handling.
Technical Problem-Solving: Navigate open-ended problems where solutions may not yet exist. Conduct deep technical investigation of customer problems, research relevant techniques from academic literature, and prototype novel approaches. Document technical decisions and communicate complex concepts to diverse audiences.
System Adoption & Impact: Drive end-user adoption of deployed systems within customer organizations. Gather feedback, iterate on implementations, and expand system impact across new use cases. Measure success metrics and continuously improve system performance and user experience.
Knowledge Transfer & Documentation: Maintain comprehensive technical documentation of system architectures, deployment procedures, and operational guidelines. Create clear explanations of complex technical work for both technical teammates and non-technical stakeholders. Contribute to company knowledge base and best practices.
Human-in-the-Loop Systems: Design and implement feedback mechanisms for continuous system improvement. Build systems that learn from human corrections and validation. Implement prompt optimization strategies and context engineering based on real-world feedback patterns.
Production Deployment & Operations: Manage full lifecycle of production deployments including environment setup, performance monitoring, troubleshooting, and optimization. Implement CI/CD pipelines, automated testing, and rollout strategies. Be responsible for system reliability and incident response.
What we look for
Technical
Complex Production LLM SystemsDemonstrated expertise building sophisticated LLM-based systems in production with multiple layers of engineering decisions. Experience should include advanced prompt engineering, context management, token optimization, and handling edge cases at scale.
Multi-Agent Architecture DesignProven experience designing and building multi-agent systems capable of coordinating complex workflows. Understanding of agent reasoning patterns, tool calling, state management, and inter-agent communication.
Evaluation Framework DevelopmentExperience designing evaluation frameworks and confidence scoring systems for LLM outputs. Knowledge of metrics for assessing model behavior, reliability, and hallucination detection.
Async & Scalable System ArchitectureStrong expertise architecting asynchronous, scalable systems capable of handling enterprise-scale workloads. Experience with distributed systems patterns, queueing systems, event-driven architectures, and performance optimization.
Prompt Engineering & Context EngineeringExpertise in advanced prompt engineering techniques, few-shot learning, chain-of-thought prompting, and dynamic context management. Experience optimizing prompts based on real-world performance data.
Human-in-the-Loop SystemsExperience implementing systems that incorporate human feedback, active learning, and iterative improvement mechanisms. Understanding of how to balance automation with human judgment and oversight.
Production Deployment & DevOpsProficiency with production deployment pipelines, containerization, monitoring, and operational best practices. Experience managing systems in AWS, GCP, or similar cloud environments.
System Integration & Data EngineeringExperience integrating with diverse data sources including knowledge bases, APIs, databases, and system logs. Comfortable building data pipelines and handling data quality challenges.
API Design & IntegrationExperience designing and building APIs that serve LLM systems and external integrations. Understanding of REST/GraphQL patterns, rate limiting, authentication, and backward compatibility.
Testing & Quality AssuranceStrong testing practices for AI systems including unit testing, integration testing, and strategy for testing non-deterministic LLM outputs.
Debugging & ObservabilityAbility to debug complex distributed systems and LLM applications. Experience with logging, tracing, metrics, and debugging AI model behavior.
Problem-Solving Under UncertaintyComfort navigating ambiguous technical problems without clear solutions, conducting research, and prototyping novel approaches.
Education
Computer Science or Related FieldBachelor's degree in Computer Science, Software Engineering, Mathematics, Physics, or related field preferred, though exceptional candidates with demonstrated expertise from alternative backgrounds are considered.
Machine Learning FundamentalsUnderstanding of machine learning concepts, including neural networks, transformers, and large language model architectures. Self-taught expertise in LLMs is acceptable if demonstrated through production work.
Advanced Mathematics or StatisticsComfort reading and interpreting machine learning research papers, understanding experimental methodologies, and statistical evaluation approaches.
Experience
3+ Years Professional Software EngineeringMinimum three years of professional software engineering experience with demonstrated progression in responsibility and technical complexity.
2+ Years LLM/Generative AI DevelopmentSignificant hands-on experience specifically with large language models and generative AI systems, including shipping LLM applications to production.
Enterprise Software ExposureExperience working with or building for enterprise customers, understanding enterprise requirements, security, scalability, and operational constraints.
Production System OwnershipTrack record of owning systems in production with end-to-end responsibility from design through operation and maintenance.
Cross-functional CollaborationExperience working effectively with diverse teams including researchers, product managers, and customers to translate requirements into technical solutions.
Startup or High-Growth EnvironmentExperience in fast-paced environments where requirements evolve and you must be self-directed with minimal oversight.
Skills
Required skills
PythonAdvanced proficiency in Python for building production LLM systems, data processing, and backend services. Experience with async patterns, type safety, and performance optimization.
Large Language ModelsDeep expertise with LLM APIs (OpenAI, Anthropic, open-source models), including understanding of model capabilities, limitations, and cost optimization.
Prompt EngineeringAdvanced prompt engineering techniques including few-shot learning, chain-of-thought prompting, and ability to optimize prompts based on real-world behavior.
Distributed SystemsUnderstanding of distributed systems concepts including eventual consistency, message passing, rate limiting, and failure handling.
Database & Data ManagementExperience with both SQL and NoSQL databases, vector databases for semantic search, and data pipeline design.
API DevelopmentProficiency building REST or GraphQL APIs, understanding authentication, rate limiting, versioning, and API best practices.
System DesignAbility to design scalable, reliable systems with attention to performance, reliability, and operational concerns.
Production DeploymentExperience with containerization (Docker), orchestration, CI/CD pipelines, monitoring, and production operational practices.
Testing & QualityExpertise writing comprehensive tests including unit, integration, and end-to-end testing. Experience testing non-deterministic AI systems.
Nice to have
Agentic AI FrameworksExperience with agentic AI frameworks such as LangChain, LlamaIndex, Autogen, or similar tools for building autonomous agent systems.
Vector DatabasesExperience with vector databases like Pinecone, Weaviate, Qdrant, or Milvus for semantic search and retrieval-augmented generation (RAG).
ML Observability ToolsFamiliarity with ML observability and monitoring tools for tracking model performance, drift, and production behavior.
TypeScript/Node.jsExperience with TypeScript and Node.js for building scalable backend services and full-stack AI applications.
KubernetesExperience with Kubernetes for container orchestration and managing production deployments at scale.
Retrieval-Augmented GenerationHands-on experience implementing RAG systems combining language models with external knowledge sources.
Fine-tuning & Model CustomizationExperience fine-tuning or customizing language models for specific use cases and domains.
Enterprise Software ArchitectureExperience with enterprise architectural patterns, API gateways, authentication systems, and multi-tenant considerations.
Customer-Facing Technical WorkExperience working directly with customers to understand requirements, prototype solutions, and gather feedback.
Research & Academic BackgroundFamiliarity with reading and implementing techniques from machine learning research papers.
AWS or GCPProficiency with major cloud platforms including compute, storage, messaging, and monitoring services.
Real-time SystemsExperience building systems with strict latency requirements or handling high-frequency data processing.
Compensation & benefits
Salary
USD 195,000 – 275,000 (annual)
Stock options
Available
Benefits
Equity Compensation
Significant equity stake in a Series A startup backed by Sequoia and leading venture capital firms with strong growth trajectory
Competitive Salary
Market-competitive compensation for AI engineering talent in New York tech market
Comprehensive Health Coverage
Full health, vision, and dental insurance coverage with employer contributions
Flexible Work Arrangements
Flexibility to work in New York or London offices, with work-from-home options for key activities
Professional Development
Budget and time for learning, attending conferences, and staying current with AI/ML advancements
Mentor Access
Work alongside AI researchers and experienced engineers from top-tier backgrounds
Technical Leadership
Opportunity to own technical decisions and drive architectural direction for AI systems
Impact-Driven Work
Direct impact on enterprise operations through AI automation and agent-based systems
Customer Proximity
Direct engagement with enterprise customers allowing you to see the real-world impact of your work
Startup Upside
Early-stage opportunity to shape product direction and benefit from company growth as it scales
Apply for this position
You'll be redirected to the company's application page
Edra
View all jobs
Edra is an artificial intelligence company that develops advanced AI solutions and platforms. The company focuses on creating intelligent systems and tools that help businesses leverage machine learning and artificial intelligence technologies. Operating in the competitive AI and software market, Edra aims to provide accessible and powerful AI capabilities to organizations seeking to enhance their operations and decision-making processes through data-driven insights and automation.
edra.ai
Tech Stack
Languages
PythonTypeScript/JavaScriptSQL
Frameworks
FastAPILangChainLlamaIndexPydanticCeleryExpress.js or FastifyAgentic Frameworks
Databases
PostgreSQLVector DatabasesRedisMongoDBElasticsearch
Tools
DockerKubernetesAWS or GCPGitGitHub Actions or GitLab CIPrometheus & GrafanaJupyter NotebooksClaude API, OpenAI API
Other
Retrieval-Augmented Generation (RAG)Prompt EngineeringMulti-Agent SystemsHuman-in-the-Loop FeedbackConfidence Scoring & EvaluationAsync Architecture PatternsAPI Rate Limiting & QuotasLLM Cost Optimization