Senior Data Engineer
Data Engineer · Senior · Full Time
Opens Codat's application page
Role
What you'll do.
Join Codat's Data and Insights team as a Senior Data Engineer, building production data pipelines and shaping the technical direction of our advisory intelligence platform. This hands-on senior role combines practical software engineering with technical leadership, requiring expertise in Python, modern data engineering tools (SQL, Spark, Databricks, Dagster/Airflow), and cloud infrastructure. You'll drive the development of AI-ready data infrastructure while maintaining production excellence and communicating technical strategy across engineering and business stakeholders.
Responsibilities
- Production Code Development & Pipeline Architecture: Write and maintain production-grade Python code daily, designing and building data pipelines that power Codat's Insights products. Take ownership of complex data transformation logic, implementing robust error handling, retry mechanisms, and data validation across the full pipeline architecture from ingestion through to analytics consumption.
- Full Project Lifecycle Ownership: Own projects end-to-end from domain analysis through pragmatic system design, implementation, deployment, and production reliability. Conduct thorough data domain exploration, design scalable solutions, manage shipping timelines, and maintain operational stability with proactive monitoring and incident response.
- Technical Direction & Platform Strategy: Set and lead the technical roadmap for the Insights platform, making architectural decisions around data modeling, processing patterns, and infrastructure choices. Document and communicate technical strategies clearly to engineering teams, product managers, and commercial stakeholders, ensuring alignment on trade-offs and rationale.
- Engineering Standards & Code Quality Leadership: Elevate engineering standards across the Data and Insights team through exemplary practices including comprehensive testing strategies, observability implementation, data quality frameworks, and clean code principles. Mentor team members on production-grade engineering practices and establish best practices for maintainability.
- AI Integration & Innovation: Identify and implement AI applications across data pipelines and products where they deliver measurable value. Move beyond code generation to strategic AI usage including domain research, knowledge building, rapid prototyping, and operational applications such as automated pipeline diagnostics and intelligent troubleshooting agents.
- AI-Ready Data Infrastructure: Design and implement foundations for semantic layers, MCP (Model Context Protocol) support, and ontology-driven data organization. Build data systems that enable both human querying and AI system reasoning, supporting enterprise-grade data access patterns and intelligent data discovery capabilities.
Qualifications
What we look for.
Technical
Python Production Engineering
Expert-level Python proficiency with proven ability to write well-tested, production-ready code at scale. Deep understanding of software engineering best practices including SOLID principles, design patterns, testing frameworks, and code maintainability. Experience building complex data processing applications with emphasis on reliability and operational excellence.
Data Pipeline Architecture & Development
Proven track record designing and building data pipelines from first principles, moving beyond configuration-focused approaches. Strong experience implementing end-to-end data flows including extraction, transformation, loading, and quality assurance. Hands-on expertise with modern orchestration tools such as Dagster, Airflow, or Temporal for workflow management and scheduling.
Modern Data Engineering Stack
Deep expertise in multiple data engineering tools with particular strength in SQL for complex transformations, Apache Spark for distributed processing, Databricks and Delta Lake for lakehouse patterns, and dbt for data transformation workflows. Solid understanding of data warehousing architecture and analytics engineering practices.
Cloud Infrastructure & Deployment
Hands-on experience with cloud-based infrastructure (AWS, GCP, or Azure) and containerization using Docker. Proficiency with CI/CD pipelines and modern deployment practices. Ideally, you've shaped deployment architecture and best practices for teams, not just operated within established systems.
Data Quality & Observability
Experience implementing data quality frameworks, data validation logic, and comprehensive observability for data systems. Proficiency in monitoring, alerting, and debugging data pipeline issues. Understanding of data lineage, data catalog principles, and data governance practices.
AI-Augmented Development
Demonstrated ability to integrate AI tools into development workflow beyond code generation, including research acceleration, domain knowledge exploration, and rapid prototyping. Experience building or operating AI systems within data platforms, such as anomaly detection agents, intelligent diagnostics, or LLM-powered data applications.
Education
Computer Science Foundation
Bachelor's degree in Computer Science, Data Engineering, Software Engineering, Mathematics, or related field. Equivalent professional experience demonstrating strong computer science fundamentals is acceptable.
Continuous Learning Commitment
Evidence of continuous professional development in data engineering, cloud platforms, and emerging technologies. Active participation in engineering communities, conferences, or open-source projects demonstrates commitment to staying current with industry evolution.
Experience
Data Engineering Leadership
5+ years building production data systems and pipelines, with at least 2-3 years in a senior or lead data engineering capacity. Demonstrated ability to influence technical direction, establish architectural patterns, and mentor junior engineers. Track record of shipping complex data projects that delivered measurable business impact.
Complex Data Domain Expertise
Hands-on experience with complex data domains such as financial data, high-volume transaction processing, or multi-source data integration. Understanding of business intelligence, analytics engineering, and data-driven decision making. Comfortable deep-diving into unfamiliar domains and learning quickly.
Fintech or Financial Services Context (Preferred)
Experience working with financial data, banking systems, or fintech platforms is valuable. Understanding of data regulatory requirements, compliance considerations, and financial data standards strengthens your fit for Codat's commercial banking advisory platform.
Skills
Required
Python
Production-grade Python expertise with proficiency in writing clean, well-tested, maintainable code. Strong understanding of Python ecosystem including package management, testing frameworks (pytest, unittest), and performance optimization.
SQL
Advanced SQL knowledge for complex data transformations, optimization, and query performance tuning. Ability to write efficient queries against large datasets and design robust data models.
Apache Spark or Distributed Data Processing
Solid experience with Apache Spark, Databricks, or similar distributed processing frameworks for large-scale data transformation and analytics pipelines.
Data Pipeline Orchestration
Hands-on experience with modern orchestration tools such as Dagster, Apache Airflow, or Temporal for scheduling, monitoring, and managing complex data workflows at scale.
dbt or Modern ELT Tools
Experience with dbt (data build tool) or similar modern ELT tools for analytics engineering, data transformation version control, and documentation practices.
Docker & Containerization
Proficiency with Docker for containerizing applications, building reproducible environments, and enabling consistent deployment across infrastructure.
CI/CD & Deployment
Strong understanding of continuous integration and deployment practices, including Git-based workflows, automated testing, and infrastructure-as-code approaches.
Cloud Platforms
Hands-on experience with major cloud providers (AWS, GCP, or Azure) for infrastructure provisioning, data warehousing services, and managed analytics platforms.
Technical Communication
Ability to articulate complex technical concepts clearly to both technical and non-technical stakeholders, design thorough documentation, and build consensus around architectural decisions.
AI-Augmented Development
Practical experience leveraging AI tools in development workflow for productivity gains, research, prototyping, and problem-solving beyond basic code generation.
Preferred
Semantic Layers & Data Modeling
Nice to haveFamiliarity with semantic layer concepts, business logic abstraction, and ontology-driven data architecture. Experience with tools that abstract data complexity for end-user querying.
Model Context Protocol (MCP)
Nice to haveEarly exposure to Model Context Protocol or similar frameworks enabling AI systems to interact with diverse data sources and tools through standardized interfaces.
Text-to-SQL & AI-Native Data Querying
Nice to haveExperience implementing or working with text-to-SQL systems, LLM-powered data exploration, or other AI-native approaches to data access and discovery.
Operational AI Systems
Nice to haveExperience building or maintaining AI systems within production data platforms, such as anomaly detection pipelines, intelligent diagnostics, or decision automation agents.
Data Quality & Governance Frameworks
Nice to haveDemonstrated expertise implementing enterprise data quality platforms, data catalogs, data lineage tracking, or comprehensive governance frameworks.
Financial Data or Fintech Experience
Nice to haveBackground in financial services, investment technology, banking systems, or fintech provides valuable context for Codat's commercial banking advisory mission.
Lakehouse Architectures
Nice to haveAdvanced experience with modern lakehouse architectures combining data lake flexibility with data warehouse structure, particularly Delta Lake or Iceberg implementations.
Open Source Contributions
Nice to haveActive contributions to open-source data engineering projects, demonstrating community engagement and commitment to advancing the field.
Tech stack
Languages
Frameworks
Databases
Tools
Other
Compensation
Pay and benefits.
Base·GBP 95,000 – 120,000
Full posting
Original listing.
About Codat
Codat is an advisory intelligence solution purpose-built for modern commercial banking. Through rich, specialized data, forward-looking insights, and integrated workflows, Codat empowers banking teams to deepen their relationships, grow their revenue, and simplify their day-to-day work.
Founded in 2017 and backed by JPMorgan, PayPal, Amex, Plaid, and Shopify, Codat has successfully powered over 350,000 connections to business customers’ financial systems — and is trusted by industry leaders to turn scattered information into actionable, strategic advantages in real time, every time.
The Role
We're looking for a Senior Data Engineer to join our Data and Insights team. You'll be hands-on every day, writing production code, building and maintaining data pipelines, and shipping features that turn raw data into intelligence our clients can act on. You'll work across the full project lifecycle, from understanding the problem through to delivery, and you'll care as much about code quality and operational reliability as you do about getting things shipped.
This is also a technical leadership role. As a senior member of the team, you'll set and lead the technical direction of our Insights platform. This is a visible position within engineering and across the wider business, so you'll explain your thinking clearly, share the reasoning behind it, and bring people with you. You'll do all of this while staying close to the code: it'll suit you if you want to keep building hands-on, rather than move into pure architecture or people management in the near term.
What You'll Do
Write production code every day, most likely in Python, building and maintaining the data pipelines that power our Insights products.
Own the full lifecycle of your projects, from understanding the data domain through to pragmatic design, shipping, and keeping things running reliably in production.
Set and lead the technical direction of the Insights platform, and communicate it openly across engineering and the wider business, so product and commercial colleagues understand the choices you are making and why.
Help raise engineering standards across the team and improve technical quality through strong engineering practice, including testing, observability, data quality checks, and clean, maintainable code.
Make AI your default way of working, and find opportunities to apply it across our products and pipelines where it delivers real value, from research and prototyping through to more operational uses such as agents that help diagnose and fix pipeline issues.
Help lay the foundations for our emerging MCP and semantic layer, so our data becomes something both people and AI systems can query and reason over.
What You'll Bring
Strong software engineering fundamentals: you write well-tested, production-ready Python and care about maintainability, observability, and operational excellence.
A track record of building data pipelines and production systems from the ground up, rather than mainly configuring managed services or wiring off-the-shelf tools together. You can describe complex logic you've written and the engineering problems you had to solve.
Solid experience with modern data engineering tools and patterns, with real depth in several of SQL, Spark, Databricks/Delta Lake, orchestration tools (Dagster, Airflow, Temporal), and dbt.
Comfort with modern deployment practices: CI/CD, containerisation (Docker), and cloud-based infrastructure. It's a bonus if you've shaped these for a team, not only worked within them.
A product mindset: you want to understand the business domain and use that understanding to shape what gets built, not only how. You're comfortable pushing back or proposing a different approach when your read of the data and the domain calls for it.
Strong communication skills: you can explain and build support for your ideas with peers, managers, and non-technical stakeholders, and you're comfortable holding a visible technical position and bringing people with you.
AI as a default part of how you work, with evidence of real efficiency gains and creative use beyond code generation, such as research, building domain knowledge, or prototyping.
Nice to have: exposure to the building blocks of AI-ready data, such as semantic layers, ontologies, or text-to-SQL, plus any experience applying AI operationally within data platforms or pipelines. This won't be your main focus, but it will help as our platform grows to support an MCP.
Redirects to Codat's application page.