# Senior Data Engineer, Knowledge Platform, Foundational Data **Company:** [Airwallex](https://scaleengineer.com/companies/airwallex) Senior Data Engineer role at Airwallex focused on building scalable data foundations for the Knowledge Platform, supporting critical AI and business use cases across the organization's financial infrastructure. This position requires 5+ years of experience designing ETL pipelines, strong SQL and data warehousing expertise, and proficiency with cloud-native technologies like Databricks and Google Cloud Platform. The role is based in Singapore and involves owning end-to-end data solutions while collaborating with engineering teams to transform ambiguous data requirements into reliable, well-structured systems that power decision-making and AI capabilities at global financial scale. **Role:** Data Engineer **Seniority:** Senior **Locations:** SG - Singapore **Salary:** 150000–220000 SGD [Apply](https://jobs.ashbyhq.com/airwallex/39069d1d-9a56-4ef2-b326-8cc1f46a25f9) Canonical: https://scaleengineer.com/jobs/airwallex/senior-data-engineer-knowledge-platform-foundational-data --- ## Responsibilities - Design and Build Scalable Data Pipelines: Design and build scalable data pipelines and data models that power critical use cases across Knowledge Platform teams and business domains, supporting both batch and real-time workflows at global financial scale. - Cross-Functional Stakeholder Collaboration: Partner closely with engineering and cross-functional stakeholders to translate ambiguous data requirements into reliable, well-structured solutions that support decision-making, product development, and AI-driven capabilities. - End-to-End Project Ownership: Own projects end to end, from system architecture and pipeline design through performance optimization, monitoring, alerting, and ongoing improvements in data quality, reliability, and governance. - Technical Leadership and Standards: Raise the bar on technical design and execution as a senior member of the team, bringing strong engineering judgment, practical problem-solving, architectural thinking, and establishing high standards of ownership across data solutions. - Data Infrastructure and Knowledge Platform Development: Build the Knowledge Platform foundation end to end, from core technologies including Databricks, ClickHouse, Spark, and Kafka to real-time analytics, knowledge retrieval, indexing, model infrastructure, and AI interaction history systems. - Governance and Reliability Framework Development: Develop evaluation, annotation, feedback, governance, permission, and freshness frameworks required to operate AI reliably at global financial scale, ensuring compliance with financial regulations and data governance best practices. ## Requirements ### education - {"name":"Bachelor's Degree in Computer Science or Related Field","description":"Bachelor's degree or higher in Computer Science, Computer Engineering, Data Science, or a closely related technical discipline."} ### technical - {"name":"ETL Pipeline Design and Implementation","description":"At least five years of hands-on experience designing and implementing scalable ETL pipelines across batch and real-time workflows, with proven expertise using technologies such as DBT, Apache Spark, and Airflow in production environments."} - {"name":"Advanced SQL and Database Management","description":"Strong proficiency in SQL query optimization, database management systems, data warehousing architecture, and complex data modeling for business intelligence, machine learning, and operational use cases."} - {"name":"Data Schema and Modeling Expertise","description":"Strong understanding of data schema design patterns including star schemas, snowflake schemas, normalized designs, and denormalized designs, with ability to choose appropriate patterns based on use case requirements."} - {"name":"Google Cloud Platform and Modern Data Stack","description":"Hands-on familiarity with Google Cloud Platform services, Databricks for data processing, Apache Airflow for workflow orchestration, and modern data infrastructure tools commonly used in cloud-native architectures."} - {"name":"Data Governance and Compliance","description":"Knowledge of data governance practices, the six key governance pillars (data quality, metadata management, security, privacy, compliance, and lifecycle management), and regulatory requirements in the financial services industry."} - {"name":"AI and Machine Learning Data Infrastructure","description":"Understanding of AI fundamentals and practical experience applying data engineering principles to support machine learning pipelines, model training infrastructure, and AI system requirements."} ### experience - {"name":"Senior-Level Data Engineering Experience","description":"Minimum five years of professional experience in data engineering roles, with demonstrated expertise in designing scalable data solutions, leading data platform initiatives, and mentoring junior team members."} - {"name":"Distributed Systems and Data Architecture","description":"Experience working with data across distributed or multi-datacenter systems, including practical knowledge of data migration strategies, data duplication patterns, and consistency challenges in distributed environments."} - {"name":"Financial Services or FinTech Background","description":"Preferred experience working in financial services, payments infrastructure, or fintech environments, with understanding of financial data requirements, compliance considerations, and transaction processing systems."} - {"name":"Cross-Functional Team Collaboration","description":"Demonstrated experience working effectively with globally distributed teams, communicating complex technical concepts to non-technical stakeholders, and collaborating with product, analytics, and ML teams."} ## Skills ### required - {"name":"Apache Spark","description":"Production-level expertise in Apache Spark for large-scale distributed data processing, including optimization of RDD and DataFrame operations, catalyst optimization, and performance tuning."} - {"name":"DBT (Data Build Tool)","description":"Advanced proficiency with DBT for analytics engineering, including data transformation workflows, testing frameworks, documentation generation, and CI/CD integration for data pipelines."} - {"name":"SQL Query Optimization","description":"Expert-level SQL skills with ability to write complex queries, optimize performance, analyze execution plans, and design efficient database schemas for analytics and operational workloads."} - {"name":"Data Warehousing Architecture","description":"Deep understanding of data warehouse design, including dimensional modeling, fact and dimension tables, slowly changing dimensions, and optimization for analytical query patterns."} - {"name":"Apache Airflow","description":"Hands-on experience with Apache Airflow for orchestrating complex ETL workflows, DAG design patterns, monitoring and alerting, and error handling in production environments."} - {"name":"Google Cloud Platform (GCP)","description":"Practical experience with GCP services including BigQuery, Dataflow, Cloud Storage, and Cloud Composer, with understanding of cloud-native architecture patterns and cost optimization."} - {"name":"Databricks","description":"Proficiency with Databricks platform for data engineering and analytics, including Unity Catalog for governance, Delta Lake for ACID transactions, and lakehouse architecture patterns."} ### preferred - {"name":"Python","description":"Strong Python programming skills for data analysis, pipeline automation, and integration scripts, with familiarity of data science libraries like Pandas, NumPy, and scikit-learn."} - {"name":"ClickHouse","description":"Experience with ClickHouse for real-time analytics on large datasets, including column-oriented data storage optimization and high-throughput query patterns."} - {"name":"Apache Kafka","description":"Experience with Apache Kafka for building real-time data pipelines, understanding of streaming data patterns, event-driven architecture, and exactly-once processing semantics."} - {"name":"R","description":"Familiarity with R programming language for statistical analysis, data visualization, and exploratory data analysis, complementing data engineering workflows."} - {"name":"Data Management Certifications","description":"Professional certifications in data management, cloud platforms, or data engineering (such as GCP Data Engineer, Databricks Lakehouse Engineer, or similar credentials)."} - {"name":"System Design and Architecture","description":"Strong background in system design principles, architectural patterns for scalable systems, and experience designing high-performance, fault-tolerant data infrastructure."} ## Tech stack ### tools - {"name":"Google Cloud Platform","description":"Cloud infrastructure provider offering BigQuery, Dataflow, Cloud Storage, and managed services for building scalable data pipelines and applications."} - {"name":"Databricks Unity Catalog","description":"Unified governance solution for data and AI assets, providing centralized access control, metadata management, and lineage tracking across data platforms."} - {"name":"Delta Lake","description":"Open-source storage format providing ACID transactions, schema enforcement, and time travel capabilities for data lakes, enabling reliable data pipelines."} - {"name":"Kafka","description":"Distributed event streaming platform for building real-time data pipelines and applications, supporting high-throughput, low-latency data ingestion."} ### others - {"name":"Data Governance Frameworks","description":"Practices and tools for managing data quality, metadata, security, privacy, compliance, and lifecycle management across the organization."} - {"name":"Real-Time Analytics","description":"Technologies and techniques for enabling sub-second query latency and real-time insights on streaming and historical data."} - {"name":"Knowledge Retrieval and Indexing","description":"Systems for organizing and efficiently searching through data to support AI systems and natural language query interfaces."} - {"name":"AI Infrastructure","description":"Platforms and tools supporting machine learning model training, serving, monitoring, and evaluation at scale within the data ecosystem."} ### databases - {"name":"Google BigQuery","description":"Serverless, highly scalable data warehouse on Google Cloud Platform for analytics, real-time querying, and machine learning-ready datasets."} - {"name":"Databricks","description":"Unified analytics platform combining data engineering and AI capabilities, built on Apache Spark with Delta Lake for ACID transactions and data governance."} - {"name":"ClickHouse","description":"Open-source column-oriented database optimized for real-time analytics, enabling fast aggregations and sub-second query response times on large datasets."} ### languages - {"name":"SQL","description":"Core language for data manipulation, transformation, and analytics queries; essential for working with databases and data warehouses at Airwallex."} - {"name":"Python","description":"Primary scripting language for data pipeline automation, ETL logic, and integration with data processing frameworks and cloud services."} ### frameworks - {"name":"Apache Spark","description":"Distributed data processing framework for large-scale batch and streaming ETL pipelines, supporting DataFrame API, SQL interface, and MLlib for machine learning."} - {"name":"DBT","description":"Analytics engineering framework for transforming data in data warehouses using SQL, enabling version control, testing, and documentation of data transformations."} - {"name":"Apache Airflow","description":"Workflow orchestration platform for scheduling, monitoring, and managing complex ETL pipelines and data workflows at scale."} ## Benefits ### benefits - {"name":"Competitive Equity Compensation","description":"Participate in company growth through stock options and equity grants, aligning personal success with Airwallex's mission to build global financial infrastructure."} - {"name":"Professional Development","description":"Opportunities to grow technical expertise through mentorship from senior engineers, training in emerging data technologies, and exposure to complex, large-scale systems challenges."} - {"name":"Global Team Collaboration","description":"Work with talented engineers across 27 offices worldwide, gaining exposure to diverse perspectives and collaborating with teams building financial infrastructure at global scale."} - {"name":"Founders-Level Culture","description":"Join a high-impact organization where decisions are made from first principles, individuals move fast with good judgment, and builder energy is valued over hierarchy."} - {"name":"Impact and Ownership","description":"Own end-to-end projects with measurable business impact, supporting critical use cases for 675,000+ businesses and helping shape the future of financial operations globally."} ## Compensation - **max:** 220000 - **min:** 150000 - **currency:** SGD - **stockOptions:** true ## Interview process ### steps - {"name":"Initial Screening","description":"Conversation with recruiting team to discuss background, experience with data engineering roles, and alignment with the Knowledge Platform team's mission to build foundational data infrastructure."} - {"name":"Technical Phone Screen","description":"Discussion with a senior data engineer covering data pipeline design patterns, SQL optimization, system architecture decisions, and approach to problem-solving in distributed data systems."} - {"name":"Take-Home Data Engineering Project","description":"Practical assessment involving designing a data pipeline solution for a realistic business scenario, evaluating SQL skills, ETL architecture thinking, and code quality in a realistic timeframe."} - {"name":"Architectural Deep Dive","description":"Interview with the Knowledge Platform team discussing complex technical challenges, design trade-offs, experience with large-scale data systems, and approach to raising technical standards in a team."} - {"name":"Cross-Functional Collaboration Discussion","description":"Conversation exploring experience working with product, analytics, and machine learning teams, communication style with non-technical stakeholders, and examples of building impact through cross-functional partnerships."} - {"name":"Leadership and Culture Alignment","description":"Final interview with team leadership discussing career goals, experience with ownership and autonomy, alignment with Airwallex's builder culture and operating principles, and vision for data-driven decision-making."} ## Full description ## **About Airwallex** Airwallex is the AI-native financial operating system for a real-time, intelligent economy. More than 675,000 businesses, including McLaren Racing, Qantas, SHEIN, and TikTok, use us, directly or through our platform partners, to run their financial operations or build and monetize financial products of their own. We started in Melbourne in 2015 to build the infrastructure global commerce runs on. We're the regulated backbone behind global payments: not by accident, but by design. A decade plus, 85+ licenses, and a financial infrastructure spanning North America, Europe, the Middle East, and Asia-Pacific. We're co-headquartered in San Francisco and Singapore, with more than 2,300 people across 27 offices. We hire builders with founder-level energy, people who move fast with good judgment, dig in with real curiosity, and make calls from first principles rather than waiting to be told what to do. Read our[ operating principles](https://www.airwallex.com/en-us/operating-principles) to see it in full. ## How you'll make impact Knowledge Platform is building the data foundations that support critical business and AI use cases across Airwallex. You'll own reliable, scalable data solutions end to end while helping raise the bar for technical design and execution. * Design and build scalable data pipelines and data models that power critical use cases across Knowledge Platform teams and business domains. * Partner closely with engineering and cross-functional stakeholders to turn ambiguous data needs into reliable, well-structured solutions that support decision-making and product development. * Own projects end to end, from system and pipeline design through performance, monitoring, and ongoing improvements in data quality and reliability. * Raise the bar on technical design and execution as a senior member of the team, bringing strong engineering judgment, practical problem-solving, and a high standard of ownership. You'll work from our Singapore office five days a week. ## What we're looking for ### Essentials * Bachelor’s degree or higher in Computer Science, Computer Engineering or a related field. * At least five years of experience designing and implementing scalable ETL pipelines, across batch and real-time workflows, using technologies such as DBT and Spark * Strong proficiency in SQL, database management systems, data warehousing, and data modelling for business intelligence, machine learning, and operational needs. * Strong understanding of data schemas, including star, snowflake, normalised, and denormalised designs. * Familiarity with Google Cloud Platform and modern data infrastructure tools, including Databricks and Airflow. * Knowledge of data governance practices, the six key governance pillars, and regulatory requirements in the financial industry. * Strong problem-solving, attention to detail, communication, and collaboration skills, including experience working with globally distributed teams. * Understanding of AI and an interest in applying data engineering and AI in practical and creative ways. ### Preferred * Experience working with data across distributed or multi-datacenter systems, including data migration, duplication, and consistency challenges. * Experience working in financial services, payments, or fintech. * Experience with scripting languages such as Python or R for data analysis and automation. * Certification in data management or related technologies. ## Learn more about your team The Knowledge Platform sits at the foundation of Airwallex’s Data and AI strategy. We are evolving traditional data infrastructure into knowledge infrastructure—turning structured and unstructured information into trusted, reusable context that helps people and AI systems make better decisions and take action. You’ll build the platform end to end, from core technologies such as Databricks, Clickhouse, Spark, and Kafka to real-time analytics, knowledge retrieval and indexing, model infrastructure, agent capabilities, and AI interaction history. You’ll also develop the evaluation, annotation, feedback, governance, permission, and freshness frameworks required to operate AI reliably at global financial scale. Our mission is making this ecosystem truly agent-ready. That means enabling teams to explore data through natural language, investigate business drivers in real time, and give AI systems secure access to the context and capabilities they need to support decisions and execute workflows. If you want to build the foundation that turns Airwallex’s data into knowledge, knowledge into intelligence, and intelligence into measurable business impact, this is where that work happens. ## **Applicant Safety Policy: Fraud and Third-Party Recruiters** _To protect you from recruitment scams, please be aware that Airwallex will not ask for bank details, sensitive ID numbers (i.e. passport), or any form of payment during the application or interview process. All official communication will come from an @_[_airwallex.com_](http://airwallex.com) _email address. Please apply only through_ [_careers.airwallex.com_](http://careers.airwallex.com) _or our official LinkedIn page._ _Airwallex does not accept unsolicited resumes from search firms/recruiters. Airwallex will not pay any fees to search firms/recruiters if a candidate is submitted by a search firm/recruiter unless an agreement has been entered into with respect to specific open position(s). Search firms/recruiters submitting resumes to Airwallex on an unsolicited basis shall be deemed to accept this condition, regardless of any other provision to the contrary._ ## **Equal opportunity** Airwallex is proud to be an equal opportunity employer. We value diversity and anyone seeking employment at Airwallex is considered based on merit, qualifications, competence and talent. We don’t regard color, religion, race, national origin, sexual orientation, ancestry, citizenship, sex, marital or family status, disability, gender, or any other legally protected status when making our hiring decisions. If you have a disability or special need that requires accommodation, please let us know.