Role Overview
This position requires a detail-oriented data engineer who can independently architect and implement data pipelines, while also serving as a trusted technical partner in client engagements and stakeholder meetings. Youll work hands-on with PySpark, Airflow, Python, and SQL, driving end-to-end data migration and platform modernization efforts across Azure and AWS.
In addition to technical execution, youll contribute to sprint planning, backlog prioritization, and continuous integration/deployment of data infrastructure. This is a senior-level individual contributor role with direct visibility across engineering, product, and client delivery functions.
Key Responsibilities
Lead design and development of enterprise-grade data pipelines and cloud data migration architectures.
Build scalable, maintainable ETL/ELT pipelines using Apache Airflow, PySpark, and modern data services.
Write efficient, modular, and well-tested Python code, grounded in clean architecture and performance principles.
Develop and optimize complex SQL queries across diverse relational and analytical databases.
Contribute to and uphold standards for data modeling, data governance, and pipeline performance.
Own the implementation of CI/CD pipelines to enable reliable deployment of data workflows and infrastructure (., GitHub Actions, Azure DevOps, Jenkins).
Embed unit testing, integration testing, and monitoring in all stages of the data pipeline lifecycle.
Participate actively in Agile ceremonies: sprint planning, daily stand-ups, retrospectives, and backlog grooming.
Collaborate directly with clients, stakeholders, and cross-functional teams to translate business needs into scalable technical solutions.
Act as a technical authority within the teamleading architectural decisions and contributing to internal best practices and documentation.