looking for immediate joiner for a product based us mnc company for the
role - data engineer
location - balewadi, pune (hybrid)
required skills - python/pyspark coding , data factory, data bricks
min exp - 5 years
technical requirements
- must be hands-on/well-verses in python/pyspark coding and scripting
- strong expertise and hands-on experience with azure data factory and azure databricks
- experience with agile methodology and azure devops
- databricks lakehouse architecture experience strongly preferred
- programming/coding skills with familiarity with software and
system engineering design principles & standards (., tdd)
- foundational knowledge of data management architectures like
data warehouse, data lake, data hub and the supporting processes like data integration, governance, metadata management
- ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, data quality and observability, schemas, metadata, and job management
- understanding of structured, semi-structured, and unstructured data sources
- experience with data integration technologies such as etl/elt, data replications/cdc, message-oriented data movement, api design and access, stream data integration, and data virtualization
- basic understanding of machine learning algorithms and
approaches
- knowledge of/proficiency using one or more popular languages
and frameworks such as sql, python, scala, spark, command
line, etc.
- familiarity with open-source and commercial (., azure cloud
services) technology
- ability to automate development via cicd patterns and
processes
- experience leveraging version control and repository
management systems (git experience recommended)
- experience leveraging ide and source code editors (., visual
studio code)
- data visualization (., power bi, tableau, qliksense)