Hadoop Job Vacancies in Mumbai

Hadoop Lead RoleExperience 9+ years- Strong architectural experience with Hortonworks, Cloudera Hadoop distributions on Appliance based and on-premise clusters.- Expertise in providing technical solutions for data lakes design and data ingestion in Hadoop.- Expertise in data modelling, data governance and architecture of large size databases.- Expertise in understanding complex data models, large scale data migrations and application development.- Expertise in designing data pipelines solutions for structured, semi-structured and unstructured data in Hadoop.- Expertise in Developing solutions for batch and real time processing data processing in Hadoop.- Hands on experience in query writing using HiveQL, Impala QL and HBase commands.- Able to provide technical guidelines assistance to development team if they face any problems related to environment- Able to coordinate with cloudera team for trouble shooting, OS / network related problems with respective teams.- Proficiency in Cloudera Manager architecture, cloudera cluster environment and cloudera manager.- Proficiency in data ingestion tools like Sqoop, Flume, Kafka, Nifi(HDF), UNIX shells scripting and python- Proficiency in building data warehouse on top of Hadoop in Hive. Defining data models and data mapping from the source to enterprise model.- Proficiency in creating solutions in Spark (Scala/Pyspark) for batch and real time processing.- Proficiency in developing solutions for NoSQL databases like HBase, Cassandra and MongoDB.- Knowledge of Data Security / Governance and Data Lineage handling on Hadoop clusters.- Strong experience on SQL Server and Oracle, MySQL.- Expertise in understanding Java , REST API concepts and troubleshooting java based services.

Technologies / Skills: Advanced SQL, Python and associated libraries like Pandas, Numpy etc., Pyspark , Shell scripting, Data- Modelling, Big data, Hadoop, Hive, ETL pipelines and IaC tools like Terraform etc.Responsibilities: Efficient communication skills to coordinate with users, technical teams and DataSolution architects. Document technical design documents for given requirements or JIRA stories. Communicate results and business impacts of insight initiatives to key stakeholders to collaboratively solve business problems. Working closely with the overall Enterprise Data & Analytics Architect and Engineering practice leads to ensure adherence with the best practices and design principles. Assures quality, security and compliance requirements are met for supported area. Develop fault-tolerance data pipelines running on cluster Ability to come up with scalable and modular solutionsRequired Qualification: 1-8 yrs of hands-on experience developing data pipelines for Data Ingestion or transformation using Python (PySpark) /Spark SQL in AWS cloud Experience in development of data pipelines and processing of data at scale using technologies like EMR, Lambda, Glue, Athena, Redshift, Step Functions. Advanced experience in writing and optimizing efficient SQL queries with Python and Hive handling Large Data Sets in Big-Data Environments Experience in debugging, tunning and optimizing PySpark data pipelines Should have implemented concepts and have good knowledge of Pyspark data frames, joins, partitioning, parallelism etc. Understanding of Spark UI, Event Timelines, DAG, Spark config parameters, in order to tune the long running data pipelines. Experience working in Agile implementations Experience with Git and CI/CD pipelines to deploy cloud applications Good knowledge of designing Hive tables with partitioning for performanceThanks and RegardsHR TEAM

Profile - Data Engineer ExecutiveQualification - Graduate With Good Communication SkillExperience - Minimum 1 Year RequiredCandidate Should Have Knowledge of AWS,Spark, Py- Spark, Python, HarkSalary - 24 LPA TO 42 LPA Gender - Male & Female Can ApplyLocation - Pen IndiaDuties & Responsibilities-----Analyze and organize raw data.Build data systems and pipelines.Evaluate business needs and objectives.Interpret trends and patterns.Conduct complex data analysis and report on results.Prepare data for prescriptive and predictive modeling.Build algorithms and prototypes.Only Serious Candidate Apply

Brief about the Company:AdZapier Corporation is a global technology and enablement services company with a vision to transform data into value for everyone. Through a simple open approach, in connecting systems and data, we provide the data foundation for the worlds best marketers. By making it safe and easy to activate, validate, enhance, and unify data. We provide marketers with the ability to deliver relevant messages at scale and tie those messages back to actual results. Our products and services enable individual-based marketing, allowing our clients to generate a higher ROI and drive better omni-channel customer experiences.Position Description:Join our Information Technology team where you will work on new technologies and find ways to meet our customers needs and make it easy for them to do business with us.You will use functional expertise to act as an advisor to management and make recommendations on more complex projects. You will use professional concepts and company policies & procedures to solve a wide range of difficult problems creatively and practically.ResponsibilitiesYou will be responsible for operations and administration of Cloudera Hadoop platform.You will work independently on day to day monitoring and operations of Data Analytics platform. You will be required to develop automation using scripting languages. After initial training, you will be able to handle critical operation tasks as well as on demand requests.Minimum Requirements: 5+ years of experience in Software Development including Big Data Analytics area Experience in Hadoop Big Data Platform Operations and Administration High Proficiency working with Hadoop platform including Hadoop, Hive, Spark/Scala, Java, Kafka, Flume etc. Experience with any scripting language such as BASH, Scala or Python Good understanding of file formats including JSON, Parquet, Avro, and otherWork Hours - 2.30 pm noon to 11.30 pm (Mon-Fri) (US Shift)