Big Data Engineer Graduate Experience Jobs in Pune

Requirements:8+ years of experience in Core Java and Spring Framework (Mandatory)Minimum 2 years of experience in Google Cloud Platform (GCP) (Mandatory)Hands-on experience with Apache Beam / Dataflow for building ETL/data pipelines (Mandatory)Strong expertise in big data processing on distributed systemsProficiency with RDBMS, NoSQL, and Cloud-native databasesExperience in handling multiple data formats (Flat file, JSON, Avro, XML, etc.) with schema/contract definitionsExperience in Microservices architecture and API integration patternsStrong understanding of data structures and data model design

Sr. Big Data Engineer Location: PuneExperience: 10+ years Mode: HybridRole Overview:We are seeking a talented Sr. Big Data Engineer to design, develop, and support a highly scalable, distributed SaaS-based Security Risk Prioritization product. You will lead the design and evolution of our data platform and pipelines, providing technical leadership to a team of engineers and architects.Key Responsibilities: Provide technical leadership on data platform design, roadmaps, and architecture. Design and implement scalable architecture for Big Data and Microservices environments. Drive technology explorations, leveraging knowledge of internal and industry prior art. Ensure quality architecture and design of systems, focusing on performance, scalability, and security. Mentor and provide technical guidance to other engineers.Required Skills & Technologies: Mandatory: Kafka, Scala, Spark. Big Data & Data Streaming: Spark, Kafka, Hadoop, Presto, Airflow, Data lakes, lambda architecture, kappa, and kappa ++ architectures with flink data streaming. Databases & Caching: RDBMS, NoSQL, Oracle, Cassandra, Redis. Search Solutions: Solr, Elastic. ML & Automation: Experience with ML models engineering and related deployment, scripting, and automation. Architecture: In-depth experience with messaging queues and caching components. Other Skills: Strong troubleshooting and performance benchmarking skills for Big Data technologies.Qualifications: Bachelors degree in Computer Science or equivalent. 8+ years of total experience, with 6+ years relevant. 2+ years in designing Big Data solutions with Spark. 3+ years with Kafka and performance testing for large infrastructure.

Role: Bigdata Developer - Scala SparkExp: 5+ YrsMode of Work: WFO All 5 daysLocation: Chennai/Bangalore/PuneInterview: Any one Level F2FJob Description: Total IT / development experience of 3+ years Experience in Spark (Scala-Spark ) developing Big Data applications on Hadoop, Hive and/or Kafka, HBase, MongoDB Deep knowledge of Scala-Spark libraries to develop and debug complex data engineering challenges Experience in developing sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies Exposure to deploying on Cloud platforms At least 2 years of development experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Spark-Scala At least 2 years of development experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS At least 2 years of developing applications in Agile with Monitoring, Build Tools, Version Control, Unit Test, Unix Shell Scripting, TDD, CI/CD, Change Management to support DevOps

Job DescriptionThe Big Data/Analytics Solution Architect is responsible for understanding emerging and evolving end user usage models and requirements in Big Data and Analytics, documenting those usage models and business, technical and user requirements and designing a solutions architecture to meet those requirements and specifying an implementation HW and SW solution stack. Solution architects document the solution architectures and solution requirements and, when needed, define end user proofs of concept to test the architectures, usage models and corresponding Intel technologies in testbed or real end user environments. They also work with end users and ecosystem partners to deploy those solutions in early adopter production environments. A strong candidate will have:Experience in Requirements Engineering, Solution Architecture, Design, Development and DeploymentA broad set of technical skills and knowledge across hardware, software, systems and solutions development and a across more than one technical domain.Demonstrated experience in real world IT or other solutions environments including creating (on your own or with a team) a product or IT solution in the area of Big Data/AnalyticsStrong communication skills including representing your company in industry standards organizations or industry technical forums or events in Cloud SecurityStrong technical team leadership, mentorship and collaboration.Ability to develop technical relationships with end-users, ISVs, OEMs and Intel platform architects and Proof-Of-Concept engineers.

Brief about the Company:AdZapier Corporation is a global technology and enablement services company with a vision to transform data into value for everyone. Through a simple open approach, in connecting systems and data, we provide the data foundation for the worlds best marketers. By making it safe and easy to activate, validate, enhance, and unify data. We provide marketers with the ability to deliver relevant messages at scale and tie those messages back to actual results. Our products and services enable individual-based marketing, allowing our clients to generate a higher ROI and drive better omni-channel customer experiences.Position Description:Join our Information Technology team where you will work on new technologies and find ways to meet our customers needs and make it easy for them to do business with us.You will use functional expertise to act as an advisor to management and make recommendations on more complex projects. You will use professional concepts and company policies & procedures to solve a wide range of difficult problems creatively and practically.ResponsibilitiesYou will be responsible for operations and administration of Cloudera Hadoop platform.You will work independently on day to day monitoring and operations of Data Analytics platform. You will be required to develop automation using scripting languages. After initial training, you will be able to handle critical operation tasks as well as on demand requests.Minimum Requirements: 5+ years of experience in Software Development including Big Data Analytics area Experience in Hadoop Big Data Platform Operations and Administration High Proficiency working with Hadoop platform including Hadoop, Hive, Spark/Scala, Java, Kafka, Flume etc. Experience with any scripting language such as BASH, Scala or Python Good understanding of file formats including JSON, Parquet, Avro, and otherWork Hours - 2.30 pm noon to 11.30 pm (Mon-Fri) (US Shift)

**Preference will be given to the candidates who can join on or before 1st of October, 2022**You will:Write excellent production code and tests and help others improve in code-reviewsAnalyze high-level requirements to design, document, estimate, and build systemsCoordinate across teams to identify, resolve, mitigate and prevent technical issuesCoach and mentor engineers within the team to develop their skills and abilitiesContinuously improve the team's practices in code-quality, reliability, performance, testing, automation, logging, monitoring, alerting, and build processesYou have:For (Full stack):2 - 10 Years of experienceStrong with DS & AlgorithmsHands on Experience in the Programming languages: JavaScript (React or Angular), Python, SQL.Experience with AWS.For (Backend):2 - 10 years of experienceHands on product development experience using Java/ C++/PythonExperience with AWS,SQL,GITStrong with Data structures and AlgorithmsAdditional nice to have skills/certifications:For Java skill set:Mockito, Grizzly, Netty, VertX, Jersey / JAX-RS, Swagger / Open API, Nginx, Protocol Buffers, Thrift, Aerospike, Redis, Kinesis, Sed, Awk, PerlFor Python skill set: Data Engineering experience, Athena, Lambda, EMR, Spark, Glue, Step Functions, Hadoop, Kinesis, Orc, Parquet, Perl, Awk, RedshiftFor (Data Engineering):2 - 10 years of experienceExperience with object-oriented/object function scripting languages: Python.Experience with AWS cloud services: EC2, RDS, Redshift,S3,Athena, GlueMust be proficient in GIT, Jenkins, CICD (Continuous Integration Continuous Deployment)Experience in big data technologies like Hadoop, Map Reduce, Spark, etcExperience with Amazon Web Services and DockersFor (Geo Team):4 - 10 years of experienceExperience with Big Data technologies like Hadoop, Spark, Map Reduce, Kafka, etcExperience using object-oriented languages (Java, Python)Experience in working with different AWS technologies.Experience in software