Data Engineer Female Graduate Jobs in Pune

As a Data Engineer, your responsibilities will include collecting and analyzing data to help inform business decisions. You will be responsible for data management, ensuring that data is accurate and up-to-date. This will involve using tools such as Informatica, PLSQL, SQL, and Oracle SQL to manipulate and query large datasets.Your skills should include a strong understanding of data analysis techniques, such as data mining and statistical analysis.

3+ years of AWS data engineering: Glue, Step Functions, Lambda, S3, DynamoDB, EC2Strong Python (boto3) scripting for automationTerraform or CloudFormation expertiseHands-on experience integrating RAG workflows or deploying LLM applicationsSolid SQL and NoSQL data-modeling skillsExcellent written and verbal communication in client-facing contexts

Sr. Big Data Engineer Location: PuneExperience: 10+ years Mode: HybridRole Overview:We are seeking a talented Sr. Big Data Engineer to design, develop, and support a highly scalable, distributed SaaS-based Security Risk Prioritization product. You will lead the design and evolution of our data platform and pipelines, providing technical leadership to a team of engineers and architects.Key Responsibilities: Provide technical leadership on data platform design, roadmaps, and architecture. Design and implement scalable architecture for Big Data and Microservices environments. Drive technology explorations, leveraging knowledge of internal and industry prior art. Ensure quality architecture and design of systems, focusing on performance, scalability, and security. Mentor and provide technical guidance to other engineers.Required Skills & Technologies: Mandatory: Kafka, Scala, Spark. Big Data & Data Streaming: Spark, Kafka, Hadoop, Presto, Airflow, Data lakes, lambda architecture, kappa, and kappa ++ architectures with flink data streaming. Databases & Caching: RDBMS, NoSQL, Oracle, Cassandra, Redis. Search Solutions: Solr, Elastic. ML & Automation: Experience with ML models engineering and related deployment, scripting, and automation. Architecture: In-depth experience with messaging queues and caching components. Other Skills: Strong troubleshooting and performance benchmarking skills for Big Data technologies.Qualifications: Bachelors degree in Computer Science or equivalent. 8+ years of total experience, with 6+ years relevant. 2+ years in designing Big Data solutions with Spark. 3+ years with Kafka and performance testing for large infrastructure.

Responsibilities: Design, develop, and deploy data solutions on Azure, leveraging SQL Azure,Azure Data Factory, and Databricks. Build and maintain scalable data pipelines to ingest, transform, and load datafrom various sources into Azure data repositories. Implement data security and compliance measures to safeguard sensitiveinformation. Collaborate with data scientists and analysts to support their data requirementsand enable advanced analytics and machine learning initiatives. Optimize and tune data workflows for performance and efficiency. Troubleshoot data-related issues and provide timely resolution. Stay updated with the latest Azure data services and technologies andrecommend best practices for data engineering.Qualifications: Bachelors degree in computer science, Information Technology, or related field. Proven experience as a data engineer, preferably in a cloud environment. Strong proficiency in SQL Azure for database design, querying, and optimization. Hands-on experience with Azure Data Factory for ETL/ELT workflows. Familiarity with Azure Databricks for big data processing and analytics. Experience with other Azure data services such as Azure Synapse Analytics,Azure Cosmos DB, and Azure Data Lake Storage is a plus. Solid understanding of data warehousing concepts, data modeling, anddimensional modeling. Excellent problem-solving and communication skills.

Role: Bigdata Developer - Scala SparkExp: 5+ YrsMode of Work: WFO All 5 daysLocation: Chennai/Bangalore/PuneInterview: Any one Level F2FJob Description: Total IT / development experience of 3+ years Experience in Spark (Scala-Spark ) developing Big Data applications on Hadoop, Hive and/or Kafka, HBase, MongoDB Deep knowledge of Scala-Spark libraries to develop and debug complex data engineering challenges Experience in developing sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies Exposure to deploying on Cloud platforms At least 2 years of development experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Spark-Scala At least 2 years of development experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS At least 2 years of developing applications in Agile with Monitoring, Build Tools, Version Control, Unit Test, Unix Shell Scripting, TDD, CI/CD, Change Management to support DevOps

Description:We are looking for a competent and motivated individual to fill this role. This is an operational MSSQL DataEngineering role. Some of the primary responsibilities of this role would include development, setup a new clientfrom scratch, owning, tracking, and resolving database related incidents and requests, fulfilling requests, andresolving incidents.Responsibilities: Read business requirements - and be able to ask questions to understand the requirements. Understand information model and database schemas and plan data/table layouts and schemas. Able to write and understand SQL code (SQL-Server) for high-speed data loading pipeline. Should have good hands-on Database objects like Tables, View, Stored Procedure, Functions etc. Understand SQL Server Performance issues, find root cause and solve the problem to ensure appropriateperformance. Maintain and improve existing processes. Able to mentor junior/entry level colleagues. Should be a team player and self-driven.Skills and Qualifications: Bachelors or masters in computer science, information technology or related field, or equivalent experience Demonstrated ability to solve problems. Ability to work in a team environment. Strong verbal and written communication Positive can-do attitude Preferred Skills 4-7 years of relevant work experience

Level of skills and experience:5 years of hands-on experience in using Python, Spark,Sql.Experienced in AWS Cloud usage and management.Experience with Databricks (Lakehouse, ML, Unity Catalog, MLflow).Experience using various ML models and frameworks such as XGBoost, Lightgbm, Torch.Experience with orchestrators such as Airflow and Kubeflow.Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).Fundamental understanding of Parquet, Delta Lake and other data file formats.Proficiency on an IaC tool such as Terraform, CDK or Cloud Formation.Strong written and verbal English communication skill and proficient in communication with non-technical stakeholderst

Work Mode: Hybrid6-9 years of overall IT experience, preferably in cloud environments.Minimum of 5 years of hands-on experience with AWS cloud development projects.Design and develop AWS data architectures and solutions.Build robust data pipelines and ETL processes using big data technologies.Utilize AWS data services such as Glue, Lambda, Redshift, and Athena effectively.Implement infrastructure as code (IaC) using Terraform.Proficiency in SQL, Python, and other relevant programming/scripting languages.Experience with orchestration tools like Apache Airflow or AWS Step Functions.Strong understanding of data warehousing concepts, data lakes, and data governance frameworks.Expertise in data modeling for both relational and non-relational databases.Excellent communication skills are essential for this role.

Work mode : Hybrid Overall 4-9 yrs of IT experience preferably in cloudMin 3 years in Azure Databricks on development projectsShould be 100% hands on in Pyspark codingShould have strong SQL expertise in writing advanced/complex SQL queriesDWH experience is a must for this roleExperience in programming using Python is an advantageExperience in data ingestion, preparation, integration, and operationalization techniques in optimally addressing the data requirementsShould be able to understand system architecture which involves Data Lakes, Data Warehouses and Data MartsExperience to own end-to-end development, including coding, testing, debugging and deploymentExcellent communication is required for this role

Looking For Immediate Joiner for a Product based US MNC company for theRole - Data EngineerLocation - Balewadi, Pune (Hybrid)Required Skills - Python/Pyspark coding , Data Factory, Data BricksMin Exp - 5 YearsTechnical Requirements- Must be hands-on/well-verses in Python/PySpark coding and scripting- Strong expertise and hands-on experience with Azure Data Factory and Azure Databricks- Experience with Agile Methodology and Azure Devops- Databricks Lakehouse architecture experience strongly preferred- Programming/Coding Skills with familiarity with software andsystem engineering design principles & standards (e.g., TDD)- Foundational knowledge of data management architectures likeData Warehouse, Data Lake, Data Hub and the supporting processes like Data Integration, Governance, Metadata Management- Ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, data quality and observability, schemas, metadata, and job management- Understanding of structured, semi-structured, and unstructured data sources- Experience with data integration technologies such as ETL/ELT, data replications/CDC, message-oriented data movement, API design and access, stream data integration, and data virtualization- Basic understanding of machine learning algorithms andapproaches- Knowledge of/proficiency using one or more popular languagesand frameworks such as SQL, Python, Scala, Spark, CommandLine, etc.- Familiarity with Open-Source and Commercial (e.g., Azure CloudServices) technology- Ability to automate development via CICD patterns andprocesses- Experience leveraging version control and repositorymanagement systems (Git experience recommended)- Experience leveraging IDE and source code editors (e.g., VisualStudio Code)- Data visualization (e.g., Power BI, Tableau, QlikSense)