Data Engineer Female Graduate Jobs in Pune

As a Data Engineer, your responsibilities will include collecting and analyzing data to help inform business decisions. You will be responsible for data management, ensuring that data is accurate and up-to-date. This will involve using tools such as Informatica, PLSQL, SQL, and Oracle SQL to manipulate and query large datasets.Your skills should include a strong understanding of data analysis techniques, such as data mining and statistical analysis.

3+ years of AWS data engineering: Glue, Step Functions, Lambda, S3, DynamoDB, EC2Strong Python (boto3) scripting for automationTerraform or CloudFormation expertiseHands-on experience integrating RAG workflows or deploying LLM applicationsSolid SQL and NoSQL data-modeling skillsExcellent written and verbal communication in client-facing contexts

Sr. Big Data Engineer Location: PuneExperience: 10+ years Mode: HybridRole Overview:We are seeking a talented Sr. Big Data Engineer to design, develop, and support a highly scalable, distributed SaaS-based Security Risk Prioritization product. You will lead the design and evolution of our data platform and pipelines, providing technical leadership to a team of engineers and architects.Key Responsibilities: Provide technical leadership on data platform design, roadmaps, and architecture. Design and implement scalable architecture for Big Data and Microservices environments. Drive technology explorations, leveraging knowledge of internal and industry prior art. Ensure quality architecture and design of systems, focusing on performance, scalability, and security. Mentor and provide technical guidance to other engineers.Required Skills & Technologies: Mandatory: Kafka, Scala, Spark. Big Data & Data Streaming: Spark, Kafka, Hadoop, Presto, Airflow, Data lakes, lambda architecture, kappa, and kappa ++ architectures with flink data streaming. Databases & Caching: RDBMS, NoSQL, Oracle, Cassandra, Redis. Search Solutions: Solr, Elastic. ML & Automation: Experience with ML models engineering and related deployment, scripting, and automation. Architecture: In-depth experience with messaging queues and caching components. Other Skills: Strong troubleshooting and performance benchmarking skills for Big Data technologies.Qualifications: Bachelors degree in Computer Science or equivalent. 8+ years of total experience, with 6+ years relevant. 2+ years in designing Big Data solutions with Spark. 3+ years with Kafka and performance testing for large infrastructure.

Responsibilities: Design, develop, and deploy data solutions on Azure, leveraging SQL Azure,Azure Data Factory, and Databricks. Build and maintain scalable data pipelines to ingest, transform, and load datafrom various sources into Azure data repositories. Implement data security and compliance measures to safeguard sensitiveinformation. Collaborate with data scientists and analysts to support their data requirementsand enable advanced analytics and machine learning initiatives. Optimize and tune data workflows for performance and efficiency. Troubleshoot data-related issues and provide timely resolution. Stay updated with the latest Azure data services and technologies andrecommend best practices for data engineering.Qualifications: Bachelors degree in computer science, Information Technology, or related field. Proven experience as a data engineer, preferably in a cloud environment. Strong proficiency in SQL Azure for database design, querying, and optimization. Hands-on experience with Azure Data Factory for ETL/ELT workflows. Familiarity with Azure Databricks for big data processing and analytics. Experience with other Azure data services such as Azure Synapse Analytics,Azure Cosmos DB, and Azure Data Lake Storage is a plus. Solid understanding of data warehousing concepts, data modeling, anddimensional modeling. Excellent problem-solving and communication skills.

Role: Bigdata Developer - Scala SparkExp: 5+ YrsMode of Work: WFO All 5 daysLocation: Chennai/Bangalore/PuneInterview: Any one Level F2FJob Description: Total IT / development experience of 3+ years Experience in Spark (Scala-Spark ) developing Big Data applications on Hadoop, Hive and/or Kafka, HBase, MongoDB Deep knowledge of Scala-Spark libraries to develop and debug complex data engineering challenges Experience in developing sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies Exposure to deploying on Cloud platforms At least 2 years of development experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Spark-Scala At least 2 years of development experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS At least 2 years of developing applications in Agile with Monitoring, Build Tools, Version Control, Unit Test, Unix Shell Scripting, TDD, CI/CD, Change Management to support DevOps

Level of skills and experience:5 years of hands-on experience in using Python, Spark,Sql.Experienced in AWS Cloud usage and management.Experience with Databricks (Lakehouse, ML, Unity Catalog, MLflow).Experience using various ML models and frameworks such as XGBoost, Lightgbm, Torch.Experience with orchestrators such as Airflow and Kubeflow.Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).Fundamental understanding of Parquet, Delta Lake and other data file formats.Proficiency on an IaC tool such as Terraform, CDK or Cloud Formation.Strong written and verbal English communication skill and proficient in communication with non-technical stakeholderst

Description:We are looking for a competent and motivated individual to fill this role. This is an operational MSSQL DataEngineering role. Some of the primary responsibilities of this role would include development, setup a new clientfrom scratch, owning, tracking, and resolving database related incidents and requests, fulfilling requests, andresolving incidents.Responsibilities: Read business requirements - and be able to ask questions to understand the requirements. Understand information model and database schemas and plan data/table layouts and schemas. Able to write and understand SQL code (SQL-Server) for high-speed data loading pipeline. Should have good hands-on Database objects like Tables, View, Stored Procedure, Functions etc. Understand SQL Server Performance issues, find root cause and solve the problem to ensure appropriateperformance. Maintain and improve existing processes. Able to mentor junior/entry level colleagues. Should be a team player and self-driven.Skills and Qualifications: Bachelors or masters in computer science, information technology or related field, or equivalent experience Demonstrated ability to solve problems. Ability to work in a team environment. Strong verbal and written communication Positive can-do attitude Preferred Skills 4-7 years of relevant work experience

Work Mode: Hybrid6-9 years of overall IT experience, preferably in cloud environments.Minimum of 5 years of hands-on experience with AWS cloud development projects.Design and develop AWS data architectures and solutions.Build robust data pipelines and ETL processes using big data technologies.Utilize AWS data services such as Glue, Lambda, Redshift, and Athena effectively.Implement infrastructure as code (IaC) using Terraform.Proficiency in SQL, Python, and other relevant programming/scripting languages.Experience with orchestration tools like Apache Airflow or AWS Step Functions.Strong understanding of data warehousing concepts, data lakes, and data governance frameworks.Expertise in data modeling for both relational and non-relational databases.Excellent communication skills are essential for this role.

Work mode : Hybrid Overall 4-9 yrs of IT experience preferably in cloudMin 3 years in Azure Databricks on development projectsShould be 100% hands on in Pyspark codingShould have strong SQL expertise in writing advanced/complex SQL queriesDWH experience is a must for this roleExperience in programming using Python is an advantageExperience in data ingestion, preparation, integration, and operationalization techniques in optimally addressing the data requirementsShould be able to understand system architecture which involves Data Lakes, Data Warehouses and Data MartsExperience to own end-to-end development, including coding, testing, debugging and deploymentExcellent communication is required for this role

Strong experience with Microsoft Azure data services, including Azure Data Factory, Azure Synapse Analytics, Azure Databricks, and Azure SQL Database.Proficiency in ETL processes, data pipeline design, and automation within the Azure ecosystem.Experience with big data technologies such as Apache Spark, Hadoop, Azure HDInsight, and Databricks.Solid understanding of data modeling, data warehousing concepts, and relational and non-relational databases.Strong knowledge of SQL and experience with scripting languages like Python, R, or Scala.Experience with Azure Storage services, such as Blob Storage, Data Lake Storage, and Azure Cosmos DB.Familiarity with Data Governance, Data Lineage, and Data Security practices in the cloud.Hands-on experience with Azure DevOps for CI/CD pipeline management, version control, and deployment automation.Familiarity with Azure Active Directory (AAD) and role-based access control (RBAC) for managing permissions and security.Experience with RESTful APIs and integrating with third-party systems for data ingestion and export.Strong problem-solving and analytical skills, with the ability to work with large datasets and derive actionable insights.Ability to collaborate effectively with stakeholders and communicate complex data engineering concepts in simple terms.Preferred Qualifications:Azure certifications such as Microsoft Certified: Azure Data Engineer Associate or Azure Solutions Architect Expert.Experience with real-time data processing using Azure Stream Analytics or Apache Kafka.Knowledge of machine learning and AI concepts for data-driven decision-making.Familiarity with Power BI or other reporting tools for data visualization.Experience with Agile methodologies in a data engineering environment.