Data Engineer Graduate Experience Jobs in Gurgaon

Level of skills and experience:5 years of hands-on experience in using Python, Spark,Sql.Experienced in AWS Cloud usage and management.Experience with Databricks (Lakehouse, ML, Unity Catalog, MLflow).Experience using various ML models and frameworks such as XGBoost, Lightgbm, Torch.Experience with orchestrators such as Airflow and Kubeflow.Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes).Fundamental understanding of Parquet, Delta Lake and other data file formats.Proficiency on an IaC tool such as Terraform, CDK or Cloud Formation.Strong written and verbal English communication skill and proficient in communication with non-technical stakeholderst

Position Overview:We are currently seeking a seasoned Data Engineer to join our dynamic team. The ideal candidate will have a minimum of 4 years of experience in data engineering roles, with a proven track record of delivering high-quality solutions in a fast-paced environment.Responsibilities:Design, build, and maintain scalable data pipelines and infrastructure.Collaborate with cross-functional teams to gather and analyze requirements.Implement best practices for data warehousing, ETL processes, and data integration.Optimize data workflows for performance, reliability, and efficiency.Stay current with emerging technologies and industry trends in data engineering.Qualifications:Bachelor's degree in Computer Science, Engineering, or related field.Minimum of 4 years of professional experience in data engineering.Proficiency in programming languages such as Python, Java, or Scala.Experience with cloud platforms (e.g., AWS, Azure, GCP) and related services.Strong understanding of data modeling, database systems, and SQL.Excellent problem-solving skills and attention to detail.Effective communication and collaboration abilities.

Desired candidate profile: Design, develop, and maintain web scraping scripts using Python. Use web scraping libraries like Beautiful Soup, Scrapy, Selenium and other scraping tools to extract data from websites. Write reusable, testable, and efficient code to extract structured and unstructured data. Develop and maintain software documentation for web scraping scripts. Collaborate with other software developers, data scientists, and other stakeholders to plan, design, develop, and launch new web scraping projects. Troubleshoot, debug, and optimize web scraping scripts. Stay up-to-date with the latest industry trends and technologies in automated data collection and cleaning Help maintain code quality and organization of projects Participate in code reviews and ensure that all solutions are aligned with standards. Create automated test cases to ensure the functionality and performance of the code Integration of data storage solutions like SQL/NoSQL databases, message brokers, and data streams for storing and analyzing the scraped data.Experience with Python development and web scraping techniques. Familiarity with web frameworks such as Django and Flask, as well as other technologies like SQL, Git, and Linux, is also required. Strong analytical and problem-solving skills, as well as good communication and teamwork abilities, are also important for the role.

The hire is expected to have good knowledge of Java based technologies as well as fair knowledge of Cloud (GCP) migration.The hire is also expected to have basic GCP understanding and hands on experience in few GCP based services like Dataflow and BigQuery.Additionally, he will also help us in setting up CI/CD pipeline in the project.Following skillset is expected from GCP Java engineer:GCP fundamentals and hands on experience in GCP managed services especially Big Query and Data Flow.Hands on Java8+ experience with understanding of OOPS and other common design patterns. Experience in developing Java based applications (on-premise and on Google Cloud) which are scalable, performant and cost effective.Provisioning new GCP infrastructure using Terraform scripting and building CI/CD in the project.Good problem-solving skills and understanding of microservices based architecture

Data Management PySpark / SparkSQL Data Modeling Python / Scala Adaptable and flexible Agility for quick learning Ability to work well in a team Commitment to quality Strong analytical skillsRoles and Responsibilities:In this role, you need to analyze and solve increasingly complex problems Your day to day interactions is with peers within Accenture You are likely to have some interaction with clients and/or Accenture management You will be given minimal instruction on daily work/tasks and a moderate level of instructions on new assignments You will need to consistently seek and provide meaningful and actionable feedback in all interactions You will be expected to be constantly on the lookout for ways to enhance value for your respective stakeholders/clients Decisions that are made by you will impact your work and may impact the work of others You would be an individual contributor and/or oversee a small work effort and/or team. Please note this role may require you to work in rotational shifts.

Job Openings for 20 Data Engineer Jobs with minimum 3 Years Experience in Gurgaon having Educational qualification of : B.C.A, B.Tech/B.E, M.C.A, M.Tech with Good knowledge in hive,spark,HBase etc. for American Express

We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:Experience with big data tools: Hadoop, Spark, Kafka, etc.Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.Experience with AWS cloud services: EC2, EMR, RDS, RedshiftExperience with stream-processing systems: Storm, Spark-Streaming, etc.Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

Good experience in computer programming languages such as: Python & SQL Minimum 3 years of working experience in Python and Python Frameworks (Flask/Django). Familiarity with concepts of MVC, Mocking, ORM, and RESTful. Solid understanding of object-oriented programming. Experience working with design architecture. Familiarity with some ORM (Object Relational Mapper) libraries. Sound knowledge of Database administration like: Managing MySQL Server, MSSQL Server. Advanced working SQL knowledge and experience working with relational databases, as wellworking familiarity with a variety of databases. Knowledge of NoSQL Databases (MongoDB, DynamoDB) is good to have. Strong knowledge on JavaScript or JQuery. Knowledge of other languages OR big data tools (Hive, Spark) is a plus. Ability to work with AWS services like Lambda, Kinesis, SQS, SNS etc., is a plus. Certificationin cloud platforms like Azure, AWS, and Google will be considered a very good asset. Strong verbal and written communication skills with ability to communicate effectively,articulate results and issues to internal and client teams.

Job Responsibilities Use different data warehousing concepts to build a data warehouse for reporting purposes. Design, develop and launch efficient and reliable data pipelines to move data across application systems and to provide intuitive analytics to business teams Actively develop and test ETL components to high standards of data quality. Assist in the creation of design best practices as well as coding and architectural guidelines, standards, and frameworks. Provide analytical support like Visualization, Business Insights, Reporting, as needed. Can guide a team of Data Engineers for a whole projectMust have: 4+ years of experience in ETL (or) data engineering role in an analytics environment. Bachelors degree in a technical field (Comp. Sci degree preferred not mandatory) Working knowledge of Relational Database Management Systems (RDBMS) like Oracle, SQL server etc. Expertise in building data pipelines & data warehousing concepts Good understanding of Big data platforms Knowledge of SQL, Python & some of the standard data science packages (Pandas, Numpy, etc.). Exposure to Visualization tools like Tableau & Power BI is a plus; Not Mandatory. Strong verbal and business communication skills.

Job Responsibilities Use different data warehousing concepts to build a data warehouse for reporting purpose. Design, develop, and launch efficient and reliable data pipelines to move data across application systems and to provide intuitive analytics to business teams Actively develop and test ETL components to high standards of data quality. Assist in the creation of design best practices as well as coding and architectural guidelines, standards, and frameworks. Provide analytical support like Visualization, Business Insights, Reporting, as needed.Must have: 2+ years of experience in ETL (or) data engineering role in an analytics environment. Bachelors degree in a technical field (Comp. Sci degree preferred not mandatory) Working knowledge of Relational Database Management Systems (RDBMS) like Oracle, SQL server etc. Expertise in building data pipelines & data warehousing concepts Good understanding of Big data platforms Knowledge of SQL, Python & some of the standard data science packages (Pandas, NumPy, etc.). Exposure to Visualization tools like Tableau & Power BI is a plus; Not Mandatory. Strong verbal and business communication skills.