Big Data Jobs

AIML Engineer Job DescriptionExp: 4+ yrsNP : Immediate / 15 daysLocation : Bhubaneswar/ Bangalore/ ChennaiPosition OverviewWe are seeking a highly skilled Artificial Intelligence & Machine Learning Engineer to design, develop, and deploy intelligent systems that solve complex business problems. The role involves applying advanced algorithms, data science techniques, and deep learning frameworks to build scalable AI solutions.Key ResponsibilitiesModel Development: Design, train, and optimize ML/DL models for predictive analytics, NLP, computer vision, and recommendation systems.Data Engineering: Collect, preprocess, and analyze large datasets to ensure quality and usability.Deployment & Integration: Implement AI models into production environments using cloud platforms (AWS, Azure, GCP).Research & Innovation: Stay updated with emerging AI/ML technologies and apply them to business challenges.Collaboration: Work closely with product managers, data scientists, and software engineers to deliver end-to-end AI solutions.Performance Monitoring: Continuously evaluate and improve deployed models for accuracy, scalability, and efficiency.Required Skills & QualificationsStrong programming skills in Python, R, Java, or C++.Expertise in ML/DL frameworks: TensorFlow, PyTorch, Keras, Scikit-learn.Solid understanding of statistics, probability, and linear algebra.Experience with cloud services: AWS SageMaker, Azure ML, Google AI Platform.Knowledge of data handling tools: SQL, Spark, Hadoop.Familiarity with NLP, computer vision, reinforcement learning.Strong problem-solving, communication, and teamwork skills.Preferred QualificationsExperience with MLOps (CI/CD pipelines for ML).Hands-on with big data technologies.Publications or contributions to AI research.Infested candidate can share CV to Rekha.C@eagledrift.com

Database analysts design, develop and administer data management solutions using database management software. Data administrators develop and implement data administration policy, standards and models. They are employed in information technology consulting firms and in information technology units throughout the private and public sectors.This group performs some or all of the following duties:Database analystsCollect and document user requirementsDesign and develop database architecture for information systems projectsDesign, construct, modify, integrate, implement and test data models and database management systemsConduct research and provide advice to other informatics professionals regarding the selection, application and implementation of database management toolsOperate database management systems to analyze data and perform data mining analysisMay lead, co-ordinate or supervise other workers in this group.Data administratorsDevelop and implement data administration policy, standards and modelsResearch and document data requirements, data collection and administration policy, data access rules and securityDevelop policies and procedures for network and/or Internet database access and usage and for the backup and recovery of dataConduct research and provide advice to other information systems professionals regarding the collection, availability, security and suitability of dataWrite scripts related to stored procedures and triggersMay lead and co-ordinate teams of data administrators in the development and implementation of data policies, standards and models.

Requirements:8+ years of experience in Core Java and Spring Framework (Mandatory)Minimum 2 years of experience in Google Cloud Platform (GCP) (Mandatory)Hands-on experience with Apache Beam / Dataflow for building ETL/data pipelines (Mandatory)Strong expertise in big data processing on distributed systemsProficiency with RDBMS, NoSQL, and Cloud-native databasesExperience in handling multiple data formats (Flat file, JSON, Avro, XML, etc.) with schema/contract definitionsExperience in Microservices architecture and API integration patternsStrong understanding of data structures and data model design

Sr. Big Data Engineer Location: PuneExperience: 10+ years Mode: HybridRole Overview:We are seeking a talented Sr. Big Data Engineer to design, develop, and support a highly scalable, distributed SaaS-based Security Risk Prioritization product. You will lead the design and evolution of our data platform and pipelines, providing technical leadership to a team of engineers and architects.Key Responsibilities: Provide technical leadership on data platform design, roadmaps, and architecture. Design and implement scalable architecture for Big Data and Microservices environments. Drive technology explorations, leveraging knowledge of internal and industry prior art. Ensure quality architecture and design of systems, focusing on performance, scalability, and security. Mentor and provide technical guidance to other engineers.Required Skills & Technologies: Mandatory: Kafka, Scala, Spark. Big Data & Data Streaming: Spark, Kafka, Hadoop, Presto, Airflow, Data lakes, lambda architecture, kappa, and kappa ++ architectures with flink data streaming. Databases & Caching: RDBMS, NoSQL, Oracle, Cassandra, Redis. Search Solutions: Solr, Elastic. ML & Automation: Experience with ML models engineering and related deployment, scripting, and automation. Architecture: In-depth experience with messaging queues and caching components. Other Skills: Strong troubleshooting and performance benchmarking skills for Big Data technologies.Qualifications: Bachelors degree in Computer Science or equivalent. 8+ years of total experience, with 6+ years relevant. 2+ years in designing Big Data solutions with Spark. 3+ years with Kafka and performance testing for large infrastructure.

Role: Bigdata Developer - Scala SparkExp: 5+ YrsMode of Work: WFO All 5 daysLocation: Chennai/Bangalore/PuneInterview: Any one Level F2FJob Description: Total IT / development experience of 3+ years Experience in Spark (Scala-Spark ) developing Big Data applications on Hadoop, Hive and/or Kafka, HBase, MongoDB Deep knowledge of Scala-Spark libraries to develop and debug complex data engineering challenges Experience in developing sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies Exposure to deploying on Cloud platforms At least 2 years of development experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Spark-Scala At least 2 years of development experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC), Resource Management, Distributed Processing and RDBMS At least 2 years of developing applications in Agile with Monitoring, Build Tools, Version Control, Unit Test, Unix Shell Scripting, TDD, CI/CD, Change Management to support DevOps

Role OverviewThis position requires a detail-oriented data engineer who can independently architect and implement data pipelines, while also serving as a trusted technical partner in client engagements and stakeholder meetings. Youll work hands-on with PySpark, Airflow, Python, and SQL, driving end-to-end data migration and platform modernization efforts across Azure and AWS.In addition to technical execution, youll contribute to sprint planning, backlog prioritization, and continuous integration/deployment of data infrastructure. This is a senior-level individual contributor role with direct visibility across engineering, product, and client delivery functions.Key ResponsibilitiesLead design and development of enterprise-grade data pipelines and cloud data migration architectures.Build scalable, maintainable ETL/ELT pipelines using Apache Airflow, PySpark, and modern data services.Write efficient, modular, and well-tested Python code, grounded in clean architecture and performance principles.Develop and optimize complex SQL queries across diverse relational and analytical databases.Contribute to and uphold standards for data modeling, data governance, and pipeline performance.Own the implementation of CI/CD pipelines to enable reliable deployment of data workflows and infrastructure (e.g., GitHub Actions, Azure DevOps, Jenkins).Embed unit testing, integration testing, and monitoring in all stages of the data pipeline lifecycle.Participate actively in Agile ceremonies: sprint planning, daily stand-ups, retrospectives, and backlog grooming.Collaborate directly with clients, stakeholders, and cross-functional teams to translate business needs into scalable technical solutions.Act as a technical authority within the teamleading architectural decisions and contributing to internal best practices and documentation.

Must have 4-6 years of experience in Data warehouse, ETL, BI projects Must have atleast 4+ years of experience in Snowflake Expertise in Snowflake architecture is must. Must have atleast 3+ years of experience and strong hold in Python/PySpark Must have experience implementing complex stored Procedures and standard DWH and ETL concepts Proficient in Oracle database, complex PL/SQL, Unix Shell Scripting, performance tuning and troubleshoot Good to have experience with AWS services and creating DevOps templates for various AWS services. Experience in using Github, Jenkins Good communication and Analytical skills Snowflake certification is desirable

Qualifications: 5-7 years of professional experience in machine learning, data science, or related roles. Should have good exposure and understanding in time series Modelling using ARIMA, ARIMAX Exposure into how to handle underfitting and overfitting. Should be capable of applying techniques which helps to generalize Models. Regularization techniques LASSO, RIDGE & ELASTIC NET and when to apply them. Good exposure in Unsupervised machine learning like clustering, dimensionality reduction, Outlier detection Ability to understand how Models are optimized using various techniques including Gradient Descent approach. Good understanding of deep learning algorithms CNN, RNN, LSTM and how to control overfitting in such cases. Good hands on in data engineering to process huge scale of data using Big Data (Spark/Hive) Good coding practices to write production ready code for creating data pipeline for Models to consume. Very good hands on in python (Pandas/Numpy/Scikit-Learn/NLTK/spaCy/Matplotlib) Able to apply the right level of ML techniques for the given problem statement. Ability to access information contained in data and engineer appropriate features. Familiar with Python language and various platforms for hosting ML models Expert in model training, tuning and validation. Expert in statistical techniques, deep learning methodologies, GenAI, alternate techniques such as Bayesian etc. Exposure to big data and related models Ability to articulate model choice and convert outcome for business decision making. Expert in Model Development Lifecycle from sourcing to model monitoring Ability to create code that is highly performance in the given platform. Ability to map model and business use case to the appropriate platform and tools needed. Understanding of technical and machine learning governance Ability to validate and articulate model choices with relevant metrics (precision, recall, confusion matrix, RMSE,

we are in the lookout for a seasoned Cloud Data Lead. We are eager to connect with you if you have extensive experience in cloud platforms, data architecture, and leadership!

Training on Java, Python, Big data technologies, Hadoop, Spark, PiSpark, Kafka, Airflow, Machine Learning, Deep learning, Tableau, Power BI, TableauMin Experience: 2 to 3 years of training experience.

Design, develop, and maintain scalable data processing pipelines using Kafka, PySpark, Python/Scala, and Spark.Work extensively with the Kafka and Hadoop ecosystem, including HDFS, Hive, and other related technologies.Write efficient SQL queries for data extraction, transformation, and analysis.Implement and manage Kafka streams for real-time data processing.Utilize scheduling tools to automate data workflows and processes.

We are looking for an only immediate joiner and experienced Big Data Developer with a strong background in Kafka, PySpark, Python/Scala, Spark, SQL, and the Hadoop ecosystem.The ideal candidate should have over 5 years of experience and be ready to join immediately. This role requires hands-on expertise in big data technologies and the ability to design and implement robust data processing solutions.Key Responsibilities:Design, develop, and maintain scalable data processing pipelines using Kafka, PySpark, Python/Scala, and Spark.Work extensively with the Kafka and Hadoop ecosystem, including HDFS, Hive, and other related technologies.Write efficient SQL queries for data extraction, transformation, and analysis.Implement and manage Kafka streams for real-time data processing.Utilize scheduling tools to automate data workflows and processes.Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.Ensure data quality and integrity by implementing robust data validation processes.Optimize existing data processes for performance and scalability.

Invitation for B2B Partnerships: Seeking Software Development Support! We are looking to collaborate with companies that can provide 10 skilled Data Engineers to support our data engineering requirements pipeline for Euoropean clients on a contract basis.Requirements:Expertise in data engineering, including data processing and ETL.Proficiency in SQL and NoSQL databases.Experience with Hadoop or Spark.Experience in AWS or Azure.Snowflake is an added advantage.If your company is equipped to provide top-notch Data Engineers, we invite you to submit your proposal with terms and conditions.Please contact us by email or Whatsapprakanalytics@gmail.com (or) 9900173022