Data Engineer Jobs in India

We are looking for an experienced Data Engineer to join our team in Thiruvananthapuram. This role requires you to design, build, and maintain data systems that enable data-driven decision-making across the organization.- **Data Pipeline Development**: You will create and manage data pipelines that ensure smooth flow and transformation of data from various sources into databases or data warehouses.- **Data Integration**: You will work on integrating diverse data sources, ensuring consistency and accuracy in the data collected from different platforms.- **Database Management**: Your responsibility will include setting up and maintaining database systems, optimizing their performance, and ensuring data security.- **Collaboration with Stakeholders**: You will work closely with data scientists, analysts, and other stakeholders to understand their data needs and provide effective solutions.- **Performance Optimization**: You will monitor and enhance the performance of data processes, identifying and resolving bottlenecks to ensure efficient data handling.To be successful in this role, you should have a strong understanding of data modeling and ETL (Extract, Transform, Load) processes. Proficiency in programming languages such as Python or Java and experience with SQL databases is essential. Familiarity with big data technologies like Hadoop or Spark will be a plus. You should also possess excellent problem-solving skills and the ability to work collaboratively in a fast-paced environment. A minimum of 5 years of experience in data engineering or a related field is required.

Plan and allocate resources (personnel, equipment, materials) effectively to meet project requirements.Responsible for developing and implementing production plans, ensuring timely execution of projects, and managing resources to optimize manufacturing processes.Prioritization of work.Work with ERP systems, manage inventory, and collaborate with various teams to meet production goals.Manage raw material and spare parts inventory, optimizing stock levels to prevent shortages or overstocking.Plan and schedule the operation of machines and equipment used in the manufacturing process.Analyze machine shop and assembly shop capacities to ensure optimal resource utilization and balance workloads across departments.Machine planning with ERP system as per work order requirements.Machine planning with Inventory Control.Maintaining good quality of machine and components as per company standards.Preparation of costing sheets for individual machines and for projects for cost benefits and for healthy negotiation with vendors to get the higher discounts on existing costing.To put completion dates against each item on BOM printouts and inform the concerned to put the same on Job Cards, Indents, printouts for Purchase, etc.Raw material planning & inventory control.Spare material planning & inventory control.Knowledge of CNC, VMC, lathe, drilling, surface grinding machines, etc.

As a Senior Data Engineer, you will be at the forefront of our data transformation, specifically architecting and building the next generation of our Financial Transaction Processing engine within Microsoft Fabric. You will lead the migration of legacy financial pipelines into a unified Lakehouse architecture, ensuring that every pence of revenue and every byte of roaming data is accounted for with 100% accuracy.This role requires a blend of high-level architectural design, hands-on pipeline engineering, and a deep understanding of telecom financial domains (billing, reconciliation, and inter-carrier settlements).Key ResponsibilitiesArchitectural Design: Lead the design and implementation of a Medallion Architecture (Bronze, Silver, Gold) within MS Fabric to handle high-velocity telecom transactions.Data Pipeline Engineering: Develop end-to-end ETL/ELT workflows using Fabric Data Factory, Spark Notebooks (PySpark), and Dataflows Gen2.Financial Migration: Lead the migration of legacy SQL-based financial workloads into Fabric Lakehouses and Warehouses, ensuring zero data loss and maintaining historical integrity.Configuration & Governance: Configure Fabric workspaces, capacities, and OneLake security. Implement rigorous data governance and Row-Level Security (RLS) to protect sensitive financial PII.Performance Optimization: Tune large-scale Spark jobs and SQL queries to process millions of Call Detail Records (CDRs) and financial events with minimal latency.Analytics & Reporting: Build high-performance Semantic Models and Direct Lake datasets for real-time financial reporting and margin analysis.Mandatory Skills & QualificationsTechnical ExpertiseMicrosoft Fabric: Expert-level knowledge of the Fabric ecosystem (Lakehouse, Eventhouse, Data Factory, and OneLake).Pipeline Development: Strong experience with Azure Data Factory or Fabric Pipelines, including complex orchestration and incremental loading patterns.Coding & Transformation: Advanced proficiency in PySpark/Python and T-SQL for complex financial logic (e.g., multi-currency conversion, tax calculations).Architecture: Deep understanding of Delta Lake, Star Schema modeling, and SaaS-based data platform design.DevOps: Hands-on experience with Git integration, CI/CD for Fabric items, and Azure DevOps deployment pipelines.Domain KnowledgeTelecom Finance: Familiarity with telecom-specific data such as CDRs, roaming settlements, prepaid/postpaid billing cycles, and Churn analytics.Financial Accuracy: Experience building automated reconciliation frameworks to catch revenue leakage or transaction mismatches.Required Experience8+ years in Data Engineering3+ years hands-on with Microsoft FabricProven experience delivering large-scale data migration programmes in UK telecom or retailStrong SQL & PySpark expertiseExperience with financial data reconciliation and ledger alignmentStrong stakeholder engagement across Finance, Billing, and Technology teamsExperience operating within UK enterprise governance frameworksPreferred ExperienceCertifications: DP-600 (Microsoft Fabric Analytics Engineer) or DP-203 (Azure Data Engineer).Legacy Knowledge: Experience migrating from on-prem SQL Server or older Azure Synapse environments.Real-Time Intelligence: Knowledge of KQL (Kusto) for monitoring real-time transaction streams.Skills to be evaluated onMicrosoft FabricAzure-Data-FactoryFabric-AnalyticsDatabricksMandatory Skills : Microsoft Fabric

Key Skills Required: 5+ years in Data Engineering / Distributed Systems Strong experience with AWS / GCP / Azure Expertise in Python, Spark, Ray, Kafka, Airflow, etc. Building scalable pipelines for large-scale video/audio datasets Knowledge of ML training & inference workflows Experience with fault-tolerant, high-availability systems Docker & Kubernetes experience is a plus

We are looking for a Data Engineer to join our team in Chennai. This part-time role allows you to work from home and is suitable for candidates with 0 to 2 years of experience. **Key Responsibilities:**- **Data Collection:** Gather data from various sources to ensure we have accurate and relevant information for analysis. This includes pulling data from databases, APIs, and other data repositories.- **Data Cleaning:** Improve data quality by identifying errors and inconsistencies. This means ensuring that all data is accurate, complete, and usable for analysis.- **Data Storage:** Help design and manage data storage solutions, ensuring that data is stored securely and can be accessed efficiently by team members.- **Data Transformation:** Convert raw data into a usable format for analysis. This involves using tools and programming languages to modify and structure data appropriately.- **Collaboration:** Work with data scientists and analysts to understand their data needs. This role requires communicating effectively with team members to support their projects.**Required Skills and Expectations:**- Basic understanding of data engineering concepts and tools, such as SQL, Python, or similar programming languages.- Strong attention to detail, as accuracy in data handling is crucial.- Ability to work independently and manage your time effectively while meeting deadlines.- Good communication skills to share ideas and collaborate with others in the team. - A curious mindset and willingness to learn about new technologies and data processes.

We are looking for a Gen AI Developer with 5 to 9 years of experience to join our team in Guindy. The successful candidate will work on developing innovative artificial intelligence solutions.**Key Responsibilities:**- **AI Model Development:** Create and implement machine learning models to solve complex problems, ensuring they meet performance expectations.- **Data Analysis:** Analyze large datasets to extract meaningful insights and improve AI functionalities, enhancing the effectiveness of AI solutions.- **Collaboration with Teams:** Work closely with cross-functional teams, including data scientists and software developers, to integrate AI capabilities into existing systems and products.- **Testing and Optimization:** Conduct thorough testing of AI models and applications, optimizing them for better performance and user experience.- **Continuous Learning:** Stay updated with the latest trends and advancements in AI technology, and apply new knowledge to improve current practices.**Required Skills and Expectations:**- Candidate should hold a Bachelors degree in Computer Applications (B.C.A), Bachelor of Science (B.Sc), or Bachelor of Engineering (B.E).- A strong understanding of machine learning algorithms, natural language processing, and deep learning frameworks is essential.- Proficiency in programming languages such as Python, R, or Java is required to develop and implement AI models.- Excellent problem-solving skills and the ability to work independently as well as part of a team are necessary.- Strong communication skills to effectively collaborate with team members and stakeholders are important for success in this role.Work Location: Chennai Guindy Tamil Nadu 626001Interested For Job Please Call Immediately & Share you updated profileCall/Whatsapp: Mr. Pradeep HR careerturn5techhrsolution@gmail.com

We are seeking a motivated and enthusiastic Data Processing Engineer to join our team. This role is perfect for recent graduates or individuals looking to start their career in data processing.As a Data Processing Engineer, you will be responsible for handling and organizing data efficiently. Your key responsibilities will include:- **Data Entry**: Accurately input data into our systems while ensuring all information is correct and up to date.- **Data Quality Assurance**: Review and validate data to identify any errors or inconsistencies, and fix them promptly.- **Data Maintenance**: Regularly update and maintain databases to keep them organized and accessible for team members.- **Reporting**: Generate basic reports from the data you process, helping the team make informed decisions.To be successful in this role, you should possess strong attention to detail and the ability to work independently from home. You must be comfortable using computers and familiar with basic data processing tools. Strong communication skills are essential to collaborate effectively with team members. Having a proactive approach to problem-solving will also be important as you work through data challenges.We welcome applications from females who have completed their 10th grade education and are eager to begin a full-time position in data processing. This is a fantastic opportunity to learn, grow, and launch your career in the field of data.

We are looking for a highly analytical Lead Process Engineer to drive operational excellence and process transformation within lending operations. The role involves leveraging advanced statistical methods, process intelligence, and AI-assisted tools to improve efficiency, reduce variation, and enable scalable business operations. This position also includes mentoring junior team members while collaborating with cross-functional stakeholders.Key Responsibilities:Lead end-to-end process improvement initiatives using DMAIC methodologyApply statistical analysis, DoE, and SPC techniques to enhance process performanceDesign and optimize operational workflows across lending value streamsBuild and maintain process intelligence dashboards for real-time insightsConduct FMEA-based risk assessments and implement mitigation strategiesDrive AI-assisted process improvements using GenAI tools and automation frameworksPerform capacity forecasting and time series modeling for scaling operationsCollaborate with Product, Data, and Operations teams for transformation initiativesPresent ROI-driven insights and business cases to senior stakeholdersMentor and develop Process Analysts and Junior EngineersRequired Skills & Qualifications:7-9 years of experience in process engineering, quality engineering, or operations improvementStrong expertise in Lean Six Sigma methodologies (DMAIC execution experience mandatory)Hands-on experience in statistical tools: DoE, SPC, regression, hypothesis testingExperience with process mining / intelligence tools (Power BI, Celonis, etc.)Proven experience in FMEA and risk assessment frameworksStrong capability in capacity planning and time series forecastingExposure to AI/GenAI tools in operational environmentsProficiency in Excel (advanced), Python or Minitab, and data visualization toolsStrong stakeholder management and communication skills

As a Data Engineer, your responsibilities will include collecting and analyzing data to help inform business decisions. You will be responsible for data management, ensuring that data is accurate and up-to-date. This will involve using tools such as Informatica, PLSQL, SQL, and Oracle SQL to manipulate and query large datasets.Your skills should include a strong understanding of data analysis techniques, such as data mining and statistical analysis.

Data Extraction Engineer designs extraction systems (and not just scripts). They build and maintain a next-generation data acquisition platform that treats web scraping as a declarative, specification-driven discipline. Instead of hard-coding XPaths for every site, Web Scraping Developer defines what data is neededusing schemas, natural language descriptions, or visual blueprintsand lets intelligent pipelines figure out how to get it.Key Responsibilities:Specification-Driven Extraction Engineering-Design and maintain declarative extraction specificationsusing Pydantic models, JSON schemas, or domain-specific languagesthat describe exactly which fields to capture, their types, and validation rules.Implement pipelines that translate these specifications into executable extraction plans, leveraging both classical (Scrapy, Playwright) and AI-augmented (LLM-based semantic parsing) backends.Build reusable specification libraries for recurring data types (product prices, tariff codes, regulatory texts) to accelerate onboarding of new sources.Autonomous & Self-Healing Systems-Deploy self-healing spiders that automatically detect website layout changes and repair themselves using Model Context Protocol (MCP) servers (e.g., Scrapy MCP Server, Playwright MCP).Integrate semantic extraction (Scrapy-LLM, custom LLM pipelines) to eliminate selector brittlenessspiders rely on field descriptions, not fragile XPaths.Orchestrate complex, multi-step browsing workflows with agentic frameworks (BMAD/TEA, AutoGPT-like agents) that reason about page state, adapt to anti-bot measures, and correct their own behaviour in real time.Platform Thinking & Reusability-Move beyond one-off scrapers: build a component-based extraction platform where selectors, login handlers, and pagination logic are shared, versioned, and tested.Implement monitoring, alerting, and automatic rollback for failed extraction runs.Champion ethical crawling by designrate limiting, robots.txt respect, and compliance with GDPR/CCPA are built into the specification layer, not retrofitted.Collaboration & Continuous Innovation-Partner with data scientists and domain experts to refine extraction specifications for complex, unstructured domains (e.g., legal texts, tariff classifications).Evaluate and pilot emerging tools to push automation coverage beyond 90%.Document and evangelise specification-driven best practices across the engineering organisation.Candidate Profile:Education and Experience -Bachelors degree in Computer Science3+ years of experience in web scraping or data extractionSkills and competences-Specification-Driven Extraction Experience defining extraction requirements via schemas (Pydantic, JSON Schema) and executing them through both traditional crawlers and LLM-based semantic parsers.SelfHealing & Semantic Extraction Handson use of ScrapyLLM, Scrapy MCP Server, or similar systems that decouple field definitions from page structure.Agentic Workflows Familiarity with frameworks that give LLMs browser control (Playwright + MCP, BMAD/TEA) to handle complex, nondeterministic crawling tasks.Classical Scraping Fundamentals You still know how to write a Scrapy spider or a Playwright script when needed, but you actively seek to replace that work with reusable, specification-driven components.Data Validation & Storage Ability to define validation rules within specifications and land clean data into SQL/NoSQL databases or data lakes.Python proficiency: the focus is on an extraction engineer who happens to use Python.HTTP, DOM, XPath, CSS.Basic API integration and authentication flows.Preferred / Nice-to-Have Skills:Contributions to open-source scraping or AI-automation projects.Experience training or fine-tuning small LLMs for domain-specific extraction.Familiarity with data privacy engineering (GDPR, CCPA) baked into specification design.DevOps light Docker, CI/CD for testing extraction specifications.Mindset & Approach (Non-Negotiable):Strong belief that the future of scraping is declarative, not imperative. Youd rather write a schema that says extract the price than debug an XPath when a website redesigns.Looking to shift from code that scrapes to systems that understand extraction.

Role: Data Engineer 2.0Location - RemoteExperience: Min. 5 YearsNotice Period- Immediate to 15 Days or serving notice periodKey Responsibilities:Design and implement manual test strategies for real-time streaming use cases using Azure Service Bus, Event Hubs, Kafka, and Azure Functions.Validate Spark Streaming applications, including unbounded data flows, streaming DataFrames, checkpoints, and streaming joins.Develop test plans for containerized microservices deployed on Kubernetes, ensuring scalability and fault tolerance.Test data ingestion and transformation workflows across open table formats like Delta Lake, Apache Iceberg, and Hudi.Good to Have:Monitoring and troubleshooting system performance using observability stacks such as Prometheus, Grafana, and ELK.functional and performance testing on analytical databases and query engines such as Trino, StarRocks, and ClickHouse.Testing and validation of data products designed under data mesh architecture, ensuring domain-oriented data quality and governance.

We are looking for a highly experienced Python Architect to design and lead large-scale, enterprise-grade Python solutions for AI, data, and analytics platforms. The role requires deep hands-on expertise in Python across architecture, development, and production systems, supporting end-to-end AI and data-driven solutions.Key Responsibilities:Architect scalable, high-performance systems using Python as the core technologyDefine technical architecture for AI, data engineering, and analytics platformsLead design and development of microservices, APIs, and backend frameworks in PythonEnsure production-ready implementations with strong focus on performance, security, and reliabilityCollaborate with data science, ML, DevOps, and product teams to operationalize AI solutionsEstablish coding standards, best practices, and architectural guidelinesReview code, mentor senior engineers, and provide technical leadershipDrive innovation and continuous improvement in Python-based system designRequired Skills & Qualifications:1218 years of experience working fully and extensively in PythonStrong expertise in Python system architecture and large-scale backend designExperience supporting AI/ML, data engineering, or analytics platformsSolid understanding of microservices, REST APIs, and distributed systemsHands-on experience with cloud platforms (AWS/Azure/GCP)Strong problem-solving skills and ability to lead complex technical initiativesExcellent communication and stakeholder collaboration skills

Job Description: We are looking for Data Engineers to join our team. You will use various methods to transform raw data into useful data systems. For example, youll create algorithms and conduct statistical analysis. Overall, youll strive for efficiency by aligning data systems with business goals. To succeed in this position, you should have strong analytical skills and the ability to combine data from different sources. Data engineer skills also include familiarity with several programming languages and knowledge of machine learning methods. Job Requirements: Participate in the customers system design meetings and collect the functional/technical requirements. Build up data pipelines for consumption by the data science team. Skillful in ETL process and tools. Clear understanding and experience with Python and PySpark or Spark and SCALA, with HIVE, Airflow, Impala, and Hadoop and RDBMS architecture. Experience in writing Python programs and SQL queries. Experience in SQL Query tuning. Experienced in Shell Scripting (Unix/Linux). Build and maintain data pipelines in Spark/Pyspark with SQL and Python or SCALA. Knowledge of Cloud (Azure/AWS/GCP, etc..) technologies is additional. Good to have knowledge of Kubernetes, CI/CD concepts, Apache Kafka Suggest and implement best practices in data integration. Guide the QA team in defining system integration tests as needed. Split the planned deliverables into tasks and assign them to the team. Needs to Maintain/Deploy the ETL code and follow the Agile methodology Needs to work on optimization wherever applicable. Good oral, written and presentation skills. Preferred Qualifications: Degree in Computer Science, IT, or a similar field; a Masters is a plus. Hands-on experience with Python and Pyspark Or Hands-on experience with Spark and SCALA. Great numerical and analytical skills. Working knowledge of cloud platforms such as MS Azure, AWS, etc..

Responsibilities: Designing and implementing data pipelines to collect, clean, and transform data from various sources Building and maintaining data storage and processing systems, such as databases, data warehouses, and data lakes Ensuring data is properly secured and protected Developing and implementing data governance policies and proceduresCollaborating with business analysts, data analysts, data scientists, and other stakeholders to understand their data needs and ensure they have access to the data they need Sharing knowledge with the wider business, working with other BAs and technology teams to make sure processes and ways of working is documented. Collaborate with Big Data Solution Architects to design, prototype, implement, and optimize data ingestion pipelines so that data is shared effectively across various business systems. Ensure the design, code and procedural aspects of solution are production ready, in terms of operational, security and compliance standards. Participate in day-to-day project and agile meetings and provide technical support for faster resolution of issues. Clearly and concisely communicating to the business, on status of items and blockers. Have an end-to-end knowledge of the data landscape within the company. Skills & Experience: 10+ years of design & development experience with big data technologies like Azure, AWS or GCP Preferred is Azure & Databricks, with experience in Azure DevOps Experience in data visualising technology in DL, like PowerBI Proficient in Python, PySpark and SQL Proficient in querying and manipulating data from various DB (relational and big data). Experience of writing effective and maintainable unit and integration tests for ingestion pipelines. Experience of using static analysis and code quality tools and building CI/CD pipelines. Excellent communication, problem-solving, and leadership skills, and be able to work well in a fast-paced, dynamic environment.

We are looking for an AI Engineer with 5 to 11 years of experience to join our team in Thiruvananthapuram. In this full-time role, you will work on developing innovative AI solutions and applications.**Key Responsibilities:**- **Develop AI Models:** Create and train advanced machine learning models to solve complex problems, ensuring they are accurate and efficient.- **Conduct Research:** Stay updated on the latest AI trends and research findings to implement cutting-edge techniques into projects.- **Collaborate with Teams:** Work closely with data scientists and software engineers to integrate AI features into existing applications and systems.- **Optimize Algorithms:** Fine-tune algorithms for performance improvements, ensuring they meet project requirements and consumer needs.- **Debug and Test:** Perform thorough testing and debugging of AI systems to identify issues and enhance functionality.- **Document Solutions:** Maintain clear documentation of algorithms and processes to facilitate knowledge sharing and onboarding of new team members.**Required Skills and Expectations:**The ideal candidate should have a strong foundation in machine learning, deep learning, and data analytics. Proficiency in programming languages such as Python or Java is essential, along with experience in frameworks like TensorFlow or PyTorch. You should possess excellent problem-solving skills and the ability to work collaboratively in a team. Strong communication skills are necessary to explain complex concepts to non-technical stakeholders and to contribute effectively to discussions. A degree in Computer Science, Engineering, or a related field is preferred.

Key ResponsibilitiesTechnical Documentation: Maintaining project blueprints, technical manuals, and compliance records.Progress Monitoring: Tracking the project timeline against the master schedule (using tools like MS Project or Excel) and highlighting potential delays.Resource Coordination: Ensuring that parts, tools, and materials arrive at the site or production line exactly when the team needs them.Quality Assurance: Assisting in site visits or shop floor inspections to ensure that work is being done according to the technical specifications.Communication Bridge: Facilitating information flow between departments-for example, explaining design changes to the production team or clarifying technical issues to the client.Risk Mitigation: Identifying small technical issues early on before they snowball into costly project delays.

Key ResponsibilitiesTechnical Design & Drafting: Using CAD software to create layouts for Medical Gas Pipeline Systems (MGPS), nurse call systems, or ambulance interior configurations.Site Supervision: Managing on-site installation teams (technicians, welders, and fitters) to ensure work meets ISO and AIS-125 safety standards.Project Scheduling: Developing timelines (Gantt charts) to ensure hospital wings or ambulance fleets are delivered on time.Quality Control & Testing: Conducting pressure tests on gas lines and load-testing stretchers/cots before handover to the client.Procurement Coordination: Working with the supply chain to ensure specialized medical-grade materials (like degreased copper pipes or reinforced alloys) are available on-site.Regulatory Compliance: Ensuring every aspect of the project adheres to healthcare building codes and medical device regulations.

Dear Candidate ,Give dynamic start to your career with VMS Group SECRET TECHNOLOGIES INDIA , we are looking for freshers and experienced candidates who want to make good career in IT or NON It field.Job Type: Full time.Eligibility criteria : Fresher /Experience 0-5 Year.Location : Pune. OR PAN INDIADesired Qualification - Any Bachelor degree Minimum BA ,B.Com - Commerce, B.E , B.Tech, M.Sc , M.C.A B.C.A ,Any Graduates, Post Graduates, can also apply. Should be comfortable to work any location of pune or Pan India.Your application for theTesting & DevelopmentBack Office / Data EntryTechnical Support, Non-Voice ProcessBDE (canadian, US, UK)-In HouseBusiness Development Executive, On FieldOperations/Customer Service , Customer Service (Web)BPO MNC and Domestic ( Hindi Marathi or Any local language)Accounts & Finance /Financial AnalystPosition stood out to us and we would like to invite you for a Face to face interview Telephone interview at our office to get to know you a bit better.You will meet with the MR. Prem Sir or Aishwarya Ma'am.The interview will last about 20 minutes and you'll have the chance to discuss the above position and learn more about our company. Please carry your updated resume at list Two CopiesDesired Candidate - Candidate should be Dynamic, Enterprising, Presentable with excellent communication and written skills.If candidate is interested please visit our office. Monday to Saturday 10 am to 5 pm bitwin anytime Meet- Prem Sir / Ms. Aishwarya Ma'am Interview Venue-building A -15th , city vista, foundation road, Kharadi, Pune, Maharashtra 411014We will take 2 rounds of an interview.-Contact No. 7066165593 Aishwarya Ma'am 7066147637 Prem Sir 1. HR Round and simple logic checking P.I round.2. Technical HR Final round by Senior HR Manager.Thanks & Regards SECRET TECHNOLOGIES INDIA VMS GROUP Placement / Contract Labour / Real EstateAddress -building A -15th , city vista, foundation road, Kharadi, Pune, Mahara

Key Responsibilities:Build and manage data pipelines and ETL processesWork with large datasets using tools like Spark, Hadoop, or SQLEnsure data quality and performance optimizationRequirements:Experience in Python/SQLHands-on with ETL tools and big data technologiesUnderstanding of data warehousing conceptsBrief Summary:Develops scalable data systems to support analytics and business insights.

Key Responsibilities: Cloud EngineerDesign, implement, and manage CI/CD pipelinesAutomate deployments and infrastructure using tools like Terraform/AnsibleMonitor system performance and ensure high availabilityRequirements:Experience with AWS/AzureKnowledge of Docker, Kubernetes, and scriptingStrong problem-solving skillsBrief Summary:Responsible for automating and optimizing cloud infrastructure and deployment processes.

Key Responsibilities Requirement Analysis: Collaborate with stakeholders to understand businessrequirements and data sources, and define the architecture and design of dataengineering models to meet these requirements. Architecture Design: Design scalable, reliable, and efficient data engineering models,including algorithms, data pipelines, and data processing systems, to support businessrequirements and quantitative analysis. Technology Selection: Evaluate using POCs and recommend appropriate technologies,frameworks, and tools for building and managing data engineering models, consideringfactors like performance, scalability, and cost-effectiveness. Data Processing: Develop and implement data processing logic, including data cleansing,transformation, and aggregation, using technologies such as AWS Glue, Batch, Lambda. Quantitative Analysis: Collaborate with data scientists and analysts to develop algorithmsand models for quantitative analysis, using techniques such as regression analysis,clustering, and predictive modeling. Model Evaluation: Evaluate the performance of data engineering models using metricsand validation techniques, and iterate on models to improve their accuracy andeffectiveness. Data Visualization: Create visualizations of data and model outputs to communicateinsights and findings to stakeholders.Data Engineering: Understanding of Data engineering principles and practices, includingdata ingestion, processing, transformation, and storage, using tools and technologiessuch as AWS Glue, Batch, Lambda. Quantitative Analysis: Proficiency in quantitative analysis techniques, including statisticalmodeling, machine learning, and data mining, with experience in implementingalgorithms for regression analysis, clustering, classification, and predictive modeling. Programming Languages: Proficiency in programming languages commonly used in dataengineering and quantitative analysis, such as Python, R, Java, or Scala,