Data Engineer Jobs

We are looking for an experienced Data Engineer to join our team in Thiruvananthapuram. This role requires you to design, build, and maintain data systems that enable data-driven decision-making across the organization.- **Data Pipeline Development**: You will create and manage data pipelines that ensure smooth flow and transformation of data from various sources into databases or data warehouses.- **Data Integration**: You will work on integrating diverse data sources, ensuring consistency and accuracy in the data collected from different platforms.- **Database Management**: Your responsibility will include setting up and maintaining database systems, optimizing their performance, and ensuring data security.- **Collaboration with Stakeholders**: You will work closely with data scientists, analysts, and other stakeholders to understand their data needs and provide effective solutions.- **Performance Optimization**: You will monitor and enhance the performance of data processes, identifying and resolving bottlenecks to ensure efficient data handling.To be successful in this role, you should have a strong understanding of data modeling and ETL (Extract, Transform, Load) processes. Proficiency in programming languages such as Python or Java and experience with SQL databases is essential. Familiarity with big data technologies like Hadoop or Spark will be a plus. You should also possess excellent problem-solving skills and the ability to work collaboratively in a fast-paced environment. A minimum of 5 years of experience in data engineering or a related field is required.

Plan and allocate resources (personnel, equipment, materials) effectively to meet project requirements.Responsible for developing and implementing production plans, ensuring timely execution of projects, and managing resources to optimize manufacturing processes.Prioritization of work.Work with ERP systems, manage inventory, and collaborate with various teams to meet production goals.Manage raw material and spare parts inventory, optimizing stock levels to prevent shortages or overstocking.Plan and schedule the operation of machines and equipment used in the manufacturing process.Analyze machine shop and assembly shop capacities to ensure optimal resource utilization and balance workloads across departments.Machine planning with ERP system as per work order requirements.Machine planning with Inventory Control.Maintaining good quality of machine and components as per company standards.Preparation of costing sheets for individual machines and for projects for cost benefits and for healthy negotiation with vendors to get the higher discounts on existing costing.To put completion dates against each item on BOM printouts and inform the concerned to put the same on Job Cards, Indents, printouts for Purchase, etc.Raw material planning & inventory control.Spare material planning & inventory control.Knowledge of CNC, VMC, lathe, drilling, surface grinding machines, etc.

Database Analysts Design, Develop and Administer Data Management Solutions Using Database Management Software. Data Administrators Develop and Implement Data Administration Policy, Standards and Models. They are Employed in Information Technology Consulting Firms and in Information Technology Units Throughout the Private and Public Sectors.This Group Performs Some or all of the Following Duties:Database AnalystsCollect and Document User RequirementsDesign and Develop Database Architecture for Information Systems ProjectsDesign, Construct, Modify, Integrate, Implement and Test Data Models and Database Management SystemsConduct Research and Provide Advice to other Informatics Professionals Regarding the Selection, Application and Implementation of Database Management ToolsOperate Database Management Systems to Analyze Data and Perform Data Mining AnalysisMay Lead, Co-ordinate or Supervise other Workers in this Group.Data AdministratorsDevelop and Implement Data Administration Policy, Standards and ModelsResearch and Document Data Requirements, Data Collection and Administration Policy, Data Access Rules and SecurityDevelop Policies and Procedures for Network And/or Internet Database Access and Usage and for the Backup and Recovery of DataConduct Research and Provide Advice to other Information Systems Professionals Regarding the Collection, Availability, Security and Suitability of DataWrite Scripts Related to Stored Procedures and TriggersMay Lead and Co-ordinate Teams of Data Administrators in the Development and Implementation of Data Policies, Standards and Models.

Key Skills Required: 5+ years in Data Engineering / Distributed Systems Strong experience with AWS / GCP / Azure Expertise in Python, Spark, Ray, Kafka, Airflow, etc. Building scalable pipelines for large-scale video/audio datasets Knowledge of ML training & inference workflows Experience with fault-tolerant, high-availability systems Docker & Kubernetes experience is a plus

As a Senior Data Engineer, you will be at the forefront of our data transformation, specifically architecting and building the next generation of our Financial Transaction Processing engine within Microsoft Fabric. You will lead the migration of legacy financial pipelines into a unified Lakehouse architecture, ensuring that every pence of revenue and every byte of roaming data is accounted for with 100% accuracy.This role requires a blend of high-level architectural design, hands-on pipeline engineering, and a deep understanding of telecom financial domains (billing, reconciliation, and inter-carrier settlements).Key ResponsibilitiesArchitectural Design: Lead the design and implementation of a Medallion Architecture (Bronze, Silver, Gold) within MS Fabric to handle high-velocity telecom transactions.Data Pipeline Engineering: Develop end-to-end ETL/ELT workflows using Fabric Data Factory, Spark Notebooks (PySpark), and Dataflows Gen2.Financial Migration: Lead the migration of legacy SQL-based financial workloads into Fabric Lakehouses and Warehouses, ensuring zero data loss and maintaining historical integrity.Configuration & Governance: Configure Fabric workspaces, capacities, and OneLake security. Implement rigorous data governance and Row-Level Security (RLS) to protect sensitive financial PII.Performance Optimization: Tune large-scale Spark jobs and SQL queries to process millions of Call Detail Records (CDRs) and financial events with minimal latency.Analytics & Reporting: Build high-performance Semantic Models and Direct Lake datasets for real-time financial reporting and margin analysis.Mandatory Skills & QualificationsTechnical ExpertiseMicrosoft Fabric: Expert-level knowledge of the Fabric ecosystem (Lakehouse, Eventhouse, Data Factory, and OneLake).Pipeline Development: Strong experience with Azure Data Factory or Fabric Pipelines, including complex orchestration and incremental loading patterns.Coding & Transformation: Advanced proficiency in PySpark/Python and T-SQL for complex financial logic (e.g., multi-currency conversion, tax calculations).Architecture: Deep understanding of Delta Lake, Star Schema modeling, and SaaS-based data platform design.DevOps: Hands-on experience with Git integration, CI/CD for Fabric items, and Azure DevOps deployment pipelines.Domain KnowledgeTelecom Finance: Familiarity with telecom-specific data such as CDRs, roaming settlements, prepaid/postpaid billing cycles, and Churn analytics.Financial Accuracy: Experience building automated reconciliation frameworks to catch revenue leakage or transaction mismatches.Required Experience8+ years in Data Engineering3+ years hands-on with Microsoft FabricProven experience delivering large-scale data migration programmes in UK telecom or retailStrong SQL & PySpark expertiseExperience with financial data reconciliation and ledger alignmentStrong stakeholder engagement across Finance, Billing, and Technology teamsExperience operating within UK enterprise governance frameworksPreferred ExperienceCertifications: DP-600 (Microsoft Fabric Analytics Engineer) or DP-203 (Azure Data Engineer).Legacy Knowledge: Experience migrating from on-prem SQL Server or older Azure Synapse environments.Real-Time Intelligence: Knowledge of KQL (Kusto) for monitoring real-time transaction streams.Skills to be evaluated onMicrosoft FabricAzure-Data-FactoryFabric-AnalyticsDatabricksMandatory Skills : Microsoft Fabric

We are looking for a Gen AI Developer with 5 to 9 years of experience to join our team in Guindy. The successful candidate will work on developing innovative artificial intelligence solutions.**Key Responsibilities:**- **AI Model Development:** Create and implement machine learning models to solve complex problems, ensuring they meet performance expectations.- **Data Analysis:** Analyze large datasets to extract meaningful insights and improve AI functionalities, enhancing the effectiveness of AI solutions.- **Collaboration with Teams:** Work closely with cross-functional teams, including data scientists and software developers, to integrate AI capabilities into existing systems and products.- **Testing and Optimization:** Conduct thorough testing of AI models and applications, optimizing them for better performance and user experience.- **Continuous Learning:** Stay updated with the latest trends and advancements in AI technology, and apply new knowledge to improve current practices.**Required Skills and Expectations:**- Candidate should hold a Bachelors degree in Computer Applications (B.C.A), Bachelor of Science (B.Sc), or Bachelor of Engineering (B.E).- A strong understanding of machine learning algorithms, natural language processing, and deep learning frameworks is essential.- Proficiency in programming languages such as Python, R, or Java is required to develop and implement AI models.- Excellent problem-solving skills and the ability to work independently as well as part of a team are necessary.- Strong communication skills to effectively collaborate with team members and stakeholders are important for success in this role.Work Location: Chennai Guindy Tamil Nadu 626001Interested For Job Please Call Immediately & Share you updated profileCall/Whatsapp: Mr. Pradeep HR careerturn5techhrsolution@gmail.com

Minimum 4+ years of hands-on experience in data engineering or data management, preferably in the energy or industrial domain. Design, develop, and maintain scalable ETL/ELT pipelines to process structured and unstructured data from diverse sources including SAP ECC / S/4HANA, SQL Server, Oracle, other RDBMS, Excel/CSV files, Parquet, JSON, XML, and APIs. Collaborate with business and functional teams to define data integration and transformation strategies aligned with enterprise objectives. Optimize data pipelines, storage, and query performance for high-volume datasets across cloud or on-prem environments. Implement robust data quality, validation, and monitoring frameworks to ensure accuracy and reliability. Develop and automate data workflows, versioning, and deployment pipelines to streamline data operations. Support the deployment, monitoring, and governance of data infrastructure and warehouse/lakehouse environments. Work closely with Data Architects to establish best practices for data modeling, warehousing, and lineage tracking. Enable incremental data processing (CDC) and efficient handling of batch and near real-time data pipelines. Collaborate with analytics teams to enable insightful visualization and reporting solutions. Prepare and maintain technical documentation for pipelines, transformations, and data flows. Collaborate with business stakeholders to understand requirements and ensure alignment with business goals.Qualification and Experience: BE/B.Tech / Science Graduate in Computer Science, Information Technology, or related field. Strong foundation in data modeling, performance tuning, and ETL/ELT orchestration. Experience working with enterprise data sources such as SAP ECC / S/4HANA, SQL/Oracle databases, and file-based data (Excel, CSV, Parquet, JSON). Experience with one or more cloud platforms such as Azure, AWS, or GCP. Familiarity with tools such as Microsoft Fabric, Databricks, Power BI, or equivalent tools is an advantage. Experience in data migration or transformation projects is a strong advantage. Understanding of data governance, data quality, and metadata/lineage concepts is a plus.Key Deliverables: ETL/ELT pipelines for all assigned data sources Data ingestion from SAP, databases, files, and APIs Bronze/Silver/Gold data layer implementation Data quality checks and monitoring setup Optimized and scalable data pipelines Develop the Power BI dashboard. Technical documentation and runbooks

We are seeking a skilled Mid-Level Data Engineer to design, build, and optimize scalable data pipelines and architectures. The ideal candidate will have strong experience in data processing, ETL development, and cloud-based data platforms, supporting data-driven decision-making across the organization.

We are looking for an AI Engineer with 5 to 11 years of experience to join our team in Thiruvananthapuram. In this full-time role, you will work on developing innovative AI solutions and applications.**Key Responsibilities:**- **Develop AI Models:** Create and train advanced machine learning models to solve complex problems, ensuring they are accurate and efficient.- **Conduct Research:** Stay updated on the latest AI trends and research findings to implement cutting-edge techniques into projects.- **Collaborate with Teams:** Work closely with data scientists and software engineers to integrate AI features into existing applications and systems.- **Optimize Algorithms:** Fine-tune algorithms for performance improvements, ensuring they meet project requirements and consumer needs.- **Debug and Test:** Perform thorough testing and debugging of AI systems to identify issues and enhance functionality.- **Document Solutions:** Maintain clear documentation of algorithms and processes to facilitate knowledge sharing and onboarding of new team members.**Required Skills and Expectations:**The ideal candidate should have a strong foundation in machine learning, deep learning, and data analytics. Proficiency in programming languages such as Python or Java is essential, along with experience in frameworks like TensorFlow or PyTorch. You should possess excellent problem-solving skills and the ability to work collaboratively in a team. Strong communication skills are necessary to explain complex concepts to non-technical stakeholders and to contribute effectively to discussions. A degree in Computer Science, Engineering, or a related field is preferred.

Key ResponsibilitiesTechnical Documentation: Maintaining project blueprints, technical manuals, and compliance records.Progress Monitoring: Tracking the project timeline against the master schedule (using tools like MS Project or Excel) and highlighting potential delays.Resource Coordination: Ensuring that parts, tools, and materials arrive at the site or production line exactly when the team needs them.Quality Assurance: Assisting in site visits or shop floor inspections to ensure that work is being done according to the technical specifications.Communication Bridge: Facilitating information flow between departments-for example, explaining design changes to the production team or clarifying technical issues to the client.Risk Mitigation: Identifying small technical issues early on before they snowball into costly project delays.

Key ResponsibilitiesTechnical Design & Drafting: Using CAD software to create layouts for Medical Gas Pipeline Systems (MGPS), nurse call systems, or ambulance interior configurations.Site Supervision: Managing on-site installation teams (technicians, welders, and fitters) to ensure work meets ISO and AIS-125 safety standards.Project Scheduling: Developing timelines (Gantt charts) to ensure hospital wings or ambulance fleets are delivered on time.Quality Control & Testing: Conducting pressure tests on gas lines and load-testing stretchers/cots before handover to the client.Procurement Coordination: Working with the supply chain to ensure specialized medical-grade materials (like degreased copper pipes or reinforced alloys) are available on-site.Regulatory Compliance: Ensuring every aspect of the project adheres to healthcare building codes and medical device regulations.

Key Responsibilities:Build and manage data pipelines and ETL processesWork with large datasets using tools like Spark, Hadoop, or SQLEnsure data quality and performance optimizationRequirements:Experience in Python/SQLHands-on with ETL tools and big data technologiesUnderstanding of data warehousing conceptsBrief Summary:Develops scalable data systems to support analytics and business insights.

Key Responsibilities: Cloud EngineerDesign, implement, and manage CI/CD pipelinesAutomate deployments and infrastructure using tools like Terraform/AnsibleMonitor system performance and ensure high availabilityRequirements:Experience with AWS/AzureKnowledge of Docker, Kubernetes, and scriptingStrong problem-solving skillsBrief Summary:Responsible for automating and optimizing cloud infrastructure and deployment processes.

Key Responsibilities Requirement Analysis: Collaborate with stakeholders to understand businessrequirements and data sources, and define the architecture and design of dataengineering models to meet these requirements. Architecture Design: Design scalable, reliable, and efficient data engineering models,including algorithms, data pipelines, and data processing systems, to support businessrequirements and quantitative analysis. Technology Selection: Evaluate using POCs and recommend appropriate technologies,frameworks, and tools for building and managing data engineering models, consideringfactors like performance, scalability, and cost-effectiveness. Data Processing: Develop and implement data processing logic, including data cleansing,transformation, and aggregation, using technologies such as AWS Glue, Batch, Lambda. Quantitative Analysis: Collaborate with data scientists and analysts to develop algorithmsand models for quantitative analysis, using techniques such as regression analysis,clustering, and predictive modeling. Model Evaluation: Evaluate the performance of data engineering models using metricsand validation techniques, and iterate on models to improve their accuracy andeffectiveness. Data Visualization: Create visualizations of data and model outputs to communicateinsights and findings to stakeholders.Data Engineering: Understanding of Data engineering principles and practices, includingdata ingestion, processing, transformation, and storage, using tools and technologiessuch as AWS Glue, Batch, Lambda. Quantitative Analysis: Proficiency in quantitative analysis techniques, including statisticalmodeling, machine learning, and data mining, with experience in implementingalgorithms for regression analysis, clustering, classification, and predictive modeling. Programming Languages: Proficiency in programming languages commonly used in dataengineering and quantitative analysis, such as Python, R, Java, or Scala,