Hirebee Resourcing Pvt Ltd

South Africa

Job Description Senior Kafka EngineerPosition: Senior Kafka EngineerExperience: 7+ YearsLocation: Bangalore / PuneOpen Positions: 1About the RoleWe are seeking a highly skilled Senior Kafka Engineer with deep expertise in Apache Kafka to design, implement, and support high-performance, scalable, and reliable messaging/streaming solutions. The ideal candidate will have extensive experience in owning end-to-end Kafka solutions, troubleshooting complex issues, and collaborating with cross-functional teams to ensure seamless data streaming architectures.Key ResponsibilitiesDesign & Architecture:Architect, implement, and optimize large-scale Kafka solutions.Define and implement best practices for Kafka cluster design, monitoring, and security.Work on high-throughput and low-latency streaming systems.Administration & Operations:Install, configure, and maintain Kafka clusters in production and non-production environments.Manage Kafka brokers, Zookeepers (or KRaft mode), schema registry, and connectors.Perform capacity planning, performance tuning, and monitoring of clusters.Troubleshooting & Support:Troubleshoot and resolve production issues related to Kafka.Implement proactive monitoring, alerting, and recovery strategies.Provide 24x7 support for business-critical streaming workloads (as required).Collaboration & Development:Partner with application developers, DevOps, and data engineering teams to integrate Kafka into enterprise solutions.Develop and manage Kafka topics, producers, consumers, and streaming pipelines.Create technical documentation and provide knowledge transfer to teams.Required Skills & ExperienceCore Expertise:7+ years of hands-on experience with Apache Kafka in production environments.Strong expertise in Kafka design, cluster administration, security, monitoring, and troubleshooting.Proven ability to design scalable, high-availability, and fault-tolerant Kafka architectures.Technical Skills:Strong knowledge of Kafka Connect, Kafka Streams, Schema Registry, and REST Proxy.Hands-on experience with Zookeeper / KRaft mode and cluster migrations.Familiarity with DevOps tools for deployment and monitoring (Prometheus, Grafana, Confluent Control Center, etc.).Proficiency in Java / Scala / Python for developing Kafka producers and consumers.Knowledge of cloud platforms (AWS/MS Azure/GCP) and container orchestration (Kubernetes, Docker) is a plus.Soft Skills:Excellent problem-solving and troubleshooting skills.Strong communication and stakeholder management abilities.Ability to work independently and collaboratively in fast-paced environments.Good to HaveExperience with Confluent Kafka Platform.Knowledge of real-time data pipelines, event-driven architecture, and microservices integration.Exposure to NoSQL databases, big data ecosystems (Hadoop, Spark, Flink).