Lead Software Engineer - DevOps

Job description

Experience

10+ Years Location: Bangalore Role: Full-time / Technical Lead

Role Summary

We are seeking a highly capable Technical Lead with deep expertise in Big Data (Spark, Hadoop, Hive), Kubernetes platforms, Azure cloud, PostgreSQL, CI/CD (GitHub Actions/Jenkins), and DevOps automation. The ideal candidate will be a strong hands-on engineer who also brings mature leadership capabilities, sound architectural judgment, and exceptional software craftsmanship practices.

Key Responsibilities

• Software Engineering & Craftsmanship
• Develop high‑quality, maintainable code using Software Craftsmanship principles such as: Test-Driven Development (TDD) Continuous Integration / Continuous Delivery / Continuous Deployment
• Clean code, refactoring, SOLID principles
• Participate actively in design reviews and code reviews to ensure engineering excellence.
• Implement quick POCs using the latest tech stack to validate ideas and technical feasibility. Apply design patterns and engineering best practices for scalable and maintainable solutions. Collaborate in all phases of the SDLC , with strong understanding of Agile/Scrum or continuous delivery environments.
• Big Data Engineering & Distributed Systems Lead development of large-scale ETL/ELT pipelines using Apache Spark (Java/Scala) . Oversee data processing frameworks in Hadoop and Hive , especially within Azure-based ecosystems . Optimize Spark/Hive workloads for cost, performance, and reliability.
• Cloud, Kubernetes & Infrastructure Engineering Lead architecture and operations of distributed applications running on: Kubernetes (On-Prem + AKS) Docker Drive adoption of Helm , container standards, and cluster best practices. Build and maintain Azure infrastructure using Terraform (IaC) with reusable, scalable modules. Ensure end-to-end observability of clusters using Prometheus, ELK, APM tools, and custom Python scripts.
• DevOps Engineering & CI/CD Leadership Define and own the platform CI/CD strategy using: GitHub Actions Jenkins Terraform automation Build standardized CI/CD frameworks for 50+ microservices, leveraging Helm and automated build workflows.

Automate DNS creation, certificate management, APM integration, user management, Vault policy creation, and repository migrations (200+ repos). Champion an automation-first mindset across the engineering and SRE teams. Coordinate with developers and testers for smooth deployment and production readiness.

Database Engineering Provide leadership on PostgreSQL cluster management

• HA setup Performance tuning Failover mechanisms Monitoring dashboards Automate PostgreSQL operations using Python and infrastructure scripts.
• Observability, Stability & Support Ensure reliability and availability of services across stacks (Java, Angular, .NET). Implement monitoring and alerting using Prometheus, Grafana, ELK, Elastic APM. Oversee production stability through proactive dashboards and automated health checks. Provide guidance for L2/L3 support and participate in incident review and resolution.
• Leadership & Collaboration Lead a cross-functional team of developers, DevOps engineers, and data engineers. Translate business requirements into technical designs and actionable backlogs. Mentor team members on coding practices, DevOps processes, and cloud-native patterns.

Collaborate with distributed teams and communicate effectively with technical and non-technical stakeholders.

Required Skills & Qualifications

10+ years of experience in software engineering / data engineering / DevOps roles.

Strong expertise in: Apache Spark (Java/Scala) Hadoop, Hive on Azure Kubernetes, Docker Azure Kubernetes Service (AKS) Ansible Jenkins & GitHub Actions PostgreSQL (HA + automation) Strong understanding of CI/CD, SDLC, Agile, and continuous delivery. Good understanding of networking fundamentals , DNS, LBs, ingress controllers. Experience working on production support and high-availability systems. Soft Skills Strong leadership and mentoring capabilities.

Excellent communication skills with distributed teams. Ability to analyze complex problems and deliver quick, practical solutions. High sense of ownership and accountability. Nice-to-Have Experience with Azure Data Lake, Databricks, or Synapse. Exposure to microservices architecture and domain-driven design.