Confluent Consultant

Job description

Job Description

• 5+ years hands-on experience with Apache Kafka / Confluent Platform
• Deep experience with Kafka Connect, Streams, ksqlDB
• Strong coding skills in Java, Scala, or Python
• Hands-on with cloud platforms (GCP/Azure) + Confluent Cloud
• Solid understanding of data serialization (Avro, Protobuf, JSON Schema)
• Knowledge of Kubernetes, Docker, and Helm
• Experience with monitoring tools (Prometheus, Grafana, Control Center)
• Monitor and troubleshoot Kafka topics, brokers, and consumers
• Support DevOps and CI/CD pipelines for Kafka-based applications
• Excellent problem-solving and communication skills Must have :
• Bachelor’s or Master’s degree in Computer Science, Information Systems, or a related field.

Architecture & Design

• Define topic strategy (naming, partitions, RF, retention/compaction), idempotency/ordering; size and configure Confluent clusters; plan DR using Cluster Linking; leverage Tiered Storage as needed.

Build & Integration

• Implement/tune producers & consumers; design Kafka Connect integrations (managed/self‑managed connectors, REST/File/JDBC/CDC); use Schema Registry for contracts and compatibility; optional ksqlDB for stream transformations.

Security & Compliance

• Enforce RBAC, API keys/service accounts, mTLS/OAuth/OIDC; manage secrets; implement encryption in transit/rest; align retention, PII handling, and audit. Support private networking (VPC peering/PrivateLink) in Confluent Cloud.

Observability & Operations

• Set up Control Center dashboards/alerts (lag, throughput, error rates, connector health); create runbooks for DLQ triage, replay, offset resets, connector/broker incidents; capacity planning and performance testing.

Delivery & Governance

• Produce HLD/LLD, participate in design reviews, release/cutover plans, DR drills, and knowledge transfer.