[Skip to main content](#main-content) [OpenNash Jobs](https://jobs.opennash.com/) [Hiring report](/reports/hiring/) · [All Tata Consultancy Services jobs](https://jobs.opennash.com/companies/tata-consultancy-services/) · [Search jobs](/search/) Tata Consultancy Services · Charlotte, NC # Hadoop,Pyspark,Hive,Kafka - Senior Developer Charlotte, NC Senior individual contributor $110,000- 125,000 Software Engineering Last seen on Tata Consultancy Services’s official careers site on August 1, 2026; it may have closed since. [Apply on Tata Consultancy Services’s careers site →](https://ibegin.tcsapps.com/candidate/jobs/416948J) [OpenNash Jobs](https://jobs.opennash.com/) / [Tata Consultancy Services](https://jobs.opennash.com/companies/tata-consultancy-services/) / Hadoop,Pyspark,Hive,Kafka - Senior Developer ## Job description ### Must Have Technical/Functional Skills Primary skills PySpark, Apache Kafka, Hadoop Ecosystem, Hive, Databricks Lakehouse Architecture, Delta Lake, Bronze/Silver/Gold Data Modeling, Big Data ETL Pipeline Development, SQL, Real-time Data Ingestion Frameworks, Data Governance & Cataloging, CI/CD Tools – Git, Jenkins, Bitbucket, Workflow Orchestration, and Cloud & On-Prem Big Data Platforms. ### Experience Minimum 10+ years Roles & Responsibilities Seeking a Senior Big Data Engineer with 10–13 years of experience specializing in Hadoop, PySpark, Kafka, Hive, and strong experience designing data solutions for large-scale financial systems. In addition, the candidate must possess advanced expertise in Databricks Lakehouse architecture, particularly around Bronze/Silver/Gold layer data modeling, Delta Lake optimizations, and building reliable, scalable pipelines for regulatory, risk, trading, and analytics workloads. This role focuses on delivering highly performant, well-governed data platforms that support the bank’s mission-critical global markets functions. ### Key Responsibilities Big Data Platform Engineering - Design, develop, and optimize PySpark-based ETL pipelines running on on‑prem Hadoop clusters and cloud environments. - Build high‑volume ingestion frameworks using Kafka for real-time and near-real-time trading and market data. - Develop, tune, and manage Hadoop ecosystem components—HDFS, YARN, MapReduce, Tez, Oozie/Airflow. - Build high-performance, optimized Hive data models for regulatory reporting, trade lifecycle, and market risk processing. ### Databricks Lakehouse & Delta Framework - Architect and implement Bronze/Silver/Gold layer modeling patterns within the Databricks Lakehouse. ### Apply Delta Lake best practices including - o optimized file management o Z-Ordering - Delta Change Data Feed (CDF) o schema evolution & enforcement o ACID transaction handling - Build reusable frameworks for ingestion, cleansing, transformation, and consumption of data across Lakehouse layers. - Enable governance, lineage, and auditability using Unity Catalog or equivalent cataloging tools. Collaboration, Leadership & Delivery - Collaborate closely with quants, product owners, architects, risk tech, and business users. - Participate in agile ceremonies — sprint planning, refinement, design reviews. - Mentor junior engineers and contribute to building strong engineering practices across tech teams. ### Required Skills & Experience - 10–13 years of hands-on experience in Big Data engineering. - Expert skills in: - PySpark — dataframe optimizations, partitioning, broadcast strategies, distributed computing. - Kafka — producer/consumer design, schema registry, streaming ETLs. - Hadoop ecosystem — HDFS, YARN, MapReduce/Tez, Oozie/Airflow. - Hive — advanced query tuning, TEZ optimization, partition/bucket management. - Extensive hands-on experience with Databricks Lakehouse, including: - Bronze/Silver/Gold layer modeling - Delta Lake optimizations - Data quality frameworks on Lakehouse - Structured & unstructured data handling - Experience in Global Markets, Risk, Treasury, Trade Surveillance, or Regulatory Reporting. - Strong SQL knowledge with experience working on massive datasets (TB/PB scale). Experience with CI/CD practices — Git, Jenkins, Bitbucket, build pipelines. ### TCS Employee Benefits Summary Discretionary Annual Incentive. ### Comprehensive Medical Coverage Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans. ### Family Support Maternal & Parental Leaves. ### Insurance Options Auto & Home Insurance, Identity Theft Protection. ### Convenience & Professional Growth Commuter Benefits & Certification & Training Reimbursement. ### Time Off Vacation, Time Off, Sick Leave & Holidays. ### Legal & Financial Assistance Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing. ### Salary Range $110,000- 125,000 a year