AWS Spark Scala - Data Engineer

Job description

Job Description

TCS Inviting Application For -- AWS Spark Scala - Data Engineer EXP Range:6-8 Yrs Location: Bangalore Desired Competencies (Technical/Behavioral Competency) Must-Have

• Strong experience in programming languages such as Python, Java, or Scala for data manipulation and engineering tasks.
• Expertise in SQL and NoSQL databases
• Hands-on experience with big data technologies like Hadoop, Spark, Kafka, and Hive to handle large-scale data processing and real-time data streams.
• In-depth knowledge of data warehousing solutions such as Amazon Redshift, Google BigQuery, and Snowflake for building and managing data warehouses.
• Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using tools like Apache NiFi, Talend, or Informatica.

Good-to-Have

• Familiarity with cloud platforms like AWS, Azure, or Google Cloud for deploying, managing, and scaling data infrastructure and services.
• * Strong understanding of data modeling concepts and techniques to create efficient and scalable data models.
• Experience with version control systems such as Git for code management and collaboration.
• Knowledge of data governance, data quality standards, and data security practices to ensure compliance and protection of sensitive information SN Responsibility of / Expectations from the Role 1
• Design, develop, and maintain scalable data pip elines for extracting, transforming, and loading data from various sources to ensure seamless data flow and accessibility. 2
• Collaborate with cross-functional teams to integrate data from multiple disparate sources, ensuring consistency, accuracy, and reliability of data. 3
• Optimize data processing workflows and storage solutions for performance, scalability, and cost-efficiency 4
• Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data throughout the data lifecycle. 5 <td valign="top" style="width:468.75pt;border-top:none;border-left: none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt; mso-border-top-alt:solid windowtext .75pt;mso-border-left-alt:solid windowtext .75pt; mso-border-alt:solid windowtex

Qualifications

BACHELOR OF ENGINEERING