Software Engineer - Big Data

Job description

About project

The Octoflow project is a tool for mapping and certifying network flows. Octoflow collects information via APIs from data sources (IPAM, DNS, etc.) to perform ingestion and mapping of flows. It will also be responsible for adding business-related features around the data.

JD

Develop code and test case scenarios by applying relevant software craftsmanship principles and meet the acceptance criteria. Complete the assigned learning path and contribute to daily meetings. Deliver on all aspects of Software Development Lifecyle (SDLC) in-line with Agile and IT craftsmanship principles. Take part in team ceremonies be it agile practices or chapter meetings. Deliver high-quality clean code and design that can be re-used.

Actively, work with other Development teams to define and implement API's and rules for data access. Perform bug-free release validations and produce test and defect reports. Contribute to developing scripts, configuring quality and automating framework usage. Run and maintain test suites with the guidance of seniors. Support existing data models, data dictionary, data pipeline standards, storage of source, process and consumer metadata.

Technical Skills

Languages: Scala, Shell Data Processing: Spark, SQL Databases: PostgreSQL, Hive, Starburst (query tools)

Big Data

Hadoop, Kafka CI/CD: Jenkins, Ansible, Control-M Methodology: Agile, backlog management, collaborative work Soft Skills Rigor and adherence to group standards (security, quality, compliance) Ability to work in an international environment Proactive in optimizing development Hands on experience 2-3 years is required.