Senior Software Engineer

Job description

You are a seasonal leader in building scalable data analytics/insights platform with PiB level data processing every day. You have a growth mindset to learn the cut-edge technology and apply it in daily work to change the status quo. You dive deep and resolve the challenging technical problem. You insist on the highest standard for system performance and reliability. You will be working on Bing mission critical analytical data and support the grounding business; processing at big data scale and need for generating real-time insights.

You will be working in a hybrid technology space that combines the best from Azure and open-source world.

• Design, build and own the optimal data processing architecture and systems for new data and analytics pipelines, dashboards and metrics leveraging Azure and open-source technologies
• Build core datasets as well as scalable and fault-tolerant pipelines
• Build data anomaly detection, data quality checks, and optimize pipelines for ideal compute and storage systems while guaranteeing high quality and SLA datasets
• Understand the data compliance and data governance
• Collaborate with cross functional partners including data scientists and product managers to design and develop end-to-end data assets that help improve our products.

Hand on at least one data process technology like Spark, Flink, and message queue technology like Kafka, EventHub Hand on data storage technology like HDFS, Azure blob etc Leverage AI technology in coding/testing/Data processing/data validation. Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or

Python OR equivalent experience. 6+ years of experience in one or more coding languages (C#, C++, Java) Familiar at least one data process technology like Spark, Flink, and message queue technology like Kafka, EventHub Have experience on cloud environment like Azure Have experience on data storage technology like HDFS, Azure blob etc Have experience in leveraging AI technology in coding or testing work.

Fluence verbal and written English communication. 6+ years' experience in backend service development, 2+ years' experience in batch or streaming data processing to process the PiB level data. Domain expert on Spark or Flink to handle PiB level data every day. Experience in data pipeline performance optimization Experience to use AI in data processing and data validation Data compliance or governance experience is a plus