Senior Big Data Engineer - Latency Monitoring System

Careers at Bloomberg

New York

Posted Mar 13, 2017 - Requisition No. 57497

You're a software engineer who has develops and integrates large systems with many parts, and can distribute data between those parts. You're interested in Big Data sets, performance analysis, and statistics. You enjoy working with low latency, high throughput systems, and are comfortable thinking about the distance between two cities in both miles and milliseconds. You're interested in using open-source technologies, but if it doesn't exist you're happy to build it.

If this sounds like you, then consider working on the Latency Monitoring System. We're building a system from scratch to explore the latency of market data delivery on Bloomberg's global network. You'll be involved nearly from the beginning, designing a system that helps both developers and business departments understand how data flows through our system and where we can improve. We're looking for someone who can contribute to all aspects of the system, from parsing to data storage to data analysis. You'll work with a small, flexible team on identifying how applications behave under load, which applications can be improved, and where the bottlenecks are.

We'll trust you to:

  • Design and implement distributed data analytics systems, using Hadoop/Spark, Python, and C/C++
  • Manage cloud resources in order to maintain resiliency and performance
  • Effectively roll out new features using an Agile methodology
  • Work with a small team on all parts of the system, from data capture to display
  • Participate with the rest of the team in analyzing the latency data, finding bottlenecks, and proposing solutions

You need to have:

  • 2+ years experience with Hadoop and Spark
  • 2+ years experience with Openstack or Amazon EC2 (or equivalent)
  • 4+ years experience with Python
  • BS or MS in Computer Science or equivalent experience
  • Experience with GitHub and a solid understanding of core concepts with Git
  • Familiarity with Linux
  • A solid understanding of basic statistics and core computer science concepts

We'd love to see:

  • A strong understanding of distributed computing
  • Familiarity with web technologies, including NGinx, Flask, and REST APIs
  • Experience with chef, puppet or ansible
  • Familiarity with system administration tasks, such as managing services, hardware, and network configurations
  • Prior experience working with trading or market data
Similar jobs