Big Data Pipeline Engineer - Network Infrastructure
New York, NY
Posted Mar 3, 2017 - Requisition No. 57310
In the financial world, time is money. And Bloomberg is the fastest provider of market data, financial news and electronic trades in an industry where every second counts. We, the Network Infrastructure team, develop systems that manage the distribution of data across Bloomberg's network for various products. Our key focus when building our high-quality distribution algorithms is the real-time monitoring of changing market conditions, loads on distribution servers and network outages.
As part of our team, you will build monitoring modules in C++ which are deployed across the entire network to collect real-time performance metrics 24/7. These metrics are then streamed to centralized big data infrastructure in our data centers for real-time analysis. You will have an opportunity to work on a challenging and always-on data pipeline: from remote C++ modules to centralized Kafka/Spark clusters and datastores.
We are self-motivated engineers who like to research, learn, and apply current technology to build new products that anticipate our customers' needs. We work in a collaborative and Agile environment, which allows us to deploy new features to production every day. As part of a rapidly growing team, you’ll have the opportunity to work on challenging problems as well as key projects and initiatives.
We’ll trust you to:
- Partner with various product and engineering teams across the company
- Be self-motivated and thrive in a fast-paced and collaborative environment
- Take full ownership of the full software development life-cycle, including researching the infrastructure needs of various Bloomberg products
You need to have:
- 3+ years of experience programming in C++ or Java and familiarity with Scala
- Experience developing multi-threaded and client-server applications
- Experience building large-scale, real-time distributed software systems in professional environment
- An agile attitude to develop new features incrementally and regularly adapt to changing customer needs
We’d love to see:
- Familiarity with Kafka/MQ and Spark streaming
- Knowledge of relational databases (MSSQL, Oracle, MySQL) or NoSQL (Cassandra/MongoDB)
- Familiarity with data science tools (Python, R, ML-lib) or search technologies (Splunk, Solr)