Spark Platform Engineer

Careers at Bloomberg

Back to Search

New York, NY

Posted Jan 20, 2017 - Requisition No. 56584

The Spark Platform team is building low-latency, distributed analytics infrastructure for Bloomberg based on Apache Spark.

Instead of building dozens of isolated Spark applications for individual problem domains, we are building a platform that makes it easy for teams to plug in their business logic without duplicating common functionality. This includes connecting to various datastores or real-time streams, figuring out a way to serverize Spark transforms or having to re-implement transforms such as currency conversion that are very common in financial analytics. But we can't just use Apache Spark as is. We need to enhance open source Spark to fit our low-latency, high throughput and high availability contexts. That's where you come in.

As part of our team, you'll collaborate with a number of the core Spark contributors to co-design and co-develop enhancements to Spark in areas ranging from Performance to HA.

We'll trust you to:

  • Contribute to the development of the Bloomberg Spark Platform
  • Interact with application teams to understand their specific requirements and drive the next set of features for the Spark Platform
  • Meet with key players in the open source community to motivate necessary development efforts
  • Contribute code to the Apache Spark community in areas ranging from Machine Learning to Spark HA

You'll need to have:

  • Strong programming proficiency in Java and Scala (a minimum of two years)
  • An expert-level understanding of Apache Spark - your work will involve modifying its internals

We'd love to see:

  • Experience with MLlib
  • Open source contributions of any sort
  • Professional experience developing analytics infrastructure

If this sounds like you, submit an application and watch our talk at Spark Summit East to learn more about Bloomberg's Spark Platform (https://spark-summit.org/east-2016/events/spark-at-bloomberg/).

Similar jobs

    The Bloomberg Talent Network

    Stay connected with us and be among the first to learn about new job opportunities. We’ll use the information you provide to help us get in touch with you to align your expertise with our opportunities and better direct our conversations.

    CONNECT WITH US