Senior Kubernetes Platform Engineer- Enterprise Data Science Infrastructure

Careers at Bloomberg

Back to Search

New York, NY

Posted May 4, 2021 - Requisition No. 90535

Bloomberg runs on data. It's our business and our product. From the biggest banks to the smallest hedge funds, financial institutions need timely, accurate data to capture opportunities and evaluate risk in fast-moving markets. With petabytes of data available, a platform to transform and analyze the data is critical to our success.

Bloomberg’s quant platform, BQuant, enables users to develop sophisticated financial applications on top of Bloomberg’s data and services. Customers are able to programmatically access Bloomberg’s data; build and analyze factors; screen securities for investable ideas; backtest custom trading strategies; and much much more, all through BQuant’s unique portal. Customers can deploy the product pre-configured on their own premises, or buy it as a service from Bloomberg. Our mission: to democratize the industry by providing every participant with advanced quantitative tools that might only be available to certain players.

The BQuant platform has further evolved to also support data-driven science, machine learning, and business analytics in a cloud-native way. Customers are enabled to integrate data science and distributed analytics into their quantitative workflows. To support this, the platform works to provide scalable compute, specialized hardware, and first-class support for a variety of workloads such as Spark, Tensorflow, and PyTorch. The platform was developed to provide a standard set of tooling for addressing the Model Development Life Cycle from experimentation and training to inference. The platform runs on top of Kubernetes thereby leveraging containerization, container orchestration, and cloud architecture on an infrastructure stack composed of open source technology.
   
The platform is poised for enormous user growth this year and has an exciting roadmap that requires scalable infrastructure as a foundation. As a member of the multidisciplinary Enterprise Data Science Infrastructure team, you’ll be tasked with hardening and scaling the underlying Kubernetes stack upon which the platform runs and committing to the team’s mission statement to design composable infrastructure substrates that can be run on a variety of deployment environments. 
  
Our team makes extensive use of open-source (e.g. Kubernetes, Tensorflow, Spark, and Jupyter) and is deeply involved in a number of communities. As part of that, we regularly upstream features we develop, present at conferences, and collaborate with our peers in the industry. We are contributors to the Kubeflow project as well as founding members of the KFServing subproject to standardize ML Inference within the Kubernetes ecosystem. For Spark, we have implemented a scalable and resilient external shuffle service for dynamic resource allocation, a pluggable interface for secure worker creation, and a token renewal service that handles privacy and security across jobs, all in line with our effort to improve security and elasticity for Spark on Kubernetes. Open source is at the heart of our team. It's not just something we do in our free time, it is how we work.

 

We’ll trust you to:

  • Design and develop Kubernetes-based products running on customer-managed clusters and as a SaaS offering
  • Design Kubernetes operators and controllers
  • Develop alerting, monitoring and remediation automation in a large scale distributed environment
  • Design distributed systems and develop solutions for problems such as elastic load distribution, effective resource management, and guaranteed scheduling
  • Regularly present your work to peers, senior stakeholders (including our CTO), and clients

We’ll expect you to:

  • Provide reliable, scalable, and composable infrastructure
  • Collaborate across data science teams on proper use/integration of our platform
  • Tinker at a low level and communicate your work at a high level
  • Research, architect and drive complex technical solutions, consisting of multiple technologies
  • Mentor junior engineers and be a strong engineering voice alongside other leaders through advising and driving the platform’s technical vision and strategy.

You’ll need to have: 

  • 3+ years experience developing infrastructure solutions, preferably within a Kubernetes infrastructure, middle-ware, or application team
  • Experience with distributed systems eg. Kubernetes, Spark, MPI, TF, PyTorch, Kafka
  • Proficiency in one or more languages (Python, Go, C++, Java, Scala, or JavaScript) and willingness to learn more as needed
  • B.S., M.S., or PhD in Computer Science, Computer Engineering, or equivalent practical experience

We’d love to see: 

  • Experience with Kubebuilder and Kubernetes operator-based frameworks
  • Open source involvement such as a well-curated blog, accepted contribution, or community presence
  • Experience with cloud providers such as AWS, GCP, or Azure
  • Linux systems experience (Network, OS, Filesystems)
  • Experience with service-mesh frameworks like Istio or serverless frameworks like Knative
  • Ability to identify and perform OS and hardware-level optimizations
  • Experience working with GPU compute software and hardware
  • Experience working with authentication & authorization systems such as Kerberos and LDAP

If this sounds like you, apply! You can also learn more about our work using the links below:

Bloomberg is an equal opportunities employer, and we value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Similar jobs

    The Bloomberg Talent Network

    Stay connected with us and be among the first to learn about new job opportunities. We’ll use the information you provide to help us get in touch with you to align your expertise with our opportunities and better direct our conversations.

    CONNECT WITH US