Senior Software Engineer - Distributed Build
New York, NY
Posted Oct 3, 2018 - Requisition No. 71077
The Distributed Build team is responsible for building and maintaining Bloomberg’s build tooling, build farm and supporting services and workflows. We leverage open standards and APIs to distribute the compilation of packages, libraries and applications across multiple machines and platforms simultaneously. C/C++ is continuously integrated and deployed daily alongside Python, Haskell, and other third party components integrated into our environment.
Our current priorities are the development and integration of BuildGrid into Bloomberg’s developer workflow to enable accelerated build and test for our developers around the world.
We work closely with the Open Source community on the development of BuildGrid which is based upon open APIs for the remote execution and caching of builds. We also maintain and enhance our existing build infrastructure that supports a weekly production load of 100,000s of package builds.
Going forward we are looking to:
- Elastically scale our build resources with demand using cloud container technologies
- Provide deep instrumentation and analytics around how our developers build their software
- Provide intelligent job distribution over the cluster based on workload and job size to improve build times and performance
- Improve caching performance through tiering via S3, Redis and Memcached as well as intelligent policies
The infrastructure is built using python 3, C++, FUSE, gRPC and protobufs. Each component is independently interoperable and open sourced on GitLab.
We will trust you to:
- Participate in community development and discussions
- Develop new features in the open for BuildGrid, BuildBox and BuildStream across a variety of languages and platforms
- Collaborate with our engineers to understand their workflows, constraints and priorities to deliver valuable build experience and performance improvements
- Provide monitoring and scaling infrastructure for our platform to ensure high uptime and performance
You will need to have:
- A proficiency in the design and build of clean, efficient distributed python and C++ infrastructure
- A familiarity with scripting languages such as shell and perl
- An interest in and understanding of the wide variety of programming languages that will be built using the platform
- BA, BS, MS, PhD in Computer Science, Engineering or related technology field
We would love to see:
- Familiarity with build systems such as Bazel, CMake, Pants or Buck
- An understanding of cloud and container technologies such as Kubernetes, Docker and Prometheus