Senior Spark Engineer-DataHub Infrastructure
New York, NY
Posted Oct 20, 2021 - Requisition No. 96668
The DataHub Engineering team provides a distributed platform for hosting datasets, complete with managed data stores, search, discovery, batch analytics and real-time stream processing capabilities. Our goals: ensure high quality content, which is indispensable to financial markets, is cataloged, standardized, discoverable, distributed and accessible in one place.
Who are you:
The ideal candidate is an innovative problem solver who enjoys working in multiple roles and thrives in a fast-paced, collaborative environment. You are curious, kind, continually learning, and happy to share what you learn. You enjoy pursuing complex issues to their root cause. You love large scale distributed data systems, performance at scale, engineering for reliability.
What's in it for you:
As a senior infrastructure engineer in the DataHub team, you will build an Apache Spark based batch data infrastructure that scales to address needs of all financial datasets at Bloomberg. You will engineer for reliability, scale, performance, efficiency, observability and ease of use. This is an opportunity to engineer systems on a massive scale, and to gain valuable experience in distributed computing. Your impact will be at the heart of financial data at the most influential financial data company in the world. You'll be surrounded by people who are passionate about distributed computing, and believe that premier service is crucial to customer success. You'll get the chance to work with engineering teams across Bloomberg as well as opportunities to be a committer to open source.
You'll need to have:
- 5+ years of systems programming experience in Java/Scala or C/C++
- Experience designing scalable and reliable high throughput systems
- Proficiency in Apache Spark, Linux and performance optimizations is a must
- Knowledge of using and debugging a cluster manager YARN/Kubernetes/Mesos for Spark
- Good understanding of distributed systems, concurrency, object stores, debugging
- Experience in software instrumentation for monitoring and observability
- BA, BS, MS, PhD in Computer Science, Engineering or related technology field
We'd love to see:
- Kubernetes operator experience
- Pride of ownership and quality in your work
- A team player with a customer-focus
- Any of your contributions to the open-source community
About Us: Meet the DataHub Engineering Team https://www.techatbloomberg.com/blog/meet-the-team-datahub-engineering/