Senior Software Engineer - Hadoop Infrastructure
New York, NY
Posted Jun 29, 2022 - Requisition No. 103697
Bloomberg runs on data! It's our business and our product. Financial institutions need timely, accurate data to capture opportunities and evaluate risk in fast-moving markets, from the biggest banks to elite hedge funds.
Bloomberg's application teams face complex challenges - large-scale data storage, low-latency retrievals, high-volume requests, and high availability over a distributed compute and storage environment. Hadoop Infrastructure's mission is to provide a multi-tenant, observable, and highly available infrastructure backed by the open-source Apache Hadoop platform (HDFS, HBase, Hive, Oozie, YARN/Spark, etc.) to support large-scale data applications across Bloomberg. The Hadoop Infrastructure team manages clusters that handle 10s of PB of storage spread across 1000s of servers running 100s of billions of requests per day and running 10s of thousands of jobs hitting 100s of thousands of tables every day. We also provide best practices and subject matter expertise on Hadoop services to applications across a variety of product domains at Bloomberg.
Who are you?
- You are a talented and motivated engineer interested in building and managing large-scale distributed systems, looking for a tight-knit, collaborative team
- You are an innovative problem solver who enjoys working in multiple roles and thrives in a fast-paced environment
- You want to make a significant impact and contribute to open-source software
- Provide and improve our capabilities to migrate and support a massive data and compute footprint (10s of PB) to newer versions
- Improve our tenants' user experience when securely interacting with powerful underlying infrastructure frameworks
- Understand and improve the usability, reliability, and scalability of open-source Apache Hadoop services to optimize for the needs of Bloomberg application teams
- Working knowledge of web frameworks (React/Angular)
- Working knowledge of modern development methodologies and tools (Jenkins/Maven/Jira)
- Solid understanding of algorithms and data structures, knowledge of object-oriented design, and multi-threaded programming
- Strong problem solving and communication skills
- Experience with distributed systems architecture and system design
- Knowledge of scripting languages such as Python
- Familiarity with Kubernetes and containers
- Exposure to modern security patterns
- Experience working with open-source software/community
You'll have the opportunity to:
- Advance how tenants from multiple product domains leverage Hadoop Infrastructure services to meet their goals
You'll need to have:
- 5+ years of experience in Java development and associated technologies (Eg: Spring / JMX / JDBC)
We'd love to see:
Bloomberg is an equal opportunity employer and we value diversity at our company. We do not discriminate on the basis of age, ancestry, color, gender identity or expression, genetic predisposition or carrier status, marital status, national or ethnic origin, race, religion or belief, sex, sexual orientation, sexual and other reproductive health decisions, parental or caring status, physical or mental disability, pregnancy or maternity/parental leave, protected veteran status, status as a victim of domestic violence, or any other classification protected by applicable law.
Bloomberg is a disability inclusive employer. Please let us know if you require any reasonable adjustments to be made for the recruitment process. If you would prefer to discuss this confidentially, please email firstname.lastname@example.org