Many of the most prevalent applications in Bloomberg need real-time data analysis and business intelligence. Instead of using off-the-shelf applications, we use Hadoop and its ecosystem to provide a large-scale data platform with low-latency SLAs and large storage capabilities. This platform has revolutionized the way we manage and analyze data in a distributed environment.

We design software and hardware systems to support low-latency/high volume requests, security, fault tolerance/high availability and easy customization. Our Hadoop Infrastructure Platform is built to fully automate deployment and operations using Chef; developed and open sourced at https://github.com/bloomberg/chef-bach. With hundreds of applications depending on our platform, we are looking to grow our Hadoop Infrastructure team. That's where you come in.

We'll trust you to:

• Evaluate Hadoop projects across the ecosystem and extend and deploy them to exacting standards (high availability, big data clusters, elastic load tolerance)
• Develop automation, installation and monitoring of Hadoop ecosystem components in our open source infrastructure stack, specifically HBase, HDFS, Map/Reduce, Yarn, Oozie, Pig, Hive, Tez, Spark and Kafka
• Dig deep into performance, scalability, capacity and reliability problems to resolve them
• Create application patterns for integrating internal application teams and external vendors into our infrastructure
• Troubleshoot and debug Hadoop ecosystem run-time issues
• Provide developer and operations documentation to educate peer teams

You'll need to have most of the following:

• Experience building out and scaling a Hadoop-based or UNIX-hosted database infrastructure for an enterprise
• 2+ years of experience with Hadoop infrastructure or a strong and diverse background of distributed cluster management and operations experience
• Experience writing software in a continuous build and automated deployment environment

We'd love to see:

• 2+ years of DevOps or System Administration experience using Chef/Puppet/Ansible for system configuration, or quality shell scripting for systems management (error handling, idempotency, configuration management)
• In-depth knowledge of low-level Linux, UNIX networking and C system calls for high performance computing
• Experience with Java, Python or Ruby development (including testing with standard test frameworks and dependency management systems, knowledge of Java garbage collection fundamentals)
• Experience or exposure to the open source community (a well-curated blog, upstream accepted contribution or community presence)

We want to work with others who are passionate about community-driven development both within the company and with the wider open source community. If this sounds like you, submit an application, and learn more about the work we do from Clay and Amit's interview at ChefConf2015: https://www.youtube.com/watch?v=LnMCFxXgDEw