Posted May 15, 2017 - Requisition No. 58557
Financial institutions thrive on data and the Bloomberg Enterprise Content business satisfies their needs. Enterprise Content is a fast growing business that provides clients with rich programmatic access to the best financial data and services in the industry such as Reference, Pricing, Corporate Actions and Regulatory content sets.
Our team of 8 is responsible for storing detailed usage metrics on every piece of data taken down by our clients. We just don't stop there - our team develops applications and services to analyze the data and apply various pricing models to be able to bill our clients. Our pricing models involve optimizing usage patterns across multiple access points that our clients use to access financial data and require us to churn through 2 years of data (trillions of data points) in 2 hours.
Let's talk technology. To build a scalable ingestion pipeline for our usage metric data, we have picked Apache Kafka, one of the fastest distributed streaming platforms. Now we need to store that data - we use Hadoop Distributed File System (HDFS) and, for some use cases, HBase. The data isn’t very useful unless we can analyze it - that’s where Apache Spark & MapReduce come in. We use other tools in the Hortonworks Hadoop distribution like Oozie for scheduling our billing jobs, and are investing in Jupyter Notebooks to give tech-savvy users more flexible and sophisticated ways to analyze our data.
As a part of building new products, our business users have to design new pricing models to offer products to our clients. Time to market is absolutely one of our key metrics. Our goal is to empower users to be able to configure various attributes that make up the pricing models and reduce the time it takes on the engineering side to build them.
These are big goals and we’re just getting started. We’re looking for bright, knowledgeable, driven people to help us make key technical decisions on the architecture, processes, best practices, and direction of this initiative. You will be responsible for the research, development, and stability of this critical platform and its impact on one of the highest revenue businesses within Bloomberg.