Senior Software Engineer - Ticker Plant
New York, NY
Posted Sep 16, 2022 - Requisition No. 102406
The Ticker Plant system is the beating heart of Bloomberg’s market data infrastructure and is responsible for the storage and distribution of Market Data -- trades, quotes, and other pricing data -- collected from every type of trading venue around the world on every possible kind of financial instrument. The system processes daily volumes of several hundreds of billions of events across 100 million instruments, aggregating, enriching, storing and disseminating these events to the user-facing Bloomberg Terminal and through a range of Bloomberg Enterprise solutions that form the backbone of today’s financial institutions. Read our blog to know more! (https://www.bloomberg.com/company/stories/how-bloomberg-handles-a-massive-wave-of-real-time-market-data-in-microseconds/)
Ticker Plant is organized into three main functional areas.
Market Data Distribution handles for custom high performance middleware that enable transmission of market data events in and out of the Ticker Plant. Our challenges are to provide a platform that can ingest and redistribute data successfully reliably (without loss), resiliently (fault-tolerant), at scale (hundreds of hosts, billions of ticks), globally and regionally, and all in real-time with minimal latency.
Data Content develops software components and systems to transform, enrich, store and publish normalized market data in real-time. In addition, the team develops tools and services that aid in automated testing and verification of the functionality provided by the Ticker Plant to ensure stability of the system.
Database Infrastructure develops a high performance time-series database that supports low latency data storage and retrieval for high-volume applications. The storage engine hosts several petabytes of data, across hundreds of millions of instruments. The query engine handles ~80 billion queries daily ranging from interactive queries to batch queries at very high throughputs of up to 100K QPS.
Being the core of the market data infrastructure, the Ticker Plant system by necessity is scalable, distributed, highly performant, and stable. To be highly performant, most of the critical software is written in C++. Python is also extensively used to build auxiliary services around the critically important software and to enable automation. In addition, we use a wide range of engineering strategies to solve problems ranging from using off the shelf software (like kafka) to implementing our own algorithms/solutions (e.g custom built database), as appropriate for the problem at hand.
What we do:
We have some unique challenges where we not only have to optimize for scale (billions of data points, hundreds of hosts) but also for very high throughput of updates and queries, all at the same time. As market data volume increases each year, these challenges get harder to meet with the current infrastructure and so we are re-architecting and re-writing the core parts of our system to make the system simpler and easier to scale. Some examples are:
- Consolidate existing data distribution mechanisms onto a fast, loss-less, resilient market data distribution infrastructure.
- Improve scalability of the Ticker Plant by separating data processing and data storage layers, making them independently scalable and ultimately moving storage and query layers to cloud infrastructure.
- Make efficient use of resources within Ticker Plant by dynamically balancing load in response to market data spikes.
In addition to re-architecture initiatives, we also work on projects to enhance the functionality and stability of Ticker Plant. Some examples of are:
- Modernize internal data representation and associated processing to seamlessly support increasing price and time precision, increasing volumes, and emergence of new products (e.g. cryptocurrencies). .
- Create tools and frameworks for real-time production data stream introspection to ensure data quality.
- Build API abstractions and self-service solutions to Ticker Plant data processing configuration layer in order to enable faster feature development and improve stability. We are looking for a motivated engineer who is excited by these types of projects/problems to help build the next generation Ticker Plant!
What is in it for you:
- Hands-on experience working on large scale distributed systems
- Join a group in the early stages of a major system re-architecture
- Work on core computer science problems such as efficient data storage, data discoverability, data transmission and distribution
- Work with high-caliber individuals and experienced teams of engineers and management
- Be challenged with trying to make our system more modular and horizontally scalable
We’ll trust you to:
- Take full ownership of technical solutions/features
- Design, implement, and own critical applications/components of our infrastructure stack
- Write quality code with automated tests
- Collaborate with other engineers in the team and with our CTO office & business partners
You’ll need to have:
- 3+ years experience programming in C++/Python or other object oriented languages
- BS, MS, PhD in Computer Science, Engineering or related technology field or relevant boot camp experience
- Deep desire to learn and understand software infrastructure
We are extremely proud of our diverse and open culture, and value diversity of thought and perspective in every form. We're looking for engineers with a real passion for writing reusable, efficient solutions to complex problems, who can adapt to an ever-changing market landscape, and who can collaborate and work effectively on small teams to develop software that impacts thousands of power users around the world. If this sounds like you, please apply below!
To find out more about Engineering at Bloomberg, check out www.bloomberg.com/careers/technology/engineering/.
Bloomberg is an equal opportunity employer and we value diversity at our company. We do not discriminate on the basis of age, ancestry, color, gender identity or expression, genetic predisposition or carrier status, marital status, national or ethnic origin, race, religion or belief, sex, sexual orientation, sexual and other reproductive health decisions, parental or caring status, physical or mental disability, pregnancy or maternity/parental leave, protected veteran status, status as a victim of domestic violence, or any other classification protected by applicable law.
Bloomberg is a disability inclusive employer. Please let us know if you require any reasonable adjustments to be made for the recruitment process. If you would prefer to discuss this confidentially, please email [select from below per region]"
- firstname.lastname@example.org / or our Diversity Partner EmployAbility email@example.com.