Senior Software Engineer – Data Platform Business Intelligence
New York, NY
Posted Dec 30, 2022 - Requisition No. 106596
Our Team:
Bloomberg runs on data. As the Data Warehouse / Business Intelligence team for the firm, our data captures the who, what, when, where and why of everything about Bloomberg - from everything about our clients, to their use of Bloomberg products, to our ticketing systems, to information about the hardware in our data centers and client locations. We are responsible for ingesting and preparing massive amounts of data (4 Petabytes and growing) to empower the whole company to make a difference through reporting, dashboards, self-service and advanced analytics. The data is used to drive new businesses within Bloomberg by providing new insights. Our goal is to have a direct or indirect hand in every decision Bloomberg makes!
What's in it for you?
You will work with modern, open source distributed system technologies to build our next generation, self-service Data Ingestion and ELT platform. Along with supporting the exponential growth of our Data Lake, this platform will need to make it easy for stakeholders to trust our data quality, and build integrated data analytics pipelines efficiently.
Our team is constantly looking for the next big thing in Big Data technology that can help improve our analytics pipelines, improve developer productivity, and make our offerings as user-friendly as possible. You will be expected to help the team make these strategic decisions, while also being a hands-on developer designing services and applications to power the Business Intelligence Data Platform. We’ll depend on you to advise on the system design, architecture, and scalability.
We'll trust you to:
- Collaborate within an Agile, multi-disciplinary, fast-moving team
- Design, build, and manage mission critical systems for accessing and managing data within our platform, including data discovery, monitoring, metadata (lineage, history, schema), and query layers
- Build and maintain libraries and integrations for data processing systems to leverage discovery, monitoring, metadata, and access functionality
- Collaborate with data scientists, engineers, and product managers to understand the emergent workloads and needs to support the product
- Evaluate and potentially adopt the use of new and emerging data technologies
You'll need to have:
- 4+ years of experience designing, building, and supporting scalable, resilient production systems in Python, Java or similar languages
- Experience with the Hadoop Ecosystem (HDFS, Oozie, Hive, etc)
- Experience with different Data Storage technologies such as Hadoop, S3, and Relational Database Systems (PostgreSQL, MySQL, etc) with understanding of the internal design and implementation
- Experience working with structured (Parquet, Avro, Orc, Protocol Buffers) and unstructured data (CSV, JSON)
- Familiarity with Apache Spark
- A Degree in Computer Science, Engineering, Mathematics, similar field of study or equivalent work experience
We'd love to see:
- A passion for learning, standardization, style guides and process efficiency, and bias for action
- Experience working with stream processing frameworks such as Flink, Spark Streaming, Kafka
- Experience working with Kubernetes to deploy and serve mission critical systems and services
- Experience configuring and operating Presto or Trino
- Experience or interest in Data Analytics and Business Intelligence
Bloomberg is an equal opportunity employer and we value diversity at our company. We do not discriminate on the basis of age, ancestry, color, gender identity or expression, genetic predisposition or carrier status, marital status, national or ethnic origin, race, religion or belief, sex, sexual orientation, sexual and other reproductive health decisions, parental or caring status, physical or mental disability, pregnancy or maternity/parental leave, protected veteran status, status as a victim of domestic violence, or any other classification protected by applicable law.
Bloomberg is a disability inclusive employer. Please let us know if you require any reasonable adjustments to be made for the recruitment process. If you would prefer to discuss this confidentially, please email amer_recruit@bloomberg.net
Salary Range: 160,000 - 240,000 USD Annually + Benefits + Bonus