Data Engineer - AIOPS (Contract)
New York, NY
Posted Aug 26, 2021 - Requisition No. 96983
Who we are:
Bloomberg runs on data. It's our business and our product. From the biggest banks to the most elite hedge funds, financial institutions need timely, accurate data to capture opportunities and evaluate risk in a dynamic market. We process and distribute this data on the Bloomberg infrastructure. We strive to provide a highly stable and reliable infrastructure to provide for the needed application performance and user experience.
In order to provide the high level of service assurance, ensuring performance, predicting, detecting, and remediating problems in real time s , the network architecture team is responsible for building next generation global network infrastructure, including automation, visibility and service assurance. As part of this initiative, we are focusing on service assurance and in particular AIOPs and insights as it applies to service assurance. Our goal is to garner and process real-time telemetry data across the stack from applications, compute and network infrastructure, to gain insights, detect and root cause problems when they occur, leveraging advanced analytics machine learning techniques.
Our team and your role:
As a member of this team you will be instrumental in all aspects of building a modern data ingestion and processing pipeline for consuming different types of operational signals such as logs, metrics, distributed traces, and events.
We’ll trust you to:
- Automate operation, installation, and monitoring of system components in our infrastructure stack
- Work with the CTO Network Architecture team to understand their software and data needs, and design solutions to ingest, understand and present related data
- Gather requirements and synthesize software solutions to meet them
- Produce developer and operational documentation, and troubleshoot run-time issues
- Provide performance analysis and capacity planning for clusters
You’ll need to have:
- Strong software development skills and ability to multitask in an Agile environment
- 3+ years experience working with Apache Kafka or other message queuing systems
- 5+ years experience programming in one of the languages like Java, Scala, Python
- 3+ years experience working with Big Data technologies such as HDFS, Hadoop, Spark, Hive, HBase, Druid, etc.
- Excellent communication skills and the ability to collaborate across teams, manage competing goals and changing priorities in a fast-paced environment
- BA, BS, MS, PhD in Computer Science, Electrical Engineering or related technology field
We’d love to see:
- Experience working with/in containerized environments using Docker, Kubernetes, Swarm, Rancher, Mesos
- Experience with Data pipelines at scale.
- Experience writing parsers/schemas for semi-structured and unstructured content
- Experience with Elasticsearch, Solr, Cassandra and other database technologies
Bloomberg is an equal opportunities employer, and we value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.