Data Engineer - Artificial Intelligence Group
New York, NY
Posted Jan 7, 2023 - Requisition No. 110059
The AI Group is the central engineering group responsible for driving Machine Learning (ML) adoption at Bloomberg, with over 200 researchers and engineers working together to provide clients with the best-in-class news, research, market data, and analytics using innovative machine learning technology. We directly impact a wide variety of our flagship products, including news, research, pricing, communications platforms, search and discovery tools. We work on a variety of ML fields, including natural language processing, information retrieval, time series analysis, and recommender systems.
Some projects where we are looking for experienced research engineers include: unified search, question answering, query parsing, financial instrument (e.g., fixed income) pricing, dialogue understanding. Our engineers are responsible for architecting and implementing services end-to-end, overcoming unique challenges that come with machine learning systems in the financial domain. In addition, we contribute to open source; contributions we have made and work with include Solr, Koan, KFServing, Cloud Native Buildpacks and PyTorch Lightning that directly impact our production services.
Broadly, we are looking for colleagues who are passionate about software engineering and who want to learn more about:
- Parallel and distributed systems like Kubernetes, OpenMPI, Apache Kafka,
- Applied Machine Learning frameworks like PyTorch, scikit-learn, TensorFlow, and
- Data-driven frameworks like Apache Spark, Apache Solr, Pandas, Apache Hadoop.
If all of this sounds like the projects you are passionate about and want to work on, apply! Do check out our blog at https://TechAtBloomberg.com/AI and learn more about our projects and research.
We'll trust you to:
While working on the team as a Senior Data Engineer, you will have the opportunity to enhance our data pipelines and platforms that enable machine learning tasks. You will work with application, product, platform, and infrastructure teams to extract data from Bloomberg’s vast data ecosystem and prepare it for analyzing, annotating, training, evaluating, serving, and other tasks. Typical activities include:
- Work with application and product teams to define data needs and SLAs
- Design, build, and deploy resilient, monitorable pipelines to transport and store data in various storage solutions
- Identify the appropriate storage solutions and for a given use case, understanding the tradeoffs in performance, cost, and complexity
- Model datasets with idiomatic schema design, tailored for a dataset’s intended use
- Design and implement efficient data retrieval and processing solutions suited for our ML environments
- Collaborate with platform and infrastructure teams to influence the direction and support of various managed solutions
Colleagues who excel in this role often exhibit these qualities:
- Experience with message queueing systems like Kafka
- Proven track record building ETL pipelines leveraging technologies such as Kafka Streams, Kafka Connect, Flink, or Spark Streaming
- Prior experience building or extending data lakes, using various storage, cataloging, and retrieval technologies such as S3, HDFS, Hadoop, HBase, Hive, Trino, Presto, Cassandra, Spark
- Instruments their pipelines, and is able to quickly pinpoint problems from dashboards and logs
- Proficiency in a programming language such as Scala, Java, or Python, and a willingness to learn new languages
- Familiarity with an workflow orchestration technology, such as Argo, Airflow, or Oozie
- Excellent communication skills and a willingness to collaborate with various stakeholders
You'll need to have:
- 4+ years of experience working with an object-oriented programming language (C/C++, Python, Java, etc.)
- A bachelor’s degree in computer science or a related field, and/or
- An equivalent combination of education, and/or
- Specialized training, and/or
- Related professional experience.
If this sounds like you:
Apply if you think we're a good match! We'll get in touch with you to let you kno w what the next steps are. In the meantime, check us out at http://www.techatbloomberg.com/ (http://www.techatbloomberg.com/)
Bloomberg is an equal opportunity employer and we value diversity at our company. We do not discriminate on the basis of age, ancestry, color, gender identity or expression, genetic predisposition or carrier status, marital status, national or ethnic origin, race, religion or belief, sex, sexual orientation, sexual and other reproductive health decisions, parental or caring status, physical or mental disability, pregnancy or parental leave, protected veteran status, status as a victim of domestic violence, or any other classification protected by applicable law.
Bloomberg is a disability inclusive employer. Please let us know if you require any reasonable adjustments to be made for the recruitment process. If you would prefer to discuss this confidentially, please email firstname.lastname@example.org
The referenced salary range is based on the Company's good faith belief at the time of posting. Actual compensation may vary based on factors such as geographic location, work experience, market conditions, education/training and skill level.We offer one of the most comprehensive and generous benefits plans available and offer a range of total rewards that may include merit increases, incentive compensation [Exempt roles only], paid holidays, paid time off, medical, dental, vision, short and long term disability benefits, 401(k) +match, life insurance, and various wellness programs, among others. The Company does not provide benefits directly to contingent workers/contractors and interns.
Salary Range: 160,000 - 240,000 USD Annually + Benefits + Bonus