You've seen the headlines. Financial firms are fined millions of dollars when just one employee does something they shouldn't. It can be as serious as insider trading or as seemingly innocent as an inappropriate instant message. Keeping track of who is saying, doing, and trading what is a daunting task for compliance officers, especially with the explosion in the volume and variety of data produced by communications platforms.

Nearly 1,000 organizations use Vault to manage, archive, and search a broad range of enterprise communications and collaboration data, including email, instant message (IM), mobile, files and documents, voice data and social media. Our multi-tenant cloud processes 300 million messages daily and stores about 90 billion archived objects.

Compliance's Data Ingestion Engineering team stands at the forefront of the large scale machinery that delivers enterprise recordkeeping compliance solution to Bloomberg's clients. Our team builds content ingestion pipelines and leverages immutable storage infrastructure over data streams from all Bloomberg's internal communications platforms - including instant messaging, emails, file sharing, trade ideas as prime examples. To efficiently process incoming data and the multiple petabytes of data in storage, our engineers rely on open source data technologies well supported by Bloomberg's Managed Services - we work with Protobuf, Kafka, Spark, and Postgres. Given the stringent demand for reliability, we have also built safeguards such as asynchronous batch reconciliations, redundant snapshots of data, and data encryption at rest. We also work closely with the Reliability Engineering team in Compliance to implement SDLC and observability best practices to ensure optimal quality

Our team is looking for software engineers who are passionate in solving data engineering problems at scale. As a team member of the Data Ingestion team, you will:

• Develop systems to fully automate data lifecycle management to meet growing business needs over richer retention policies
• Experiment with internal and remote cloud-based storage solutions and evolve our software to complement business strategies
• Work closely with our upstream data producers within the Bloomberg world to understand their recordkeeping needs and the data specifications to then help shape our team's development roadmap
• Collaborate with our data consumers to understand their specific data access needs then develop robust APIs and push-based delivery mechanism that meets their needs within the expected SLA

You'll need to have:

• 4+ years of experience working with Java, C++ or Scala
• 2+ years experience in designing distributed systems architecture, optimizing for scalability, testing & staged deployment
• Strong communication and interpersonal skills
• Experience with high volume, high availability distributed systems
• Driven and is self-organized
• A Degree in Computer Science, Engineering, Mathematics, similar field of study or equivalent work experience

We'd love to see:

• Familiarity with electronic communications protocols and data model
• Familiarity with Python, Linux Shell, or other scripting languages
• Experience in Spring Boot, Spark, Kafka, Protobuf, Redis, Airflow, Postgres or Cassandra
• Familiarity developing software on containers (docker, kubernetes, et al)
• Experience building data pipelines and robust APIs in form of REST/Web services

Bloomberg is an equal opportunity employer and we value diversity at our company. We do not discriminate on the basis of age, ancestry, color, gender identity or expression, genetic predisposition or carrier status, marital status, national or ethnic origin, race, religion or belief, sex, sexual orientation, sexual and other reproductive health decisions, parental or caring status, physical or mental disability, pregnancy or parental leave, protected veteran status, status as a victim of domestic violence, or any other classification protected by applicable law. 

Bloomberg is a disability inclusive employer. Please let us know if you require any reasonable adjustments to be made for the recruitment process. If you would prefer to discuss this confidentially, please email amer_recruit@bloomberg.net.
 

Salary Range: 160,000 - 240,000 USD Annually + Benefits + Bonus