Data Platform Engineer
San Francisco, CA
Posted Mar 22, 2021 - Requisition No. 89690
Bloomberg Second Measure is a leading provider of data analytics that deliver valuable insights into company performance and consumer behavior. Using data from billions of anonymized transactions, we have created self-service analytics products for daily tracking and real-time exploration of 5,200+ public and private companies. Clients use our product to discover new markets, gain an advantage in financial investments, and inform their competitive strategies. Second Measure was acquired by Bloomberg in 2020.
As a Data Platform Engineer, your goal is to provide an efficient, stable platform that reliably and predictably delivers accurate, high-quality data to our clients. You will collaborate with data engineers to build and extend systems necessary productionizing insights and analyses. You will collaborate with data scientists to provide tools and systems to enable their insights.
We’ll trust you to:
- Design, build, and maintain the control plane for Second Measure’s platform for processing data built with Kubernetes, Argo Workflows, Presto, and Apache Spark
- Build tools and abstractions for using the platform that empower data scientists and data engineers with the autonomy to deliver value to our clients
You’ll need to have:
- 3+ years experience in software development using Python, Scala, or Go
- Experience working with Kubernetes, especially extending Kubernetes using admission webhooks, controllers, and other building services that interact directly with the Kubernetes API
- Built systems in event-driven or streaming architectures using systems such as Kafka/Kinesis, RabbitMQ, NATS, and AWS SNS/SQS
- Experience with building, managing, and deploying systems used for data processing
- Architected, designed, developed, and operationalized scalable, resilient, and reliable services
- Software engineering fundamentals, including using participating in Agile, knowledge and use distributed version control systems such as Git/Mercurial, code reviews, emphasis on quality, and commitment to documentation
- BA, BS, MS, PhD in Computer Science, Electrical Engineering or related technology field
We’d love to see:
- Experience working with structured and unstructured data, especially CSV, JSON, Parquet
- Argo Workflows orchestration system experience
- Experience with data processing systems using data processing frameworks such as Apache Spark, Apache Flink, Dask
- Distributed query experience including Presto/Trino, AWS Redshift
Bloomberg is an equal opportunities employer, and we value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.