Company Description
The Senior Data Engineer will collaborate with our team to build, enhance, and maintain high-performance data systems and machine learning models, ensuring their scalability and reliability.
Responsibilities include designing and maintaining high-throughput data pipelines, optimizing real-time and batch data processing using Scala, and integrating advanced data solutions to improve performance. The role involves working with large-scale distributed systems, handling terabytes of data per second and processing millions of writes per second.
Our team is at the forefront of AdTech innovation, enabling cutting-edge solutions for data-driven advertising. We value hard work and flexibility, offering an environment where engineers can experiment, innovate, and push technological boundaries.
Our client is a leading AdTech company specializing in high-performance data systems and machine learning models, processing terabytes of data per second and millions of writes per second.
It is a large-scale AdTech platform designed to process massive amounts of data in real time, optimizing data-driven advertising with advanced machine learning models. It ensures scalability and reliability, enabling businesses to maximize efficiency and performance while leveraging cutting-edge data solutions.
Job Description
- Work with Apache Spark batch and real-time streaming to process data at scale
- Work with Scala microservices hosted on K8s (GKE) to support our products
- Work with MlLib and TensorFlow on Vertex AI to solve prediction and simulation issues, as well as a variety of machine learning challenges
- Deploy new models and engines for new ideas and analysis.
Qualifications
- 5+ years of experience as a Data Engineer
- Advanced knowledge of different non-relational schema models (Column Family, Graph, Document, and Object)
- Experience with advanced topologies and architecture design, such as Kappa and Lambda architectures
- GCP Dataflow experience
- Experience in Scala
- Experience in Python
- Experience with high-throughput systems
- Strong communication skills and at least C1 level of English
WOULD BE A PLUS
- Experience in the AdTech industry
Additional Information
PERSONAL PROFILE
- Problem-solving mindset
- Ability to work closely with cross-functional teams
- Willingness to experiment with new technologies and approaches
