Senior Data Architect - Real-time Streaming

Job description

About us

Who we are? We are Big Data experts, working with international clients, creating and leading innovative projects related to the Big Data environment. We offer tailor-made solutions. It does not matter whether we are talking about building a Data Lake, conducting training in the field of data management, or performing detailed Big Data analysis. We don’t just focus on one technology instead, we specialize in a whole range of open-source and public cloud tools. Our team brings together over 130 specialists in their fields. We have participated in dozens of conferences, and written countless amounts of code, we are the organizers of Big Data Tech Summit Warsaw, the largest Polish conference related to Big Data topics. We run webinars, share knowledge on blogs, create whitepapers, and more. Why? Because we believe that Big Data is an indispensable future of business.
Thanks to that, we always select the most optimal Big Data solutions.


Projects

A core part of Getindata’s value proposition in projects we do is to bring the best expert knowledge in the applications of Big Data & AI (including Real-time Stream Processing). We mentor clients on what problems they can solve with modern technologies, how to architect and implement reliable solutions with the best engineering standards, and how to deliver them effectively. This expertise is brought by every Getindata employee but especially by solutions architects. Being a solution architect is to be the ambassador of the company by setting technological direction, roadmap, architecture, and feasibility in multiple projects, especially at their earliest stages. The goal is to make sure our clients get the best possible solution to their real-world challenges and a path to get there.


Role description

Some of your time you will work as a part of an architect's team which consults and supports each other in their ventures. Each of the architects has T-shaped skills i.e. has wide knowledge about all aspects of data engineering, but specializes and gravitates towards one/some.

Most of the time you will spend working with clients, contract owners, or operational teams that implement the project.

There will be a changing portion of the time you spend hands-on, on doing PoC of solutions, crunching the hardest problems, and testing new approaches and technologies.

As an architect of Streaming solutions, you will bring specialization in solutions that leverage Stream Processing to solve cases in real-time. That could be in the area of data ingestion, integration, transformation, business automation, Machine Learning application, or other data & event-based applications. 

Responsibilities

  • Launch new projects
    • Facilitate workshops with new clients - explore requirements, propose a solution, technologies, architecture, implementation process, and organization
    • Write high quality and technically deep solution proposals
    • Define prerequisites for clients, acceptance criteria of the solution
    • Propose the work estimations, roadmap, and timeline of the project
    • Implement the prototype while building the design of the final product
  • Look after running projects
    • Adapt architectural choices to (always changing) client conditions and requirements
    • Bring deep knowledge and advice to the project team on the hardest problems in the project
    • Suggest future steps, extensions of the projects, reuse artifacts developed in Getindata
  • Build internal GID knowledge base
    • Unify architectures and solutions across many GID projects and build the company’s knowledge base
    • Be the go-to person on “how we do it at Getindata?”

Technologies used:

  • Flink, GCP Dataflow, Beam, Kafka, Spark Structured Streaming
  • GCP, AWS, Azure
  • Kubernetes, Docker

Requirements

  • Strong engineering background, preferred in fields such as Computer Science, Mathematics, Software Engineering, Physics, and Data Science
  • Great fundamental knowledge of Software Engineering, Data Engineering, building whole data platforms and solutions
  • Experience or interest in working with and using large scale data to solve valuable business problems
  • Understanding of stream processing, especially stateful processing, CEP, real-time analytics
  • Some experience in solving stateful stream processing problems with the use of Spark, Kafka, Flink or similar technologies

Preferred Qualifications:

  • Deep experience in building systems with Apache Flink
  • Experience with cloud technologies (GCP, AWS, Azure)
  • Experience with large-scale OLAP and technologies like Druid, Pinot, Clickhouse
  • Experience with Kubernetes and Docker
  • Experience in technical leadership, leading project teams, and setting technical direction
  • Experience in working with clients, doing requirement analysis, and writing solution proposals
  • Experience in technical sales, professional consulting, or working as a customer engineer
We offer
  • Salary: 180-230 PLN net + VAT/h B2B (depending on knowledge and experience)

  • 100% remote work

  • Elastic working hours

  • Possibility to work from the office located in the heart of Warsaw

  • Opportunity to learn and develop with the best Big Data specialists in Poland

  • International projects

  • Possibility of conducting workshops and training.

  • Clear career path and certifications

  • Co-financing sport card

  • Co-financing health care

  • All equipment needed for work