(Senior) DevOps Engineer

deepset • Posted June 17th

Software Development
deepset logo

Texts are the fabric of the digital world. There's so much information at our fingertips, but making sense of it efficiently is still very hard. Deepset's mission is to break this barrier, and empower any developer to harness powerful NLP pipelines to efficiently access and comprehend any kind of data. Our open-source project Haystack is already used all around the world and we set out to become the semantic layer in every enterprise tech stack.

Our Haystack platform is based on recent breakthroughs in NLP that tremendously improve the accuracy of search results. However, with great performance breakthroughs come great engineering challenges: we are building a solution to scale to millions of documents for our enterprise clients. Just a small set of the challenges you will tackle at deepset: How to distribute GPU workloads efficiently in a distributed cluster? How to keep a warm cache of models? How to enable customers to deploy our platform in their own AWS environment?

As a DevOps Engineer, you will not only be responsible for operating our commercial SaaS platform (Haystack Hub) reliably and efficiently, but also work closely with our backend engineers to tackle the above challenges and architect the infrastructure in a scalable, sustainable way.

Exemplary tasks: Enable efficient scaling of GPU-heavy machine learning services; Enable tracing of requests across multiple microservices in Kubernetes; Design together with our backend team the (auto)scaling logic of Haystack pipelines

Your Responsibilities

  • Design & build with our backend engineers the core cloud infrastructure for Haystack Hub
  • Make sure we have the right metrics in place to steer stability and performance
  • Improve the scalability & reliability of Haystack Hub
  • Be our internal infrastructure expert that guides us through the waters of multi-cloud or hybrid cloud environments (e.g. via AWS PrivateLink)
  • Simplify & automate all kinds of deployment, provisioning and dev processes
  • Own & optimize our cloud infrastructure


  • 3+ years experience of having worked in a related field
  • Excellent in designing & building service-oriented architectures
  • Experience in the lifecycle of building up a SaaS infrastructure (ideally from early days to massive scale)
  • Expert in containerization and orchestration
  • Fast, analytical thinker who understands where to use best-practices and where to leave the common paths
  • Good communication skills
  • Good understanding of networking and security
  • Experience with AWS, Kubernetes, CI/CD, Monitoring tools
  • Interest in Machine Learning/NLP and the motivation to learn about it
  • Pragmatism, sense of responsibility, and independent work style


  • Fast & expanding environment - Team lead possibilities, fast decisions, direct impact and flexibility
  • We live and breathe open source: See Haystack, FARM, German BERT ...
  • Latest machine learning & tech - no legacies
  • Collaborate with tech experts - we understand your language & problems
  • Work on real-world cases - no AI hype
  • Possibility to speak at conferences and workshops
  • True ownership - stock options to participate in our common success
  • Working in a remote setting (Timezone: CET +/- 2 hours)

About deepset

  • One of the leading companies in transformer-based NLP
  • Living Open Source - in products and culture
  • We know the problem that we are solving like nobody else
  • Backed by renowned and mission-driven investors (Ex-Google, Ex-Microsoft ...)
  • Collaborations with AWS, Nvidia & more