Staff Software Engineer (Inference)

Join our Inference team as a Staff Software Engineer, where you will be responsible for building and maintaining critical systems that serve our AI model, Claude, to millions of users worldwide. You will work on complex distributed systems challenges, optimize compute efficiency, and enable breakthrough AI research. The role offers comprehensive benefits, a competitive salary, and the opportunity to make a significant impact in the field of AI.

• Concevoir des algorithmes de routage intelligents qui optimisent la distribution des demandes à travers des milliers d'accélérateurs.
• Autoscaler notre flotte de calcul pour s'adapter dynamiquement à l'offre et à la demande.
• Construire des pipelines de déploiement de production pour la mise en œuvre de nouveaux modèles.
• LLM inference optimization, batching, and caching strategies
• Implementing and deploying machine learning systems at scale
• Thrive in environments where technical excellence directly drives both business results and research breakthroughs
• Familiarity with LLM inference optimization, batching strategies, and multi-accelerator deployments is highly encouraged but not strictly necessary
• Python or Rust
• Want to learn more about machine learning systems and infrastructure
• You may be a good fit if you:
• Care about the societal impacts of your work
• Are results-oriented, with a bias towards flexibility and impact
• Education requirements: We require at least a Bachelor's degree in a related field or equivalent experience
• Have significant software engineering experience, particularly with distributed systems
• Strong candidates should have familiarity with performance optimization, distributed systems, large-scale service orchestration, and intelligent request routing
• Strong candidates may also have experience with:
• High-performance, large-scale distributed systems
• Pick up slack, even if it goes outside your job description
• Load balancing, request routing, or traffic management systems
• Kubernetes and cloud infrastructure (AWS, GCP)
• We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed