Member of Technical Staff (Training Performance Engineer)

Join Cohere as a Member of Technical Staff (Training Performance Engineer) in the Pre-Training team. You will optimize the performance of advanced language models and systems, focusing on key model training metrics and ensuring high accelerator utilization. Your work will involve identifying and removing performance bottlenecks, developing training and profiling tools, and designing high-performant software for training. You will also have the opportunity to learn from and work with top researchers in the field. Enjoy a range of benefits, including six weeks of paid vacation, equity options, and a monthly fitness and wellness allowance.

• Optimizing the performance of advanced language models and systems, focusing on improving key model training metrics.
• Identifying and removing performance bottlenecks, developing cutting-edge training and profiling tools to enhance model performance.
• Designing and writing high-performant and scalable software for training, including low-level CUDA and triton kernels.
• Experience using large-scale distributed training strategies
• Extremely strong software engineering skills
• Familiarity with autoregressive sequence models, such as Transformers
• Proficiency in Python and related ML frameworks such as JAX, Pytorch and XLA/MLIR
• Experience writing kernels for GPUs using CUDA, triton, etc
• Bonus: paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)
• If some of the above doesn’t line up perfectly with your experience, we still encourage you to apply!