Senior Member of Technical Staff (Web Data)

Join Cohere as a Senior Member of Technical Staff specializing in web data for pre-training. In this pivotal role, you will develop and maintain the large-scale web data pipeline that underpins our advanced language models. You will work extensively with large-scale web corpora, transforming raw internet data into high-quality training data for pretraining. Your work will be essential to our mission of delivering efficient and reliable language understanding and generation capabilities. Enjoy a range of benefits including six weeks of paid vacation, equity options, and a remote work culture.

• Develop and maintain large-scale web data pipelines, including extraction, parsing, deduplication, and filtering.
• Analyze the composition and quality of web data, study its impact on downstream model performance, and collaborate with cross-functional teams.
• Work on filtering and quality-scoring systems to identify high-value web documents and ensure data pipelines meet the demands of cutting-edge language models.
• Familiarity with data processing frameworks such as Apache Spark, Apache Beam, Pandas, or similar tools
• A passion for bridging research and engineering to solve complex data-related challenges in AI model training
• Experience working with large-scale web datasets
• Knowledge of data quality assessment techniques and experimentation with data mixtures
• Strong software engineering skills, with proficiency in Python and experience building data pipelines
• Paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)
• If you are passionate about transforming data into the foundation of AI systems, this role offers a unique opportunity to make a meaningful impact
• If some of the above doesn’t line up perfectly with your experience, we still encourage you to apply!