Research Engineer/Research Scientist (Pre-training)

Join our Pre-training team as a Research Engineer/Research Scientist, where you will develop the next generation of large language models. You will conduct research, implement solutions, and optimize our training infrastructure. This role requires an advanced degree in Computer Science or a related field, strong problem-solving skills, and expertise in Python and deep learning frameworks. Enjoy comprehensive benefits, a competitive salary, and the opportunity to make a significant impact in the field of AI.

• Conduct research and implement solutions in areas such as model architecture, algorithms, data processing, and optimizer development.
• Independently lead small research projects while collaborating with team members on larger initiatives, and design, run, and analyze scientific experiments.
• Optimize and scale our training infrastructure to improve efficiency and reliability, and develop and improve dev tooling to enhance team productivity.
• We encourage you to apply even if you do not believe you meet every single qualification
• Advanced degree (MS or PhD) in Computer Science, Machine Learning, or a related field
• Ability to balance research goals with practical engineering constraints
• Strong problem-solving skills and a results-oriented mindset
• Familiarity with large-scale machine learning, particularly in the context of language models
• Care about the societal impacts of your work
• Excellent communication skills and ability to work in a collaborative environment
• Strong software engineering skills with a proven track record of building complex systems
• Expertise in Python and experience with deep learning frameworks (PyTorch preferred)
• Are working to align state of the art models with human values and preferences, understand and interpret deep neural networks, or develop new models to support these areas of research
• Have significant software engineering experience
• Are results-oriented with a bias towards flexibility and impact
• Enjoy pair programming and collaborative work
• Familiarity with GPUs, Kubernetes, and OS internals
• Willingly take on tasks outside your job description to support the team
• View research and engineering as two sides of the same coin, and seek to understand all aspects of our research program as well as possible, to maximize the impact of your insights
• Are eager to learn more about machine learning research
• Background in large-scale ETL processes
• Work on high-performance, large-scale ML systems
• Knowledge of reinforcement learning techniques
• Are enthusiastic to work at an organization that functions as a single, cohesive team pursuing large-scale AI research projects
• Experience with language modeling using transformer architectures
• Have ambitious goals for AI safety and general progress in the next few years, and you’re working to create the best outcomes over the long-term