Product Data Scientist (AI Evaluation & Quality)

Join our team as a Product Data Scientist specializing in AI evaluation and quality. In this role, you will own and extend our offline evaluation suite across products, build and maintain online quality dashboards, and close the production feedback loop. You will also harden our methodology and translate numbers into decisions. The ideal candidate will have a solid foundation in statistics, experience in quality analytics for ML systems, and hands-on experience evaluating LLM applications.

• Posséder et étendre notre suite d'évaluation hors ligne à travers les produits, y compris les ensembles de données, les juges et les métriques.
• Construire et maintenir des tableaux de bord de qualité en ligne, en surveillant des indicateurs clés tels que le taux de résolution et le taux de satisfaction client.
• Exploiter les retours de production pour identifier les modèles d'échec dans le trafic réel et les transformer en cas de régression.
• Solid foundation in statistics — sampling, hypothesis testing, variance, understanding what a noisy metric is
• Python and SQL — you can build an analysis end-to-end
• 3+ years in analyst / data scientist roles, at least one in a product context
• Analytical mindset — you start from the business question, not from the tool
• Experience in quality analytics for ML systems — ranking, recommendations, classification, etc
• Hands-on experience evaluating LLM applications (RAG, agents, tool use, judges)
• Experience building LLM agents — side projects, toy builds, personal experiments all count