Product Data Scientist (AI Evaluation & Quality)

Join Finom's AI Team as a Product Data Scientist, where you'll be responsible for owning the evaluation loop for all AI products. You'll work closely with AI engineers, product managers, and domain experts to ensure the quality and continuous improvement of our AI offerings. Your core responsibilities will include extending our offline evaluation suite, building and maintaining online quality dashboards, and translating data into actionable decisions. The ideal candidate will have at least 3 years of experience in analyst or data scientist roles, a solid foundation in statistics, and experience in quality analytics for ML systems.

• Own and extend the offline evaluation suite across AI products, including datasets, judges, and metrics.
• Build and maintain online quality dashboards to monitor key performance indicators and quality metrics.
• Close the production feedback loop by mining failure patterns from real traffic and turning them into regression cases.
• 3+ years in analyst / data scientist roles, at least one in a product context
• Analytical mindset — you start from the business question, not from the tool
• Solid foundation in statistics — sampling, hypothesis testing, variance, understanding what a noisy metric is
• Python and SQL — you can build an analysis end-to-end
• Experience in quality analytics for ML systems — ranking, recommendations, classification, etc
• Experience building LLM agents — side projects, toy builds, personal experiments all count
• Hands-on experience evaluating LLM applications (RAG, agents, tool use, judges)