Data Scientist (Evals, Member of Technical Staff)

Join Perplexity as a Data Scientist, where you will architect and maintain automated evaluation pipelines to assess answer quality across our products. You will design evaluation sets and methods, develop VLM-based solutions, and continuously review public benchmarks and academic evaluations. You will work within a small, high-impact team, collaborating closely with technical leadership to measure and improve answer quality.

• Architect and maintain automated evaluation pipelines to assess answer quality across Perplexity's products.
• Design evaluation sets and methods specifically to measure the impact of tool calls on the final answer's quality.
• Continuously review public benchmarks and academic evaluations for their applicability to the Perplexity product, adapting and incorporating them into our regular performance measurements.
• Experience building within a modern cloud data stack, specifically AWS and Databricks
• Strong proficiency in Python and SQL (expected to write production-grade code)
• PhD or MS in a technical field or equivalent experience
• Comfortable with agentic coding workflows and using AI-assisted development tools to iterate faster
• 4+ years of experience in data science or machine learning
• 1+ years of experience working with LLMs at scale, specifically with LLM-as-a-judge setups
• Prior experience working on customer-facing web products or consumer apps, with real user traffic at scale
• A strong research background, with experience applying research methods to real-world ML problems
• Experience defining evaluation metrics (e.g., factual consistency, hallucination rate, retrieval precision) and building ground truth datasets