Staff Product Manager (Evals)

Join Workato as a Staff Product Manager, where you'll own the evaluation framework for AI agents and build the customer-facing evaluation experience. You'll work closely with internal teams and customers to drive adoption and improve agent performance. This role requires a deep understanding of evaluation methodology and a track record of shipping technical products.

• Definir y ser responsable del marco de evaluación para las características internas de los agentes de IA de Workato, impulsando la adopción en varios equipos.
• Construir la experiencia de evaluación orientada al cliente, permitiendo a los constructores probar, medir y mejorar los agentes que crean en Workato.
• Establecer métricas para lo que significa "bueno" en términos de calidad del agente interno y adopción de la evaluación por parte del cliente.
• Practitioner depth in evaluations. You've written evals yourself — built test suites, designed rubrics, debugged why agents underperformed. You understand evaluation methodology not only from reading about it, but from doing it. You have opinions about what works, what doesn't, and where current approaches fall short
• Track record of shipping technical products to both internal and external users
• Greenfield comfort. You've defined products from ambiguity — scoped v1s, made bets with incomplete information, and iterated based on what you learned. You don't need an existing playbook to be effective
• Experience driving adoption of frameworks or practices across engineering teams
• 7+ years in Product Management
• B2B product sensibility. You see enterprise conventions as problems to solve, not constraints to accept. You're drawn to products that make complex workflows feel elegant
• Strong written and verbal communication skills
• Hands-on experience writing evaluations for AI/ML systems (agents, LLMs, or similar)
• Bachelor's degree or equivalent experience
• Strong product management experience. You've shipped products, driven roadmaps, and led cross-functional teams. You know how to translate technical capabilities into user value and write specs that don't leave details to chance
• Technical translation ability. You can take complex evaluation concepts and make them accessible to business technologists without dumbing them down. You understand the difference between hiding complexity and organizing it
• Internal influence skills. You've driven adoption of frameworks, practices, or tools across teams. You can be a credible partner to ML engineers while advocating for what internal teams actually need
• Experience with agent architectures, RAG systems, or LLM application development
• Background in ML engineering, solutions architecture, or technical program management before PM
• Experience building developer tools or platform products
• Familiarity with evaluation frameworks (e.g., human eval pipelines, automated benchmarks, red-teaming)