Senior AI Engineer

Join Ro, a healthcare technology company, as a Senior AI Engineer. You will be part of a new team focused on developing LLM-powered products for patient experience, clinical operations, and internal tooling. Your responsibilities will include designing, building, and shipping production LLM-powered features, developing evaluation infrastructure, and ensuring the safety and reliability of AI systems. You will also have the opportunity to raise engineering standards across the team and work closely with clinical operators to understand the workflows you are automating.

• Design, build, and ship production LLM powered features end-to-end, including prompt orchestration, tool calling, retrieval, embeddings, agent workflows, and structured outputs.
• Develop evaluation infrastructure, including regression suites, LLM-as-a-judge evaluations, synthetic datasets, human review workflows, and quality metrics that catch regressions before users do.
• Raise engineering standards across the team through technical leadership, code reviews, mentorship, and thoughtful system design.
• Strong Python engineer with experience building backend services and integrating modern AI APIs and frameworks
• Bonus: Experience with orchestration frameworks (LangChain, LangGraph, AgentCore), RAG architectures, vector databases, prompt/version management, and evaluation or observability platforms (e.g., LangSmith, Braintrust, Arize Phoenix)
• 5+ years building production software, with at least 1–2 years building and deploying LLM-powered applications in production (or equivalent depth through substantial side projects or open-source work)
• You have strong product instincts and thrive in ambiguous environments, translating operational problems into technical solutions without relying on detailed specifications
• Comfortable using AI-assisted development tools throughout the software development lifecycle
• You've shipped AI features that served real users, and you can clearly articulate how you measured quality, reliability, latency, cost, and business impact