Head of Data Labeling

ABOUT US

White Circle https://whitecircle.ai/ is an AI Safety company building the safety, reliability, and optimization layer for AI systems. At the core of our platform are policies – simple natural-language rules that define what an AI model should and shouldn’t do. We automatically test, enforce, and continuously improve these policies at scale.

• We’ve raised $11M from top funds, founders, and senior leaders at OpenAI, Anthropic, HuggingFace, Mistral, DeepMind, Datadog, Sentry, and others
• We process over 100M+ API calls every month
• We fine-tune and train our own LLMs so they run faster and cheaper than any open or proprietary model

We’re at a multi-million dollar run rate already having signed customers like Lovable and multiple neobanks. You’ll be joining at the most exciting time - early enough that the equity can be life changing but at a point where demand has been proven.

IN THIS ROLE, YOU WILL

• Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management)
• Define annotation guidelines, quality standards, and evaluation frameworks
• Develop quality assurance processes, calibration sessions, and auditing systems
• Partner with AI researchers and engineers to translate research objectives into labeling workflows
• Prioritise labeling projects based on business and research needs
• Monitor operational metrics including quality, consistency, throughput, and cost
• Improve annotation tooling, automation, and workflow efficiency
• Lead complex AI evaluation projects, including safety, preference ranking, RLHF, policy evaluation, and benchmark creation
• Analyse disagreement patterns and edge cases to improve guidelines and model performance
• Manage vendor relationships and ensure consistent quality across distributed teams
• Build reporting dashboards and communicate operational insights to leadership
• Foster a culture of continuous improvement, accountability, and operational excellence

WE'RE LOOKING FOR SOMEONE WHO

• Has experience leading data annotation or AI evaluation teams
• Has strong operational and people management skills
• Understands AI model evaluation, LLM behavior, and modern annotation workflows
• Can design scalable processes without sacrificing quality
• Communicates clearly across technical and non-technical teams
• Thrives in fast-moving startup environments

YOU MIGHT BE A GREAT FIT IF YOU

• Have managed annotation programs for LLMs, generative AI, or machine learning
• Have experience with RLHF, preference data collection, safety evaluations, or benchmark creation
• Have worked in Trust & Safety, AI Safety, Content Moderation, or ML Ops
• Have managed distributed or global annotation teams
• Have experience with vendor management and outsourcing operations

BONUS POINTS

• Familiarity with prompt engineering and AI safety policies
• SQL, Python, or data analysis experience
• Experience building internal annotation platforms or workflow automation
• Background in linguistics, cognitive science, machine learning, or data operations

• Important note
• This role involves overseeing projects that may include offensive, harmful, violent, sexual, or otherwise disturbing content.
• You'll be responsible for ensuring reviewers have the tools, guidance, and support necessary to perform this work safely and consistently.

WHY WHITE CIRCLE

• Competitive salary + equity
• Work from Paris (hybrid) with a relocation package available, or work from London (note: we are currently unable to provide relocation support and medical insurance for London-based roles)
• Paid time off in line with your local regulations
• All the hardware, tools, and services you need
• Covered subscriptions for AI agents and IDEs
• Team off-sites twice a year: we’ve recently been to the Alps and Saint-Tropez

HOW WE HIRE

1. Intro call with HR (30 min)
2. Take-home exercise
3. Final conversation with our CEO (45 min)

Please submit your application in English.