Computational Linguist (AI Evaluation)

Join TwelveLabs, a cutting-edge AI company, as a Computational Linguist. In this role, you will be a key member of our ML Data Operations Team, responsible for defining dataset and evaluation requirements, designing and building data pipelines, and coordinating with vendor partners. You will also automate repetitive tasks and work cross-functionally with research and product teams. The ideal candidate has 5+ years of experience in AI-focused data operations, excellent communication and project management skills, and a foundational understanding of LLMs/VLMs and multimodal AI.

• Defining dataset and evaluation requirements in consultation with research and product teams, designing and building data pipelines, and coordinating with vendor partners.
• Defining and building evaluation protocols for video-language model quality, working with Research and Product teams to translate ambiguous quality questions into measurable benchmarks.
• Designing and executing video-language data collection and labeling projects, automating repetitive processes, and coordinating with vendor and outsourcing partners.
• Proficiency with Python, agentic coding, or other popular industry tools for automation
• Conviction that data is the key ingredient for the performance of AI models
• Direct experience building or running model evaluation pipelines (benchmark design, human eval frameworks, automated scoring systems), and translating ambiguous quality questions into measurable criteria
• The ability to analyze messy and complex data, identify overarching patterns, and distill your findings into crisp annotation guidelines or other accessible documentation
• 5+ years of experience working in an AI focused data operations organization
• Excellent communication and project management skills, and the ability to support several projects simultaneously
• A foundational understanding of and interest in LLMs/VLMs and multimodal AI
• A proven track record designing and executing large scale data projects, including gathering, labeling, and post-processing data
• Even if there are a few checkboxes that aren’t ticked through your prior experience, we still encourage you to apply! If you are a 0-1 achiever, a ferocious learner, and a kind and fun team player who motivates others, you will find a home at TwelveLabs
• Experience planning and operating new data tools and labeling systems
• Experience working with research scientists and engineers
• Expertise or interest in video-centric domains, such as sports, advertising, and content creation
• Experience in data collection and labeling for multimodal language models