Senior Software Engineer (Trust & Safety)

Join Replit's Trust & Safety team as a Senior Software Engineer. You'll be on the front lines of defending our platform from exploitation, building detection systems and automated responses to stay ahead of attackers. This unique role offers hands-on experience applying AI to security problems in a production environment. You'll own the full abuse response lifecycle, analyze attack patterns, and maintain internal detection tools. Enjoy a remote-first work environment with flexible hours and a comprehensive benefits package.

• Design and implement LLM guardrails that detect abuse scenarios in AI-generated code and agent interactions.
• Build AI-powered detection systems that use LLMs to identify malicious patterns, classify threats, and automate response decisions.
• Own the full abuse response lifecycle: detection, investigation, enforcement, and handling appeals alongside Support and Legal.
• 4+ years of experience in security engineering, anti-abuse, trust & safety, or fraud detection
• Ability to investigate complex abuse patterns and translate findings into automated defenses
• Familiarity with common attack patterns: phishing infrastructure, account takeover, credential stuffing, resource abuse
• Familiarity with prompt injection, jailbreaking, and other LLM-specific attack vectors
• Experience building or fine-tuning ML/LLM-based classifiers for security or abuse detection
• Strong programming skills in Python and/or TypeScript for building detection systems and automation
• Experience with SQL and data analysis at scale (BigQuery, Snowflake, or similar)
• Clear communication skills for working across Security, Support, Legal, and Engineering teams
• Experience at a platform company dealing with user-generated content or compute abuse (hosting providers, cloud platforms, developer tools)
• Background in fraud detection, payment abuse, or financial crime
• Familiarity with device fingerprinting, IP reputation, and email validation services
• Knowledge of container security, Linux internals, or cloud infrastructure (GCP preferred)
• Experience with CI/CD security tooling (SAST, SCA, Dependabot, Snyk)
• Prior work with abuse reporting pipelines, trust & safety tooling, or content moderation systems
• CI/CD Security: Dependabot, Snyk, SAST/SCA scanners
• Data: BigQuery, Hex
• Detection tools: Slurper, Netwatch, Stytch (device fingerprint); ClearOut (email reputation)
• Languages: Python, TypeScript, Go, SQL
• Infrastructure: GCP, Kubernetes
• Collaboration: Linear, Slack, Zendesk (for abuse reports)
• Tools + Tech Stack for this role: