Senior Software Development Engineer in Test (Machine Learning)

Join PlayStation as a Senior Software Development Engineer in Test (Machine Learning) in San Mateo. In this role, you will lead quality efforts for machine learning-powered products and services, partnering closely with ML engineers, software engineers, product managers, and data partners. You will design, develop, and maintain automated test frameworks, validate model behavior, and drive high standards of quality for engineering decisions and customer-facing features. The ideal candidate will have 5+ years of experience as an SDET, proficiency in Python/Java/Scala/go, and experience testing machine learning systems.

• Design, develop, and maintain automated test frameworks and test scripts for machine learning inference services.
• Validate score distributions, ranking behavior, output quality, and other model signals for incoming ML models prior to release.
• Work closely with ML engineers and ops teams to ensure that all features and bug fixes come with automated test coverage.
• Excellent problem-solving skills and excellent communication skills
• Hands-on experience with API testing tools (e.g., Postman or similar) for distributed systems
• Familiarity with CI/CD pipelines and test execution in Jenkins or similar environments such as jenkins, GitHub Actions, ArgoCD, etc
• Bachelor's degree or equivalent
• 5+ years of experience as an SDET, with strong experience testing backend systems
• Proficiency in Python/Java/Scala/go for test automation
• Familiarity with monitoring and observability tools (e.g., Prometheus, Grafana, Datadog, etc.) for validating service health
• Hands-on experience building and maintaining automated test frameworks (e.g., pytest, JUnit, TestNG, etc.)Experience in working with databases for backend validation
• Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker, etc.)
• Strong understanding of software development lifecycle (SDLC) and agile methodologies
• Experience testing machine learning systems, such as recommendation systems, search ranking systems, or other probabilistic ML application systems
• Experience creating synthetic or seeded test data to simulate realistic customer/account behaviors
• Experience evaluating model outputs using statistical techniques, distribution analysis, or scenario-based validation, rather than deterministic assertions
• Knowledge of data pipelines, feature stores, inference systems or model-serving infrastructure (Seldon, KServe, Ray Serve, etc.) is a plus
• Experience with performance and load testing tools (Locust, k6, Gatling, JMeter, etc.)
• Experience testing online services with high RPS and low latency requirements
• Experience with Databricks or similar ML platform tooling