Senior Software Engineer (ML Infrastructure)

Join SambaNova, an inference-first company, as a Senior Software Engineer specializing in ML Infrastructure. In this role, you will design, build, and operate the production-grade inference infrastructure that powers SambaNova's serving stack on our Reconfigurable Dataflow Unit (RDU) architecture. You will own end-to-end systems, partner closely with various teams, and contribute to technical design discussions and architectural decisions. Enjoy benefits such as flexible PTO, parental leave, and gym access.

• Design, build, and operate the production-grade inference infrastructure for SambaNova's serving stack.
• Own end-to-end systems spanning request scheduling, advanced decoding algorithms, caching layers, API surfaces, and the accuracy infrastructure.
• Partner closely with ML, compiler, runtime, and product teams to ship inference features from prototype to production.
• Working knowledge of modern LLM inference techniques and familiarity with open-source serving stacks such as vLLM, TensorRT-LLM, or SGLang
• Bachelor's degree in Computer Science, Electrical Engineering, or related field
• Strong software engineering fundamentals: algorithms, data structures, concurrency, and systems design
• Proficiency in Python
• 5+ years of industry experience building and operating large-scale distributed systems, ideally in ML serving
• Experience collaborating across teams to deliver complex, system-level engineering solutions
• Experience designing and maintaining production services with strict latency, throughput, and availability requirements