Software Engineer (Model Performance Tooling)

Join Baseten, a remote-first company at the forefront of AI infrastructure. As a Software Engineer specializing in Model Performance Tooling, you'll work on high-performance computing and large language model engineering. Your responsibilities will include performance benchmarking, infrastructure validation, model development experience, tool development, deep hardware profiling, monitoring and observability, continuous integration, and optimization automation. You'll have the opportunity to make a direct impact on defining "good" performance for customers and gain world-class expertise in GPU orchestration and LLM inference.

• Automating the performance testing and benchmarking of large language models (LLMs) and high-performance computing (HPC) systems.
• Building tools and infrastructure for measuring GPU performance, stress-testing clusters, and defining benchmarks.
• Developing and maintaining internal GPU-enabled development environments to ensure seamless model experimentation.
• Mathematical Curiosity: A desire to understand the underlying math of Transformers and how it translates into FLOPs and memory requirements
• This is a fresher-friendly role. We care more about your trajectory, curiosity, and technical depth than your years of experience. We want to talk to you if you have:
• Interest in Optimization: You are excited to learn about (or already play with) quantization, speculative decoding, disaggregated serving, and kernel-level optimizations
• An Automation Mindset: You believe that if a task has to be done twice, it should be scripted. You have a passion for stress-testing and fuzzy testing to find the "breaking point" of a system
• A Love for Systems & Hardware: You aren’t just interested in the AI; you want to understand GPU memory subsystems, InfiniBand, and how data moves across a cluster
• Technical Toolkit: Familiarity with Python, and an eagerness to master the NVIDIA software stack. C++ familiarity is good to have