Company
Decart
Annual salary
$350k – $450k/yr
Location
San Francisco, CA
Listed
29d ago
- Experience:
- 5+ years
- Workplace:
- On-site in San Francisco, CA, 5 days per week at Decart's office near the Embarcadero/Ferry Building area.
- Equity:
- Competitive equity
Open to people who live in the US, Canada or the UK
We can only put candidates forward for this role from those three countries for now. See the contract AI training jobs open in more countries.
On-site in San Francisco, CA.
Send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role.
Apply for a referral →The recruiter emails you before anything happens and may suggest other jobs that suit you better.
What you'll do
- Design, operate, and scale highly available Kubernetes clusters across AWS, GCP, and specialized GPU cloud providers to serve global inference demand at 1,000+ tokens per second
- Take ownership of the production serving layer, debug NVIDIA systems end-to-end including drivers, CUDA, NCCL, node health, and network fabric
- Work closely with the model optimization team to ensure kernel-level speed improvements survive contact with real traffic, real hardware, and real customers
- Extend deployment tooling so models ship consistently across NVIDIA, Trainium, and TPU hosts through a unified workflow
- Build observability infrastructure that keeps the platform honest: time-to-first-token, inter-token latency, throughput, and availability, measured per model, per chip, and per region
- Own reliability engineering including alerting, automated failover, self-healing infrastructure, and intelligent traffic routing across models, chips, and regions
What you need
- We need someone with 5+ years of infrastructure engineering experience who has deep hands-on expertise operating production Kubernetes at scale and working with LLM inference serving systems. You should be comfortable debugging NVIDIA GPU systems end-to-end (drivers, CUDA, NCCL, network fabric) and have a track record of building highly available, multi-cloud infrastructure for latency-sensitive AI workloads. Bonus points if you've deployed across heterogeneous accelerator types (NVIDIA GPUs, AWS Trainium, Google TPUs) or contributed to open-source inference frameworks like vLLM or SGLang.
Common questions
How do I apply for the Infrastructure Engineer, Managed Inference (US) role at Decart? +
Use the Apply for a referral button on this page to send us your LinkedIn and CV. If your experience fits, we'll introduce you to the recruiter filling this role. They'll email you to check you're interested, then put you forward for this job or others that suit you better. It's free.
What does this Decart role pay? +
The listing gives $350k – $450k/yr.
Is this role remote? +
The listing gives the location as San Francisco, CA. On-site in San Francisco, CA, 5 days per week at Decart's office near the Embarcadero/Ferry Building area.
Related roles
Browse all referral roles
Salaried roles at startups, AI companies and established businesses, all filled by referral. One application covers every role.
See all referral roles →